Re-adding caching of Metas & fixing how files are scanned and ref dependencies #91
Conversation
|
Currently I don't know. I remember seeing this weirdness where some things were seen as dependencies when they should not be and I have not tracked that down yet. |
|
I'm still checking into this - and have identified several related items. |
weaverryan
force-pushed
the
re-add-meta-caching
branch
2 times, most recently
from
ac3905a
to
c8b06a6
Compare
weaverryan
changed the title
|
This is still WIP, but it's close and I'm now fixing a bunch of things. Most important, the way that we were scanning for files was wrong and WAY too complex. I checked Sphinx: if i have a file I've now fixed and simplified that: |
weaverryan
changed the title
It's much simpler: Sphinx finds all files in the given directory, recursively, that have the .rst file extension. And it parses all of them. There is no need to follow "dependencies" to find more things to parse: just parse all files you find. Sure, then later you might be able to give a warning if some documents you rendered were not included in any toctree's - but you will still render those orphan documents.
The problem is that a ref called "some-link" is not a document filename, but it is/was being added into dependencies. The solution is to track which dependencies are unresolved, and resolve them later once we can.
Before this commit, if there were 2 :ref: to the same spot, because "references" were not stored uniquely on Environment, we only resolved the first, not the second. But, if we simply stored $this->dependencies as a unique array on Environment, we could have a collision between a :doc: and a :ref: with the same name. The hack of prefixing the unresolved dependencies was added to work around this.
For example, if we had :ref:`foo` in subdir/index, then it would be changed to `subdir/foo` and we wouldn't be able to resolve the `foo` reference later.
weaverryan
force-pushed
the
re-add-meta-caching
branch
from
9c322fd
to
27979a9
Compare
|
Feedback handled! :) |
Hi!
I'm trying to re-add the caching of the metas files. It's fairly trivial, except that there are/were two problems:
In
Scanner, the$rstvariable is always missing its.rstextension. And so, thefile_exists($rst)call always returned false.Fixing that meant that a whole sub-system of "loading from an existing meta instead of re-parsing" was activated, and it's acting strangely:
You can see this in the new failed test: it's trying to get the
filectimeof sometest-anchor.rstfile. That is not a file, but actually an inline anchor from one of the source files (tests/Builder/input/subdir/index.rst):For some reason, this
test-anchoris being seen as a dependency of the MetaEntry for thissubdir/index.rstfile. I don't know much about how the anchors are resolved and how this relates to dependencies, so I would love any guidance you may have.Thanks!