Force full crawl on spec when previous crawl reported an error #1602
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Reffy happily reused the result of a previous crawl on a spec when a spec wasn't modified even though these results indicated an error. In most cases, the error is a network error, and reusing the result is actually quite fine because that result was for when the spec could be crawled without error, and we should just forget about the error. In some cases though, the error is for something else, and reusing the result while pretending the error is gone would be wrong. These cases are not straightforward to distinguish. This update does not try to be smart and simply forces a full crawl on specs for which the previous result contains an error.
dontcallmedom
approved these changes
Jun 19, 2024
tidoust
added a commit
that referenced
this pull request
Jun 19, 2024
Dependencies bumped: - Bump undici from 6.18.2 to 6.19.2 (#1596) - Bump respec from 35.1.0 to 35.1.1 (#1597) - Bump ws from 8.17.0 to 8.17.1 (#1598) - Bump puppeteer from 22.10.0 to 22.11.2 (#1598) Patches: - Force full crawl on spec when previous crawl reported an error (#1602) - Don't sleep between specs when useCrawl option is set (#1600) - Bump braces from 3.0.2 to 3.0.3 (#1592)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reffy happily reused the result of a previous crawl on a spec when a spec wasn't modified even though these results indicated an error.
In most cases, the error is a network error, and reusing the result is actually quite fine because that result was for when the spec could be crawled without error, and we should just forget about the error.
In some cases though, the error is for something else, and reusing the result while pretending the error is gone would be wrong.
These cases are not straightforward to distinguish. This update does not try to be smart and simply forces a full crawl on specs for which the previous result contains an error.