[feature] Implement metadata storage in daily directory names and fast path for list target #313
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Features:
list
target, both simple and detailed) is encoded via custom string-compression (OS-directory name and length restriction safe, probably would even work under Windows 95 🤣), e.g. (first one without metadata, second one with):goQuery
version has to be at least the same as thegoProbe
version since the old goQuery version isn't aware of the suffix (and will not ignore it). This will not affectglobal-query
or the API though since that one theDBWorkManager
insidegoProbe
itself to read from the DB and hence will automatically receive valid data no matter what format the respective local DB will be in. So this in reality isn't a restriction, because it's rather unlikely that you updategoProbe
on your local system but notgoQuery
...goProbe
: Once updated, the DB writer will simply write the metadata for the current directory - old ones are untouched by default. Manual conversion can be done with theanalyze-meta
tool for now (not the final solution, but it was a simple addition, so I opted for not writing yet another tool) in case someone wants to benefit from the additional performance for the whole history.list
target is 10-15 fold, depending on the storage / partition type.list
target query log in this commit.Closes #282