You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Tracking the I/O of a scalar index search we can see that the search is loading the dataset metadata (it appears to do so twice). That data should already be cached. Loading it can be quite costly and defeat the purpose of doing an indexed search in the first place. This gets even worse when there are many fragments in a dataset because the manifest is quite large.
The text was updated successfully, but these errors were encountered:
We were not using the cache before which meant every scalar index query
had to load a bunch of metadata from the index files (some of which was
quite expensive)
Closes#2313
Tracking the I/O of a scalar index search we can see that the search is loading the dataset metadata (it appears to do so twice). That data should already be cached. Loading it can be quite costly and defeat the purpose of doing an indexed search in the first place. This gets even worse when there are many fragments in a dataset because the manifest is quite large.
The text was updated successfully, but these errors were encountered: