Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update lz4 flex #33

Closed
wants to merge 650 commits into from
Closed

update lz4 flex #33

wants to merge 650 commits into from

Conversation

PSeitz
Copy link
Owner

@PSeitz PSeitz commented Jan 31, 2023

PSeitz and others added 30 commits September 30, 2022 10:06
Co-authored-by: Paul Masurel <paul@quickwit.io>
fix docstore settings for temp docstore
These show on nightly clippy.
This case is inverted from the previous cases fixed.

This is from nightly clippy.
This also fixes a couple of typos, but plenty remain!
Tantivy needs the `total_cmp` feature to compile, which was stabilized
in Rust 1.62.
remove get_val in serialization and mark as unimplemented!()
replace get_val with iter in linear codec
remove MultivalueStartIndexRandomSeeker
replace MultivalueStartIndexIter with closure
Sample 100 values in linear codec
* use new format for docstore blocks

* move index to end of block

it makes writing the block faster due to one less memcopy
Removing alloc on all .next() in MultiValueColumn
don't materialize index column on merge
use simpler chain() variant
Co-authored-by: Paul Masurel <paul@quickwit.io>
disable linear codec for multivalue values
The pool of searchers was removed in 23fe73a as part of #1411.
guilload and others added 28 commits January 16, 2023 22:58
…om-stats-agg

Remove standard deviation from stats aggregation
start migrate Field to &str in preparation of columnar
return Result for get_field
* add way to get up to `limit` terms from sstable

* make some function of sstable load less data

* add some tests to sstable

* add tests on sstable dictionary

* fix some bugs with sstable
* Update text_options.rs

* Update src/schema/text_options.rs

Co-authored-by: Paul Masurel <paul@quickwit.io>
* Improvement on the scalar / random bitpacker code.

Added proptesting
Added simple benchmark
Added assert and comments on the very non trivial hidden contract
Remove the need for an extra padding.

The last point introduces a small performance regression (~10%).

* Fixing unit tests
Some parts are outdated

For the debugging tutorial, debugging is really easy now with VSCode, and there are plenty of other sources for debugging rust
* collect columns for merge

* return column_type from, fix visibility

* fix

Co-authored-by: Paul Masurel <paul@quickwit.io>
…1816)

* Removed the sorting logic that forced column type to be sorted like
ColumnTypes.

* add comments

Co-authored-by: PSeitz <PSeitz@users.noreply.github.com>
* fix doc store cache docs

addresses an issue reported in #1820

* rename doc_store_cache_size
formatting on nightly changed
@PSeitz PSeitz closed this Jan 31, 2023
@fulmicoton fulmicoton deleted the update_lz4_flex branch February 9, 2023 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Expose a public API for interacting with JSON field types outside of the QueryParser