Follow search performance on basic queries

[fmassot]

We could use a restricted set of queries used in tantivy benchmark.

Would be also interesting to know the perf with parallel queries.

[PSeitz]

Let's split this into subtasks:

• Load test-data
  • Create indexes via CLI - https://github.com/PSeitz/qw_build_index
  • Ingest via CLI
  • Store and retrieve datasets from S3
• Run set of queries
  • Have different queries defined in a yaml or toml
  • Start server via CLI (multiple configs, matrix?, memory layout randomization)
  • Warmup caches
  • Run queries
• Retrieve and store benchmarks in a DB (could be Quickwit :)
• Have a dedicated machine running continuously

[zamazan4ik]

By the way, preparing a good benchmark set will help a lot with using PGO for Quickwit. Without some "generic" sample load it is much more time consuming to prepare PGO-optimized binary.

[fmassot]

Interesting to see how databend is doing it: databendlabs/databend#3084

[zamazan4ik]

Regarding PGO (and Bolt) possibly these links could be helpful:

• ScyllaDB results: build: introduce LTO, PGO and BOLT to the build scylladb/scylladb#10808
• Vector results: PGO applicability to Vector vectordotdev/vector#15631
• Rust experience with LTO + PGO + BOLT: https://kobzol.github.io/rust/rustc/2022/10/27/speeding-rustc-without-changing-its-code.html