AirIndex

Learned index on external memory model for key-value data

Progress Tracker

• Implement io module to file system
• Setup unit test (github)
• Setup lint (github)
• Static storage profiler
• Block-based variable-length store (micro pages)
  • Writer: key-bytes
  • Reader: read partial in range
• Fix-length array store
• Add dataset: fixed-sized elements (e.g. SOSD)
  • Improve keysets with answers
  • Read and query from keysets
• Step function
  • Model builder
  • Layer builder
• Linear function
  • Model builder
  • Layer builder
• Stack and balance index builder
• Parallel builder connector
• Partition builder connector
• Meta-serializable structs
• Rank DB
• Key-value DB with writing
• Azure connector

Optimization List

• Async IO
• Zero-copy block store reads
• IO buffer pool (or mmap)
  • IO buffer pool explicit
• Root layer in metadata

Refactoring List

• Metaserde cleaner pattern