generational rewriting (IO locality tracks)

[spacejam]

we have an opportunity to be smarter about locality when rewriting pages. If we 2 or more IoBufs structures for different generations/tracks, we could avoid rewriting data that is mostly being merged into base nodes.

This is mostly a spitball issue for different ideas and thoughts on the constraints.

Random thoughts:

• Give each track its own IoBufs struct
• put replaces on one track, links on another, maybe have rewrite counters on pages for placement in one of several more mature tracks
• during recovery, we could zip along all levels/tracks/generations in a single pass, merging updates together. replace the current Lsn byte counter with a logical counter, so we know when we hit the edge while zipping along the N levels/generations/tracks
• periodic fsync'ing on segments becomes pretty much mandatory, as more mature generations will fill up more slowly, and block recovery nevertheless.
• low water mark tracking needs to happen per-track
• for each track, keep the highest reserved ID stored in-memory as well as the highest ID fsync'ed
• to make_stable(ID), we need to iterate over all tracks and ensure that the highest fsync'ed ID is either above the given ID, or that the highest fsync'ed ID is the same as the highest reserved ID.
• to mark_interval, provide (track, highest reserved ID, highest synced ID) and set the current low water mark to the lowest synced ID among tracks for which highest reserved != highest synced