Skip to content

Commit

Permalink
Allow seeking backwards within a prefetch stream
Browse files Browse the repository at this point in the history
Linux asynchronous readahead confuses our prefetcher by sometimes making
the stream appear to go backwards, even though the customer is actually
just reading sequentially (awslabs#488). The problem is that with parallel FUSE
threads, the two asynchronous read operations can arrive to the
prefetcher out of order.

This change allows us to tolerate a little bit of backwards seeking in a
prefetch stream. We keep around a little bit of previously read data and
can reload it in the event that a seek goes backwards. We do this by
creating a fake new request containing the rewound bytes, so that the
existing read logic will pick them up. I chose an arbitrary max for the
backwards seek buffer, big enough to handle Linux readahead.

This should fix the readahead issue: in my testing, I no longer saw slow
sequential reads, and the logs confirmed this seeking logic was being
triggered in both directions (forwards and backwards), consistent with
the readahead requests sometimes arriving out of order.
  • Loading branch information
jamesbornholt committed Oct 13, 2023
1 parent 1cc6e72 commit 057e7db
Show file tree
Hide file tree
Showing 4 changed files with 243 additions and 32 deletions.
Loading

0 comments on commit 057e7db

Please sign in to comment.