perf: re-enable late materialization on full scans

[westonpace]

This also fixes the "disable pushdown if custom batch size requested" logic which wasn't working correctly. I also added a test to ensure that we are getting the benefits we expect.

[westonpace]

Leaving in draft until #2288 (a prereq) is merged

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 77.51196% with 47 lines in your changes are missing coverage. Please review.

Project coverage is 80.67%. Comparing base (be4110d) to head (65bc2cd).

Files	Patch %	Lines
rust/lance/src/utils/test.rs	45.88%	32 Missing and 14 partials ⚠️
rust/lance/src/dataset/scanner.rs	99.19%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2290      +/-   ##
==========================================
+ Coverage   80.66%   80.67%   +0.01%     
==========================================
  Files         191      191              
  Lines       56113    56308     +195     
  Branches    56113    56308     +195     
==========================================
+ Hits        45262    45426     +164     
- Misses       8319     8340      +21     
- Partials     2532     2542      +10     

Flag	Coverage Δ
unittests	80.67% <77.51%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[wjones127]

Nice. Thanks for good tests.