refactor: add scan_to_stream() to Table trait to postpone the stream generation

[waynexia]

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

Intension

scan() method on Table returns a physical plan (execution plan in DF) directly. This manner has several limitations:

• Our scan options are highly tightened to DataFusion. Table implementations are usually simply adapted to DataFusion's table source, and it is hard to add new options
• The scan options may change during plan optimization. Thus we may generate the execution plan multiple times.
• There is still some I/O happens during scan().

Changes

Add a new interface scan_to_stream() to replace the existing scan(). It returns a stream that can be adapted to execution plan. And leverage the middle layer DfTableProviderAdapter as the only entry point of all the Table implementations. It holds a Table reference and a mutable ScanRequest that can be changed during the optimization phase. We can further postpone the call to scan_to_stream() to the return value of DfTableProviderAdapter::scan(), to avoid I/O during planning.

The main change is the new interface scan_to_stream() and deprecating scan(). InformationSchema and file table engine have some noticeable changes due to this.

Checklist

• I have written the necessary rustdoc comments.
• I have added the necessary unit tests and integration tests.

Refer to a related PR or issue link (optional)

[waynexia]

This is a temporary regression due to the lack of an optimization rule.

[codecov]

Codecov Report

Merging #1639 (96236b5) into develop (d072947) will decrease coverage by 0.26%.
The diff coverage is 86.87%.

❗ Current head 96236b5 differs from pull request most recent head 892d9f5. Consider uploading reports for the commit 892d9f5 to get more accurate results

@@             Coverage Diff             @@
##           develop    #1639      +/-   ##
===========================================
- Coverage    85.73%   85.48%   -0.26%     
===========================================
  Files          566      566              
  Lines        90433    90795     +362     
===========================================
+ Hits         77537    77614      +77     
- Misses       12896    13181     +285     

[v0y4g3r]

Looks like the failing CI checks checked out an unrelated commit id 9ddb79ae91e23e17b16421b5a34f7d70aa6a1fd8. Let force-merge this PR and proceed.