Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce SelectiveStreamReader #12991

Merged
merged 16 commits into from
Jul 1, 2019
Merged

Commits on Jun 26, 2019

  1. Configuration menu
    Copy the full SHA
    f5ffcfa View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2019

  1. Configuration menu
    Copy the full SHA
    603cf54 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a1029d6 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    137ee50 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    5e436ea View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    29b5131 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    a9714f5 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    9eb640c View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    aa90c8a View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    33cad09 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    29d2319 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    b5aff54 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    b9dbde6 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    4910e70 View commit details
    Browse the repository at this point in the history
  14. Fold BenchmarkOrcDecimalReader into BenchmarkBatchStreamReaders

    ```
    
    Benchmark                                             Mode  Cnt  Score   Error  Units
    BenchmarkBatchStreamReaders.readLongDecimalNoNull     avgt   60  0.829 ± 0.032   s/op
    BenchmarkBatchStreamReaders.readLongDecimalWithNull   avgt   60  0.566 ± 0.015   s/op
    BenchmarkBatchStreamReaders.readShortDecimalNoNull    avgt   60  0.544 ± 0.026   s/op
    BenchmarkBatchStreamReaders.readShortDecimalWithNull  avgt   60  0.364 ± 0.006   s/op
    ```
    mbasmanova committed Jun 27, 2019
    Configuration menu
    Copy the full SHA
    b4da3ea View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2019

  1. Introduce SelectiveStreamReader

    SelectiveStreamReader enables more efficient scan by applying a filter while
    extracting values only from the specified rows.
    
    This commit includes an implementation of the new SelectiveStreamReader for the
    boolean type.
    
    The new benchmark BenchmarkSelectiveStreamReaders is similar to
    BenchmarkBatchStreamReaders, but uses SelectiveStreamReader. The performance
    of a single column no filter case is the same for batch and selective readers.
    
    BenchmarkSelectiveStreamReaders.readAllNull                            boolean  avgt   20  0.003 ±  0.001   s/op
    BenchmarkSelectiveStreamReaders.readBooleanNoNull                          N/A  avgt   20  0.029 ±  0.001   s/op
    BenchmarkSelectiveStreamReaders.readBooleanNoNullWithFilter                N/A  avgt   20  0.098 ±  0.006   s/op
    BenchmarkSelectiveStreamReaders.readBooleanWithNull                        N/A  avgt   20  0.098 ±  0.004   s/op
    BenchmarkSelectiveStreamReaders.readBooleanWithNullWithFilter              N/A  avgt   20  0.134 ±  0.006   s/op
    
    BenchmarkBatchStreamReaders.readAllNull                  boolean  avgt   20  0.003 ±  0.001   s/op
    BenchmarkBatchStreamReaders.readBooleanNoNull                N/A  avgt   20  0.030 ±  0.001   s/op
    BenchmarkBatchStreamReaders.readBooleanWithNull              N/A  avgt   20  0.106 ±  0.002   s/op
    mbasmanova committed Jun 29, 2019
    Configuration menu
    Copy the full SHA
    d89159d View commit details
    Browse the repository at this point in the history