Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Read ORC does not ignoreCorruptFiles #4795

Closed
wbo4958 opened this issue Feb 16, 2022 · 0 comments · Fixed by #4809
Closed

[BUG] Read ORC does not ignoreCorruptFiles #4795

wbo4958 opened this issue Feb 16, 2022 · 0 comments · Fixed by #4809
Assignees
@wbo4958
Labels
bug Something isn't working

Comments

@wbo4958
Copy link
Collaborator

wbo4958 commented Feb 16, 2022

This is a follow-up of #4727. ORC mutlithreaded and coalescing reading will throw exception even ignoreCorruptFiles is set when reading the corrupt file.
@wbo4958 wbo4958 added bug Something isn't working ? - Needs Triage Need team to review and classify labels Feb 16, 2022
@wbo4958 wbo4958 self-assigned this Feb 16, 2022
@wbo4958 wbo4958 mentioned this issue Feb 17, 2022
Merged
@jlowe jlowe linked a pull request Feb 18, 2022 that will close this issue
Add ignoreCorruptFiles for ORC readers [databricks] #4809
Merged
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label Feb 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wbo4958 wbo4958

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add ignoreCorruptFiles for ORC readers [databricks] wbo4958/spark-rapids
2 participants
@wbo4958 @sameerz