Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ballista] Fix regression in roundtrip_logical_plan_custom_ctx test #481

Open
andygrove opened this issue May 16, 2022 · 2 comments
Open
Labels
ballista bug Something isn't working

Comments

@andygrove
Copy link
Member

andygrove commented May 16, 2022

Describe the bug
PR apache/datafusion#2537 fixed an issue where the csv scan methods were using the full URI instead of the path when serializing csv scans, which was not consistent with the way other scans worked (parquet, avro, json). Making this consistent led to a regression in roundtrip_logical_plan_custom_ctx so the test was ignored for now.

We should re-enable this test.

To Reproduce
Run roundtrip_logical_plan_custom_ctx.

Expected behavior
Functionality should be consistent between file types.

Additional context
None

@andygrove andygrove added bug Something isn't working ballista labels May 16, 2022
@andygrove andygrove changed the title [Ballista] Custom object stores only work for CSV files [Ballista] Fix regression in roundtrip_logical_plan_custom_ctx test May 16, 2022
@andygrove
Copy link
Member Author

@thinkharderdev @mingmwang I could use some help with this if you have time to look.

@thinkharderdev
Copy link
Contributor

Yeah, the issue here is that the serde logic uses the file scheme to resolve the ObjectStore. So we were relying on the test scheme to resolve the TestObjectStore. Why would we not serialize the entire file URI?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ballista bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants