Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Explore the Apache Arrow ecosystem for better alternatives to HDF5 #170

Open
jacg opened this issue Nov 10, 2023 · 1 comment
Open

Explore the Apache Arrow ecosystem for better alternatives to HDF5 #170

jacg opened this issue Nov 10, 2023 · 1 comment

Comments

@jacg
Copy link
Owner

jacg commented Nov 10, 2023

No description provided.

@jacg jacg added this to nain4 Nov 10, 2023
@github-project-automation github-project-automation bot moved this to Triage in nain4 Nov 10, 2023
@jacg jacg moved this from Triage to In Progress in nain4 Nov 10, 2023
@jacg
Copy link
Owner Author

jacg commented Jan 9, 2024

We're using Parquet proof of concepts for IO.

We have figured out how to do most things we want. Writing the client code in C++ is an immense PITA, but no more so than writing HDF5 code in C++.

Ideally we should create abstractions for parquet IO code in C++, which is going to be difficult for arbitrarily general schemas, but still more feasible than the equivalent in HDF5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
nain4
Status: In Progress
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jacg