Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cast fixed-length string -> std::string missing #505

Closed
JATC1024 opened this issue Feb 23, 2022 · 11 comments
Closed

Cast fixed-length string -> std::string missing #505

JATC1024 opened this issue Feb 23, 2022 · 11 comments
Labels
bug enhancement String Anything related to handling strings, sequences of chars.

Comments

@JATC1024
Copy link

Is your feature request related to a problem? Please describe.
H5Easy.load cannot load fixed length strings. I always get the error Can't output std::string as fixed-length. Use raw arrays or FixedLenStringArray. I tried to use HighFive::FixedLenStringArray<N> and it worked. However, I had to know the length of the strings beforehand since N must be a constant. This does not seem viable to me.

Describe the solution you'd like
Something like

int size = dataset.getStringSize();
HighFive::FixedLenStringArray strs(size);

would be nice. Or even better if H5Easy will handle everything so that I don't have to care if the dataset is fixed length or not.

@ferdonline
Copy link
Contributor

Hey.
Your request is understandable but I believe HighFive already implements the required API.
Have you tried along the lines? (not tested):

int size = dataset.getDataType().getSize();
HighFive::FixedLenStringArray strs(size);

@JATC1024
Copy link
Author

Thanks for replying. I think my comment here best describes my problem: #94 (comment).

Looking through the document, I cannot find a way to construct FixedLenStringArray without knowing N beforehand. Did I miss something?

@JATC1024
Copy link
Author

JATC1024 commented Feb 23, 2022

I think the problem here is that FixedLenStringArray<N> is a std::vector<std::array<T, N>> in disguise, which requires knowing N in compile time.

Therefore,

const int len = 10;
HighFive::FixedLenStringArray<len> strs;

works but

const int len = dataset.getDataType().getSize();
HighFive::FixedLenStringArray<len> strs;

doesn't.

@ferdonline
Copy link
Contributor

Oh, I'm sorry I overlooked the fact that the signature of FixedLenStringArray is FixedLenStringArray<N>, then N must be known at compile time.
I find your use case valid. There are plans to create a replacement of FixedLenStringArray but unfortunately our bandwidth is limited. If you feel comfortable to draft a solution, along the lines of using regular std::vector with std::string_view but keep compat with c++11 (like https://github.com/bitwizeshift/string_view-standalone) I would be happy to review.

@JATC1024
Copy link
Author

I'll try to look into this in my free time. For now, I will simply set N as a big enough integer.

@tdegeus
Copy link
Collaborator

tdegeus commented Feb 23, 2022

@JATC1024 You probably have a good reason to do what you do. Still, std::string may not be fixed-size, but is still pretty 'easy':

auto mystring = H5Easy::load<std::string>(file, "/path/to/my/string");

Also, unlike @ferdonline I'm not super sure that a 'fixed-size' string object that is dynamically allocated is relevant. Wouldn't that we std::string that you reserve to a certain size? Also this you can do currently (though not in H5Easy but in HighFive).
The current compile-time allocated truly fixed-size string is different. It presumably lives on the stack on might gain you some speed (though I doubt a lot, because I think that disk-IO will always be limiting).

@JATC1024
Copy link
Author

JATC1024 commented Feb 23, 2022

@tdegeus
Oh I think I didn't describe the problem properly. I want to load an array of strings, not just a single string. What I'm trying to do is:

std::vector<std::string> strs;
strs = H5Easy::load<std::vector<std::string>>(file, "/path/to/strings")`

For variable-length strings, this works. For fixed-length strings, it shows the error Can't output std::string as fixed-length. Use raw arrays or FixedLenStringArray.

Or are you suggesting that I can load each string in the array individually? Something like

auto mystring = H5Easy::load<std::string>(file, "/path/to/strings", index);

I see you do have a function like that. I will try it tomorrow, though I think I will get the same error.

@tdegeus
Copy link
Collaborator

tdegeus commented Feb 23, 2022

Thanks for the clarification @JATC1024 . This makes a lot of sense. I should say that I'm not at all an expert on HDF5 fixed-length strings. That being said, I see no reason why HighFive should not provide a method to load strings that are stored as fixed-length to std::string. I think that this should be implemented in the 'core' for std::string types. In that case it should be automatically available in H5Easy as well.

@tdegeus
Copy link
Collaborator

tdegeus commented Mar 9, 2022

Shall we rename this issue (or open a new one) indicating that it is more than missing conversion from fixed-length strings to std::string?

@JATC1024
Copy link
Author

@tdegeus sure, whatever way you see fit. If you create a new one then I think we can close this one.

@tdegeus tdegeus changed the title H5Easy is not easy enough Cast fixed-length string -> std::string missing Apr 7, 2022
@1uc 1uc added bug String Anything related to handling strings, sequences of chars. labels Mar 17, 2023
@1uc
Copy link
Collaborator

1uc commented Oct 31, 2023

With #744 merged std::strings and containers thereof, e.g. std::vectorstd::string can be read and written to fixed or variable length HDF5 strings. There's an example here:
https://github.com/BlueBrain/HighFive/blob/master/src/examples/read_write_std_strings.cpp

@1uc 1uc closed this as completed Oct 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug enhancement String Anything related to handling strings, sequences of chars.
Projects
None yet
Development

No branches or pull requests

4 participants