Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] JSON reader should parse quoted integer values as null #4793

Closed
andygrove opened this issue Feb 15, 2022 · 2 comments
Closed

[FEA] JSON reader should parse quoted integer values as null #4793

andygrove opened this issue Feb 15, 2022 · 2 comments
Assignees
Labels
feature request New feature or request

Comments

@andygrove
Copy link
Contributor

Is your feature request related to a problem? Please describe.
Given the following JSON file, Spark will produce the values 1 and null, but on GPU we would produce 1 for both rows (once #4790 is merged).

{ "number": 1 }
{ "number": "1" }

Describe the solution you'd like
We should have consistent behavior with Spark. We need to remove the XFAIL from the test in json_test.py that references this issue.

Describe alternatives you've considered
None

Additional context
Depends on rapidsai/cudf#10283

@andygrove andygrove added feature request New feature or request ? - Needs Triage Need team to review and classify labels Feb 15, 2022
@jlowe jlowe removed the ? - Needs Triage Need team to review and classify label Feb 22, 2022
@revans2
Copy link
Collaborator

revans2 commented Feb 21, 2024

I should clarify a bit here. This is true for integer types, ByteType, ShortType, IntegerType, and LongType when using ScanJson or JsonToStructs. It is not true for GetJsonObject or JsonTuple. It is not true for DecimalType, which happily accepts quoted strings.

@revans2
Copy link
Collaborator

revans2 commented Mar 13, 2024

This was fixed as a part of #10542 but one of the tests marked with it was still failing because of a separate regression related to how ints are parsed as dates. That test is being updated.

@revans2 revans2 closed this as completed Mar 13, 2024
@revans2 revans2 self-assigned this Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants