Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggestion: Invalid lines are ignored, or more easily implemented, lines must start with trimmed '{' or '[' #65

Open
jvlake opened this issue Mar 1, 2023 · 2 comments

Comments

@jvlake
Copy link

jvlake commented Mar 1, 2023

Suggestion: Invalid lines are ignored, or more easily implemented -> lines must start with trimmed '{' or '['

I love this format but the restriction that every line MUST be valid json would be better if invalid lines were just ignored.

Implementing an invalid-json-checker that actually checks the validity of each line has too great of a complexity for the specification, but adding a very simple rule. lines that don't start with '{' or '[' are ignored from processing. now that's cooking with gas.

If this breaks the strict "jsonlines standard" maybe call it jsonlines-loose.

---- begin of example jsonlines.jsonll ---- 
# I can now be a comment because the row will be skipped.
{ "msg": "I'll be processed" }
{ "msg": "I'll blow up by whatever parses the row as json is not closed ->"
    {"msg":"I can be a legit  row because string.trim() is simple in most languages and starts with a known good character"}
---- end of example jsonlines.jsonll ---- 
@jvlake jvlake changed the title Suggestion: Invalid lines are ignored, or more easily implemented lines must start with trimmed '{' or '[' Suggestion: Invalid lines are ignored, or more easily implemented, lines must start with trimmed '{' or '[' Mar 1, 2023
@wardi
Copy link
Owner

wardi commented Mar 1, 2023

Adding comments/ignored lines makes it more difficult to tell which line of a file corresponds to which record. Without comments, record number == line number, making using unix text tools much easier.

The suggestion here would also silently ignore some records that used to be processed (ones starting with whitespace, plain strings, numbers, boolean, null values) and makes implementation more complicated.

@jvlake
Copy link
Author

jvlake commented Mar 2, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants