Add option to parse single objects at a time #196
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Ensure primitives are "true", "false", "null", or an RFC 8259 compliant number. (Still need to add test cases.)
String parsing previously did not differ between strict and non-strict modes, but was not fully compliant with RFC 8259. RFC 8259 requires that control characters (code points < 0x20) be escaped. This is now enforced in strict mode. In addition, non-strict mode now does *no* validations on string contents, much like primitives in non-strict mode.
dominickpastore
force-pushed
the
single-parsing
branch
from
cf00588
to
b494feb
Compare
|
Apologize for the history rewrite. Rebased onto the latest changes from the pull requests this was built on top of. |
Parent links and strict parsing are now the default behavior. New macros
JSMN_LOW_MEMORY and JSMN_NON_STRICT disable these behaviors.
JSMN_PARENT_LINKS still exists, but is defined by default unless
JSMN_LOW_MEMORY is defined.
JSMN_STRICT no longer exists. Instead, we have three new macros:
JSMN_PERMISSIVE_PRIMITIVES - Relaxes validation of primitives. Any
characters except whitespace and {}[],:" become allowed. (Normally, only
"true", "false", "null", and RFC 8259 numbers are permitted.)
JSMN_PERMISSIVE_STRINGS - Relaxes validation of strings. Any characters
allowed. (Normally, control characters (<0x20) and invalid escape
sequences are foridden.)
JSMN_PRIMITIVE_KEYS - Allows primitives to be used as object keys.
These can be defined individually, or defining JSMN_NON_STRICT will
cause all to be defined.
Tests have not yet been adapted for these changes.
Previously, jsmn parsed all input provided, parsing multiple objects if present. If the last object is incomplete, it would return JSMN_ERROR_PART, even if there was at least one complete object before it. This makes it difficult to parse streams of objects: The input reader must ensure the input buffer ends on an object boundary. The JSMN_SINGLE macro provides a solution to this by configuring jsmn to parse objects one at a time. As soon as a complete object is parsed, jsmn returns, ignoring the rest of the input. The parser state will be reinitialized, so to parse the next object, simply advance the input buffer pointer ahead by tokens[0].end characters and call jsmn_parse() again.
dominickpastore
force-pushed
the
single-parsing
branch
from
b494feb
to
fe33cb8
Compare
Open
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request is built on top of PR #194 and PR #195 (thus the commits for those branches are showing as well). If those PRs are not accepted but there is interest in this one, I can rebase this branch onto
master.This PR allows parsing one object at a time:
Normally, jsmn parses the entire input string. If multiple objects are present, it will parse all of them consecutively into the
tokensarray. If the last object is incomplete, jsmn returnsJSMN_ERROR_PART, even if there were one or more complete objects before it. This can make parsing a stream of JSON objects difficult. The input reader must ensure the input buffer passed to jsmn ends on an object boundary.This PR adds a new macro,
JSMN_SINGLE, to provide a solution to this. When defined, jsmn will only parse one object at a time. Once it has parsed a complete object, it returns immediately, ignoring the rest of the input string. The parser state will be reinitialized, so to parse the next object, simply advance the input buffer pointer ahead bytokens[0].endand calljsmn_parse()again.