Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: general value extractor #95

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

lonerapier
Copy link
Collaborator

@lonerapier lonerapier commented Oct 8, 2024

closes #94 #67

  • adds a general serde-like Value extractor from arbitrary JSON
  • Adds 2 different templates for object and array

Logic is similar to before:

  • check key matches
  • check inside value
  • propogate matched key to value
  • create mask
  • create a subarray from mask

@lonerapier lonerapier changed the title Feat/general value extractor feat: general value extractor Oct 8, 2024
Copy link
Contributor

@Autoparallel Autoparallel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thoughts on this: Do we want to have a bit more logical separation that is conducive to NIVC?

For example, perhaps we can parse the JSON into the machine. Pass the machine as step_out into interpreter stage that grabs the location (indices) of where the intended object is and then pass on this new subarray of the machine into the next iteration of this process.

For example, consider:

{
   "a": {
       "inner_a": { 
           "inner_inner_a": "my_string_value"
        }
    }
}

where we want to extract a.inner_a.inner_inner_a. We parse the whole JSON, pass machine into ExtractValue which outputs data of the start and end of a (and here we can even trim bottom off stack and truncate the machine state further possibly, though let's not get into pre-mature optimization), then this is passed again into ExtractValue, and it returns start and end indices of inner_a as step_out. Finally, we know we are at the final retrieval so we call a new circuit instance in NIVC ExtractStringValue which then hunts within the range output previously for inner_inner_a and gets the String value there.

I imagine the role of array is similar and we can just choose to use ExtractArray in NIVC

@lonerapier lonerapier linked an issue Oct 11, 2024 that may be closed by this pull request
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat(extractor): general serde-based value extractor feat(extractor): improvements
2 participants