Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add event logging to Malloy #1929

Merged
merged 16 commits into from
Oct 3, 2024
Merged

Add event logging to Malloy #1929

merged 16 commits into from
Oct 3, 2024

Conversation

christopherswenson
Copy link
Contributor

@christopherswenson christopherswenson commented Sep 25, 2024

Adds:

  • A new EventStream interface with an emit(id: string, data: any): void function (for which an EventEmitter can be used)
  • A new parameter eventStream to Runtime, QueryModel, and MalloyTranslator, as well as Malloy.parse, Malloy.compile, SingleConnectionRuntime, and preparedQuery.getPreparedResult*, which is an EventStream
  • DB Jest Matchers toEmitDuringCompile and toEmitDuringTranslation, which ensure that a given Malloy query/model produces a given set of events**
  • A few initial events:
    • source-argument-compiled: the argument to a source parameter was compiled in an actual query
    • parameterized-source-compiled: a source that has parameters is used in a query (either in the pipeline or a join)
    • join-used: a joined source is used
    • translation-error: an error is encountered during translation

*preparedQuery.preparedResult creates a QueryModel and runs the compiler; preparedQuery.getPreparedResult now allows passing in the eventStream; this is automatically done when using queryMaterializer.getPreparedResult() or queryMaterializer.run(), passing the eventStream from the runtime that produced the query materializer.

**The exact behavior of toEmitDuringCompile and toEmitDuringTranslation is: 1) load the model or query: if ...DuringCompile, runs loadModel().loadFinalQuery().getPreparedResult(), if ...DuringTranslation, runs loadModel().getModel() 2) check that the events that were emitted match those expected: filters the emitted events to only those with IDs matching any in the list of expected events, then ensures that ALL of those events match the events given, in order, by ensuring that the ID matches and the data object matches (loosely: keys present in expected must be equal if scalar, or recursively match if arrays or objects)

@christopherswenson christopherswenson merged commit 0b18922 into main Oct 3, 2024
10 checks passed
@christopherswenson christopherswenson deleted the crs-event-logging branch October 3, 2024 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants