Skip to content
This repository has been archived by the owner on Mar 21, 2024. It is now read-only.

Telemetry - Missing events and options #186

Merged
merged 12 commits into from
Nov 29, 2022
Merged

Telemetry - Missing events and options #186

merged 12 commits into from
Nov 29, 2022

Conversation

gmourier
Copy link
Member

@gmourier gmourier commented Oct 18, 2022 •
edited
Loading

Summary

Catch-up specification to add missing events and add/rework properties of existing events.

Changes

  • Add Documents Deleted aggregated event
    • Add user-agent property
    • Add requests.total_received property.
    • Add per_document_id property.
    • Add clear_all property.
    • Add per_batch property.
  • Add Health Seen aggregated event
    • Add user-agent property
    • Add requests.total_received property
  • Add Version Seen event
    • Add user-agent property
  • Update SearchableAttributes Updated event
    • Add searchable_attributes.with_wildcard
  • Add DisplayedAttributes Updated event
    • Add user-agent property
    • Add displayed_attributes.total property
    • Add displayed_attributes.with_wildcard property
  • Add DistinctAttribute Updated event
    • Add user-agent property
    • Add distinct_attribute.set property
  • Add StopWords Updated event
    • Add user-agent property
    • Add stop_words.total property
  • Add Synonyms Updated event
    • Add user-agent property
    • Add synonyms.total property
  • Update Settings Updated event
    • Add searchable_attributes.with_wildcard
    • Add displayed_attributes.total property
    • Add displayed_attributes.with_wildcard property
    • Add distinct_attribute.set property
    • Add stop_words.total property
    • Add synonyms.total property
  • Update Documents Searched POST and Documents Searched GET events
    • Add formatting.max_attributes_to_retrieve property
    • Add formatting.max_attributes_to_highlight property
    • Add formatting.max_attributes_to_crop property
    • Add facets.avg_facets_number property
    • Remove facets property
    • Rename most_used_matching_strategy to matching_strategy.most_used_strategy
  • Update Settings Updated event
    • Add ranking_rules.words_position property
    • Add ranking_rules.typo_position property
    • Add ranking_rules.proximity_position property
    • Add ranking_rules.attribute_position property
    • Add ranking_rules.exactness_position property
    • Add ranking_rules.values property
  • Update RankingRules Updated event
    • Add ranking_rules.words_position property
    • Add ranking_rules.typo_position property
    • Add ranking_rules.proximity_position property
    • Add ranking_rules.attribute_position property
    • Add ranking_rules.exactness_position property
    • Add ranking_rules.values property
  • Add instance properties
    • infos.ssl_auth_path
    • infos.ssl_cert_path
    • infos.ssl_key_path
    • infos.ssl_ocsp_path
    • infos.ssl_require_auth
    • infos.ssl_resumption
    • infos.ssl_tickets
  • Tasks Seen is now an aggregated event.
    • Add requests.total_received property

Out Of Scope

N/A

Attention To Reviewers

This is a catch-up. Dedicated new features will add their telemetry in their given specification PR.

Misc

  • Update telemetry datapoints

@gmourier gmourier added Draft Feature specification is in draft state. Summary and Motivation parts need to be written. v0.30 Telemetry Update the telemetry collect. labels Oct 18, 2022
@gmourier gmourier mentioned this pull request Oct 18, 2022
Merged
1 task
@gmourier gmourier changed the title Telemetry changes Telemetry catch-up Oct 18, 2022
@gmourier gmourier changed the title Telemetry catch-up Telemetry - Missing events and options Oct 18, 2022
@gmourier gmourier marked this pull request as ready for review October 24, 2022 15:31
@gmourier gmourier added Ready For Review Feature specification must be reviewed. and removed Draft Feature specification is in draft state. Summary and Motivation parts need to be written. labels Oct 24, 2022
@gmourier gmourier requested review from curquiza and irevoire October 24, 2022 15:52
curquiza
curquiza approved these changes Oct 27, 2022
View reviewed changes
Copy link
Member

@curquiza curquiza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approve to say I saw it but I would rather @irevoire approved

@gmourier gmourier mentioned this pull request Nov 2, 2022
Closed
1 task
irevoire
irevoire suggested changes Nov 2, 2022
View reviewed changes
Copy link
Member

@irevoire irevoire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're good in terms of code to write

text/0034-telemetry-policies.md Show resolved Hide resolved
text/0034-telemetry-policies.md Outdated Show resolved Hide resolved
text/0034-telemetry-policies.md Outdated Show resolved Hide resolved
gmourier and others added 2 commits November 2, 2022 19:45
@gmourier @irevoire
Update text/0034-telemetry-policies.md
f690029 
Co-authored-by: Tamo <tamo@meilisearch.com>
@gmourier @irevoire
Update text/0034-telemetry-policies.md
174825e 
Co-authored-by: Tamo <tamo@meilisearch.com>
@gmourier gmourier requested a review from irevoire November 14, 2022 13:46
irevoire
irevoire approved these changes Nov 14, 2022
View reviewed changes
Copy link
Member

@irevoire irevoire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see anything wrong, but it's hard to be 100% sure.
I'll let you know if I find anything while implementing all these new analytics 👀

@gmourier gmourier force-pushed the telemetry-catch-up branch 2 times, most recently from 149200c to 2c36375 Compare November 23, 2022 10:57
@gmourier
Copy link
Member Author

Signal to @maryamsulemani97; I'm sorry for the last minute additions 🙇‍♂️

31b437b

irevoire
irevoire suggested changes Nov 23, 2022
View reviewed changes
Copy link
Member

@irevoire irevoire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gmourier @maryamsulemani97, Sorry, I had to change the ranking rules values from an array to a string because amplitude doesn’t know how to group by array, and thus the metrics were unusable.

text/0034-telemetry-policies.md Outdated Show resolved Hide resolved
text/0034-telemetry-policies.md Outdated Show resolved Hide resolved
gmourier and others added 3 commits November 23, 2022 15:03
@gmourier @irevoire
Update text/0034-telemetry-policies.md
66199f3 
Co-authored-by: Tamo <tamo@meilisearch.com>
@gmourier @irevoire
Update text/0034-telemetry-policies.md
32ef84e 
Co-authored-by: Tamo <tamo@meilisearch.com>
@gmourier
Update text/0034-telemetry-policies.md
5f4b6d2
@gmourier gmourier added Implemented Feature specification has been implemented. and removed Ready For Review Feature specification must be reviewed. labels Nov 29, 2022
@gmourier gmourier merged commit 4ea6141 into release-v0.30.0 Nov 29, 2022
@gmourier gmourier deleted the telemetry-catch-up branch November 29, 2022 14:24
gmourier added a commit that referenced this pull request Nov 29, 2022
@gmourier @curquiza
@irevoire @Kerollmops @bidoubiwa @brunoocasali @ManyTheFish @alallema @dureuill
Release v0.30.0 (#176)
b2e4b3d 
* Bump Meilisearch OpenAPI spec to v0.30.0

* Compression Headers (#178)

* Start specification

* Rename spec file

* Rephrase summary

* Rephrase receiving a compressed response section

* Add links to RFC9910

* Introduce missing_master_key error (#179)

* Update error message (#180)

* Specify the error variant (#181)

* replaces 127.0.0.1 by localhost (#184)

* Configuration File Support (#185)

* init spec file

* Fill-in spec

* Fix typing errors

* adds errors case

* Update text/0185-configuration-file.md

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>

* Replace config file by configuration file in the error message

* Update text/0185-configuration-file.md

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>

* Update text/0185-configuration-file.md

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>

* Update text/0185-configuration-file.md

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>

* Update text/0185-configuration-file.md

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>

* Add infos.with_configuration_file to telemetry-policies

* Update text/0185-configuration-file.md

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>

* Update text/0185-configuration-file.md

Co-authored-by: Tamo <tamo@meilisearch.com>

* Update text/0185-configuration-file.md

Co-authored-by: Tamo <tamo@meilisearch.com>

* Update text/0185-configuration-file.md

Co-authored-by: Tamo <tamo@meilisearch.com>

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>

* Telemetry - Missing events and options (#186)

* wip spec changes

* update telemetry

* add missing property to the full properties table

* Update text/0034-telemetry-policies.md

Co-authored-by: Tamo <tamo@meilisearch.com>

* Update text/0034-telemetry-policies.md

Co-authored-by: Tamo <tamo@meilisearch.com>

* Add distrinct_attribute.set property, Task Seen becomes an aggregated event, add missing sub settings properties to Settings Updated event

* Add requests.total_received property to Tasks Seen event

* add missing ranking rules metrics

* Update text/0034-telemetry-policies.md

Co-authored-by: Tamo <tamo@meilisearch.com>

* Update text/0034-telemetry-policies.md

Co-authored-by: Tamo <tamo@meilisearch.com>

* Update text/0034-telemetry-policies.md

Co-authored-by: Tamo <tamo@meilisearch.com>

* Precise max-index-size option (#187)

* Tasks API - Rename `receivedDocumentsIds` by `providedIds` (#188)

* Rename receivedDocumentsIds by matchedDocuments

* Update open-api.yaml

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Update open-api.yaml

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Update text/0060-tasks-api.md

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Add missing env vars, brings consistency (#190)

* Swap Indexes API (#192)

* Add new spec file as draft file

* Add specification file

* update spec

* draft open-api.yml changes

* Add errors

* fix typos

* Add Indexes Swapped event

* duplicate_index_found is synchronous

* Apply suggestions from code review

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>

* Add review suggestions

* Precis mention for tasks indexUid target swap in the history

* Update open-api.yaml

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Update open-api.yaml

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Update open-api.yaml

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Update text/0061-error-format-and-definitions.md

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Update text/0191-swap-indexes-api.md

Co-authored-by: Clément Renault <renault.cle@gmail.com>

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Add snapshotCreation as a new task type to make it visible in the task history (#193)

* Tasks API - Add task priority order section (#194)

* Add 2.1 Task Priority section

* Precise other tasks priority ordering

* Search API - Page Selection Component (#196)

* Search API - numbered pagination

* Improve numbered pagination explaination

* Update text/0118-search-api.md

* Update text/0118-search-api.md

* Apply suggestions from code review

* Add links

* Update open api

* Add a dot at the end of the title

* Fix links

* Improve page and hitsperpage query params

* Refactor pagination guide

* Fix wording

* Add corner case

* Update open-api.yaml

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update open-api.yaml

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update open-api.yaml

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Apply wording changes from code review

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Apply suggestions from code review

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
Co-authored-by: Bruno Casali <brunoocasali@gmail.com>

* Fix example

* Apply suggestions from code review

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update open-api.yaml

* Update text/0118-search-api.md

* Update links

* Add link to maxtotalhits

* Update text/0118-search-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Improve message in page reference

* Update text/0118-search-api.md

Co-authored-by: Many the fish <legendre.maxime.isn@gmail.com>

* Add telemetry to know the most used search results navigation methods in a batch

* Update text/0118-search-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0118-search-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
Co-authored-by: Bruno Casali <brunoocasali@gmail.com>
Co-authored-by: Many the fish <legendre.maxime.isn@gmail.com>

* Tasks API - Add task cancelation capability (#195)

* Add cancel to task spec

* Remove missing error

* Fix typo on cancels

* Fix IndexUid always null

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Add canceledBy and missing_filters error

* Merge 11.1 filter iwth get and 11.2 filter with post

* Specified every date filter

* Change uid as integer and not as string

* Add auto-batching and error

* Add new route task/cancel api-key

* Add global task note

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0061-error-format-and-definitions.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Modification due to review

* Modification due to review

* Modification due to review

* Modification due to review

* Fix global task link

* Fix format link

* Add open-api modification

* Remove trailing space

* Add clarification

* Add errors definition

* Update open-api.yml

* fix task uid query parameter name

* Fix collission between url path and query parameters

* fmt open-api.yml

* Add telemetry details

* Add 202_taskCancelation example

* Update originalQuery to originalFilters and update missing_task_filter to missing_task_filters

* matchedTasks is computed when the task is enqueued

* Apply suggestions from code review

Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>

* Recall explanation for filter query parameter and mentions that limit and from are valid on

* Fix typo in property name

* Update query paramters name and related error codes when invalid

* Fix missing part of error msg

* Precise valid date formats

* Add supported format for date filters in open-api.yml

* Update open-api.yaml

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Update text/0060-tasks-api.md

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* fix typo

* Update open-api.yaml

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Update text/0060-tasks-api.md

* Update text/0060-tasks-api.md

* Tasks API - Add task deletion capability (#198)

* Describes task deletion feature additions

* Update open-api.yml

* Update telemetry-policies

* Update originalQuery to originalFilters

* Follow title structure

* Introduce review feedback

* Branch the right components

* Update text/0060-tasks-api.md

Co-authored-by: Clément Renault <renault.cle@gmail.com>

Co-authored-by: Clément Renault <renault.cle@gmail.com>

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>
Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Update the dump support table with the dumpv6 (#199)

* Settings API - Specify handling of null values when patching settings (#202)

* Specify behavior when patching a setting with partial values and null values.

- Partial value: the setting is unchanged.
- null value: the setting is reset to default.

* Settings: Update error messages to accept the null value

* Add missing task types

* Add canceledTasks details

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Clément Renault <renault.cle@gmail.com>
Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>
Co-authored-by: Bruno Casali <brunoocasali@gmail.com>
Co-authored-by: Many the fish <legendre.maxime.isn@gmail.com>
Co-authored-by: Amélie <alallema@users.noreply.github.com>
Co-authored-by: Louis Dureuil <louis.dureuil@gmail.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@irevoire irevoire irevoire requested changes

@curquiza curquiza curquiza approved these changes

Assignees
No one assigned
Labels
Implemented Feature specification has been implemented. Q4:2022 Telemetry Update the telemetry collect. v0.30
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@gmourier @irevoire @curquiza