Skip to content
This repository has been archived by the owner on Mar 21, 2024. It is now read-only.

Documents API — Support a csv delimiter customization #221

Merged
merged 4 commits into from
Apr 3, 2023

Conversation

irevoire
Copy link
Member

@irevoire irevoire commented Feb 20, 2023

Summary

Specify the usage of the new query parameter csvDelimiter that lets you customize the csv delimiter you wants to use in your document addition.


Attention To Reviewers

Added the primaryKey in the OpenAPI schema as a catch up


Misc

  • Update OpenAPI specification file (if needed; Apply the OpenApi label)
  • [] Update telemetry datapoints (if needed; Apply the Telemetry label)
  • Implemented in Csv delimiter meilisearch#3505

@irevoire irevoire added Implemented Feature specification has been implemented. v1.1.0 labels Feb 20, 2023
@irevoire irevoire requested a review from gmourier February 20, 2023 12:27
@gmourier gmourier added the Ready For Review Feature specification must be reviewed. label Feb 20, 2023
@gmourier gmourier mentioned this pull request Feb 20, 2023
1 task
@gmourier gmourier changed the title specify the support of the csv-delimiter Documents API — Support a csv delimiter customization Feb 20, 2023
bors bot added a commit to meilisearch/meilisearch that referenced this pull request Feb 20, 2023
3505: Csv delimiter r=irevoire a=irevoire

Fixes #3442
Closes #2803
Specified in meilisearch/specifications#221

This PR is a reimplementation of #2803, on the new engine. Thanks for your idea and initial PR `@MixusMinimax;` sorry I couldn’t update/merge your PR. Way too many changes happened on the engine in the meantime.

**Attention to reviewer**; I had to update deserr to implement the support of deserializing `char`s

-------

It introduces four new error messages;
- Invalid value in parameter csvDelimiter: expected a string of one character, but found an empty string
- Invalid value in parameter csvDelimiter: expected a string of one character, but found the following string of 5 characters: doggo
- csv delimiter must be an ascii character. Found: 🍰 
- The Content-Type application/json does not support the use of a csv delimiter. The csv delimiter can only be used with the Content-Type text/csv.

And one error code;
- `invalid_index_csv_delimiter`

The `invalid_content_type` error code is now also used when we encounter the `csvDelimiter` query parameter with a non-csv content type.

Co-authored-by: Tamo <tamo@meilisearch.com>
Copy link
Member

@gmourier gmourier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WDYT @irevoire?

text/0028-indexing-csv.md Outdated Show resolved Hide resolved
text/0061-error-format-and-definitions.md Outdated Show resolved Hide resolved
text/0061-error-format-and-definitions.md Outdated Show resolved Hide resolved
text/0061-error-format-and-definitions.md Outdated Show resolved Hide resolved
@gmourier gmourier requested a review from curquiza February 22, 2023 09:40
Copy link
Member

@curquiza curquiza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok with this with the change of the error code to invalid_document_csv_delimiter

@gmourier
Copy link
Member

📡 @irevoire I've updated the spec to replace invalid_index_csv_delimiter with invalid_document_csv_delimiter

Copy link
Member

@gmourier gmourier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM Thanks @irevoire 🎉 :goberserk:

@gmourier gmourier added the OpenAPI Update OpenAPI specification. label Mar 13, 2023
@github-actions
Copy link

🤖 API change detected:

Modified (2)

  • POST /indexes/{indexUid}/documents
    • Query parameters added: primaryKey, csvDelimiter
  • PUT /indexes/{indexUid}/documents
    • Query parameters added: primaryKey, csvDelimiter

View documentation diff

Powered by Bump

@gmourier gmourier removed their assignment Mar 14, 2023
@ahmednfwela
Copy link

will this also be able to customize new line delimiters (instead of \n)?

@gmourier gmourier removed the Ready For Review Feature specification must be reviewed. label Mar 15, 2023
@gmourier
Copy link
Member

gmourier commented Mar 15, 2023

Hello @ahmednfwela 👋

No, it's out-of-scope for this change!

If you need this customization, I invite you to share your need and use case here by creating a new discussion 🙇‍♂️

Thanks!

@gmourier gmourier merged commit 88a5fa1 into release-v1.1.0 Apr 3, 2023
@gmourier gmourier deleted the csv-delimiter branch April 3, 2023 14:16
gmourier added a commit that referenced this pull request Apr 3, 2023
* Bump API version

* Metrics API — Introduce a `/metrics` route for observability / monitoring purposes (#174)

* init spec for /metrics

* Rename spec filename and add description for metrics

* update metrics name and type col in the metrics table; add auth errors

* Add --enable-metrics-route to instance-options.md specification

* Precise metrics

* update open-api.yml

* Add telemetry instance property

* Mention metrics route on stats.get action

* Add  action

* Precise  action description

* Rephrase future possibility

* Add MEILI_ENABLE_METRICS_ROUTE

* rename all the cli parameter to the experimental variant

* Update text/0174-metrics-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update open-api.yaml

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0085-api-keys.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Apply suggestions from code review

Co-authored-by: Tamo <tamo@meilisearch.com>

---------

Co-authored-by: Tamo <tamo@meilisearch.com>

* API Keys - Introduce `*` wildcard char at the last position to express access to multiple indexes starting with the same string (#189)

* Bump open-api spec

* Precise that the * wildcard char can be used at the last position to match multiple indexes starting with the same string

* Update future possibilities

* Update the index wildcard pattern error message

---------

Co-authored-by: Clément Renault <clement@meilisearch.com>

* Documents API — Support a csv delimiter customization (#221)

* specify the support of the csv-delimiter

* Apply suggestions from code review

* get rid of the last 'future possibility' that is being described in this PR

* Update openAPI specification

---------

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update the geo-search spec to mention the fact that null is a valid geo field (#222)

* GeoSearch — Add the `_geoBoundingBox` built-in filter (#223)

* Add geoBoundingBox built-in filter description

* Add missing parts

* Update text/0059-geo-search.md

Co-authored-by: Tamo <tamo@meilisearch.com>

* Replace the top left / bottom right signature to a top right / bottom left signature

---------

Co-authored-by: Tamo <tamo@meilisearch.com>

* Search API — Add `facetStats` description (#224)

* update search-api.md

* rephrase

* update open-api.yml

* rephrase, add example on open-api.yml

* Add examples

* fix

* Update text/0118-search-api.md

Co-authored-by: Louis Dureuil <louis.dureuil@gmail.com>

* rephrase

* Clarify that facet stats are for all search results

---------

Co-authored-by: Louis Dureuil <louis.dureuil@gmail.com>

* Multi Search API (#225)

* Add multi-search spec

* Change possible HTTP return codes for index_not_found

* Add multi-search to openAPI

* Update telemetry

* openAPI: shorter summary for multi-search

* openAPI: attempt to remove superfluous "values" field in multisearch request/response

* Apply review suggestions for openAPI

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Apply review suggestions for multi search API

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Add queries/results keys to openAPI examples

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* fix indent

* Apply review suggestions for multi search API

Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>

* OpenAPI: Change summary to `Multi Search`

* use backticks instead of ticks in multi search API example

* Apply review suggestions for multi search API

Co-authored-by: Maryam <90181761+maryamsulemani97@users.noreply.github.com>

* remove superfluous newline from telemetry file

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

---------

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>
Co-authored-by: Maryam <90181761+maryamsulemani97@users.noreply.github.com>

* Add small blurb on index count / max size (#226)

---------

Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: Louis Dureuil <louis.dureuil@gmail.com>
Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>
Co-authored-by: Maryam <90181761+maryamsulemani97@users.noreply.github.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Implemented Feature specification has been implemented. OpenAPI Update OpenAPI specification. Q1:2023 v1.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants