Search API - Formatting Search Results (#120)

* Draft a revamp of the formatting search results spec

* Removes formatting-search-results specification

* wip

* Fix links

* Add precision

* Update OpenAPI

* Fix sentences

* Add consistency

* Add details and fix sentences

* Fix default value in OpenAPI spec

* Add Future Possibilities for _matchesInfo

* Remove bullet point title

* Add wip examples

* Add _formatted behavior regarding attributesToRetrieve, attributesToCrop and attributesToHighlight

* Apply suggestions from code review

Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>

* Fix typo

* Mention no effect case of  cropMarker / cropLength / highlightPostTag / highlightPreTag  on attributesToCrop  & attributesToHighlight

* Re-explain _formatted in details

* Mention Synonyms

* Add precision after team feedback

* Apply suggestions from code review

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

* Add precisions

* Add boolean analytics for fields dedicated to customize formatting of search results behaviors

* Precise behavior for _formatted

* Add clearer explanations for the cropping algorithm and the fact that it keep the phrase context when extending around

* Apply suggestions from code review

Co-authored-by: Many <legendre.maxime.isn@gmail.com>
Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>
Co-authored-by: Tommy <68053732+dichotommy@users.noreply.github.com>

* Precise highlighting around every matched query term

* Apply suggestions from code review

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>
Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>

* Precise cropLength behavior when 0 is specified

* Remove filter-and-facet-behavior spec to merge it into search-api, and correct some types description

* Improve formatted spec (#146)

* Improve formatted spec

* Update text/0118-search-api.md

Co-authored-by: Tamo <irevoire@protonmail.ch>

* Update text/0118-search-api.md

Co-authored-by: Tamo <irevoire@protonmail.ch>

* Update text/0118-search-api.md

Co-authored-by: Tamo <irevoire@protonmail.ch>

* Update text/0118-search-api.md

Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>

* Update text/0118-search-api.md

Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>

* Improve according to reviews

Co-authored-by: Tamo <irevoire@protonmail.ch>
Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>

* Add missing precision for highlightPostTag

* fix broken links

Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>
Co-authored-by: gui machiavelli <hey@guimachiavelli.com>
Co-authored-by: Many <legendre.maxime.isn@gmail.com>
Co-authored-by: Tommy <68053732+dichotommy@users.noreply.github.com>
Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
Co-authored-by: Tamo <irevoire@protonmail.ch>

open-api.yaml

@@ -720,6 +720,22 @@ components:
           default:
             - ''
         example: 'title,description'
+    highlightPreTag:
+      name: highlightPreTag
+      in: query
+      required: false
+      description: Specify the tag to put before the highlighted query terms.
+      schema:
+        type: string
+        default: '<em>'
+    highlightPostTag:
+      name: highlightPostTag
+      in: query
+      required: false
+      description: Specify the tag to put after the highlighted query terms.
+      schema:
+        type: string
+        default: '</em>'
     attributesToCrop:
       name: attributesToCrop
       in: query
@@ -728,15 +744,23 @@ components:
         type: string
         example: 'overview:10'
       description: Comma-separated list of attributes whose values have to be cropped. Cropped attributes are returned in `_formatted` response object.
+    cropMarker:
+      name: cropMarker
+      in: query
+      description: Sets the crop marker to apply before and/or after cropped part selected within an attribute defined in `attributesToCrop` parameter.
+      required: false
+      schema:
+        type: string
+        default: '…'
     cropLength:
       name: cropLength
       in: query
       required: false
       schema:
         type: integer
         example: 5
-        default: 200
-      description: Length used to crop field values.
+        default: 10
+      description: Sets the total number of words to keep around the matched part of an attribute specified in the `attributesToCrop` parameter.
     facetsDistribution:
       name: facetsDistribution
       in: query
@@ -1431,7 +1455,10 @@ paths:
         - $ref: '#/components/parameters/q'
         - $ref: '#/components/parameters/attributesToRetrieve'
         - $ref: '#/components/parameters/attributesToHighlight'
+        - $ref: '#/components/parameters/highlightPreTag'
+        - $ref: '#/components/parameters/highlightPostTag'
         - $ref: '#/components/parameters/attributesToCrop'
+        - $ref: '#/components/parameters/cropMarker'
         - $ref: '#/components/parameters/cropLength'
         - $ref: '#/components/parameters/facetsDistribution'
         - $ref: '#/components/parameters/filter'
@@ -1515,17 +1542,31 @@ paths:
                     type: string
                     example: '["title", "overview"]'
                     default: '[]'
+                highlightPreTag:
+                  type: string
+                  description: Specify the tag to put before the highlighted query terms.
+                  example: '<mark>'
+                  default: '<em>'
+                highlightPostTag:
+                  type: string
+                  description: Specify the tag to put after the highlighted query terms.
+                  example: '</mark>'
+                  default: '</em>'
                 attributesToCrop:
                   type: array
                   description: Array of attributes whose values have to be cropped. Cropped attributes are returned in `_formatted` response object.
                   items:
                     type: string
                     example: '["overview", "author"]'
                     default: '[]'
+                cropMarker:
+                  type: string
+                  description: Sets the crop marker to apply before and/or after cropped part selected within an attribute defined in `attributesToCrop` parameter.
+                  default: '…'
                 cropLength:
                   type: number
-                  description: Length used to crop field values.
-                  default: 200
+                  description: Sets the total number of **words** to keep for the cropped part of an attribute specified in the `attributesToCrop` parameter.
+                  default: 10
                 matches:
                   type: boolean
                   description: Defines whether an `_matchesInfo` object that contains information about the matches should be returned or not.

text/0034-telemetry-policies.md

@@ -109,6 +109,11 @@ The collected data is sent to [Segment](https://segment.com/). Segment is a plat
 | `q.max_terms_number`                    | Highest number of terms given for the `q` parameter in this batch | 5 | `Documents Searched POST`, `Documents Searched GET` |
 | `pagination.max_limit`                  | Highest value given for the `limit` parameter in this batch | 60 | `Documents Searched POST`, `Documents Searched GET` |
 | `pagination.max_offset`                 | Highest value given for the `offset` parameter in this batch | 1000 | `Documents Searched POST`, `Documents Searched GET` |
+| `formatting.highlight_pre_tag`       | `true` if `highlightPreTag` was used in this batch, otherwise `false` | false | `Documents Searched POST`, `Documents Searched GET` |
+| `formatting.highlight_post_tag`       | `true` if `highlightPostTag` was used in this batch, otherwise `false` | false | `Documents Searched POST`, `Documents Searched GET` |
+| `formatting.crop_length`                | `true` if `cropLength` was used in this batch, otherwise `false` | false | `Documents Searched POST`, `Documents Searched GET` |
+| `formatting.crop_marker`                | `true` if `cropMarker` was used in this batch, otherwise `false` | false | `Documents Searched POST`, `Documents Searched GET` |
+| `formatting.matches`                    | `true` if `matches` was used in this batch, otherwise `false` | false | `Documents Searched POST`, `Documents Searched GET` |
 | `primary_key`                           | Value given for the `primaryKey` parameter if used, otherwise `null` | id | `Index Created`, `Index Updated`, `Documents Added`, `Documents Updated`|
 | `payload_type`                          | All `payload_type` encountered in this batch | ["application/json", "text/plain", "application/x-ndjson"] | `Documents Added`, `Documents Updated` |
 | `index_creation`                        | `true` if a document addition or update request triggered index creation in this batch, otherwise `false` | true | `Documents Added`, `Documents Updated` |
@@ -206,6 +211,11 @@ This property allows us to gather essential information to better understand on
 | q.max_terms_number | The maximum number of terms for the `q` parameter among all requests in the aggregated event. | `5` |
 | pagination.max_limit | The maximum limit encountered among all requests in the aggregated event. | `20` |
 | pagination.max_offset | The maxium offset encountered among all requests in the aggregated event. | `1000` |
+| formatting.highlight_pre_tag | Does `highlightPreTag` has been used in the aggregated event? If yes, `true` otherwise `false` | `false` |
+| formatting.highlight_post_tag | Does `highlightPostTag` has been used in the aggregated event? If yes, `true` otherwise `false` | `false` |
+| formatting.crop_length | Does `cropLength` has been used in the aggregated event? If yes, `true` otherwise `false` | `false` |
+| formatting.crop_marker | Does `cropMarker` has been used in the aggregated event? If yes, `true` otherwise `false` | `false` |
+| formatting.matches | Does `matches` has been used in the aggregated event? If yes, `true` otherwise `false` | `false` |
 
 ---
 
@@ -228,6 +238,11 @@ This property allows us to gather essential information to better understand on
 | q.max_terms_number | The maximum number of terms for the `q` parameter among all requests in the aggregated event. | `5` |
 | pagination.max_limit | The maximum limit encountered among all requests in the aggregated event. | `20` |
 | pagination.max_offset | The maxium offset encountered among all requests in the aggregated event. | `1000` |
+| formatting.highlight_pre_tag | Does `highlightPreTag` has been used in the aggregated event? If yes, `true` otherwise `false` | `false` |
+| formatting.highlight_post_tag | Does `highlightPostTag` has been used in the aggregated event? If yes, `true` otherwise `false` | `false` |
+| formatting.crop_length | Does `cropLength` has been used in the aggregated event? If yes, `true` otherwise `false` | `false` |
+| formatting.crop_marker | Does `cropMarker` has been used in the aggregated event? If yes, `true` otherwise `false` | `false` |
+| formatting.matches | Does `matches` has been used in the aggregated event? If yes, `true` otherwise `false` | `false` |
 
 ---