Skip to content
This repository has been archived by the owner on Mar 21, 2024. It is now read-only.

Define fields to search on at runtime #251

Merged
merged 9 commits into from
Jul 31, 2023
Merged

Define fields to search on at runtime #251

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
f5db7b6
Update 0118-search-api.md
ManyTheFish Jul 3, 2023
705121c
Update 0061-error-format-and-definitions.md
ManyTheFish Jul 3, 2023
6d22d58
Update 0118-search-api.md
ManyTheFish Jul 3, 2023
97cea19
Update open-api.yaml
ManyTheFish Jul 3, 2023
8bf29c6
Update 0034-telemetry-policies.md
ManyTheFish Jul 13, 2023
eba4c93
Update open-api.yaml
ManyTheFish Jul 13, 2023
12211c7
Update text/0118-search-api.md
ManyTheFish Jul 13, 2023
0f97847
Update text/0118-search-api.md
ManyTheFish Jul 13, 2023
015cfd1
Merge branch 'release-v1.3.0' into Define-fields-to-search-on-at-sear…
macraig Jul 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions open-api.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -716,6 +716,10 @@ components:
type: string
description: Defines which strategy to use to match the query terms within the documents as search results. Two different strategies are available, `last` and `all`. By default, the `last` strategy is chosen.
default: 'last'
attributesToSearchOn:
type: array
description: Defines which `searchableAttributes` the query will search on.
default: '["*"]'
filter:
$ref: '#/components/schemas/filter'
facets:
Expand Down
3 changes: 3 additions & 0 deletions text/0034-telemetry-policies.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@ The collected data is sent to [Segment](https://segment.com/). Segment is a plat
| `filter.with_geoRadius` | `true` if the filter rule `_geoRadius` was used in this batch, otherwise `false` | false | `Documents Searched POST`, `Documents Searched GET` |
| `filter.with_geoBoundingBox` | `true` if the filter rule `_geoBoundingBox` was used in this batch, otherwise `false`| false | `Documents Searched POST`, `Documents Searched GET` |
| `filter.most_used_syntax` | Most used filter syntax among all requests containing the `filter` parameter in this batch | string | `Documents Searched POST`, `Documents Searched GET` |
| `attributes_to_search_on.total_number_of_uses`| Total number of queries where `attributesToSearchOn` is set | 5 | `Documents Searched POST`, `Documents Searched GET` |
macraig marked this conversation as resolved.
Show resolved Hide resolved
| `q.max_terms_number` | Highest number of terms given for the `q` parameter in this batch | 5 | `Documents Searched POST`, `Documents Searched GET` |
| `pagination.max_limit` | Highest value given for the `limit` parameter in this batch | 60 | `Documents Searched POST`, `Documents Searched GET`, `Documents Fetched GET`, `Documents Fetched POST` |
| `pagination.max_offset` | Highest value given for the `offset` parameter in this batch | 1000 | `Documents Searched POST`, `Documents Searched GET`, `Documents Fetched GET`, `Documents Fetched POST` |
Expand Down Expand Up @@ -267,6 +268,7 @@ This property allows us to gather essential information to better understand on
| filter.with_geoBoundingBox | Does the built-in filter rule _geoBoundingBox has been used in the aggregated event?| `false` |
| filter.avg_criteria_number | The average number of filter criteria among all the requests containing the `filter` parameter in the aggregated event. `"filter": []` equals to `0` while not sending `filter` does not influence the average in the aggregated event. | `4` |
| filter.most_used_syntax | The most used filter syntax among all the requests containing the requests containing the `filter` parameter in the aggregated event. `string` / `array` / `mixed` | `mixed` |
| attributes_to_search_on.total_number_of_uses| Total number of queries where `attributesToSearchOn` is set | `5` |
| q.max_terms_number | The maximum number of terms for the `q` parameter among all requests in the aggregated event. | `5` |
| pagination.max_limit | The maximum limit encountered among all requests in the aggregated event. | `20` |
| pagination.max_offset | The maximum offset encountered among all requests in the aggregated event. | `1000` |
Expand Down Expand Up @@ -301,6 +303,7 @@ This property allows us to gather essential information to better understand on
| filter.with_geoBoundingBox | Does the built-in filter rule _geoBoundingBox has been used in the aggregated event?| `false` |
| filter.avg_criteria_number | The average number of filter criteria among all the requests containing the `filter` parameter in the aggregated event. `"filter": []` equals to `0` while not sending `filter` does not influence the average in the aggregated event. | `4` |
| filter.most_used_syntax | The most used filter syntax among all the requests containing the requests containing the `filter` parameter in the aggregated event. `string` / `array` / `mixed` | `mixed` |
| attributes_to_search_on.total_number_of_uses | Total number of queries where `attributesToSearchOn` is set | 5 |
| q.max_terms_number | The maximum number of terms for the `q` parameter among all requests in the aggregated event. | `5` |
| pagination.max_limit | The maximum limit encountered among all requests in the aggregated event. | `20` |
| pagination.max_offset | The maximum offset encountered among all requests in the aggregated event. | `1000` |
Expand Down
43 changes: 43 additions & 0 deletions text/0061-error-format-and-definitions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1808,6 +1808,49 @@ HTTP Code: `400 Bad Request`
}
```

## invalid_search_attributes_to_search_on

`Synchronous`

### Context

This error occurs if a value with a different type than `Array of String`(POST), `String`(GET) or `null` and other than attributes names contained in the settings `searchableAttributes` as a value for `attributesToSearchOn` is specified.

### Error Definition

HTTP Code: `400 Bad Request`

```json
{
"message": "`:deserr_helper`",
"code": "invalid_search_attributes_to_search_on",
"type": "invalid_request",
"link": "https://docs.meilisearch.com/errors#invalid_search_attributes_to_search_on"
}
```

#### Variant: one of the values is not part of the settings `searchableAttributes` list

```json
{
"message": "Attribute `:value` is not searchable. Available searchable attributes are: `:searchableAttributes`.",
"code": "invalid_search_attributes_to_search_on",
"type": "invalid_request",
"link": "https://docs.meilisearch.com/errors#invalid_search_attributes_to_search_on"
}
```

#### Variant: one of the values is not part of the settings `searchableAttributes` list and not all of the `searchableAttributes` are displayable
gmourier marked this conversation as resolved.
Show resolved Hide resolved

```json
{
"message": "Attribute `:value` is not searchable. Available searchable attributes are: `:DisplayableSearchableAttributes, <..hidden-attributes>`.",
"code": "invalid_search_attributes_to_search_on",
"type": "invalid_request",
"link": "https://docs.meilisearch.com/errors#invalid_search_attributes_to_search_on"
}
```

---

## invalid_document_geo_field
Expand Down
15 changes: 15 additions & 0 deletions text/0118-search-api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ If a master key is used to secure a Meilisearch instance, the auth layer returns
| [`cropMarker`](#3115-cropmarker) | String | False |
| [`showMatchesPosition`](#3116-showmatchesposition) | Boolean | False |
| [`matchingStrategy`](#3117-matchingStrategy) | String | False |
| [`attributesToSearchOn`](#3118-attributesToSearchOn) | Array of String - String | False |


#### 3.1.1. `q`
Expand Down Expand Up @@ -912,6 +913,20 @@ The documents containing ALL the query words (i.e. in the `q` parameter) are ret

Only the documents containing ALL the query words (i.e. in the `q` parameter) are returned by Meilisearch. If Meilisearch doesn't have enough documents to fit the requested `limit`, it returns the documents found without trying to match more documents.

#### 3.1.17. `attributesToSearchOn`

- Type: Array of String (POST) | String (GET)
- Required: False
- Default: `["*"]`

Defines which `searchableAttributes` the query will search on.

- If `attributesToSearchOn` is not set, set to `["*"]` or set to `null`, then the query will search on all `searchableAttributes`.
- Sending the attributes in a different order than the order set in the settings `searchableAttributes` doesn't reorder the fields' rank for the `Attributes` ranking rule
- 🔴 Sending a value with a different type than `Array of String`(POST), `String`(GET) or `null` for `attributesToSearchOn` returns an [invalid_attributes_to_search_on](0061-error-format-and-definitions.md#invalid_search_attributes_to_search_on) error.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if you send an empty array?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty result

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should that be specified or is it common practice in the API?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that defining an empty attributesToSearchOn list means that you search in no attributes, which implies no results. Do you see any other behavior?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess not, I was just wondering if it might be surprising for a user to leave it empty and then be confused about getting no results

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm more on the side of the error if the array is sent with an empty form, so the developer can't break the search and is aware of it; By doing this, we can catch it before they have to understand what's wrong and fix it; I see no point in returning 0 results, am I wrong?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could return the hits as if the query was empty?

Copy link
Member Author

@ManyTheFish ManyTheFish Jul 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm more on the side of the error if the array is sent with an empty form, so the developer can't break the search and is aware of it; By doing this, we can catch it before they have to understand what's wrong and fix it; I see no point in returning 0 results, am I wrong?

I agree about the fact that would be useless to set [], however, it's a logical behavior from Meilisearch in my sense. Moreover, it's consistent with the searchableAttributes that accepts an empty array

Maybe we could return the hits as if the query was empty?

It doesn't seems logical to me to replace the search with a placeholder one when attributeToSearchOn is empty. 🤔

Copy link
Member

@gmourier gmourier Jul 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right @ManyTheFish; accepting an empty array is an aspect of "consistency" that we could revisit with a v2 eventually because it doesn't seem great for DX (letting the developer put himself in a situation where he/she gets nothing out of the product)

I see it as an edge case personally, so the current implementation works for me, and I'm not against it

cc @macraig (if it helps!)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 @gmourier , let's be consistent and revisit this when we're ready for a v2

- 🔴 Sending an attribute that is not part of the settings `searchableAttributes` list returns an [invalid_attributes_to_search_on](0061-error-format-and-definitions.md#invalid_search_attributes_to_search_on) error.


### 3.2. Search Response Properties

| Field | Type | Required |
Expand Down