2.10.20.1

📦 Features

• Ability to change sort direction per category #3259

🐛 Bugfixes

• Fix php7.4 compatibility for category query cache by @PierreGauthier in #3248
• [Optimizer] Prevent selecting a disabled category for previewing by @rbayet in #3261
• [Terms Merchandiser] Mview reindexing of pinned/blacklisted products by @rbayet in #3262
• [Core] Prevent fatal error when fuzziness disabled by @rbayet in #3263
• Could not load category condition in virtual category conditions: Wrong cast order by @thomas-kl1 in #3265
• [Category Query Cache] Prevent to cache null query by @PierreGauthier in #3266
• [Tracker] Add page type in add to cart event by @PierreGauthier in #3268

🔧 CI/CD

• [Tools] Reintegrate PHP 7.4 tests and bump PHP 8.1 versions by @rbayet in #3249
• [Tools] Reintegrate PHP 7.4 tests and bump PHP 8.1 versions by @rbayet in #3250

Full Changelog: 2.11.6...2.11.6.1

2.11.6

Main features

Magento 2.4.7 compatibility

Thanks to @Bashev for spotting the dependency issue on opensearch-project/opensearch-php.
If you migrated to Magento 2.4.7 and you were suddenly back in Elasticsuite 2.11.4.3, that's now fixed in this release !

Ability to ignore leading, trailing and consecutive zeroes in SKUs

New settings are available in a new section of the Store Configuration in Stores > Configuration > Elasticsuite > Analyzers Settings > Reference analyzer configuration

• Remove leading zeroes of numeric parts
• Remove trailing zeroes of numeric parts
• Reduce series of contiguous zeroes in numeric parts

Those new settings (disabled by default) apply to fields using the reference analyzer which is dedicated to searchable fields containing SKUs, UPCs or manufacturer part numbers.

Depending on your SKUs or part number schemes and the way your users search for them, it could help them find your products.
Please be aware that

• any change to those three new settings will require a full catalogsearch_fulltext reindex to take effect
• those settings will fully work only if you also enable the following "SKU search" dedicated experimental settings
  • Elasticsuite > Search Relevance > Spellchecking Configuration > Term Vectors Configuration > [Experimental] Use reference analyzer in term vectors
  • (possibly) Elasticsuite > Search Relevance > Spellchecking Configuration > Term Vectors Configuration > [Experimental] Use all tokens from term vectors
  • Elasticsuite > Search Relevance > Relevance Configuration > Exact Match Configuration > [Experimental] Use default analyzer in exact matching filter query

Admin notification for non-indexed tracker events

A new admin notification will pop up in the "Elasticsuite > Analytics > Search Usage" page when the tracker events storage table contains events created more than 6 hours ago, as a way to indicate to you a possible reason for behavioral data missing from the dashboard.

A new settings available under Stores > Configuration > Elasticsuite > Analytics > Pending events configuration > Hours before warning allows to control the timespan between a tracker event being recorded in the database and the admin notification complaining that it has not been indexed yet so could

• augment that timespan on a non-production environment where you almost never run the cronjobs
• reduce that timespan on a trafic heavy production environment where you do not want tracker events to accumulate for too long

📦 Features

• [Analysis] Allow ignoring leading, trailing and consecutive zeroes in… by @rbayet in #3243
• [Analytics] Notice/warning about old events left in the events queue … by @rbayet in #3226
• [CatalogRule] added missing form name in product conditions by @mvenghaus in #3167
• [Core] Server distr. and version in admin footer info by @rbayet in #3225
• [Core] Add Ingest Pipeline support. by @romainruaud in #3229
• [Core] Change opensearch deps to make it compatible with magento 2.4.7-beta3/2.4.7 by @Bashev in #3215
• [Core] Fix #3205, remove type parameter in order to compatibility with Elasticsearch 8 by @vahonc in #3233
• [Search] Use case insensitive query for span matching. by @romainruaud in #3240

🐞 Fixes

• [GraphQL] Fix #3231 allow fetching products with category_uid IN clauses. by @romainruaud in #3241
• [VirtualCategories] Fix #3234 Prevent to cache null query by @PierreGauthier in #3237

🧰 Quality

• [Tools] CI update to support of 2.4.7 and PHP 8.3 by @rbayet in #3244
• [Tools] Updating Quality workflow to node16 by @rbayet in #3227
• [Tools] Updating PHPStan/Integration workflows to node16 by @rbayet in #3228

Full Changelog: 2.11.5.3...2.11.6

2.10.20

Main features

Ability to ignore leading, trailing and consecutive zeroes in SKUs

New settings are available in a new section of the Store Configuration in Stores > Configuration > Elasticsuite > Analyzers Settings > Reference analyzer configuration

• Remove leading zeroes of numeric parts
• Remove trailing zeroes of numeric parts
• Reduce series of contiguous zeroes in numeric parts

Those new settings (disabled by default) apply to fields using the reference analyzer which is dedicated to searchable fields containing SKUs, UPCs or manufacturer part numbers.

Depending on your SKUs or part number schemes and the way your users search for them, it could help them find your products.
Please be aware that

• any change to those three new settings will require a full catalogsearch_fulltext reindex to take effect
• those settings will fully work only if you also enable the following "SKU search" dedicated experimental settings
  • Elasticsuite > Search Relevance > Spellchecking Configuration > Term Vectors Configuration > [Experimental] Use reference analyzer in term vectors
  • (possibly) Elasticsuite > Search Relevance > Spellchecking Configuration > Term Vectors Configuration > [Experimental] Use all tokens from term vectors
  • Elasticsuite > Search Relevance > Relevance Configuration > Exact Match Configuration > [Experimental] Use default analyzer in exact matching filter query

Admin notification for non-indexed tracker events

A new admin notification will pop up in the "Elasticsuite > Analytics > Search Usage" page when the tracker events storage table contains events created more than 6 hours ago, as a way to indicate to you a possible reason for behavioral data missing from the dashboard.

A new settings available under Stores > Configuration > Elasticsuite > Analytics > Pending events configuration > Hours before warning allows to control the timespan between a tracker event being recorded in the database and the admin notification complaining that it has not been indexed yet so could

• augment that timespan on a non-production environment where you almost never run the cronjobs
• reduce that timespan on a trafic heavy production environment where you do not want tracker events to accumulate for too long

📦 Features

• [Analysis] Allow ignoring leading, trailing and consecutive zeroes in… by @rbayet in #3243
• [Analytics] Notice/warning about old events left in the events queue … by @rbayet in #3226
• [Core] Server distr. and version in admin footer info by @rbayet in #3225
• [Search] Use case insensitive query for span matching. by @romainruaud in #3240

🐞 Fixes

• [GraphQL] Fix #3231 allow fetching products with category_uid IN clauses. by @romainruaud in #3241
• [VirtualCategories] Fix #3234 Prevent to cache null query by @PierreGauthier in #3237

🧰 Quality & Tools

• [Tools] Updating Quality workflow to node16 by @rbayet in #3227
• [Tools] Updating PHPStan/Integration workflows to node16 by @rbayet in #3228

Full Changelog: 2.10.19.3...2.10.20

2.11.5.3

Emergency maintenance release

Releases 2.10.19.2 and 2.11.5.2 published yesterday can cause an issue if you minify your JS sources, preventing JS scripts from working in Magento's admin area.
Thanks @thomas-kl1 for a quick reaction !

🐛 Fixes

• [Quality] Remove invalid characters in new JS files by @thomas-kl1 in #3222

Full Changelog: 2.11.5.2...2.11.5.3

2.10.19.3

Emergency maintenance release

Releases 2.10.19.2 and 2.11.5.2 published yesterday can cause an issue if you minify your JS sources, preventing JS scripts from working in Magento's admin area.
Thanks @thomas-kl1 for a quick reaction !

🐛 Fixes

• [Quality] Remove invalid characters in new JS files by @thomas-kl1 in #3222

Full Changelog: 2.10.19.2...2.10.19.3

2.11.5.2

Main features

Additional KPIs in Analytics > Search Usage

KPIs about category views, product views, products added to cart and sales event are now extracted from your behavioral data and visible in the "Analytics > Search Usage dashboard".

Removal of the hardcoded 'sku' field from exact queries filtering part

Historically, when filtering (not scoring !) results, exact match queries always targeted at least the search collector field (containing all searchable attributes content) and the sku field, whatever search weight was assigned to the sku field.
As

• on one hand, we have introduced back in releases 2.10.17 and 2.11.3 (experimental) settings to specifically target in that filtering query part all fields using the reference search analyzer (used by the sku attribute by default)
• on the other hand, you might be in a situation where you do not want at all the sku to be searchable

we have decided to remove that hardcoded sku in exact queries filtering part.

Long story short, if your users frequently perform searches by SKU and you're still using the reference analyzer for that attribute, we urge you to go enable the following Search Relevance settings listed below :

• Elasticsuite > Search Relevance > Spellchecking Configuration > Term Vectors Configuration > Use all tokens from term vectors
• Elasticsuite > Search Relevance > Spellchecking Configuration > Term Vectors Configuration > Use reference analyzer in term vectors
• Elasticsuite > Search Relevance > Relevance Configuration > Exact matching configuration > Use default analyzer in exact matching filter query

Ability to set custom number of primary shards and replicas by index type

If you're using a 3 nodes cluster in your production environment, you could have been tempted to set 3 primary shards and 2 replicas in the legacy Elasticsuite base settings.
While it is our opinion that having multiple primary shards is hardly necessary unless you really have a sizeable product index, it is now possible to go this route while keeping usually light indices (categories, thesaurus, and tracker indices on low traffic sites) with a single primary shard.
It is now possible to definer per index type the number of primary shards and replicas of indices.

This will allow you to minimize the memory footprint per node of opened shards/indices in your cluster (the rule of thumb being that 1 GB of Heap is required per node per 20 shards just for those indices being opened).

Admin notifications about invalid tracker events and automated removal

Releases 2.10.18.3 and 2.11.4.3 introduced some protections to avoid indexing invalid tracker events missing unique visitor or session ids into the behavioral data indices (as well as CLI tools to remove those already indexed).
Those invalid tracker events were left in the Elasticsuite tracker DB table with a flag is_invalid = 1.
This release introduces a cronjob task that will automatically and periodically remove them and a notification system to warn you about their existence (if you want to investigate the potential issue in your tracker tags).

You also have the ability to remove all of them at once.

📦 Features

• [Analytics] Additional product/category/sales KPIs by @rbayet in #3214
• [Config] Make getIndicesSettingsConfigParam public. by @rbayet in #3195
• [Core] Feature custom number of shards replicas by @romainruaud in #3185
• [Core] Append default CONFIG cache tag to ES configuration and also a generic ES cache tag by @romainruaud in #3191
• [Core] Enable Elasticsuite ES cache tag automatically. by @romainruaud in #3197
• [Cache] Add cache on search query building by @PierreGauthier in #3193 and #3213
• [GraphQL] Add suggestions in GraphQl results by @rbayet in #3053
• [LayeredNavigation] Optimize "see more" action on navigation filter by @thomas-kl1 in #3201
• [Search] Allow configurable msm for fuzzy search by @rbayet in #3196
• [Search] Remove hardcoded SKU from query builder since new options should allow to explicitely target it by @romainruaud in #3198
• [Search] Introducing knn query by @romainruaud in #3175
• [Search] Adding knn field config by @romainruaud in #3176
• [Tracker] Automated removal of queued invalid events by @rbayet in #3204

🐛 Fixes

• [Optimizer] Clean Applier Definition by @PierreGauthier in #3181
• [Spellchecker] Fix multi index by alias error by @PierreGauthier in #3199
• [Tracker] Ensuring tracker provided store_id is numeric. by @romainruaud in #3187
• [VirtualCategories] Add isActiveFilter to prevent disabled categories from being found by @wahidnory in #3159

🧰 Quality

• [Quality] Spellchecker unit tests by @rbayet in #3207
• [Quality] Allow own mapping impl. by using interface instead of mapping directly by @mvenghaus in #3170
• [Quality] M2 2.4.7 compatiblity: Adding missing parameter into constructor by @romainruaud in #3180

New Contributors

• @mvenghaus made their first contribution in #3170
• @wahidnory made their first contribution in #3159
• @thomas-kl1 made their first contribution in #3201

Full Changelog: 2.11.5.1...2.11.5.2

2.10.19.2

Main features

Additional KPIs in Analytics > Search Usage

KPIs about category views, product views, products added to cart and sales event are now extracted from your behavioral data and visible in the "Analytics > Search Usage dashboard".

Removal of the hardcoded 'sku' field from exact queries filtering part

Historically, when filtering (not scoring !) results, exact match queries always targeted at least the search collector field (containing all searchable attributes content) and the sku field, whatever search weight was assigned to the sku field.
As

• on one hand, we have introduced back in releases 2.10.17 and 2.11.3 (experimental) settings to specifically target in that filtering query part all fields using the reference search analyzer (used by the sku attribute by default)
• on the other hand, you might be in a situation where you do not want at all the sku to be searchable

we have decided to remove that hardcoded sku in exact queries filtering part.

Long story short, if your users frequently perform searches by SKU and you're still using the reference analyzer for that attribute, we urge you to go enable the following Search Relevance settings listed below :

• Elasticsuite > Search Relevance > Spellchecking Configuration > Term Vectors Configuration > Use all tokens from term vectors
• Elasticsuite > Search Relevance > Spellchecking Configuration > Term Vectors Configuration > Use reference analyzer in term vectors
• Elasticsuite > Search Relevance > Relevance Configuration > Exact matching configuration > Use default analyzer in exact matching filter query

Ability to set custom number of primary shards and replicas by index type

If you're using a 3 nodes cluster in your production environment, you could have been tempted to set 3 primary shards and 2 replicas in the legacy Elasticsuite base settings.
While it is our opinion that having multiple primary shards is hardly necessary unless you really have a sizeable product index, it is now possible to go this route while keeping usually light indices (categories, thesaurus, and tracker indices on low traffic sites) with a single primary shard.
It is now possible to definer per index type the number of primary shards and replicas of indices.

This will allow you to minimize the memory footprint per node of opened shards/indices in your cluster (the rule of thumb being that 1 GB of Heap is required per node per 20 shards just for those indices being opened).

Admin notifications about invalid tracker events and automated removal

Releases 2.10.18.3 and 2.11.4.3 introduced some protections to avoid indexing invalid tracker events missing unique visitor or session ids into the behavioral data indices (as well as CLI tools to remove those already indexed).
Those invalid tracker events were left in the Elasticsuite tracker DB table with a flag is_invalid = 1.
This release introduces a cronjob task that will automatically and periodically remove them and a notification system to warn you about their existence (if you want to investigate the potential issue in your tracker tags).

You also have the ability to remove all of them at once.

📦 Features

• [Analytics] Additional product/category/sales KPIs by @rbayet in #3214
• [Config] Make getIndicesSettingsConfigParam public. by @rbayet in #3195
• [Core] Feature custom number of shards replicas by @romainruaud in #3185
• [Core] Append default CONFIG cache tag to ES configuration and also a generic ES cache tag by @romainruaud in #3191
• [Core] Enable Elasticsuite ES cache tag automatically. by @romainruaud in #3197
• [Cache] Add cache on search query building by @PierreGauthier in #3193 and #3213
• [Search] Allow configurable msm for fuzzy search by @rbayet in #3196
• [Search] Remove hardcoded SKU from query builder since new options should allow to explicitely target it by @romainruaud in #3198
• [Tracker] Automated removal of queued invalid events by @rbayet in #3204

🐛 Fixes

• [Optimizer] Clean Applier Definition by @PierreGauthier in #3181
• [Spellchecker] Fix multi index by alias error by @PierreGauthier in #3199
• [Tracker] Ensuring tracker provided store_id is numeric. by @romainruaud in #3187

🧰 Quality

• [Quality] Spellchecker unit tests by @rbayet in #3207

Full Changelog: 2.10.19.1...2.10.19.2

2.11.5.1

Main changes

We fixed the elision/contraction management for French, Italian and Catalan when indexing/searching content.

Historically the text analyzers for those languages were supposed to ignore the elision/contraction of some articles or pronouns but didn't due to an improper order of text transformation components.
This could be problematic if you had a Minimum Should Match lower than 100%

For instance :

• in French: l'avion (mandatory form of "le avion" - "the plane") was supposed to be indexed as avion (plane) but was still indexed as l avion, so searching for l'avion bleu vole could match l'écureuil bleu vole with a 75% minimum should match (matches on l, bleu and vole)
• in Italian: comedia dell'arte was supposed to be indexed as comedia arte but was indexed as comedia dell arte
• in Catalan: l'avi (the grandfather) was supposed to be indexed as avi (grandfather) but was indexed as l avi

This release fixes that issues and now the following elisions/contractions will be removed as intended :

• for French: l, m, t, qu, n, s, j, d, c as defined in that language specific elision filter configuration
• for Italian: c, l, all, dall, dell, nell, sull, coll, pell, gl, agl, dagl, degl, negl, sugl, un, m, t, s, v, d as defined in that language specific elision filter configuration
• for Catalan: d, l, m, n, s, t as defined in that language specific elision filter configuration

Should this changes affect you negatively, please be aware that you can override those settings (as all of those in the elasticsuite_analysis.xml configuration file) in a custom module.
Feel also free to let us know, especially about the configuration Italian and Catalan since we are not native speakers and we are just following official Elasticsearch recommendations.

📦 Features

• [Community] Integrating HS form in dashboard. by @romainruaud in #3155
• [Thesaurus] Feature #3115, add a warning message for existing terms in other thesaurus by @vahonc in #3128

🐛 Fixes

• [Analytics] Exclude search usage chart type from i18n by @rbayet in #3161
• [Search] Fix french stemmer on "clef/clefs" and better elision management by @PierreGauthier in #3156

Full Changelog: 2.11.5...2.11.5.1

2.10.19.1

Main changes

We fixed the elision/contraction management for French, Italian and Catalan when indexing/searching content.

Historically the text analyzers for those languages were supposed to ignore the elision/contraction of some articles or pronouns but didn't due to an improper order of text transformation components.
This could be problematic if you had a Minimum Should Match lower than 100%

For instance :

• in French: l'avion (mandatory form of "le avion" - "the plane") was supposed to be indexed as avion (plane) but was still indexed as l avion, so searching for l'avion bleu vole could match l'écureuil bleu vole with a 75% minimum should match (matches on l, bleu and vole)
• in Italian: comedia dell'arte was supposed to be indexed as comedia arte but was indexed as comedia dell arte
• in Catalan: l'avi (the grandfather) was supposed to be indexed as avi (grandfather) but was indexed as l avi

This release fixes that issues and now the following elisions/contractions will be removed as intended :

• for French: l, m, t, qu, n, s, j, d, c as defined in that language specific elision filter configuration
• for Italian: c, l, all, dall, dell, nell, sull, coll, pell, gl, agl, dagl, degl, negl, sugl, un, m, t, s, v, d as defined in that language specific elision filter configuration
• for Catalan: d, l, m, n, s, t as defined in that language specific elision filter configuration

Should this changes affect you negatively, please be aware that you can override those settings (as all of those in the elasticsuite_analysis.xml configuration file) in a custom module.
Feel also free to let us know, especially about the configuration Italian and Catalan since we are not native speakers and we are just following official Elasticsearch recommendations.

📦 Features

• [Community] Integrating HS form in dashboard. by @romainruaud in #3155
• [Thesaurus] Feature #3115, add a warning message for existing terms in other thesaurus by @vahonc in #3128

🐛 Fixes

• [Analytics] Exclude search usage chart type from i18n by @rbayet in #3161
• [Search] Fix french stemmer on "clef/clefs" and better elision management by @PierreGauthier in #3156

Full Changelog: 2.10.19...2.10.19.1

2.11.5

Main feature

Due to breaking changes in the Elasticsearch PHP client on one side and our comitment for the 2.11.x releases to be still compatible with Elasticsearch 7 and Elasticsearch 8 as well as to ensure our compatibility with a growing installation base of Magentos using OpenSearch on the other side, we've decided to switch to the OpenSearch PHP client which addresses all flavours (ES 7, ES 8, OS 1, OS 2) seamlessly.

📦 Features

• [Core] Replace the Elasticsearch client by the Opensearch client. by @romainruaud in #3131
• [Configuration] update of BO comment LU-140 related to SSL certificate validation and OpenSearch by @gabrielLumao in #3152

🐛 Fixes

• [Analytics] Make the dashboard display handle long search terms by @rbayet in #3147
• [Layered Navigation] Fixes #3148 Replacing _term agg sort order by _key by @rbayet in #3150

Full Changelog: 2.11.4.3...2.11.5