Update Japanese and Korean language analyzers

[macrakis]

Is your feature request related to a problem? Please describe.
The current Japanese (kuromoji) and Korean (seunjeon) language analyzers / tokenizers are out of date.

Describe the solution you'd like
Update to the latest versions.

[minalsha]

@macrakis what exactly is out of date and could you please share some additional details to help us investigate?

[dbwiddis]

Looks like the versions are tied to the Lucene dependency version:

OpenSearch/plugins/analysis-kuromoji/build.gradle

Line 38 in 7694a27

api "org.apache.lucene:lucene-analysis-kuromoji:${versions.lucene}"

OpenSearch/plugins/analysis-nori/build.gradle

Line 38 in 7694a27

api "org.apache.lucene:lucene-analysis-nori:${versions.lucene}"

Currently kuromoji is on 9.1.0 (#2487) and nori is on 9.0.0 (#1109) but neither was included in the 9.3.0 update (#3537).

[dbwiddis]

Actually, those were the latest code changes, but the dependency should have been updated with the latest Lucene version used in the build. There is no way to independently upgrade those plugins.

[minalsha]

For OpenSearch 2.0, 2.0 we did upgrade to Lucene 9 and these dependencies@st should have been upgraded. Next steps would be to validate with customer on what version of OpenSearch they are using. @macrakis as discussed today, closing this issue but feel free to open it if you have additional information from customer.