This is a fork of https://github.com/o19s/elasticsearch-learning-to-rank to work with OpenSearch. It's a rewrite of some parts to be able to work with OpenSearch. Please refer to official documentation of Elasticsearch Learning to Rank for usage.
The OpenSearch Learning to Rank plugin uses machine learning to improve search relevance ranking. The original Elasticsearch LTR plugin powers search at places like Wikimedia Foundation and Snagajob.
To install, you'd run a command like this but replacing with the appropriate prebuilt version zip:
| OS | Command |
|---|---|
| 1.0.0 | bin/opensearch-plugin install https://github.com/aparo/opensearch-learning-to-rank/releases/download/1.0.0/ltr-1.5.4-os1.0.0.zip |
| 1.1.0 | bin/opensearch-plugin install https://github.com/aparo/opensearch-learning-to-rank/releases/download/1.1.0/ltr-1.5.4-os1.1.0.zip |
| 1.2.0 | bin/opensearch-plugin install https://github.com/aparo/opensearch-learning-to-rank/releases/download/1.2.0/ltr-1.5.4-os1.2.0.zip |
| 1.2.2 | bin/opensearch-plugin install https://github.com/aparo/opensearch-learning-to-rank/releases/download/1.2.2/ltr-1.5.4-os1.2.2.zip |
| 1.2.3 | bin/opensearch-plugin install https://github.com/aparo/opensearch-learning-to-rank/releases/download/1.2.3/ltr-1.5.4-os1.2.3.zip |
| 2.2.1 | bin/opensearch-plugin install https://github.com/aparo/opensearch-learning-to-rank/releases/download/2.2.1/ltr-2.0.0-os2.2.1.zip |
| 2.5.0 | bin/opensearch-plugin install https://github.com/gsingers/opensearch-learning-to-rank-base/releases/download/release-v2.1.0/ltr-plugin-v2.1.0.zip |
(It's expected you'll confirm some security exceptions, you can pass -b to opensearch-plugin to automatically install)
If you already are running OpenSearch, don't forget to restart!
Releases can be found at https://github.com/gsingers/opensearch-learning-to-rank-base/releases.
Releases are done through Github Workflows (see .github/workflows in the root directory) on an as needed basis. If you do ./gradlew build as per above under building,
it will build all the artifacts that are in the release.
These releases are alpha because some issues with the tests due to securemock that depends on ElasticSearch security stuff. And there are 14 failing tests.
Tests with failures:
- com.o19s.es.ltr.feature.store.StoredFeatureSetParserTests.testExpressionDoubleQueryParameter
- com.o19s.es.ltr.feature.store.StoredFeatureSetParserTests.testExpressionMissingQueryParameter
- com.o19s.es.ltr.feature.store.StoredFeatureSetParserTests.testExpressionIntegerQueryParameter
- com.o19s.es.ltr.feature.store.StoredFeatureSetParserTests.testExpressionShortQueryParameter
- com.o19s.es.ltr.feature.store.StoredFeatureSetParserTests.testExpressionInvalidQueryParameter
- com.o19s.es.termstat.TermStatQueryBuilderTests.testMustRewrite
- com.o19s.es.termstat.TermStatQueryBuilderTests.testToQuery
- com.o19s.es.termstat.TermStatQueryBuilderTests.testCacheability
- com.o19s.es.ltr.feature.store.StoredFeatureParserTests.testExpressionOptimization
- com.o19s.es.termstat.TermStatQueryTests.testEmptyTerms
- com.o19s.es.termstat.TermStatQueryTests.testUniqueCount
- com.o19s.es.termstat.TermStatQueryTests.testBasicFormula
- com.o19s.es.termstat.TermStatQueryTests.testQuery
- com.o19s.es.termstat.TermStatQueryTests.testMatchCount
228 tests completed, 14 failed
To build, you need to disable the Java security manager
./gradlew -Dtests.security.manager=false clean build
- Edit
gradle.propertiesto have the appropriate versions (it's often easiest to go download the latest tarball from OpenSearch and simply check the versions that ship) and to increment the version of this plugin - Build and test as above
- Update this README with the version info in the table above
- Upgrade the Docker file versions in the
dockerdirectory - Test the docker image, per below.
A custom image of OpenSearch with the OpenSearch Learning to Rank plugin installed.
This image was created for the Search with Machine Learning course and Search Fundamentals taught by Grant Ingersoll and Daniel Tunkelang.
See the Elasticsearch Learning to Rank documentation for details on how to us.
Building the docker image is triggered via the Github Actions workflows automatically (for releases) or via the commands below.
Note, we are use Docker ARGs to pass through variables via the --build-arg. All args have defaults
docker build -f docker/local.Dockerfile .
docker build -f docker/Dockerfile --tag=YOUR/IMAGE_NAME .
docker build -f docker/Dockerfile --tag=YOUR/IMAGE_NAME --build-arg opensearch_version=2.2.1 --build-arg ltrversion=2.0.0 .
docker build -f docker/Dockerfile --tag=YOUR/IMAGE_NAME --build-arg plugin="https://github.com/gsingers/opensearch-learning-to-rank-base/releases/download/release-test-release/ltr-plugin-test-release.zip" .
See the OpenSearch docs for official instructions, but this should work:
docker run -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" YOUR/IMAGE_NAME:latest
To publish the Docker image to Docker Hub, you need to kick off the Docker action workflow:
gh workflow run .github/workflows/docker.yml