Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LTR feature logger #648

Merged
merged 12 commits into from
Jan 12, 2024
Merged

LTR feature logger #648

merged 12 commits into from
Jan 12, 2024

Conversation

afoucret
Copy link
Contributor

@afoucret afoucret commented Jan 5, 2024

The PR provides tools for LTR query feature extraction into Eland.

  1. Some base class have been defined to declare the LTR config:
ltr_config = LTRModelConfig(
  feature_extractors=[
    QueryFeatureExtractor(
      feature_name="title_bm25", 
      query={"match": {"title": "{{query}}"}},
    ),
    QueryFeatureExtractor(
      feature_name="popularity",
      query={
        "script_score": {
          "query": {"exists": {"field": "popularity"}},
          "script": {"source": "return doc['popularity'].value"},
        }
      },
    )
  ]
)
  1. A feature logger can be created using:
feature_logger = FeatureLogger(es_client="http://localhost:9200",  es_index="my-index", ltr_model_config=ltr_model_config)
  1. Document features can be extracted for a set of query params using:
feature_logger.extract_features(
  query_params={ "query": "my search query" },
  doc_ids= ["doc-1", "doc-2"] 
)

@afoucret afoucret marked this pull request as ready for review January 11, 2024 23:23
Copy link

@ioanatia ioanatia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since movies.json.gz is only used for tests, is it possible to reduce its size? with 7.43 MB it seems to be the largest artefact in tests.

"""
List of the feature names for the model.
"""
if not hasattr(self, "_feature_names"):
Copy link

@ioanatia ioanatia Jan 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we seem to reuse this pattern in multiple methods:

if not hasattr(self, attr_name)
  # assign attribute
return attribute

this seems like a good candidate for using the cached_property decorator: https://docs.python.org/3/library/functools.html#functools.cached_property

I see from the README that Eland supports python 3.8, 3.9, 3.10. So cached_property should be available.

- Supports Python 3.8, 3.9, 3.10 and Pandas 1.5

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did change the implementation to use cached_property. Thank you for the tip.

@afoucret afoucret merged commit d3ed669 into elastic:main Jan 12, 2024
4 checks passed
@@ -50,3 +50,6 @@ Permission is hereby granted, free of charge, to any person obtaining a copy of
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

--
This product contains a adapted version of the "us-national-parks" dataset, https://data.world/kevinnayar/us-national-parks, by Kevin Nayar, https://data.world/kevinnayar, is licensed under CC BY, https://creativecommons.org/licenses/by/4.0/legalcode
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo - this should be "an adapted version"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants