Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update results_to_dataframe to use BenchmarkResults class #1628

Merged

Conversation

AlexeyVatolin
Copy link
Contributor

@AlexeyVatolin AlexeyVatolin commented Dec 24, 2024

Changed the results_to_dataframe function to make the example from the documentation work.
Error before

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[32], line 11
      7 models = [mteb.get_model_meta(name) for name in results_eval_mteb_df["full_name"].unique()]
      9 results = mteb.load_results(models=models, tasks=tasks)
---> 11 df = results_to_dataframe(results)

File ~/miniconda3/envs/llm_emb/lib/python3.12/site-packages/mteb/task_selection.py:50, in results_to_dataframe(mteb_results, drop_na, **kwargs)
     42 """Convert the results of the MTEB evaluation to a pandas DataFrame.
     43 
     44 Args:
   (...)
     47     **kwargs: Additional keyword arguments to be passed to the `get_score` method of the `MTEBResults` class.
     48 """
     49 data = []
---> 50 for model_name, revisions in mteb_results.items():
     51     for rev, tasks_results in revisions.items():
     52         for task_result in tasks_results:

File ~/miniconda3/envs/llm_emb/lib/python3.12/site-packages/pydantic/main.py:892, in BaseModel.__getattr__(self, item)
    889     return super().__getattribute__(item)  # Raises AttributeError if appropriate
    890 else:
    891     # this is the current error
--> 892     raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')

AttributeError: 'BenchmarkResults' object has no attribute 'items'

Checklist

  • Run tests locally to make sure nothing is broken using make test.
  • Run the formatter to format the code using make lint.

Copy link
Collaborator

@isaac-chung isaac-chung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching this!

@isaac-chung isaac-chung merged commit 02ae4fa into embeddings-benchmark:main Dec 24, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants