1.23.2

1.23.2 (2024-12-09)

Fix

• fix: Added radar chart displaying capabilities on task types (#1570)

• Added radar chart displaying capabilities on task types

• Fixed table aggregation in leaderboard

• Spelled out why instructionretrieval is excluded (c49f838)

1.23.1

1.23.1 (2024-12-09)

Fix

• fix: Added metadata for miscellaneous models (#1557)

• Added script for generating metadata, and metadata for the listed models

• Added misc models to overview

• Fixed misc metas

• Removed unnecessary imports

• Added logic to retrieve base model information

• Added base models to misc meta

• Added superseded_by to sentence-croissant models

• Added training datasets to mis models (ce8c175)

1.23.0

1.23.0 (2024-12-08)

Documentation

• docs: Fix dependency library name for bm25s (#1568)

• fix: bm25s implementation

• correct library name

---

Co-authored-by: Daniel Buades Marcos <daniel.buades@clinia.com> (03347eb)

Feature

• feat: (cohere_models) cohere_task_type issue, batch requests and tqdm for visualization (#1564)

• feat: batch requests to cohere models

• fix: use correct task_type

• feat: use tqdm with openai

• fix: explicitely set show_progress_bar to False (1d21818)

Fix

• fix(publichealth-qa): ignore rows with None values in question or answer (#1565) (68bd8ac)

• fix: Add training dataset to model meta (#1561)

• fix: Add training dataset to model meta

Adresses #1556

• Added docs

• format (6489fca)

1.22.1

1.22.1 (2024-12-07)

Fix

• fix(bm25s): search implementation (#1566)

fix: bm25s implementation (ac44e58)

1.22.0

1.22.0 (2024-12-07)

Documentation

• docs: Correction of SICK-R metadata (#1558)

• Correction of SICK-R metadata

• Correction of SICK-R metadata

---

Co-authored-by: rposwiata <rposwiata@opi.org.pl> (fc64791)

Feature

• feat(google_models): fix issues and add support for text-embedding-005 and text-multilingual-embedding-002 (#1562)

• fix: google_models batching and prompt

• feat: add text-embedding-005 and text-multilingual-embedding-002

• chore: make lint errors

• fix: address PR comments (611b6a1)

1.21.8

1.21.8 (2024-12-06)

Fix

• fix: Add Model2vec (#1546)

• Added Model2Vec wrapper

• Added Model2vec models

• Added model2vec models to registry

• Added model2vec as a dependency

• Ran linting

• Update mteb/models/model2vec_models.py

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com>

• Update mteb/models/model2vec_models.py

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com>

• Added adapted_from and superseeded_by to model2vec models.

• Added missing import

• Moved pyproject.toml to optional dependencies

• Fixed typos

• Added import error and changed model to model_name

• Added Numpy to frameworks

• Added Numpy to frameworks

• Corrected false info on model2vec models

• Replaced np.inf with maxint

• Update mteb/models/model2vec_models.py

Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>

• Added option to have infinite max tokens, added it to Model2vec

---

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com>
Co-authored-by: Isaac Chung <chungisaac1217@gmail.com> (2ee8d44)

Unknown

• Made result loading more permissive, changed eval splits for HotPotQA and DBPedia (#1554)

• Removed train and dev from eval splits on HotpotQA

• Removed dev from eval splits on DBPedia

• Made task_results validation more permissive

• Readded exception in get_score

• Ran linting (2905813)

1.21.7

1.21.7 (2024-12-05)

Fix

• fix: remove curev1 from multlingual (#1552)

Seems like it was added here:
1cc6c9e (279a4ee)

1.21.6

1.21.6 (2024-12-04)

Fix

• fix: Fixed metadata errors (#1547) (a44a46c)

1.21.5

1.21.5 (2024-12-04)

Fix

• fix: add sentence trimming to OpenAIWrapper (#1526)

• fix: add sentence trimming to OpenAIWrapper

• fix: import tiktoken library inside encode function

• fix: check tokenizer library installed and update ModelMeta to pass tokenizer_name

• fix: pass tokenizer_name, max_tokens to loader

• fix: make tokenizer_name None for default

• fix: delete changes for ModelMeta

• fix: fix revision to 2 for OpenAI models

• fix: add docstring for OpenAIWrapper

• fix: lint

• feat: add openai optional dependency set

• fix: add sleep for too many requests

• fix: add lint

• fix: delete evaluate file (37fdfa1)

• fix: Added arctic models (#1541)

#1515 (df11c38)

• fix: Added all-minilm-l12-v2 (#1542)

#1515 (97ab272)

• fix: add nomic models (#1543)

#1515 (5013df8)

Unknown

• Add cohere models (#1538)

• fix: bug cohere names

• format (c2f4c26)

1.21.4

1.21.4 (2024-12-04)

Documentation

• docs: Add Model Meta parameters and metadata (#1536)

• add multi_qa_MiniLM_L6_cos_v1 model meta

• add all_mpnet_base_v2

• add parameters to model meta

• make lint

• add extra params to meta (5fa7b7b)

Fix

• fix: add more model meta (jina, e5) (#1537)

• add e5 model meta

• address review comments (36bab4d)