The LLM Evaluation Framework
evaluation-metrics evaluation-framework llm-evaluation llm-evaluation-framework llm-evaluation-metrics
-
Updated
Dec 12, 2024 - Python
#
evaluation-metrics
Here are 457 public repositories matching this topic...
Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks like CrewAI, Langchain, and Autogen
agent ai openai evaluation-metrics mistral cost-estimation autogen groq agentops llm langchain anthropic evals ollama crewai
-
Updated
Dec 13, 2024 - Python
(IROS 2020, ECCVW 2020) Official Python Implementation for "3D Multi-Object Tracking: A Baseline and New Evaluation Metrics"
tracking machine-learning real-time computer-vision robotics evaluation evaluation-metrics multi-object-tracking kitti 3d-tracking 3d-multi-object-tracking 2d-mot-evaluation 3d-mot 3d-multi kitti-3d
-
Updated
Apr 3, 2024 - Python
Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!
-
Updated
Dec 4, 2024 - Jupyter Notebook
Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
-
Updated
Dec 12, 2024 - Python
[NeurIPS'21 Outstanding Paper] Library for reliable evaluation on RL and ML benchmarks, even with only a handful of seeds.
-
Updated
Aug 12, 2024 - Jupyter Notebook
OCTIS: Comparing Topic Models is Simple! A python package to optimize and evaluate topic models (accepted at EACL2021 demo track)
nlp natural-language-processing hyperparameter-optimization topic-modeling nlp-library bayesian-optimization hyperparameter-tuning latent-dirichlet-allocation evaluation-metrics neural-topic-models latent-semantic-analysis topic-models hyperparameter-search non-negative-matrix-factorization nlproc
-
Updated
Jul 25, 2024 - Python
Evaluate your speech-to-text system with similarity measures such as word error rate (WER)
-
Updated
Nov 1, 2024 - Python
📈 Implementation of eight evaluation metrics to access the similarity between two images. The eight metrics are as follows: RMSE, PSNR, SSIM, ISSM, FSIM, SRE, SAM, and UIQ.
-
Updated
Aug 31, 2024 - Python
A Neural Framework for MT Evaluation
nlp machine-learning natural-language-processing machine-translation artificial-intelligence evaluation-metrics
-
Updated
Dec 5, 2024 - Python
⚡️A Blazing-Fast Python Library for Ranking Evaluation, Comparison, and Fusion 🐍
python information-retrieval evaluation comparison numba recommender-systems evaluation-metrics metasearch data-fusion score-fusion ranking-metrics information-retrieval-evaluation information-retrieval-metrics rank-fusion
-
Updated
Jul 1, 2024 - Python
PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language model. There are also more complex data types and algorithms. Mor…
python nlp machine-learning natural-language-processing library linguistics computational-linguistics text-processing nlp-library search-algorithms evaluation-metrics folia language-modelling
-
Updated
Sep 14, 2023 - Python
Data-Driven Evaluation for LLM-Powered Applications
information-retrieval evaluation-metrics evaluation-framework rag llmops retrieval-augmented-generation llm-evaluation
-
Updated
Sep 2, 2024 - Python
Source code for "Taming Visually Guided Sound Generation" (Oral at the BMVC 2021)
audio video pytorch transformer gan multi-modal evaluation-metrics video-understanding vas video-features vqvae bmvc melgan audio-generation vggsound
-
Updated
Jul 12, 2024 - Jupyter Notebook
Resources for the "Evaluating the Factual Consistency of Abstractive Text Summarization" paper
-
Updated
Jul 22, 2023 - Python
Metrics to evaluate the quality of responses of your Retrieval Augmented Generation (RAG) applications.
evaluation-metrics evaluation-framework rag large-language-models llm llms llmops retrieval-augmented-generation
-
Updated
Nov 14, 2024 - Python
[ICLR'24] Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
evaluation vision vqa llama object-detection gpt evaluation-metrics iclr multimodal vision-and-language hallucination vicuna gpt-4 foundation-models prompt-engineering chatgpt llava iclr2024
-
Updated
Mar 13, 2024 - Python
A Python wrapper for the ROUGE summarization evaluation package
-
Updated
Feb 10, 2021 - Python
Code base for the precision, recall, density, and coverage metrics for generative models. ICML 2020.
diversity machine-learning deep-learning evaluation generative-adversarial-network generative-model recall precision evaluation-metrics fidelity icml icml-2020 icml2020
-
Updated
Jan 9, 2023 - Python
Python SDK for running evaluations on LLM generated responses
evaluation evaluation-metrics evaluation-framework llmops llm-eval llm-ops llm-evaluation llm-evaluation-toolkit
-
Updated
Dec 12, 2024 - Python
Improve this page
Add a description, image, and links to the evaluation-metrics topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the evaluation-metrics topic, visit your repo's landing page and select "manage topics."