Skip to content

Pull requests: EleutherAI/lm-evaluation-harness

New pull request New
91 Open 1,190 Closed
91 Open 1,190 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

BEEP, BLM-IT, TraceIT, Multi-IT and VeryfIT
#2403 opened Oct 14, 2024 by Jj-source Loading…
1
Support for IBM watsonx_llm
#2397 opened Oct 11, 2024 by Medokins Loading…
1
Fix: Turkish MMLU Regex Pattern
#2393 opened Oct 10, 2024 by ArdaYueksel Loading…
Update citation links to Zenodo and DOI to 0.4.5
#2391 opened Oct 9, 2024 by LSinev Loading…
Add new tasks to spanish_bench and fix duplicates
#2390 opened Oct 9, 2024 by zxcvuser Loading…
4
log_samples File name too long. Need truncation or override #1454 FIX
#2384 opened Oct 5, 2024 by kroc99 Loading…
1
add Russian mmlu
#2378 opened Oct 3, 2024 by tatiana-iazykova Loading…
2
Add the BlueBench benchmark
#2369 opened Oct 1, 2024 by shachardon Loading…
3
Remove unnecessary space prefix
#2368 opened Oct 1, 2024 by eldarkurtic Loading…
1
MMLU Pro Plus
#2366 opened Sep 30, 2024 by asgsaeid Loading…
1
Add Unitxt Multimodality Support
#2364 opened Sep 29, 2024 by elronbandel Loading…
5
fix cost_estimate script
#2359 opened Sep 26, 2024 by baberabb • Draft
Add metabench task to LM Evaluation Harness
#2357 opened Sep 26, 2024 by kozzy97 Loading…
1
Support pipeline parallel with OpenVINO models
#2349 opened Sep 25, 2024 by sstrehlk Loading…
1
Mathvista
#2321 opened Sep 18, 2024 by baberabb • Draft
mmlu translated professionally by OpenAI
#2312 opened Sep 17, 2024 by giuliolovisotto Loading…
1
Scrolls branch
#2309 opened Sep 16, 2024 by blitzionic Loading…
2
avoid timeout errors with high concurrency in api_model
#2307 opened Sep 16, 2024 by dtrawins Loading…
4
add new truncation strategy
#2300 opened Sep 15, 2024 by artemorloff • Draft
3
Gen Prefix
#2274 opened Sep 2, 2024 by baberabb Loading…
1
2
Nvidia TensorRT-LLM
#2271 opened Sep 1, 2024 by abhishekvijeev • Draft
1
Add Yue-Benchmark and update tasks description
#2270 opened Aug 31, 2024 by cpa2001 Loading…
1
Ifeval: Dowload punkt_tab on rank 0
#2267 opened Aug 30, 2024 by baberabb Loading…
1
6
[Draft] llm-as-judge
#2251 opened Aug 25, 2024 by baberabb • Draft
1
Minor features
#2249 opened Aug 25, 2024 by artemorloff Loading…
8
ProTip! Exclude everything labeled bug with -label:bug.