Skip to content

v0.6.0

Latest
Compare
Choose a tag to compare
@NathanHB NathanHB released this 23 Oct 16:02
· 10 commits to main since this release

What's New

Lighteval becomes massively multilingual!

We now have extensive coverage in many languages, as well as new templates to manage multilinguality more easily.

Other Tasks

Features

Bug Fixes

Significant community contributions

The following contributors have made significant changes to the library over the last release:

  • @hynky1999
    • Support for multilingual generative metrics (#293)
    • Adds tasks templating (#335)
    • Multilingual NLI Tasks (#329)
    • Multilingual COPA tasks (#330)
    • Multilingual Hellaswag tasks (#332)
    • Multilingual Reading Comprehension tasks (#333)
    • Multilingual General Knowledge tasks (#338)
    • Selecting tasks using their superset (#308)
    • Fix Tokenization + misc fixes (#354)
    • Misc-multilingual tasks (#339)
    • add iroko bench + nicer output on task search failure (#357)
    • Translation literals (#356)
    • selected tasks for multilingual evaluation (#371)
    • Adds Baseline workflow + fixes (#363)
  • @DeanChugall
    • Serbian LLM Benchmark Task (#340)
  • @NathanHB
    • readme rewrite (#327)
    • refacto judge and add mixeval (#337)
    • bump lighteval versoin (#328)
    • fix (#347)
    • Nathan llm judge quickfix (#348)
    • Nathan llm judge quickfix (#350)
    • adds openai models (#359)

New Contributors

Full Changelog: v0.5.0...v0.6.0