Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mmlu translated professionally by OpenAI #2312

Open
wants to merge 25 commits into
base: main
Choose a base branch
from

Commits on Sep 17, 2024

  1. First commit.

    giuliolovisotto committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    ca2dd8b View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2024

  1. Update README.md (EleutherAI#2297)

    * Update README.md
    
    I encounter some Git buffer size limits when trying to download all commits history of the repository, such as:
    ```error: RPC failed; curl 18 transfer closed with outstanding read data remaining
    error: 5815 bytes of body are still expected
    fetch-pack: unexpected disconnect while reading sideband packet
    fatal: early EOF```
    
    therefore the installation is faster and there are not errors when I download only the last version of the repository
    
    * Fix linting issue
    SYusupov authored and giuliolovisotto committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    bad44e7 View commit details
    Browse the repository at this point in the history
  2. repr bug (EleutherAI#2315)

    baberabb authored and giuliolovisotto committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    b1aedcd View commit details
    Browse the repository at this point in the history
  3. Update neuron backend (EleutherAI#2314)

    * feat(neuron): align with latest optimum-neuron
    
    * feat(neuron): support pre-exported neuron models
    
    * fix(neuron): correctly use max_length
    
    * fix(neuron): adapt loglikelihood
    
    The evaluation of log likelihood was not working for neuron models
    using continuous batching, such as all cached neuron LLama models.
    
    * refactor(neuron): remove dead code
    dacorvo authored and giuliolovisotto committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    a0d66fb View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    5be4d01 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    b59b9b9 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    81b13e4 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    2476ad3 View commit details
    Browse the repository at this point in the history
  8. Treat tags in python tasks the same as yaml tasks (EleutherAI#2288)

    * Treat python tasks same as yaml tasks.
    
    * Add tests.
    
    * Re-add fixture decorators.
    
    * Fix typing specification error for Python 3.9.
    giuliolovisotto committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    bebc227 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    e6726a7 View commit details
    Browse the repository at this point in the history
  10. change glianorex to test split (EleutherAI#2332)

    * change glianorex to test set
    
    * nit
    
    * fix test; doc_to_target can be str for multiple_choice
    
    * nit
    baberabb authored and giuliolovisotto committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    969ea42 View commit details
    Browse the repository at this point in the history
  11. mmlu-pro: add newlines to task descriptions (not leaderboard) (Eleuth…

    …erAI#2334)
    
    * add newlines to task descriptions; increment versions
    
    * fix task tests (with groups)
    
    * Apply suggestions from code review
    
    ---------
    
    Co-authored-by: Hailey Schoelkopf <65563625+haileyschoelkopf@users.noreply.github.com>
    2 people authored and giuliolovisotto committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    4ca6f35 View commit details
    Browse the repository at this point in the history
  12. Added TurkishMMLU to LM Evaluation Harness (EleutherAI#2283)

    * Added TurkishMMLU to LM Evaluation Harness
    
    * Fixed COT name
    
    * Fixed COT name
    
    * Updated Readme
    
    * Fixed Test issues
    
    * Completed  Scan for changed tasks
    
    * Updated Readme
    
    * Update README.md
    
    * fixup task naming casing + ensure yaml template stubs aren't registered
    
    ---------
    
    Co-authored-by: Hailey Schoelkopf <65563625+haileyschoelkopf@users.noreply.github.com>
    Co-authored-by: haileyschoelkopf <hailey@eleuther.ai>
    3 people authored and giuliolovisotto committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    641411f View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    0bb44af View commit details
    Browse the repository at this point in the history
  14. openai: better error messages; fix greedy matching (EleutherAI#2327)

    * better error message; fix greedy matching
    
    * Update lm_eval/models/openai_completions.py
    
    Co-authored-by: Hailey Schoelkopf <65563625+haileyschoelkopf@users.noreply.github.com>
    
    * Update lm_eval/models/openai_completions.py
    
    Co-authored-by: Hailey Schoelkopf <65563625+haileyschoelkopf@users.noreply.github.com>
    
    * pre-commit
    
    ---------
    
    Co-authored-by: Hailey Schoelkopf <65563625+haileyschoelkopf@users.noreply.github.com>
    2 people authored and giuliolovisotto committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    c51deda View commit details
    Browse the repository at this point in the history
  15. Rename this file.

    giuliolovisotto committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    0e5ef9d View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    ad1ce4e View commit details
    Browse the repository at this point in the history
  17. Refactored structure.

    giuliolovisotto committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    6f58636 View commit details
    Browse the repository at this point in the history
  18. Configuration menu
    Copy the full SHA
    d685ed5 View commit details
    Browse the repository at this point in the history
  19. Updated readme.md

    giuliolovisotto committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    b6f2fe3 View commit details
    Browse the repository at this point in the history
  20. Configuration menu
    Copy the full SHA
    a2a9e8e View commit details
    Browse the repository at this point in the history
  21. Re-add english.

    giuliolovisotto committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    4fff660 View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2024

  1. Configuration menu
    Copy the full SHA
    41e8a92 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    6807c19 View commit details
    Browse the repository at this point in the history

Commits on Oct 8, 2024

  1. Configuration menu
    Copy the full SHA
    c8ca981 View commit details
    Browse the repository at this point in the history