feat: use chunk data in NIAH and QA evals

[jalling97]

Description

This PR adds chunk data to the NIAH and QA evaluations to better evaluate the retrieval stage of RAG

• Changes the NIAH retrieval metric to be based on the actual chunk data, not just the annotations
• Adds chunk_rank metric to NIAH evals to evaluate how well ranked the chunks are
• Adds 2 new LLM-as-judge metrics to QA evals: Contextual Relevancy and Faithfulness
• Updates package versions
• Fixes bug when not using padding data in NIAH evals

BREAKING CHANGES

NIAH Retrieval now measures slightly differently, meaning prior NIAH retrieval metrics should not be compared to these new values.

CHANGES

Replaces the current proxy annotation measure for NIAH retrievals with one based on the chunk data directly.

Related Issue

Relates to #1067

Checklist before merging

• Tests, documentation, ADR added or updated as needed
• Followed the Contributor Guide Steps

[netlify]

✅ Deploy Preview for leapfrogai-docs canceled.

Name	Link
🔨 Latest commit	855cec5
🔍 Latest deploy log	https://app.netlify.com/sites/leapfrogai-docs/deploys/67000ac85a9ad60008a999e9

[jalling97]

Evaluation results from most recent run using new metrics:

Final Results:
INFO:root:Average Needle in a Haystack (NIAH) Retrieval: 1.0
INFO:root:Average Needle in a Haystack (NIAH) Response: 1.0
INFO:root:Average Needle in a Haystack (NIAH) Chunk Rank: 0.9600000000000001
INFO:root:Average Correctness (GEval): 0.82
INFO:root:Average Answer Relevancy: 0.9583333333333335
INFO:root:Average Contextual Relevancy: 0.504
INFO:root:Average Faithfulness: 0.9278174603174603
INFO:root:Average Annotation Relevancy: 0.9359999999999999
INFO:root:MMLU: 0.696969696969697
INFO:root:HumanEval: 0.95
INFO:root:Eval Execution Runtime (seconds): 1655.2433378696442