Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speech Translation Evals #54

Merged
merged 8 commits into from
Jul 25, 2024
Merged

Speech Translation Evals #54

merged 8 commits into from
Jul 25, 2024

Conversation

farzadab
Copy link
Contributor

This PR adds a sample-level BLEU score as well as some evaluations using this metric.

BLEU is supposed to be a corpus-level metric, but this approach was simpler to get working at the moment (given that our eval framework will be overhauled soon). As such, the final score will not be comparable to other results as a result, but that was a given since we don't run over the whole datasets.

Closes #50

@farzadab farzadab marked this pull request as ready for review July 24, 2024 18:33
@farzadab farzadab requested a review from zqhuang211 July 24, 2024 18:33
@farzadab farzadab changed the title Speech Translation Evals [Blocked by #53] Speech Translation Evals Jul 24, 2024
@farzadab farzadab merged commit cc48b74 into main Jul 25, 2024
1 check passed
@farzadab farzadab deleted the farzad-bleu-eval branch July 25, 2024 01:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Tracking] Speech Translation Evaluation: CoVoST 2
3 participants