Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In the task "balance_chemical_equation", many instances have incorrect labels. #1386

Open
dongZheX opened this issue Oct 19, 2023 · 1 comment
Labels
bug Something isn't working

Comments

@dongZheX
Copy link

Describe the bug

For example

You are ChemistGPT, can help user balance chemical equation. For example, if user's input is "C6H5COOH + O2 = CO2 + H2O", you will reply the balanced chemical equation: "2C6H5COOH + 15O2 = 14CO2 + 6H2O", without explanation. If you can't balance the equation, just reply "Unknown"

Input: Al(OH)3 + H2SO4 = Al2(SO4)3 + H2O
Answer: 

The given groundtruth is

2Al(OH)3 + 3H2SO4 = Al2(SO4)3 + 3H2O

but the real correct label is

2Al(OH)3 + 3H2SO4 = Al2(SO4)3 + 6H2O

There are many problems like this, hope to fix.
@scruel

To Reproduce

Nothing

Code snippets

Nothing

OS

macOS

Python version

Python3.11

Library version

openai-evals latest

@dongZheX dongZheX added the bug Something isn't working label Oct 19, 2023
@scruel
Copy link
Contributor

scruel commented Oct 20, 2023

Hi, thank you for pointing out those incorrect labels, I would like to fix them, however currently I don't have enough time for this.
Anyway, you're welcome to create a PR to fix this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants