Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Metrics import path in community task template file. #309

Merged
merged 8 commits into from
Oct 1, 2024

Conversation

chuandudx
Copy link
Contributor

Fix the metric import path for new users who want to base their custom task on this template.

@clefourrier
Copy link
Member

It's likely not going to pass for style reasons as it's duplicating an import

@chuandudx
Copy link
Contributor Author

I fixed the duplicate import and ran ruff format according to the following - hope this works now

Run make quality
ruff format --check .
[7](https://github.com/huggingface/lighteval/actions/runs/10897826103/job/30244895251#step:5:8)8 files already formatted
ruff check .
community_tasks/_template.py:31:1: I001 [*] Import block is un-sorted or un-formatted
Found 1 error.
[*] 1 fixable with the `--fix` option.
make: *** [Makefile:[11](https://github.com/huggingface/lighteval/actions/runs/10897826103/job/30244895251#step:5:12): quality] Error 1
Error: Process completed with exit code 2.

Copy link
Member

@clefourrier clefourrier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, will merge if tests pass

…g automated tests for authentication purposes.
@chuandudx
Copy link
Contributor Author

chuandudx commented Sep 24, 2024

Hi Clémentine, I made a new commit (minor change in comments) in order to push code along with my HF_TOKEN since it appeared that the previous errors were related to user/password - would we be open to trying to submit again? Thank you!

Edit: Upon further investigation, I also set the HF_TOKEN in the environment secret of my fork of the repository. Maybe this will help?
image

@NathanHB
Copy link
Member

Hi ! You need a token that is able to access the https://huggingface.co/lighteval-tests org, I restarted the tests to see if it's because I need to be the one to trigger it

@chuandudx
Copy link
Contributor Author

Hi Nathan, thank you! Thanks for re-triggering it. Does the test check for the token from the person who triggered it or me as the submitter of the PR? Is there any action I need to take to obtain a token that has the appropriate access? Thank you!

Currently I have a user access token following these steps: https://huggingface.co/docs/hub/en/security-tokens

@clefourrier
Copy link
Member

@chuandudx You should not need any token on your side, it should be managed by our workflow entirely (and the token attached should be sent by the one triggering the pipeline).
We changed a small config thing recently and it broke this, so it's on us 😅 - normally one test should be skipped in the absence of the relevant token and it does not seem like it is.

@chuandudx
Copy link
Contributor Author

@clefourrier ohhh I see! Thank you for looking into this 😊!!

@clefourrier clefourrier merged commit 01e7240 into huggingface:main Oct 1, 2024
2 checks passed
@chuandudx chuandudx deleted the fix_template_import branch October 15, 2024 00:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants