Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

📻 (AST) Audio data classification optimization and data pre-process #762

Merged
merged 6 commits into from
Dec 1, 2023

Conversation

trajepl
Copy link
Contributor

@trajepl trajepl commented Nov 29, 2023

Describe your changes

For this issue #735, I tried to add an example for AST model optimization with Olive data configs on huggingface examples to achieve script-free for model optimization.

Checklist before requesting a review

  • Add unit tests for this change.
  • Make sure all tests can pass.
  • Update documents if necessary.
  • Lint and apply fixes to your code by running lintrunner -a
  • Is this a user-facing change? If yes, give a description of this change to be included in the release notes.

(Optional) Issue link

@trajepl trajepl changed the title (AST) Audio data classification optimization and data pre-process 📻 (AST) Audio data classification optimization and data pre-process Nov 29, 2023
@vymao
Copy link

vymao commented Nov 29, 2023

Thanks. Though one thing to note is that the actual speech_commands dataset by itself doesn't match what the AST-v2 model was fine-tuned on, and you probably need to do some pre-processing to remove extra labels: huggingface/datasets#6446

@vymao
Copy link

vymao commented Nov 29, 2023

Also I think you need to query file from the examples, currently getting an error from here:

  File "/Users/victor/anaconda3/envs/transformers-v2/lib/python3.9/site-packages/olive/data/component/pre_process_data.py", line 259, in _tokenizer_and_align_labels
    for files in tokenized_inputs[kwargs.get("file_column_name", "file")]:
  File "/Users/victor/anaconda3/envs/transformers-v2/lib/python3.9/site-packages/transformers/feature_extraction_utils.py", line 86, in __getitem__
    return self.data[item]
KeyError: 'file'

@trajepl
Copy link
Contributor Author

trajepl commented Nov 30, 2023

Thanks. Though one thing to note is that the actual speech_commands dataset by itself doesn't match what the AST-v2 model was fine-tuned on, and you probably need to do some pre-processing to remove extra labels: huggingface/datasets#6446

Yes, I manually align the labels by file prefix. The way you show is the better way. Updated.

@trajepl
Copy link
Contributor Author

trajepl commented Nov 30, 2023

Also I think you need to query file from the examples, currently getting an error from here:

  File "/Users/victor/anaconda3/envs/transformers-v2/lib/python3.9/site-packages/olive/data/component/pre_process_data.py", line 259, in _tokenizer_and_align_labels
    for files in tokenized_inputs[kwargs.get("file_column_name", "file")]:
  File "/Users/victor/anaconda3/envs/transformers-v2/lib/python3.9/site-packages/transformers/feature_extraction_utils.py", line 86, in __getitem__
    return self.data[item]
KeyError: 'file'

fixed.

@trajepl
Copy link
Contributor Author

trajepl commented Nov 30, 2023

[Not merge] Let us wait for customer's feedback.

@vymao
Copy link

vymao commented Nov 30, 2023

From the linked issue, you probably need to remove the _silence_ label before running align_labels_with_mapping, currently getting a _silence_ KeyError. Not very clean but maybe you can make this an option.

@jambayk
Copy link
Contributor

jambayk commented Nov 30, 2023

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@trajepl
Copy link
Contributor Author

trajepl commented Dec 1, 2023

From the linked issue, you probably need to remove the _silence_ label before running align_labels_with_mapping, currently getting a _silence_ KeyError. Not very clean but maybe you can make this an option.

https://github.com/microsoft/Olive/pull/762/files#:~:text=%23%20align%20labels%20with,%5B0%5D)
I already did that. The config example worked for me. When you trigger the error of _silence_ key error? Have you updated your config json with the latest one in this pr? @vymao

latest changes for json config:
https://github.com/microsoft/Olive/pull/762/files#diff-f9735dfb6fa445feeb666b7f11c9b670283b799fecf876683962741bee8733fe:~:text=%22component_kwargs%22%3A%20%7B,%7D

@trajepl trajepl merged commit 32322a8 into main Dec 1, 2023
31 checks passed
@trajepl trajepl deleted the jiapli/audio_data_preprocess branch December 1, 2023 06:28
@vymao
Copy link

vymao commented Dec 2, 2023

Thanks, works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants