Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HF pipeline to run Ultravox independent of Ultravox repo #49

Merged
merged 7 commits into from
Jul 17, 2024

Conversation

farzadab
Copy link
Contributor

@farzadab farzadab commented Jul 11, 2024

Right now you can load the model without having access to the repo using:

# pip install transformers peft

import transformers
import numpy as np

pipe = transformers.pipeline(model='fixie-ai/ultravox-v0_2', trust_remote_code=True)

pipe({'audio': np.random.randn(16000), 'sampling_rate': 16000}, max_new_tokens=30)

The result is gonna be:

"assistant\n\nI'm here to help! What's on your mind?"

I ran into multiple issues with HF Hub to get this working. As a result I had to make two sacrifices:

  1. change HF Hub repo name so it doesn't contain any dots (.): fixie-ai/ultravox-v0.2 -> fixie-ai/ultravox-v0_2
    • The original name should still work as it redirects to the new address
  2. imports in ultravox/model directory had to break the style guide, e.g. from .ultravox_config import UltravoxConfig

@farzadab farzadab requested a review from juberti July 11, 2024 00:13
@farzadab farzadab marked this pull request as ready for review July 11, 2024 00:16
@farzadab farzadab requested a review from zkoch July 11, 2024 00:16
ultravox/data/data_processing.py Outdated Show resolved Hide resolved
ultravox/model/ultravox_model.py Outdated Show resolved Hide resolved
ultravox/model/ultravox_pipeline.py Outdated Show resolved Hide resolved
ultravox/model/ultravox_pipeline.py Outdated Show resolved Hide resolved
ultravox/model/ultravox_pipeline.py Show resolved Hide resolved
ultravox/tools/push_to_hub.py Outdated Show resolved Hide resolved
ultravox/tools/push_to_hub.py Outdated Show resolved Hide resolved
@farzadab farzadab merged commit 772f292 into main Jul 17, 2024
1 check passed
@farzadab farzadab deleted the farzad-hf-pipeline branch July 17, 2024 20:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants