Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate Moondream into Twitter and Discord #5

Closed
lalalune opened this issue Jul 9, 2024 · 0 comments
Closed

Integrate Moondream into Twitter and Discord #5

lalalune opened this issue Jul 9, 2024 · 0 comments
Assignees

Comments

@lalalune
Copy link
Member

lalalune commented Jul 9, 2024

The "Whisper Turbo" package comes from Ratchet, a new project by Huggingface.

Also included is an example of using Moondream. Moondream is a model that can describe images.

If a user uploads an image to Discord or Twitter, we want to get the description of the image for Ruby as context. i.e. the agent can "see" the image.

We can use Moondream directly with WebGPU here:
https://github.com/huggingface/ratchet/tree/master/examples/ratchet-moondream

If this proves to be difficult, we can use Moondream from Replicate: https://replicate.com/lucataco/moondream2

However our goal is to have everything running locally.

@lalalune lalalune closed this as completed Sep 4, 2024
sirkitree pushed a commit that referenced this issue Oct 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants