Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add example of LLava encoder #2375

Closed
wants to merge 2 commits into from
Closed

Add example of LLava encoder #2375

wants to merge 2 commits into from

Conversation

iseeyuan
Copy link
Contributor

Summary: Add the example to start enabling LLava, one multimodal model in generative AI area. In this example, we initiate the process of running LLava through ExecuTorch. Refer to the added README.md for details.

Differential Revision: D54812717

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Mar 12, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/2375

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures

As of commit ec64e19 with merge base 75284d2 (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 12, 2024
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D54812717

Martin Yuan added 2 commits March 12, 2024 11:10
@facebook-github-bot
Add torch.no_grad() guard on export (#2373)
ff83562 
Summary:

For some models where the grad is enabled (like LLava model), we need to disable grad before exporting. This PR is to add this guard when using export script. It would not affect the models that grad is not enabled.

Reviewed By: cccclai

Differential Revision: D54812718
@facebook-github-bot
Add example of LLava encoder (#2375)
ec64e19 
Summary:

Add the example to start enabling LLava, one multimodal model in generative AI area. In this example, we initiate the process of running LLava through ExecuTorch. Refer to the added README.md for details.

Reviewed By: cccclai

Differential Revision: D54812717
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D54812717

@facebook-github-bot
Copy link
Contributor

This pull request has been merged in 42eeebc.

@guangy10
Copy link
Contributor

@huydhn Test for this model still fails the documentation build even after #2376 got merged, which blocks the stable branch moving forward.

@huydhn
Copy link
Contributor

huydhn commented Mar 13, 2024

@guangy10

The doc build job looks fixed in trunk now I think https://hud.pytorch.org/hud/pytorch/executorch/main, so we should be good there.

For this PR, doing a rebase will fix the doc build failure. On the other hand, let me retry those remaining failures in trunks, some of them look flaky.

@facebook-github-bot
Copy link
Contributor

This pull request has been reverted by 3e414fb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cccclai cccclai cccclai approved these changes

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported Merged Reverted
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@iseeyuan @facebook-github-bot @guangy10 @huydhn @cccclai