Repositories list

• LOVA3

  Public
  (NeurIPS 2024) Learning to Visual Question Answering, Asking and Assessment

  benchmarkvisual-question-answeringmultimodal-deep-learning+ 3visual-question-generationmultimodal-large-language-modelsdata-asse

  Python

  •1•63•0•0•Updated Nov 7, 2024Nov 7, 2024

• Awesome-Unified-Multimodal-Models

  Public
  📖 This is a repository for organizing papers, codes and other resources related to unified multimodal models.

  8•203•0•0•Updated Nov 7, 2024Nov 7, 2024

• computer_use_ootb

  Public
  An out-of-the-box (OOTB) version of Anthropic Claude Computer Use

  Python

  •

  MIT License

  •22•226•8•2•Updated Nov 6, 2024Nov 6, 2024

• Awesome-Video-Diffusion

  Public
  A curated list of recent diffusion models for video generation, editing, restoration, understanding, etc.

  awesomevideo-editingvideo-understanding+ 5video-generationdiffusion-modelstext-to-videovideo-restorationtext-to-motion

  200•3.4k•0•0•Updated Nov 6, 2024Nov 6, 2024

• Show-o

  Public
  Repository for Show-o, One Single Transformer to Unify Multimodal Understanding and Generation.

  multimodaldiffusion-modelslarge-language-models

  Python

  •

  Apache License 2.0

  •44•1k•29•0•Updated Nov 5, 2024Nov 5, 2024

• VideoLISA

  Public
  [NeurlPS 2024] One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videos

  0•25•2•0•Updated Nov 3, 2024Nov 3, 2024

• ShowUI

  Public
  0•2•0•0•Updated Oct 31, 2024Oct 31, 2024

• VisInContext

  Public
  Official implementation of Leveraging Visual Tokens for Extended Text Contexts in Multi-Modal Learning

  efficientin-context-learningllm+ 1mllm

  Python

  •1•11•1•0•Updated Oct 30, 2024Oct 30, 2024

• Exo2Ego-V

  Public
  0•2•0•0•Updated Oct 29, 2024Oct 29, 2024

• Awesome-GUI-Agent

  Public
  💻 A curated list of papers and resources for multi-modal Graphical User Interface (GUI) agents.

  awesomegraphical-user-interfaceai-assistant+ 2llm-agentgui-agents

  10•174•0•0•Updated Oct 27, 2024Oct 27, 2024

• watermark-steganalysis

  Public
  Python

  •0•0•0•0•Updated Oct 24, 2024Oct 24, 2024

• videogui

  Public
  [NeurIPS2024] VideoGUI: A Benchmark for GUI Automation from Instructional Videos

  guivideo-languagellm-agent

  JavaScript

  •0•21•0•0•Updated Oct 22, 2024Oct 22, 2024

• EvolveDirector

  Public
  [NeurIPS 2024] EvolveDirector: Approaching Advanced Text-to-Image Generation with Large Vision-Language Models.

  Python

  •0•40•0•0•Updated Oct 14, 2024Oct 14, 2024

• Awesome-MLLM-Hallucination

  Public
  📖 A curated list of resources dedicated to hallucination of multimodal large language models (MLLM).

  13•438•1•0•Updated Oct 10, 2024Oct 10, 2024

• MovieSeq

  Public
  [ECCV2024] Learning Video Context as Interleaved Multimodal Sequences

  Jupyter Notebook

  •1•28•1•0•Updated Oct 1, 2024Oct 1, 2024

• GUI-Narrator

  Public
  Repository of GUI Action Narrator

  JavaScript

  •0•3•0•0•Updated Sep 22, 2024Sep 22, 2024

• RingID

  Public
  Python

  •0•16•1•0•Updated Aug 30, 2024Aug 30, 2024

• MotionDirector

  Public
  [ECCV 2024 Oral] MotionDirector: Motion Customization of Text-to-Video Diffusion Models.

  video-generationdiffusion-modelstext-to-video+ 3text-to-motiontext-to-video-generationmotion-customization

  Python

  •

  Apache License 2.0

  •50•841•21•0•Updated Aug 21, 2024Aug 21, 2024

• videollm-online

  Public
  VideoLLM-online: Online Video Large Language Model for Streaming Video (CVPR 2024)

  Python

  •

  Apache License 2.0

  •27•221•16•0•Updated Aug 15, 2024Aug 15, 2024

• X-Adapter

  Public
  [CVPR 2024] X-Adapter: Adding Universal Compatibility of Plugins for Upgraded Diffusion Model

  Python

  •

  Apache License 2.0

  •43•740•17•4•Updated Aug 14, 2024Aug 14, 2024

• afformer

  Public
  Affordance Grounding from Demonstration Video to Target Image (CVPR 2023)

  deep-learningpytorch

  Python

  •2•38•6•0•Updated Jul 26, 2024Jul 26, 2024

• BoxDiff

  Public
  [ICCV 2023] BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion

  text-to-image-synthesisdiffusion-models

  Python

  •15•247•7•0•Updated Jul 21, 2024Jul 21, 2024

• cvpr2024-tutorial-video-diffusion-models

  Public
  HTML

  •

  MIT License

  •0•1•0•0•Updated Jul 16, 2024Jul 16, 2024

• DragAnything

  Public
  [ECCV 2024] DragAnything: Motion Control for Anything using Entity Representation

  Python

  •13•421•20•0•Updated Jul 2, 2024Jul 2, 2024

• AssistGaze

  Public
  Python

  •0•1•0•0•Updated Jun 25, 2024Jun 25, 2024

• cosmo

  Public
  Python

  •4•72•2•2•Updated May 10, 2024May 10, 2024

• EgoVLP

  Public
  [NeurIPS2022] Egocentric Video-Language Pretraining

  pretrainingvideo-languageegocentric-vision+ 1pytorch

  Python

  •20•226•5•0•Updated May 9, 2024May 9, 2024

• UniVTG

  Public
  [ICCV2023] UniVTG: Towards Unified Video-Language Temporal Grounding

  video-summarizationvideo-groundingpretraining+ 3moment-retrievalhighlight-detectionvideo-language

  Python

  •

  MIT License

  •28•320•19•0•Updated May 8, 2024May 8, 2024

• VisorGPT

  Public
  [NeurIPS 2023] Customize spatial layouts for conditional image synthesis models, e.g., ControlNet, using GPT

  image-generationgptdiffusion-models+ 1controlnet

  Python

  •

  MIT License

  •2•131•4•0•Updated May 4, 2024May 4, 2024

• Long-form-Video-Prior

  Public
  Python

  •0•23•0•0•Updated May 3, 2024May 3, 2024