Official implementation of Video Seal: training and inference code for video watermarking, and state-of-the-art open-sourced models.
This repository includes pre-trained models, training code, inference code, and evaluation tools, all released under the MIT license, as well as baselines of state-of-the-art image watermarking models adapted for video watermarking (including MBRS, CIN, TrustMark, and WAM) allowing for free use, modification, and distribution of the code and models.
[paper]
[arXiv]
[Colab]
[Demo]
| Original | Video Seal output | The watermark (normalized for visibility) |
|---|---|---|
 |
 |
 |
import torchvision.io
import videoseal
from videoseal.evals.metrics import bit_accuracy
# Load video and normalize to [0, 1]
video_path = "assets/videos/1.mp4"
video = torchvision.io.read_video(video_path, output_format="TCHW")
video = video.float() / 255.0
# Load the model
model = videoseal.load("videoseal")
# Video Watermarking
outputs = model.embed(video, is_video=True) # this will embed a random msg
video_w = outputs["imgs_w"] # the watermarked video
msgs = outputs["msgs"] # the embedded message
# Extract the watermark message
msg_extracted = model.extract_message(imgs_w, aggregation="avg", is_video=True)
# VideoSeal can do Image Watermarking
img = video[0:1] # 1 x C x H x W
outputs = model.embed(img, is_video=False)
img_w = outputs["imgs_w"] # the watermarked image
msg_extracted = model.extract_message(imgs_w, aggregation="avg", is_video=False)Version of Python is 3.10 (pytorch 2.5.1, torchvision 0.20.1, torchaudio 2.5.1, cuda 12.1). Install pytorch:
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
pip install -e .
The video model is automatically downloaded through Hugging Face by doing videoseal.load("videoseal").
If you wish to download the model manually, you can do so by running the following command:
wget https://dl.fbaipublicfiles.com/videoseal/checkpoint.pth(or click here). You then have to update the checkpoint path in the model card file.
We do not own any third-party models, so you have to download them manually. We provide a guide on how to download the models at docs/baselines.md.
We provide a guide on how to check and install VMAF at docs/vmaf.md.
To watermark both audio and video from a video file. It loads the full video in memory, so it is not suitable for long videos.
To watermark a video file in streaming. It loads the video clips by clips, so it is suitable for long videos, even on laptops.
Example:
python inference_streaming.py --input assets/videos/1.mp4 --output_dir outputs/Will output the watermarked video in outputs/1.mp4 and the binary message in outputs/1.txt.
To run full evaluation of models and baselines.
Example to evaluate a trained model:
python -m videoseal.evals.full \
--checkpoint /path/to/videoseal/checkpoint.pth \or, to run a given baseline:
python -m videoseal.evals.full \
--checkpoint baselines/wam \Example run of a simple run on a node of 8 GPUs:
torchrun --nproc_per_node=8 train.py --local_rank 0 --debug_slurm \
--balanced True --total_gnorm 1.0 --lambda_dec 1.0 --lambda_det 0.0 --lambda_d 0.1 --lambda_i 0.5 \
--scaling_w 1.0 --scaling_i 1.0 --nbits 96 --perceptual_loss mse --seed 444 \
--scheduler CosineLRScheduler,lr_min=1e-6,t_initial=1000,warmup_lr_init=1e-8,warmup_t=50 --optimizer AdamW,lr=1e-5 \
--epochs 1000 --eval_freq 5 --full_eval_freq 5 --batch_size_video_eval 1 --batch_size_video 1 --batch_size_eval 32 --batch_size 16 --iter_per_valid 10 \
--videoseal_step_size 16 --video_start 800 --prop_img_vid 0.5 --iter_per_epoch 1500 \
--extractor_model sam_small --embedder_model unet_small2 --augmentation_config configs/augs.yaml \
The model is licensed under an MIT license.
See contributing and the code of conduct.
If you find this repository useful, please consider giving a star ⭐ and please cite as:
@article{fernandez2024video,
title={Video Seal: Open and Efficient Video Watermarking},
author={Fernandez, Pierre and Elsahar, Hady and Yalniz, I. Zeki and Mourachko, Alexandre},
journal={arXiv preprint arXiv:2412.09492},
year={2024}
}