Skip to content

This is a paper on deepfake generation and how to evaluate it.

Notifications You must be signed in to change notification settings

freak-jaeuk/Deepfake-generator

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
 
 

Repository files navigation

Deepfake-generator

The authors would like to thank Leo Hyun Park, Myung Gyo Oh and Jaewoo Park for helpful discussion.

Papers

Face Swapping

  • [StyleSwap] StyleSwap: Style-Based Generator Empowers Robust Face Swapping, ECCV 2022: Paper
  • [MegaFS] One Shot Face Swapping on Megapixels, CVPR 2021: Paper
  • [FaceShifter] FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping, CVPR 2020: Paper
  • MaskGAN: Towards Diverse and Interactive Facial Image Manipulation, CVPR 2020: Paper
  • Reference-guided Face Component Editing, IJCAI 2020: Paper
  • SimSwap: An Efficient Framework For High Fidelity Face Swapping, ACM MM 2020: Paper
  • FSGAN: Subject Agnostic Face Swapping and Reenactment, ICCV 2019: Paper

Face Reenactment

  • [Dual-Generator] Dual-Generator Face Reenactment, CVPR 2022: Paper
  • FSGAN: Subject Agnostic Face Swapping and Reenactment, ICCV 2019: Paper
  • Talking Face Generation by Conditional Recurrent Adversarial Network, IJCAI 2019: Paper arxiv
  • X2Face: A network for controlling face generation using images, audio, and pose codes, ECCV 2018: Paper
  • Semi-supervised Adversarial Learning to Generate Photorealistic Face Images of New Identities from 3D Morphable Model, ECCV 2018: Paper

Eentire Face Synthesis

  • TediGAN: Text-Guided Diverse Face Image Generation and Manipulation, CVPR 2021: Paper
  • Diagonal Attention and Style-based GAN for Content-Style Disentanglement in Image Generation and Translation, ICCV 2021: Paper
  • Dual Projection Generative Adversarial Networks for Conditional Image Generation, ICCV 2021: Paper
  • Unsupervised Image Generation with Infinite Generative Adversarial Networks, ICCV 2021: Paper
  • [StyleGAN3] Alias-Free Generative Adversarial Networks, NeurIPS 2021: Paper
  • CcGAN: Continuous Conditional Generative Adversarial Networks for Image Generation, ICLR 2021: Paper
  • [StyleGAN2] Analyzing and Improving the Image Quality of StyleGAN, CVPR 2020: Paper
  • From Inference to Generation: End-to-end Fully Self-supervised Generation of Human Face from Speech, ICLR 2020: Paper
  • A Style-Based Generator Architecture for Generative Adversarial Networks, CVPR 2019: Paper
  • Progressive Growing of GANs for Improved Quality, Stability, and Variation, ICLR 2018: Paper

Attribute Manipulation

  • A Style-aware Discriminator for Controllable Image Translation, CVPR 2022: Paper
  • [HIfaFACE] High-Fidelity and Arbitrary Face Editing, CVPR 2021: Paper
  • A Latent Transformer for Disentangled Face Editing in Images and Videos, ICCV 2021: Paper
  • [LoFGAN] LoFGAN: Fusing Local Representations for Few-shot Image Generation, ICCV 2021: Paper
  • [StarGAN2] StarGAN v2: Diverse Image Synthesis for Multiple Domains, CVPR 2020: Paper
  • [DAAE] Hierarchical Face Aging through Disentangled Latent Characteristics, ECCV 2020: Paper
  • [CONFIG] CONFIG: Controllable Neural Face Image Generation, ECCV 2020: Paper
  • [CAFE-GAN] CAFE-GAN: Arbitrary Face Attribute Editing with Complementary Attention Feature, ECCV 2020: Paper
  • High Resolution Zero-Shot Domain Adaptation of Synthetically Rendered Face Images, ECCV 2020: Paper
  • [GDFace] GDFace: Gated Deformation for Multi-view Face Image Synthesis, AAAI 2020: Paper
  • [DeepFacePencil] DeepFacePencil: Creating Face Images from Freehand Sketches, ACM MM 2020: Paper
  • [ETGAN] Talking Face Generation with Expression-Tailored Generative Adversarial Network, ACM MM 2020: Paper
  • [InterFaceGAN] InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs, TPAMI 2020: Paper
  • Semantic Component Decomposition for Face Attribute Manipulation, CVPR 2019: Paper
  • Make a Face: Towards Arbitrary High Fidelity Face Manipulation, ICCV 2019: Paper
  • SC-FEGAN: Face Editing Generative Adversarial Network with User’s Sketch and Color, ICCV 2019: Paper
  • Towards Automatic Face-to-Face Translation, ACM MM 2019: Paper
  • LinesToFacePhoto: Face Photo Generation From Lines With Conditional Self-Attention Generative Adversarial Networks, ACM MM 2019: Paper
  • StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation, CVPR 2018: Paper
  • Generative Adversarial Network with Spatial Attention for Face Attribute Editing, ECCV 2018: Paper
  • ELEGANT: Exchanging Latent Encodings with GAN for Transferring Multiple Face Attributes, ECCV 2018: Paper
  • A Hybrid Model for Identity Obfuscation by Face Replacement, ECCV 2018: Paper

Evaluation

Face Swapping

  • ID retrieval
    • CosFace: Large Margin Cosine Loss for Deep Face Recognition, CVPR 2018
    • One Shot Face Swapping on Megapixels, CVPR 2021
  • Pose error
    • Fine-Grained Head Pose Estimation Without Keypoints, CVPR 2018
    • One Shot Face Swapping on Megapixels, CVPR 2021
  • Expression error
    • A Compact Embedding for Facial Expression Similarity, CVPR 2019
    • One Shot Face Swapping on Megapixels, CVPR 2021
  • Frechét inception distance(FID)
    • FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping, CVPR 2020
    • One Shot Face Swapping on Megapixels, CVPR 2021

Face Reenactment

  • Structural similarity (SSIM)
    • Image Quality Assessment: From Error Visibility to Structural Similarity, IEEE TIP 2004
    • **Dual-Generator Face Reenactment, CVPR 2022
    • FSGAN: Subject Agnostic Face Swapping and Reenactment, ICCV 2019
    • Talking Face Generation by Conditional Recurrent Adversarial Network, IJCAI 2019
  • Frechét inception distance(FID)
    • Dual-Generator Face Reenactment, CVPR 2022
  • Cosine Similarity (CSIM)
    • Dual-Generator Face Reenactment, CVPR 2022
  • Face Verification
    • Dlib-ml: A machine learning toolkit, JML 2009
    • FSGAN: Subject Agnostic Face Swapping and Reenactment, ICCV 2019
  • Euclidean distance
    • Euclidean distance mapping, Computer Graphics and image processing 1980
  • Pose error
    • FSGAN: Subject Agnostic Face Swapping and Reenactment, ICCV 2019
  • Peak Signal-to-Noise Ratios (PSNR)
    • Talking Face Generation by Conditional Recurrent Adversarial Network, IJCAI 2019
  • Landmark Distance Error
    • Lip movements generation at a glance, ECCV 2018
    • Talking Face Generation by Conditional Recurrent Adversarial Network, IJCAI 2019

Entire Face Synthesis

  • Frechét inception distance(FID)
    • GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium, NeurIPS 2017
    • TediGAN: Text-Guided Diverse Face Image Generation and Manipulation, CVPR 2021: Paper
    • Diagonal Attention and Style-based GAN for Content-Style Disentanglement in Image Generation and Translation, ICCV 2021: Paper
    • Dual Projection Generative Adversarial Networks for Conditional Image Generation, ICCV 2021: Paper
    • Unsupervised Image Generation with Infinite Generative Adversarial Networks, ICCV 2021: Paper
    • Alias-Free Generative Adversarial Networks, NeurIPS 2021: Paper
    • CcGAN: Continuous Conditional Generative Adversarial Networks for Image Generation, ICLR 2021: Paper
    • Analyzing and Improving the Image Quality of StyleGAN, CVPR 2020: Paper
    • A Style-Based Generator Architecture for Generative Adversarial Networks, CVPR 2019: Paper
  • Learned perceptual Image Patch Similarity (LPIPS)
    • The Unreasonable Effectiveness of Deep Features as a Perceptual Metric, CVPR 2018: Paper
    • TediGAN: Text-Guided Diverse Face Image Generation and Manipulation, CVPR 2021: Paper
    • Diagonal Attention and Style-based GAN for Content-Style Disentanglement in Image Generation and Translation, ICCV 2021: Paper
    • Dual Projection Generative Adversarial Networks for Conditional Image Generation, ICCV 2021: Paper
  • Accuracy (text2image)
    • TediGAN: Text-Guided Diverse Face Image Generation and Manipulation, CVPR 2021: Paper
  • Perceptual Path Length (PPL)
    • A Style-Based Generator Architecture for Generative Adversarial Networks, CVPR 2019: Paper
    • Diagonal Attention and Style-based GAN for Content-Style Disentanglement in Image Generation and Translation, ICCV 2021: Paper
    • Analyzing and Improving the Image Quality of StyleGAN, CVPR 2020: Paper
    • A Style-Based Generator Architecture for Generative Adversarial Networks, CVPR 2019: Paper
  • Inception Score
    • Improved techniques for training gans, NeurIPS 2016
    • Dual Projection Generative Adversarial Networks for Conditional Image Generation, ICCV 2021
  • Cluster Purity
    • Introduction to Information Retrieval, Cambridge University Press 2008
    • Unsupervised Image Generation with Infinite Generative Adversarial Networks, ICCV 2021
  • EQ-T, EQ-R
    • Alias-Free Generative Adversarial Networks, NeurIPS 2021
  • Naturalness Image Quality Evaluator (NIQE)
    • Making a “completely blind” image quality analyzer, Signal processing letter 2012
    • CcGAN: Continuous Conditional Generative Adversarial Networks for Image Generation, ICLR 2021
  • Diversity
    • CcGAN: Continuous Conditional Generative Adversarial Networks for Image Generation, ICLR 2021
  • Label Score
    • CcGAN: Continuous Conditional Generative Adversarial Networks for Image Generation, ICLR 2021
  • Sliced Wasserstein distance
    • Sliced Wasserstein distance: Wasserstein barycenter and its application to texture mixing, SSVM 2011
    • Progressive Growing of GANs for Improved Quality, Stability, and Variation, ICLR 2018
  • Structural similarity (SSIM)
    • Structural similarity (SSIM): Multiscale structural similarity for image quality assessment, Systems & Computers 2003
    • Progressive Growing of GANs for Improved Quality, Stability, and Variation, ICLR 2018
  • Linear separability
    • A Style-Based Generator Architecture for Generative Adversarial Networks, CVPR 2019

Attribute Mapulation

  • Frechét inception distance(FID)
    • GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium, NeurIPS 2017: Paper
    • (StyleGAN) A Style-Based Generator Architecture for Generative Adversarial Networks, CVPR 2019: Paper
    • (StarGAN) Stargan v2: Diverse image synthesis for multiple domains, CVPR 2020**: Paper**
    • (HifaFACE) High-Fidelity and Arbitrary Face Editing, CVPR 2021: Paper
    • (LoFGAN) LoFGAN: Fusing Local Representations for Few-shot Image Generation, ICCV 2021: Paper
    • (CONFIG) CONFIG: Controllable Neural Face Image Generation, ECCV 2020: Paper
    • (CAFE-GAN) CAFE-GAN: Arbitrary Face Attribute Editing with Complementary Attention Feature, ECCV 2020: Paper
    • High Resolution Zero-Shot Domain Adaptation of Synthetically Rendered Face Images, ECCV 2020: Paper
    • (DeepFacePencil) DeepFacePencil: Creating Face Images from Freehand Sketches, ACM MM 2020: Paper
  • Learned Perceptual Image Patch Similarity(LPIPS)
    • The Unreasonable Effectiveness of Deep Features as a Perceptual Metric, CVPR 2018: Paper
    • (StarGAN) Stargan v2: Diverse image synthesis for multiple domains, CVPR 2020**: Paper**
    • (LoFGAN) LoFGAN: Fusing Local Representations for Few-shot Image Generation, ICCV 2021: Paper
  • Quality Score(QS)
    • GIQA: Generated Image Quality Assessment, ECCV 2020: Paper
    • (HifaFACE) High-Fidelity and Arbitrary Face Editing, CVPR 2021: Paper
  • Self Reconstruction Error(SRE)
    • High-Fidelity and Arbitrary Face Editing, CVPR 2021: Paper
  • kernel Inception Distance(KID)
    • Demystifying mmd gans, ICLR 2018: Paper
    • A Style-aware Discriminator for Controllable Image Translation, CVPR 2022: Paper
  • AUC
    • The use of the area under the ROC curve in the evaluation of machine learning algorithms, Pattern recognition 1997: Paper
    • (DAAE) Hierarchical Face Aging through Disentangled Latent Characteristics, ECCV 2020: Paper
    • (GDFace) GDFace: Gated Deformation for Multi-view Face Image Synthesis, AAAI 2020: Paper
  • Attribute class Accuracy
    • CAFE-GAN: Arbitrary Face Attribute Editing with Complementary Attention Feature, ECCV 2020: Paper
  • Face verification Accuracy
    • GDFace: Gated Deformation for Multi-view Face Image Synthesis, AAAI 2020: Paper
  • Inception Score
    • Improved techniques for training gans, NeurIPS 2016: Paper
    • High Resolution Zero-Shot Domain Adaptation of Synthetically Rendered Face Images, ECCV 2020: Paper
    • (DeepFacePencil) DeepFacePencil: Creating Face Images from Freehand Sketches, ACM MM 2020: Paper
  • Peak Signal-to-Noise Ratios (PSNR)
    • (ETGAN) Talking Face Generation with Expression-Tailored Generative Adversarial Network, ACM MM 2020: Paper
  • Average Content Distance
    • MoCoGAN: Decomposing Motion and Content for Video Generation, CVPR 2018: Paper
    • (ETGAN) Talking Face Generation with Expression-Tailored Generative Adversarial Network, ACM MM 2020: Paper

..

About

This is a paper on deepfake generation and how to evaluate it.

Topics

Resources

Stars

Watchers

Forks