Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022;130(7):1678-1734.
doi: 10.1007/s11263-022-01606-8. Epub 2022 May 4.

Countering Malicious DeepFakes: Survey, Battleground, and Horizon

Felix Juefei-Xu  1 , Run Wang  2 , Yihao Huang  3 , Qing Guo  4   5 , Lei Ma  6 , Yang Liu  5   7
Affiliations

Countering Malicious DeepFakes: Survey, Battleground, and Horizon

Felix Juefei-Xu et al. Int J Comput Vis. 2022.

Abstract

The creation or manipulation of facial appearance through deep generative approaches, known as DeepFake, have achieved significant progress and promoted a wide range of benign and malicious applications, e.g., visual effect assistance in movie and misinformation generation by faking famous persons. The evil side of this new technique poses another popular study, i.e., DeepFake detection aiming to identify the fake faces from the real ones. With the rapid development of the DeepFake-related studies in the community, both sides (i.e., DeepFake generation and detection) have formed the relationship of battleground, pushing the improvements of each other and inspiring new directions, e.g., the evasion of DeepFake detection. Nevertheless, the overview of such battleground and the new direction is unclear and neglected by recent surveys due to the rapid increase of related publications, limiting the in-depth understanding of the tendency and future works. To fill this gap, in this paper, we provide a comprehensive overview and detailed analysis of the research work on the topic of DeepFake generation, DeepFake detection as well as evasion of DeepFake detection, with more than 318 research papers carefully surveyed. We present the taxonomy of various DeepFake generation methods and the categorization of various DeepFake detection methods, and more importantly, we showcase the battleground between the two parties with detailed interactions between the adversaries (DeepFake generation) and the defenders (DeepFake detection). The battleground allows fresh perspective into the latest landscape of the DeepFake research and can provide valuable analysis towards the research challenges and opportunities as well as research trends and future directions. We also elaborately design interactive diagrams (http://www.xujuefei.com/dfsurvey) to allow researchers to explore their own interests on popular DeepFake generators or detectors.

Keywords: DeepFake Detection; DeepFake Generation; DeepFakes; Disinformation; Face; Misinformation.

PubMed Disclaimer

Figures

Fig. 1
Fig. 1
From top to bottom, the four panels illustrate the four categories of DeepFakes. Four examples are shown for both ‘identity swap’ and ‘expression swap’, with each example associated with a target, real, and DeepFake sequences of 5 frames. Within each panel, the two examples in the top row show the DeepFake manipulation that is pretty subtle, which demonstrate the minuscule manipulations that some DeepFakes can present, and the two examples in the bottom row show more drastic DeepFake manipulations. Across ‘identity swap’ and ‘expression swap’, as a comparison, one example is shown in both scenarios and is highlighted by formula image, to showcase the difference in the DeepFake frames for these two modalities coming from the same ‘target’ and ‘real’ sources. Readers are encouraged to zoom in on the image. Actual full-resolution videos are available on the project website (http://www.xujuefei.com/dfsurvey) to better illustrate the DeepFake phenomenon. For ‘attribute manipulation’ and ‘entire face synthesis’ on the bottom panel, both real and DeepFakes are shown. In terms of popularity as being attempted by DeepFake detectors according to the survey results, the ranking is as the following: identity swap > entire face synthesis > attribute manipulation ≫ expression swap
Fig. 2
Fig. 2
This figure shows a growing trend in the number of papers in the DeepFake field in recent years. The papers are collected according to the criteria introduced in Sect. 2.1, including arXiv, conference, and journal articles. They are categorized by the year of the last updated version. (L) The number of published DeepFake related papers year over year since its inception in 2016. (R) The cumulative number of published DeepFake related papers year over year since its inception in 2016. It shows that over 78% papers were published in the last 2 years. The bars with shadow are the projected number of published papers
Fig. 3
Fig. 3
Tree diagram showing the paper structure
Fig. 4
Fig. 4
a For the papers in DeepFake research area, we can find that a large amount of papers are from Others and arXiv. The papers published in top conferences and journals only account for a third of the total. Furthermore, a lot of top papers are published in CVPR, which accounts for about half the published top papers. b For DeepFake generation methods, we can find that a large amount of the papers are from CVPR and arXiv. Two thirds of the generation papers are published in top conferences and journals. c For DeepFake detection methods, we can find that a large amount of the papers are from Others and CVPR. The published top papers only make up a small part of the total. This suggests that progress in DeepFake detection is not enough. d For DeepFake evasion methods, we can find that the volume of articles is not large and a large amount of the papers are from arXiv. One fourth of the papers are published in top conferences
Fig. 5
Fig. 5
The evolution of DeepFake generation techniques with a fishbone diagram for each DeepFake generation type
Fig. 6
Fig. 6
The difference between real and fake from the spatial domain, especially the discrepancies across the blending the boundary (Li et al. 2020c)
Fig. 7
Fig. 7
The difference between real and fake from the frequency domain, especially noticing the difference in their spectra (Zhang et al. 2019)
Fig. 8
Fig. 8
The difference between real and fake from the biological signal domain, especially the colorful motion-magnified spatial-temporal (MMST) maps between them (Qi et al. 2020)
Fig. 9
Fig. 9
(L) Summary of various types of DeepFake detection methods, including the type ID and the name of each type. (R) The proportion of various types of DeepFake detection methods in our collected DeepFake detection papers
Fig. 10
Fig. 10
The evolution of DeepFake detection techniques with a fishbone diagram. In the main fishbone, the weakness and strengths of each detection method are presented as well. For each DeepFake detection method in the sub-fishbone diagram, the milestone studies are added for presenting the significant progress, especially their novelty on technical, the problem addressed, and new insight for defending DeepFakes
Fig. 11
Fig. 11
Battleground diagram between DeepFake generation and detection. The Sankey diagram shows the interactions between various DeepFake detection methods (right column) and various DeepFake generation methods (left column). Both of the generation and detection methods are sorted by the release time and labeled with the corresponding years (same as the order in Tables 6, 7). Four colors represent the different types of detection methods introduced in Tables 7: Blue is Type-I (spatial based) methods, green is Type-II (frequency based) methods, yellow is Type-III (biological signal based) methods, and red is Type-IV (others) methods. Interactive diagram is available at http://www.xujuefei.com/dfsurvey
Fig. 12
Fig. 12
(L) Top-9 most popular DeepFake generation methods or datasets based on the battleground. (R) 2020’s Top-11 most popular DeepFake generation methods or datasets based on the battleground
Fig. 13
Fig. 13
Relation pairs of the image- and video-based DeepFake generation methods that are simultaneously evaluated by some DeepFake detection methods. Interactive diagram is available at http://www.xujuefei.com/dfsurvey
Fig. 14
Fig. 14
A chord diagram represents the comparison among the existing detection methods. The node indicates the method for DeepFake detection and the link represents that one of the work is served as the baseline in the evaluation. The baselines include typical CNN models and the works with/without the peer review. An interactive diagram is available at http://www.xujuefei.com/dfsurvey
Fig. 15
Fig. 15
(L) Top-11 most popular DeepFake detection methods chosen as baselines. (R) Top-10 most popular ML-based methods chosen as baselines
Fig. 16
Fig. 16
(L) Top-9 DeepFake detection methods that benchmark against the most number of baselines. (R) Top-8 DeepFake detection methods that benchmark against the most number of baselines in 2020
Fig. 17
Fig. 17
(L) Top-10 DeepFake generation methods or datasets based on citations. (R) Top-10 DeepFake generation methods or datasets based on normalized citations
Fig. 18
Fig. 18
(L) Top-10 DeepFake detection methods based on citations. (R) Top-10 DeepFake detection methods based on normalized citations
Fig. 19
Fig. 19
Top-10 DeepFake generation methods or datasets based on Elo rating (Wikipedia 2021a). Default score is 1400
Fig. 20
Fig. 20
Evasion of DeepFake detection via shallow reconstruction (Huang et al. 2020b)
See this image and copyright information in PMC

References

    1. 115th Congress. (2018). S.3805—Malicious Deep Fake Prohibition Act of 2018. https://www.congress.gov/bill/115th-congress/senate-bill/3805
    1. 116th Congress. (2019a). H.R.3230—Defending Each and Every Person from False Appearances by Keeping Exploitation Subject to Accountability Act of 2019. https://www.congress.gov/bill/116th-congress/house-bill/3230/
    1. 116th Congress. (2019b). S.2065—Deepfake Report Act of 2019. https://www.congress.gov/bill/116th-congress/senate-bill/2065
    1. Abiantun, R., Juefei-Xu, F., Prabhu, U., & Savvides, M. (2019). SSR2: Sparse signal recovery for single-image super-resolution on faces with extreme low resolutions. Pattern Recognition, 90, 308–324.
    1. Adobe. (2021a). Adobe audition. Retrieved August 1, 2021, from https://www.adobe.com/products/audition.html (online).

LinkOut - more resources