Video retargeting
Video synthesis is a broad term used for describing all forms of video generation. This can include generating video from random noise or words, to colorize black-and-white video, and so on, much like image generation.
In this section, we will look at a subgroup of video synthesis known as video retargeting. We will first look at two applications – face reenactment and pose transfer – and then introduce a powerful model that uses motion to generalize video targeting.
Face reenactment
Face reenactment was introduced along with face swapping in Chapter 9, Video Synthesis. Face reenactment in video synthesis involves transferring the facial expression of the driving video to the face in the target video. This is useful in animation and movie making. Recently, Zakharov et al. proposed a generative model that requires only a few target 2D images. This is done by using facial landmarks as intermediate features, as shown in the following diagram:
...
