Utilizing Motion LoRA to control animation motion

Besides the motion adapter model, the author of the paper also introduced Motion LoRA to control the motion style. Motion LoRA is the same LoRA adapter we introduced in Chapter 8. As mentioned before, the AnimateDiff pipeline supports all other community-shared LoRAs. You can find these Motion LoRAs on the author’s Hugging Face repository [8].

These Motion LoRAs can be used to control the camera view. Here, we will use zoom-in LoRA – guoyww/animatediff-motion-lora-zoom-in – as an example. The zoom-in will guide the model to generate a video with zoom-in motion.

The usage is simply one additional line of code:

pipe.load_lora_weights("guoyww/animatediff-motion-lora-zoom-in", 
    adapter_name="zoom-in")

Here is the complete generation code. We are mostly reusing the code from the previous section:

import torch
from diffusers.utils import export_to_gif, export_to_video...