Diffusers inference using TI

Before diving into how TI works internally, let’s take a look at how to use TI using Diffusers.

There are countless pretrained TIs shared in the Hugging Face’s Stable Diffusion concepts library [3] and CIVITAI [4]. For example, one of the most downloaded TIs from the Stable Diffusion concepts library is sd-concepts-library/midjourney-style [5]. We can start using it by simply referencing this name in the code; Diffusers will download the model data automatically:

1. Let’s initialize a Stable Diffusion pipeline:

   # initialize model

   from diffusers import StableDiffusionPipeline

   import torch

   model_id = "stablediffusionapi/deliberate-v2"

   pipe = StableDiffusionPipeline.from_pretrained(

       model_id,

       torch_dtype=torch.float16

   ).to("cuda")

2. Generate an image without TI involved:

   # without using TI

   prompt = "a high quality photo of a futuristic city in deep \ 

   space, midjourney...