Building a custom TI loader

In this section, we are going to build a TI loader by implementing the preceding understanding into code and giving a TI weight parameter for the loader function.

Before writing the function code, let’s first take a look at how a TI looks internally. Before running the following code, you will need to first download the TI file to your storage.

TI in the pt file format

Load a TI in the pt file format:

# load a pt TI
import torch
loaded_learned_embeds = torch.load("badhandsv5-neg.pt",
    map_location="cpu")
keys = list(loaded_learned_embeds.keys())
for key in keys:
    print(key,":",loaded_learned_embeds[key])

We can clearly see the key and paired value from the TI file:

string_to_token : {'*': 265}
string_to_param : {'*': tensor([[ 0.0399, -0.2473,  0.1252,  ...,  0.0455,  0.0845, -0.1463],
  ...