Introduction to pre-trained models and ZSL

Building on the foundations of GenAI discussed in the chapter so far, we will now introduce some core concepts related to pre-trained models and zero-shot learning (ZSL). These concepts underly how models can take vast amounts of existing data to create realistic, new outputs for scenarios that have not yet been encountered, with little to no additional training. With a focus on text data, we will discuss how contextual embeddings and semantic proximity are two key concepts that facilitate this capability. With this knowledge, you will be equipped to understand and apply these concepts in this chapter and the ones to come.

Contextual embeddings

Contextual embeddings, enabled by advancements such as the LSTM and GPT models discussed earlier, are fundamental to how large language models (LLMs) interpret and generate language. As discussed in Chapter 5, embeddings are dense vector representations of data that capture key features in...