Understanding RAG and AI orchestration
The process of training an LLM involves taking a massive amount of training data and building a mapping of how different tokens in the source data are associated with each other.
Because the training process uses the process of self-attention, the resulting models are able to have a larger context of the relationship between different words in sentences and even different sentences in a paragraph.
The model training process requires large scales of both data and computing resources – typically many graphical processing units (GPUs) – and a significant amount of time. This process is illustrated in Figure 12.1:
Figure 12.1 – Training an LLM on source data
As we’ve seen, these trained LLMs are very powerful, but they’re limited to the data used in the training process.
This means that a model trained on news would not include any news stories published after the model began...
