Building a RAG pipeline
We will now build a RAG pipeline by implementing the pipeline described in the previous section and illustrated in Figure 2.3. We will implement three components assuming that three teams (Team #1, Team #2, and Team #3) work in parallel to implement the pipeline:
- Data collection and preparation by
Team #1 - Data embedding and storage by
Team #2 - Augmented generation by
Team #3
The first step is to set up the environment for these components.
Setting up the environment
Let’s face it here and now. Installing cross-platform, cross-library packages with their dependencies can be quite challenging! It is important to take this complexity into account and be prepared to get the environment running correctly. Each package has dependencies that may have conflicting versions. Even if we adapt the versions, an application may not run as expected anymore. So, take your time to install the right versions of the packages and dependencies...
