What this book covers

Chapter 1, Understanding Large Language Models, serves as an introduction to generative AI and LLMs. It explains what LLMs are, their role in modern technology, and their strengths and weaknesses. The chapter aims to provide you with a foundational understanding of the capabilities of LLMs that LlamaIndex builds upon.

Chapter 2, LlamaIndex: The Hidden Jewel - An Introduction to the LlamaIndex Ecosystem, introduces the LlamaIndex ecosystem and how it can augment LLMs. It explains the general structure of the book – starting with basic concepts and gradually introducing more complex elements of the LlamaIndex framework. The chapter also introduces the PITS – Personalized Intelligent Tutoring System project, which will be used to apply the concepts studied in the book and covers the preparation of the development environment.

Chapter 3, Kickstarting Your Journey with LlamaIndex, covers the basics of starting your first LlamaIndex project. It explains the essential components of a RAG application in LlamaIndex, such as documents, nodes, indexes, and query engines. The chapter provides a typical workflow model and a simple hands-on example, where readers will begin building the PITS project.

Chapter 4, Ingesting Data into Our RAG Workflow, focuses on importing our proprietary data into LlamaIndex, emphasizing the usage of the LlamaHub connectors. We learn how to break down and organize documents by parsing them into coherent, indexable chunks of information. The chapter also covers ingestion pipelines, important data privacy considerations, metadata extraction, and simple cost estimation methods.

Chapter 5, Indexing with LlamaIndex, explores the topic of data indexing. It provides an overview of how indexing works, comparing different indexing techniques to help readers choose the most suitable one for their use cases. The chapter also explains the concept of layered indexing and covers persistent index storage and retrieval, cost estimation, embeddings, vector stores, similarity search, and storage contexts.

Chapter 6, Querying Our Data, Part 1 – Context Retrieval, explains the mechanics of querying data and various querying strategies and architectures within LlamaIndex, with a deep focus on retrievers. It covers advanced concepts such as asynchronous retrieval, metadata filters, tools, selectors, retriever routers, and query transformations. The chapter also discusses fundamental paradigms such as dense retrieval and sparse retrieval, along with their strengths and weaknesses.

Chapter 7, Querying Our Data, Part 2 – Postprocessing and Response Synthesis, continues the query mechanics topic, explaining the role of node post-processing and response synthesizers in the RAG workflow. It presents the overall query engine construct and its usage, as well as output parsing. The hands-on part of this chapter focuses on using LlamaIndex to generate personalized content in the PITS application.

Chapter 8, Building Chatbots and Agents with LlamaIndex, introduces the essentials of chatbots, agents, and conversation tracking with LlamaIndex, applying this knowledge to the hands-on project. You will learn how LlamaIndex facilitates fluid interaction, retains context, and manages custom retrieval/response strategies, which are essential aspects for building effective conversational interfaces.

Chapter 9, Customizing and Deploying Our LlamaIndex Project, provides a comprehensive guide to personalizing and launching LlamaIndex projects. It covers tailoring different components of the RAG pipeline, a beginner-friendly tutorial on deploying with Streamlit, advanced tracing methods for debugging, and techniques for evaluating and fine-tuning a LlamaIndex application.

Chapter 10, Prompt Engineering Guidelines and Best Practices, explains the essential role of prompt engineering in enhancing the effectiveness of a RAG pipeline, highlighting how prompts are used “under the hood” of the LlamaIndex framework. It guides readers on the nuances of customizing and optimizing prompts to harness the full power of LlamaIndex and ensure more reliable and tailored AI outputs.

Chapter 11, Conclusion and Additional Resources, serves as a comprehensive conclusion, highlighting other projects and pathways for extended learning and summarizing the core insights from the book. It offers an overview of the main features of the framework, provides a curated list of additional resources for further exploration, and includes an index for quick terminology reference.