This chapter is theoretical. It covers the logical components of intelligent applications and how they fit together.

This chapter assumes fundamental knowledge of traditional full stack application development components, such as servers, clients, databases, and APIs.

Traditional applications typically consist of a client-side user interface, a server-side backend, and a database for data storage and retrieval. They perform tasks following a strict set of instructions. Intelligent applications require a client, server, and database as well, but they augment the traditional stack with AI components.

Intelligent applications stand out by understanding complex, unstructured data to enable natural, adaptive interactions and decision-making. Intelligent applications can engage in open-ended interactions, generate novel content, and make autonomous decisions.

Examples of intelligent applications include the following:

• Chatbots that provide natural language responses based on external data using retrieval-augmented generation (RAG). For example, Perplexity.ai (https://www.perplexity.ai/) is an AI-powered search engine and chatbot that provides users with AI-generated answers to their queries based on sources...

LLMs are the key technology of intelligent applications, unlocking whole new classes of AI-powered systems. These models are trained on vast amounts of text data to understand language, generate human-like text, answer questions, and engage in dialogue.

LLMs undergo continuous improvement with the release of new models. featuring billions or trillions of parameters and enhanced reasoning, memory, and multi-modal capabilities.

Use cases for LLM reasoning engines

LLMs have emerged as a powerful general-purpose technology for AI systems, analogous to the central processing unit (CPU) in traditional computing. Much like CPUs, LLMs serve as general-purpose computational engines that can be programmed for many tasks and play a similar role in language-based reasoning and generation. The general-purpose nature of LLMs lets developers use their capabilities for a wide range of reasoning tasks.

A crop of techniques to leverage...

In addition to the reasoning capabilities provided by LLMs, intelligent applications require semantic long-term memory for storing and retrieving information.

Semantic memory typically consists of two core components—AI vector embedding models and vector databases. Vector embedding models represent the semantic meaning of unstructured data, such as text or images, in large arrays of numbers. Vector databases efficiently store and retrieve these vectors to support semantic search and context retrieval. These components work together to enable the reasoning engine to access relevant context and information as needed.

Embedding models

Embedding models are AI models that map text and other data types, such as images and audio, into high-dimensional vector representations. These vector representations capture the semantic meaning of the input data, allowing for efficient similarity comparisons and semantic...

With LLMs, embedding models, vector databases, and model hosting, you have the key building blocks for creating intelligent applications. While the specific architecture will vary depending on your use case, a common pattern emerges:

• LLMs for reasoning and generation
• Embeddings and vector search for retrieval and memory
• Model hosting to serve these components at scale

This AI stack is integrated with traditional application components, such as backend services, APIs, frontend user interfaces, databases, and data pipelines. Additionally, intelligent applications often include components for AI-specific concerns, such as prompt management and optimization, data preparation and embedding generation, and AI safety, testing, and monitoring.

The rest of this section walks through an example architecture for a RAG-powered chatbot, showcasing how these components work together. The subsequent chapters will dive deeper into the end...

Intelligent applications represent a new paradigm in software development, combining AI with traditional application components to deliver highly personalized, context-aware experiences. This chapter details the core components of intelligent applications, highlighting the pivotal role of LLMs as reasoning engines. LLMs serve as versatile computational tools capable of performing diverse tasks, including chat, summarization, and classification, due to their general-purpose design.

Complementing these reasoning engines are embedding models and vector databases, which function as the semantic memory of intelligent applications. These components enable the reasoning engine to retrieve pertinent context and information as needed. Additionally, the hosting of AI models demands dedicated infrastructure, as their unique hardware requirements differ significantly from traditional software needs. Using building blocks such as LLMs, embedding models, vector databases, and model hosting...