Retrieval-Augmented Generation and AI Agents

What Is Retrieval-Augmented Generation (RAG)?

Retrieval-Augmented Generation, or RAG, combines information retrieval with generative AI. Instead of relying only on the model's learned knowledge, a RAG system retrieves relevant information from an external source and supplies it as context for generating a response.

A typical RAG workflow is:

1. Collect documents from a trusted source.
2. Split documents into manageable chunks.
3. Convert chunks into searchable representations, often embeddings.
4. Retrieve relevant chunks for a user's question.
5. Provide those chunks to the language model.
6. Generate an answer grounded in the retrieved information.

For example, a company could build a support assistant that answers questions using internal product documentation.

RAG helps incorporate updated or private knowledge without retraining the entire language model. However, retrieval quality, document permissions, and source accuracy remain important.

What Are AI Agents?

An AI agent is a system that uses a model to decide what to do next in pursuit of a goal. Depending on its design, it may use tools, maintain state, observe results, and perform multiple steps.

For example, a travel-planning agent might retrieve destination information, compare options, and prepare an itinerary.

A basic LLM application usually responds to a prompt. An agent can use tools and take additional actions as part of a larger workflow.

RAG vs AI Agents

RAG and agents can be combined. An agent might retrieve documents through RAG, use a calculator, and then generate a response.

When agents can modify data, send messages, or perform external actions, they should use restricted permissions, confirmation for consequential actions, and logging.