What Is RAG (Retrieval Augmented Generation)?

An AI model is only as good as what it’s taught. For it to thrive, it needs the proper context and large amounts of factual data. An off-the-shelf LLM is not always up-to-date, nor will it have trustworthy access to your data or understand the ins and outs of your business.

That’s where RAG-powered AI can help.

RAG is a method where AI pulls information from your own data, such as PDFs, knowledge bases, or chat logs, and uses it to answer questions/prompts. Instead of relying only on what the model was trained on, it searches your content in real time, then gives accurate answers to your team members' prompts.

• How retrieval augmented generation works
• What are the benefits of RAG?
• Real-life applications of RAG
• Types of RAG models
• How RAG supports different teams
• How Salesforce uses RAG AI
• The ROI of RAG

In a nutshell, RAG helps companies retrieve and use their data from various internal sources for better generative AI results. Since the source material comes from your own trusted data, it helps reduce hallucinations and other incorrect outputs.

To achieve this improved accuracy, RAG works in conjunction with a specialised type of database (called a vector database) to store data in a numeric format that makes sense for AI and retrieve it when prompted.

At a high level, RAG works like this:

• Retrieval: Companies can retrieve their most relevant data for use in their prompts to steer LLMs towards useful and trusted outputs.
• Augmented: With RAG, the LLM doesn’t need to be trained on your data in advance. Instead, the most relevant information is added to the prompt in real time, so the model can use it to generate a more up-to-date response.
• Generation: The LLM uses that context to produce a response, like a customer service reply or outreach email, based on current and reliable information.

Retrieval-augmented generation is a cost-effective way to improve how your AI performs across employee and customer experiences. Here are some of the key benefits for businesses:

• Always up to date: RAG pulls in your latest data every time you ask a question, meaning it won’t use outdated information.
• Hallucination reduction: Adding search results to a prompt reduces hallucinations in generative AI responses without the complexity of fine-tuning or retraining an LLM.
• Clear sources: With RAG, generative AI responses can cite the source of their data, allowing your team to quickly see where the information was gathered from.
• Relevant searches: RAG allows companies to retrieve and embed both structured and unstructured data directly into their LLM prompts.
• Risk mitigation: Your data stays separate from the AI model. With Salesforce, it runs through the Einstein Trust Layer, so users only see what they’re allowed to.
• More control: Off-the-shelf LLMs are typically trained on an opaque and far-ranging set of data sources. With RAG, you choose the data the AI uses.

Companies across industries are already using RAG to solve practical problems. The examples below show how teams are applying it to improve customer service, speed up workflows, and make better use of their internal data.

Australian travel company TripADeal introduced a virtual travel consultant to help customers find and plan their next holiday. Using RAG, the agent searches their site and knowledge base to answer questions quickly and accurately. It also uses Prompt Templates and Flows to surface the right holiday packages based on customer preferences.

Built in just five weeks, the agent now helps customers navigate the website, get instant answers, and receive personalised deal recommendations.

Canada-based communications company, Algo, turned to generative AI to speed up customer service onboarding during a period of rapid growth. They found that off-the-shelf LLMs lacked the context they needed, so the team adopted retrieval augmented generation (RAG), loading in chat logs and email history to support their customer support representatives.

After testing on 10% of their product base, their CSRs grew confident using the tool, and within two months, case resolution sped up by 67%. RAG now supports 60% of their products, their onboarding time has halved, and CSRs can spend more time adding a human touch to every interaction.

Australian property developer Geocon introduced a virtual agent to help customers log building defects and get support 24/7. Using RAG, the agent searches account records, cases, and knowledge articles to verify property details and determine whether an issue falls within the warranty period.

Built on Experience Cloud, the agent has replaced manual inboxes with a self-service portal and can now route cases instantly to subcontractors. As a result, tasks that once took an average of 6.5 hours now happen in seconds.

There are a few different ways to set up retrieval-augmented generation (RAG), depending on how your data is stored and how you want the AI to access it. Here are the three most common approaches.

1. Vector-based: Your data is stored in a vector database, which converts information (like documents or emails) into a numeric format that AI can understand and search through efficiently.
2. Knowledge graph: Data is structured more like a network, with entities (people, places, products) connected by relationships. This helps the AI make more human-like connections and answer more complex questions based on how things relate to each other.
3. Ensemble: Instead of relying on one retriever, multiple systems are used at once. Their results are then combined or cross-checked, which helps improve accuracy and fill in gaps one model might miss.

RAG helps you connect AI with your own data and systems, meaning that different teams can get useful, task-specific support. Instead of using generic chatbots, you can set up AI agents that answer questions, complete tasks, and pull in current information from your CRM or any other integrated database.

So what does this look like in action? Here are some examples of how you could use RAG across your business.

An AI service agent enables your support staff to provide customer service that's faster and more accurate. Using RAG, an AI agent can give personalised responses, anticipate needs, and manage service across multiple channels.

With an AI sales agent integrated into your CRM, you can use RAG throughout your sales pipeline to nurture leads. It can tackle everything from answering key questions in the buying process for inbound leads to providing coaching to upskill sales reps.

A platform that has a marketing AI agent can deepen customer engagement and boost productivity. Marketers can generate personalised content and offers across the entire customer lifecycle, and optimise autonomously based on real-time results.

A commerce AI agent built into your commerce platform can increase sales by recommending products, answering questions, and making the checkout process easier. It can also help manage inventory and write product descriptions in the background.

RAG is a broadly used concept, but at Salesforce, it works in a specific and powerful way to make AI more trusted and relevant.

• Consolidate data: RAG starts with Data Cloud, which brings together structured and unstructured data in one place. This includes PDFs, call transcripts, emails, and any other content that AI can use to give better answers.
• Increase sales: In Sales Cloud, RAG gives reps access to insights from past emails, meetings, and documents to improve outreach and follow-ups.
• Speed-up service: In Service Cloud, RAG powers suggested replies based on knowledge articles, past cases, emails, and conversation history. Building a service chatbot can also help service teams respond faster and more accurately.
• Chat support: Agentforce combines data, AI, automation, and human input to deploy useful AI agents. These agents use RAG and large language models to access business data, reason through tasks, and take action when needed.