Retriever Builder

Salesforce

June 2023 - Current (Generally Available)

Design Systems, User Research, Experience Design, Information Architecture, Design Strategy

Retrievers function as personalized research assistants that query unstructured data, enhancing agent accuracy and reducing hallucinations through Retrieval-Augmented Generation (RAG): RAG in Salesforce

I researched, collaborated with product managers and engineers, and designed the building experience for custom retrievers from concept to launch.

To comply with my non-disclosure agreement, I have omitted and obfuscated confidential information in this case study. All information in this case study is my own and does not necessarily reflect the views of Salesforce.

“My personal anecdotes and thoughts will be in quotes and in blue”

Tools

  • Figma (Design and FigJam)

Skills

  • User Research

  • Information Architecture

  • Design Strategy

  • Experience Design

  • Design Systems

  • Communication and Presentation Skills

Team

  • Som Liengtiraphan (Me, Senior Product Designer)

  • Reinier van Leuken (Senior Director, Product Management)

  • Chang Lu (Engineering)

  • Amruta Chitnis (Engineering)

  • Oksana Dzivinska (Principal Product Designer)

  • Adrian Massey (Principal Product Designer)

  • Anubha Dubey (Design Manager)

Impact & Contribution

Lead designer for the custom retriever builder and retriever playground projects, owning end-to-end UX design from initial concept through GA release.

Key Metrics

  • 2+ years of dedicated design leadership

  • Multiple feature releases from concept to production

  • Cross-functional collaboration across Engineering, Product, Content Strategy

  • Platform-level impact - foundational AI building block for agent development

Key Deliverables

Outcomes

Retriever Builder Design Architecture

  • Comprehensive UX for retriever building configuration, versioning, and setup workflows

Retriever Playground (Testing Experience)

  • Led design from concept through GA release

  • Designed testing workflows for individual and ensemble retrievers

  • Created validation, optimization, and fine-tuning experiences

Feature Designs

  • Retriever versioning system

  • Retriever citations

  • Web retriever support

  • Multi-modal retrievers exploration

  • Data kits integration for retrievers

“ A full case study for the Retriever Playground Design will be added soon “

Shipped to Production

Research & Validation

  • Created research plan for retriever playground testing

  • Conducted blue sky explorations for future iterations

  • Gathered continuous feedback from internal users to inform design decisions

An overview of how to create an individual retriever in our MVP product

Project Context

Retrievers help prevent LLM (large language model) hallucinations through its utilization in the RAG (retriever augmented generation) process, but building a custom one is complex.

Think of hallucinations in LLMs as someone making up sources for their thesis. RAG is like the research process of writing a thesis—it makes sure that every generation is grounded in facts. And if RAG is the research process, then retrievers are your personal research assistant. Instead of reading through the entire library yourself, you have a little robot that does it for you. It reads the entire library catalogue and comes back with exactly what you need, sources cited and all. This is the beauty of retrievers: they cut down on the work you actually need to do and help minimize hallucinations in your generations.

Here is a simplified view of the RAG process.

However, configuring a retriever involves a complex, multi-feature workflow that presents significant barriers to user adoption. Users must navigate Data Cloud's intricate data ingestion pipeline and search index creation, master its sophisticated data models, and possess the technical expertise to architect retrievers that precisely match their retrieval requirements.

This convergence of technical demands creates a user experience characterized by cognitive overload and a critical need for guided workflows.

The Problem

How might we guide users through a flexible retriever creation while reducing technical complexity?

The Users

There are 2 main categories of users custom retriever builder: Admin and Data Steward Personas.

Salesforce Admins

  • Enterprise Admins/Support - Users debugging retriever issues (like troubleshooting why a retriever isn't pulling the expected documents)

Data Stewards

  • Data Stewards - Users responsible for managing and maintaining data quality

  • AI Researchers - Users who need to fine-tune and optimize RAG configurations

Core JTBD (Jobs to be Done) for Custom Retriever Builder Users:

1. Configure Retrieval for Specific Use Cases

  • When I need to build an AI agent or copilot with specialized knowledge, help me configure a retriever that pulls the most relevant data for my specific business context.

2. Improve Retrieval Relevancy

  • When my retriever returns too many irrelevant results, help me apply filters and refine search parameters to increase precision without starting from scratch.

3. Optimize Data Selection

  • When I need to control what information gets returned, help me specify which fields and data sources to include so my AI outputs are accurate and useful.

4. Debug Retrieval Issues

  • When my retriever isn't pulling the expected documents, help me understand why and fix the configuration without needing deep Data Cloud expertise.

5. Adapt General Search Indexes

  • When I have a search index designed for general usage, help me apply it to my specific use case without rebuilding the entire index.

6. Bridge Technical Complexity

  • When I need to create a custom retriever, help me accomplish this without mastering Data Cloud's data ingestion pipeline, complex data models, and technical retrieval architecture.

The overall platform workflow for custom retriever building in the overall Salesforce ecosystem

Scope and Constraints

This project's scope covered developing features, identifying integration touchpoints, and delivering the MVP. The primary constraint were technical complexities and platform integration.

The simplified version of what is needed to create the RAG process

Scope

The Custom Retriever Builder Project was a 2+ year marathon creating Salesforce's foundational AI retrieval platform with no-code and pro-code capabilities. As lead designer, I owned end-to-end UX for the Retriever Builder Framework (Einstein Studio configuration) and Retriever Playground (testing environment). Key features included SQL-based retriever building, ensemble support, web crawling, and integrations across Data Cloud, Agentforce, and Prompt Builder. Phase 1 reached GA in Q4 2025/Q1 2026, establishing retrievers as the critical bridge between AI agents and Data360 assets while enabling tailored RAG solutions.

Constraints

The biggest constraints for this project were the technical complexity and a sprawling web of platform integrations needed to make Retriever a powerful AI building block across agents, citations, Prompt Builder, and beyond. This required extensive cross-functional collaboration and careful planning to position Retriever as the central bridge between agents and Data360 assets—all while maintaining our platform-first approach. It didn't help that our engineering and product teams were scattered across 5 time zones and 3 continents, turning scheduling into an Olympic sport. I eventually shifted to recording video updates to keep things moving asynchronously.

One of the best things my manager ever gave me — how retrievers can break as it makes away around the platform

Developing Expertise

Mastering RAG and retriever technology and the complexity of the Salesforce ecosystem was the first crucial step for this project—because apparently, diving headfirst into a technical labyrinth without a map is the designer's equivalent of extreme sports.

Retrievers are highly reliant on a network of features across Salesforce for data ingestion, set-up, and utilization—basically, they're the needy friend who requires everyone else to function. Retrievers and RAG in of itself was something I was not familiar it at the beginning of this project. I spent a few weeks becoming fluent in both subjects and mapping out the stakeholders who'd either make or break the parts of Salesforce that retrievers would inevitably invade. This upfront investment in becoming my own subject matter expert paid dividends later, cutting down the ping-pong matches with engineers and accelerating the design process.

As part of my learning process, I ran a quick workshop to collect all the retriever knowledge I could. Then I created a Venn diagram for myself, because apparently my brain works best with circles and intersections.

Experience Maps

After I understood the project details, I created user journeys as my first deliverables. This helped me validate my understanding and also served as onboarding documentation for future team members.

The workflow for the different types of retrievers we support

These user journeys were the first artifacts I created in my design process. I used them to validate my understanding of the project complexities with engineers and co-create the experience with my product partners. They were intentionally scrappy and loose—ideas were created and discarded quickly, because sometimes you need to kill your darlings before they become problems. This approach helped shape the experience and keep everyone aligned. Once these maps were created, they were validated with internal users before we dove deep into the design process.

As our team grew, the finalized experience map for each release was also used to onboard new colleagues. This was especially helpful paired with videos as our engineering teams were split between the US and India.

The three panel design for the retriever builder is as follows left to right: the overall steps, the main step canvas (advanced workflows will be split off into modals as needed), and a FAQ panel on the right to help guide users as needed.

Retriever Building Framework

The Retriever Building Framework was built with reducing cognitive load, flexibility, and scalability as its most important tenets.

Structure Overview

At its core, the builder encompasses 3 steps:

  1. Select the location to retrieve information from

  2. Determine what information you want to filter in and out

  3. Configure the output of your retrieved information

This simplicity is intentional. By distilling the process into just 3 broadly defined steps, we reduce the technical knowledge users need to create a custom retriever, allow space for new features, and enable the builder to scale and encompass other retriever types in the future.

The simplified filtration step for users with no SQL knowledge

Reducing Cognitive Load

The builder follows a standard navigation pattern for Salesforce. It utilizes progressive disclosure, chunked forms, and auto-filling to help streamline the creation process. I collaborated closely with our content team to use concise text, avoid jargon to improve readability, and include plenty of help text to guide users.

Filteration step with custom SQL editor

Flexibility

Vagueness was intentional in the design of the retriever builder framework to allow flexibility. Within each section, forms are organized into different areas that use a consistent library of custom-made components. This means components can be swapped in and out as requirements for different retrievers emerge. For example, as we expand to support more code-based filters, the filter condition components can be replaced with a custom SQL editor. Because both are placed in the "data filtration" step, the experience remains consistent.

Above is our ensemble retriever view, where we allow users to combine retrievers so they can retrieve information from multiple sources at once.

Scalability

Similar to flexibility, scalability was built into Retriever Builder through a consistent navigation framework and swappable components. At MVP, we only allowed users to build one type of custom retriever with minimal output configurations. Over the last two years, we've been able to add multiple types of retrievers, customized output configurations and citations, a testing environment, and scaled out our filtering experience to include both no-code and pro-code personas. All of this while retaining the same original framework.

The advanced mode of retriever playground for our MVP design.

Retriever Playground

Retriever Playground follows the natural user journey of creating and testing retrievers. It's natively integrated into the retriever building experience.

The primary focus was developing a comprehensive testing and optimization environment for retrievers. This included workflows for debugging existing retrievers, validation and fine-tuning capabilities for custom retrievers, and support for individual, ensemble, and custom retriever types. The design followed the natural progression of our experience map, which revealed what users actually want: the ability to kick the tires on their retrievers after building them and before unleashing them into the wild.

For the full case study of the design of retriever playground, click here.

Prioritization

Design proposals were prioritized based on technical feasibility, user feedback, and logical next steps from our experience map.

As with all design projects, more designs were discarded than created—a reminder that the delete key is every designer's most underrated tool. Our prioritization process considered technical feasibility (creation and platform integration), feedback from users about what they actually needed versus wanted, and logical next expansions based on the experience map we developed at the beginning of the project.

Retrievers being used in Salesforce’s agent builder. This was created with collaboration with the Agentforce Design Team

Platform Integrations

Retrievers are innately a building block of artificial intelligence products—not a product in and of themselves. Thus, their strategic value lies in integration with cross-product experiences.

Platform Integrations

  1. Prompt Builder Integration - The retriever integrates with Prompt Builder for Retrieval-Augmented Generation (RAG), allowing users to invoke retriever queries within prompt templates

  2. Ensemble Retriever - Allows users to group individual retrievers together into on super retriever

  3. Einstein Studio & Data Cloud Integration - The Retriever Builder is housed under Einstein Studio and requires an org configured with Data Cloud to support the full RAG pipeline

  4. Web Search Retriever Integration in Prompt Builder - Allows the users to use specific website domains as their sources of data for data retrieval

  5. Google Vertex Integration - Allows users to use Google Vertex in Retriever Builder UI

  6. Citations - When enabled, citations are added to the Prompt Response when that Retriever is used in RAG, providing crucial transparency for enterprise customers at any point of usage.

  7. Pluggable Query Editor Integration - For Custom SQL Retriever support, enabling a new set of users to write queries and "save as retriever"

Playground is can be launched directly from the retriever details page

Retriever Playground Integrations

  1. Retriever Builder Integration - The Playground launches directly from Retriever Builder via a "Test Retriever" button, allowing users to test and tune no-code retrievers seamlessly

  2. Testing Center/Workbench Integration - Phase 2 of Playground includes integration with Testing Center for a seamless workflow

  3. Dynamic Filters Support - Multiple dynamic filters were added to both Retriever Builder and Retriever Playground, with a modal in Playground for users to input these filters

Retriever Builder is ever evolving! Based on user feedback, here is an exploration of how we're iterating toward a fully agentic retriever-building experience.

Lessons Learned

Mapped strategic information gathering

Mapped all touchpoints across Salesforce's platform before design execution because the project's massive scope and numerous integration points throughout Salesforce made comprehensive upfront research essential. This approach prevented missed integrations and ensured alignment with the platform-first product strategy, avoiding costly redesigns and ensuring seamless cross-product functionality from the start.

Developed technical expertise

Mastered RAG and retriever technology independently to become my own subject matter expert, which was crucial given the highly technical nature of retrievers and RAG systems. This investment accelerated design velocity by reducing dependency on engineering consultations, allowing the design process to move faster despite slower initial startup time, and enabling more informed design decisions without constant technical validation.

Mastered invisible design

Prioritized seamless user experience over visual polish because this project required focusing on system connections and underlying functionality rather than interface aesthetics. This approach taught me that exceptional product design lies in creating reliable, seamless experiences that work flawlessly—much like choosing a dependable Toyota over a beautiful Porsche that never starts—delivering lasting value through robust functionality over superficial visual appeal.