“Where knowledge begins.” Perplexity’s pithy motto reflects its mission to save users time by providing precise knowledge as an AI “answer engine.”

Recently, the Perplexity team launched Pro Search, a feature that can answer complex, nuanced questions using multi-step reasoning. Unlike Perplexity’s quick search, which is designed for off-the-cuff questions, this advanced modality helps students, researchers, and enterprises gain precise, relevant responses to even the most complex and detailed questions.  

Thanks to the Perplexity team’s thoughtful approach to crafting user experience and agent architecture, they didn’t have to compromise on speed and accuracy, even when increasing the complexity of its systems.

Traditional search engines may struggle to answer complex queries that require connecting the dots across multiple ideas or extracting detailed information. For instance, searching "What’s the educational background of the founders of LangChain?" involves not only identifying the founders but also researching into each individual founder’s background.

This is where Perplexity Pro Search shines. Their AI agent breaks down multi-step questions to deliver well-organized, factual answers. Instead of sifting through countless pages of search results, users get direct responses from Perplexity Pro Search that summarize the most relevant information.

In fact, query search volume of Perplexity Pro Search has increased by over 50% in the past few months, as more users discover its ability to answer tricky questions quickly and efficiently.

Perplexity Pro’s AI agent separates planning from execution, which yields better results for multi-step search. 

When a user submits a query, the AI creates a plan— a step-by-step guide to answering it. For each step in the plan, a list of search queries are generated and executed. These steps are executed sequentially, and results from previous steps are passed when executing steps after. These search queries return a list of documents, which are grouped and then filtered down to the most relevant ones. The highly-ranked documents are then passed to an LLM to generate a final answer.

Perplexity Pro Search also supports specialized tools such as code interpreters, which allow users to run calculations or analyze files on the fly, as well as mathematics evaluations tools like Wolfram Alpha.

Perplexity uses a variety of language models to break down web search tasks, giving users the flexibility to choose the model that best fits the problem they’re trying to solve. Since each language model processes and interprets prompts differently, Perplexity customizes prompts on the backend that are tailored to each individual model. 

In order to guide the model’s behavior, Perplexity leverages techniques like few-shot prompt examples and chain-of-thought prompting. Few-shot examples allow engineers to steer the search agent’s behavior. When constructing few-shot examples, maintaining the right balance in prompt length was crucial. Crafting the rules that the language model should follow also involved several rounds of iteration. 

William Zhang, the engineer who led this effort at Perplexity, shared:

"It’s harder for models to follow the instructions of really complex prompts. Much of the iteration involves asking queries after each prompt change and checking that not only the output made sense, but that the intermediate steps were sensible as well."

By keeping the rules in the system prompt simple and precise, Perplexity reduced the cognitive load for models to understand the task and generate relevant responses.

Perplexity relied on both answer quality metrics and internal dogfooding before shipping the upgrade of Pro Search. The team conducted manual evaluations by testing Pro Search on a wide range of queries and comparing its answers side-by-side with other AI products. The ability to inspect intermediate steps was also critical in helping identify common errors before shipping to users. 

To scale up their evaluations, Perplexity gathered a large batch of questions and used an LLM-as-a-Judge to rank the answers. Additionally, A/B tests were run on users to gauge their reactions to different possible configurations of the product, such as tradeoffs between latency and costs across different models. The product was ready to be shipped after the Perplexity team was satisfied with the product experience from both an answer quality and UX perspective.

One of the biggest challenges for the team was designing the Perplexity Pro Search user interface. Perplexity found that users were more willing to wait for results if the product would display the intermediate progress.

This led to the development of an interactive UI that shows the plan being executed step-by-step. The team iterated on expandable sections that allow the user to click on individual steps to see more details on a search. They also introduced the ability to hover over citations to see snippets from sources that the user could click on to open in a new window. 

Perplexity’s Pro Search represents a significant advancement in AI-powered search and question-answering. By breaking down complex queries into manageable steps and providing a transparent, interactive interface, Perplexity has created a powerful tool that works at the speed of curiosity. 

As Zhang emphasizes:

“It is important that we design our product with the user in mind, since our users span a wide range of familiarity with AI systems. Some are experts while others are new to AI search interfaces – so we have to make sure we’re creating a positive experience for everyone, regardless of their expertise level.” 

Their development process offers valuable lessons for others building AI agents:

1. Have the LLM do an explicit planning step when doing more complicated research

2. Speed alongside answer quality is important for creating a good user experience. Keep users engaged with dynamic UI feedback instead of leaving them waiting.