Enhancing LLM accuracy with Coveo Passage Retrieval on Amazon Bedrock

This put up is co-written with Keith Beaudoin and Nicolas Bordeleau from Coveo.

As generative AI transforms enterprise operations, enterprises face a vital problem: how can they assist massive language fashions (LLMs) present correct and reliable responses? With out dependable knowledge foundations, these AI fashions can generate deceptive or inaccurate responses, doubtlessly decreasing person belief and organizational credibility.

As an AWS Companion, Coveo addresses this problem with its Passage Retrieval API. This answer enhances the reliability of LLM-powered purposes by offering them with related, context-aware enterprise information to tell generated responses. In Retrieval Augmented Era (RAG) techniques, the retrieval course of is essentially the most complicated element. It requires extracting essentially the most related, exact info from enterprise knowledge sources. By integrating the Coveo AI-Relevance Platform with Amazon Bedrock Agents, organizations acquire entry to an industry-leading enterprise search service that includes a secured, unified hybrid index that respects enterprise permission fashions and affords strong connectivity. The Coveo AI-Relevance Platform makes use of machine studying (ML) and in-depth utilization analytics to repeatedly optimize relevance. This permits Amazon Bedrock Brokers to ship grounded, contextually related responses tailor-made to complicated enterprise content material.

The Coveo AI-Relevance Platform is an industry-leading service that connects and unifies the content material of cloud and on-premises repositories in a single index, making it quick and easy to search out related content material shortly. Its ML algorithms analyze person habits, in-app context, in addition to profile and permissions knowledge to retrieve customized search outcomes and suggestions. It then aggregates and reviews insights again to content material and expertise managers. By integrating seamlessly with enterprise techniques (comparable to web sites, information bases, and CRM) and implementing safety permissions, Coveo helps customers get essentially the most pertinent info whereas sustaining knowledge safety.

On this put up, we present tips on how to deploy Coveo’s Passage Retrieval API as an Amazon Bedrock Brokers motion group to reinforce response accuracy, so Coveo customers can use their present index to quickly deploy new generative experiences throughout their group.

Coveo’s Passage Retrieval API

The Coveo Passage Retrieval API enhances LLM purposes by passing ranked textual content passages (or chunks) retrieved from the unified index, together with applicable metadata comparable to supply URLs for citations, in order that generated solutions are grounded in a company’s proprietary information. Constructed on Coveo’s unified hybrid index, the Passage Retrieval API applies a two-stage retrieval course of to extract essentially the most related passages from structured and unstructured content material sources, offering accuracy, safety, and real-time relevance. The next diagram illustrates these levels.

The method consists of the next key parts:

• Related passage extraction utilizing a two-stage retrieval course of – Within the first retrieval stage, Coveo’s hybrid search system is used to establish essentially the most related paperwork. Then, it extracts essentially the most related textual content passages from these paperwork, together with rating scores, quotation hyperlinks, and different key metadata. This two-stage method permits Coveo to establish accessible and related paperwork from the sources after which extra exactly establish essentially the most related passages.
• Enhanced search outcomes with hybrid rating – Combining semantic (vector) search, and lexical (key phrase) matching helps Coveo retrieve the precise info in the precise context.
• ML relevancy – AI repeatedly learns from person interactions, tailoring retrieval to every person’s journey, habits, and profile for context-aware responses.
• Content material linked throughout sources with a unified index – A centralized hybrid index connects structured and unstructured content material throughout sources. This unified hybrid index supplies higher multi-source relevancy than a federated search method by making use of the rating operate throughout sources. Coveo additionally supplies a sturdy library of pre-built connectors to take care of seamless integration with third-party providers (comparable to Salesforce, SharePoint, and Google Drive), facilitating knowledge freshness with computerized updates for real-time retrieval.
• Analytics and insights for efficiency monitoring – With occasions monitoring by the Knowledge Platform and Data Hub, you may see precisely how your generated solutions carry out, the place info is lacking or underused, and which content material wants tuning. With these insights, you may enhance reply high quality and drive measurable enterprise influence.
• Enterprise-grade safety – Coveo supplies the native permission mannequin of every linked content material supply by importing merchandise‑degree permissions at crawl time by an early‑binding method. Resolving entry rights earlier than indexing helps forestall knowledge leakage and boosts search efficiency by filtering out objects a person can’t see earlier than a question is run.
• Redefining enterprise-ready RAG – Coveo reimagines what RAG techniques can obtain by going past fundamental vector search, providing a dynamic and clever retrieval pipeline designed for enterprise wants. At its core is a unified hybrid index that seamlessly connects structured, unstructured, and permission-sensitive knowledge. This basis, mixed with real-time ML-driven tuning, helps validate that each response delivered to an LLM is related and securely grounded in the precise context.

By means of native entry management enforcement, behavior-based relevance adjustment, and deep analytics into content material utilization and efficiency, Coveo empowers organizations to repeatedly refine their generative AI experiences. Backed by constant recognition from main analyst companies comparable to Gartner, Forrester, and IDC, Coveo delivers a dependable, safe, and scalable basis for enterprise-grade generative AI.

Resolution overview

This answer demonstrates how one can combine Coveo’s Passage Retrieval API with Amazon Bedrock Brokers to construct LLM-powered assistants that ship correct, context-aware, and grounded responses. It applies broadly throughout use circumstances the place brokers have to entry proprietary information, whether or not to assist prospects, help workers, or empower gross sales and repair groups. This method helps AI responses keep grounded in your most related and trusted info throughout structured and unstructured content material. Amazon Bedrock Brokers acts because the clever spine of this answer, deciphering pure language queries and orchestrating the retrieval course of to ship grounded, contextually related insights. For this use case, the agent is provided to reply a person’s questions on Coveo providers capabilities, API documentation, and integration guides. The agent is designed to fetch exact passages from enterprise content material in response to person questions, enabling purposes comparable to digital assistants, assist copilots, or inner information bots. Through the use of structured definitions and directions, the agent understands when and tips on how to set off Coveo’s Passage Retrieval API, ensuring that LLM-generated responses are backed by correct and trusted content material.

The motion group defines the structured API operations that the Amazon Bedrock agent can invoke. Utilizing OpenAPI specs, it defines the interface between the agent and AWS Lambda capabilities. On this use case, fetching related passages primarily based on the person’s search intent is the one out there operation.

The next diagram illustrates the answer structure.

For this demo, the agent is ready with the next directions throughout creation:

You'll act as an knowledgeable on Coveo documentation, platform, APIs, analytics, and integration guides.
        Use the CoveoPRAPI Motion Group to retrieve related info on Coveo documentation, platform, APIs, analytics, and integration guides.
        Summarize the knowledge retrieved from the Motion Group response clearly and precisely.
        OuputFormatting tips:
        - Present clear, direct solutions with out introductory phrases comparable to “As an knowledgeable,” “Positive,” or “Right here is...”
        - When applicable, arrange content material utilizing:
          - Numbered or bulleted lists
        - Structured sections (e.g., options, steps, key factors)
        Hold solutions concise, informative, and freed from conversational filler.

The Lambda operate outlined within the motion group is important for enabling the Amazon Bedrock agent to name the Coveo Passage Retrieval API. The Lambda operate performs the next duties:

• Receives incoming requests from the Amazon Bedrock agent
• Queries the Coveo Passage Retrieval API utilizing the person’s enter
• Returns the related search outcomes again to the Amazon Bedrock agent

Lastly, the Coveo AI-Relevance Platform supplies listed and structured search outcomes by the Passage Retrieval API.

Conditions

Earlier than you start, it’s essential to have the next stipulations:

Deploy the answer with AWS CloudFormation

To deploy your agent, full the next steps:

1. Launch the CloudFormation stack:
   
2. Enter a stack identify and values for AgentModelID, AgentName, CoveoApiKey, CoveoOrgID, and CoveoSearchHub.
3. Within the Capabilities part, choose the acknowledge test field.
4. Select Create stack.
5. Look forward to the stack creation to finish.
6. Confirm all sources are created on the stack particulars web page.

Take a look at the answer

To check the answer, full the next steps:

1. On the Amazon Bedrock console, select Brokers within the navigation pane.
2. Select the agent created by the CloudFormation stack.

3. Below Take a look at, enter your query within the message field.

For this demo, Coveo technical documentation (from the web site) was ingested in an current Coveo Search index. Let’s begin with the question “What’s the distinction between Coveo Atomic and Headless?”

Right here’s how the agent answered the query when Claude 3 Haiku v1 is used because the LLM.

4. Select Present hint in the precise pane and broaden Hint step 1 to see the agent’s rationale.

The next screenshot demonstrates how Amazon Bedrock Brokers processed and answered the query. First, it fashioned a rationale:

To know the distinction between Coveo Atomic and Headless, I might want to retrieve related info from the Coveo technical documentation utilizing the CoveoPRAPI Motion Group.

Then it invokes the CoveoPRAP motion group, which is particularly designed to retrieve related passages, by a Lambda operate to make an API name to /relaxation/search/v3/passages/retrieve.

This instance illustrates the agent’s systematic method to planning and executing essential actions by the CoveoPRAP1 motion group, and retrieving related doc chunks earlier than formulating its closing response.

The Lambda operate code features a debugging function that logs every retrieved passage from the Passage Retrieval API. This logging mechanism iterates by the returned chunks, numbering them sequentially for fast reference. These logs can be found in Amazon CloudWatch, so you may see precisely which passages have been retrieved for every person question, and the way they contributed to the ultimate response. To visualise the logs, open the CloudWatch console, and on the Log teams web page, find the Lambda operate identify.

The next screenshot exhibits the agent detailed logs in CloudWatch. Within the logs, the Coveo Passage Retrieval API returns the 5 most related chunks to the LLM.

Clear up

To keep away from ongoing prices, delete the sources deployed as a part of the CloudFormation stack:

1. On the AWS CloudFormation console, select Stacks within the navigation pane.
2. Select the stack you created, then select Delete.
3. Select Delete stack when prompted.

For extra info, consult with Deleting a stack on the AWS CloudFormation console.

Conclusion

By integrating Amazon Bedrock capabilities with Coveo’s AI-driven retrieval, organizations can develop AI purposes that present validated responses primarily based on their enterprise content material. This method helps scale back inaccuracies whereas delivering real-time, safe responses.

We encourage you to discover pre-built examples within the GitHub repository that will help you get began with Amazon Bedrock.

To be taught extra concerning the Coveo AI-Relevance Platform and tips on how to implement the Passage Retrieval API in your Coveo group, consult with Passage Retrieval (CPR) implementation overview.

Concerning the authors

Yanick Houngbedji is a Options Architect for Unbiased Software program Distributors (ISV) at Amazon Net Providers (AWS), primarily based in Montréal, Canada. He focuses on serving to prospects architect and implement extremely scalable, performant, and safe cloud options on AWS. Earlier than becoming a member of AWS, he spent over 8 years offering technical management in knowledge engineering, large knowledge analytics, enterprise intelligence, and knowledge science options.

Keith Beaudoin is a Senior Resolution Architect at Coveo Labs. He’s specialised in designing and implementing clever search and AI-powered relevance options, with experience in Agentic options, cloud applied sciences, search structure, and third-party integrations. Keith helps organizations harness the total potential of Coveo’s platform, optimizing digital transformation methods for seamless and impactful search implementations that drive enterprise worth

Nicolas Bordeleau is Head of Product Relations at Coveo. With over 19 years of expertise within the search {industry}, together with 11 years in product administration, Nicolas has a eager understanding of enterprises and builders’ wants associated to go looking and data retrieval. He has utilized this information to develop award-winning merchandise that fulfill these wants in progressive methods.