How Infosys Topaz leverages Amazon Bedrock to rework technical assist desk operations

AI-powered apps and AI-powered service supply are key differentiators within the enterprise area in the present day. A generative AI-based useful resource can vastly scale back the onboarding time for brand spanking new staff, improve enterprise search, help in drafting content material, examine for compliance, perceive the authorized language of information, and extra.

Generative AI purposes are an rising and sought-after resolution within the enterprise world for buyer care facilities, buyer relationship administration facilities, and assist desks.

Infosys Topaz, an AI-first providing that accelerates enterprise worth for enterprises utilizing generative AI, is integrating AWS generative AI capabilities to future proof enterprise AI options together with Infosys Cortex, Infosys Personalized Smart Video (PSV), Infosys Conversational AI Suite, Infosys Live Enterprise Automation Platform (LEAP), and Infosys Cyber Next.

On this publish, we study the use case of a giant power provider whose technical assist desk assist brokers reply buyer calls and assist meter technicians within the subject. We use Amazon Bedrock, together with capabilities from Infosys Topaz, to construct a generative AI software that may scale back name dealing with occasions, automate duties, and enhance the general high quality of technical assist.

Enterprise challenges

Meter technicians go to buyer areas to put in, trade, service, and restore meters. Generally they name assist brokers from the technical assist desk to get steerage and assist to repair points that they will’t repair by themselves. The approximate quantity of those calls is 5,000 per week, roughly 20,000 per thirty days.

A number of the challenges confronted by assist brokers and meter technicians embody:

• Finding the suitable data or sources to deal with inquiries or issues successfully.
• The typical dealing with time for these calls varies based mostly on the difficulty class, however calls within the prime 10 classes, which signify over 60% of calls, are over 5 minutes.
• 60–70% points are repetitive, and the remainder are new points.

Sustaining an sufficient workforce to offer immediate responses may be expensive. It’s costly and never scalable to rent extra assist brokers and prepare them with the information wanted to offer assist. We constructed an AI-powered technical assist desk that may ingest previous name transcripts and new name transcripts in close to actual time. It will assist assist brokers present resolutions based mostly on previous calls, thereby decreasing guide search time to allow them to attend to different priorities.

Answer overview

The answer entails making a information base by ingesting and processing name transcripts, in order that the AI assistant can present resolutions based mostly on historical past. The advantages of an AI-powered technical assist desk embody:

• Offering all-day availability
• Saving effort for the assistance desk brokers
• Permitting companies to give attention to new points
• Lowering wait time and shortening name length
• Automating actions that the assistance desk brokers tackle the backend based mostly on their evaluation of the difficulty
• Bettering the standard of technical assist desk responses, and thereby communication and outcomes

This publish showcases the implementation particulars, together with user-based entry controls, caching mechanisms for environment friendly FAQ retrieval and updates, consumer metrics monitoring, and response technology with time-tracking capabilities.

The next diagram reveals the stream of information and processes from left to proper, beginning with name transcripts, going by means of preprocessing, storage, and retrieval, and ending with consumer interplay and response technology. It emphasizes the role-based entry management all through the system.

We used Amazon Bedrock as a result of it integrates seamlessly with different AWS providers proven within the diagram, reminiscent of AWS Step Functions, Amazon DynamoDB, and Amazon OpenSearch Service. This integration improves information stream and administration inside a single cloud system.

Constructing the information base: Information stream

Calls to the technical assist desk are recorded for high quality and evaluation functions, and the transcripts are saved in JSON format in an AWS Simple Storage Service (Amazon S3) bucket.

The conversations are parsed right into a CSV file for sorting and a big language mannequin (LLM), reminiscent of Anthropic’s Claude Sonnet on Amazon Bedrock, is used to summarize the dialog and decide if the context has helpful data, based mostly on the size of the decision, key phrases that point out related context, and so forth.

The shortlisted conversations are chunked, and embeddings are generated and saved in an Amazon OpenSearch Serverless vector retailer. The conversations decided to be irrelevant go into one other S3 bucket for future reference. This course of is automated, as proven within the following determine.

A digital assistant is then constructed on prime of the information base that can help the assist agent.

The conversations are parsed right into a CSV file for easy sorting and an LLM reminiscent of Anthropic’s Claude Sonnet on Amazon Bedrock is used to summarize the dialog and decide if the context has helpful data, based mostly on the size of the decision, key phrases that point out related context, and so forth.

An event-driven AWS Lambda operate is triggered when new name transcripts are loaded into the S3 bucket. It will set off a Step Capabilities workflow.

From the uncooked CSV file of name transcripts, only some fields are extracted: a contact ID that’s distinctive for a specific name session between a buyer and a assist agent, the participant column indicating the speaker (who may be both a assist agent or a buyer) and the content material column, which is the dialog.

To construct the information base, we used Step Capabilities to ingest the uncooked CSV recordsdata, as proven within the following workflow.

The automated workflow begins when a consumer uploads the JSON file to an S3 bucket.

1. The Step Capabilities workflow receives the Amazon S3 URL of the CSV transcripts from a Lambda operate. The contactid is exclusive for a specific name session between the client and the agent, who’re the members, and the content material is the precise dialog.
2. The Lambda operate (Parse Transcripts from CSV) makes use of this Amazon S3 URL to obtain the CSV recordsdata and makes use of Pandas to preprocess the CSV in a format with the contact ID and transcript solely. Conversations with the identical contact ID are concatenated right into a single row.
3. The second step is a classification activity that ingests, classifies, and retains or discards conversions. The conversations are handed to the map state. In map state, conversations are dealt with concurrently. For every dialog row, this state triggers concurrent execution of one other Lambda operate (Verify for Irrelevant Conversations) that can classify every dialog as related or irrelevant.
   1. For this classification activity, the Lambda operate makes use of Anthropic’s Claude Sonnet mannequin on Amazon Bedrock. It makes use of zero-shot chain-of-thought prompting, to first summarize the dialog after which to find out the relevance. If the dialog is disconnected or disjointed (due to sign disturbances or different causes), or has no significant context (when the agent is unable to offer decision), it’s labeled as irrelevant.
4. Lastly, the map state takes every occasion of the dialog (labeled as related or irrelevant) and passes to the selection state, which is able to log the irrelevant conversations into an S3 bucket and related conversations are handed to a different Lambda operate (Deal with Related Conversations) for additional processing.
5. The ultimate Lambda operate (Log Irrelevant Conversations) reads the related conversations and generates the abstract, drawback, and determination steps utilizing Anthropic’s Claude Sonnet. The abstract generated is used for creating the abstract embeddings.

The next is an instance of an irrelevant dialog that’s discarded.

The next is an instance of a related dialog.

The next desk reveals the ultimate information base schema.

Constructing an efficient RAG pipeline

The success of retrieval techniques depends on an efficient embedding mannequin. The Amazon Titan Text Embeddings mannequin is optimized for textual content retrieval to allow Retrieval Augmented Technology (RAG). As a substitute of processing huge paperwork on the identical time, we used chunking methods to enhance retrieval. We used a bit dimension of 1,000 with an overlapping window of 150–200 for greatest outcomes. Chunking mixed with web page boundaries is a straightforward but extremely efficient method. Sentence window retrieval additionally returns correct outcomes.

Prompting methods play a vital function in acquiring efficient outcomes. For instance, as a substitute of “pointers for sensible meter set up,” an expanded immediate reminiscent of “directions, procedures, laws, and greatest practices together with agent experiences for set up of a sensible meter” yields higher outcomes.

Constructing production-ready RAG purposes requires a performant vector database as effectively. The vector engine for OpenSearch Serverless supplies a scalable and high-performing vector storage and search functionality; key options embody including, updating, and deleting vector embeddings in close to actual time with out impacting question efficiency. See Build a contextual chatbot application using Amazon Bedrock Knowledge Bases for extra data.

Safety issues

This structure implements complete safety measures throughout the parts. We use AWS Secrets Manager to securely retailer and handle delicate credentials, API keys, and database passwords, with computerized rotation insurance policies in place. S3 buckets are encrypted utilizing AWS Key Management Service (AWS KMS) with AES-256 encryption, and versioning is enabled for audit functions. Personally identifiable data (PII) is dealt with with excessive care— PII information is encrypted and entry is strictly managed by means of AWS Identity and Access Management (IAM) insurance policies and AWS KMS. For OpenSearch Serverless implementation, we be sure that information is encrypted each at relaxation utilizing AWS KMS and in transit utilizing TLS 1.2. Session administration contains timeout for inactive periods, requiring re-authentication for continued entry. The system interacts with entry management record (ACL) information saved in DynamoDB by means of a safe middleware layer, the place the DynamoDB desk is encrypted at relaxation utilizing AWS managed KMS keys. Information transmissions between providers are encrypted in transit utilizing TLS 1.2, and we keep end-to-end encryption throughout our complete infrastructure. Entry controls are granularly outlined and recurrently audited by means of AWS CloudTrail.

Implementing role-based entry management

We used three totally different personas to implement role-based entry management: an administrator with full entry, a technical desk analyst with a medium stage of entry, and a technical agent with minimal entry. We used OpenSearch Serverless collections to handle totally different entry ranges. Totally different name transcripts are ingested into totally different collections; that is to allow consumer entry to the content material they’re approved to based mostly on their roles. An inventory of consumer IDs and their roles and allowed entry are saved in a DynamoDB desk together with the OpenSearch assortment and index identify.

We used the authenticate.login methodology in a Streamlit authenticator to retrieve the consumer ID.

Consumer interface and agent expertise

We used Streamlit as a frontend framework to construct the TECHNICAL HELP DESK, with entry to the content material managed by the consumer’s function. The UI options an FAQ part displayed on the prime of the primary web page and a search metrics insights part within the sidebar, as proven within the following screenshot.

The UI contains the next parts:

• Dialog part – The dialog part accommodates interactions between the consumer and the assistance desk assistant. Customers can present suggestions by selecting both the like or dislike button for every response obtained, as proven within the following screenshot. This suggestions is continued in a DynamoDB desk.

• Consumer metrics insights – As proven within the following screenshot, the sidebar accommodates metrics data, together with:
  • Variety of queries within the final week
  • Variety of complete transcripts
  • Variety of transcripts added within the final week
  • Variety of useful responses generated
  • Variety of dislikes
  • Variety of misses (no reply discovered)

These fields are up to date asynchronously after every consumer question. Further metrics are additionally saved, reminiscent of sentiment, tone of the audio system, nature of responses generated, and satisfaction proportion.

• FAQ – The queries are saved in a DynamoDB desk together with a question depend column. When the assistance desk agent indicators in, the queries with probably the most counts are displayed on this part, as proven within the following desk.

The Counter column is created as the worldwide secondary index to retrieve the highest 5 FAQs.

After the consumer submits a question, the technical assist desk fetches the highest related objects from the information base. That is in contrast with the consumer’s question and, when a match is discovered, the Counter column is incremented.

Cache administration

We used the st.cache_data() operate in Streamlit to retailer the legitimate ends in reminiscence. The outcomes are continued throughout the consumer periods.

The caching operate employs an inside hashing mechanism that may be overridden if required. The cached information may be saved both in reminiscence or on disk. Moreover, we are able to set the info persistence length as wanted for the use case. Cache invalidation or updates may be accomplished when the info modifications or after each hour. This, together with the FAQ part, has considerably enhanced efficiency of the technical assist desk, creating sooner response occasions and bettering the consumer expertise for purchasers and assist brokers.

Conclusion

On this publish, we confirmed you ways we constructed a generative AI software to considerably scale back name dealing with occasions, automate repetitive duties, and enhance the general high quality of technical assist.

The enterprise AI assistant from the Infosys Agentic Foundry, a part of Infosys Topaz, now handles 70% of the beforehand human-managed calls. For the highest 10 subject classes, common dealing with time has decreased from over 5 minutes to underneath 2 minutes, a 60% enchancment. The continual growth of the information base has decreased the share of points requiring human intervention from 30–40% to twenty% inside the first 6 months after deployment.

Submit-implementation surveys present a 30% improve in buyer satisfaction scores associated to technical assist interactions.

To be taught extra about different options constructed with Amazon Bedrock and Infosys Topaz, see Create a multimodal assistant with advanced RAG and Amazon Bedrock and Infosys Topaz Unlocks Insights with Advanced RAG Processing for Oil & Gas Drilling Data.

In regards to the authors

Meenakshi Venkatesan is a Principal Marketing consultant at Infosys and part of the AWS Centre Of Excellence at Infosys Topaz. She helps design, develop, and deploy options in AWS environments and has pursuits in exploring the brand new choices and providers.

Karthikeyan Senthilkumar is a Senior Techniques Engineer at Infosys and part of the AWS COE at iCETS. He focuses on AWS generative AI and database providers.

Aninda Chakraborty is a Senior Techniques Engineer at Infosys and part of the AWS COE at iCETS. He focuses on generative AI and is obsessed with leveraging know-how to create progressive options that drive progress on this subject.

Ashutosh Dubey is an completed software program technologist and Technical Chief at Amazon Net Companies, the place he focuses on Generative AI options structure. With a wealthy background in software program growth and information engineering, he architects enterprise-scale AI options that bridge innovation with sensible implementation. A revered voice within the tech group, he recurrently contributes to trade discourse by means of talking engagements and thought management on Generative AI purposes, Information engineering, and moral AI practices.

Vishal Srivastava is a Senior Options Architect with a deep specialization in Generative AI. In his present function, he collaborates carefully with NAMER System Integrator (SI) companions, offering knowledgeable steerage to architect enterprise-scale AI options. Vishal’s experience lies in navigating the complicated panorama of AI applied sciences and translating them into sensible, high-impact implementations for companies. As a thought chief within the AI area, Vishal is actively engaged in shaping trade conversations and sharing information. He’s a frequent speaker at public occasions, webinars, and conferences, the place he gives insights into the newest developments and greatest practices in Generative AI.

Dhiraj Thakur is a Options Architect with Amazon Net Companies, specializing in Generative AI and information analytics domains. He works with AWS prospects and companions to architect and implement scalable analytics platforms and AI-driven options. With deep experience in Generative AI providers and implementation, end-to-end machine studying implementation, and cloud-native information architectures, he helps organizations harness the ability of GenAI and analytics to drive enterprise transformation. He may be reached by way of LinkedIn.