Asure’s strategy to enhancing their name heart expertise utilizing generative AI and Amazon Q in QuickSight

Asure, an organization of over 600 workers, is a number one supplier of cloud-based workforce administration options designed to assist small and midsized companies streamline payroll and human assets (HR) operations and guarantee compliance. Their choices embrace a complete suite of human capital administration (HCM) options for payroll and tax, HR compliance companies, time monitoring, 401(ok) plans, and extra.

Asure anticipated that generative AI may assist contact heart leaders to know their staff’s assist efficiency, determine gaps and ache factors of their merchandise, and acknowledge the simplest methods for coaching buyer assist representatives utilizing name transcripts. The Asure staff was manually analyzing 1000’s of name transcripts to uncover themes and tendencies, a course of that lacked scalability. The overarching purpose of this engagement was to enhance upon this guide strategy. Failing to undertake a extra automated strategy may have probably led to decreased buyer satisfaction scores and, consequently, a loss in future income. Subsequently, it was priceless to offer Asure a post-call analytics pipeline able to offering helpful insights, thereby enhancing the general buyer assist expertise and driving enterprise development.

Asure acknowledged the potential of generative AI to additional improve the consumer expertise and higher perceive the wants of the shopper and wished to discover a companion to assist notice it.

Pat Goepel, chairman and CEO of Asure, shares,

“In collaboration with the AWS Generative AI Innovation Middle, we’re using Amazon Bedrock, Amazon Comprehend, and Amazon Q in QuickSight to know tendencies in our personal buyer interactions, prioritize objects for product growth, and detect points sooner in order that we might be much more proactive in our assist for our clients. Our partnership with AWS and our dedication to be early adopters of revolutionary applied sciences like Amazon Bedrock underscore our dedication to creating superior HCM expertise accessible for companies of any measurement.”

“We’re thrilled to companion with AWS on this groundbreaking generative AI venture. The strong AWS infrastructure and superior AI capabilities present the right basis for us to innovate and push the boundaries of what’s attainable. This collaboration will allow us to ship cutting-edge options that not solely meet however exceed our clients’ expectations. Collectively, we’re poised to rework the panorama of AI-driven expertise and create unprecedented worth for our shoppers.”

—Yasmine Rodriguez, CTO of Asure.

“As we launched into our journey at Asure to combine generative AI into our options, discovering the appropriate companion was essential. Having the ability to companion with the Gen AI Innovation Middle at AWS brings not solely technical experience with AI however the expertise of growing options at scale. This collaboration confirms that our AI options usually are not simply revolutionary but additionally resilient. Collectively, we consider that we will harness the ability of AI to drive effectivity, improve buyer experiences, and keep forward in a quickly evolving market.”

—John Canada, VP of Engineering at Asure.

On this submit, we discover why Asure used the Amazon Web Services (AWS) post-call analytics (PCA) pipeline that generated insights throughout name facilities at scale with the superior capabilities of generative AI-powered companies similar to Amazon Bedrock and Amazon Q in QuickSight. Asure selected this strategy as a result of it supplied in-depth shopper analytics, categorized name transcripts round frequent themes, and empowered contact heart leaders to make use of pure language to reply queries. This in the end allowed Asure to offer its clients with enhancements in product and buyer experiences.

Answer Overview

At a excessive degree, the answer consists of first changing audio into transcripts utilizing Amazon Transcribe and producing and evaluating abstract fields for every transcript utilizing Amazon Bedrock. As well as, Q&A might be achieved at a single name degree utilizing Amazon Bedrock or for a lot of calls utilizing Amazon Q in QuickSight. In the remainder of this part, we describe these parts and the companies utilized in larger element.

We added upon the present PCA solution with the next companies:

Customer support and name heart operations are extremely dynamic, with evolving buyer expectations, market tendencies, and technological developments reshaping the business at a speedy tempo. Staying forward on this aggressive panorama calls for agile, scalable, and clever options that may adapt to altering calls for.

On this context, Amazon Bedrock emerges as an distinctive selection for growing a generative AI-powered answer to investigate customer support name transcripts. This absolutely managed service gives entry to cutting-edge basis fashions (FMs) from main AI suppliers, enabling the seamless integration of state-of-the-art language fashions tailor-made for textual content evaluation duties. Amazon Bedrock gives fine-tuning capabilities that let you customise these pre-trained fashions utilizing proprietary name transcript information, facilitating excessive accuracy and relevance with out the necessity for intensive machine studying (ML) experience. Furthermore, Amazon Bedrock gives integration with different AWS companies like Amazon SageMaker, which streamlines the deployment course of, and its scalable structure makes positive the answer can adapt to growing name volumes effortlessly.

With strong safety measures, information privateness safeguards, and a cheap pay-as-you-go mannequin, Amazon Bedrock gives a safe, versatile, and cost-efficient service to harness generative AI’s potential in enhancing customer support analytics, in the end resulting in improved buyer experiences and operational efficiencies.

Moreover, by integrating a data base containing organizational information, insurance policies, and domain-specific data, the generative AI fashions can ship extra contextual, correct, and related insights from the decision transcripts. This information base permits the fashions to know and reply based mostly on the corporate’s distinctive terminology, merchandise, and processes, enabling deeper evaluation and extra actionable intelligence from buyer interactions.

On this use case, Amazon Bedrock is used for each era of abstract fields for pattern name transcripts and analysis of those abstract fields in opposition to a floor reality dataset. Its worth comes from its easy integration into present pipelines and numerous analysis frameworks. Amazon Bedrock additionally permits you to select numerous fashions for various use instances, making it an apparent selection for the answer because of its flexibility. Utilizing Amazon Bedrock permits for iteration of the answer utilizing data bases for easy storage and entry of name transcripts in addition to guardrails for constructing accountable AI purposes.

Amazon Bedrock

Amazon Bedrock is a totally managed service that makes FMs accessible by way of an API, so you may select from a variety of FMs to search out the mannequin that’s finest suited to your use case. With the Amazon Bedrock serverless expertise, you will get began rapidly, privately customise FMs with your personal information, and rapidly combine and deploy them into your purposes utilizing AWS instruments with out having to handle the infrastructure.

Amazon Q in QuickSight

Amazon Q in QuickSight is a generative AI assistant that accelerates decision-making and enhances enterprise productiveness with generative enterprise intelligence (BI) capabilities.

The unique PCA answer consists of the next companies:

The answer consisted of the next parts:

• Name metadata era – After the file ingestion step when transcripts are generated for every name transcript utilizing Amazon Transcribe, Anthropic’s Claude Haiku FM in Amazon Bedrock is used to generate call-related metadata. This features a abstract, the class, the basis trigger, and different high-level fields generated from a name transcript. That is orchestrated utilizing AWS Step Functions.
• Particular person name Q&A – For questions requiring a particular name, similar to, “How did the shopper react in name ID X,” Anthropic’s Claude Haiku is used to energy a Q&A assistant positioned in a CloudFront software. That is powered by the online app portion of the structure diagram (supplied within the subsequent part).
• Mixture name Q&A – To reply questions requiring a number of calls, similar to “What are the commonest points detected,” Amazon Q on QuickSight is used to reinforce the Agent Help interface. This step is proven by enterprise analysts interacting with QuickSight within the storage and visualization step by way of pure language.

To be taught extra concerning the architectural parts of the PCA answer, together with file ingestion, perception extraction, storage and visualization, and internet software parts, check with Post call analytics for your contact center with Amazon language AI services.

Structure

The next diagram illustrates the answer structure. The analysis framework, name metadata era, and Amazon Q in QuickSight had been new parts launched from the unique PCA answer.

Ragas and a human-in-the-loop UI (as described within the customer blogpost with Tealium) had been used to guage the metadata era and particular person name Q&A parts. Ragas is an open supply analysis framework that helps consider FM-generated textual content.

The high-level takeaways from this work are the next:

• Anthropic’s Claude 3 Haiku efficiently took in a name transcript and decided its abstract, root trigger, if the problem was resolved, and, if it was a callback, subsequent steps by the shopper and agent (generative AI-powered fields). When utilizing Anthropic’s Claude 3 Haiku versus Anthropic’s Claude Instantaneous, there was a discount in latency. With chain-of-thought reasoning, there was a rise in total high quality (consists of how factual, comprehensible, and related responses are on a 1–5 scale, described in additional element later on this submit) as measured by subject material specialists (SMEs). With using Amazon Bedrock, numerous fashions might be chosen based mostly on totally different use instances, illustrating its flexibility on this software.
• Amazon Q in QuickSight proved to be a robust analytical instrument in understanding and producing related insights from information by way of intuitive chart and desk visualizations. It will probably carry out easy calculations each time essential whereas additionally facilitating deep dives into points and exploring information from a number of views, demonstrating nice worth in perception era.
• The human-in-the-loop UI plus Ragas metrics proved efficient to guage outputs of FMs used all through the pipeline. Notably, reply correctness, reply relevance, faithfulness, and summarization metrics (alignment and protection rating) had been used to guage the decision metadata era and particular person name Q&A parts utilizing Amazon Bedrock. Its flexibility in numerous FMs allowed the testing of many forms of fashions to generate analysis metrics, together with Anthropic’s Claude Sonnet 3.5 and Anthropic’s Claude Haiku 3.

Name metadata era

The decision metadata era pipeline consisted of changing an audio file to a name transcript in a JSON format utilizing Amazon Transcribe after which producing key data for every transcript utilizing Amazon Bedrock and Amazon Comprehend. The next diagram exhibits a subset of the previous structure diagram that demonstrates this.

The unique PCA submit linked beforehand exhibits how Amazon Transcribe and Amazon Comprehend are used within the metadata era pipeline.

The decision transcript enter that was outputted from the Amazon Transcribe step of the Step Capabilities workflow adopted the format within the following code instance:

{
call_id: <name id>,
agent_id: <agent_id>
customer_id: <customer_id>
transcript: """
   Agent: <Agent message>.
   Buyer: <Buyer message>
   Agent: <Agent message>.
   Buyer: <Buyer message>
   Agent: <Agent message>.
   Buyer: <Buyer message>
   ...........
    """
}

Metadata was generated utilizing Amazon Bedrock. Particularly, it extracted the abstract, root trigger, subject, and subsequent steps, and answered key questions similar to if the decision was a callback and if the problem was in the end resolved.

Prompts had been saved in Amazon DynamoDB, permitting Asure to rapidly modify prompts or add new generative AI-powered fields based mostly on future enhancements. The next screenshot exhibits how prompts might be modified by way of DynamoDB.

Particular person name Q&A

The chat assistant powered by Anthropic’s Claude Haiku was used to reply pure language queries on a single transcript. This assistant, the decision metadata values generated from the earlier part, and sentiments generated from Amazon Comprehend had been displayed in an software hosted by CloudFront.

The consumer of the ultimate chat assistant can modify the immediate in DynamoDB. The next screenshot exhibits the overall immediate for a person name Q&A.

The UI hosted by CloudFront permits an agent or supervisor to investigate a particular name to extract extra particulars. The next screenshot exhibits the insights Asure gleaned for a pattern customer support name.

The next screenshot exhibits the chat assistant, which exists in the identical webpage.

Analysis Framework

This part outlines parts of the analysis framework used. It in the end permits Asure to spotlight vital metrics for his or her use case and gives visibility into the generative AI software’s strengths and weaknesses. This was achieved utilizing automated quantitative metrics supplied by Ragas, DeepEval, and conventional ML metrics in addition to human-in-the-loop analysis achieved by SMEs.

Quantitative Metrics

The outcomes of the generated name metadata and particular person name Q&A had been evaluated utilizing quantitative metrics supplied by Ragas: reply correctness, reply relevance, and faithfulness; and DeepEval: alignment and protection, each powered by FMs from Amazon Bedrock. Its easy integration with exterior libraries allowed Amazon Bedrock to be configured with present libraries. As well as, conventional ML metrics had been used for “Sure/No” solutions. The next are the metrics used for various parts of the answer:

• Name metadata era – This included the next:
  • Abstract – Alignment and protection (discover a description of those metrics within the DeepEval repository) and reply correctness
  • Subject resolved, callback – F1-score and accuracy
  • Subject, subsequent steps, root trigger – Reply correctness, reply relevance, and faithfulness
• Particular person name Q&A – Reply correctness, reply relevance, and faithfulness
• Human within the loop – Each particular person name Q&A and name metadata era used human-in-the-loop metrics

For an outline of reply correctness, reply relevance, and faithfulness, check with the shopper blog post with Tealium.

Using Amazon Bedrock within the analysis framework allowed for a flexibility of various fashions based mostly on totally different use instances. For instance, Anthropic’s Claude Sonnet 3.5 was used to generate DeepEval metrics, whereas Anthropic’s Claude 3 Haiku (with its low latency) was preferrred for Ragas.

Human within the Loop

The human-in-the-loop UI is described within the Human-in-the-Loop part of the shopper blog post with Tealium. To make use of it to guage this answer, some modifications needed to be made:

• There’s a selection for the consumer to investigate one of many generated metadata fields for a name (similar to a abstract) or a particular Q&A pair.
• The consumer can usher in two mannequin outputs for comparability. This may embrace outputs from the identical FMs however utilizing totally different prompts, outputs from totally different FMs however utilizing the identical immediate, and outputs from totally different FMs and utilizing totally different prompts.
• Extra checks for fluency, coherence, creativity, toxicity, relevance, completeness, and total high quality had been added, the place the consumer provides in a measure of this metric based mostly on the mannequin output from a variety of 0–4.

The next screenshots present the UI.

The human-in-the-loop system establishes a mechanism between area experience and Amazon Bedrock outputs. This in flip will result in improved generative AI purposes and in the end to excessive buyer belief of such methods.

To demo the human-in-the-loop UI, comply with the directions within the GitHub repo.

Pure Language Q&A utilizing Amazon Q in Quicksight

QuickSight, built-in with Amazon Q, enabled Asure to make use of pure language queries for complete buyer analytics. By decoding queries on sentiments, name volumes, difficulty resolutions, and agent efficiency, the service delivered data-driven visualizations. This empowered Asure to rapidly determine ache factors, optimize operations, and ship distinctive buyer experiences by way of a streamlined, scalable analytics answer tailor-made for name heart operations.

Combine Amazon Q in QuickSight with the PCA answer

The Amazon Q in QuickSight integration was achieved by following three high-level steps:

1. Create a dataset on QuickSight.
2. Create a subject on QuickSight from the dataset.
3. Question utilizing pure language.

Create a dataset on QuickSight

We used Athena as the information supply, which queries information from Amazon S3. QuickSight might be configured by way of a number of information sources (for extra data, check with Supported data sources). For this use case, we used the information generated from the PCA pipeline as the information supply for additional analytics and pure language queries in Amazon Q in QuickSight. The PCA pipeline shops information in Amazon S3, which might be queried in Athena, an interactive question service that permits you to analyze information straight in Amazon S3 utilizing normal SQL.

1. On the QuickSight console, select Datasets within the navigation pane.
2. Select Create new.
3. Select Athena as the information supply and enter the actual catalog, database, and desk that Amazon Q in QuickSight will reference.

Affirm the dataset was created efficiently and proceed to the subsequent step.

Create a subject on Amazon QuickSight from the dataset created

Customers can use matters in QuickSight, powered by Amazon Q integration, to carry out pure language queries on their information. This characteristic permits for intuitive information exploration and evaluation by posing questions in plain language, assuaging the necessity for advanced SQL queries or specialised technical expertise. Earlier than organising a subject, ensure that the customers have Professional degree entry. To arrange a subject, comply with these steps:

1. On the QuickSight console, select Matters within the navigation pane.
2. Select New subject.
3. Enter a reputation for the subject and select the information supply created.
4. Select the created subject after which select Open Q&A to begin querying in pure language

Question utilizing pure language

We carried out intuitive pure language queries to achieve actionable insights into buyer analytics. This functionality permits customers to investigate sentiments, name volumes, difficulty resolutions, and agent efficiency by way of conversational queries, enabling data-driven decision-making, operational optimization, and enhanced buyer experiences inside a scalable, name center-tailored analytics answer. Examples of the straightforward pure language queries “Which buyer had optimistic sentiments and a fancy question?” and “What are the commonest points and which brokers handled them?” are proven within the following screenshots.

These capabilities are useful when enterprise leaders wish to dive deep on a specific difficulty, empowering them to make knowledgeable selections on numerous points.

Success metrics

The first success metric gained from this answer is boosting worker productiveness, primarily by rapidly understanding buyer interactions from calls to uncover themes and tendencies whereas additionally figuring out gaps and ache factors of their merchandise. Earlier than the engagement, analysts had been taking 14 days to manually undergo every name transcript to retrieve insights. After engagement, Asure noticed how Amazon Bedrock and Amazon Q in QuickSight may cut back this time to minutes, even seconds, to acquire each insights queried straight from all saved name transcripts and visualizations that can be utilized for report era.

Within the pipeline, Anthropic’s Claude 3 Haiku was used to acquire preliminary name metadata fields (similar to abstract, root trigger, subsequent steps, and sentiments) that was saved in Athena. This allowed every name transcript to be queried utilizing pure language from Amazon Q in QuickSight, letting enterprise analysts reply high-level questions on points, themes, and buyer and agent insights in seconds.

Pat Goepel, chairman and CEO of Asure, shares,

“In collaboration with the AWS Generative AI Innovation Middle, we now have improved upon a post-call analytics answer to assist us determine and prioritize options that would be the most impactful for our clients. We’re using Amazon Bedrock, Amazon Comprehend, and Amazon Q in QuickSight to know tendencies in our personal buyer interactions, prioritize objects for product growth, and detect points sooner in order that we might be much more proactive in our assist for our clients. Our partnership with AWS and our dedication to be early adopters of revolutionary applied sciences like Amazon Bedrock underscore our dedication to creating superior HCM expertise accessible for companies of any measurement.”

Takeaways

We had the next takeaways:

• Enabling chain-of-thought reasoning and particular assistant prompts for every immediate within the name metadata era part and calling it utilizing Anthropic’s Claude 3 Haiku improved metadata era for every transcript. Primarily, the flexibleness of Amazon Bedrock in using numerous FMs allowed full experimentation of many forms of fashions with minimal modifications. Utilizing Amazon Bedrock can enable for using numerous fashions relying on the use case, making it the plain selection for this software because of its flexibility.
• Ragas metrics, significantly faithfulness, reply correctness, and reply relevance, had been used to guage name metadata era and particular person Q&A. Nonetheless, summarization required totally different metrics, alignment, and protection, which didn’t require floor reality summaries. Subsequently, DeepEval was used to calculate summarization metrics. Total, the convenience of integrating Amazon Bedrock allowed it to energy the calculation of quantitative metrics with minimal modifications to the analysis libraries. This additionally allowed using several types of fashions for various analysis libraries.
• The human-in-the-loop strategy can be utilized by SMEs to additional consider Amazon Bedrock outputs. There is a chance to enhance upon an Amazon Bedrock FM based mostly on this suggestions, however this was not labored on on this engagement.
• The post-call analytics workflow, with using Amazon Bedrock, might be iterated upon sooner or later utilizing options similar to Amazon Bedrock Knowledge Bases to carry out Q&A over a particular variety of name transcripts in addition to Amazon Bedrock Guardrails to detect dangerous and hallucinated responses whereas additionally creating extra accountable AI purposes.
• Amazon Q in QuickSight was in a position to reply pure language questions on buyer analytics, root trigger, and agent analytics, however some questions required reframing to get significant responses.
• Knowledge fields inside Amazon Q in QuickSight wanted to be outlined correctly and synonyms wanted to be added to make Amazon Q extra strong with pure language queries.

Safety finest practices

We suggest the next safety tips for constructing safe purposes on AWS:

Conclusion

On this submit, we showcased how Asure used the PCA answer powered by Amazon Bedrock and Amazon Q in QuickSight to generate shopper and agent insights each at particular person and mixture ranges. Particular insights included these centered round a typical theme or difficulty. With these companies, Asure was in a position to enhance worker productiveness to generate these insights in minutes as an alternative of weeks.

This is among the some ways builders can ship nice options utilizing Amazon Bedrock and Amazon Q in QuickSight. To be taught extra, check with Amazon Bedrock and Amazon Q in QuickSight.

Concerning the Authors

Suren Gunturu is a Knowledge Scientist working within the Generative AI Innovation Middle, the place he works with numerous AWS clients to resolve high-value enterprise issues. He focuses on constructing ML pipelines utilizing giant language fashions, primarily by way of Amazon Bedrock and different AWS Cloud companies.

Avinash Yadav is a Deep Studying Architect on the Generative AI Innovation Middle, the place he designs and implements cutting-edge GenAI options for numerous enterprise wants. He focuses on constructing ML pipelines utilizing giant language fashions, with experience in cloud structure, Infrastructure as Code (IaC), and automation. His focus lies in creating scalable, end-to-end purposes that leverage the ability of deep studying and cloud applied sciences.

John Canada is the VP of Engineering at Asure Software program, the place he leverages his expertise in constructing revolutionary, dependable, and performant options and his ardour for AI/ML to steer a gifted staff devoted to utilizing Machine Studying to reinforce the capabilities of Asure’s software program and meet the evolving wants of companies.

Yasmine Rodriguez Wakim is the Chief Expertise Officer at Asure Software program. She is an revolutionary Software program Architect & Product Chief with deep experience in creating payroll, tax, and workforce software program growth. As a results-driven tech strategist, she builds and leads expertise imaginative and prescient to ship environment friendly, dependable, and customer-centric software program that optimizes enterprise operations by way of automation.

Vidya Sagar Ravipati is a Science Supervisor on the Generative AI Innovation Middle, the place he leverages his huge expertise in large-scale distributed methods and his ardour for machine studying to assist AWS clients throughout totally different business verticals speed up their AI and cloud adoption.