Summarize name transcriptions securely with Amazon Transcribe and Amazon Bedrock Guardrails

Given the quantity of conferences, interviews, and buyer interactions in trendy enterprise environments, audio recordings play a vital function in capturing helpful data. Manually transcribing and summarizing these recordings generally is a time-consuming and tedious process. Happily, developments in generative AI and computerized speech recognition (ASR) have paved the way in which for automated options that may streamline this course of.

Customer support representatives obtain a excessive quantity of calls every day. Beforehand, calls had been recorded and manually reviewed later for compliance, rules, and firm insurance policies. Name recordings needed to be transcribed, summarized, after which redacted for private identifiable data (PII) earlier than analyzing calls, leading to delayed entry to insights.

Redacting PII is a vital observe in safety for a number of causes. Sustaining the privateness and safety of people’ private data is just not solely a matter of moral duty, but additionally a authorized requirement. On this submit, we present you the best way to use Amazon Transcribe to get close to real-time transcriptions of calls despatched to Amazon Bedrock for summarization and delicate information redaction. We’ll stroll by means of an structure that makes use of AWS Step Functions to orchestrate the method, offering seamless integration and environment friendly processing

Amazon Bedrock is a totally managed service that provides a selection of high-performing basis fashions (FMs) from main mannequin suppliers corresponding to AI21 Labs, Anthropic, Cohere, Meta, Stability AI, Mistral AI, and Amazon by means of a single API, together with a broad set of capabilities it is advisable to construct generative AI functions with safety, privateness, and accountable AI. You should utilize  Amazon Bedrock Guardrails to redact sensitive information such as PII discovered within the generated name transcription summaries. Clear, summarized transcripts are then despatched to analysts. This supplies faster entry to name developments whereas defending buyer privateness.

Answer overview

The structure of this answer is designed to be scalable, environment friendly, and compliant with privateness rules. It consists of the next key parts:

1. Recording – An audio file, corresponding to a gathering or assist name, to be transcribed and summarized
2. Step Capabilities workflow – Coordinates the transcription and summarization course of
3. Amazon Transcribe – Converts audio recordings into textual content
4. Amazon Bedrock – Summarizes the transcription and removes PII
5. Amazon SNS – Delivers the abstract to the designated recipient
6. Recipient – Receives the summarized, PII-redacted transcript

The next diagram reveals the structure overflow –

The workflow orchestrated by Step Capabilities is as follows:

1. An audio recording is offered as an enter to the Step Capabilities workflow. This might be performed manually or mechanically relying on the precise use case and integration necessities.
2. The workflow invokes Amazon Transcribe, which converts the multi-speaker audio recording right into a textual, speaker-partition transcription. Amazon Transcribe makes use of superior speech recognition algorithms and machine studying (ML) fashions to precisely partition audio system and transcribe the audio, dealing with varied accents, background noise, and different challenges.
3. The transcription output from Amazon Transcribe is then handed to Anthropic’s Claude 3 Haiku mannequin on Amazon Bedrock by means of AWS Lambda. This mannequin was chosen as a result of it has comparatively decrease latency and price than different fashions. The mannequin first summarizes the transcript in keeping with its abstract directions, after which the summarized output (the mannequin response) is evaluated by Amazon Bedrock Guardrails to redact PII. To study the way it blocks dangerous content material, consult with How Amazon Bedrock Guardrails works. The directions and transcript are each handed to the mannequin as context.
4. The output from Amazon Bedrock is saved in Amazon Simple Storage Service (Amazon S3) and despatched to the designated recipient utilizing Amazon Simple Notification Service (Amazon SNS). Amazon SNS helps varied supply channels, together with electronic mail, SMS, and cell push notifications, ensuring that the abstract reaches the meant recipient in a well timed and dependable method

The recipient can then evaluation the concise abstract, shortly greedy the important thing factors and insights from the unique audio recording. Moreover, delicate data has been redacted, sustaining privateness and compliance with related rules.

The next diagram reveals the Step Capabilities workflow –

Conditions

Comply with these steps earlier than beginning:

1. Amazon Bedrock customers must request entry to fashions earlier than they’re out there to be used. It is a one-time motion. For this answer, it is advisable to allow entry to Anthropic’s Claude 3 Haiku mannequin on Amazon Bedrock. For extra data, consult with Access Amazon Bedrock foundation models. Deployment, as described under, is at present supported solely within the US West (Oregon) us-west-2 AWS Region. Customers might discover different fashions if desired. You would possibly want some customizations to deploy to different Areas with completely different mannequin availability (corresponding to us-east-1, which hosts Anthropic’s Claude 3.5 Sonnet). Be sure to contemplate mannequin high quality, pace, and price tradeoffs earlier than selecting a mannequin.
2. Create a guardrail for PII redaction. Configure filters to dam or masks delicate data. This selection will be discovered on the Amazon Bedrock console on the Add delicate data filters web page when making a guardrail. To discover ways to configure filters for different use instances, consult with Remove PII from conversations by using sensitive information filters.

Deploy answer assets

To deploy the answer, download an AWS CloudFormation template to mechanically provision the required assets in your AWS account. The template units up the next parts:

• A Step Capabilities workflow
• Lambda capabilities
• An SNS matter
• An S3 bucket
• AWS Key Management Service (AWS KMS) keys for information encryption and decryption

By utilizing this template, you possibly can shortly deploy the pattern answer with minimal handbook configuration. The template requires the next parameters:

• E mail deal with used to ship abstract – The abstract will likely be despatched to this deal with. You could acknowledge the preliminary Amazon SNS affirmation electronic mail earlier than receiving extra notifications.
• Abstract directions – These are the directions given to the Amazon Bedrock mannequin to generate the abstract
• Guardrail ID – That is the ID of your not too long ago created guardrail, which will be discovered on the Amazon Bedrock Guardrails console in Guardrail overview

The Abstract directions are learn into your Lambda perform as an atmosphere variable.

 
# Use the offered directions to supply the abstract. Use a default if no intructions are offered.
SUMMARY_INSTRUCTIONS = os.getenv('SUMMARY_INSTRUCTIONS')
 
These are then used as a part of your payload to Anthropic’s Claude 3 Haiku mannequin. That is shared to offer you an understanding of the best way to move the directions and textual content to the mannequin.
 
# Create the payload to supply to the Anthropic mannequin.
        user_message = {"function": "person", "content material": f"{SUMMARY_INSTRUCTIONS}{transcript}"}
        messages = [user_message]
response = generate_message(bedrock_client, 'anthropic.claude-3-haiku-20240307-v1:0"', "", messages, 1000)
 
The generate_message() perform comprises the invocation to Amazon Bedrock with the guardrail ID and different related parameters.
 
def generate_message(bedrock_runtime, model_id, system_prompt, messages, max_tokens):
    physique = json.dumps(
        {
            "anthropic_version": "bedrock-2023-05-31",
            "max_tokens": max_tokens,
            "system": system_prompt,
            "messages": messages
        }
    )
print(f'Invoking mannequin: {BEDROCK_MODEL_ID}')
 
    response = bedrock_runtime.invoke_model(
        physique=physique,
        modelId=BEDROCK_MODEL_ID,
        # contentType=contentType,
        guardrailIdentifier =BEDROCK_GUARDRAIL_ID,
        guardrailVersion ="1",
        hint ="ENABLED")
    response_body = json.hundreds(response.get('physique').learn())
    print(f'response: {response}')
    return response_body

Deploy the answer

After you deploy the assets utilizing AWS CloudFormation, full these steps:

1. Add a Lambda layer.

Though AWS Lambda frequently updates the model of AWS Boto3 included, on the time of penning this submit, it nonetheless supplies model 1.34.126. To make use of Amazon Bedrock Guardrails, you want model 1.34.90 or increased, for which we’ll add a Lambda layer that updates the Boto3. You’ll be able to observe the official developer guide on the best way to add a Lambda layer.

There are other ways to create a Lambda layer. A easy methodology is to make use of the steps outlined in Packaging the layer content, which references a pattern utility repo. You must be capable to change requests==2.31.0 inside necessities.txt content material to boto3, which can set up the newest out there model, then create the layer.

So as to add the layer to Lambda, guarantee that the parameters laid out in Creating the layer match the deployed Lambda. That’s, it is advisable to replace compatible-architectures to x86_64.

2. Acknowledge the Amazon SNS electronic mail affirmation that you need to obtain a number of moments after creating the CloudFormation stack
3. On the AWS CloudFormation console, discover the stack you simply created
4. On the stack’s Outputs tab, search for the worth related to AssetBucketName. It should look one thing like summary-generator-assetbucket-xxxxxxxxxxxxx.
5. On the Amazon S3 console, discover your S3 property bucket.

That is the place you’ll add your recordings. Legitimate file codecs are MP3, MP4, WAV, FLAC, AMR, OGG, and WebM.

6. Add your recording to the recordings folder in Amazon S3

Importing recordings will mechanically set off the AWS Step Capabilities state machine. For this instance, we use a pattern crew assembly recording from the sample recording.

7. On the AWS Step Capabilities console, discover the summary-generator state machine. Select the identify of the state machine run with the standing Working.

Right here, you possibly can watch the progress of the state machine because it processes the recording. After it reaches its Success state, you need to obtain an emailed abstract of the recording. Alternatively, you possibly can navigate to the S3 property bucket and think about the transcript there within the transcripts folder.

Develop the answer

Now that you’ve a working answer, listed below are some potential concepts to customise the answer on your particular use instances:

• Attempt altering the method to suit your out there supply content material and desired outputs:
  • For conditions the place transcripts can be found, create an alternate AWS Step Capabilities workflow to ingest present text-based or PDF-based transcriptions
  • As an alternative of utilizing Amazon SNS to inform recipients by means of electronic mail, you need to use it to ship the output to a distinct endpoint, corresponding to a crew collaboration website or to the crew’s chat channel
• Attempt altering the abstract directions for the AWS CloudFormation stack parameter offered to Amazon Bedrock to provide outputs particular to your use case. The next are some examples:
  • When summarizing an organization’s earnings name, you might have the mannequin concentrate on potential promising alternatives, areas of concern, and issues that you need to proceed to observe
  • Should you’re utilizing the mannequin to summarize a course lecture, it might establish upcoming assignments, summarize key ideas, record info, and filter out small discuss from the recording
• For a similar recording, create completely different summaries for various audiences:
  • Engineers’ summaries concentrate on design choices, technical challenges, and upcoming deliverables
  • Venture managers’ summaries concentrate on timelines, prices, deliverables, and motion objects
  • Venture sponsors get a short replace on challenge standing and escalations
  • For longer recordings, strive producing summaries for various ranges of curiosity and time dedication. For instance, create a single sentence, single paragraph, single web page, or in-depth abstract. Along with the immediate, you would possibly wish to alter the max_tokens_to_sample parameter to accommodate completely different content material lengths.

Clear up

Clear up the assets you created for this answer to keep away from incurring prices. You should utilize an AWS SDK, the AWS Command Line Interface (AWS CLI), or the console.

1. Delete Amazon Bedrock Guardrails and the Lambda layer you created
2. Delete the CloudFormation stack

To make use of the console, observe these steps:

3. On the Amazon Bedrock console, within the navigation menu, choose Guardrails. Select your guardrail, then choose Delete.
4. On the AWS Lambda console, within the navigation menu, choose Layers. Select your layer, then choose Delete.
5. On the AWS CloudFormation console, within the navigation menu, choose Stacks. Select the stack you created, then choose Delete.

Deleting the stack received’t delete the related S3 bucket. Should you not require the recordings or transcripts, you possibly can delete the bucket individually. Amazon Transcribe is designed to mechanically delete transcription jobs after 90 days. Nevertheless, you possibly can decide to manually delete these jobs earlier than the 90-day retention interval expires.

Conclusion

As companies flip to information as a basis for decision-making, being able to effectively extract insights from audio recordings is invaluable. By utilizing the facility of generative AI with Amazon Bedrock and Amazon Transcribe, your group can create concise summaries of audio recordings whereas sustaining privateness and compliance. The proposed structure demonstrates how AWS companies will be orchestrated utilizing AWS Step Capabilities to streamline and automate complicated workflows, enabling organizations to concentrate on their core enterprise actions.

This answer not solely saves effort and time, but additionally makes positive that delicate data is redacted, mitigating potential dangers and selling compliance with information safety rules. As organizations proceed to generate and course of giant volumes of audio information, options like it will change into more and more necessary for gaining insights, making knowledgeable choices, and sustaining a aggressive edge.

Concerning the authors

Yash Yamsanwar is a Machine Studying Architect at Amazon Internet Providers (AWS). He’s accountable for designing high-performance, scalable machine studying infrastructure that optimizes the complete lifecycle of machine studying fashions, from coaching to deployment. Yash collaborates intently with ML analysis groups to push the boundaries of what’s doable with LLMs and different cutting-edge machine studying applied sciences.

Sawyer Hirt is a Options Architect at AWS, specializing in AI/ML and cloud architectures, with a ardour for serving to companies leverage cutting-edge applied sciences to beat complicated challenges. His experience lies in designing and optimizing ML workflows, enhancing system efficiency, and making superior AI options extra accessible and cost-effective, with a selected concentrate on Generative AI. Exterior of labor, Sawyer enjoys touring, spending time with household, and staying present with the newest developments in cloud computing and synthetic intelligence.