Create summaries of recordings utilizing generative AI with Amazon Bedrock and Amazon Transcribe

Assembly notes are a vital a part of collaboration, but they typically fall by way of the cracks. Between main discussions, listening intently, and typing notes, it’s straightforward for key data to slide away unrecorded. Even when notes are captured, they are often disorganized or illegible, rendering them ineffective.

On this put up, we discover methods to use Amazon Transcribe and Amazon Bedrock to routinely generate clear, concise summaries of video or audio recordings. Whether or not it’s an inner workforce assembly, convention session, or earnings name, this method may help you distill hours of content material right down to salient factors.

We stroll by way of an answer to transcribe a challenge workforce assembly and summarize the important thing takeaways with Amazon Bedrock. We additionally talk about how one can customise this answer for different frequent situations like course lectures, interviews, and gross sales calls. Learn on to simplify and automate your note-taking course of.

Resolution overview

By combining Amazon Transcribe and Amazon Bedrock, it can save you time, seize insights, and improve collaboration. Amazon Transcribe is an automated speech recognition (ASR) service that makes it easy so as to add speech-to-text functionality to functions. It makes use of superior deep studying applied sciences to precisely transcribe audio into textual content. Amazon Bedrock is a totally managed service that provides a selection of high-performing basis fashions (FMs) from main AI firms like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon with a single API, together with a broad set of capabilities you want to construct generative AI functions. With Amazon Bedrock, you possibly can simply experiment with quite a lot of prime FMs, and privately customise them together with your knowledge utilizing methods resembling fine-tuning and Retrieval Augmented Era (RAG).

The answer offered on this put up is orchestrated utilizing an AWS Step Functions state machine that’s triggered once you add a recording to the designated Amazon Simple Storage Service (Amazon S3) bucket. Step Features enables you to create serverless workflows to orchestrate and join elements throughout AWS companies. It handles the underlying complexity so you possibly can give attention to software logic. It’s helpful for coordinating duties, distributed processing, ETL (extract, rework, and cargo), and enterprise course of automation.

The next diagram illustrates the high-level answer structure.

The answer workflow contains the next steps:

1. A consumer shops a recording within the S3 asset bucket.
2. This motion triggers the Step Features transcription and summarization state machine.
3. As a part of the state machine, an AWS Lambda operate is triggered, which transcribes the recording utilizing Amazon Transcribe and shops the transcription within the asset bucket.
4. A second Lambda operate retrieves the transcription and generates a abstract utilizing the Anthropic Claude mannequin in Amazon Bedrock.
5. Lastly, a closing Lambda operate makes use of Amazon Simple Notification Service (Amazon SNS) to ship a abstract of the recording to the recipient.

This answer is supported in Areas the place Anthropic Claude on Amazon Bedrock is available.

The state machine orchestrates the steps to carry out the precise duties. The next diagram illustrates the detailed course of.

Stipulations

Amazon Bedrock customers must request entry to fashions earlier than they’re accessible to be used. This can be a one-time motion. For this answer, you’ll must allow entry to the Anthropic Claude (not Anthropic Claude Instantaneous) mannequin in Amazon Bedrock. For extra data, consult with Model access.

Deploy answer assets

The answer is deployed utilizing an AWS CloudFormation template, discovered on the GitHub repo, to routinely provision the required assets in your AWS account. The template requires the next parameters:

• E mail handle used to ship abstract – The abstract shall be despatched to this handle. You need to acknowledge the preliminary Amazon SNS affirmation electronic mail earlier than receiving extra notifications.
• Abstract directions – These are the directions given to the Amazon Bedrock mannequin to generate the abstract.

Run the answer

After you deploy the answer utilizing AWS CloudFormation, full the next steps:

1. Acknowledge the Amazon SNS electronic mail affirmation that you must obtain a couple of moments after creating the CloudFormation stack.
2. On the AWS CloudFormation console, navigate to stack you simply created.
3. On the stack’s Outputs tab, and search for the worth related to AssetBucketName; it’ll look one thing like summary-generator-assetbucket-xxxxxxxxxxxxx.
4. On the Amazon S3 console, navigate to your asset bucket.

That is the place you’ll add your recordings. Legitimate file codecs are MP3, MP4, WAV, FLAC, AMR, OGG, and WebM.

5. Add your recording to the recordings folder.

Importing recordings will routinely set off the Step Features state machine. For this instance, we use a pattern workforce assembly recording within the sample-recording listing of the GitHub repository.

6. On the Step Features console, navigate to the summary-generator state machine.
7. Select the title of the state machine run with the standing Working.

Right here, you possibly can watch the progress of the state machine because it processes the recording.

8. After it reaches its Success state, you must obtain an emailed abstract of the recording.

Alternatively, you possibly can navigate to the S3 belongings bucket and examine the transcript there within the transcripts folder.

Overview the abstract

You’ll get the recording abstract emailed to the handle you offered once you created the CloudFormation stack. Should you don’t obtain the e-mail in a couple of moments, just be sure you acknowledged the Amazon SNS affirmation electronic mail that you must have obtained after you created the stack after which add the recording once more, which can set off the abstract course of.

This answer features a mock workforce assembly recording that you should use to check the answer. The abstract will look much like the next instance. Due to the character of generative AI, nonetheless, your output will look a bit totally different, however the content material needs to be shut.

Listed below are the important thing factors from the standup:

• Joe completed reviewing the present state for activity EDU1 and created a brand new activity to develop the longer term state. That new activity is within the backlog to be prioritized. He’s now beginning EDU2 however is blocked on useful resource choice.
• Rob created a tagging technique for SLG1 based mostly on finest practices, however might must coordinate with different groups who’ve created their very own methods, to align on a uniform method. A brand new activity was created to coordinate tagging methods.
• Rob has made progress debugging for SLG2 however may have extra assist. This activity shall be moved to Dash 2 to permit time to get further assets.

Subsequent Steps:

• Joe to proceed engaged on EDU2 as ready till useful resource choice is determined
• New activity to be prioritized to coordinate tagging methods throughout groups
• SLG2 moved to Dash 2
• Standups transferring to Mondays beginning subsequent week

Increase the answer

Now that you’ve got a working answer, listed here are some potential concepts to customise the answer in your particular use instances:

• Strive altering the method to suit your accessible supply content material and desired outputs:
  • For conditions the place transcripts can be found, create an alternate Step Features workflow to ingest current text-based or PDF-based transcriptions.
  • As an alternative of utilizing Amazon SNS to inform recipients through electronic mail, you should use it to ship the output to a distinct endpoint, resembling a workforce collaboration website, or to the workforce’s chat channel.
• Strive altering the abstract directions CloudFormation stack parameter offered to Amazon Bedrock to supply outputs particular to your use case (that is the generative AI immediate):
  • When summarizing an organization’s earnings name, you would have the mannequin give attention to potential promising alternatives, areas of concern, and issues that you must proceed to observe.
  • In case you are utilizing this to summarize a course lecture, the mannequin may determine upcoming assignments, summarize key ideas, record details, and filter out any small discuss from the recording.
• For a similar recording, create totally different summaries for various audiences:
  • Engineers’ summaries give attention to design choices, technical challenges, and upcoming deliverables.
  • Mission managers’ summaries give attention to timelines, prices, deliverables, and motion gadgets.
  • Mission sponsors get a quick replace on challenge standing and escalations.
  • For longer recordings, attempt producing summaries for various ranges of curiosity and time dedication. For instance, create a single sentence, single paragraph, single web page, or in-depth abstract. Along with the immediate, chances are you’ll wish to alter the max_tokens_to_sample parameter to accommodate totally different content material lengths.

Clear up

To scrub up the answer, delete the CloudFormation stack that you just created earlier. Word that deleting the stack is not going to delete the asset bucket. Should you not want the recordings or transcripts, you possibly can delete this bucket individually. Amazon Transcribe will routinely delete transcription jobs after 90 days, however you possibly can delete these manually earlier than then.

Conclusion

On this put up, we explored methods to use Amazon Transcribe and Amazon Bedrock to routinely generate clear, concise summaries of video or audio recordings. We encourage you to proceed evaluating Amazon Bedrock, Amazon Transcribe, and different AWS AI companies, like Amazon Textract, Amazon Translate, and Amazon Rekognition, to see how they may help meet your small business targets.

In regards to the Authors

Rob Barnes is a principal guide for AWS Skilled Companies. He works with our prospects to handle safety and compliance necessities at scale in complicated, multi-account AWS environments by way of automation.

Jason Stehle is a Senior Options Architect at AWS, based mostly within the New England space. He works with prospects to align AWS capabilities with their best enterprise challenges. Outdoors of labor, he spends his time constructing issues and watching comedian guide motion pictures along with his household.