New technical deep dive course: Generative AI Foundations on AWS

Generative AI Foundations on AWS is a brand new technical deep dive course that offers you the conceptual fundamentals, sensible recommendation, and hands-on steerage to pre-train, fine-tune, and deploy state-of-the-art basis fashions on AWS and past. Developed by AWS generative AI worldwide foundations lead Emily Webber, this free hands-on course and the supporting GitHub supply code launched through AWS Youtube. If you’re searching for a curated playlist of the highest sources, ideas, and steerage to stand up to hurry on basis fashions, and particularly people who unlock generative capabilities in your information science and machine studying tasks, then look no additional.

Throughout this 8-hour deep dive, you may be launched to the important thing strategies, companies, and tendencies that can assist you perceive basis fashions from the bottom up. This implies breaking down concept, arithmetic, and summary ideas mixed with hands-on workouts to achieve useful instinct for sensible software. All through the course, we deal with a large spectrum of progressively complicated generative AI strategies, providing you with a robust base to grasp, design, and apply your individual fashions for the most effective efficiency. We’ll begin with recapping basis fashions, understanding the place they arrive from, how they work, how they relate to generative AI, and what you may to do customise them. You’ll then study selecting the correct basis mannequin to fit your use case.

When you’ve developed a robust contextual understanding of basis fashions and the right way to use them, you’ll be launched to the core topic of this course: pre-training new basis fashions. You’ll be taught why you’d wish to do that in addition to how and the place it’s aggressive. You’ll even learn to use the scaling legal guidelines to select the fitting mannequin, dataset, and compute sizes. We’ll cowl getting ready coaching datasets at scale on AWS, together with selecting the correct situations and storage strategies. We’ll cowl fine-tuning your basis fashions, evaluating latest strategies, and understanding the right way to run these along with your scripts and fashions. We’ll dive into reinforcement studying with human suggestions, exploring the right way to use it skillfully and at scale to really maximize your basis mannequin efficiency.

Lastly, you’ll learn to apply concept to manufacturing by deploying your new basis mannequin on Amazon SageMaker, together with throughout a number of GPUs and utilizing high design patterns like retrieval augmented era and chained dialogue. As an added bonus, we’ll stroll you thru a Steady Diffusion deep dive, immediate engineering greatest practices, standing up LangChain, and extra.

Extra of a reader than a video client? You may try my 15-chapter e book “Pretrain Imaginative and prescient and Massive Language Fashions in Python: Finish-to-end strategies for constructing and deploying basis fashions on AWS,” which launched Could 31, 2023, with Packt publishing and is on the market now on Amazon. Wish to leap proper into the code? I’m with you—each video begins with a 45-minute overview of the important thing ideas and visuals. Then I’ll provide you with a 15-minute walkthrough of the hands-on portion. All the instance notebooks and supporting code will ship in a public repository, which you should use to step by by yourself. Be at liberty to achieve out to me on Medium, LinkedIn, GitHub, or by your AWS groups. Be taught extra about generative AI on AWS.

Completely happy trails!

Course define

1. Introduction to Basis Fashions

  • What are giant language fashions and the way do they work?
  • The place do they arrive from?
  • What are different kinds of generative AI?
  • How do you customise a basis mannequin?
  • How do you consider a Generative mannequin?
  • Arms-on stroll by: Basis Fashions on SageMaker

Lesson 1 slides

Lesson 1 hands-on demo resources

2. Choosing the right basis mannequin

  • Why beginning with the fitting basis mannequin issues
  • Contemplating dimension
  • Contemplating accuracy
  • Contemplating licensing
  • Contemplating earlier examples of this mannequin working effectively in your trade
    • Contemplating exterior benchmarks

Lesson 2 slides

Lesson 2 hands-on demo resources

3. Utilizing pretrained basis fashions: immediate engineering and fine-tuning

  • The advantages of beginning with a pre-trained basis mannequin
  • Immediate engineering:
    • Zero-shot
    • Single-shot
    • Few-shot
    • Summarization
    • Translation
  • Tremendous-tuning
    • Traditional fine-tuning
    • Parameter environment friendly fine-tuning
    • Hugging Face’s new library
    • Arms-on stroll by: immediate engineering and fine-tuning on SageMaker

Lesson 3 slides

Lesson 3 hands-on demo resources

4. Pretraining a brand new basis mannequin

  • Why would you need or have to create a brand new basis mannequin?
    • Evaluating pretraining to fine-tuning
  • Getting ready your dataset for pretraining
  • Distributed coaching on SageMaker: libraries, scripts, jobs, sources
  • Why and the right way to adapt a brand new script to SageMaker distributed coaching

Lesson 4 slides

Lesson 4 hands-on demo resources

5. Getting ready information and coaching at scale

  • Choices for prepping information at scale on AWS
  • Clarify SageMaker job parallelism on CPU situations
  • Clarify modes of sending information to SageMaker Coaching
  • Introduction to FSx for Lustre
  • Utilizing FSx for Lustre at scale for SageMaker Coaching
  • Arms-on stroll by: configuring Lustre for SageMaker Coaching

Lesson 5 slides

Lesson 5 hands-on demo resources

6. Reinforcement studying with human suggestions

  • What’s this method and why can we care about it
  • The way it will get round issues with subjectivity and objectivity by rating human preferences at scale
  • How does it work?
  • How to do that with SageMaker Floor Fact
  • Up to date reward modeling
  • Arms-on stroll by: RLFH on SageMaker

Lesson 6 slides

Lesson 6 hands-on demo resources

7. Deploying a basis mannequin

  • Why can we wish to deploy fashions?
  • Completely different choices for deploying FM’s on AWS
  • How one can optimize your mannequin for deployment
  • Massive mannequin deployment container deep dive
  • Prime configuration ideas for deploying FM’s on SageMaker
  • Immediate engineering ideas for invoking basis fashions
  • Utilizing retrieval augmented era to mitigate hallucinations
  • Arms-on stroll by: Deploying an FM on SageMaker

Lesson 7 slides

Lesson 7 hands-on demo resources

Concerning the creator

Emily Webber joined AWS simply after SageMaker launched, and has been attempting to inform the world about it ever since! Outdoors of constructing new ML experiences for purchasers, Emily enjoys meditating and learning Tibetan Buddhism.

Leave a Reply

Your email address will not be published. Required fields are marked *