Superior fine-tuning methods for multi-agent orchestration: Patterns from Amazon at scale

Our work with massive enterprise clients and Amazon groups has revealed that top stakes use circumstances proceed to learn considerably from superior massive language mannequin (LLM) fine-tuning and post-training methods. On this submit, we present you the way fine-tuning enabled a 33% discount in harmful remedy errors (Amazon Pharmacy), engineering 80% human effort discount (Amazon World Engineering Companies), and content material high quality assessments enhancing 77% to 96% accuracy (Amazon A+). These aren’t hypothetical projections—they’re manufacturing outcomes from Amazon groups. Whereas many use circumstances might be successfully addressed by way of immediate engineering, Retrieval Augmented Technology (RAG) techniques, and switch key agent deployment,, our work with Amazon and huge enterprise accounts reveals a constant sample: One in 4 high-stakes functions—the place affected person security, operational effectivity, or buyer belief are on the road—demand superior fine-tuning and post-training methods to realize production-grade efficiency.

This submit particulars the methods behind these outcomes: from foundational strategies like Supervised Fine-Tuning (SFT) (instruction tuning), and Proximal Policy Optimization (PPO), to Direct Preference Optimization (DPO) for human alignment, to cutting-edge reasoning optimizations akin to Grouped-based Reinforcement Learning from Policy Optimization (GRPO), Direct Advantage Policy Optimization (DAPO), and Group Sequence Policy Optimization (GSPO) purpose-built for agentic techniques. We stroll by way of the technical evolution of every strategy, look at real-world implementations at Amazon, current a reference structure on Amazon Web Services (AWS), and supply a call framework for choosing the precise approach primarily based in your use case necessities.

The continued relevance of fine-tuning within the agentic AI

Regardless of the rising capabilities of basis fashions and agent frameworks, roughly one among 4 enterprise use circumstances nonetheless require superior fine-tuning to realize the mandatory efficiency ranges. These are sometimes situations the place the stakes are excessive from income or buyer belief views, domain-specific data is important, enterprise integration at scale is required, governance and management are paramount, enterprise course of integration is complicated, or multi-modal assist is required. Organizations pursuing these use circumstances have reported increased conversion to manufacturing, better return on funding (ROI), and as much as 3-fold year-over-year development when superior fine-tuning is appropriately utilized.

Evolution of LLM fine-tuning methods for agentic AI

The evolution of generative AI has seen a number of key developments in mannequin customization and efficiency optimization methods. Beginning with SFT, which makes use of labeled knowledge to show fashions to observe particular directions, the sphere established its basis however confronted limitations in optimizing complicated reasoning. To deal with these limitations, reinforcement learning (RL) refines the SFT course of with a reward-based system that gives higher adaptability and alignment with human desire. Amongst a number of RL algorithms, a major leap comes with PPO, which consists of a workflow with a price (critic) community and a coverage community. The workflow comprises a reinforcement studying coverage to regulate the LLM weights primarily based on the steering of a reward mannequin. PPO scales properly in complicated environments, although it has challenges with stability and configuration complexity.

DPO emerged as a breakthrough in early 2024, addressing PPO’s stability points by eliminating the express reward mannequin and as a substitute working immediately with desire knowledge that features most popular and rejected responses for given prompts. DPO optimizes the LLM weights by evaluating the popular and rejected responses, permitting the LLM to study and regulate its habits accordingly. This simplified strategy gained widespread adoption, with main language fashions incorporating DPO into their coaching pipelines to realize higher efficiency and extra dependable outputs. Different alternate options together with Odds Ratio Policy Optimization (ORPO), Relative Preference Optimization (RPO), Identity preference optimization (IPO), Kahneman-Tversky Optimization (KTO), they’re all RL strategies for human desire alignment. By incorporating comparative and identity-based desire constructions, and grounding optimization in behavioral economics, these strategies are computationally environment friendly, interpretable, and aligned with precise human decision-making processes.

As agent-based functions gained prominence in 2025, we noticed rising calls for for customizing the reasoning mannequin in brokers, to encode domain-specific constraints, security tips, and reasoning patterns that align with brokers’ meant features (job planning, software use, or multi-step drawback fixing). The target is to enhance brokers’ efficiency in sustaining coherent plans, avoiding logical contradictions, and making acceptable choices for the area particular use circumstances. To satisfy these wants, GRPO was launched to boost reasoning capabilities and have become notably notable for its implementation in DeepSeek-V1.

The core innovation of GRPO lies in its group-based comparability strategy: somewhat than evaluating particular person responses in opposition to a hard and fast reference, GRPO generates teams of responses and evaluates every in opposition to the common rating of the group, rewarding these performing above common whereas penalizing these under. This relative comparability mechanism creates a aggressive dynamic that encourages the mannequin to supply higher-quality reasoning. GRPO is especially efficient for enhancing chain-of-thought (CoT) reasoning, which is the vital basis for agent planning and sophisticated job decomposition. By optimizing on the group stage, GRPO captures the inherent variability in reasoning processes and trains the mannequin to constantly outperform its personal common efficiency.

Some complicated agent duties would possibly require extra fine-grained and crisp corrections inside lengthy reasoning chains, DAPO addresses these use circumstances by constructing upon GRPO sequence-level rewards, using a better clip ratio (roughly 30% increased than GRPO) to encourage extra various and exploratory pondering processes, implementing dynamic sampling to get rid of much less significant samples and enhance total coaching effectivity, making use of token-level coverage gradient loss to supply extra granular suggestions on prolonged reasoning chains somewhat than treating whole sequences as monolithic items, and incorporating overlong reward shaping to discourage excessively verbose responses that waste computational assets. Moreover, when the agentic use circumstances require lengthy textual content outputs within the Mixture-of-Experts (MoE) mannequin coaching, GSPO helps these situations by shifting the optimization from GRPO’s token-level significance weights to the sequence stage. With these enhancements, the brand new strategies (DAPO and GSPO) allow extra environment friendly and complex agent reasoning and planning technique, whereas sustaining computational effectivity and acceptable suggestions decision of GRPO.

Actual-world functions at Amazon

Utilizing the fine-tuning methods described within the earlier sections, the post-trained LLMs play two essential roles in agentic AI techniques. First is within the growth of specialised tool-using parts and sub-agents throughout the broader agent structure. These fine-tuned fashions act as area specialists, every optimized for particular features. By incorporating domain-specific data and constraints throughout the fine-tuning course of, these specialised parts can obtain considerably increased accuracy and reliability of their designated duties in comparison with general-purpose fashions. The second key utility is to function the core reasoning engine, the place the muse fashions are particularly tuned to excel at planning, logical reasoning, and decision-making, for brokers in a extremely particular area. The purpose is to enhance the mannequin’s capability to keep up coherent plans and make logically sound choices—important capabilities for any agent system. This twin strategy, combining a fine-tuned reasoning core with specialised sub-components, was rising as a promising structure in Amazon for evolving from LLM-driven functions to agentic techniques, and constructing extra succesful and dependable generative AI functions. The next desk depicts multi-agent AI orchestration with of superior fine-tuning approach examples.

Amazon Healthcare Companies (AHS) started its journey with generative AI with a major problem two years in the past, when the staff tackled customer support effectivity by way of a RAG-based Q&A system. Preliminary makes an attempt utilizing conventional RAG with basis fashions yielded disappointing outcomes, with accuracy hovering between 60 and 70%. The breakthrough got here after they fine-tuned the embedding mannequin particularly for pharmaceutical area data, resulted in a major enchancment to 90% accuracy and an 11% discount in buyer assist contacts. In remedy security, remedy route errors can pose critical security dangers and value as much as $3.5 billion yearly to appropriate. By fine-tuning a mannequin with 1000’s of expert-annotated examples, Amazon Pharmacy created an agent element that validates remedy instructions utilizing pharmacy logic and security tips. This lowered near-miss occasions by 33%, as indicated of their Nature Medicine publication. In 2025, AHS is increasing their AI capabilities and rework these separate LLM-driven functions right into a holistic multi-agent system to boost affected person expertise. These particular person functions pushed by fine-tuned fashions play a vital position within the total agentic structure, serving as area knowledgeable instruments to deal with particular mission-critical features in pharmaceutical companies.

The Amazon World Engineering Companies (GES) staff, liable for overseeing lots of of Amazon achievement facilities worldwide, launched into an bold journey to make use of generative AI of their operations. Their preliminary foray into this expertise targeted on creating a complicated Q&A system designed to help engineers in effectively accessing related design info from huge data repositories. The team’s approach was fine-tuning a basis mannequin utilizing SFT, which resulted in a major enchancment in accuracy (measured by semantic similarity rating) from 0.64 to 0.81. To raised align with the suggestions from the subject material specialists (SMEs), the staff further refined the model utilizing PPO incorporating the human suggestions knowledge, which boosted the LLM-judge scores from 3.9 to 4.2 out of 5, a exceptional achievement that translated to a considerable 80% discount within the effort required from the area specialists. Much like the Amazon Pharmacy case, these fine-tuned specialised fashions will proceed to operate as area knowledgeable instruments throughout the broader agentic AI system.

In 2025, the GES staff ventured into uncharted territory by making use of agentic AI techniques to optimize their enterprise course of. LLM fine-tuning methodologies represent a vital mechanism for enhancing the reasoning capabilities in AI brokers, enabling efficient decomposition of complicated targets into executable motion sequences that align with predefined behavioral constraints and goal-oriented outcomes. It additionally serves as vital structure element in facilitating specialised job execution and optimizing for task-specific efficiency metrics.

Amazon A+ Content material powers wealthy product pages throughout lots of of tens of millions of annual submissions. The A+ staff wanted to judge content material high quality at scale—assessing cohesiveness, consistency, and relevancy, not simply surface-level defects. Content material high quality immediately impacts conversion and model belief, making this a high-stakes utility.

Following the architectural sample seen in Amazon Pharmacy and World Engineering Companies, the staff constructed a specialised analysis agent powered by a fine-tuned mannequin. They utilized feature-based fine-tuning to Nova Lite on Amazon SageMaker—coaching a light-weight classifier on imaginative and prescient language mannequin (VLM)-extracted options somewhat than updating full mannequin parameters. This strategy, enhanced by expert-crafted rubric prompts, improved classification accuracy from 77% to 96%. The end result: an AI agent that evaluates tens of millions of content material submissions and delivers actionable suggestions. This demonstrates a key precept from our maturity framework—approach complexity ought to match job necessities. The A+ use case, whereas high-stakes and working at large scale, is basically a classification job well-suited to those strategies. Not each agent element requires GRPO or DAPO; deciding on the precise approach for every drawback is what delivers environment friendly, production-grade techniques.

Reference structure for superior AI orchestration utilizing fine-tuning

Though fine-tuned fashions serve various functions throughout totally different domains and use circumstances in an agentic AI system, the anatomy of an agent stays largely constant and might be encompassed in element groupings, as proven within the following structure diagram.

This modular strategy adopts plenty of AWS generative AI companies, together with Amazon Bedrock AgentCore, Amazon SageMaker, and Amazon Bedrock, that maintains construction of key groupings that make up an agent whereas offering numerous choices inside every group to enhance an AI agent.

1. LLM customization for AI brokers

Builders can use numerous AWS companies to fine-tune and post-train the LLMs for an AI agent utilizing the methods mentioned within the earlier part. If you happen to use LLMs on Amazon Bedrock to your brokers, you should utilize multiple model customization approaches to fine-tune your fashions. Distillation and SFT by way of parameter-efficient fine-tuning (PEFT) with low-rank adaptation (LoRA) can be utilized to deal with easy customization duties. For superior fine-tuning, Continued Pre-training (CPT) extends a basis mannequin’s data by coaching on domain-specific corpora (medical literature, authorized paperwork, or proprietary technical content material), embedding specialised vocabulary and area reasoning patterns immediately into mannequin weights. Reinforcement fine-tuning (RFT), launched at re:Invent 2025, teaches fashions to know what makes a high quality response with out massive quantities of pre-labeled coaching knowledge. There are two approaches supported for RFT: Reinforcement Studying with Verifiable Rewards (RLVR) makes use of rule-based graders for goal duties like code era or math reasoning, whereas Reinforcement Studying from AI Suggestions (RLAIF) makes use of AI-based judges for subjective duties like instruction following or content material moderation.

If you happen to require deeper management over mannequin customization infrastructure to your AI brokers, Amazon SageMaker AI supplies a complete platform for customized mannequin growth and fine-tuning. Amazon SageMaker JumpStart accelerates the customization journey by providing pre-built options with one-click deployment of common basis fashions (Llama, Mistral, Falcon, and others) and end-to-end fine-tuning notebooks that deal with knowledge preparation, coaching configuration, and deployment workflows. Amazon SageMaker Training jobs present managed infrastructure for executing customized fine-tuning workflows, routinely provisioning GPU situations, managing coaching execution, and dealing with cleanup after completion. This strategy fits most fine-tuning situations the place customary occasion configurations present ample compute energy and coaching completes reliably throughout the job length limits. You should use SageMaker Coaching jobs with custom Docker containers and code dependencies housing any machine studying (ML) framework, coaching library, or optimization approach, enabling experimentation with rising strategies past managed choices.

At re:Invent 2025, Amazon SageMaker HyperPod launched two capabilities for large-scale mannequin customization: Checkpointless training reduces checkpoint-restart cycles, shortening restoration time from hours to minutes. Elastic training routinely scales workloads to make use of idle capability and yields assets when higher-priority workloads peak. These options construct on the core strengths of HyperPod—resilient distributed coaching clusters with automated fault restoration for multi-week jobs spanning 1000’s of GPUs. HyperPod helps NVIDIA NeMo and AWS Neuronx frameworks, and is right when coaching scale, length, or reliability necessities exceed what job-based infrastructure can economically present.

In SageMaker AI, for builders who wish to customise fashions with out managing infrastructure, Amazon SageMaker AI serverless customization, launched at re:Invent 2025, supplies a totally managed, UI- and SDK-driven expertise for mannequin fine-tuning. This functionality supplies infrastructure administration—SageMaker routinely selects and provisions acceptable compute assets (P5, P4de, P4d, and G5 situations) primarily based on mannequin dimension and coaching necessities. By means of the SageMaker Studio UI, you may customise common fashions (Amazon Nova, Llama, DeepSeek, GPT-OSS, and Qwen) utilizing superior methods together with SFT, DPO, RLVR, and RLAIF. You may also run the identical serverless customization utilizing SageMaker Python SDK in your Jupyter pocket book. The serverless strategy supplies pay-per-token pricing, automated useful resource cleanup, built-in MLflow experiment monitoring, and seamless deployment to each Amazon Bedrock and SageMaker endpoints.

If you might want to customise Amazon Nova fashions to your agentic workflow, you are able to do it by way of recipes and train them on SageMaker AI. It supplies end-to-end customization workflow together with mannequin coaching, analysis, and deployment for inference. with better flexibility and management to fine-tune the Nova fashions, optimize hyperparameters with precision, and implement methods akin to LoRA PEFT, full-rank SFT, DPO, RFT, CPT, PPO, and so forth. For the Nova fashions on Amazon Bedrock, you too can prepare your Nova fashions by SFT and RFT with reasoning content material to seize intermediate pondering steps or use reward-based optimization when actual appropriate solutions are tough to outline. When you have extra superior agentic use circumstances that require deeper mannequin customization, you should utilize Amazon Nova Forge—launched at re:Invent 2025—to construct your individual frontier fashions from early mannequin checkpoints, mix your datasets with Amazon Nova-curated coaching knowledge, and host your customized fashions securely on AWS.

2. AI agent growth environments and SDKs

The event setting is the place builders creator, take a look at, and iterate on agent logic earlier than deployment. Builders use built-in growth environments (IDEs) akin to SageMaker AI Studio (Jupyter Notebooks in comparison with code editors), Amazon Kiro, or IDEs on native machines like PyCharm. Agent logic is applied utilizing specialised SDKs and frameworks that summary orchestration complexity—Strands supplies a Python framework purpose-built for multi-agent techniques, providing declarative agent definitions, built-in state administration, and native AWS service integrations that deal with the low-level particulars of LLM API calls, software invocation protocols, error restoration, and dialog administration. With these growth instruments dealing with the low-level particulars of LLM API calls, builders can deal with enterprise logic somewhat than infrastructure design and upkeep.

3. AI agent deployment and operation

After your AI agent growth is accomplished and able to deploy in manufacturing, you should utilize Amazon Bedrock AgentCore to deal with agent execution, reminiscence, safety, and power integration with out requiring infrastructure administration. Bedrock AgentCore supplies a set of built-in companies, together with:

1. 1. AgentCore Runtime presents purpose-built environments that summary away infrastructure administration, whereas container-based alternate options (SageMaker AI jobs, AWS Lambda, Amazon Elastic Kubernetes Service (Amazon EKS), and Amazon Elastic Container Service (Amazon ECS)) present extra management for customized necessities. Primarily, the runtime is the place your rigorously crafted agent code meets actual customers and delivers enterprise worth at scale.
   2. AgentCore Memory provides your AI brokers the flexibility to recollect previous interactions, enabling them to supply extra clever, context-aware, and customized conversations. It supplies a simple and highly effective approach to deal with each short-term context and long-term data retention with out the necessity to construct or handle complicated infrastructure.
   3. With AgentCore Gateway, builders can construct, deploy, uncover, and hook up with instruments at scale, offering observability into software utilization patterns, error dealing with for failed invocations, and integration with identification techniques for accessing instruments on behalf of customers (utilizing OAuth or API keys). Groups can replace software backends, add new capabilities, or modify authentication necessities with out redeploying brokers as a result of the gateway structure decouples software implementation from agent logic—sustaining flexibility as enterprise necessities evolve.
   4. AgentCore Observability helps you hint, debug, and monitor agent efficiency in manufacturing environments. It supplies real-time visibility into agent operational efficiency by way of entry to dashboards powered by Amazon CloudWatch and telemetry for key metrics akin to session rely, latency, length, token utilization, and error charges, utilizing the OpenTelemetry (OTEL) protocol customary.

4. LLM and AI agent analysis

When your fine-tuned LLM pushed AI brokers are operating in manufacturing, it’s necessary to judge and monitor your fashions and brokers repeatedly to make sure top quality and efficiency. Many enterprise use circumstances require customized analysis standards that encode area experience and enterprise guidelines. For the Amazon Pharmacy remedy route validation course of, analysis standards embody: drug-drug interplay detection accuracy (share of identified contraindications appropriately recognized), dosage calculation precision (appropriate dosing changes for age, weight, and renal operate), near-miss prevention charge (discount in remedy errors that might trigger affected person hurt), FDA labeling compliance (adherence to accepted utilization, warnings, and contraindications), and pharmacist override charge (share of agent suggestions accepted with out modification by licensed pharmacists).

In your fashions on Amazon Bedrock, you should utilize Amazon Bedrock evaluations to generate predefined metrics and human overview workflows. For superior situations, you should utilize SageMaker Coaching jobs to fine-tune specialised choose fashions on domain-specific analysis datasets. For holistic AI agent analysis, AgentCore Evaluations, launched at re:Invent 2025, supplies automated evaluation instruments to measure your agent or instruments efficiency on finishing particular duties, dealing with edge circumstances, and sustaining consistency throughout totally different inputs and contexts.

Choice information and really useful phased strategy

Now that you just perceive the technical evolution of superior fine-tuning methods—from SFT to PPO, DPO, GRPO, DAPO and GSPO—the vital query turns into when and why it’s best to use them. Our expertise reveals that organizations utilizing a phased maturity strategy obtain 70–85% manufacturing conversion charges (in comparison with the 30–40% trade common) and 3-fold year-over-year ROI development. The 12–18 month journey from preliminary agent deployment to superior reasoning capabilities delivers incremental enterprise worth at every section. The secret’s letting your use case necessities, accessible knowledge, and measured efficiency information development—not technical sophistication for its personal sake.

The maturity path progresses by way of 4 phases (proven within the following desk). Strategic endurance on this development builds reusable infrastructure, collects high quality coaching knowledge, and validates ROI earlier than main investments. As our examples reveal, aligning technical sophistication with human and enterprise wants delivers transformative outcomes and sustainable aggressive benefits in your most crucial AI functions.

Conclusion

Whereas brokers have reworked how we construct AI techniques, superior fine-tuning stays a vital element for enterprises searching for aggressive benefit in high-stakes domains. By understanding the evolution of methods like PPO, DPO, GRPO, DAPO and GSPO, and making use of them strategically inside agent architectures, organizations can obtain vital enhancements in accuracy, effectivity, and security. The true-world examples from Amazon reveal –that the mix of agentic workflows with rigorously fine-tuned fashions delivers dramatic enterprise outcomes.

AWS continues to speed up these capabilities with a number of key launches at re:Invent 2025. Reinforcement fine-tuning (RFT) on Amazon Bedrock now permits fashions to study high quality responses by way of RLVR for goal duties and RLAIF for subjective evaluations—with out requiring massive quantities of pre-labeled knowledge. Amazon SageMaker AI Serverless Customization eliminates infrastructure administration for fine-tuning, supporting SFT, DPO, and RLVR methods with pay-per-token pricing. For big-scale coaching, Amazon SageMaker HyperPod launched checkpointless coaching and elastic scaling to cut back restoration time and optimize useful resource utilization. Amazon Nova Forge empowers enterprises to construct customized frontier fashions from early checkpoints, mixing proprietary datasets with Amazon-curated coaching knowledge. Lastly, AgentCore Analysis supplies automated evaluation instruments to measure agent efficiency on job completion, edge circumstances, and consistency—closing the loop on production-grade agentic AI techniques.

As you consider your generative AI technique, use the choice information and phased maturity strategy outlined on this submit to establish the place superior fine-tuning can tip the scales from adequate to transformative. Use the reference structure as a baseline to construction your agentic AI techniques, and use the capabilities launched at re:Invent 2025 to speed up your journey from preliminary agent deployment to production-grade outcomes.

In regards to the authors

Yunfei Bai is a Principal Options Architect at AWS. With a background in AI/ML, knowledge science, and analytics, Yunfei helps clients undertake AWS companies to ship enterprise outcomes. He designs AI/ML and knowledge analytics options that overcome complicated technical challenges and drive strategic targets. Yunfei has a PhD in Digital and Electrical Engineering. Exterior of labor, Yunfei enjoys studying and music.

Kristine Pearce is a Principal Worldwide Generative AI GTM Specialist at AWS, targeted on SageMaker AI mannequin customization, optimization, and inference at scale. She combines her MBA, BS Industrial Engineering background, and human-centered design experience to convey strategic depth and behavioral science to AI-enabled transformation. Exterior work, she channels her creativity by way of artwork.

Harsh Asnani is a Worldwide Generative AI Specialist Options Architect at AWS specializing in ML principle, MLOPs, and manufacturing generative AI frameworks. His background is in utilized knowledge science with a deal with operationalizing AI workloads within the cloud at scale.

Sung-Ching Lin is a Principal Engineer at Amazon Pharmacy, the place he leads the design and adoption of AI/ML techniques to enhance buyer expertise and operational effectivity. He focuses on constructing scalable, agent-based architectures, ML analysis frameworks, and production-ready AI options in regulated healthcare domains.

Elad Dwek is a Senior AI Enterprise Developer at Amazon, working inside World Engineering, Upkeep, and Sustainability. He companions with stakeholders from enterprise and tech facet to establish alternatives the place AI can improve enterprise challenges or utterly rework processes, driving innovation from prototyping to manufacturing. With a background in development and bodily engineering, he focuses on change administration, expertise adoption, and constructing scalable, transferable options that ship steady enchancment throughout industries. Exterior of labor, he enjoys touring all over the world along with his household.

Carrie Track is a Senior Program Supervisor at Amazon, engaged on AI-powered content material high quality and buyer expertise initiatives. She companions with utilized science, engineering, and UX groups to translate generative AI and machine studying insights into scalable, customer-facing options. Her work focuses on enhancing content material high quality and streamlining the purchasing expertise on product element pages.