Introducing structured output for Customized Mannequin Import in Amazon Bedrock

With Amazon Bedrock Custom Model Import, you possibly can deploy and scale fine-tuned or proprietary basis fashions in a totally managed, serverless atmosphere. You’ll be able to convey your personal fashions into Amazon Bedrock, scale them securely with out managing infrastructure, and combine them with different Amazon Bedrock capabilities.

At the moment, we’re excited to announce the addition of structured output to Customized Mannequin Import. Structured output constrains a mannequin’s era course of in actual time so that each token it produces conforms to a schema you outline. Slightly than counting on prompt-engineering methods or brittle post-processing scripts, now you can generate structured outputs straight at inference time.

For sure manufacturing functions, the predictability of mannequin outputs is extra vital than their artistic flexibility. A customer support chatbot would possibly profit from various, natural-sounding responses, however an order processing system wants actual, structured knowledge that conforms to predefined schemas. Structured output bridges this hole by sustaining the intelligence of basis fashions whereas verifying their outputs meet strict formatting necessities.

This represents a shift from free-form textual content era to outputs which are constant, machine-readable, and designed for seamless integration with enterprise techniques. Whereas free-form textual content excels for human consumption, manufacturing functions require extra precision. Companies can’t afford the anomaly of pure language variations when their techniques rely upon structured outputs to reliably interface with APIs, databases, and automatic workflows.

On this submit, you’ll learn to implement structured output for Customized Mannequin Import in Amazon Bedrock. We are going to cowl what structured output is, the right way to allow it in your API calls, and the right way to apply it to real-world situations that require structured, predictable outputs.

Understanding structured output

Structured output, often known as constrained decoding, is a technique that directs LLM outputs to adapt to a predefined schema, reminiscent of legitimate JSON. Slightly than permitting the mannequin to freely choose tokens primarily based on likelihood distributions, it introduces constraints throughout era that restrict selections to solely those who keep structural validity. If a specific token would violate the schema by producing invalid JSON, inserting stray characters, or utilizing an sudden subject identify the structured output rejects it and requires the mannequin to pick one other allowed choice. This real-time validation helps hold the ultimate output constant, machine readable, and instantly usable by downstream functions with out the necessity for extra post-processing.

With out structured output, builders usually try and implement construction via immediate directions like “Reply solely in JSON.” Whereas this strategy generally works, it stays unreliable as a result of inherently probabilistic nature of LLMs. These fashions generate textual content by sampling from likelihood distributions, introducing pure variability that makes responses really feel human however creates important challenges for automated techniques.

Contemplate a buyer assist software that classifies tickets: if responses range between “This looks as if a billing concern,” “I’d classify this as: Billing,” and “Class = BILLING,” downstream code can’t reliably interpret the outcomes. What manufacturing techniques require as an alternative is predictable, structured output. For instance:

{
  "class": "billing",
  "precedence": "excessive",
  "sentiment": "adverse"
}

With a response like this, your software can routinely route tickets, set off workflows, or replace databases with out human intervention. By offering predictable, schema-aligned responses, structured output transforms LLMs from conversational instruments into dependable system parts that may be built-in with databases, APIs, and enterprise logic. This functionality opens new potentialities for automation whereas sustaining the clever reasoning that underpin the worth of those fashions.

Past enhancing reliability and simplifying post-processing, structured output gives extra advantages that strengthens efficiency, safety and security in manufacturing environments.

• Decrease token utilization and sooner responses: By constraining era to an outlined schema, structured output removes pointless verbose, free-form textual content, leading to decreased token depend. As a result of token era is sequential, shorter outputs straight translate to sooner responses and decrease latency, enhancing general efficiency and value effectivity.
• Enhanced safety towards immediate injection: Structured output narrows the mannequin’s expression area and helps forestall it from producing arbitrary or unsafe content material. Unhealthy actors can’t inject directions, code or sudden textual content exterior the outlined construction. Every subject should match its anticipated kind and format, ensuring outputs stay inside protected boundaries.
• Security and coverage controls: Structured output allows you to design schemas that inherently assist forestall dangerous, poisonous, or policy-violating content material. By limiting fields to accepted values, implementing patterns, and limiting free-form textual content, schemas be certain that outputs align with regulatory necessities.

Within the subsequent part, we’ll discover how structured output works with Customized Mannequin Import in Amazon Bedrock and walks via an instance of enabling it in your API calls.

Utilizing structured output with Customized Mannequin Import in Amazon Bedrock

Let’s begin by assuming you will have already imported a Hugging Face mannequin into Amazon Bedrock utilizing the Customized Mannequin Import characteristic.

Conditions

Earlier than continuing, ensure you have:

• An energetic AWS account with entry to Amazon Bedrock
• A customized mannequin created in Amazon Bedrock utilizing the Customized Mannequin Import characteristic
• Applicable AWS Identity and Access Management (IAM) permissions to invoke fashions via the Amazon Bedrock Runtime

With these conditions in place, let’s discover the right way to implement structured output along with your imported mannequin.

To begin utilizing structured output with a Customized Mannequin Import in Amazon Bedrock, start by configuring your atmosphere. In Python, this includes making a Bedrock Runtime consumer and initializing a tokenizer out of your imported Hugging Face mannequin.

The Bedrock Runtime consumer gives entry to your imported mannequin utilizing the Bedrock InvokeModel API. The tokenizer applies the right chat template that aligns with the imported mannequin, which defines how consumer, system, and assistant messages are mixed right into a single immediate, how the function markers (for instance, <|consumer|>, <|assistant|>) are inserted, and the place the mannequin’s response ought to start.

By calling tokenizer.apply_chat_template(messages, tokenize=False) you possibly can generate a immediate that matches the precise enter format your mannequin expects, which is important for constant and dependable inference, particularly when structured encoding is enabled.

import boto3
from transformers import AutoTokenizer
from botocore.config import Config

# HF mannequin identifier imported into Bedrock
hf_model_id = "<<huggingface_model_id>>" # Instance: "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
model_arn = "arn:aws:bedrock:<<aws-region>>:<<account-id>>:imported-model/your-model-id"
area      = "<<aws-region>>"

# Initialize tokenizer aligned along with your imported mannequin 
tokenizer = AutoTokenizer.from_pretrained(hf_model_id)

# Initialize Bedrock consumer
bedrock_runtime = boto3.consumer(
    service_name="bedrock-runtime",
    region_name=area)

Implementing structured output

Whenever you invoke a customized mannequin on Amazon Bedrock, you will have the choice to allow structured output by including a response_format block to the request payload. This block accepts a JSON schema that defines the structured of the mannequin’s response. Throughout inference, the mannequin enforces this schema in real-time, ensuring that every generated token conforms to the outlined construction. Beneath is a walkthrough demonstrating the right way to implement structured output utilizing a easy tackle extraction activity.

Step 1: Outline the information construction

You’ll be able to outline your anticipated output utilizing a Pydantic mannequin, which serves as a typed contract for the information you wish to extract.

from pydantic import BaseModel, Discipline

class Handle(BaseModel):
    street_number: str = Discipline(description="Road quantity")
    street_name: str = Discipline(description="Road identify together with kind (Ave, St, Rd, and so forth.)")
    metropolis: str = Discipline(description="Metropolis identify")
    state: str = Discipline(description="Two-letter state abbreviation")
    zip_code: str = Discipline(description="5-digit ZIP code")

Step 2: Generate the JSON schema

Pydantic can routinely convert your knowledge mannequin right into a JSON schema:

schema = Handle.model_json_schema()
address_schema = {
    "identify": "Handle",
    "schema": schema
}

This schema defines every subject’s kind, description, and requirement, making a blueprint that the mannequin will observe throughout era.

Step 3: Put together your enter messages

Format your enter utilizing the chat format anticipated by your mannequin:

messages = [{
    "role": "user",
    "content": "Extract the address: 456 Tech Boulevard, San Francisco, CA 94105"
}]

Step 4: Apply the chat template

Use your mannequin’s tokenizer to generate the formatted immediate:

immediate = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

Step 5: Construct the request payload

Create your request physique, together with the response_format that references your schema:

request_body = {
    'immediate': immediate,
    'temperature': 0.1,
    'max_gen_len': 1000,
    'top_p': 0.9,
    'response_format': {
        "kind": "json_schema",
        "json_schema": address_schema
    }
}

Step 6: Invoke the mannequin

Ship the request utilizing the InvokeModel API:

response = bedrock_runtime.invoke_model(
    modelId=model_arn,
    physique=json.dumps(request_body),
    settle for="software/json",
    contentType="software/json"
)

Step 7: Parse the response

Extract the generated textual content from the response:

end result = json.hundreds(response['body'].learn().decode('utf-8'))
raw_output = end result['choices'][0]['text']
print(raw_output)

As a result of the schema defines required fields, the mannequin’s response will include them:

{
"street_number": "456",
"street_name": "Tech Boulevard",
"metropolis": "San Francisco",
"state": "CA",
"zip_code": "94105"
}

The output is clear, legitimate JSON that may be consumed straight by your software with no additional parsing, filtering, or cleanup required.

Conclusion

Structured output with Customized Mannequin Import in Amazon Bedrock gives an efficient option to generate constructions, schema-aligned outputs out of your fashions. By shifting validation into the mannequin inference itself, structured output scale back the necessity for advanced post-processing workflows and error dealing with code.

Structured output generates outputs which are predictable and simple to combine into your techniques and helps a wide range of use circumstances, for instance, constructing monetary functions that require exact knowledge extraction, healthcare techniques that want structured scientific documentation, or customer support techniques that demand constant ticket classification.

Begin experimenting with structured output along with your Custom Model Import immediately and rework how your AI functions ship constant, production-ready outcomes.

In regards to the authors

Manoj Selvakumar is a Generative AI Specialist Options Architect at AWS, the place he helps organizations design, prototype, and scale AI-powered options within the cloud. With experience in deep studying, scalable cloud-native techniques, and multi-agent orchestration, he focuses on turning rising improvements into production-ready architectures that drive measurable enterprise worth. He’s captivated with making advanced AI ideas sensible and enabling prospects to innovate responsibly at scale—from early experimentation to enterprise deployment. Earlier than becoming a member of AWS, Manoj labored in consulting, delivering knowledge science and AI options for enterprise shoppers, constructing end-to-end machine studying techniques supported by robust MLOps practices for coaching, deployment, and monitoring in manufacturing.

Yanyan Zhang is a Senior Generative AI Knowledge Scientist at Amazon Internet Companies, the place she has been engaged on cutting-edge AI/ML applied sciences as a Generative AI Specialist, serving to prospects use generative AI to realize their desired outcomes. Yanyan graduated from Texas A&M College with a PhD in Electrical Engineering. Outdoors of labor, she loves touring, figuring out, and exploring new issues.

Lokeshwaran Ravi is a Senior Deep Studying Compiler Engineer at AWS, specializing in ML optimization, mannequin acceleration, and AI safety. He focuses on enhancing effectivity, decreasing prices, and constructing safe ecosystems to democratize AI applied sciences, making cutting-edge ML accessible and impactful throughout industries.

Revendra Kumar is a Senior Software program Growth Engineer at Amazon Internet Companies. In his present function, he focuses on mannequin internet hosting and inference MLOps on Amazon Bedrock. Previous to this, he labored as an engineer on internet hosting Quantum computer systems on the cloud and growing infrastructure options for on-premises cloud environments. Outdoors of his skilled pursuits, Revendra enjoys staying energetic by enjoying tennis and mountaineering.

Muzart Tuman is a software program engineer using his expertise in fields like deep studying, machine studying optimization, and AI-driven functions to assist resolve real-world issues in a scalable, environment friendly, and accessible method. His objective is to create impactful instruments that not solely advance technical capabilities but additionally encourage significant change throughout industries and communities.