The best way to configure cross-account mannequin deployment utilizing Amazon Bedrock Customized Mannequin Import

In enterprise environments, organizations typically divide their AI operations into two specialised groups: an AI analysis crew and a mannequin internet hosting crew. The analysis crew is devoted to creating and enhancing AI fashions utilizing model training and fine-tuning methods. In the meantime, a separate internet hosting crew is accountable for deploying these fashions throughout their very own improvement, staging, and manufacturing environments.

With Amazon Bedrock Custom Model Import, the internet hosting crew can import and serve customized fashions utilizing supported architectures resembling Meta Llama 2, Llama 3, and Mistral utilizing On-Demand pricing. Groups can import fashions with weights in Hugging Face safetensors format from Amazon SageMaker or from Amazon Simple Storage Service (Amazon S3). These imported customized fashions work alongside present Amazon Bedrock foundation models (FMs) by a single, unified API in a serverless method, assuaging the necessity to handle mannequin deployment and scaling.

Nevertheless, in such enterprise environments, these groups typically work in separate AWS accounts for safety and operational causes. The mannequin improvement crew’s coaching outcomes, often known as mannequin artifacts, for instance mannequin weights, are usually saved in S3 buckets throughout the analysis crew’s AWS account, however the internet hosting crew must entry these artifacts from one other account to deploy fashions. This creates a problem: how do you securely share mannequin artifacts between accounts?

That is the place cross-account entry turns into vital. With Amazon Bedrock Customized Mannequin Import cross-account help, we will help you configure direct entry between the S3 buckets storing mannequin artifacts and the internet hosting account. This streamlines your operational workflow whereas sustaining safety boundaries between groups. Considered one of our clients quotes:

Bedrock Customized Mannequin Import cross-account help helped AI Platform crew to simplify the configuration, scale back operational overhead and safe fashions within the unique location.

– Scott Chang, Principal Engineer, AI Platform at Salesforce

On this information, we stroll you thru step-by-step directions for configuring cross-account entry for Amazon Bedrock Customized Mannequin Import, protecting each non-encrypted and AWS Key Management Service (AWS KMS) based mostly encrypted eventualities.

Instance situation

For this walkthrough, take into account two AWS accounts:

Mannequin Improvement account (111122223333):
- Shops mannequin artifacts (customized weights and configurations) in an S3 bucket known as model-artifacts-111122223333
- Optionally encrypts artifacts utilizing AWS KMS buyer managed key kms-cmk-111122223333
Mannequin Internet hosting account (777788889999):
- Hosts fashions utilizing Amazon Bedrock Customized Mannequin Import
- Makes use of a brand new AWS Identity and Access Management (IAM) execution function BedrockCMIExecutionRole-777788889999
- Can optionally encrypt artifacts utilizing AWS KMS key kms-cmk-777788889999

The next determine illustrates this setup, displaying how the cross-account entry is configured between the S3 bucket, KMS keys, and Amazon Bedrock Customized Mannequin Import.

Figure shows how the cross-account access is configured

To efficiently implement the described situation whereas adhering to the precept of least privilege entry, the next steps should be executed:

The Mannequin Improvement account should present entry to the Mannequin Internet hosting account’s IAM function BedrockCMIExecutionRole-777788889999, permitting it to make the most of their S3 bucket and, if relevant, the encryption key, utilizing resource-based insurance policies.
The Mannequin Internet hosting account ought to set up an IAM function, resembling BedrockCMIExecutionRole-777788889999. The identity-based insurance policies wanted could be for the Mannequin Improvement S3 bucket and buyer managed keys for decrypting mannequin artifacts, like utilizing kms-cmk-111122223333.
The Mannequin Internet hosting account should allow the Amazon Bedrock service to imagine the IAM function BedrockCMIExecutionRole-777788889999, created in step 2, by together with the Amazon Bedrock service as a trusted entity. This IAM function shall be utilized by the Mannequin Internet hosting account to provoke the customized mannequin import job.

Conditions

Earlier than you can begin a customized mannequin import job, it’s essential fulfill the next stipulations:

When you’re importing your mannequin from an S3 bucket, put together your mannequin information within the Hugging Face weights format. For extra info consult with Import source.
(Non-compulsory) Arrange further safety configurations.

Step-by-step execution

The next part gives the step-by-step execution of the beforehand outlined high-level course of, from the attitude of an administrator managing each accounts:

Step 1: Arrange the S3 bucket coverage (within the Mannequin Improvement account) to allow entry for the Mannequin Internet hosting account’s IAM function:

Check in to the AWS Management Console for account 111122223333, then entry the Amazon S3 console.
On the Normal function buckets view, find model-artifacts-111122223333, the bucket utilized by the mannequin improvement crew to retailer their mannequin artifacts.

On the Permissions tab, choose Edit within the Bucket coverage part, and insert the next IAM resource-based coverage. Make sure to replace the AWS account IDs (proven in purple) within the coverage along with your info.

{
    "Model": "2012-10-17",
    "Id": "AllowCrossAccountS3Access",
    "Assertion": [
        {
            "Sid": "cross-account-list-get",
            "Effect": "Allow",
            "Principal": {
 "AWS": "arn:aws:iam::777788889999:root"             },
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Useful resource": [
 "arn:aws:s3:::model-artifacts-111122223333", "arn:aws:s3:::model-artifacts-111122223333/*"             ],
            "Situation": {
                "ArnLike": {
 "aws:PrincipalArn": "arn:aws:iam::777788889999:function/BedrockCMIExecutionRole-777788889999*"                 }
            }
        }
    ]
}

Step 2: Set up an IAM function (within the Mannequin Internet hosting account) and authorize Amazon Bedrock to imagine this function:

Check in to the AWS console for account 777788889999 and launch the IAM console.

Within the left navigation pane, choose Insurance policies after which select Create coverage. Throughout the Coverage Editor, swap to the JSON tab and insert the next identity-based coverage. This coverage is designed for read-only entry, enabling customers or a task to listing and obtain objects from a specified S3 bucket, however provided that the bucket is owned by account 111122223333. Customise the AWS account ID and S3 bucket identify/prefix (proven in purple) along with your info.

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Sid": "1",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Useful resource": [
 "arn:aws:s3:::model-artifacts-111122223333", "arn:aws:s3:::model-artifacts-111122223333/*"             ],
            "Situation": {
                "StringEquals": {
  "aws:ResourceAccount": "111122223333"                 }
            }
        }
    ]
}

Select Subsequent, assign the coverage identify as BedrockCMIExecutionPolicy-777788889999, and finalize by selecting Create coverage.

Within the left navigation pane, select Roles and choose Customized belief coverage because the Trusted entity kind. Insert the next trusted entity coverage, which restricts the function assumption to the Amazon Bedrock service, particularly for mannequin import jobs in account 777788889999 situated within the US East (N. Virginia) us-east-1 Area. Modify the AWS account ID and Region (proven in purple) along with your info.

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Sid": "1",
            "Effect": "Allow",
            "Principal": {
                "Service": "bedrock.amazonaws.com"
            },
            "Action": "sts:AssumeRole",
            "Condition": {
                "StringEquals": {
 "aws:SourceAccount": "777788889999"                 },
                "ArnEquals": {
 "aws:SourceArn": "arn:aws:bedrock:us-east-1:777788889999:model-import-job/*"                 }
            }
        }
    ]
}

Select Subsequent and within the Add permissions part, seek for the coverage created within the earlier step BedrockCMIExecutionPolicy-777788889999, choose the checkbox, and proceed by selecting Subsequent.
Assign the Position identify as BedrockCMIExecutionRole-777788889999, present a Description as “IAM execution function for use by CMI jobs,” and finalize by selecting Create function.

Essential: When you’re utilizing an AWS KMS encryption key for mannequin artifacts within the Mannequin Improvement account or for imported mannequin artifacts with the Amazon Bedrock managed AWS account, proceed with steps 3 by 5. If not, skip to step 6.

Step 3: Alter the AWS KMS key coverage (within the Mannequin Improvement account) to permit the Amazon Bedrock CMI execution IAM function to decrypt mannequin artifacts:

Transition again to the Mannequin Improvement account and discover the AWS KMS key named kms-cmk-111122223333 within the AWS KMS console. Be aware the AWS KMS key Amazon Useful resource Title (ARN).

On the Key coverage tab, swap to the Coverage view, and incorporate the next resource-based coverage assertion to allow the Mannequin Internet hosting account’s IAM function BedrockCMIExecutionRole-777788889999 to decrypt mannequin artifacts. Revise objects in purple along with your info.

{
      "Sid": "Enable use of the important thing by the vacation spot account",
      "Impact": "Enable",
      "Principal": {
 "AWS": "arn:aws:iam::777788889999:function/BedrockCMIExecutionRole-777788889999"       },
      "Motion": [
        "kms:Decrypt",
        "kms:DescribeKey"
      ],
      "Useful resource": "*"
}

Step 4: Set the AWS KMS key coverage (within the Mannequin Internet hosting account) for the CMI execution IAM function to encrypt and decrypt mannequin artifacts to securely retailer within the Amazon Bedrock AWS account:

Return to the Mannequin Internet hosting account and find the AWS KMS key named kms-cmk-777788889999 within the AWS KMS console. Be aware the AWS KMS key ARN.

Insert the next assertion into the AWS KMS key’s resource-based coverage to allow the BedrockCMIExecutionRole-777788889999 IAM function to encrypt and decrypt mannequin artifacts at relaxation within the Amazon Bedrock managed AWS account. Revise objects in purple along with your info.

{
      "Sid": "Enable use of the important thing",
      "Impact": "Enable",
      "Principal": {
 "AWS": "arn:aws:iam::777788889999:function/BedrockCMIExecutionRole-777788889999"       },
      "Motion": [
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:ReEncrypt*",
        "kms:GenerateDataKey*",
        "kms:DescribeKey"
      ],
      "Useful resource": "*"
}

Step 5: Modify the CMI execution function’s permissions (within the Mannequin Internet hosting account) to offer entry to encryption keys:

Entry the IAM console and discover the IAM coverage BedrockCMIExecutionPolicy-777788889999. To the present identity-based coverage, append the next statements (substitute the ARNs in purple with one famous in steps 4 and 5):

{
    "Impact": "Enable",
    "Motion": [
        "kms:Decrypt",
        "kms:DescribeKey"
    ],
 "Useful resource": "arn:aws:kms:us-east-1:111122223333:key/b5b6e052-fb27-4dbb-bf0d-daf3375a9fda" },
{
    "Impact": "Enable",
    "Motion": [
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:ReEncrypt*",
        "kms:GenerateDataKey*",
        "kms:DescribeKey"
    ],
 "Useful resource": "arn:aws:kms:us-east-1:777788889999:key/6cd5d3bf-3d9b-4d1c-83d5-8df6284435a1" }

Step 6: Provoke the Mannequin import job (within the Mannequin Internet hosting account)

On this step, we execute the mannequin import job utilizing the AWS Command Line Interface (AWS CLI) command. You can even use AWS SDKs or APIs for a similar function. Run the next command out of your terminal session with an IAM consumer or function that has the required privileges to create a customized mannequin import job. You don’t have to explicitly present an ARN or particulars of the CMK utilized by the Mannequin Improvement crew.

aws bedrock create-model-import-job 
    --job-name "cmi-job-777788889999-01" 
    --imported-model-name "mistral-777788889999-01" 
    --role-arn "arn:aws:iam::777788889999:function/BedrockCMIExecutionRole-777788889999" 
    --model-data-source "s3DataSource={s3Uri="s3://model-artifacts-111122223333/mistral-model-weights/"}"

When encrypting mannequin artifacts with Amazon Bedrock Customized Mannequin Import, use the --imported-model-kms-key-id flag and specify the ARN of the Mannequin Internet hosting account’s CMK key.

aws bedrock create-model-import-job 
    --job-name "cmi-job-777788889999-04" 
    --imported-model-name "mistral-777788889999-01" 
    --role-arn "arn:aws:iam::777788889999:function/BedrockCMIExecutionRole-777788889999" 
    --model-data-source "s3DataSource={s3Uri="s3://model-artifacts-111122223333/mistral-model-weights/"}" 
    --imported-model-kms-key-id "arn:aws:kms:us-east-1:777788889999:key/6cd5d3bf-3d9b-4d1c-83d5-8df6284435a1"

Cross-account entry to the S3 bucket utilizing the customized mannequin import job is just supported by AWS CLI, AWS SDKs, or APIs. Console help just isn’t but out there.

Troubleshooting

When IAM coverage misconfigurations stop a customized mannequin import job, you may encounter an error like:

Amazon Bedrock doesn't have entry to the S3 location (s3://model-artifacts-111122223333/mistral-model-weights). Replace the permissions and take a look at once more.

To resolve this, manually confirm entry to Mannequin Improvement’s S3 bucket from the Mannequin Internet hosting account by assuming the BedrockCMIExecutionRole-777788889999. Comply with these steps:

Step 1: Determine the present IAM function or consumer within the CLI with the next and duplicate the ARN from the output:

aws sts get-caller-identity

Step 2: Replace belief relationships. Append the belief coverage of the BedrockCMIExecutionRole-777788889999 to permit the present consumer or IAM function to imagine this function:

{
    "Impact": "Enable",
    "Principal": {
        "AWS": "arn:aws:sts::777788889999:function/current-user-role"
    },
    "Motion": "sts:AssumeRole"
}

Step 3: Checklist or copy the S3 bucket contents assuming the Amazon Bedrock Customized Mannequin Import execution function

Assume the CMI execution function (substitute the ARN along with your info):

aws sts assume-role 
    --role-arn "arn:aws:iam::776941257690:function/BedrockCMIExecutionRole-777788889999" 
    --role-session-name "BedrockCMISession"

Export the returned short-term credentials as atmosphere variables:

export AWS_ACCESS_KEY_ID="ASIA..."
export AWS_SECRET_ACCESS_KEY="..."
export AWS_SESSION_TOKEN="..."

Run instructions to troubleshoot permission points:

aws s3 ls s3://model-artifacts-111122223333/mistral-model-weights/
aws s3 cp s3://model-artifacts-111122223333/mistral-model-weights/config.json .

If errors persist, think about using Amazon Q Developer or consult with extra sources outlined within the IAM User Guide.

Cleanup

There isn’t a extra cost to import a customized mannequin to Amazon Bedrock (consult with step 6 within the Step-by-step execution part). Nevertheless, in case your mannequin isn’t in use for inference, and also you need to keep away from paying storage prices (consult with Amazon Bedrock pricing), delete the imported mannequin utilizing the AWS console or AWS CLI reference or API Reference. For instance (substitute the textual content in purple along with your imported mannequin identify):

aws bedrock delete-imported-model 
    --model-identifier "mistral-777788889999-01"

Conclusion

Through the use of cross-account entry in Amazon Bedrock Customized Mannequin Import, organizations can considerably streamline their AI mannequin deployment workflows.

Amazon Bedrock Customized Mannequin Import is mostly out there as we speak in Amazon Bedrock within the US East (N. Virginia) us-east-1 and US West (Oregon) us-west-2 AWS Regions. Discuss with the full Region list for future updates. To be taught extra, consult with the Amazon Bedrock Custom Model Import product web page and Amazon Bedrock pricing web page. Give Amazon Bedrock Customized Mannequin Import a attempt within the Amazon Bedrock console as we speak and ship suggestions to AWS re:Post for Amazon Bedrock or by your traditional AWS Assist contacts.

Thanks to our contributors Scott Chang (Salesforce), Raghav Tanaji (Salesforce), Rupinder Grewal (AWS), Ishan Singh (AWS), and Dharinee Gupta (AWS)

In regards to the Authors

Hrushikesh Gangur is a Principal Options Architect at AWS. Primarily based in San Francisco, California, Hrushikesh is an professional in AWS machine studying. As a thought chief within the subject of generative AI, Hrushikesh has contributed to AWS’s efforts in serving to startups and ISVs construct and deploy AI purposes. His experience extends to numerous AWS providers, together with Amazon SageMaker, Amazon Bedrock, and accelerated computing that are essential for constructing AI purposes.

Sai Darahas Akkineni is a Software program Improvement Engineer at AWS. He holds a grasp’s diploma in Pc Engineering from Cornell College, the place he labored within the Autonomous Programs Lab with a specialization in laptop imaginative and prescient and robotic notion. At present, he helps deploy giant language fashions to optimize throughput and latency.

Prashant Patel is a Senior Software program Improvement Engineer in AWS. He’s obsessed with scaling giant language fashions for enterprise purposes. Previous to becoming a member of AWS, he labored at IBM on productionizing large-scale AI/ML workloads on Kubernetes. Prashant has a grasp’s diploma from NYU Tandon Faculty of Engineering. Whereas not at work, he enjoys touring and enjoying together with his canines.