Use Amazon SageMaker Mannequin Card sharing to enhance mannequin governance

As Synthetic Intelligence (AI) and Machine Studying (ML) applied sciences have turn out to be mainstream, many enterprises have been profitable in constructing crucial enterprise purposes powered by ML fashions at scale in manufacturing. Nonetheless, since these ML fashions are making crucial enterprise choices for the enterprise, it’s necessary for enterprises so as to add correct guardrails all through their ML lifecycle. Guardrails be sure that safety, privateness, and high quality of the code, configuration, and knowledge and mannequin configuration utilized in mannequin lifecycle are versioned and preserved.

Implementing these guardrails is getting tougher for enterprises as a result of the ML processes and actions inside enterprises have gotten extra complicated because of the inclusion of deeply concerned processes that require contributions from a number of stakeholders and personas. Along with knowledge engineers and knowledge scientists, there have been inclusions of operational processes to automate & streamline the ML lifecycle. Moreover, the surge of enterprise stakeholders and in some circumstances authorized and compliance evaluations want capabilities so as to add transparency for managing entry management, exercise monitoring, and reporting throughout the ML lifecycle.

The framework that offers systematic visibility into ML mannequin growth, validation, and utilization is named ML governance. Throughout AWS re:Invent 2022, AWS introduced new ML governance tools for Amazon SageMaker which simplifies entry management and enhances transparency over your ML initiatives. One of many instruments out there as a part of the ML governance is Amazon SageMaker Model Cards, which has the aptitude to create a single supply of fact for mannequin data by centralizing and standardizing documentation all through the mannequin lifecycle.

SageMaker mannequin playing cards allow you to standardize how fashions are documented, thereby reaching visibility into the lifecycle of a mannequin, from designing, constructing, coaching, and analysis. Mannequin playing cards are meant to be a single supply of fact for enterprise and technical metadata concerning the mannequin that may reliably be used for auditing and documentation functions. They supply a reality sheet of the mannequin that’s necessary for mannequin governance.

As you scale your fashions, initiatives, and groups, as a finest apply we advocate that you simply undertake a multi-account technique that gives mission and group isolation for ML mannequin growth and deployment. For extra details about bettering governance of your ML fashions, discuss with Improve governance of your machine learning models with Amazon SageMaker.

Structure overview

The structure is applied as follows:

Knowledge Science Account – Knowledge Scientists conduct their experiments in SageMaker Studio and construct an MLOps setup to deploy fashions to staging/manufacturing environments utilizing SageMaker Projects.
ML Shared Companies Account – The MLOps arrange from the Knowledge Science account will set off steady integration and steady supply (CI/CD) pipelines utilizing AWS CodeCommit and AWS CodePipeline.
Dev Account – The CI/CD pipelines will additional set off ML pipelines on this account masking knowledge pre-processing, mannequin coaching and put up processing like mannequin analysis and registration. Output of those pipelines will deploy the mannequin in SageMaker endpoints to be consumed for inference functions. Relying in your governance necessities, Knowledge Science & Dev accounts could be merged right into a single AWS account.
Knowledge Account – The ML pipelines operating within the Dev Account will pull the info from this account.
Take a look at and Prod Accounts – The CI/CD pipelines will proceed the deployment after the Dev Account to arrange SageMaker endpoint configuration in these accounts.
Safety and Governance – Companies like AWS Identification and Entry Administration (IAM), AWS IAM Identification Middle, AWS CloudTrail, AWS Key Administration Service (AWS KMS), Amazon CloudWatch, and AWS Safety Hub can be used throughout these accounts as a part of safety and governance.

The next diagram illustrates this structure.

For extra details about setting scalable multi account ML structure, discuss with MLOps foundation for enterprises with Amazon SageMaker.

Our clients want the aptitude to share mannequin playing cards throughout accounts to enhance visibility and governance of their fashions by way of data shared within the mannequin card. Now, with cross-account mannequin playing cards sharing, clients can get pleasure from the advantages of multi-account technique whereas having accessibility into the out there mannequin playing cards of their group, to allow them to speed up collaboration and guarantee governance.

On this put up, we present how you can arrange and entry mannequin playing cards throughout Mannequin Growth Lifecycle (MDLC) accounts utilizing the brand new cross-account sharing function of the mannequin card. First, we are going to describe a state of affairs and structure for establishing the cross-account sharing function of the mannequin card, after which dive deep into every element of how you can arrange and entry shared mannequin playing cards throughout accounts to enhance visibility and mannequin governance.

Resolution overview

When constructing ML fashions, we advocate establishing a multi-account structure to offer workload isolation bettering safety, reliability, and scalability. For this put up, we are going to assume constructing and deploying a mannequin for Buyer Churn use case. The structure diagram that follows reveals one of many advisable approaches – centralized mannequin card – for managing a mannequin card in a multi-account Machine Studying Mannequin-Growth Lifecycle (MDLC) structure. Nonetheless, you may also undertake one other method, a hub-and-spoke mannequin card. On this put up, we are going to focus solely on a centralized mannequin card method, however the identical ideas could be prolonged to a hub-and-spoke method. The primary distinction is that every spoke account will preserve their very own model of mannequin card and it’ll have processes to combination and duplicate to a centralized account.

The next diagram illustrates this structure.

The structure is applied as follows:

Lead Knowledge Scientist is notified to unravel the Buyer Churn use case utilizing ML, and so they begin the ML mission by way of creation of a mannequin card for Buyer Churn V1 mannequin in Draft standing within the ML Shared Companies Account
By way of automation, that mannequin card is shared with ML Dev Account
Knowledge Scientist builds the mannequin and begins to populate data through APIs into the mannequin card based mostly on their experimentation outcomes and the mannequin card standing is ready to Pending Assessment
By way of automation, that mannequin card is shared with the ML take a look at account
ML Engineer (MLE) runs integration and validation assessments in ML Take a look at account and the mannequin within the central registry is marked Pending Approval
Mannequin Approver evaluations the mannequin outcomes with the supporting documentation supplied within the central mannequin card and approves the mannequin card for manufacturing deployment.
By way of automation, that mannequin card is shared with ML Prod account in read-only mode.

Stipulations

Earlier than you get began, ensure you have the next stipulations:

Two AWS accounts.
In each AWS accounts, an IAM federation position with administrator entry to do the next:
- Create, edit, view, and delete mannequin playing cards inside Amazon SageMaker.
- Create, edit, view, and delete useful resource share inside AWS RAM.

For extra data, discuss with Example IAM policies for AWS RAM.

Organising mannequin card sharing

The account the place the mannequin playing cards are created is the mannequin card account. Customers within the mannequin card account share them with the shared accounts the place they are often up to date. Customers within the mannequin card account can share their mannequin playing cards by way of AWS Resource Access Manager (AWS RAM). AWS RAM helps you share sources throughout AWS accounts.

Within the following part, we present how you can share mannequin playing cards.

First, create a mannequin card for a Buyer Churn use case as beforehand described. On the Amazon SageMaker console, develop the Governance part and select Mannequin playing cards.

We create the mannequin card in Draft standing with the identify Buyer-Churn-Mannequin-Card. For extra data, discuss with Create a model card. On this demonstration, you may go away the rest of the fields clean and create the mannequin card.

Alternatively, you should utilize the next AWS CLI command to create the mannequin card:

aws sagemaker create-model-card --model-card-name Buyer-Churn-Mannequin-Card --content "{"model_overview": {"model_owner": "model-owner","problem_type": "Buyer Churn Mannequin"}}" --model-card-status Draft

Now, create the cross-account share utilizing AWS RAM. Within the AWS RAM console, choose Create a useful resource share.

Enter a reputation for the useful resource share, for instance “Buyer-Churn-Mannequin-Card-Share”. Within the Sources – optionally available part, choose the useful resource sort as SageMaker Mannequin Playing cards. The mannequin card we created within the earlier step will seem within the itemizing.

Choose that mannequin and it’ll seem within the Chosen sources part. Choose that useful resource once more as proven within the following steps and select Subsequent.

On the following web page, you may choose the Managed permissions. You possibly can create customized permissions or use the default possibility “AWSRAMPermissionSageMakerModelCards” and choose Subsequent. For extra data, discuss with Managing permissions in AWS RAM.

On the following web page, you may choose Principals. Underneath Choose principal sort, select AWS Account and enter the ID of the account of the share the mannequin card. Choose Add and proceed to the following web page.

On the final web page, evaluate the knowledge and choose “Create useful resource share”. Alternatively, you should utilize the next AWS CLI command to create a useful resource share:

aws ram create-resource-share --name <Identify of the Mannequin Card>

aws ram associate-resource-share --resource-share-arn <ARN of useful resource share create from the earlier command> --resource-arns <ARN of the Mannequin Card>

On the AWS RAM console, you see the attributes of the useful resource share. Be sure that Shared sources, Managed permissions, and Shared principals are within the “Related” standing.

After you employ AWS RAM to create a useful resource share, the principals specified within the useful resource share could be granted entry to the share’s sources.

For those who activate AWS RAM sharing with AWS Organizations, and your principals that you simply share with are in the identical group because the sharing account, these principals can obtain entry as quickly as their account administrator grants them permissions.
For those who don’t activate AWS RAM sharing with Organizations, you may nonetheless share sources with particular person AWS accounts which can be in your group. The administrator within the consuming account receives an invite to affix the useful resource share, and so they should settle for the invitation earlier than the principals specified within the useful resource share can entry the shared sources.
You may as well share with accounts outdoors of your group if the useful resource sort helps it. The administrator within the consuming account receives an invite to affix the useful resource share, and so they should settle for the invitation earlier than the principals specified within the useful resource share can entry the shared sources.

For extra details about AWS RAM, discuss with Terms and concepts for AWS RAM.

Accessing shared mannequin playing cards

Now we will log in to the shared AWS account to entry the mannequin card. Just be sure you are accessing the AWS console utilizing IAM permissions (IAM position) which permit entry to AWS RAM.

With AWS RAM, you may view the useful resource shares to which you might have been added, the shared sources you could entry, and the AWS accounts which have shared sources with you. You may as well go away a useful resource share once you not require entry to its shared sources.

To view the mannequin card within the shared AWS account:

Navigate to the Shared with me: Shared resources web page within the AWS RAM console.
Just be sure you are working in the identical AWS area the place the share was created.
The mannequin shared from the mannequin account can be out there within the itemizing. If there’s a lengthy record of sources, then you may apply a filter to search out particular shared sources. You possibly can apply a number of filters to slender your search.
The next data is accessible:
1. Useful resource ID – The ID of the useful resource. That is the identify of the mannequin card that we created earlier within the mannequin card account.
2. Useful resource sort – The kind of useful resource.
3. Final share date – The date on which the useful resource was shared with you.
4. Useful resource shares – The variety of useful resource shares wherein the useful resource is included. Select the worth to view the useful resource shares.
5. Proprietor ID – The ID of the principal who owns the useful resource.

You may as well entry the mannequin card utilizing the AWS CLI possibility. For the AWS IAM coverage configured with the proper credentials, just be sure you have permissions to create, edit, and delete mannequin playing cards inside Amazon SageMaker. For extra data, discuss with Configure the AWS CLI.

You need to use the next AWS IAM permissions coverage as template:

{
     "Model": "2012-10-17",
     "Assertion": [
        {
             "Effect": "Allow",
             "Action": [
                 "sagemaker:DescribeModelCard",
                 "sagemaker:UpdateModelCard",
                 "sagemaker:CreateModelCardExportJob",
                 "sagemaker:ListModelCardVersions",
                 "sagemaker:DescribeModelCardExportJob"
             ],
             "Useful resource": [
                 "arn:aws:sagemaker:AWS-Region:AWS-model-card-account-id:model-card/example-model-card-name-0",
                 "arn:aws:sagemaker:AWS-Region:AWS-model-card-account-id:model-card/example-model-card-name-1/*"
             ]
        },
        { 
             "Impact": "Enable", 
             "Motion": "s3:PutObject",
             "Useful resource": "arn:aws:s3:::Amazon-S3-bucket-storing-the-pdf-of-the-model-card/model-card-name/*"
        }
    ]
}

You possibly can run the next AWS CLI command to entry the small print of the shared mannequin card.

aws sagemaker describe-model-card --model-card-name <ARN of the mannequin card>

Now you can also make modifications to this mannequin card from this account.

aws sagemaker update-model-card --model-card-name <ARN of the Mannequin Card> --content "{"model_overview": {"model_owner": "model-owner","problem_type": "Buyer Churn Mannequin"}}"

After you make modifications, return to the mannequin card account to see the modifications that we made on this shared account.

The issue sort has been up to date to “Buyer Churn Mannequin” which we had supplied as a part of the AWS CLI command enter.

Clear up

Now you can delete the mannequin card you created. Just be sure you delete the AWS RAM useful resource share that you simply created to share the mannequin card.

Conclusion

On this put up, we supplied an summary of multi-account structure for scaling and governing your ML workloads securely and reliably. We mentioned the structure patterns for establishing mannequin card sharing and illustrated how centralized mannequin card sharing patterns work. Lastly, we arrange mannequin card sharing throughout a number of accounts for bettering visibility and governance in your mannequin growth lifecycle. We encourage you check out the brand new mannequin card sharing function and tell us your suggestions.

In regards to the authors

Vishal Naik is a Sr. Options Architect at Amazon Net Companies (AWS). He’s a builder who enjoys serving to clients accomplish their enterprise wants and resolve complicated challenges with AWS options and finest practices. His core space of focus consists of Machine Studying, DevOps, and Containers. In his spare time, Vishal loves making quick movies on time journey and alternate universe themes.

Ram Vittal is a Principal ML Options Architect at AWS. He has over 20 years of expertise architecting and constructing distributed, hybrid, and cloud purposes. He’s captivated with constructing safe and scalable AI/ML and large knowledge options to assist enterprise clients with their cloud adoption and optimization journey to enhance their enterprise outcomes. In his spare time, he rides his bike and walks together with his 2-year-old sheep-a-doodle!