Falcon 3 fashions now obtainable in Amazon SageMaker JumpStart

As we speak, we’re excited to announce that the Falcon 3 household of fashions from TII can be found in Amazon SageMaker JumpStart. On this put up, we discover tips on how to deploy this mannequin effectively on Amazon SageMaker AI.

Overview of the Falcon 3 household of fashions

The Falcon 3 household, developed by Know-how Innovation Institute (TII) in Abu Dhabi, represents a big development in open supply language fashions. This assortment contains 5 base fashions starting from 1 billion to 10 billion parameters, with a give attention to enhancing science, math, and coding capabilities. The household consists of Falcon3-1B-Base, Falcon3-3B-Base, Falcon3-Mamba-7B-Base, Falcon3-7B-Base, and Falcon3-10B-Base together with their instruct variants.

These fashions showcase improvements comparable to environment friendly pre-training strategies, scaling for improved reasoning, and information distillation for higher efficiency in smaller fashions. Notably, the Falcon3-10B-Base mannequin achieves state-of-the-art efficiency for fashions underneath 13 billion parameters in zero-shot and few-shot duties. The Falcon 3 household additionally contains varied fine-tuned variations like Instruct fashions and helps totally different quantization codecs, making them versatile for a variety of purposes.

Presently, SageMaker JumpStart presents the bottom variations of Falcon3-3B, Falcon3-7B, and Falcon3-10B, together with their corresponding instruct variants, in addition to Falcon3-1B-Instruct.

Get began with SageMaker JumpStart

SageMaker JumpStart is a machine studying (ML) hub that may assist speed up your ML journey. With SageMaker JumpStart, you’ll be able to consider, evaluate, and choose pre-trained basis fashions (FMs), together with Falcon 3 fashions. These fashions are totally customizable on your use case along with your knowledge.

Deploying a Falcon 3 mannequin by SageMaker JumpStart presents two handy approaches: utilizing the intuitive SageMaker JumpStart UI or implementing programmatically by the SageMaker Python SDK. Let’s discover each strategies that can assist you select the strategy that most accurately fits your wants.

Deploy Falcon 3 utilizing the SageMaker JumpStart UI

Full the next steps to deploy Falcon 3 by the JumpStart UI:

1. To entry SageMaker JumpStart, use one of many following strategies:
   1. In Amazon SageMaker Unified Studio, on the Construct menu, select JumpStart fashions underneath Mannequin improvement.
      
   2. Alternatively, in Amazon SageMaker Studio, select JumpStart within the navigation pane.
      

2. Seek for Falcon3-10B-Base within the mannequin browser.
   
3. Select the mannequin and select Deploy.
   
4. For Occasion kind, both use the default occasion or select a distinct occasion.
5. Select Deploy.
   After a while, the endpoint standing will present as InService and it is possible for you to to run inference towards it.
   

Deploy Falcon 3 programmatically utilizing the SageMaker Python SDK

For groups trying to automate deployment or combine with present MLOps pipelines, you should utilize the SageMaker Python SDK:

from sagemaker.serve.builder.model_builder import ModelBuilder
from sagemaker.serve.builder.schema_builder import SchemaBuilder
from sagemaker.jumpstart.mannequin import ModelAccessConfig
from sagemaker.session import Session
import logging

sagemaker_session = Session()

artifacts_bucket_name = sagemaker_session.default_bucket()
execution_role_arn = sagemaker_session.get_caller_identity_arn()


js_model_id = "huggingface-llm-falcon-3-10B-base"

gpu_instance_type = "ml.g5.12xlarge"  

response = "Hey, I am a language mannequin, and I am right here that can assist you along with your English."

sample_input = {
    "inputs": "Hey, I am a language mannequin,",
    "parameters": {"max_new_tokens": 128, "top_p": 0.9, "temperature": 0.6},
}

sample_output = [{"generated_text": response}]

schema_builder = SchemaBuilder(sample_input, sample_output)

model_builder = ModelBuilder(
    mannequin=js_model_id,
    schema_builder=schema_builder,
    sagemaker_session=sagemaker_session,
    role_arn=execution_role_arn,
    log_level=logging.ERROR
)

mannequin= model_builder.construct()

predictor = mannequin.deploy(model_access_configs={js_model_id:ModelAccessConfig(accept_eula=True)}, accept_eula=True)

Run inference on the predictor:

predictor.predict(sample_input)

If you wish to arrange the power to scale all the way down to zero after deployment, consult with Unlock cost savings with the new scale down to zero feature in SageMaker Inference.

Clear up

To wash up the mannequin and endpoint, use the next code:

predictor.delete_model()
predictor.delete_endpoint()

Conclusion

On this put up, we explored how SageMaker JumpStart empowers knowledge scientists and ML engineers to find, entry, and run a variety of pre-trained FMs for inference, together with the Falcon 3 household of fashions. Go to SageMaker JumpStart in SageMaker Studio now to get began. For extra info, consult with SageMaker JumpStart pretrained models, Amazon SageMaker JumpStart Foundation Models, and Getting started with Amazon SageMaker JumpStart.

In regards to the authors

Niithiyn Vijeaswaran is a Generative AI Specialist Options Architect with the Third-Social gathering Mannequin Science crew at AWS. His space of focus is generative AI and AWS AI Accelerators. He holds a Bachelor’s diploma in Laptop Science and Bioinformatics.

Marc Karp is an ML Architect with the Amazon SageMaker Service crew. He focuses on serving to prospects design, deploy, and handle ML workloads at scale. In his spare time, he enjoys touring and exploring new locations.

Raghu Ramesha is a Senior ML Options Architect with the Amazon SageMaker Service crew. He focuses on serving to prospects construct, deploy, and migrate ML manufacturing workloads to SageMaker at scale. He makes a speciality of machine studying, AI, and pc imaginative and prescient domains, and holds a grasp’s diploma in Laptop Science from UT Dallas. In his free time, he enjoys touring and images.

Banu Nagasundaram leads product, engineering, and strategic partnerships for SageMaker JumpStart, SageMaker’s machine studying and GenAI hub. She is keen about constructing options that assist prospects speed up their AI journey and unlock enterprise worth.