
Building on the success of the Segment Anything Model (SAM), Meta has released an upgraded version called the Segment Anything Model 2 (SAM 2).
SAM 2 is a computer vision model designed to identify and separate objects in images or videos quickly. It operates in real-time and can be "prompted" to focus on specific objects, making it highly effective and advanced at recognizing and isolating objects from their backgrounds.
In this article, we’ll explore how to deploy the SAM 2 model to a REST API using Modelbit.
Prerequisites
To get the most out of this article, follow these steps:
1. Access the SAM 2 Model
Start by downloading the SAM 2 model from the official Meta AI repository. Open your command line interface and run the following commands:
Next, download the model checkpoints by navigating to the checkpoints directory and running the script:
For more detailed installation instructions, refer to the SAM 2 GitHub repository.
2. Set Up Modelbit
To deploy the SAM 2 model, you'll need a Modelbit account. Head over to the Modelbit website and sign up.
Once registered, install the Modelbit Python library by running:
This will allow you to interact with Modelbit and deploy your SAM 2 model as a REST API endpoint.
Overview of SAM 2
The SAM 2 model, an advanced iteration of Meta AI's Segment Anything Model, significantly enhances image and video segmentation. It is engineered to deliver rapid and precise segmentation, making it six times faster than its predecessor.
SAM 2's core features include its ability to handle real-time video segmentation and its superior accuracy across complex and diverse scenarios.
Built on an extensive dataset of over 50,000 videos and millions of segmentation masks, SAM 2 can segment objects in both images and videos with exceptional detail. These capabilities make it ideal for applications in augmented reality, autonomous driving, environmental monitoring, and more.
Key Features and Enhancements of SAM 2
Memory Mechanism: Incorporates a memory encoder, memory bank, and memory attention module to store and use object information, enhancing user interaction throughout the video.
Streaming Architecture: Processes video frames sequentially, enabling real-time segmentation of long videos.
Enhanced Image Segmentation: Offers superior performance in image segmentation compared to the original SAM, with exceptional capabilities in video tasks.
Multiple Mask Prediction: Provides several potential segmentation masks when faced with uncertain image or video data.
Occlusion Prediction: Enhances the model’s ability to handle objects that are temporarily obscured or leave the frame.
Video Segmentation: Tracks objects across all video frames, effectively managing occlusion.
SAM 2 in Action
You can easily test and use SAM 2 through the Web UI provided by Meta. To get started, visit SAM 2 Web UI.
Working With SAM 2
Now that you have an understanding of SAM 2's capabilities, it's time to put it into action programmatically. Getting started with SAM 2 is straightforward. In this tutorial, we'll use SAM 2 to generate segmentation masks for an image.
In the context of image segmentation, a mask is typically a binary or multi-class image that matches the size of the input image. Each pixel in the mask is labeled or assigned a value indicating whether it belongs to a specific object or region of interest.
When you feed an image into the SAM 2 model, the mask generator will output an image where different objects—such as cars, people, or animals—are highlighted with distinct colours or binary values.
This capability is important for various real-world applications, including:
Autonomous driving: Helping vehicles recognize and differentiate between roads, pedestrians, other vehicles, and more.
Medical imaging: Allowing for the segmentation of different tissues, organs, or abnormalities within an image.
Image editing: Facilitating the isolation of specific objects from their background for easier manipulation.
Let's dive into how to get SAM 2 up and running.
Make sure you’ve downloaded SAM 2 in your development environment. If not, refer back to the prerequisites.
Next, check if a CUDA-compatible GPU is available on the system and optimize the execution of the PyTorch model accordingly by running the following code:
Next, create a function called
display_image_with_annotations to visually represent segmentation masks on an image. This function will overlay the masks with random colours and can optionally draw borders around the segmented regions, enhancing visibility and differentiation between various segments in the image. Below is the code:To test this model, we need an image. For this tutorial, let's use a free image from Unsplash. You can download the image using the following link: Download the image from Unsplash
Make sure to download the image to your local environment for the demonstration.
Load the image in your notebook:
Next, initialize the SAM 2 model for the image segmentation task by running this code:
Finally, call the
display_image_with_annotations function to show the segmentation mask on the image.Here is the result:
You can see that the model accurately segments each region of the image, highlighting different sections with precision. You can repeat this for different images and see how powerful SAM 2 is.
Deploying SAM 2 Model With Modelbit
The true value of an AI model is only realized when it is made available to end users, typically through deployment in a production environment. One effective method for achieving this is by deploying the model as a REST API. Modelbit offers a straightforward approach for rapidly deploying your AI models. You can learn more about this solution at Modelbit.
To begin deploying the SAM 2 model, import Modelbit and activate it with the following code:
Let's deploy using Modelbit's Python method. Remember, we already have a function,
display_image_with_annotations, to mask an image with SAM 2. Here’s how to do it:Modelbit will manage all dependencies for you, including any other Python functions and variables that the function depends on.
Result:
Accessing the Model
The model has been successfully deployed as a REST API endpoint using Modelbit. You can access the model easily via various methods, including curl or Python. Once accessed, you can integrate the API endpoint into your applications to make inferences effortlessly.
Here is an example using Python:
Final Thoughts
Deploying a model as a REST API endpoint using Modelbit simplifies the process of integrating advanced functionality into your applications. With easy access through tools like curl or Python, you can incorporate the model into your workflows, enabling efficient and scalable inferences.
In this article, you’ve learned how to effortlessly deploy one of the latest and most advanced AI models as a REST API. Whether you're working with image recognition, natural language processing, or any other AI domain, the ease of integration provided by Modelbit can help you bring sophisticated AI features to your projects with minimal effort.
0 likes · 10 views · Apr 2, 2025
Building on the success of the Segment Anything Model (SAM), Meta has released an upgraded version...