SAM 3.1 video selection

Choose an object on one frame and keep its selection through the video. SAM 3.1 takes an original clip plus text, points or boxes and returns one lossless black-and-white mask MP4. White marks the selected object. The mask keeps the source dimensions, frame count and frame rate, so it can be used directly for compositing, occlusion and moving selections.

The selection can start on any frame. Tracking runs forward and backward from that frame, following the chosen instance rather than adding other objects that match its name.

Task	Model ID	Input	Output
Video object selection	`sam3_video_segment_fp16`	`referenceVideo` and `sam3VideoPrompt`	One lossless binary mask MP4

Use the JavaScript or Python SDK, or the Creative Agent CLI. For an original still, use the separate SAM 3 image workflow.

#Source requirements

Up to 30 seconds and 900 frames, with neither edge exceeding 2560 pixels.
A constant frame rate from 1 to 60 fps, including fractional rates.
100 MB or less, SDR, square pixels, with rotation already applied.
Upload the original clip. The server reads its actual geometry and timestamps before charging. If you supply width, height, frames or fps, they must match the upload.

The output is a hard selection mask. It has no audio or transparency channel. Feathered alpha mattes and transparent cutout videos are separate compositing steps.

#Selection prompt

Use normalized coordinates from 0 to 1 in the original video's frame. The origin is its top-left corner. The existing image selection rules for signed points, boxes and text also apply.

Field	Meaning
`text`	A short name for the visible object, such as `unicorn horn`.
`points`	Positive and negative `{ x, y, label }` clicks on the selected frame.
`boxes`	Positive and negative `{ x0, y0, x1, y1, label }` boxes. Negative boxes require text.
`frameIndex`	Zero-based selection frame; default `0`.
`objectIndex`	Zero-based index among matches sorted by score; default `0`, the strongest match.
`threshold`	Confidence threshold from 0 to 1; default `0.5`.
`output`	`mask`.
`edges`	`hard`.

One job tracks one selected instance. Text and points cannot be combined. applyMask and multi-instance unions belong to the image workflow; use objectIndex to choose a video instance.

#JavaScript SDK

import { readFileSync } from 'node:fs';
import { SogniClient, SAM3_VIDEO_SEGMENT_MODEL_ID } from '@sogni-ai/sogni-client';

const client = await SogniClient.createInstance({
  appId: crypto.randomUUID(),
  apiKey: process.env.SOGNI_API_KEY,
  network: 'fast'
});

const project = await client.projects.create({
  type: 'video',
  modelId: SAM3_VIDEO_SEGMENT_MODEL_ID,
  positivePrompt: '',
  numberOfMedia: 1,
  referenceVideo: new Blob([readFileSync('clip.mp4')], { type: 'video/mp4' }),
  sam3VideoPrompt: { text: 'unicorn horn', frameIndex: 127 }
});

const [maskUrl] = await project.waitForCompletion();
console.log(maskUrl);
client.disconnect();

The upload is preserved at its original size and timing. Omit generation controls such as steps, resizing, interpolation or trimming.

#Python SDK

import asyncio
import os
from sogni_client import SogniClient

async def main():
    async with await SogniClient.create(
        api_key=os.environ['SOGNI_API_KEY'], app_id='video-selection'
    ) as client:
        project = await client.projects.create(
            type='video',
            model_id='sam3_video_segment_fp16',
            positive_prompt='',
            number_of_media=1,
            reference_video='clip.mp4',
            sam3_video_prompt={'text': 'unicorn horn', 'frameIndex': 127},
        )
        print(await project.wait_for_completion())

asyncio.run(main())

#CLI

sogni-agent --segment-video clip.mp4 --segment-text "unicorn horn" \
  --segment-frame 127 -o mask.mp4

Use --segment-point, --segment-exclude and --segment-box for spatial prompts. Signed boxes are available through the SDK. --segment-object chooses another ranked instance. --segment-frame is a frame index, not a timestamp in seconds.

#Pricing and timing

Video selection is priced from the source's verified frame count and pixel area, with a per-frame cost for the tracker. Request a live estimateVideoCost() using the source's width, height, frames, fps and steps: 1; the submitted job is charged from the uploaded bytes.

A 1920 × 1080 clip with 255 frames at 24 fps took 38.27 seconds end to end on an RTX 4090 in qualification. A 720-frame, 30-second clip at the same dimensions took 106.12 seconds. These single-run measurements include local delivery; Supernet upload, queuing and download time depend on the job and connection.

#Result provenance

Each completed job carries the mask SHA-256, source-video SHA-256, normalized selection-prompt SHA-256, mask width, height, frame count and frame rate, plus the SAM version. Keep that receipt with the source and mask when a later edit depends on exact alignment.

Workers need Comfy Worker 1.0.252 or newer and the independent SAM3 video optional pack on a local NVIDIA CUDA 13 runtime. The existing image SAM pack remains separate.