Source: https://docs.sogni.ai/api-reference/media-utilities/

# Media utilities, 3D and speech

Submit these media tools as [durable workflow](https://docs.sogni.ai/api-reference/workflows/) steps to `POST /v1/creative-agent/workflows`, or use them through hosted chat. The synchronous `POST /v1/creative-agent/tools/execute` endpoint runs only the composition and planning tools and returns `400` for media tools.

## [#](https://docs.sogni.ai/api-reference/media-utilities/#supply-source-media)Supply source media

List source files in the request's `media_references` array, then select them by index. Negative indices select request media of that type, in order: `-1` is the first image (or video, or audio file), `-2` the second. Non-negative indices select results generated earlier in the same workflow. Use `dependsOn` to pass one step's output to the next. URLs must be HTTP(S); inline `data:` URIs are rejected. Upload local files first with the [media URL endpoints](https://docs.sogni.ai/api-reference/media/).

## [#](https://docs.sogni.ai/api-reference/media-utilities/#estimate-before-starting)Estimate before starting

The examples send `confirm_cost: false`, which checks the plan and estimates it without saving or starting anything. The API responds `400` with `details.estimatedCapacity` (a per-step estimate in capacity units, not Spark) and `details.confirmationRequired: true`. Resubmit with `confirm_cost: true`, or without the field, to start, then read the workflow or subscribe to its event stream for results. The exact charge is computed per job when each step runs.

Every tool on this page also respects the request-level `safe_content_filter` setting (default `true`); there is no per-step filter argument.

## [#](https://docs.sogni.ai/api-reference/media-utilities/#remove-a-background)Remove a background

```
{
  "media_references": [
    { "kind": "image", "url": "https://your-host.example/original.png" }
  ],
  "input": {
    "steps": [
      {
        "id": "remove_background",
        "toolName": "remove_background",
        "arguments": { "sourceImageIndex": -1, "applyMask": true }
      }
    ]
  },
  "token_type": "spark",
  "confirm_cost": false
}
```

BiRefNet preserves the original foreground and returns a transparent PNG. Set `applyMask: false` to return the soft foreground mask instead of the cutout. No prompt is required. The SDK project equivalent uses `modelId: 'birefnet_image_background_removal_fp16'`, `startingImage`, and `applyMask`.

A transparent cutout is not a valid first frame, last frame, or reference for Wan 3 video models, which reject PNGs with a transparency channel. Flatten it first.

## [#](https://docs.sogni.ai/api-reference/media-utilities/#reconstruct-a-textured-3d-model)Reconstruct a textured 3D model

```
{
  "media_references": [
    { "kind": "image", "url": "https://your-host.example/front.png" },
    { "kind": "image", "url": "https://your-host.example/left.png" },
    { "kind": "image", "url": "https://your-host.example/back.png" },
    { "kind": "image", "url": "https://your-host.example/right.png" }
  ],
  "input": {
    "steps": [
      {
        "id": "image_to_3d",
        "toolName": "image_to_3d",
        "arguments": {
          "sourceImageIndex": -1,
          "leftViewImageIndex": -2,
          "backViewImageIndex": -3,
          "rightViewImageIndex": -4,
          "meshTargetFaces": 50000,
          "textureSize": 2048,
          "shapeResolution": 1024
        }
      }
    ]
  },
  "token_type": "spark",
  "confirm_cost": false
}
```

The front image (`sourceImageIndex`) is required. Any subset of the optional orbit views selects Pixal3D multi-view reconstruction; front-only uses single-view reconstruction. Views describe the subject's own sides: left means its left side faces camera (subject faces screen-left), right means screen-right. Use original stills of the same subject at consistent height and framing. The orbit views also accept URLs directly as `left_view_image_url`, `back_view_image_url`, and `right_view_image_url`.

| Control | Allowed values | Default |
| --- | --- | --- |
| `meshTargetFaces` | Integer 5,000–700,000 | 700,000 |
| `textureSize` | Integer 1,024–4,096 | 4,096 |
| `normalMapSize` | Integer 512–2,048 | 2,048 |
| `ambientOcclusionSize` | Integer 256–1,024 | 1,024 |
| `shapeResolution` | 1,024 or 1,536 | 1,024 |

The result is a binary GLB (`model/gltf-binary`). Workflow artifacts report `kind: "model"`; hosted chat tool results report `mediaType: "model"`. Download or display it as a 3D model rather than passing it to an image tool. The SDK uses `pixal3d_int8_i23d` or `pixal3d_multiview_int8_i23d`, `startingImage`, and named `leftViewImage`, `backViewImage`, and `rightViewImage` binary inputs. Generic `contextImages` do not identify orbit views.

## [#](https://docs.sogni.ai/api-reference/media-utilities/#select-an-object)Select an object

`segment_image` takes `sourceImageIndex` plus `text`, `points`, or `boxes`. Point coordinates (`x`, `y`) and box coordinates (`x0`, `y0`, `x1`, `y1`) are normalized to the original image's 0–1 range. Points carry a `positive` or `negative` label; boxes default to positive and can carry an exclusion label with a text prompt.

-   Provide `text`, at least one positive point, or a box. Negative points alone are rejected.
-   `text` is 1–240 characters, `points` up to 32, and `boxes` up to 16, with `x0 < x1` and `y0 < y1`.
-   Text can combine with boxes. Points can combine with at most one positive box and cannot combine with text.
-   `threshold` is 0–1 (default 0.5) and `maxInstances` is 1–16. `multimask` defaults to `true` with points and is rejected without them.

`applyMask: true` returns a transparent cutout; the default returns a PNG mask. Output matches the source's dimensions. The SDK equivalent is `sam3_image_segment_bf16` with `startingImage` and `sam3Prompt`.

## [#](https://docs.sogni.ai/api-reference/media-utilities/#upscale-a-video)Upscale a video

`upscale_video` runs FlashVSR on an existing video without a prompt. It keeps every frame, the frame rate, and the audio.

| Argument | Values |
| --- | --- |
| `sourceVideoIndex` | Negative for request videos, non-negative for earlier results. Omitted means the latest video. |
| `targetResolution` | Short edge `1080` or `1440`. Defaults to 1440 when the source short edge is at least 720 px, otherwise 1080. |
| `detailPreference` | `stable` (default) or `sharper` |
| `processingSpeed` | `stable` (default) or `faster` |
| `seed` | 0–4294967295 (default 0); `-1` picks a random seed |

The source needs a short edge of 540–768 px (at least 720 px for 1440), a constant frame rate from 1 to 60 fps, standard dynamic range, and at most 100 MB. Output is at most 2560×1440. Options do not change the price. If no worker that supports a non-default option is connected, the step reports that it is not available.

## [#](https://docs.sogni.ai/api-reference/media-utilities/#speech-and-reference-audio-cloning)Speech and reference-audio cloning

`generate_speech` reads `prompt` as the literal script (up to 4,096 characters). Select `model: 'voice'` for a studio speaker, `'clone'` for a supplied recording, or `'design'` for a voice described in `voiceDescription`.

-   **Studio voices** (`voice`): `serena` (default), `vivian`, `uncle_fu`, `ryan`, `aiden`, `ono_anna`, `sohee`, `eric`, and `dylan`. `voiceDescription` can add direction.
-   **Clone** uses `voiceSourceIndex` (or `voice_source_url`) and optional `voiceTranscript` (up to 1,024 characters). Supply the original recording; a clean 3–30 second clip of one speaker works best. Cloning conditions synthesis on that recording and does not create a persistent trained voice. `voiceDescription` is ignored in clone mode.
-   **Design** uses `voiceDescription` (up to 512 characters) and takes no recording.

Other controls: `language` (`auto` by default, or `english`, `chinese`, `japanese`, `korean`, `german`, `french`, `russian`, `portuguese`, `spanish`, or `italian`), `creativity` (0.1–2, default 0.9), `outputFormat` (`wav`, `mp3`, `flac`), `seed`, and `numberOfVariations` (1–16).

## [#](https://docs.sogni.ai/api-reference/media-utilities/#minimax-h3-two-stage-and-audio-driven-video)MiniMax H3 Two-Stage and audio-driven video

**Two-Stage.** Use `videoModel: 'minimax-h3-fasth3-t2v-turbo-2stage'` with `generate_video`, or the corresponding `i2v` or `flf2v` selector with `animate_photo`. Set `targetResolution` to `720`, `1080`, or `1440` (default). The worker renders the base clip, then enlarges and refines it, so this is upscaled delivery rather than native 2K. Landscape output is 1344×768, 1920×1088, or 2688×1536; portrait is mirrored. Other values are rejected. Price the selected mode and resolution together.

**Audio-driven.** `sound_to_video` accepts these FastH3 selectors, each also in a `-2stage` form:

| `videoModel` | Inputs |
| --- | --- |
| `minimax-h3-fasth3-ia2v-turbo` | Audio plus a first frame (`sourceImageIndex`) |
| `minimax-h3-fasth3-flfa2v-turbo` | Audio plus first and last frames (`sourceImageIndex`, `endImageIndex`) |
| `minimax-h3-fasth3-a2v-turbo` | Audio only |

Select audio with `audioSourceIndex` (`-1` is the first request audio). The uploaded audio drives the clip and is kept as its soundtrack. Clips render between about 5.2 and 15.1 seconds, snapped to 24 fps. These modes reject `generateAudio: false`, `negativePrompt`, LoRAs, and loose reference arrays.

Non-audio H3 modes can use LoRAs, including compatible [personal LoRAs](https://docs.sogni.ai/api-reference/personal-loras/).