ControlNet
ControlNet is structural guidance — instead of (or in addition to) describing a scene in words, you give Sogni Studio a reference image and a type of guidance to extract from it: poses, depth, edges, masks, segmentations. The result keeps that structure while the prompt fills in everything else.
Sketch-to-image with ControlNet — drawing on the left, generated render on the right.
#Setting up ControlNet
- Pick a ControlNet-compatible model. In the Model Explorer, filter by the ControlNet layer icon.
- Drag a reference image or video into the ControlNet panel in the sidebar.
- Choose a preprocessor (optional but recommended for most types — see Preprocessors).
- Pick a ControlNet model matching your guidance type — see ControlNet Models.
- Tune strength and the active window (see "Per-ControlNet parameters" below).
Video preprocessing can be time-consuming. Reduce the animation duration and frame-rate settings to limit the number of imported frames if you only need motion guidance, not every frame.
#ControlNet types in Sogni Studio
The Studio sidebar exposes these ControlNet types, picked from the CN Model dropdown below your reference image:
| Type | What it does | Best paired with |
|---|---|---|
| Sketch | Line-art extraction; renders from outlines | Sketch / Outline preprocessor |
| OpenPose | Body, hand, and (optionally) face pose detection | Pose Capture (and Face Capture for expressions) |
| Depth | Depth-map-based composition | Depth Map preprocessor |
| Segmentation | Semantic regions; preserve spatial layout | U2Net / IS-Net / RMBG preprocessors |
| InstantID | Reference face for identity preservation | Reference image with a clear face |
| Inpaint | Masked-region filling and outpainting | Mask tool or reference with transparent areas |
For Sketch, OpenPose, Depth, and Segmentation you can stack a preprocessor on the same reference to extract the cue the model expects — see Preprocessors.
ControlNet is the legacy spatial-control system. Most modern Sogni Studio workflows lean on Flux Context (multi-image conditioning) instead — see below. ControlNet remains useful for specific tasks like InstantID identity locking and Inpaint masking.
#Per-ControlNet parameters
For each ControlNet you add, the sidebar exposes:
- CN Model — the type (Sketch, OpenPose, Depth, Segmentation, InstantID, Inpaint).
- Strength — how much weight this guidance carries. Lower = subtler influence, higher = more literal.
- Emphasis — Prompt (let the prompt lead), Balanced (default), or ControlNet (let the structural cue lead).
- CN Scheduling — Start% / End% — the window during the denoising process where this ControlNet is active. Default is 0% to 100% (full duration).
A common pattern: high-strength ControlNet from 0% to ~60%, then let the diffusion finish freely for naturalness.
#Flux Context: multi-image conditioning
When you're rendering with a Flux model that supports Context Images, Sogni Studio gives you up to three reference images that condition the generation alongside the prompt — like ControlNet, but operating at a higher semantic level. Use it to:
- Lock identity across renders by adding the same person as a context image.
- Mix two reference styles into a new output.
- Edit-by-prompt: drop in an image and say "change the jacket to red."
Flux Context lives in its own panel alongside ControlNet and uses Flux-specific models. See Context Editing & Mixing for the full workflow.
#ControlNet on the Supernet vs on-device
- Supernet — no downloads. The Model Explorer auto-shows ControlNet variants that the active Supernet supports. See About Switching Models on Supernet.
- On-device — Sogni Studio downloads the ControlNet model weights and a small set of on-device preprocessor models (see Preprocessors page). The first time you use a new preprocessor, expect a small download.
#See also
Need help? Ask in our Discord! Join the Sogni Discord ✨