Three ways to start a video job, and when each one wins

Updated 2026-10-17

A video generation API usually exposes several ways to say what you want. They differ in how much of the result you control and how much you leave to the model. On the VideoRouter endpoint all of them go through the same POST /videos; which fields you send selects the mode. That is convenient, and it is also where most mistakes happen, so this page lists the fields and failure modes for each.

Text-to-video

The baseline: a prompt, a model, optionally a duration, aspect ratio and resolution.

{
  "model": "alibaba/wan-3.0",
  "prompt": "a paper airplane gliding over a city at dusk, slow tracking shot",
  "duration_secs": 8,
  "aspect_ratio": "16:9"
}

Use when you are exploring, the subject is generic (landscapes, abstract motion, b-roll), or nothing existing needs to be preserved.

Pitfalls. Identity is not stable: ask for the same character twice and you get two characters. duration_secs is snapped to a value the model supports, and some models bill a fixed length regardless of what you request, so read what the job returns rather than assuming your number was honoured.

Image-to-video

Add start_image_url, a public https URL or an inline data:image/...;base64,... URI. The model animates that frame.

{
  "model": "bytedance/seedance-2.0/fal",
  "prompt": "the subject turns and smiles, gentle camera push-in",
  "start_image_url": "https://example.com/hero.jpg"
}

Use when a specific frame must be the opening shot: product photos, key art, a storyboard frame you approved, or continuing shot two from the last frame of shot one.

Pitfalls.

• On most models the id is the same as for text-to-video; omit the image and you get text-to-video. A few models have no text mode and return a 400 without start_image_url; conversely, models that do not support image input reject it with a 400.
• Describe motion, not content. The image already says what is in the frame; repeating it in the prompt tends to add noise rather than control.
• If the file only exists locally, POST /uploads stages it and returns a presigned URL that expires in 30 minutes. Use it immediately in the next call; do not store it.

Reference-to-video

This is a different mode, not a bigger image-to-video. Instead of fixing the first frame, you hand the model several references and it composes a new scene using them. The fields extend the same shape as image references:

{
  "model": "bytedance/seedance-2.0/fal",
  "prompt": "the cat from the reference image walks across the scene",
  "input_references": [{"type": "image_url", "image_url": {"url": "https://example.com/cat.jpg"}}],
  "input_video_references": [{"type": "video_url", "video_url": {"url": "https://example.com/motion.mp4"}}],
  "input_audio_references": [{"type": "audio_url", "audio_url": {"url": "https://example.com/ambience.mp3"}}],
  "duration_secs": 5
}

Use when you need a recurring character, product or location across many clips, or want to borrow motion or sound from another clip. Per VideoRouter's docs, models that support all three kinds take up to 9 images, 3 videos and 3 audio clips (12 combined); some narrower rows accept images only. Check the model page for what yours accepts.

Pitfalls. At least one reference across the three arrays is required, and exceeding any per-kind cap returns a 400. Reference-to-video is not available on every model, so do not assume a model that does image-to-video also does this.

Side-by-side

A fourth mode you should not confuse with these

Editing an existing clip uses input_video_url, which is mutually exclusive with the fields above on nearly every model. If you pass it together with start_image_url or references you will get an error, not a blend.

Writing prompts per mode

The prompt does different work in each mode. In text-to-video it carries everything: subject, setting, camera and motion, so spend your words there. In image-to-video the picture already carries subject and setting, so the prompt should be about motion and camera only: what moves, how fast, where the camera goes. In reference-to-video, refer to the references by role ("the character in the first image", "the room in the second") so the model can bind each prompt phrase to an input. Over-long prompts are rarely better in any mode; one clear action per clip is easier to get right than a sequence of events squeezed into five seconds.

A practical progression

1. Prototype the idea in text-to-video on a cheaper model or lower resolution to settle camera and mood.
2. Generate or choose a still for the key frame and move to image-to-video.
3. If the same character or product recurs, collect references and move to reference-to-video on a model that documents it.
4. Re-render only the winners on a premium model. Compare the cost spread in the table below first.

Per second, before VideoRouter's 2.5% platform fee. For tiered models each row compares the resolution tier with the widest host-to-host gap. Built 2026-10-17 from the live catalog.

Because every mode is billed once at creation from the requested duration, an unsupported-mode 400 costs nothing, but a wrong-but-valid request costs a full clip. Validate fields locally before you submit. For concrete scenarios see use cases; for a working request, the quickstart.

Frequently asked questions

Keep reading

• What Is a Video Generation API? How It Works and What It Costs
• Video Generation API Use Cases and the Workflow Each Needs
• Build vs Buy Video Generation: Hosted API or Self-Hosting?
• Best Video Generation APIs: How to Evaluate Them (Checklist)