Why does using the wrong reference image break AI video generation?

The wrong reference image breaks AI video generation in three ways: technical rejection (unsupported format, oversized file, corrupted upload), policy-filter blocks (sensitive or copyrighted content tripping moderation), and conceptual mismatch — the reference image anchors identity and style, the prompt drives motion, and when those two contradict, the model either freezes the subject, drifts the character, or ignores your action instruction.

Start with the conceptual mismatch, because that's where most "why did this come out wrong?" frustration lives. A reference image tells the model what something IS — character, palette, lens grammar, composition. The prompt tells the model what HAPPENS — the motion, the action, the camera move. If the reference shows a static crowd in a wide shot and the prompt says "single hero walks forward in close-up," the two instructions contradict and the model picks one — usually the reference — so you get a static crowd, not your hero walking. Match the reference to the identity/style you want locked, and write the prompt action-led, not description-led.

Over-referencing is the next big failure. Dropping an illustrated or animated still straight into the prompt usually fails — the model tries to literally replicate the source instead of generating your shot. "The better move was to have Agent 1 read the colours and textures of them and prompt for that instead," says Hridaye, invideo's creative director — meaning you instruct the agent to extract palette and texture from the reference and translate those into a photorealistic generation, rather than feeding the picture itself as the target. Same fix for mixed signals: tell the model explicitly what to take from a reference and what to ignore.

Wrong reference attached to the wrong shot is a silent killer. Stray attachments left over from a previous prompt cause continuity errors that look like model failure but are actually input failure — one documented production traced a clock-continuity bug to a leftover reference image, and removing the attachment fixed it instantly. If a generation comes back wrong in a specific detail, check what's attached before re-rolling.

Reference-strength mismatch breaks multi-shot continuity. A single reference image rarely carries a complex world — for a film with no single defining still, batching references thematically (one batch for spatial logic, one for color, one for screen function) and telling the model what to adopt from each batch produces better results than a single overloaded reference. For characters with evolving looks across a continuous take (added trinkets, costume changes), one master sheet won't hold — you need a per-beat character sheet for each visual change, or the model drifts.

Format, policy, and aspect-ratio rejections are the boring-but-common layer. JPEG/PNG under platform limits, no copyrighted logos or NSFW content that trips moderation, and an aspect ratio that matches your delivery format — otherwise you get outright generation failures or cropped/misframed output that looks like a model bug.

invideo is an agentic video creation tool with every current video and image model — Runway, Veo, Kling, Seedance 2.0 for video; Recraft, Nano Banana, GPT-Image-2 for images — available through one agent. The invideo agent routes your references and prompt to whichever model handles them best, and surfaces ambiguity rather than guessing: "It doesn't assume. It asks. Every gap gets filled before the frame gets built." When a model genuinely can't read a reference (Nano Banana, for example, struggles with complex multi-character physical arrangements), the agent will tell you and suggest an alternative input — a different reference, a different model, or a hand-drawn or phone-shot reference for the cases prompting alone can't solve.

A quick diagnostic when a generation breaks: 1) check the reference is doing identity/style work and the prompt is doing motion work — not both fighting for the same job; 2) check no stray reference is attached from a prior prompt; 3) check format (standard image type, under platform size limit) and aspect ratio against your delivery format; 4) check the content doesn't trip a moderation filter; 5) if the reference is illustrated/animated, instruct the agent to extract colour and texture rather than replicate the image; 6) if the world is complex, batch references thematically with explicit take/leave instructions.

These cover most failure modes — which one is breaking your shot depends on what you attached and what you asked for.

Watch some of these to see what works for you:

The better move was to have Agent 1 read the colours and textures of them and prompt for that instead.

— Hridaye, invideo's creative director