How We Keep AI Characters Consistent Across Dozens of Story Images

Ask an AI image model to draw “a tall woman with dark curly hair and a scar over one eyebrow” three times and you will get three different women. Same prompt, same model, three strangers. Each generation is a fresh roll of the dice, and the model has no idea the three requests were supposed to be the same person.

For one-off image generation, that is a quirk. For illustrated fiction, it is fatal. A story is a relationship with a cast. If the love interest has brown eyes in chapter two and green eyes in chapter six, the reader notices instantly, and the illusion that they are following real people collapses. Readers forgive a lot from AI stories. They do not forgive the main character becoming a different human being between scenes.

A full-length NovelFlame story generates illustrations at key moments from the opening scene to the finale, plus character portraits along the way. Keeping a cast visually stable across that many independent generations is one of the hardest problems we work on, and this post explains the actual system, including the parts that still fail.

Why prompts alone cannot fix this

The obvious first idea is to write a very detailed character description and reuse it in every image prompt. We tried it. Everyone tries it. It fails for two reasons.

First, text is a lossy way to describe a face. “Angular jaw, hazel eyes, auburn hair in a loose braid” narrows the space from millions of faces to thousands. The model still picks a different one of those thousands each time.

Second, image models weight the whole prompt at once. As the scene description grows (the burning warehouse, the rain, the crowd, the lighting), the character description becomes a smaller fraction of the model’s attention, and consistency degrades exactly when scenes get interesting.

So text descriptions are necessary, but they are the floor, not the system.

Layer 1: every important character gets a visual anchor

The core idea in our pipeline is that a character should never be rendered from prose alone. Before a character appears in scene illustrations, we establish a visual anchor for them, which takes one of two forms.

The strong form is a reference portrait. Characters you create and save get a canonical portrait generated once, up front. Characters who are invented mid-story get an inline portrait generated the moment they become important. That portrait, an actual image, becomes the character’s visual ground truth for the rest of the story.

The weaker form is an identity lock. When a portrait does not exist yet, we capture a structured set of appearance attributes from the character’s first render (things like hair, eyes, build, skin tone, distinguishing features) and freeze them. A lock is not as strong as a picture, but it is machine-checkable, which matters in layer three.

Layer 2: the right model for the job

Not all image models can use a reference image, and the ones that can are more expensive. So the pipeline routes by task. Baseline scene illustration goes to a fast model that is good at atmosphere and composition. When a scene contains an anchored character, generation routes to a model that accepts the reference portrait as a conditioning input, so it is not imagining the character from text but working from the actual face.

This split matters for cost too. Reference-conditioned generation costs roughly three times our baseline per image. Spending that only where a known character is on screen is the difference between a sustainable product and a very pretty bonfire.

Layer 3: an AI that rejects the AI’s work

Here is the part most platforms skip. Even with a reference image, some renders come back wrong. The face drifts, the hair color shifts, the build changes. If you ship whatever the model returns, your consistency is only as good as your worst roll.

So every render of an anchored character goes through an automated vision check before the reader sees it. A vision model compares the generated image against the character’s anchor and answers a blunt question: is this the same person? A second check looks for visual defects, such as broken anatomy or garbled composition. Fail either check and the image is rejected and regenerated. The reader never sees the reject; they just see a slightly later, correct image.

This is the unglamorous secret of the whole system. Consistency does not come from one great generation. It comes from being willing to throw away bad ones automatically.

Layer 4: style is part of identity

A character is not just a face. If chapter three is rendered as a photograph and chapter four as a watercolor, the cast feels replaced even when the faces match. Every NovelFlame story locks an art style at the start, chosen to fit the genre, and every image prompt in that story carries the same style contract. Horror stays in its palette. A Regency romance never suddenly looks like a comic book.

What still breaks, honestly

The system is not perfect, and pretending otherwise would undercut the point of writing this. Minor supporting characters are the current weak spot: when a scene is crowded, background cast members may render without a full anchor, and they can drift between appearances. There are also timing races in a live story, where a scene needs to render before a brand-new character’s portrait has finished generating. We measure both cases and we are actively closing them, but if you read enough NovelFlame stories you will still occasionally catch a side character who changed haircuts between scenes.

The protagonist and the leads, though, are the priority, and that is where the anchoring, routing, and rejection layers concentrate. The person you spend the story with should stay the person you spend the story with.

Why we spend this much effort on it

Because it is the difference between generating pictures and illustrating a story. Anyone can bolt an image model onto a text model. The hard part is making image seventeen agree with image three about who someone is. That agreement is what lets a reader stop evaluating the pictures and start caring about the people in them, and it is the single feature we hear about most from readers who came from other AI story tools.

If you want to check our work, the demo below is free and needs no account. Watch the characters between scenes. That stability is what this whole post was about.