Zero to Director #5 — The Reference Bust: Style & Identity with IP-Adapter
From Last Time
Post #4 bent wire: an armature that locks structure — edges, depth, pose — while the blueprint rewrites everything else. One thing still had no handle at all.
"Bronze, like the photo" is a sentence. "Like this photograph" is showing. Words are a clumsy way to describe a look: the blueprint translator does its best, but style and identity live in pixels, not in sentences. This post adds the desk-side reference.
Why a Reference Bust
Every sculptor's desk has one: a favorite bust, kept at arm's reach. Not a blueprint — it carries no instructions. Not an armature — it promises no structure. It is just the look: a surface quality, a palette, a way of rendering that the eye copies faster than any sentence could describe it.
The three tools now on the bench, and what each one promises:
- Blueprint (text) — what to carve.
- Armature (ControlNet) — what shape to hold.
- Reference bust (IP-Adapter) — what look to copy.
And just as the armature ate a map, the bust eats an image — but the two images mean different things: the armature's map is a drawing of structure, the bust's reference is a sample of appearance.
How IP-Adapter Works
IP-Adapter is the technique; the name hides a plain idea. Your reference image is pushed through an image encoder into the same kind of embedding space the text translator uses for sentences. From there it rides along the conditioning — every stroke of the carve, the carver is shown the reference alongside the blueprint, like a sculptor glancing sideways at the bust.
Two consequences worth burning in:
- It rides with the text, not over it. Reference and blueprint are teammates: a clear blueprint plus a style bust gives the style something to dress. A vague blueprint hands the bust the whole composition — sometimes that is exactly what you want, more often it is a surprise.
- No structure is promised. The armature pins geometry; the bust pins atmosphere. Feed a portrait as a bust and you get its rendering, not its layout.
The dial is strength again — how often the carver glances at the bust — and, in most implementations, a weight type that biases the glance toward style or toward identity.
The Z-Image Route
ZImage Turbo has no first-party IP-Adapter; the community route fills the gap. The IP-Adapter-for-ZImage node set (Boyo's build) ports the SD3-family adapter weights onto the same Turbo carver you have been running since post #2 — a patch family, loaded beside the model like post #4's armature, not a new machine.
Installation is two folders: the node set into custom_nodes/, the adapter weights into models/ipadapter/ — about one to two gigabytes, one download, and the series route was verified against the spec before this post was written.
From the canvas's point of view, the station map barely changes: loaders, reference image, one apply node, the same sampler.
Examples
Steal the brush
05-steal-the-brush.json is the heron photograph from post #3 with a new passenger: a reference image chosen for its surface — brushwork, palette, light.
The blueprint stays honest and thin — "the same heron, in this rendering" — and lets the bust do the describing: {OBSERVE-STYLE}
Two busts
05-two-busts.json puts two references on the desk at once, each with its own weight: {OBSERVE-TWOBUSTS}
Two busts negotiate the same way two armatures did in post #4 — weights decide who talks louder, and complementary looks blend better than competing ones.
The bust and the harness
05-bust-and-harness.json stacks the two posts: post #4's canny armature for structure, this post's bust for the look — one holds the wire, one holds the glance.
Neither dial fights the other because they promise different things: {OBSERVE-STACK}
Cast the actor
05-cast-the-actor.json is the identity demo: one face, held across scenes — the same actor shot in three lightings, three costumes, three rooms, because the bust carries who, and the blueprints each carry where.
Identity lock is the bust's signature trick — style rides surfaces, identity rides geometry of the face: {OBSERVE-ACTOR}
Pitfalls
- A loud bust drowns the blueprint. Strength too high and every carve collapses toward the reference's composition — the "same image with a new caption" failure. Drop the glance, sharpen the sentence.
- Identity and style are different busts. A portrait reference injects its rendering and its face; a landscape reference injects its mood and its emptiness. Pick the reference for what you actually want copied.
- The reference's crop is part of the reference. A tight face-crop teaches close framing; a wide shot teaches layout you did not ask for.
- Garbage bust, garbage glance. The encoder reads the reference like the translator reads the blueprint — blurry, cluttered, or watermarked references get copied faithfully, flaws included.
Your Move
Pick an image whose look you would steal — a painter, a film still, a product shot — and run it as a bust on your own photograph at a middling denoise. Then cast an actor: one face reference, three blueprints, and watch who shows up in all three rooms.
Next post puts the reference desk away and changes the machine itself: a carver that reads images as instructions — edit by conversation.