4.5 LoRA Training: The Complete Workshop

Caption what you want to be able to change. Leave uncaptioned what is part of the person.

If you want to be able to change the outfit, name the outfit in the caption. If you want to change the background, name the background. If you want the face to be constant — and you do — do not describe the face at all.

A caption set for a persona therefore looks like a trigger token plus the variables:

ohwx_woman, red jacket, outdoor street, overcast light, smiling
ohwx_woman, black turtleneck, studio backdrop, softbox, neutral expression
ohwx_woman, denim shirt, kitchen interior, window light, laughing

And deliberately not:

ohwx_woman, oval face, brown eyes, sharp jaw, straight nose

The second style is what a naive auto-captioner produces. It is not wrong in the abstract — it describes the image accurately. It is wrong for this purpose, because it teaches the model that the words "brown eyes" are the explanation of the face. Then those words become a lever: change them, and the face changes, independently of the trigger. That is drift, reintroduced through the captions.

Trigger token. A rare string the model has no prior for, so it binds entirely to your concept. The convention is a nonsense token that does not appear in natural language. Use the same token every time you retrain this persona, because it is the interface between your prompts and the artifact.

Manual or automatic? Automatic captioners are fast and produce the verbose style above. If you use one, strip the facial descriptions afterward. For a dataset of 25 images, writing captions by hand takes twenty minutes and gives you full control over the one rule that matters. Do it by hand.

Time and money

Compare that to the alternative in 4.3, where the same two hours a week become an annual figure. A training run plus an afternoon is under ten hours. The economics of the training path are not close once your volume justifies it.

Choosing which checkpoint to keep

This is the section that saves the run, and the mistake it prevents is universal: keeping the last checkpoint.

Training loss is a weak signal for this task. It tells you the model is fitting the dataset, which is not the same as holding an identity. A run that has been going too long shows low loss and a rigid face — the persona can no longer smile differently, or turn, or be lit differently, without the likeness collapsing or the image developing a plastic quality.

The method:

1. Save every 250 to 500 steps. Configure this before the run. It costs disk and nothing else.
2. Sample with a fixed prompt set at the same cadence. Same three or four prompts every time, covering a frontal frame, a 3/4 frame, and one scene change.
3. After the run, score the checkpoints using the held-out images from 4.4 and the protocol in 4.6 — not on the training set.
4. Keep at least three checkpoints: the one that scored best, the one immediately before it, and the last one. Disk is cheap; a rerun is not.
5. Record the chosen step count in the iteration register alongside the dataset version and parameters.

The pattern to expect as steps increase: the likeness improves, then plateaus, then the flexibility degrades while the likeness holds or even improves. The best checkpoint is usually near the end of the plateau — not the first frame that looks like your persona, and not the last one the trainer produced.

The trap: a LoRA belongs to its base

An adapter is a set of weights that modifies specific layers of a specific model. Change the base and the modification either fails to load or produces something unintended.

What this means in practice:

• Record the base model's exact file and hash with the LoRA. "FLUX" is not an identifier; a specific checkpoint file is.
• The LoRA works best at the base's native training resolution and precision. A quantized base combined with a LoRA trained at full precision is a measurable mismatch. [UNVERIFIED]
• Within a family, sub-variants are not interchangeable. A distilled variant and a full-quality variant of the same family do not share the layers a LoRA touches.
• Update the base deliberately, and expect to retrain. Chapter 12.4 treats a base rotation as a project with a retraining step, not a configuration change.
• Keep the old base file. If you have to reproduce an asset from six months ago, the base plus the LoRA plus the settings is the recipe, and the base is the ingredient people forget to archive.

Where this goes wrong

Training on a non-commercial base and discovering it after launch. Check the license first.

Captioning the face. You have built a lever that changes the identity, and you will pull it by accident.

Leaving the learning rate high because the images "look sharper." They look over-processed.

Training the text encoder by default. Slower, more VRAM, worse prompt following, no benefit for a face.

Keeping only the final checkpoint. The last step of a run is rarely the best version of the identity, and it is often the most rigid one.

Scoring checkpoints on the training images. You will select the most memorized one.

Not archiving the base model. The recipe becomes unexecutable.

Expecting a LoRA to fix inconsistent lighting or wardrobe. It cannot; those are 5.4's problem.

The gate

1. Which base, and what is its license?
2. Where is your training configuration saved, and can you rerun it exactly?
3. Are captions free of facial descriptions and full of variable descriptions?
4. Is save-every-N configured, with at least three checkpoints retained?
5. Which step count won, and what did it score on the held-out set?
6. Is the base model file archived, with its hash recorded?

What you leave with

A base choice made on license and hardware rather than benchmark, a trainer defaulted to a reproducible config file, every parameter that matters with a starting value and the failure it causes, the captioning rule that prevents identity from leaking into your prompt vocabulary, a two-to-fourteen-dollar cost model, and a checkpoint selection method that measures identity on images the model has never seen.