Not one AI, but a chain: how I set up AI for a big job

One question, hundreds of pages

Some assignments don’t fit in a chat window. A recent job of mine looked like this: hundreds of pages of documents, dozens of parts that each had to be assessed separately, connections between those parts that nobody could oversee anymore — and one final product in which everything is correct and verifiable.

The reflex is: take a powerful AI model, give it good instructions and let it read. That doesn’t work. Not because the model is too weak, but because one long conversation has no memory, no structure and no control. Halfway in, nobody knows what has already been assessed, why something is where it is, or what happens if one step has to be redone.

What does work is setting up AI the way you set up a team: small roles, clear handover, and a fixed way of recording.

Not one AI, but a chain of roles

Instead of one AI that “does everything”, there’s a chain of small AI roles, each with one task and its own instruction: one role extracts the parts from the raw documents, one sorts them through fixed gates, one assesses each part, one finds the connections, one assembles the final piece — and in between, a separate role checks the others’ work.

That may sound cumbersome, but it’s exactly why it works. A role with one task can be instructed sharply, improved separately and tested separately. If something goes wrong in step four, you fix step four — not the whole system.

Everything goes through files

The roles don’t talk to each other in a conversation; they hand their work over through structured files. Each step reads the files of the previous step and writes one of its own, in a fixed form: what was found, what score it got, how certain that judgment is.

Besides the full report per part, each assessment also writes a small signal file: a few fields that say where this part belongs — which parts influence each other (an error in one carries through to another), how heavy something weighs, which approach fits. A next role groups on those signals, and decides the approach per coherent group instead of per loose piece. That’s how you see cross-connections that reading piece-by-piece is guaranteed to miss.

The file principle delivers two more things. Traceability: from every conclusion in the final product you can page back to the file, the step and the source it came from. And restartability: every step can be run again without touching the rest, because everything the step needs lives in files — not in the memory of a conversation.

The same yardstick for every part

Every part is assessed along the same fixed angles, with a score and a confidence level per angle. It seems like a detail, but it’s the step that turns loose judgments into a whole: parts become comparable, priorities become visible (“this first, that can wait”), and the report per part states precisely what’s missing, what contradicts itself and what doesn’t add up.

Without a fixed yardstick you get what loose AI answers always give you: every judgment in a different form, with a different strictness, incomparable with the rest.

Knowledge lives apart from execution

The professional knowledge that feeds the assessments — the questions to be asked per part, the reference knowledge that judgments rest on — doesn’t live inside the roles’ instructions, but in separate knowledge files the roles consult.

That way you can update the knowledge without touching the roles, and improve the roles without touching the knowledge. It’s the same principle as a good team: the way of working and the reference library are two different things, and you want to be able to improve both.

The AI itself gets tested

The part that’s skipped most often: how do you know such a chain delivers good work? Answer: you test it the way you test software.

For every role there are test cases with a known outcome — documents with three errors deliberately hidden in them, for example. If the role finds all three, it scores 1.0; if it reports an error that isn’t there, that’s a zero. Change an instruction, run the tests, and you see whether it got better or worse. And on top of the tests, a person reviews the critical steps at fixed checkpoints — the chain prepares decisions, people make them.

See the modular AI platform

What this means for your job

The setup above isn’t tied to one kind of work. Wherever many documents, many parts and one verifiable end product come together — assessing, checking, assembling — the same pattern works: small roles, handover through structured files, one yardstick, knowledge kept separate, and tests that prove it’s right.

Have a job like that lying around, the one everyone says “surely AI can do something with this” about? Book a call via the button above — or browse the projects first to see what this pattern looks like in working systems.