The Right Model for the Job: Building a Multi-Provider AI Harness

Software development involves several different kinds of thinking. Understanding an unfamiliar codebase, weighing architectural trade-offs, implementing a change and finding fault with a proposed solution are different jobs. Why should I expect one AI model to be the best choice for all of them?

That question sits behind the custom AI harness I built for our work at CRDX, including the development of Longstead.

I wanted to use different AI providers according to their competencies. I also wanted work to move between them without losing its purpose, its constraints or the evidence needed to decide whether it was done properly.

By harness, I mean the working environment around the AI agent: how it receives a task, gets the relevant context, carries out the work and returns a result that can be checked. This article covers the thinking behind it and the general workflow. The implementation remains private.

The problem around the model

A substantial software repository contains much more than code. It holds architectural decisions, business rules, unfinished work, experiments and assumptions that have already been challenged.

An agent needs enough of that context to make a useful change. A request to “implement this feature” only tells part of the story.

As a session progresses, the missing detail accumulates in the conversation. We clarify a requirement. An approach fails. A test exposes something unexpected. A decision changes what should happen next.

Then the session ends, or I want another model to take over.

How much of that understanding survives?

Without a proper handoff, the next agent has to reconstruct the situation. It might repeat an investigation, overlook a constraint or treat an earlier proposal as an accepted decision. A capable model can still do the wrong work when it starts with an incomplete picture.

Using several providers makes this more apparent. Their tools handle instructions, memory, permissions and session history differently. Changing provider should not mean explaining the project from the beginning.

The first requirement was therefore straightforward: the project’s working knowledge had to remain usable across sessions and providers.

Give every agent the same starting point

The harness started with shared project context. Whichever agent picked up a task needed to establish the same essentials:

• What outcome had been requested?
• Which decisions and constraints applied?
• Which sources should it consult?
• What had already been done?
• What evidence would establish completion?
• What was the next concrete step?

There is an important distinction here between information that is relevant and information that is authoritative.

A repository search might return an accepted architectural decision, an old plan and an exploratory note. All three can mention the same feature. They do not carry the same weight.

The workflow needs to preserve that distinction. Project guidance establishes the rules. The task establishes the requested outcome. Supporting material helps the agent investigate. Finding a document does not turn it into permission to build what it describes.

It also means important decisions need a durable home. Another session should be able to inspect them without access to the original conversation. Private model memory can help, but the project needs to remain understandable without it.

Choose the model for the assignment

Once that foundation exists, using multiple providers becomes much more practical.

The question I want to ask is: which model and tool combination is suited to this assignment?

Broad synthesis, a narrow implementation task and an adversarial review place different demands on an agent. The consequences of getting them wrong differ too.

Competency includes the tools around the model. How well does the agent navigate the repository? Can it make a controlled change? Does it follow the project’s constraints? Does it clearly report a blocker, or leave me to discover that part of the task never happened?

Those behaviours matter alongside the quality of its reasoning.

The harness lets me choose different providers and models while keeping the task context and completion standard consistent. At this stage, task assignment is operator-directed. I choose the agent for the job and review the result.

I do not treat that choice as a permanent ranking of vendors. Models change, tooling improves and a good result on one kind of work does not establish competence on every other kind. The useful evidence comes from representative tasks under the conditions in which the agent will actually work.

Provider flexibility gives me room to revisit those choices without rebuilding the project’s working practices each time.

How the workflow works

At a high level, there are five parts.

The console captures below show a real, read-only demonstration using Claude Code, Codex and OpenCode. That is three tools across two providers: Anthropic and OpenAI. For this demonstration, each receives the same inspection brief so the handoff and execution states are easy to follow. It is a workflow demonstration, not a model benchmark.

The images use a publication-only console view built from recorded runner states. Private prompts, source code, paths, identifiers and report contents are omitted. Open an image to see it at full size.

1. Define the assignment

Start with a clear outcome, a bounded scope and observable acceptance criteria. The agent and the reviewer need something concrete to assess.

This also gives an investigation a stopping point. Finding another possible improvement does not automatically make it part of the current task.

2. Prepare the context

Bring together the project guidance, current task state and relevant source material.

The aim is to provide useful context. Essential requirements need to stay visible, and supporting information needs to help answer the question at hand. A larger pile of documents is not necessarily a better brief.

3. Run the selected agent

The chosen agent works through its supported tooling, with the access and scope established for the assignment.

The surrounding workflow provides consistency while accommodating differences between providers.

Independent runs can overlap. Here, the coordinator explicitly launches all three, then observes their individual states. The threads shown are separate agent runs, each with its own lifecycle. Task selection remains operator-directed; there is no automatic scheduler choosing the next job.

4. Collect and verify the result

Inspect the actual changes, check outcomes and unresolved issues alongside the agent’s explanation.

Verification needs to apply to the version under consideration. If the work changes after a successful check, that earlier result cannot automatically establish that the new version is correct.

5. Review and decide what happens next

Accept the result, return it for a further bounded task or hand it to another agent with a different role.

The handoff needs to preserve what happened and what remains unresolved. The next session should be able to continue from the recorded state.

Use different perspectives to challenge the work

One useful application is to separate producing a solution from challenging it.

An agent can develop a proposal or implementation. Another can review that candidate against the requirements, looking for missing cases, weak assumptions or unnecessary complexity.

This gives the second agent a specific job. Asking it to find concrete problems is more useful than asking whether it agrees with the first agent.

Agreement between models is not proof. They can share a mistaken assumption. A reviewer can also confidently describe a defect that does not exist.

The way to resolve that is to return to the evidence. What requirement is being violated? Where does the implementation demonstrate the problem? Which check would establish whether the finding is valid?

Cross-model review is useful when it produces questions that can be investigated. The number of agents involved tells me very little on its own.

Treat interruption as part of the design

A working harness also has to deal with tasks that stop halfway through.

Processes fail. Sessions run out of context. An agent makes some changes and then encounters an error. A check starts but never finishes.

What can I establish afterwards?

I need to distinguish completed work from partial work, verified results from untested claims, and known failures from uncertain outcomes. The next step depends on which of those states I am actually in.

This is a familiar problem in cloud engineering. Long-running operations need explicit state, observable results and considered recovery behaviour. The same discipline is useful when an AI agent is doing the work.

An interrupted task should leave an inspectable record and a sensible next step. Otherwise, continuity depends on someone remembering what happened.

Keep engineering judgement explicit

The current harness supports shared context, bounded execution and evidence-based review. Task selection, integration and acceptance remain operator-directed.

Someone still needs to decide whether the requested behaviour is appropriate, whether the trade-off is acceptable and whether the evidence is sufficient. Passing checks is useful evidence, but it does not answer every product or engineering question.

For me, the value is being able to make those decisions with a clearer record of the work. I can see the assignment, inspect the result and decide what should happen next.

That matters when the output contributes to a real product. A plausible implementation still needs to meet the actual requirement.

The broader idea

The question behind this work is how to turn varying model capabilities into a dependable engineering process.

Model quality matters. So do task definition, context quality, continuity, tool behaviour and independent verification. The harness gives those concerns a consistent place in the workflow.

That is the design philosophy: keep the project’s knowledge and engineering standards consistent, while allowing the model doing the work to change.

For CRDX, a multi-provider approach gives us the freedom to choose a suitable model for each assignment, preserve what the project has learned and assess the result against evidence.

The model can change. The standard of engineering should hold.