---
title: 'AI Model vs. Agentic Harness: What Really Makes AI Work'
source: 'https://youtube.com/watch?v=ZELPNFXJ4_o'
video_id: 'ZELPNFXJ4_o'
date: 2026-09-02
duration_sec: 510
channel: 'IBM Technology'
---

# AI Model vs. Agentic Harness: What Really Makes AI Work

> Source: [AI Model vs. Agentic Harness: What Really Makes AI Work](https://youtube.com/watch?v=ZELPNFXJ4_o)

## Summary

This video explains why two AI products running the same underlying model can perform very differently. The key distinction is between the AI model itself (the 'brain in a jar') and the 'agentic harness' that wraps it, providing tools, memory, and an agentic loop. Most recent capability gains come from improvements to the harness, not just the model.

### Key Points

- **AI Model vs. Agentic Harness** [00:18] — The difference between AI products comes down to two components: the AI model (the actual neural network) and the agentic harness (everything wrapped around it). The model alone cannot interact with the world.
- **Model as 'Brain in a Jar'** [01:06] — AI models are capable but trapped—they cannot open files, run code, or browse the web natively. They are essentially a 'brain in a jar' without the harness.
- **Benchmark Gaps and Harness Impact** [01:19] — Benchmark gaps between top labs have narrowed to within a few points. When one AI product clearly outperforms another, the model is usually not the explanation—the harness is.
- **Three Components of the Harness** [02:13] — The agentic harness consists of three components: tools, memory, and the agentic loop. These enable the model to act in the world, persist information, and work toward goals.
- **Tools: Files, Code, Web, Computer Use** [02:28] — Tools give the model access to files (read/write), the ability to run code (often in a sandbox), retrieve internet information, and even drive a computer screen (computer use). Harnesses also provide command-line access and connect to external services via MCP (Model Context Protocol).
- **Memory: Context Window and Persistence** [03:36] — Models have a fixed context window (working memory) that is wiped after a conversation. The harness persists information via instruction files (e.g., AGENTS.md), compacts context when full, and allows search (grep, semantic) to pull in only needed code pieces.
- **Agentic Loop: Plan-Act-Observe-Repeat** [05:01] — The agentic loop is where the harness and model work together: the model plans a step, the harness performs an action, the model observes results, and the loop repeats—sometimes for seconds, sometimes for hours. Modern harnesses run continuous verification (tests, screenshots, separate reviewer models).
- **Why the Distinction Matters** [06:31] — Most recent capability gains come from the harness getting better (tools, memory, loops, verification), not just the model. The question 'Is AI good at X?' should be reframed as 'Which model and which harness?'
- **The Line is Fluid** [07:33] — Capabilities once in the harness (long-horizon planning, self-verification) are being trained into models, while model behaviors (consistency) are increasingly shaped by harness conventions. Both the brain and the jar are improving.

### Conclusion

To understand AI performance, you must separate the model from the harness. Both are evolving, and the harness—with its tools, memory, and agentic loop—is often the real driver of capability differences.

## Transcript

You might have noticed that two AI products can feel completely different even when they're running the same underlying model, like how the same model could be a basic back-and-forth chatbot, but if you put it inside a developer tool with access to files and a terminal, it can suddenly work for hours on a software task.
So what accounts for the difference? Well, it comes down to AI model and agentic harness. and let's start with AI model because what that is is pretty simple we all
know what these AI models are things like chatGPT or maybe CORD so when we talk about a model we really mean the actual artificial neural network and what those models can't really do on their own is reach out into the world
they can't open a file or run a piece of code natively they can't really browse the web either. So on their own these AI models are essentially, well, a brain in a jar. They're
capable but they're trapped. They're trapped in that jar. Now benchmark gaps between the top labs have narrowed to within a few points each other on most evals, so when one AI product clearly
outperforms another, the brain itself, by which I mean the AI model, that usually isn't the explanation it's what's wrapped around the brain and that wrapping has a name
when we take a model and we wrap it we are calling this thing that wraps it the agentic harness and then if we think of an AI agent overall that encompasses
both of these things so essentially an AI agent consists of the AI model and and the agentic harness That the tools the model uses the way it remembers things the loop that it keeps running the rules about what it allowed to do All of that stuff lives inside the agentic harness So let get inside that harness and what inside is essentially three components
We've got tools, we've got memory, and we've got the agentic loop. So take a look at these one at a time. We'll start with tools. So through tools the model can do things out in the world. For example, one of the things it can do is it has access
us to files so it can read and write to them. Another thing we can do with tools is run code and that code might run in a sandbox as well. We can go and get
information on the Internet and we can also often actually drive a computer screen itself. This is called computer use and here the model can move a cursor
around and operate any software similar to the way that a person would do it. Now many of these capabilities are actually built into the harness directly. For anything else already on the machine
well a harness usually gives the model access to the command line so it can just run whatever software is already installed which is really the same way that a developer would do it and
It's connected to external services that don't live on the machine, so maybe a corporate database or a third-party app. There's a standard called MCP, Model Context Protocol. And a tool built for MCP can plug into any harness that supports it without being rebuilt each time.
So it's tools. What about memory? Well, models have a fixed context window, which is basically their working memory. and once the conversation ends, that working memory is wiped clean.
But the harness can help persist things. For example, it can persist instruction files, so things like an agents.md file,
which sits in a project folder and gets loaded into the model's context at the start of every session. They how the model knows the code base conventions or what libraries to use Now within a session when the context window starts filling up the harness runs another process where it
compacts that context window so it summarizes what happened earlier so the important parts stay around but stuff like redundant tool outputs all that
stuff kind of gets pruned out and then rather than dumping an entire codebase into the model's context all at once the harness lets the model perform a search
so we can search the project sometimes with grep like text search sometimes with semantic search or code indexes and then just kind of pull in only the pieces it needs that's memory and then there is the agentic loop now this is
where the harness and the model work together to achieve a goal so the model decides on a next step to achieve that goal, that's the plan stage, and then the
harness takes that plan and actually performs an action on it. When that action is performed we get some results, the model observes those results, and
then the loop repeats around and around, sometimes just for a few seconds, sometimes for hours. And in addition to this cycle, modern harnesses run verification continuously all the way through the loop. The model runs tests as
it goes, maybe takes screenshots of what it's built, and sometimes even spins up a separate model to act as a reviewer. So verification is pretty cool, running these tests and checking the outputs and taking a peek at screenshots. And a model
that double checks itself might run a lot longer without going off the rails. So that's tools, memory, and the agentic loop. Once a harness has all three of these running together, the kinds of things AI can actually do start looking
very different from anything an AI model can do on its own. So I think we should also ask why should we care about this distinction AI model and agentic harness Well because once we can see the model and the harness as separate things I think a lot of what been going on
in AI starts to make a bit more sense, because most of the capability gains we've seen lately, they've come actually from the harness getting better, by which I mean better tools and memory handling and smarter agentic loops for verification. And of course, with every new version, the
models have improved as well. So when somebody asks, is AI good at something? Is AI good at writing code? Or is AI good as a customer support chatbot? I think the more useful version
of that question is to better define what we mean by AI. And specifically, we mean two things. We mean which AI model are we going to use in this scenario, but also if this is an agent,
we need to ask which harness are we going to use with that AI model as well, because the model can be brilliant in one harness and get stuck in another. And the line between these two
things is also a bit fluid, because capabilities that used to live entirely in the harness, like long horizon planning or self-verification, they're starting to be trained directly into the models.
And things that used to be model behavior, like consistency over a long task, are increasingly shaped by harness conventions and project files. Which brings us back to that brain in a jar.
The brain itself, the AI model, continues to get smarter. There's no doubt about that. But you know what? it's not just the model getting smarter, it's not just the brain,
it's also the jar, the agentic harness.