[00:03] Multi-agent pipelines, routers, fine-tuned models. But inside the top labs, the trend runs the other way. OpenAI's internal data agent serves thousands of employees every day, and the team describes it as pretty simple.
[00:18] We met with Emma Tong, data platform lead at OpenAI, to learn how the agent is built, how the team uses Codex internally, and five important internally, and five important engineering lessons from OpenAI.
[00:33] engineer types into Slack, "How many active users did we have over the last few weeks?" Seconds later, the agent replies with the answer, the SQL it ran, and the tables it used. Under the hood, every agent is an LLM plus a harness.
[00:49] The LLM reasons. The harness supplies context and tools, and runs the model in a loop so it can act on what it reasons. This is where the systems usually get complex. A router for easy versus hard questions, multiple models, fine-tuning
[01:04] on internal data, a separate retrieval pipeline per content type. Each piece can help, and each adds cost, latency, and new ways to fail. OpenAI's agent skips all of it. One model, which is GPT 5.5 on every
[01:21] model, which is GPT 5.5 on every request, 13 tools, a runtime loop, and a context assembly layer, which is where the real engineering lives. So, what's in that context layer? Start with the problem it solves. A user asks a
[01:35] question in plain English, and to answer it, the agent has to write SQL against the right tables. But OpenAI's warehouse has over 70,000 of them. So, the first challenge is just finding the ones that matter. This is where the context
[01:49] assembler comes in. It gathers what the model needs to make that choice. To help the model understand the tables, the most obvious approach is to include the schema of each table in the model's context.
[02:02] Two tables can both have a user ID column and look nearly identical, yet one includes logged out users and the other doesn't. Nothing in the schema tells the model which one to pick. Since schemas alone aren't enough to
[02:17] describe a table, the team created table descriptions from three sources. The first is table usage metadata, which captures how people have queried the table in the past. The second is human annotations, the
[02:30] notes written by the table's owners. And the third is Codex enrichment, a nightly Codex job that reads the code behind each table and records what it really contains and when to use it. Once a day, these three are merged into
[02:45] for search. Table descriptions in the context give the LLM a lot to work with. But in addition to them, two more sources enrich the context. Company knowledge from Slack, Docs, and Notion, and
[02:59] corrections the agent remembers from past conversations. When a question arrives, the context assembly step retrieves the relevant table descriptions and knowledge, then fetches relevant memory to build the
[03:11] context. Once the context is assembled, the rest is simple. The agent writes SQL, runs it, checks the result, and refines until the answer holds up. That's the design. The agent stays
[03:24] simple because the hard work happens before any question arrives in the before any question arrives in the context assembly layer. descriptions earlier, but that's not the only place OpenAI uses it internally.
[03:40] The team relies on Codex for several tasks that keep the platform running. Here are the three main ones. Data migration. When the platform ran out of capacity on one cloud, Codex generated the pull
[03:52] one cloud, Codex generated the pull request to migrate 90,000 tables and 600 petabytes, finished in about 2 months. Patch releaser. The platform maintains more than a dozen open-source forks. A Codex release agent
[04:06] now tests and ships patches for all of them, running unattended for months without an incident. Support bot. And on support, the on-call engineer hands tickets to Codex, which
[04:18] investigates and applies fixes for review. Around 100 fixes per day per review. Around 100 fixes per day per engineer. apply to every engineer. One.
[04:33] The foundation matters more than the agent. Every table at OpenAI comes from one mono repo with enforced conventions and required annotations. That's what makes a vanilla agent reliable. If your data is scattered, the agent isn't the
[04:48] first investment. The foundation is. Number two. Fewer tools beat more tools. The team started with around 40 tools, and the agent kept picking wrong ones or
[05:00] getting conflicting answers from tools doing similar jobs. Capping at 13 with doing similar jobs. Capping at 13 with no overlap fixed it. Number three. Not every past query is worth learning from. The team first embedded every SQL query
[05:14] ever written as examples for the agent. That failed. Most queries are one-off experiments, not good examples of how a table should be used. The fix was ranking them by trust. Queries behind heavily used dashboards at the top,
[05:30] one-offs at the bottom. The agent then relies on the top queries. Number four. Guide the goal, not the Detailed step-by-step prompts made answers worse. High-level goals, good
[05:46] better. Number five, and finally, be more The migration was estimated to take much longer than two months. OpenAI pushed
[05:58] for two, and the team hit it. Timeline estimates from before AI no And the bigger risk now is playing it safe.