---
title: 'How OpenAI Built Its Data Agent'
source: 'https://youtube.com/watch?v=5V8tP5jDZ8U'
video_id: '5V8tP5jDZ8U'
date: 2026-08-06
duration_sec: 368
---

# How OpenAI Built Its Data Agent

> Source: [How OpenAI Built Its Data Agent](https://youtube.com/watch?v=5V8tP5jDZ8U)

## Summary

OpenAI's internal data agent, built on GPT-5.5, serves thousands of employees daily by answering data questions in Slack. The agent's simplicity—one model, 13 tools, and a runtime loop—contrasts with complex multi-agent systems, with the real engineering concentrated in a context assembly layer that pre-processes table descriptions, company knowledge, and memory. The video also highlights five engineering lessons from OpenAI's experience, including the importance of data foundation, fewer tools, and high-level guidance.

### Key Points

- **Simplicity over complexity** [00:03] — OpenAI's internal data agent is described as 'pretty simple' despite industry trends toward multi-agent pipelines and fine-tuned models. It uses one model (GPT-5.5) for every request, 13 tools, and a runtime loop.
- **Agent architecture** [00:33] — Every agent is an LLM plus a harness. The LLM reasons, while the harness supplies context and tools, running the model in a loop. OpenAI's agent skips routers and fine-tuning, focusing on a context assembly layer.
- **Context assembly challenge** [01:35] — The warehouse has over 70,000 tables, so the agent must find relevant ones. Schemas alone are insufficient because tables can look identical but differ in important ways (e.g., logged-out users).
- **Table descriptions from three sources** [02:17] — Table descriptions are built from usage metadata, human annotations, and Codex enrichment (a nightly job reading code behind each table). These are merged daily for search.
- **Additional context sources** [02:45] — Company knowledge from Slack, Docs, and Notion, plus corrections from past conversations, enrich the context. The context assembly step retrieves relevant descriptions, knowledge, and memory.
- **Simple runtime loop** [03:24] — After context assembly, the agent writes SQL, runs it, checks results, and refines until the answer holds up. The hard work happens before the question arrives.
- **Codex internal use cases** [03:40] — Codex is used for data migration (90,000 tables, 600 PB in 2 months), patch releasing for open-source forks (unattended for months), and support (100 fixes per day per engineer).
- **Lesson 1: Foundation matters** [04:33] — Every table comes from one mono repo with enforced conventions and annotations. A vanilla agent is reliable only if the data foundation is solid.
- **Lesson 2: Fewer tools beat more** [05:00] — Starting with 40 tools led to wrong picks and conflicting answers. Capping at 13 with no overlap fixed the issue.
- **Lesson 3: Not all queries are worth learning** [05:14] — Embedding every SQL query as examples failed because most are one-off experiments. Ranking by trust (dashboard queries top) improved performance.
- **Lesson 4: Guide the goal, not the steps** [05:30] — Detailed step-by-step prompts made answers worse; high-level goals worked better.
- **Lesson 5: Be more ambitious** [05:46] — The migration was estimated to take much longer than two months, but OpenAI pushed for two and hit it. Timeline estimates from before AI are outdated.

### Conclusion

OpenAI's data agent demonstrates that simplicity, backed by a strong data foundation and smart context assembly, can outperform complex multi-agent systems. The five engineering lessons emphasize the importance of data quality, tool minimalism, selective learning, high-level guidance, and ambitious timelines in the AI era.

## Transcript

Multi-agent pipelines, routers, fine-tuned models. But inside the top labs, the trend runs the other way. OpenAI's internal data agent serves thousands of employees every day, and the team describes it as pretty simple.
We met with Emma Tong, data platform lead at OpenAI, to learn how the agent is built, how the team uses Codex internally, and five important internally, and five important engineering lessons from OpenAI.
engineer types into Slack, "How many active users did we have over the last few weeks?" Seconds later, the agent replies with the answer, the SQL it ran, and the tables it used. Under the hood, every agent is an LLM plus a harness.
The LLM reasons. The harness supplies context and tools, and runs the model in a loop so it can act on what it reasons. This is where the systems usually get complex. A router for easy versus hard questions, multiple models, fine-tuning
on internal data, a separate retrieval pipeline per content type. Each piece can help, and each adds cost, latency, and new ways to fail. OpenAI's agent skips all of it. One model, which is GPT 5.5 on every
model, which is GPT 5.5 on every request, 13 tools, a runtime loop, and a context assembly layer, which is where the real engineering lives. So, what's in that context layer? Start with the problem it solves. A user asks a
question in plain English, and to answer it, the agent has to write SQL against the right tables. But OpenAI's warehouse has over 70,000 of them. So, the first challenge is just finding the ones that matter. This is where the context
assembler comes in. It gathers what the model needs to make that choice. To help the model understand the tables, the most obvious approach is to include the schema of each table in the model's context.
Two tables can both have a user ID column and look nearly identical, yet one includes logged out users and the other doesn't. Nothing in the schema tells the model which one to pick. Since schemas alone aren't enough to
describe a table, the team created table descriptions from three sources. The first is table usage metadata, which captures how people have queried the table in the past. The second is human annotations, the
notes written by the table's owners. And the third is Codex enrichment, a nightly Codex job that reads the code behind each table and records what it really contains and when to use it. Once a day, these three are merged into
for search. Table descriptions in the context give the LLM a lot to work with. But in addition to them, two more sources enrich the context. Company knowledge from Slack, Docs, and Notion, and
corrections the agent remembers from past conversations. When a question arrives, the context assembly step retrieves the relevant table descriptions and knowledge, then fetches relevant memory to build the
context. Once the context is assembled, the rest is simple. The agent writes SQL, runs it, checks the result, and refines until the answer holds up. That's the design. The agent stays
simple because the hard work happens before any question arrives in the before any question arrives in the context assembly layer. descriptions earlier, but that's not the only place OpenAI uses it internally.
The team relies on Codex for several tasks that keep the platform running. Here are the three main ones. Data migration. When the platform ran out of capacity on one cloud, Codex generated the pull
one cloud, Codex generated the pull request to migrate 90,000 tables and 600 petabytes, finished in about 2 months. Patch releaser. The platform maintains more than a dozen open-source forks. A Codex release agent
now tests and ships patches for all of them, running unattended for months without an incident. Support bot. And on support, the on-call engineer hands tickets to Codex, which
investigates and applies fixes for review. Around 100 fixes per day per review. Around 100 fixes per day per engineer. apply to every engineer. One.
The foundation matters more than the agent. Every table at OpenAI comes from one mono repo with enforced conventions and required annotations. That's what makes a vanilla agent reliable. If your data is scattered, the agent isn't the
first investment. The foundation is. Number two. Fewer tools beat more tools. The team started with around 40 tools, and the agent kept picking wrong ones or
getting conflicting answers from tools doing similar jobs. Capping at 13 with doing similar jobs. Capping at 13 with no overlap fixed it. Number three. Not every past query is worth learning from. The team first embedded every SQL query
ever written as examples for the agent. That failed. Most queries are one-off experiments, not good examples of how a table should be used. The fix was ranking them by trust. Queries behind heavily used dashboards at the top,
one-offs at the bottom. The agent then relies on the top queries. Number four. Guide the goal, not the Detailed step-by-step prompts made answers worse. High-level goals, good
better. Number five, and finally, be more The migration was estimated to take much longer than two months. OpenAI pushed
for two, and the team hit it. Timeline estimates from before AI no And the bigger risk now is playing it safe.