Our AI Agents Ran $10k+ of Data Analysis for $253

Three AI tools ran 1,059 queries across 45 behavioural areas in our youth trends dataset, surfaced 475 unique findings, & produced a designed slide deck. Total cost : $253. Total time : 2 hours 40 minutes.¹ That's a 39x reduction in cost.

This isn't the first time this has happened. In 1956, loading a ship by hand cost $5.86 per ton. The shipping container dropped it to $0.16 - a 36x reduction. That single ratio made it cheaper to manufacture goods 8,000 miles away, restructured the global economy, and lifted 800m people out of poverty.

The compression ratio for AI-driven intellectual work is almost identical. I don't know what that means for the global economy, but I can tell you how we did it.

70 of those findings were headline-worthy. Not because previous analysis was wrong : because a human team would never run 1,059 exploratory cross-tabs. The economics don't allow it. AI changes that.

These are the first in a series of agentic research tools we're building, initially to interrogate our own data, & increasingly to help clients do the same with theirs. This post is about how they work.

Why most data goes unexplored

An analyst exploring a dataset picks the most promising angles first, tests a handful of hypotheses, & stops when time runs out.

1,059 queries at 10 minutes each is 176 hours. A month of analyst time. Over $10k.

So what normally happens : the team runs ~50 queries, writes up the strongest findings, & moves on. The other 1,009 questions never get asked. AI removes that constraint. It doesn't replace the analyst : it removes the ceiling on how many questions they can afford to ask.

The tools

We built three agentic tools, each with a specific job & a strict scope.²

AI Analyst

The AI Analyst takes a free-form research question & finds relevant data. It searches the survey database by meaning, not by schema : vector embeddings map "video consumption" to questions about TikTok, YouTube, & Netflix without knowing the question IDs.³

The search is two-stage. Vector similarity casts a wide net of possible matches. An LLM then ranks them by actual relevance to the research question & cuts to the best.

Once it has the right questions, it writes SQL. It knows the schema : question types (multi-select, single-choice, grid, frequency), how items & variants & answer codes work. It executes, interprets, & records findings.

Every finding passes through a multi-gate validation before it's recorded. An LLM parses the SQL to extract which questions, items, & answer codes it references. A programmatic check validates those references against the database. Then the SQL is re-run & an LLM checks whether the results actually support the claimed finding.⁴

One of the most common errors : querying a multi-variant question without specifying which variant, silently mixing data from different platforms. Looks plausible in the output. You'd only catch it reading the SQL.⁵

Deep Research

The Deep Research tool is the orchestrator. It maps the full question space first (~20% of budget), then dispatches the Analyst into each area (~80%).⁶

The separation is driven by context windows. If one agent is both mapping the territory & running SQL, it burns context on query results & validation errors. Splitting them means the orchestrator stays focused on "where to look next" & the Analyst gets a fresh context window for each area.

The harder problem was knowing when to move on. The Analyst exploring "TikTok usage by age" needs to recognise when to stop splitting by age & try a different behavioural cut, like device type or time of day. Two signals : the findings stop being interesting (two consecutive low-value results means switch dimension, three means stop the area entirely), or the sample sizes get too small to split further.⁷

Report Designer

A good report should surprise the reader & help them make smarter decisions. The Report Designer surfaces the findings most likely to do both, scoring 475 data points into four buckets : headline, interesting, context, skip. Batches of 15, scored in parallel.⁸

51.4% rated interesting or better.

Then a tension pass. The scorer identifies contradiction pairs : findings where direction & magnitude conflict, or where a dominant platform is absent from an expected behaviour. "High awareness but near-zero usage" is more interesting than either stat alone. These pairs become the strongest angles for the report.⁹

The biggest iteration was fighting obviousness. Early reports led with volume : the topic with the most findings dominated. We added two corrections. A volume bias rule : a topic with 30 findings is not more important than a topic with 3. And a "parent test" : if someone who works with children would say "well, obviously", it's not insight, it's confirmation.¹⁰

From the scored findings, the Report Designer builds a narrative, writes the report in MDX, generates charts from live SQL, & lays it out as a slide deck.¹¹ Every slide is rendered as an image & visually inspected. If the layout is wrong, text overflows, or a chart is unreadable, the agent adjusts & re-renders.

After writing, a separate model fact-checks every claim. It extracts every number from the report & verifies it maps to a real finding. Three issue types : unsupported (hallucination), inaccurate (wrong number), misleading (findings combined in a way that changes meaning). Two revision rounds max.¹²

The writing takes about 20 minutes. The design & visual validation take another 20.

Different models for different jobs

Opus reasons. Sonnet scores. GPT validates. OpenAI embeds.¹³

The expensive model does creative work where accumulated context matters : exploring data, writing narrative, designing layouts. Cheap models do stateless jobs where it doesn't : scoring findings into buckets, checking SQL references, verifying numbers.

We've written about this architectural pattern separately.¹⁴

The numbers

The scoring pipeline reliably separated signal from noise : headline findings averaged 24.5pp effect sizes, versus 17.1pp for interesting findings. The deeper the agents went, the better the results. Two-dimensional splits had a 51% hit rate. Three dimensions : 55%. Four or more : 69%, with a skip rate of just 5.6%.

The agents didn't just slice by existing dimensions. They created composite segments from combinations of behavioural flags : daily TikTok users who also create content, or parents who distrust a platform but whose children use it daily. A human analyst could build those cross-tabs, but wouldn't prioritise them without a hypothesis driving it. The agents had no hypothesis. They just ran everything.

Our agents can unlock insights in your data too

We built these tools for our own data : a longitudinal youth trends dataset covering media consumption, platform behaviour, & brand engagement across Gen Alpha & Gen Z. But the tools work against any structured dataset.

If you're sitting on research, survey data, or behavioural logs, there are likely findings in there that nobody's had the budget to look for. We can run these tools against your data, or if you work with young audiences, we can share what we're finding in ours.

Agent architecture

If you love agents as much as I do, you can nerd out on the full tool-call pipeline...