Listen to the full conversation on DataScienceWithSam, available on Apple Podcasts and Spotify.
Most data teams have lived some version of this same story: an AI agent runs a query, the query returns a result, the number looks reasonable, and the dashboard ships. Weeks later, someone traces a bad decision back to that query and finds a join that never should have executed the way it did. Nobody caught it in the moment because nothing about the failure looked like a failure.
On this episode of DataScienceWithSam, Pradnesh Patil, Co-Founder and CEO of Altimate AI argues that this is not a symptom of weak models. It is the predictable result of asking an agent to act without the infrastructure required to know what is true about the data. The gap is not in what the model can do, it is in what the model has been given to work with.
The core distinction: prompts vs. harnesses
A system prompt tells a model what to do. A harness tells it what is actually true. That distinction, Patil says, is the one most teams get wrong when their first AI data agent rollout stalls out or quietly produces bad numbers. Teams write increasingly elaborate instructions into the prompt, hoping better wording will compensate for the model having no grounded view of the schema, the lineage, or the cost of the action it is about to take. It rarely does.
Patil breaks the agentic data engineering harness into five components: context, governance, MCP tools, shared skills, and agent infrastructure. Each one answers a different question an agent needs answered before it acts responsibly. Context tells the agent what the data actually looks like today. Governance tells it what it is allowed to do. Tools give it the concrete means to act. Shared skills capture the institutional knowledge a team has already built up. Agent infrastructure ties all of it together into something that runs reliably at scale.
Miss any single one of those five components, Patil says, and the resulting failures are silent. The query runs, it returns a result, maybe the result looks correct to any human checking... but it is not, and nothing in the interface tells anyone that until the damage has already spread downstream into a report, a model, or a decision.
Between 27% and 33% of AI-generated queries reference tables that do not exist in the schema, phantom references that a grounded agent would never produce in the first place. A further 78% contain what Patil calls "silent wrong joins": joins that execute cleanly, return a syntactically valid result, and still return the wrong data. Neither statistic describes a model that is bad at SQL. Both describe what happens in the absence of infrastructure that keeps a model grounded in the real, current state of the data it is querying.
Harness beats model size
The instinct when an agent underperforms is to reach for a bigger model. Patil pushes back on that instinct directly, and points to Altimate Code's performance on ADE-Bench as the clearest evidence against it. Altimate Code topped the leaderboard running on Sonnet, while competing systems relied on Opus, a materially larger and more expensive model, to reach lower scores.
The takeaway is not that a smaller model can outperform a larger one under identical conditions, but that the system surrounding the model, the harness that grounds it in context and governs what it is allowed to do, determines outcomes more than raw model capability does. For data teams evaluating where to invest, that reframes the question. The more valuable investment is often not the next model upgrade. It is the infrastructure that makes any model reliable once it is put to work on real data.
Where LLMs should, and shouldn't, make decisions
A recurring theme here is the boundary between deterministic and probabilistic reasoning. Validation, cost checks, and query correctness are deterministic problems, Patil argues: they have a right answer that does not depend on interpretation, and they should never be handed to a model to reason about probabilistically. A query either references a table that exists or it does not. A join either matches the keys correctly or it does not. A cost estimate either falls inside budget or it does not. None of that needs a language model's judgment.
Hard rules should handle hard rules, and the model's judgment should be reserved for genuine ambiguity, the calls that actually require weighing context and making a reasoned choice. Building a harness that respects this boundary, rather than routing everything through the model, is what keeps an agent from confidently producing a wrong answer with the same tone of voice it uses for a right one.
Fixing context compaction for long-running tasks
Standard LLM context compaction was designed for general-purpose chat, where discarding older turns of a conversation carries little cost. Applied to a long-running data engineering task, the same compaction approach tends to discard exactly the schema and lineage context an agent needs to keep working correctly. An agent that loses that context mid-task does not usually stop. It keeps going, with a thinner and thinner grip on what is actually true, until its output drifts from correct to merely plausible.
Altimate's response is a compaction approach built specifically for this kind of task, one that treats schema and lineage as context to be preserved rather than as expendable conversational history. The distinction matters most on the tasks that take the longest: multi-step pipeline builds, large-scale migrations, and anything else where an agent is expected to stay grounded over many turns rather than a handful.
The $5,000 query bill
Governance is not an abstract concern here. Patil cites a real $5,000 Cortex AI query bill as the kind of outcome that permission-based governance helps avoid. The bill was not the result of malicious intent or a rare edge case. It was the predictable outcome of an agent operating without limits on what it was allowed to query and at what cost.
Without controls on scope and spend, Patil argues, agents can go rogue on cloud cost just as easily as they can go rogue on correctness, and the two failure modes compound each other. An ungoverned agent that is also ungrounded in the real schema is not just likely to produce a wrong answer. It is likely to produce a wrong and expensive answer, executed against the wrong tables, at a cost nobody approved in advance.
Where data engineering is headed
Patil's broader claim is that the era of hand-writing SQL is ending. The role is shifting toward building and governing the systems that let agents do that work reliably: defining the context that grounds them, the governance that constrains them, and the shared skills that let institutional knowledge outlive any one person's tenure on a team.
His advice reflects that same emphasis on infrastructure over individual features: Build open source, build cross-platform, so that the tools work across Snowflake, Databricks, and every other environment your data estate actually spans. Avoid siloed AI features that solve a narrow problem for one vendor's users without moving the broader field forward. For Patil, the teams that internalize this shift early, and start investing in the harness rather than the next model swap, are the ones that will find AI agents genuinely trustworthy in production, rather than merely fast and occasionally, silently, wrong.
Listen to the full conversation on DataScienceWithSam, available on Apple Podcasts and Spotify.