# Altimate AI — full site content > Generated from 138 indexable pages. Canonical index: https://altimate.ai/llms.txt --- # Altimate.ai | Best AI Data Engineering Agent URL: https://altimate.ai/ AI data engineering agent you can trust in production: bring context, governance, and execution together; build and manage data workflows safely. Free to start. ## Data agents you can trust in production. ◈ Tech Stack Agnostic Token-Efficient ⬡ Open Source Build data pipelines // Migrate workloads // Build data apps // Optimize workloads // Build data pipelines // Migrate workloads // Build data apps // Optimize workloads // TRUSTED BY HUNDREDS OF ENTERPRISES StubHub Coinbase Booking.com Peloton Workday Capgemini Slack Siemens Healthineers Oodrive Explorium Outreach Elastic Chatbooks Ooni World of Books Clicksign Qover Carta DAZN Ritchie Bros. WHY ALTIMATE ## Benchmarked Across the Whole Data Stack ◈ Tech Stack Agnostic Agents that work in your best interests, not your vendor's. Token-Efficient Designed to minimize token use, and maximize impact. ⬡ Open Source Retain control over your stack and avoid vendor lock-in. ADE-Bench results As of June 2026 altimate-code 78% DeepSeek V4 Pro Cortex Code CLI 65% Opus 4.6 dbt Labs 59% Sonnet 4.5 Claude Code 40% Sonnet 4.6 · baseline View full results → https://altimate.ai/benchmarks/ade-bench DAB Bench results As of June 2026 altimate-code 71.7% GPT-5.5 + Sonnet 4.6 Spacedock (Recce) 67.2% Opus 4.8 MinusX 65.2% Sonnet 4.6 + GPT-5.5-mini + Haiku 4.5 Pi Coding Agent 61% Opus 4.6 View full results → https://altimate.ai/benchmarks/dab-bench USE CASES ## With Altimate You Can Automatically: ### Build data pipelines Build, test, and deploy production-quality dbt pipelines with agent-generated models, tests, and docs. https://altimate.ai/use-cases#build-pipelines ### Migrate workloads Migrate Informatica, SSIS, or legacy SQL to dbt. Business logic extracted and contracts preserved. https://altimate.ai/use-cases#workload-migrations ### Build data apps Generate full-stack data applications: APIs, dashboards, and pipelines from a plain-language spec. https://altimate.ai/use-cases#data-apps ### Optimize workloads Auto-tune warehouses and optimize queries across Snowflake, BigQuery, and Databricks, autonomously. https://altimate.ai/use-cases#cost-optimization YOUR JOURNEY ## Level Up Your Agentic Data Engineering 01 LEVEL 01 ### Experience it first Open-source CLI. No sign-up. Any model, warehouse, or framework. Start shipping agentic data pipelines in minutes. $ npm install -g altimate-code 02 LEVEL 02 ### Power up your team Get 10M free tokens and enable team-level memory and context graph. Shared agents learn from every engineer on your team. [Sign Up Free →](https://app.myaltimate.com/register) 03 LEVEL 03 ### Go fully autonomous Fully autonomous experience for enterprise data teams. Governance, audit trails, and SOC 2 compliance built in. [Book a Demo →](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) CUSTOMERS ## What Data Teams Actually Say ► INTUITIVE █ — Krishna Nimmagadda INTUITIVE ▼ ◄ PREV NEXT ► ▸ ALSO TRUSTED BY StubHub Coinbase Booking.com Peloton Workday Capgemini Slack Siemens Healthineers Oodrive Explorium Outreach Elastic Chatbooks Ooni World of Books Clicksign Qover Carta DAZN Ritchie Bros. GET STARTED ## 10 Million Tokens. No Credit Card. No Reason Not To. Start in 60 seconds. $ npm install -g altimate-code --- # Altimate Code | Better than Claude Code for Data Engineering URL: https://altimate.ai/products/altimate-code Use Altimate Code for better results than Claude Code and Cursor on data engineering. Build and manage SQL, dbt, lineage, and warehouse workflows, open source. ## Open-source agentic data engineering harness. 100+ deterministic tools for SQL, lineage, dbt, FinOps, and warehouse connectivity. $ npm install -g altimate-code [View the project on GitHub](https://github.com/AltimateAI/altimate-code) ## Benchmarked Across the Whole Data Stack ADE-Bench results As of June 2026 altimate-code 78% DeepSeek V4 Pro Cortex Code CLI 65% Opus 4.6 dbt Labs 59% Sonnet 4.5 Claude Code 40% Sonnet 4.6 · baseline View full results → https://altimate.ai/benchmarks/ade-bench DAB Bench results As of June 2026 altimate-code 71.7% GPT-5.5 + Sonnet 4.6 Spacedock (Recce) 67.2% Opus 4.8 MinusX 65.2% Sonnet 4.6 + GPT-5.5-mini + Haiku 4.5 Pi Coding Agent 61% Opus 4.6 View full results → https://altimate.ai/benchmarks/dab-bench Works with your data stack Snowflake Snowflake BigQuery BigQuery Databricks Databricks PostgreSQL PostgreSQL dbt dbt Airflow Airflow GitHub GitHub Redshift Redshift MySQL MySQL Jira Jira Linear Linear DuckDB DuckDB Dagster Dagster Google Sheets Google Sheets Snowflake Snowflake BigQuery BigQuery Databricks Databricks PostgreSQL PostgreSQL dbt dbt Airflow Airflow GitHub GitHub Redshift Redshift MySQL MySQL Jira Jira Linear Linear DuckDB DuckDB Dagster Dagster Google Sheets Google Sheets Bring your own LLM Anthropic Anthropic OpenAI OpenAI Google Google AWS Bedrock AWS Bedrock Azure Azure Ollama Ollama OpenRouter OpenRouter Mistral Mistral Groq Groq xAI xAI Copilot Copilot Anthropic Anthropic OpenAI OpenAI Google Google AWS Bedrock AWS Bedrock Azure Azure Ollama Ollama OpenRouter OpenRouter Mistral Mistral Groq Groq xAI xAI Copilot Copilot Why altimate-code ## Six Things a Generic Coding Agent Does Not Give You. Every capability is open source, MIT-licensed, and reproducible from a published benchmark. ### Bring your own LLM Anthropic, OpenAI, Google, Bedrock, Azure, Ollama, Snowflake Cortex, Databricks AI Gateway, and more. No vendor lock-in. 12+ providers · BYO key ### Cross-platform Claude Code Data Engineering, Cursor, Windsurf, VS Code, and any MCP-compatible client. One install, everywhere. MCP-native · one install ### 100+ deterministic tools SQL analysis, column-level lineage, dbt integration, FinOps, warehouse connectivity. No hallucinations. 100+ tools · 34 SQL dialects https://altimate.ai/skills ### Validation layer SQL, lineage, and equivalence checks run in compiled Rust, not the model. Same input, same output, every run. 100% F1 · ~2 ms · 0 tokens ### Token efficiency Context compaction trims the schema payload per task. The gateway routes each call to the cheapest model that clears your accuracy bar. per-task routing · context compaction ### Data governance Policy gates, PII detection, and compliance checks run before the agent writes a column. Audit-ready. policy · PII · audit trail ## See It in Action Real-world pipelines built end-to-end with a single prompt US Home Sales Data Science Dashboard $ Download all available public US home sales data sets. Process and merge them into a unified format. Perform advanced data science — K-means, OLS regressions, and more. Build a single interactive dashboard. US Home Sales Data Science Dashboard Snowflake vs Databricks Benchmark $ Deploy the NovaMart platform to both Snowflake and Databricks, testing multiple warehouse sizes on each. Run the full data pipeline and benchmark queries, capturing execution time, credits/DBUs consumed, and bytes scanned. Snowflake vs Databricks Benchmark NYC Taxi Coverage Dashboard DuckDB dbt Airflow $ Take the New York City taxi cab public dataset, bring up a DuckDB instance, and build a dashboard showing areas of maximum coverage and lowest coverage. Set up a complete dbt project with staging, intermediate, and mart layers, and create an Airflow DAG to orchestrate the pipeline. NYC Taxi Coverage Dashboard Olist E-Commerce Analytics Pipeline $ Build an end-to-end e-commerce analytics pipeline using the Olist Brazilian E-Commerce dataset. Use Azure Data Factory to ingest CSV files from Blob Storage into Snowflake raw tables, then orchestrate Snowflake stored procedures to transform data through raw → staging → mart layers. Olist E-Commerce Analytics Pipeline Global CO2 & Climate Explorer $ Build me an interactive Global CO2 & Climate Explorer dashboard using DuckDB-WASM running entirely in the browser, sourcing data from Our World in Data’s CO2 dataset. Global CO2 & Climate Explorer NYC Taxi Coverage Dashboard Curious to know more? Check out the [examples](https://docs.altimate.sh/examples/) ## Join the Community altimate-code is built in the open. Get involved. ### Star & contribute on GitHub View on GitHub https://github.com/AltimateAI/altimate-code ### Join the community on Slack Join Slack https://bit.ly/agentic-data-engineering-slack ### Report an issue with /feedback Report Issue https://github.com/AltimateAI/altimate-code/issues ## Open Source. MIT Licensed. One Install. [Get Started](https://docs.altimate.sh/getting-started) [Read the Docs](https://docs.altimate.sh) [GitHub](https://github.com/AltimateAI/altimate-code) ## Frequently Asked Questions What is Altimate Code? Altimate Code is an open-source agentic harness for data engineering. It gives AI agents like Claude and GPT-4 100+ deterministic tools for SQL analysis, column-level lineage, dbt, FinOps, and warehouse connectivity across Snowflake, BigQuery, Databricks, Redshift, and Postgres. It scored 78.0% on ADE-Bench using DeepSeek V4 Pro. MIT licensed, install with `npm install -g altimate-code`. How does Altimate Code compare to Claude Code for data engineering? Altimate Code outperforms Claude Code on data engineering benchmarks — 78.0% vs 40% on ADE-Bench. The difference is the harness: Altimate Code provides 100+ purpose-built, deterministic tools for SQL, lineage, dbt, and warehouses. Claude Code uses general-purpose coding tools. Altimate Code also works with any LLM, not just Anthropic models — achieving 78.0% with DeepSeek V4 Pro on DuckDB. How does Altimate Code work alongside Claude Code for data engineering? Altimate Code and Claude Code cover different halves of the same job, so many teams run both. Claude Code handles orchestration: it reads the repo, plans the change, and edits the files. Altimate Code supplies the data engineering layer underneath it, including warehouse connectivity, column-level lineage, PII classification, schema diffs, impact analysis, and a real dbt build against your warehouse. When it comes to data engineering, Claude Code ships no deterministic lineage tracer, PII classifier, or schema diff engine. Connecting the two takes one install. `npm install -g altimate-code` ships the data engineering skills, and `/configure-claude` registers an `/altimate` command inside Claude Code so you can call Altimate's deterministic tools from the session you are already working in. `mcp_discover` reads the MCP servers you have already configured in `~/.claude.json`, so existing connections carry over. Altimate Code stays model-agnostic, so the same tools work if your team also uses Codex, Cursor, or a different provider. Start by installing the CLI and running `altimate-code check` on your project, which works without a model provider or API key. What data warehouses does Altimate Code support? Altimate Code supports Snowflake, BigQuery, Databricks, Redshift, Postgres, MySQL, DuckDB, and SQLite. It auto-detects your warehouse on setup and provides warehouse-specific tools for cost optimization, query analysis, and schema management. Cross-warehouse data parity checks run across all supported platforms simultaneously. Is Altimate Code free to use? Yes — Altimate Code is fully open-source under the MIT license. There is no paid tier, no platform fee, and no vendor lock-in. You bring your own LLM (Anthropic, OpenAI, Google, AWS Bedrock, Azure, Ollama, and 10+ more providers). The only cost is your LLM API tokens. What AI agents and IDEs does Altimate Code work with? Altimate Code works with Claude Code, Cursor, Windsurf, VS Code, and any MCP-compatible client, enabling data engineering workflows in each of them. It exposes tools via the Model Context Protocol (MCP), so any agent that supports MCP can use it. A team using Cursor for data engineering gets the same lineage, dbt, and warehouse tools as the terminal. It also has a built-in TUI (terminal UI) for running skills directly from the command line. What are ADE-Bench and DAB-Bench, and why do they matter? ADE-Bench is a benchmark created by Benn Stancil (founder of Mode) with dbt Labs that evaluates AI agents on real-world analytics engineering tasks. Tasks run in Docker sandboxes against actual dbt projects and databases. Altimate Code scores 78.0% using DeepSeek V4 Pro on DuckDB — ahead of Cortex Code CLI (65%), dbt Labs (59%), and Claude Code baseline (40%). DAB-Bench (Data Agent Benchmark) is a heterogeneous multi-database benchmark from UC Berkeley’s EPIC Lab that tests agents across PostgreSQL, MongoDB, SQLite, and DuckDB. Altimate Code scores 71.7% Pass@1 using GPT-5.5 + Claude Sonnet 4.6, ahead of Spacedock/Recce (67.2% on Opus 4.8), MinusX (65.2%), and Pi Coding Agent (61.0% on Opus 4.6). Together, these two independent benchmarks confirm that purpose-built data engineering tools consistently outperform general-purpose agents regardless of the underlying LLM. --- # Power User for dbt: VS Code Extension URL: https://altimate.ai/products/dbt-power-user Use the dbt VS Code extension for column lineage, autocomplete, AI docs, query previews, and health checks. Open source and built for modern data teams. ## Power User for dbt A dbt VS Code extension that makes VS Code work with dbt™ core, dbt™ cloud, and dbt™ Fusion. Autocomplete, column lineage, AI documentation, query preview, health checks, and more - without leaving the editor. [Install in VS Code](https://marketplace.visualstudio.com/items?itemName=innoverio.vscode-dbt-power-user) [Read the Docs](https://help.altimate.ai/dbt-power-user/) 1M+ VS Code + Open VSX installs 5.0★ VS Code Marketplace rating dbt 1.6+ Core, Cloud and Fusion supported ## Everything for dbt Across Develop, Test, and Collaborate Core features are free, no account required. Add a free Altimate AI key to unlock AI-powered features. ### Develop - Autocomplete model, macro, source names + go to definition - Compiled SQL preview as you write - Click to run parent / child models and tests - Generate and edit dbt models with natural language - Visualize compiled SQL as an interactive node graphbeta ### Test - Column-level and model-level lineage - Defer to prod - run only changed models - SQL validation without execution - Query preview, CTE profiling, and CSV export ### Collaborate - AI documentation generation for models and columns - Documentation editor - write and save to YAML - Project governance in IDE, Git, and CI/CD - Code collaboration and query bookmarks ## Model and column lineage with full visibility View lineage across sources, seeds, models, tests, metrics, and exposures. Click into any model to see its columns, tests, and linkage types. Column lineage traces individual columns through your entire dbt project. - Model-level lineage (no API key required) - Column-level lineage (with free Altimate AI key) - Shows linkage types: unchanged, aliased, transformed - Expandable depth: adjust scope without reopening - Export lineage for audits and downstream tooling Lineage - fct\_orders.amount\_usd raw\_orders amount ──▶ stg\_orders amount\_usd \[transformed\] ──▶ fct\_orders amount\_usd Downstream mart\_revenue / monthly\_total \[aggregated\] mart\_customer\_ltv / lifetime\_value \[aggregated\] Tests: not\_null, unique Documented ✓ ## Auto-complete dbt code and one-click definitions access Auto-fill model names, macros, sources, and docs as you type with the dbt VS Code extension. Click on any model name, macro, or source reference to jump directly to its definition. - Autocomplete for models, macros, sources, columns, and docs - Go-to-definition - click any ref() or source() to navigate - Jinja-SQL aware - understands dbt template syntax - Works across dbt Core and dbt Cloud projects - Supports dbt Mesh and multi-project workspaces fct\_orders.sql WITH orders AS (   SELECT \* FROM {{ ref('stg\_orders') }} Suggestions stg\_orders model stg\_customers model stg\_payments model ), ⌘ Click to go to definition → staging/stg\_orders.sql ## Preview query results, profile CTEs, and export See your query results instantly without leaving VS Code. Debug complex models CTE by CTE, export to CSV, and analyze data inline with graphs and filters - no context switching, no copy-pasting to a SQL client. - Preview results for your full model or just a selection - Debug step by step - preview individual CTEs separately - Export as CSV or analyze inline with graphs and filters Query Results - fct\_orders.sql Results SQL Graph | order\_id | customer\_id | amount\_usd | status | | --- | --- | --- | --- | | 1001 | C\_421 | 142.50 | completed | | 1002 | C\_089 | 88.00 | completed | | 1003 | C\_312 | 210.75 | pending | 3 of 2,847 rows · 0.8s Export CSV ↓ ## Generate model and column descriptions, saved directly to YAML Generate documentation for models and columns automatically, or write descriptions manually in the built-in UI editor. Descriptions are formatted and saved to YAML files. Hover over a column name in schema.yml to get instant AI suggestions for descriptions and tests. - AI-generated model and column descriptions - Hover shortcut - hover a column name: in YAML for instant AI actions - Manual editor - write and save descriptions in the UI - Auto-saved and formatted into YAML files - Code and documentation collaboration without creating a PR schema.yml - Documentation Editor models:   \- name: stg\_orders     columns:       \- name: order\_id (column) order\_id ✏ Suggest description 🧪 Suggest tests         tests: \[not\_null, unique\] dbt Governance ## Catch Issues Before They Reach Production ### Project health check Scan your entire dbt project for issues via the IDE, Python package, and in CI/CD pipelines. ### SQL validation without execution Identify SQL issues before running a query. Every error popup includes a quick-fix shortcut to resolve the issue. ### Defer to prod Build only the models you changed by deferring upstream models to production state. ### BigQuery cost estimator Estimate how much data will be processed by a dbt model in BigQuery directly inside VS Code, before you run it. ### SQL Visualizer Translate complex SQL queries into interactive graphs: CTEs, joins, filters, and unions visualized as connected components. beta ### Query history and bookmarks View the history of all queries and models. Compare results across code changes. Bookmark useful queries, and share across your team. Security ## SOC 2 Type 2 Certified Altimate AI is SOC 2 Type 2 certified. Data is encrypted in transit via TLS. Runs in a private VPC with RBAC access controls. SOC 2 Type 2 Certified TLS All data in transit encrypted AWS VPC Private cloud infrastructure MIT Open source license ## What Analytics Engineers Say "Does everything you need it to do - and it supports Athena!" James Hopgood VS Code Marketplace · Feb 2025 "Once properly configured it's a huge help in fast dbt development and also is an awesome leg-up in onboarding new team members." Oleg Brovko VS Code Marketplace · Oct 2024 "This extension is even better than the dbt Cloud IDE. It allows me to see the lineage of the models within VS Code, which is all I need." Gonzalo Hernandez VS Code Marketplace · Jun 2024 "Definitely the best extension out there for dbt with VS Code! The query preview and lineage screens are extremely useful, as is the feature to just run, build or test a single model. Doing this through the CLI would cost a lot of time." Ruben Helsloot VS Code Marketplace · May 2024 Common questions ## Frequently Asked Is the extension free? Yes - the core dbt VS Code extension is fully open source under the MIT license. Features like autocomplete, model lineage, compiled SQL preview, click-to-run, defer to prod, SQL validation, and query preview are all free with no account required. AI features (column lineage, explanation, translation, docs generation, test generation) require a free Altimate AI API key, available at altimate.ai. Does it work with dbt Core, dbt Cloud, and dbt Fusion? Yes - dbt Core, dbt Cloud, and dbt Fusion are all supported by the dbt VS Code extension. The extension auto-detects your dbt project type and reads your profiles.yml. Defer to prod works with dbt Core via the extension without requiring dbt Cloud, which makes the extension a practical dbt Cloud IDE alternative. You can use a local manifest file or a SaaS-hosted manifest for team sharing. What dbt versions are supported? The extension supports dbt versions 1.0 and above. The BigQuery cost estimator specifically requires dbt 1.6 or above. The dbt VS Code extension is fully compatible with dev containers, GitHub Codespaces, and VS Code Remote (WSL and SSH). Does it support dbt Mesh and multi-project setups? Yes. Power User supports dbt Mesh and federated multi-project workspaces. Cross-project refs resolve correctly in the editor, so autocomplete, go-to-definition, and lineage all work across project boundaries inside a single VS Code window. Teams that run several dbt projects get one dbt VS Code extension for all of them. Is Power User a good dbt Cloud IDE alternative? Yes, for teams that prefer to work in VS Code. Power User covers the core work a dbt developer does in the dbt Cloud IDE: autocomplete, model and column lineage, query preview, SQL validation, documentation generation, tests, and project navigation. Defer to prod runs against dbt Core, so a dbt Cloud account is optional. Can Power User help with AI-powered dbt development? Yes. Power User is an AI tool for dbt that runs inside your editor. It generates model and column descriptions, generates tests, explains and translates SQL, and lets you generate and edit dbt models in natural language. The AI features sit next to autocomplete, lineage, query preview, and SQL validation, so you use them without leaving the dbt project. What data does the extension send to Altimate AI? For AI features, the extension sends query context to the Altimate AI backend to generate responses. The only data stored is feedback you choose to provide, retained for 30 days for quality improvement. Telemetry follows VS Code's standard telemetry framework and can be disabled entirely via VS Code settings. All transmission is encrypted via TLS. Altimate AI is SOC 2 Type 2 certified. Where can I get help? Join the AI in DE Community on Slack - the most active place for questions. Documentation is at help.altimate.ai/dbt-power-user. You can also reach the team at altimate.ai/support. ## Join the Community Power User for dbt is built in the open. Get involved. ### Star & contribute on GitHub View on GitHub https://github.com/AltimateAI/vscode-dbt-power-user ### Join AI in DE Community on Slack Join Slack https://bit.ly/agentic-data-engineering-slack ### Report an issue on GitHub Report Issue https://github.com/AltimateAI/vscode-dbt-power-user/issues Get started ## Install in VS Code. Free to install. No account required for core features. [Install in VS Code](https://marketplace.visualstudio.com/items?itemName=innoverio.vscode-dbt-power-user) [Read the Docs](https://help.altimate.ai/dbt-power-user/) Open source · MIT dbt core · cloud · fusion SOC 2 Type 2 Free to install --- # Altimate Lite for Snowflake | Native App Cost Optimization URL: https://altimate.ai/products/altimate-lite Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user, and service. Altimate Lite · Snowflake Native App ## Cut Snowflake Costs on Autopilot Auto Tune suspends idle warehouses and scales clusters down the moment demand drops — private-preview customers saved an average of 19%, with no change in query performance. - Also included: detailed AI cost analysis across models, users, and services - Runs entirely inside your account — no outbound calls, ever [Install from Snowflake Marketplace](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) See how it works Free 21-day trial. Installs in under 5 minutes. Altimate Lite Warehouses dashboard showing 30-day total cost, potential annual savings, Auto Tune eligible and enabled counts, and a daily spend and savings chart. Warehouse cost & savings dashboard 19% Average warehouse savings in private preview 57.5% Highest single-day savings observed $100/mo Plus 2.5% of daily cost, per enabled warehouse — following the free 21-day trial How it works ## What Auto Tune and AI Cost Visibility Actually Do ### Auto Tune takes the idle spend out of every warehouse Auto Tune suspends a warehouse the moment it goes idle and scales clusters down the instant demand drops — continuously, without anyone watching the dashboard. - Only touches the warehouses you explicitly enable - Never modifies your data, queries, or roles — it tunes compute settings only - Turn it on or off per warehouse at any time Altimate Lite Warehouses dashboard showing 30-day total cost, potential annual savings, Auto Tune eligible and enabled counts, and a daily spend and savings chart. ### Savings you can point at, per warehouse Each warehouse shows what Auto Tune actually recovered, not a projection dressed up as a result. Warehouses that are still off show what they would recover if you turned them on. - Realized savings for enabled warehouses, in dollars - Projected annual savings for warehouses still eligible - Daily chart separating spend, recoverable idle, and recovered idle Altimate Lite warehouse detail page for a dbt pipeline warehouse showing Auto Tune enabled, realized savings of $351, and projected annual savings of $4,269. ### Every decision is timestamped and exportable Auto Tune keeps a full audit trail of every action it takes — cluster suspensions, scale-downs, resume events. - Agent Decisions chart showing tuning volume over time - Line-by-line history with timestamps in your local zone - Filterable by event type, exportable as CSV - Turn Auto Tune off at any time without uninstalling the app Agent Decisions chart and Auto Tune history log showing a timestamped list of warehouse suspend actions with a CSV export button. ### See what your AI services actually cost Most teams get a token count at the end of the month with no breakdown of what drove it. AI Services itemizes Cortex spend so you can tell finance why this week is double last week. - Daily spend stacked by service: Cortex Analyst, Search, Code, fine-tuning - Filter by service, user, and model - Spot the handful of users or the one expensive model driving the bill - Cortex is not required — the section stays empty if you do not use it Altimate Lite per-user AI usage page showing 30-day cost, tokens, queries, credits per million tokens, a daily spend chart, and an activity table with CSV export. Per-user detail — cost, tokens, queries, activity Native app, walled garden ## Nothing Leaves Your Snowflake Account Auto Tune, the web UI, every ACCOUNT\_USAGE query, and every Cortex call run inside your account. Only operational signals travel out, through Snowflake's own Native App event sharing — never query or table contents. ### Zero outbound calls Every query, every tuning decision, and the web UI itself execute inside your Snowflake account. Nothing is sent to an external endpoint. ### No InfoSec queue Nothing leaves your account, so the review that a hosted SaaS tool usually triggers does not apply. Billing runs through Snowflake credits. ### Opt-in per warehouse Auto Tune only touches warehouses you switch on, and only their compute settings. Your data, tables, and queries are never modified. ### One command to remove DROP APPLICATION takes the app, its services, and its warehouse with it. Toggle Auto Tune off at any time without uninstalling. Getting started ## Installed in Under Five Minutes All you need is ACCOUNTADMIN access in Snowsight. 01 ### Get the app Sign in to Snowsight as ACCOUNTADMIN, open Marketplace, search for Altimate AI, and click Get. Pick any small warehouse for the install. 02 ### Approve the privileges Six account-level privileges are listed in the install dialog. Approving the dialog is the grant — there is no SQL to run by hand. 03 ### Start the app Go to Apps → Altimate AI, click Start app, then Open web app. Snowflake authenticates you automatically. 04 ### Pick your warehouses Onboarding verifies ACCOUNT\_USAGE access, then you choose which warehouses to auto-tune and whether to monitor AI Services. [See the Full Install Guide](https://help.altimate.ai/snowflake-native-app/) Walkthrough ## Watch the Launch Demo Pradnesh walks through the cost dashboard, the per-warehouse Auto Tune toggle, the decision history, and AI cost observability. [Announcing Altimate Lite for AI and warehouse cost optimization](https://www.youtube.com/embed/pPDlkjMCSJY) Pricing ## Transparent, and No Sales Call Needed Turn it on for one or two warehouses, watch the savings and the audit history for a while, and expand once you are comfortable. 30 days Free trial, no cost $100 Per month after the trial 2.5% Of daily cost, after savings, per warehouse you enable You only pay for the warehouses you turn on, however many others exist in the account. Billing runs through Snowflake, so it can be paid with pre-purchased credits. Where it fits ## Lite Is the Front Door to the Platform Altimate Lite covers Auto Tune and AI cost visibility. The enterprise platform, running in production for customers processing billions of queries, adds the rest. ### Altimate Lite - Auto Tune: idle suspension and cluster scale-down - Warehouse cost, savings, and decision history - Cortex AI spend by service, user, and model - Runs entirely inside your Snowflake account ### Altimate Enterprise - Auto-resize — automatic warehouse right-sizing as workloads shift - Altimate Studio — a collaborative agent workspace with scheduled reporting, showback and chargeback, and SSO - Assisted optimization — guided fixes for query code, table and storage configuration, and dbt models, assignable to teams - Altimate Code — the open-source CLI and VS Code extension that catches costly SQL and dbt mistakes during development - …and many more [Explore the Platform](https://altimate.ai/platform) Common questions ## Frequently Asked Does my data leave my Snowflake account? No. Auto Tune, the web UI, every ACCOUNT\_USAGE query, and every Cortex call run inside your Snowflake account, with zero outbound network calls. Only operational signals travel out through Snowflake's Native App event sharing — never query or table contents. How much can I save? Private preview customers saved an average of 19% on warehouse costs. The lowest savings observed on any warehouse on any day was 12.4%, and the highest single-day figure was 57.5% — typically weekends, when workloads are least consistent. No degradation in query performance or queue times. How is this different from Snowflake's built-in auto-suspend? Snowflake's auto-suspend probes warehouse activity every 30 seconds. Auto Tune adjusts configuration every five to six seconds based on idle state, workload shifts, and predicted upcoming load — roughly ten checks in the same window. How does Auto Tune decide which warehouses to touch? It only touches warehouses you explicitly enable on the Warehouses page. It responds within about a minute of a warehouse going idle, and you can disable it at any time without uninstalling the app. Snowflake-managed warehouses, such as Adaptive Compute, are not eligible. What does Altimate Lite cost? A 21-day free trial, then $100/month plus 2.5% of the daily cost, after savings, of each warehouse you enable Auto Tune on. You only pay for the warehouses you turn on, however many others exist in the account. Billing runs through Snowflake, so you can pay with pre-purchased credits. What does the app itself cost to run? Auto Tune and the web UI share a compute pool, and the app's own warehouse only spins up when the UI runs queries. Net savings comfortably outweigh that runtime cost — the first warehouse Auto Tune touches usually pays for the app many times over. Which Snowflake editions are supported? Standard edition and above. Cortex AI is not required — the AI Services section simply stays empty if you are not using Cortex. Can I uninstall and reinstall later? Yes. Run DROP APPLICATION, or use Apps → Altimate AI → Manage → Uninstall in Snowsight. That removes the app, its services, and the NATIVE\_APP\_WH warehouse, while idle-timeout changes already made to your warehouses stay in place. Reinstalling lets you re-enable Auto Tune on the warehouses you choose. Get started ## Install It from the Marketplace. Free for 30 days. The first warehouse Auto Tune touches usually pays for the app many times over. [Install from Snowflake Marketplace](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) [Read the Docs](https://help.altimate.ai/snowflake-native-app/) Want auto-resize, Studio, and assisted optimization? [Talk to the Team](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings). --- # Agentic Data Engineering Platform URL: https://altimate.ai/platform Build with trusted data agents on an agentic data engineering platform with context, governance, and execution infrastructure for reliable data workflows. ## The Altimate Agentic Data Engineering Platform ## One Core Agentic Data Engineering Platform with Multiple Interfaces Interfaces CLI IDE EXTENSIONS STUDIO UI APPS Altimate Agentic Harness Context Governance Tools Skills Infrastructure Your Stack Your Data Stack Snowflake · Databricks · BigQuery · dbt · Airflow · Dagster · Redshift · 80+ more LLMs OpenAI · Anthropic · Gemini · open-source ## Context Graph Altimate is a powerful agentic data engineering platform for modern data teams built on a combination of domain-specific tools trained on years of Fortune 500 query patterns that delivers 85% accuracy on cost optimization and 30 to 50% documented savings on real customer workloads. ### Decision memory Auto-extracts ADRs from every merged PR so agents can avoid expensive mistakes. ### Technical metadata & cost intel Cost profiles, cross-platform lineage, incident history. ### Business metadata Ownership, PII / SOX / GDPR classification, retention policies. ## Governance Governance gives agents the same instinct a good engineer has: know what this will cost, know what it will break, know when to ask. ### Guardrails Pre-execution cost checks, blast radius analysis across downstream models, dashboards, PII columns, and incident history. ### Validation & reasoning traces Every agent action leaves a paper trail with cost estimate, blast radius, confidence score, and the actual rationale for the decision. ### Agent observatory Track agents working in real time. What each one knows, what it tried, what it escalated, and why. ## MCP, Tools & Skills Altimate provides a unified suite of domain-specific skills and tools, eliminating the need for multiple servers and credentials, enabling data agents to work effectively across the data stack. ### Local and remote, in the same config Agents reach the tools they need whether the server runs locally on an engineer's laptop or as a hosted endpoint behind OAuth. ### Vendor MCPs without the vendor sprawl Use a vendor's MCP, or a hosted version we manage instead. The agent sees one unified tool surface, not a dozen disconnected ones. ### Altimate MCP is the engine underneath Altimate MCP validates every connection, stores credentials in one place instead of scattered config files, and gives your team a clean interface for managing connections. ### Knowledge Hub context built in The agent gets context from Confluence, Google Docs, and Notion through the same connection layer that handles the tools. ### What's in the skills toolkit ### SQL Ten tools built on a real SQL parser including static analysis against 19 anti-patterns, translation across leading SQL databases and data warehouses. Optimization, formatting, diffing, fixing, explaining. ### Column-level lineage Trace columns end-to-end through joins and CTEs. Confidence drops for SELECT \*, Jinja, or unqualified tables. ### Schema, dbt, warehouse, FinOps Live schema indexing, dbt manifest parsing and verified dbt builds. Warehouse connectivity and query history. FinOps surfaces query cost and right-sizing on real workloads. ### Memory and custom State that survives long sessions. Bring your own tools and the agent treats them with the same compile-and-validate guarantees as the built-ins. ## Infrastructure An agent needs an environment where it can spin up the runtime, run the transformation against real data, execute the tests, and verify the output before anything touches production. It has to work across the stack you actually run, not just one warehouse. ### Sandboxes for the modern data stack Provisioned, isolated environments with the runtimes your team already uses, ready in seconds. Pair them with zero-copy clones of your warehouse and the agent runs against your real data, at production scale, without duplicating a byte. ### Bring your own LLM, or use ours Use what your security team already approved, or use the Altimate LLM Gateway to route across Sonnet 4.6, Opus 4.6, GPT-5.4, and more, picking the right model for the task. ALTIMATE STUDIO ## Your Entire Data Stack, in One Conversation. Studio is a specialized, multi-agent AI solution to interact with entire data stacks using natural language, providing rich analysis across platforms and technologies. Altimate Studio — slide 1 Altimate Studio — slide 2 ### Cross-stack analysis Lineage exploration, docs search, root cause analysis, optimization recommendations, and more — all from a single query. ### Enriched context Enrich queries with additional context. A shared Prompt Library lets teams launch common analyses instantly. ### Structured outputs Results are delivered with executive summaries, key metrics, root cause findings, and actionable next steps. ### Always in context Studio is context-aware, and persistent chat history means you can seamlessly resume past investigations. 90 deterministic tools ## Works with Your Stack. DATA PLATFORMS Snowflake Snowflake Databricks Databricks BigQuery BigQuery Microsoft Fabric Microsoft Fabric ORCHESTRATION dbt dbt Airflow Airflow Dagster Dagster Prefect Prefect BI & ANALYTICS Tableau Tableau Looker Looker Power BI Power BI Metabase Metabase DEV TOOLS GitHub GitHub Jira Jira Slack Slack VS Code VS Code and many more... ## Agentic Data Engineering, Answered What is an Agentic Data Engineering Platform? An Agentic Data Engineering Platform uses AI agents to automate and streamline data engineering tasks across the modern data stack. It helps teams manage data workflows, analyze data, troubleshoot issues, optimize pipelines, and work with tools such as Snowflake, Databricks, dbt, and other data solutions. Altimate connects to over a dozen of the most popular data platforms: Snowflake, Databricks, BigQuery, Redshift, Postgres, DuckDB, Trino, ClickHouse, MongoDB, MySQL, SQL Server, Oracle and SQLite. It exposes over 100 deterministic tools to the agents that work across them. What is Agentic Data Engineering? Agentic Data Engineering (ADE) applies AI agents to data engineering workflows so teams can automate complex tasks, investigate data issues, understand lineage, optimize costs, and improve pipeline development. Unlike traditional automation, AI agents can reason through tasks and use available data tools to complete multi-step workflows. Unlike general purpose LLMs or coding frameworks, ADE incorporates dedicated tools, both through inference and deterministic approaches, to automate the work of data engineering teams. That distinction is measurable rather than rhetorical. On ADE-Bench, dbt Labs' analytics-engineering benchmark, Altimate Code scores 78.0%, solving 32 of 41 tasks against real dbt projects in Docker sandboxes, putting it ahead of general-purpose coding agents running the same models. What are Data Agents? Data Agents are AI-powered agents designed specifically for data engineering. They can help analyze data, investigate pipeline failures, explore lineage, identify optimization opportunities, and perform other data engineering tasks while using the context and metadata available across an organization's data stack. Ours are model-agnostic: the same agents run on Anthropic, OpenAI, Google, Bedrock, Azure, Ollama, Snowflake Cortex or Databricks AI Gateway, so the deterministic layer stays constant when the model underneath changes. Can AI agents help build data pipelines? Yes. AI agents can help teams build data pipelines by assisting with pipeline development, SQL generation, testing, debugging, documentation, and data validation. They can also use existing metadata and lineage information to provide more context-aware recommendations throughout the development process. Two independent benchmarks put numbers on it. Altimate Code scores 78.0% on ADE-Bench (32 of 41 dbt tasks), and 71.7% Pass@1 on DAB-Bench, UC Berkeley EPIC Lab's multi-database benchmark, across PostgreSQL, MongoDB, SQLite and DuckDB. Our dbt Power User extension has passed 1M+ installs. How does an Agentic Data Engineering Platform improve data engineering workflows? An Agentic Data Engineering Platform can automate repetitive engineering tasks while helping teams investigate issues and make data-driven decisions faster. By connecting AI agents with data tools, metadata, lineage, and governance controls, teams can streamline workflows from development through production. A well-designed agentic data engineering harness does not sacrifice control or security: the team sees every change an agent makes, reviews the code it writes before that code runs, and sees the cost implication of each decision. The harness itself is open source under the MIT license, so the layer driving the agents can be read line by line. GET STARTED ## See What Your Agents Don't Know Yet. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) --- # Pricing URL: https://altimate.ai/pricing Start free with 10 million tokens — no credit card required. Upgrade to Pro at $29/seat/month when your agents start saving you money. ## Start free. Pay when it pays off. 10 million tokens free on sign-up, no credit card required. Upgrade to Pro at $29/seat when you need more. | | COMMUNITY | PRO | ENTERPRISE | | --- | --- | --- | --- | | PRICE | Free · No credit card. No time limit. · [Claim 10M Tokens →](https://app.myaltimate.com/register) | $29 /seat / month · Billed monthly. · [Start Pro →](https://app.myaltimate.com/register) | Custom · Annual. Outcome-based pricing. · [Contact Us →](https://calendly.com/d/crs2-7g8-xgm/enterprise-plan-pricing) | | FUNCTIONALITY | Everything except Apps & Enterprise Security features | Everything except Apps & Enterprise Security features | [Altimate Studio](https://altimate.ai/platform#studio) · Everything in Pro, plus: · - ✓ SOC2 · - ✓ SSO (SAML / OIDC) · - ✓ Dedicated instance · - ✓ On-premises proxy — AI traffic stays entirely within the client's own infrastructure | | AI-NATIVE APPS | N/A | N/A | - ✓ Warehouse Cost Optimization · - ✓ dbt Development Acceleration · - ✓ Data Pipelines Migration | | LLMs | - ✓ BYO LLM · - ✓ [Model Gateway](https://help.altimate.ai/workspaces/user-guide/components/llm-gateway/) | - ✓ BYO LLM · - ✓ [Model Gateway](https://help.altimate.ai/workspaces/user-guide/components/llm-gateway/) | Everything in Pro, plus: · - ✓ Custom LLM Deployments | | Monthly TOKENS | N/A (One-time 10M token allocation) | 20M tokens / seat / month · Top-up at $5 per 1M tokens · Max: 100M tokens / month | [Custom. Contact Us →](https://calendly.com/d/crs2-7g8-xgm/enterprise-plan-pricing) | | SUPPORT | Community Slack channels | Chatbot in product | - ✓ Enterprise support SLAs · - ✓ Custom contracts · - ✓ Email, Slack / Teams, Video | HOW LLM ACCESS WORKS ## How LLM access works Altimate is model-agnostic. Every plan (including Community) can connect to any LLM. Two paths: bring your own subscription, or route through our Gateway. ### BYO LLM Connect any OpenAI-compatible endpoint using your own API keys. No tokens consumed from your Altimate balance — you pay your provider directly. ### Model Gateway Access leading models through Altimate without a separate LLM subscription. Requests are routed through our infrastructure. Fastest way to get started with no API key management. ### Custom LLM Deployments Enterprise-only. Deploy models within your own infrastructure or a dedicated Altimate instance. Full data isolation, custom fine-tuning, and compliance controls. ### Token visibility Every LLM call is logged and attributed. Community and Pro dashboards show per-tool, per-developer token usage so you know exactly where capacity goes. FAQ ## Pricing Questions Can I use Altimate Code for free forever? Yes. Altimate Code is MIT licensed. When you exhaust the free tier, upgrade to Pro to keep going — Community users cannot purchase additional tokens separately. What happens when I run out of free tokens? You'll need to upgrade to Pro ($29/seat/month) to continue using the Altimate AI LLM. Alternatively, configure your own LLM via BYO LLM or Model Gateway — all tools continue to work regardless. What is the Model Gateway? The Model Gateway lets you access leading models through Altimate AI without a separate LLM subscription. Requests are routed through our infrastructure. It's the fastest way to get started without managing API keys. Is Pro per-seat or flat? Per-seat at $29/month. Each seat includes 20M tokens. Need more? Pro plans can go up to 100M tokens/month total (base included) at $5/1M. What are the Enterprise Apps? Three purpose-built applications for large data teams, such as: [Warehouse Cost Optimization](https://altimate.ai/use-cases/altimate-for-databricks) (find and eliminate wasteful spend), [dbt Development Acceleration](https://altimate.ai/use-cases/analytics-engineering) (AI-assisted dbt authoring at scale), and [Data Pipelines Migration](https://altimate.ai/use-cases/workload-migration) (automate cross-platform migrations). What's included in Enterprise Security? SOC2 compliance, SSO (SAML/OIDC), dedicated instance, on-premises proxy, and Enterprise SLA. These are available exclusively on the Enterprise plan. Can we run Altimate on our own infrastructure? Yes! Enterprise plans include an on-premises proxy and custom LLM deployments. AI traffic stays within your environment, and you retain full control over which models process your data. ## Ten Million Tokens. No Credit Card. No Reason Not To. $ npm install -g altimate-code --- # Customers URL: https://altimate.ai/customers Enterprise data teams — Elastic, Booking.com, Carta, Coinbase, and 20+ more — run Altimate in production. 3× faster development. 10–15× faster migrations. REAL WORLD IMPACT ## Enterprise Data Teams Are Building with Altimate Every team that deploys Altimate builds a context graph that gets smarter with every PR, every incident, every session. TRUSTED BY StubHub Coinbase Booking.com Peloton Workday Capgemini Slack Siemens Healthineers Oodrive Explorium Outreach Elastic Chatbooks Ooni World of Books Clicksign Qover Carta DAZN Ritchie Bros. Pepsi Docusign ScalePad Sunrun ...AND MANY MORE. --- # Integrations URL: https://altimate.ai/integrations altimate-code works with your entire data stack — warehouses, orchestrators, collaboration tools, and any LLM provider. ## Integrations altimate-code works with your entire data stack — warehouses, orchestrators, collaboration tools, and any LLM provider. ## All integrations All 35 Data Stores 9 Data Stack Tools 5 Collaboration 3 LLM Providers 12 Coming Soon 6 35 integrations Snowflake ### Snowflake Supported Run queries, manage warehouses, explore data BigQuery ### BigQuery Supported Execute queries, manage datasets, explore tables Databricks ### Databricks Supported Cost intelligence and Auto Tune for clusters, jobs, SQL warehouses, and AI/ML services PostgreSQL ### PostgreSQL Supported Execute SQL, manage databases, explore schemas MySQL ### MySQL Supported Execute queries, manage databases, explore schemas Redshift ### Redshift Supported Run queries, manage clusters, analyze performance DuckDB ### DuckDB Supported Local analytics, in-process SQL, file queries SQL Server ### SQL Server Supported Execute T-SQL, manage databases, explore schemas Oracle ### Oracle Supported Execute PL/SQL, manage schemas, explore tables dbt ### dbt Supported Compile models, run tests, generate documentation Airflow ### Airflow Supported Manage DAGs, monitor workflows, trigger runs Dagster ### Dagster Supported Manage DAGs, monitor workflows, trigger runs Google Sheets ### Google Sheets Supported Get, update, and manage Sheets Custom API ### Custom API Supported Custom API integration capabilities GitHub ### GitHub Supported Manage repositories, issues, and pull requests Jira ### Jira Supported Create/update tickets, track progress Linear ### Linear Supported Create/update issues, track progress, get documents Anthropic ### Anthropic Supported Claude Opus 4.6, Sonnet 4.6, Haiku 4.5 OpenAI ### OpenAI Supported GPT-4o, Codex models via API or ChatGPT Google Gemini ### Google Gemini Supported Gemini 2.5 Pro and other Gemini models AWS Bedrock ### AWS Bedrock Supported Access Anthropic, Meta, and other models via AWS Azure OpenAI ### Azure OpenAI Supported GPT-4o and other models via Azure Google Vertex AI ### Google Vertex AI Supported Gemini and Anthropic models via Google Cloud Ollama ### Ollama Supported Run Llama, Mistral, and other models locally OpenRouter ### OpenRouter Supported Access 150+ models through a single API GitHub Copilot ### GitHub Copilot Supported Use GPT-4o via your GitHub Copilot subscription Mistral ### Mistral Supported Mistral Large, Medium, and other models Groq ### Groq Supported Ultra-fast inference for Llama and Mixtral xAI ### xAI Supported Grok models via xAI API Alation ### Alation Coming soon Access data catalogs, query metadata Azure DevOps ### Azure DevOps Coming soon Manage work items, pipelines, repositories ServiceNow ### ServiceNow Coming soon Create and manage incidents Apache Kafka ### Apache Kafka Coming soon Manage topics, monitor consumers, stream data Apache Spark ### Apache Spark Coming soon Run jobs, manage clusters, process data at scale Kubernetes ### Kubernetes Coming soon Manage pods, deployments, and cluster resources --- # Company URL: https://altimate.ai/company Altimate AI is on a mission to build autonomous agents that can be trusted with production data. Our founding story, principles, track record, and team. COMPANY ## We Build the Infrastructure That Makes Data Agents Actually Work. Altimate AI has a specific obsession: to build autonomous agents that can be trusted with production data. 74.4% ADE-Bench$840K avg. saved/yrOpen source [See the Platform →](https://altimate.ai/platform) [Book a Demo →](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) Looking to join us? See open roles ↓ WHY WE EXIST ## Humans Carry Context The data stack is full of context that lives in engineers' heads: The revenue recognition model was built a certain way because of an ASC 606 ruling in 2021; the warehouse is sized the way it is because of a $340K incident in Q3 2024; the join logic in that pipeline encodes a business rule that was never written down anywhere except in a Slack message that's now three years old. For twenty years, that was fine. Humans carry context. They slow down when they're uncertain. They ask questions. Agents don't. When agents act on production data, they hit that context gap at machine speed. They don't slow down. They execute — confidently, incorrectly, at scale. We built Altimate because that gap has a solution: a context graph structured for agent consumption, governance guardrails that know your stack, and infrastructure built for how agents actually work. Not retrofitted from human systems. Built from the ground up for the agent-first data stack. That's what we've been building. That's what we're still building. HUMAN AGENT TEAM ## The People Building It. An agentic-first team that has shipped a lot. Pradnesh Patil \[ CO-FOUNDER · VERIFIED \] ### Pradnesh Patil CEO & Co-founder https://www.linkedin.com/in/pradneshpatil/ Anand Gupta \[ CO-FOUNDER · VERIFIED \] ### Anand Gupta CTO & Co-founder https://www.linkedin.com/in/anandguptas/ INVESTORS & ADVISORS ## Backed by World-Class Investors and Advisors. Altimate ALTIMATE Michel Tricot Michel Tricot CEO & Co-Founder at Airbyte LinkedIn → https://www.linkedin.com/in/micheltricot/ Shashank Tiwari Shashank Tiwari Entrepreneur, CEO, Executive, Engineer, Mathematician LinkedIn → https://www.linkedin.com/in/tshanky/ Pradeep Padala Pradeep Padala Entrepreneur and Investor LinkedIn → https://www.linkedin.com/in/ppadala/ Ashish Agrawal Ashish Agrawal Entrepreneur and Investor LinkedIn → https://www.linkedin.com/in/agrawalashishkumar CAREERS STATUS: HIRING · 4 OPEN ROLES ## We're Hiring. Small team. Real problems. Every engineer ships things that end up on the platform. ~/altimate/fit\_check.sh WHO WE'RE LOOKING FOR - 01 PASS Shipped something non-trivial you can walk us through. - 02 PASS Opinions about data engineering from doing it. - 03 PASS Know the difference between prototype and production. ~/altimate/not\_for.sh NOT FOR YOU IF - 01 FAIL Read about it. Never built it. - 02 FAIL Resume optimizers. - 03 FAIL "Move fast, break production" types. [See open roles →](https://jobs.ashbyhq.com/altimate) CONTACT ## Get in Touch. SALES Book a Demo → https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings GENERAL ENQUIRIES Contact Us → mailto:hello@altimate.ai COMMUNITY & SUPPORT [help.altimate.ai/code](https://help.altimate.ai/code/) [GitHub Issues](https://github.com/AltimateAI/altimate-code/issues) [Slack community](https://bit.ly/agentic-data-engineering-slack) --- # Security URL: https://altimate.ai/security Altimate AI is built security-first. Metadata only, never your data. SOC 2 Type II certified, multi-tenant isolated, encrypted in transit and at rest. SECURITY ## Trusted by Security Teams in Leading Enterprises Altimate AI is committed to the privacy and security of every customer environment. The platform operates on metadata only — no customer data is ever retrieved, stored, or used to train models. SOC 2 TYPE II AICPA CERTIFIED HOW WE PROTECT YOU ## Six Layers. No Exceptions. ### Metadata only Altimate never reads, stores, or processes your actual data. Only metadata — table names, schemas, query shapes — enters the platform. Your data stays in your environment. ### Multi-tenant isolation Every customer instance gets its own isolated environment with a unique URL. No shared compute, no shared storage between tenants. ### Encryption everywhere All metadata is encrypted in transit (TLS 1.2+) and at rest using AES-256 via our cloud providers. Keys are rotated automatically. ### Human approval on critical steps Agents operating on production systems require explicit human sign-off before executing write actions. No autonomous mutation without governance clearance. ### Customer-configurable guardrails Define your own security perimeter: which warehouses agents can touch, which schemas are off-limits, which actions require dual approval. ### LLM isolation We do not use customer data or metadata to train any LLM. Prompts are scoped to the session and never persisted to model providers. THE ARCHITECTURE ## Your Data Never Leaves Your Environment When an agent analyses your Snowflake warehouse, it reads query execution plans, cost metadata, and schema definitions — not your actual rows. The same principle applies across every integration: Airflow, dbt, Databricks, GitHub. Shape, not substance. Write actions (query changes, pipeline edits, warehouse config updates) are gated by governance guardrails and, where configured, human approval workflows. Agents don't act on production systems without clearance. SECURITY FAQ ## Common Questions What data do you use? We do not use actual data for any Altimate AI features. We use only metadata e.g. table names, column schemas, query shapes. Your data never leaves your environment. Do you use customer data or metadata to train LLMs? No. We do not train any LLM using customer data or metadata. What's the quality of AI features? Our agents are calibrated with confidence thresholds. They proceed autonomously at high confidence, present options at medium confidence, and escalate below a minimum threshold rather than guessing. You can always see the agent's reasoning trace. Is Altimate SOC 2 certified? Yes. Altimate AI is SOC 2 Type II certified. Certification documentation is available under NDA upon request. Can I run Altimate in my own cloud environment? Yes. Enterprise customers can deploy Altimate in a VPC or on-premise environment. Contact us to discuss your architecture requirements. More questions? [Check the documentation](https://docs.myaltimate.com/arch/faq/) or [contact our security team](mailto:security@altimate.ai). GET STARTED ## Security Documentation Available on Request SOC 2 Type II report, penetration test summaries, and architecture diagrams are available to enterprise prospects under NDA. [Start Free](https://app.myaltimate.com/register) [Request Security Docs](mailto:security@altimate.ai) --- # Changelog URL: https://altimate.ai/changelog Every shipped update to the Altimate platform: dbt Power User, Altimate Code, Datamates, and cost intelligence for Snowflake and Databricks. ## Changelog Release notes for the whole Altimate suite: dbt Power User, Altimate Code, Datamates, Snowflake, and Databricks. Filter Power user for dbt11 Altimate Code11 MCP Server8 Snowflake App4 Databricks App6 [RSS](https://altimate.ai/changelog.rss.xml) Timeline Showing 15 of 33 updates ## August 2026 1. New Altimate Code Aug 8 ### YOLO mode toggle in Altimate Code Chat Altimate Code Chat now lets you enable YOLO mode directly from the message composer. When active, the agent can complete multi-step work without pausing for approval at every step, making longer workflows more seamless. 2. Improved Altimate Code Aug 1 ### Install Altimate Code skills by typing a source Installing skills in Altimate Code is now more intuitive. In the \`/skills\` list, enter a GitHub URL, an \`owner/repo\`, or a local path and press Enter to install it directly, without relying on terminal keyboard shortcuts. ## July 2026 1. New Altimate Code MCP Server Jul 25 ### Altimate Code catches incomplete grain-key testing in dbt reviews Altimate Code now catches incomplete dbt uniqueness testing during pull request review. If a grain-key column is tested for uniqueness but not for null values, the review flags the gap before it can allow duplicate or incomplete data into a model. 2. Improved Altimate Code MCP Server Jul 25 ### Live progress in Altimate Code Altimate Code now shows live progress while preparing your request. Phase updates, such as preparing context or connecting to your project, make it clear that work is moving forward during startup. 3. Improved Altimate Code MCP Server Jul 25 ### Risky dbt metadata changes now get a full review in Altimate Code Altimate Code now sends higher-risk dbt metadata changes through a full review instead of automatically approving them. This adds an extra safeguard for changes such as weakened uniqueness tests on contracted marts or updates to finance-relevant SQL. 4. Improved MCP Server Jul 18 ### Altimate Code chat shows what it's doing, live Altimate Code now makes agent work visible as it happens. When the chat needs time to inspect project files, call tools, or gather context, users can see live progress instead of waiting without feedback. This makes the experience feel faster, clearer, and more reliable, especially for longer-running workflows. Tool steps are also shown with friendly labels and source badges, so users can quickly understand what the agent is doing and which type of tool it is using. 5. Improved MCP Server Jul 18 ### Altimate Code installs on Windows without Node Installing or updating Altimate Code on Windows no longer requires Node or npm. The Install panel and the background updater now use a PowerShell installer that drops the Altimate Code binary straight into place, for both first-time installs and later updates. If a locked-down Windows machine ever made installing via npm painful, that step is gone. 6. NewPower user for dbt Jul 18 ### Explain and profile queries with Altimate Code Altimate Code now brings query explanation and performance profiling directly into the Query Results panel. After running a query, users can ask Altimate Code to explain the compiled SQL or profile the query with the full context already loaded. This makes it easier for analytics teams to understand results, investigate slow queries, and move from question to next step without leaving dbt Power User. 7. Improved MCP Server Jul 5 ### Altimate Code auto-installs on first chat open Opening the chat panel for the first time now installs Altimate Code automatically in the background instead of sending you to a manual install screen. You'll see a short "Preparing your Altimate Code experience" notification while it sets up, with the old manual install flow kept as a fallback if auto-install fails. Prefer to do it yourself? Turn off the new auto-install setting in your editor preferences. 8. ImprovedSnowflake AppDatabricks App Jul 5 ### Better error messaging when no team is selected in Studio Asking Studio any question without a team selected now shows a clear message prompting you to pick a team, instead of a generic error with no obvious next step. 9. Improved MCP Server Jul 5 ### Datamate MCP server defaults to stdio transport The datamate MCP server now uses the stdio transport by default, so external MCP clients like Claude Code, Claude Desktop, and Cursor connect out of the box. Previously you had to manually switch the transport setting to get this; now it's the default, and you can still switch back if you prefer the old behavior. 10. ImprovedSnowflake AppDatabricks App Jul 5 ### Enterprise AI token usage is now metered and visible Enterprise and premium plans previously bypassed usage metering entirely, so there was no way to see how many AI tokens your team had used. Usage is now tracked against your provisioned grant, the same way every other plan already works, and shows up in the same token usage view — so you can see exactly what's been used and what's left. 11. ImprovedDatabricks App Jul 5 ### Filter AI/ML opportunities by endpoint name You can now filter Databricks AI/ML cost-saving opportunities by endpoint name. Type or paste part of a name to jump straight to the model-serving, vector-search, or AI Gateway endpoint you're looking for. The filter also works alongside the existing Type filter, and accepts letters, numbers, and symbols. 12. ImprovedDatabricks App Jul 5 ### Fixed Blank Runtime Engine and Photon fields The Runtime Engine and the Photon Enabled fields always showed a blank dash no matter how the cluster was actually configured. Both now populate correctly from the cluster's runtime version, so you can see at a glance whether Photon is turned on. 13. ImprovedDatabricks App Jul 5 ### Fixed Workspace-scoped visibility for model-serving endpoint opportunities Fixed: Access scoping for model-serving endpoint opportunities Model-serving endpoint opportunities are now properly scoped by workspace access, matching the behavior of cluster, job, and warehouse opportunities. Load more --- # Privacy Policy URL: https://altimate.ai/privacy Read the Altimate AI privacy policy: what personal data we collect, how we use, keep, and share it, and your rights under GDPR and the CCPA/CPRA. Legal ## Privacy Policy Last updated: November 06, 2023 This Privacy Policy describes Our policies and procedures on the collection, use and disclosure of Your information when You use the Service and tells You about Your privacy rights and how the law protects You. We use Your Personal data to provide and improve the Service. By using the Service, You agree to the collection and use of information in accordance with this Privacy Policy. ## Interpretation and Definitions ### Interpretation The words of which the initial letter is capitalized have meanings defined under the following conditions. The following definitions shall have the same meaning regardless of whether they appear in singular or in plural. ### Definitions For the purposes of this Privacy Policy: - **Account** means a unique account created for You to access our Service or parts of our Service. - **Affiliate** means an entity that controls, is controlled by or is under common control with a party, where "control" means ownership of 50% or more of the shares, equity interest or other securities entitled to vote for election of directors or other managing authority. - **Application** refers to DataPilot, the software program provided by the Company. - **Business**, for the purpose of CCPA/CPRA, refers to the Company as the legal entity that collects Consumers' personal information and determines the purposes and means of the processing of Consumers' personal information, or on behalf of which such information is collected and that alone, or jointly with others, determines the purposes and means of the processing of consumers' personal information, that does business in the State of California. - **CCPA** and/or **CPRA** refers to the California Consumer Privacy Act (the "CCPA") as amended by the California Privacy Rights Act of 2020 (the "CPRA"). - **Company** (referred to as either "the Company", "We", "Us" or "Our" in this Agreement) refers to Altimate Inc., 440 North Wolfe Road #150, Sunnyvale, CA 94085 Phone: +1 (650) 254-6266. Email: [legal@altimate.ai](mailto:legal@altimate.ai) For the purpose of the GDPR, the Company is the Data Controller. - **Consumer**, for the purpose of the CCPA/CPRA, means a natural person who is a California resident. A resident, as defined in the law, includes (1) every individual who is in the USA for other than a temporary or transitory purpose, and (2) every individual who is domiciled in the USA who is outside the USA for a temporary or transitory purpose. - **Cookies** are small files that are placed on Your computer, mobile device or any other device by a website, containing the details of Your browsing history on that website among its many uses. - **Country** refers to: California, United States - **Data Controller**, for the purposes of the GDPR (General Data Protection Regulation), refers to the Company as the legal person which alone or jointly with others determines the purposes and means of the processing of Personal Data. - **Device** means any device that can access the Service such as a computer, a cellphone or a digital tablet. - **Do Not Track** (DNT) is a concept that has been promoted by US regulatory authorities, in particular the U.S. Federal Trade Commission (FTC), for the Internet industry to develop and implement a mechanism for allowing internet users to control the tracking of their online activities across websites. - **GDPR** refers to EU General Data Protection Regulation. - **Personal Data** is any information that relates to an identified or identifiable individual. For the purposes of GDPR, Personal Data means any information relating to You such as a name, an identification number, location data, online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity. For the purposes of the CCPA/CPRA, Personal Data means any information that identifies, relates to, describes or is capable of being associated with, or could reasonably be linked, directly or indirectly, with You. - **Service** refers to the Application or the Website or both. - **Service Provider** means any natural or legal person who processes the data on behalf of the Company. It refers to third-party companies or individuals employed by the Company to facilitate the Service, to provide the Service on behalf of the Company, to perform services related to the Service or to assist the Company in analyzing how the Service is used. For the purpose of the GDPR, Service Providers are considered Data Processors. - **Usage Data** refers to data collected automatically, either generated by the use of the Service or from the Service infrastructure itself (for example, the duration of a page visit). - **Website** refers to Altimate AI, accessible from [https://altimate.ai](https://altimate.ai) - **You** means the individual accessing or using the Service, or the company, or other legal entity on behalf of which such individual is accessing or using the Service, as applicable. Under GDPR, You can be referred to as the Data Subject or as the User as you are the individual using the Service. ## Collecting and Using Your Personal Data ### Types of Data Collected #### Personal Data While using Our Service, We may ask You to provide Us with certain personally identifiable information that can be used to contact or identify You. Personally identifiable information may include, but is not limited to: - Email address - First name and last name - Usage Data #### Usage Data Usage Data is collected automatically when using the Service. Usage Data may include information such as Your Device's Internet Protocol address (e.g. IP address), browser type, browser version, the pages of our Service that You visit, the time and date of Your visit, the time spent on those pages, unique device identifiers and other diagnostic data. When You access the Service by or through a mobile device, We may collect certain information automatically, including, but not limited to, the type of mobile device You use, Your mobile device unique ID, the IP address of Your mobile device, Your mobile operating system, the type of mobile Internet browser You use, unique device identifiers and other diagnostic data. We may also collect information that Your browser sends whenever You visit our Service or when You access the Service by or through a mobile device. #### Tracking Technologies and Cookies We use Cookies and similar tracking technologies to track the activity on Our Service and store certain information. Tracking technologies used are beacons, tags, and scripts to collect and track information and to improve and analyze Our Service. The technologies We use may include: - **Cookies or Browser Cookies.** A cookie is a small file placed on Your Device. You can instruct Your browser to refuse all Cookies or to indicate when a Cookie is being sent. However, if You do not accept Cookies, You may not be able to use some parts of our Service. Unless you have adjusted Your browser setting so that it will refuse Cookies, our Service may use Cookies. - **Web Beacons.** Certain sections of our Service and our emails may contain small electronic files known as web beacons (also referred to as clear gifs, pixel tags, and single-pixel gifs) that permit the Company, for example, to count users who have visited those pages or opened an email and for other related website statistics (for example, recording the popularity of a certain section and verifying system and server integrity). Cookies can be "Persistent" or "Session" Cookies. Persistent Cookies remain on Your personal computer or mobile device when You go offline, while Session Cookies are deleted as soon as You close Your web browser. Learn more about cookies on the [Free Privacy Policy website](https://www.freeprivacypolicy.com/blog/sample-privacy-policy-template/#Use_Of_Cookies_And_Tracking) article. We use both Session and Persistent Cookies for the purposes set out below: - **Necessary / Essential Cookies** Type: Session Cookies Administered by: Us Purpose: These Cookies are essential to provide You with services available through the Website and to enable You to use some of its features. They help to authenticate users and prevent fraudulent use of user accounts. Without these Cookies, the services that You have asked for cannot be provided, and We only use these Cookies to provide You with those services. - **Cookies Policy / Notice Acceptance Cookies** Type: Persistent Cookies Administered by: Us Purpose: These Cookies identify if users have accepted the use of cookies on the Website. - **Functionality Cookies** Type: Persistent Cookies Administered by: Us Purpose: These Cookies allow us to remember choices You make when You use the Website, such as remembering your login details or language preference. The purpose of these Cookies is to provide You with a more personal experience and to avoid You having to re-enter your preferences every time You use the Website. - **Tracking and Performance Cookies** Type: Persistent Cookies Administered by: Third-Parties Purpose: These Cookies are used to track information about traffic to the Website and how users use the Website. The information gathered via these Cookies may directly or indirectly identify you as an individual visitor. This is because the information collected is typically linked to a pseudonymous identifier associated with the device you use to access the Website. We may also use these Cookies to test new pages, features or new functionality of the Website to see how our users react to them. For more information about the cookies we use and your choices regarding cookies, please visit our Cookies Policy or the Cookies section of our Privacy Policy. ### Use of Your Personal Data The Company may use Personal Data for the following purposes: - **To provide and maintain our Service**, including to monitor the usage of our Service. - **To manage Your Account:** to manage Your registration as a user of the Service. The Personal Data You provide can give You access to different functionalities of the Service that are available to You as a registered user. - **For the performance of a contract:** the development, compliance and undertaking of the purchase contract for the products, items or services You have purchased or of any other contract with Us through the Service. - **To contact You:** To contact You by email, telephone calls, SMS, or other equivalent forms of electronic communication, such as a mobile application's push notifications regarding updates or informative communications related to the functionalities, products or contracted services, including the security updates, when necessary or reasonable for their implementation. - **To provide You** with news, special offers and general information about other goods, services and events which we offer that are similar to those that you have already purchased or enquired about unless You have opted not to receive such information. - **To manage Your requests:** To attend and manage Your requests to Us. - **For business transfers:** We may use Your information to evaluate or conduct a merger, divestiture, restructuring, reorganization, dissolution, or other sale or transfer of some or all of Our assets, whether as a going concern or as part of bankruptcy, liquidation, or similar proceeding, in which Personal Data held by Us about our Service users is among the assets transferred. - **For other purposes**: We may use Your information for other purposes, such as data analysis, identifying usage trends, determining the effectiveness of our promotional campaigns and to evaluate and improve our Service, products, services, marketing and your experience. We may share Your personal information in the following situations: - **With Service Providers:** We may share Your personal information with Service Providers to monitor and analyze the use of our Service, for payment processing, to contact You. - **For business transfers:** We may share or transfer Your personal information in connection with, or during negotiations of, any merger, sale of Company assets, financing, or acquisition of all or a portion of Our business to another company. - **With Affiliates:** We may share Your information with Our affiliates, in which case we will require those affiliates to honor this Privacy Policy. Affiliates include Our parent company and any other subsidiaries, joint venture partners or other companies that We control or that are under common control with Us. - **With business partners:** We may share Your information with Our business partners to offer You certain products, services or promotions. - **With other users:** when You share personal information or otherwise interact in the public areas with other users, such information may be viewed by all users and may be publicly distributed outside. - **With Your consent**: We may disclose Your personal information for any other purpose with Your consent. ### Retention of Your Personal Data The Company will retain Your Personal Data only for as long as is necessary for the purposes set out in this Privacy Policy. We will retain and use Your Personal Data to the extent necessary to comply with our legal obligations (for example, if we are required to retain your data to comply with applicable laws), resolve disputes, and enforce our legal agreements and policies. The Company will also retain Usage Data for internal analysis purposes. Usage Data is generally retained for a shorter period of time, except when this data is used to strengthen the security or to improve the functionality of Our Service, or We are legally obligated to retain this data for longer time periods. ### Transfer of Your Personal Data Your information, including Personal Data, is processed at the Company's operating offices and in any other places where the parties involved in the processing are located. It means that this information may be transferred to — and maintained on — computers located outside of Your state, province, country or other governmental jurisdiction where the data protection laws may differ than those from Your jurisdiction. Your consent to this Privacy Policy followed by Your submission of such information represents Your agreement to that transfer. The Company will take all steps reasonably necessary to ensure that Your data is treated securely and in accordance with this Privacy Policy and no transfer of Your Personal Data will take place to an organization or a country unless there are adequate controls in place including the security of Your data and other personal information. ### Delete Your Personal Data You have the right to delete or request that We assist in deleting the Personal Data that We have collected about You. Our Service may give You the ability to delete certain information about You from within the Service. You may update, amend, or delete Your information at any time by signing in to Your Account, if you have one, and visiting the account settings section that allows you to manage Your personal information. You may also contact Us to request access to, correct, or delete any personal information that You have provided to Us. Please note, however, that We may need to retain certain information when we have a legal obligation or lawful basis to do so. ### Disclosure of Your Personal Data #### Business Transactions If the Company is involved in a merger, acquisition or asset sale, Your Personal Data may be transferred. We will provide notice before Your Personal Data is transferred and becomes subject to a different Privacy Policy. #### Law enforcement Under certain circumstances, the Company may be required to disclose Your Personal Data if required to do so by law or in response to valid requests by public authorities (e.g. a court or a government agency). #### Other legal requirements The Company may disclose Your Personal Data in the good faith belief that such action is necessary to: - Comply with a legal obligation - Protect and defend the rights or property of the Company - Prevent or investigate possible wrongdoing in connection with the Service - Protect the personal safety of Users of the Service or the public - Protect against legal liability ### Security of Your Personal Data The security of Your Personal Data is important to Us, but remember that no method of transmission over the Internet, or method of electronic storage is 100% secure. While We strive to use commercially acceptable means to protect Your Personal Data, We cannot guarantee its absolute security. ## Detailed Information on the Processing of Your Personal Data The Service Providers We use may have access to Your Personal Data. These third-party vendors collect, store, use, process and transfer information about Your activity on Our Service in accordance with their Privacy Policies. ### Analytics We may use third-party Service providers to monitor and analyze the use of our Service. - **Posthog** Their Privacy Policy can be viewed at [https://posthog.com/](https://posthog.com/) ### Email Marketing We may use Your Personal Data to contact You with newsletters, marketing or promotional materials and other information that may be of interest to You. You may opt-out of receiving any, or all, of these communications from Us by following the unsubscribe link or instructions provided in any email We send or by contacting Us. We may use Email Marketing Service Providers to manage and send emails to You. - **Mailchimp** Mailchimp is an email marketing sending service provided by The Rocket Science Group LLC. For more information on the privacy practices of Mailchimp, please visit their Privacy policy: [https://mailchimp.com/legal/privacy/](https://mailchimp.com/legal/privacy/) ### Payments We may provide paid products and/or services within the Service. In that case, we may use third-party services for payment processing (e.g. payment processors). We will not store or collect Your payment card details. That information is provided directly to Our third-party payment processors whose use of Your personal information is governed by their Privacy Policy. These payment processors adhere to the standards set by PCI-DSS as managed by the PCI Security Standards Council, which is a joint effort of brands like Visa, Mastercard, American Express and Discover. PCI-DSS requirements help ensure the secure handling of payment information. - **Stripe** Their Privacy Policy can be viewed at [https://stripe.com/us/privacy](https://stripe.com/us/privacy) ## GDPR Privacy ### Legal Basis for Processing Personal Data under GDPR We may process Personal Data under the following conditions: - **Consent:** You have given Your consent for processing Personal Data for one or more specific purposes. - **Performance of a contract:** Provision of Personal Data is necessary for the performance of an agreement with You and/or for any pre-contractual obligations thereof. - **Legal obligations:** Processing Personal Data is necessary for compliance with a legal obligation to which the Company is subject. - **Vital interests:** Processing Personal Data is necessary in order to protect Your vital interests or of another natural person. - **Public interests:** Processing Personal Data is related to a task that is carried out in the public interest or in the exercise of official authority vested in the Company. - **Legitimate interests:** Processing Personal Data is necessary for the purposes of the legitimate interests pursued by the Company. In any case, the Company will gladly help to clarify the specific legal basis that applies to the processing, and in particular whether the provision of Personal Data is a statutory or contractual requirement, or a requirement necessary to enter into a contract. ### Your Rights under the GDPR The Company undertakes to respect the confidentiality of Your Personal Data and to guarantee You can exercise Your rights. You have the right under this Privacy Policy, and by law if You are within the EU, to: - **Request access to Your Personal Data.** The right to access, update or delete the information We have on You. Whenever made possible, you can access, update or request deletion of Your Personal Data directly within Your account settings section. If you are unable to perform these actions yourself, please contact Us to assist You. This also enables You to receive a copy of the Personal Data We hold about You. - **Request correction of the Personal Data that We hold about You.** You have the right to have any incomplete or inaccurate information We hold about You corrected. - **Object to processing of Your Personal Data.** This right exists where We are relying on a legitimate interest as the legal basis for Our processing and there is something about Your particular situation, which makes You want to object to our processing of Your Personal Data on this ground. You also have the right to object where We are processing Your Personal Data for direct marketing purposes. - **Request erasure of Your Personal Data.** You have the right to ask Us to delete or remove Personal Data when there is no good reason for Us to continue processing it. - **Request the transfer of Your Personal Data.** We will provide to You, or to a third-party You have chosen, Your Personal Data in a structured, commonly used, machine-readable format. Please note that this right only applies to automated information which You initially provided consent for Us to use or where We used the information to perform a contract with You. - **Withdraw Your consent.** You have the right to withdraw Your consent on using your Personal Data. If You withdraw Your consent, We may not be able to provide You with access to certain specific functionalities of the Service. ### Exercising of Your GDPR Data Protection Rights You may exercise Your rights of access, rectification, cancellation and opposition by contacting Us. Please note that we may ask You to verify Your identity before responding to such requests. If You make a request, We will try our best to respond to You as soon as possible. You have the right to complain to a Data Protection Authority about Our collection and use of Your Personal Data. For more information, if You are in the European Economic Area (EEA), please contact Your local data protection authority in the EEA. ## CCPA/CPRA Privacy Notice This privacy notice section for California residents supplements the information contained in Our Privacy Policy and it applies solely to all visitors, users, and others who reside in the State of California. ### Categories of Personal Information Collected We collect information that identifies, relates to, describes, references, is capable of being associated with, or could reasonably be linked, directly or indirectly, with a particular Consumer or Device. The following is a list of categories of personal information which we may collect or may have been collected from California residents within the last twelve (12) months. Please note that the categories and examples provided in the list below are those defined in the CCPA/CPRA. This does not mean that all examples of that category of personal information were in fact collected by Us, but reflects our good faith belief to the best of Our knowledge that some of that information from the applicable category may be and may have been collected. For example, certain categories of personal information would only be collected if You provided such personal information directly to Us. - **Category A: Identifiers.** Examples: A real name, alias, postal address, unique personal identifier, online identifier, Internet Protocol address, email address, account name, driver's license number, passport number, or other similar identifiers. Collected: Yes. - **Category B: Personal information categories listed in the California Customer Records statute (Cal. Civ. Code § 1798.80(e)).** Examples: A name, signature, Social Security number, physical characteristics or description, address, telephone number, passport number, driver's license or state identification card number, insurance policy number, education, employment, employment history, bank account number, credit card number, debit card number, or any other financial information, medical information, or health insurance information. Some personal information included in this category may overlap with other categories. Collected: Yes. - **Category C: Protected classification characteristics under California or federal law.** Examples: Age (40 years or older), race, color, ancestry, national origin, citizenship, religion or creed, marital status, medical condition, physical or mental disability, sex (including gender, gender identity, gender expression, pregnancy or childbirth and related medical conditions), sexual orientation, veteran or military status, genetic information (including familial genetic information). Collected: No. - **Category D: Commercial information.** Examples: Records and history of products or services purchased or considered. Collected: Yes. - **Category E: Biometric information.** Examples: Genetic, physiological, behavioral, and biological characteristics, or activity patterns used to extract a template or other identifier or identifying information, such as, fingerprints, faceprints, and voiceprints, iris or retina scans, keystroke, gait, or other physical patterns, and sleep, health, or exercise data. Collected: No. - **Category F: Internet or other similar network activity.** Examples: Interaction with our Service or advertisement. Collected: Yes. - **Category G: Geolocation data.** Examples: Approximate physical location. Collected: No. - **Category H: Sensory data.** Examples: Audio, electronic, visual, thermal, olfactory, or similar information. Collected: No. - **Category I: Professional or employment-related information.** Examples: Current or past job history or performance evaluations. Collected: No. - **Category J: Non-public education information (per the Family Educational Rights and Privacy Act (20 U.S.C. Section 1232g, 34 C.F.R. Part 99)).** Examples: Education records directly related to a student maintained by an educational institution or party acting on its behalf, such as grades, transcripts, class lists, student schedules, student identification codes, student financial information, or student disciplinary records. Collected: No. - **Category K: Inferences drawn from other personal information.** Examples: Profile reflecting a person's preferences, characteristics, psychological trends, predispositions, behavior, attitudes, intelligence, abilities, and aptitudes. Collected: No. - **Category L: Sensitive personal information.** Examples: Account login and password information, geolocation data. Collected: Yes. Under CCPA/CPRA, personal information does not include: - Publicly available information from government records - Deidentified or aggregated consumer information - Information excluded from the CCPA/CPRA's scope, such as: - Health or medical information covered by the Health Insurance Portability and Accountability Act of 1996 (HIPAA) and the California Confidentiality of Medical Information Act (CMIA) or clinical trial data - Personal Information covered by certain sector-specific privacy laws, including the Fair Credit Reporting Act (FRCA), the Gramm-Leach-Bliley Act (GLBA) or California Financial Information Privacy Act (FIPA), and the Driver's Privacy Protection Act of 1994 ### Sources of Personal Information We obtain the categories of personal information listed above from the following categories of sources: - **Directly from You**. For example, from the forms You complete on our Service, preferences You express or provide through our Service, or from Your purchases on our Service. - **Indirectly from You**. For example, from observing Your activity on our Service. - **Automatically from You**. For example, through cookies We or our Service Providers set on Your Device as You navigate through our Service. - **From Service Providers**. For example, third-party vendors to monitor and analyze the use of our Service, third-party vendors for payment processing, or other third-party vendors that We use to provide the Service to You. ### Use of Personal Information We may use or disclose personal information We collect for "business purposes" or "commercial purposes" (as defined under the CCPA/CPRA), which may include the following examples: - To operate our Service and provide You with Our Service. - To provide You with support and to respond to Your inquiries, including to investigate and address Your concerns and monitor and improve our Service. - To fulfill or meet the reason You provided the information. For example, if You share Your contact information to ask a question about our Service, We will use that personal information to respond to Your inquiry. If You provide Your personal information to purchase a product or service, We will use that information to process Your payment and facilitate delivery. - To respond to law enforcement requests and as required by applicable law, court order, or governmental regulations. - As described to You when collecting Your personal information or as otherwise set forth in the CCPA/CPRA. - For internal administrative and auditing purposes. - To detect security incidents and protect against malicious, deceptive, fraudulent or illegal activity, including, when necessary, to prosecute those responsible for such activities. - Other one-time uses. Please note that the examples provided above are illustrative and not intended to be exhaustive. For more details on how we use this information, please refer to the "Use of Your Personal Data" section. If We decide to collect additional categories of personal information or use the personal information We collected for materially different, unrelated, or incompatible purposes We will update this Privacy Policy. ### Disclosure of Personal Information We may use or disclose and may have used or disclosed in the last twelve (12) months the following categories of personal information for business or commercial purposes: - Category A: Identifiers - Category B: Personal information categories listed in the California Customer Records statute (Cal. Civ. Code § 1798.80(e)) - Category D: Commercial information - Category F: Internet or other similar network activity Please note that the categories listed above are those defined in the CCPA/CPRA. This does not mean that all examples of that category of personal information were in fact disclosed, but reflects our good faith belief to the best of our knowledge that some of that information from the applicable category may be and may have been disclosed. When We disclose personal information for a business purpose or a commercial purpose, We enter a contract that describes the purpose and requires the recipient to both keep that personal information confidential and not use it for any purpose except performing the contract. ### Share of Personal Information We may share, and have shared in the last twelve (12) months, Your personal information identified in the above categories with the following categories of third parties: - Service Providers - Payment processors - Our affiliates - Our business partners - Third party vendors to whom You or Your agents authorize Us to disclose Your personal information in connection with products or services We provide to You ### Sale of Personal Information As defined in the CCPA/CPRA, "sell" and "sale" mean selling, renting, releasing, disclosing, disseminating, making available, transferring, or otherwise communicating orally, in writing, or by electronic or other means, a Consumer's personal information by the Business to a third party for valuable consideration. This means that We may have received some kind of benefit in return for sharing personal information, but not necessarily a monetary benefit. We do not sell personal information as the term sell is commonly understood. We do allow Service Providers to use Your personal information for the business purposes described in Our Privacy Policy, for activities such as advertising, marketing, and analytics, and these may be deemed a sale under CCPA/CPRA. We may sell and may have sold in the last twelve (12) months the following categories of personal information: - Category A: Identifiers - Category B: Personal information categories listed in the California Customer Records statute (Cal. Civ. Code § 1798.80(e)) - Category D: Commercial information - Category F: Internet or other similar network activity Please note that the categories listed above are those defined in the CCPA/CPRA. This does not mean that all examples of that category of personal information were in fact sold, but reflects our good faith belief to the best of Our knowledge that some of that information from the applicable category may be and may have been shared for value in return. ### Sale of Personal Information of Minors Under 16 Years of Age We do not knowingly collect personal information from minors under the age of 16 through our Service, although certain third party websites that we link to may do so. These third-party websites have their own terms of use and privacy policies and We encourage parents and legal guardians to monitor their children's Internet usage and instruct their children to never provide information on other websites without their permission. We do not sell the personal information of Consumers We actually know are less than 16 years of age, unless We receive affirmative authorization (the "right to opt-in") from either the Consumer who is between 13 and 16 years of age, or the parent or guardian of a Consumer less than 13 years of age. Consumers who opt-in to the sale of personal information may opt-out of future sales at any time. To exercise the right to opt-out, You (or Your authorized representative) may submit a request to Us by contacting Us. If You have reason to believe that a child under the age of 13 (or 16) has provided Us with personal information, please contact Us with sufficient detail to enable Us to delete that information. ### Your Rights under the CCPA/CPRA The CCPA/CPRA provides California residents with specific rights regarding their personal information. If You are a resident of California, You have the following rights: - **The right to notice.** You have the right to be notified which categories of Personal Data are being collected and the purposes for which the Personal Data is being used. - **The right to know/access.** Under CCPA/CPRA, You have the right to request that We disclose information to You about Our collection, use, sale, disclosure for business purposes and share of personal information. Once We receive and confirm Your request, We will disclose to You: - The categories of personal information We collected about You - The categories of sources for the personal information We collected about You - Our business or commercial purposes for collecting or selling that personal information - The categories of third parties with whom We share that personal information - The specific pieces of personal information We collected about You - If we sold Your personal information or disclosed Your personal information for a business purpose, We will disclose to You: - The categories of personal information categories sold - The categories of personal information categories disclosed - **The right to say no to the sale or sharing of Personal Data (opt-out).** You have the right to direct Us to not sell Your personal information. To submit an opt-out request, please see the "Do Not Sell My Personal Information" section or contact Us. - **The right to correct Personal Data.** You have the right to correct or rectify any inaccurate personal information about You that We collected. Once We receive and confirm Your request, We will use commercially reasonable efforts to correct (and direct our Service Providers to correct) Your personal information, unless an exception applies. - **The right to limit use and disclosure of sensitive Personal Data.** You have the right to request to limit the use or disclosure of certain sensitive personal information We collected about You, unless an exception applies. To submit, please see the "Limit the Use or Disclosure of My Sensitive Personal Information" section or contact Us. - **The right to delete Personal Data.** You have the right to request the deletion of Your Personal Data under certain circumstances, subject to certain exceptions. Once We receive and confirm Your request, We will delete (and direct Our Service Providers to delete) Your personal information from our records, unless an exception applies. We may deny Your deletion request if retaining the information is necessary for Us or Our Service Providers to: - Complete the transaction for which We collected the personal information, provide a good or service that You requested, take actions reasonably anticipated within the context of our ongoing business relationship with You, or otherwise perform our contract with You. - Detect security incidents, protect against malicious, deceptive, fraudulent, or illegal activity, or prosecute those responsible for such activities. - Debug products to identify and repair errors that impair existing intended functionality. - Exercise free speech, ensure the right of another consumer to exercise their free speech rights, or exercise another right provided for by law. - Comply with the California Electronic Communications Privacy Act (Cal. Penal Code § 1546 et. seq.). - Engage in public or peer-reviewed scientific, historical, or statistical research in the public interest that adheres to all other applicable ethics and privacy laws, when the information's deletion may likely render impossible or seriously impair the research's achievement, if You previously provided informed consent. - Enable solely internal uses that are reasonably aligned with consumer expectations based on Your relationship with Us. - Comply with a legal obligation. - Make other internal and lawful uses of that information that are compatible with the context in which You provided it. - **The right not to be discriminated against.** You have the right not to be discriminated against for exercising any of Your consumer's rights, including by: - Denying goods or services to You - Charging different prices or rates for goods or services, including the use of discounts or other benefits or imposing penalties - Providing a different level or quality of goods or services to You - Suggesting that You will receive a different price or rate for goods or services or a different level or quality of goods or services ### Exercising Your CCPA/CPRA Data Protection Rights Please see the "Do Not Sell My Personal Information" section and "Limit the Use or Disclosure of My Sensitive Personal Information" section for more information on how to opt out and limit the use of sensitive information collected. Additionally, in order to exercise any of Your rights under the CCPA/CPRA, and if You are a California resident, You can contact Us: - By email: [legal@altimate.ai](mailto:legal@altimate.ai) Only You, or a person registered with the California Secretary of State that You authorize to act on Your behalf, may make a verifiable request related to Your personal information. Your request to Us must: - Provide sufficient information that allows Us to reasonably verify You are the person about whom We collected personal information or an authorized representative - Describe Your request with sufficient detail that allows Us to properly understand, evaluate, and respond to it We cannot respond to Your request or provide You with the required information if We cannot: - Verify Your identity or authority to make the request - And confirm that the personal information relates to You We will disclose and deliver the required information free of charge within 45 days of receiving Your verifiable request. The time period to provide the required information may be extended once by an additional 45 days when reasonably necessary and with prior notice. Any disclosures We provide will only cover the 12-month period preceding the verifiable request's receipt. For data portability requests, We will select a format to provide Your personal information that is readily usable and should allow You to transmit the information from one entity to another entity without hindrance. ### Do Not Sell My Personal Information As defined in the CCPA/CPRA, "sell" and "sale" mean selling, renting, releasing, disclosing, disseminating, making available, transferring, or otherwise communicating orally, in writing, or by electronic or other means, a Consumer's personal information by the Business to a third party for valuable consideration. This means that We may have received some kind of benefit in return for sharing personal information, but not necessarily a monetary benefit. We do not sell personal information as the term sell is commonly understood. We do allow Service Providers to use Your personal information for the business purposes described in Our Privacy Policy, for activities such as advertising, marketing, and analytics, and these may be deemed a sale under CCPA/CPRA. You have the right to opt-out of the sale of Your personal information. Once We receive and confirm a verifiable consumer request from You, we will stop selling Your personal information. To exercise Your right to opt-out, please contact Us. The Service Providers we partner with (for example, our analytics or advertising partners) may use technology on the Service that sells personal information as defined by the CCPA/CPRA law. If you wish to opt out of the use of Your personal information for interest-based advertising purposes and these potential sales as defined under CCPA/CPRA law, you may do so by following the instructions below. Please note that any opt out is specific to the browser You use. You may need to opt out on every browser that You use. #### Website If applicable, click "Privacy Preferences", "Update Privacy Preferences" or "Do Not Sell My Personal Information" buttons listed on the Service to review your privacy preferences and opt out of cookies and other technologies that We may use. Please note that You will need to opt out from each browser that You use to access the Service. Additionally, You can opt out of receiving ads that are personalized as served by our Service Providers by following our instructions presented on the Service: - The NAI's opt-out platform: [https://www.networkadvertising.org/choices/](https://www.networkadvertising.org/choices/) - The EDAA's opt-out platform [https://www.youronlinechoices.com/](https://www.youronlinechoices.com/) - The DAA's opt-out platform: [https://optout.aboutads.info/?c=2&lang=EN](https://optout.aboutads.info/?c=2&lang=EN) The opt out will place a cookie on Your computer that is unique to the browser You use to opt out. If you change browsers or delete the cookies saved by your browser, You will need to opt out again. #### Mobile Devices Your mobile device may give You the ability to opt out of the use of information about the apps You use in order to serve You ads that are targeted to Your interests: - "Opt out of Interest-Based Ads" or "Opt out of Ads Personalization" on Android devices - "Limit Ad Tracking" on iOS devices You can also stop the collection of location information from Your mobile device by changing the preferences on Your mobile device. ### Limit the Use or Disclosure of My Sensitive Personal Information If You are a California resident, You have the right to limit the use and disclosure of Your sensitive personal information to that use which is necessary to perform the services or provide the goods reasonably expected by an average Consumer who requests such services or goods. We collect, use and disclose sensitive personal information in ways that are necessary to provide the Service. For more information on how We use Your personal information, please see the "Use of Your Personal Data" section or contact us. ## "Do Not Track" Policy as Required by California Online Privacy Protection Act (CalOPPA) Our Service does not respond to Do Not Track signals. However, some third party websites do keep track of Your browsing activities. If You are visiting such websites, You can set Your preferences in Your web browser to inform websites that You do not want to be tracked. You can enable or disable DNT by visiting the preferences or settings page of Your web browser. ## Your California Privacy Rights (California's Shine the Light law) Under California Civil Code Section 1798 (California's Shine the Light law), California residents with an established business relationship with us can request information once a year about sharing their Personal Data with third parties for the third parties' direct marketing purposes. If you'd like to request more information under the California Shine the Light law, and if You are a California resident, You can contact Us using the contact information provided below. ## California Privacy Rights for Minor Users (California Business and Professions Code Section 22581) California Business and Professions Code Section 22581 allows California residents under the age of 18 who are registered users of online sites, services or applications to request and obtain removal of content or information they have publicly posted. To request removal of such data, and if You are a California resident, You can contact Us using the contact information provided below, and include the email address associated with Your account. Be aware that Your request does not guarantee complete or comprehensive removal of content or information posted online and that the law may not permit or require removal in certain circumstances. ## Children's Privacy Our Service does not address anyone under the age of 13. We do not knowingly collect personally identifiable information from anyone under the age of 13. If You are a parent or guardian and You are aware that Your child has provided Us with Personal Data, please contact Us. If We become aware that We have collected Personal Data from anyone under the age of 13 without verification of parental consent, We take steps to remove that information from Our servers. If We need to rely on consent as a legal basis for processing Your information and Your country requires consent from a parent, We may require Your parent's consent before We collect and use that information. ## Links to Other Websites Our Service may contain links to other websites that are not operated by Us. If You click on a third party link, You will be directed to that third party's site. We strongly advise You to review the Privacy Policy of every site You visit. We have no control over and assume no responsibility for the content, privacy policies or practices of any third party sites or services. ## Changes to this Privacy Policy We may update Our Privacy Policy from time to time. We will notify You of any changes by posting the new Privacy Policy on this page. We will let You know via email and/or a prominent notice on Our Service, prior to the change becoming effective and update the "Last updated" date at the top of this Privacy Policy. You are advised to review this Privacy Policy periodically for any changes. Changes to this Privacy Policy are effective when they are posted on this page. ## Contact Us If you have any questions about this Privacy Policy, You can contact us: - By email: [legal@altimate.ai](mailto:legal@altimate.ai) --- # Terms & Conditions URL: https://altimate.ai/terms Read the terms and conditions for Altimate AI products. They cover orders, subscriptions, billing, refunds, free trials, user accounts, and content rules. Legal ## Altimate Platform: Terms and Conditions Last updated: November 06, 2023 Please read these terms and conditions carefully before using Our Service. ## Interpretation and Definitions ### Interpretation The words of which the initial letter is capitalized have meanings defined under the following conditions. The following definitions shall have the same meaning regardless of whether they appear in singular or in plural. ### Definitions For the purposes of these Terms and Conditions: - **Application** means the software program provided by the Company downloaded by You on any electronic device, named DataPilot - **Application Store** means the digital distribution service operated and developed by Apple Inc. (Apple App Store) or Google Inc. (Google Play Store) in which the Application has been downloaded. - **Affiliate** means an entity that controls, is controlled by or is under common control with a party, where "control" means ownership of 50% or more of the shares, equity interest or other securities entitled to vote for election of directors or other managing authority. - **Account** means a unique account created for You to access our Service or parts of our Service. - **Country** refers to: California, United States - **Company** (referred to as either "the Company", "We", "Us" or "Our" in this Agreement) refers to Altimate Inc., 440 N Wolfe Road #150 Sunnyvale CA 94085 Phone: +1 (650) 254-6266. Email: [legal@altimate.ai](mailto:legal@altimate.ai). - **Content** refers to content such as text, images, or other information that can be posted, uploaded, linked to or otherwise made available by You, regardless of the form of that content. - **Device** means any device that can access the Service such as a computer, a cellphone or a digital tablet. - **Feedback** means feedback, innovations or suggestions sent by You regarding the attributes, performance or features of our Service. - **Free Trial** refers to a limited period of time that may be free when purchasing a Subscription. - **Goods** refer to the items offered for sale on the Service. - **In-app Purchase** refers to the purchase of a product, item, service or Subscription made through the Application and subject to these Terms and Conditions and/or the Application Store's own terms and conditions. - **Orders** mean a request by You to purchase Goods from Us. - **Service** refers to the Application or the Website or both. - **Subscriptions** refer to the services or access to the Service offered on a subscription basis by the Company to You. - **Terms and Conditions** (also referred as "Terms") mean these Terms and Conditions that form the entire agreement between You and the Company regarding the use of the Service. - **Third-party Social Media Service** means any services or content (including data, information, products or services) provided by a third-party that may be displayed, included or made available by the Service. - **Website** refers to Altimate AI, accessible from [https://altimate.ai](https://altimate.ai) - **You** means the individual accessing or using the Service, or the company, or other legal entity on behalf of which such individual is accessing or using the Service, as applicable. ## Acknowledgment These are the Terms and Conditions governing the use of this Service and the agreement that operates between You and the Company. These Terms and Conditions set out the rights and obligations of all users regarding the use of the Service. Your access to and use of the Service is conditioned on Your acceptance of and compliance with these Terms and Conditions. These Terms and Conditions apply to all visitors, users and others who access or use the Service. By accessing or using the Service You agree to be bound by these Terms and Conditions. If You disagree with any part of these Terms and Conditions then You may not access the Service. You represent that you are over the age of 18. The Company does not permit those under 18 to use the Service. Your access to and use of the Service is also conditioned on Your acceptance of and compliance with the Privacy Policy of the Company. Our Privacy Policy describes Our policies and procedures on the collection, use and disclosure of Your personal information when You use the Application or the Website and tells You about Your privacy rights and how the law protects You. Please read Our Privacy Policy carefully before using Our Service. ## Placing Orders for Goods By placing an Order for Goods through the Service, You warrant that You are legally capable of entering into binding contracts. ### Your Information If You wish to place an Order for Goods available on the Service, You may be asked to supply certain information relevant to Your Order including, without limitation, Your name, Your email, Your phone number, Your credit card number, the expiration date of Your credit card, Your billing address, and Your shipping information. You represent and warrant that: (i) You have the legal right to use any credit or debit card(s) or other payment method(s) in connection with any Order; and that (ii) the information You supply to us is true, correct and complete. By submitting such information, You grant us the right to provide the information to payment processing third parties for purposes of facilitating the completion of Your Order. ### Order Cancellation We reserve the right to refuse or cancel Your Order at any time for certain reasons including but not limited to: - Goods availability - Errors in the description or prices for Goods - Errors in Your Order We reserve the right to refuse or cancel Your Order if fraud or an unauthorized or illegal transaction is suspected. #### Your Order Cancellation Rights Any Goods you purchase can only be returned in accordance with these Terms and Conditions and Our Returns Policy. Our Returns Policy forms a part of these Terms and Conditions. Please read our Returns Policy to learn more about your right to cancel Your Order. Your right to cancel an Order only applies to Goods that are returned in the same condition as You received them. You should also include all of the product's instructions, documents and wrappings. Goods that are damaged or not in the same condition as You received them or which are worn simply beyond opening the original packaging will not be refunded. You should therefore take reasonable care of the purchased Goods while they are in Your possession. We will reimburse You no later than 14 days from the day on which We receive the returned Goods. We will use the same means of payment as You used for the Order, and You will not incur any fees for such reimbursement. You will not have any right to cancel an Order for the supply of any of the following Goods: - The supply of Goods made to Your specifications or clearly personalized. - The supply of Goods which according to their nature are not suitable to be returned, deteriorate rapidly or where the date of expiry is over. - The supply of Goods which are not suitable for return due to health protection or hygiene reasons and were unsealed after delivery. - The supply of Goods which are, after delivery, according to their nature, inseparably mixed with other items. - The supply of digital content which is not supplied on a tangible medium if the performance has begun with Your prior express consent and You have acknowledged Your loss of cancellation right. ### Availability, Errors and Inaccuracies We are constantly updating Our offerings of Goods on the Service. The Goods available on Our Service may be mispriced, described inaccurately, or unavailable, and We may experience delays in updating information regarding our Goods on the Service and in Our advertising on other websites. We cannot and do not guarantee the accuracy or completeness of any information, including prices, product images, specifications, availability, and services. We reserve the right to change or update information and to correct errors, inaccuracies, or omissions at any time without prior notice. ### Prices Policy The Company reserves the right to revise its prices at any time prior to accepting an Order. The prices quoted may be revised by the Company subsequent to accepting an Order in the event of any occurrence affecting delivery caused by government action, variation in customs duties, increased shipping charges, higher foreign exchange costs and any other matter beyond the control of the Company. In that event, You will have the right to cancel Your Order. ### Payments All Goods purchased are subject to a one-time payment. Payment can be made through various payment methods we have available, such as Visa, MasterCard, Affinity Card, American Express cards or online payment methods (PayPal, for example). Payment cards (credit cards or debit cards) are subject to validation checks and authorization by Your card issuer. If we do not receive the required authorization, We will not be liable for any delay or non-delivery of Your Order. ## Subscriptions ### Subscription period The Service or some parts of the Service are available only with a paid Subscription. You will be billed in advance on a recurring and periodic basis (such as daily, weekly, monthly or annually), depending on the type of Subscription plan you select when purchasing the Subscription. At the end of each period, Your Subscription will automatically renew under the exact same conditions unless You cancel it or the Company cancels it. ### Subscription cancellations You may cancel Your Subscription renewal either through Your Account settings page or by contacting the Company. You will not receive a refund for the fees You already paid for Your current Subscription period and You will be able to access the Service until the end of Your current Subscription period. If the Subscription has been made through an In-app Purchase, You can cancel the renewal of Your Subscription with the Application Store. ### Billing You shall provide the Company with accurate and complete billing information including full name, address, state, zip code, telephone number, and a valid payment method information. Should automatic billing fail to occur for any reason, the Company will issue an electronic invoice indicating that you must proceed manually, within a certain deadline date, with the full payment corresponding to the billing period as indicated on the invoice. If the Subscription has been made through an In-app Purchase, all billing is handled by the Application Store and is governed by the Application Store's own terms and conditions. ### Fee Changes The Company, in its sole discretion and at any time, may modify the Subscription fees. Any Subscription fee change will become effective at the end of the then-current Subscription period. The Company will provide You with reasonable prior notice of any change in Subscription fees to give You an opportunity to terminate Your Subscription before such change becomes effective. Your continued use of the Service after the Subscription fee change comes into effect constitutes Your agreement to pay the modified Subscription fee amount. ### Refunds Except when required by law, paid Subscription fees are non-refundable. Certain refund requests for Subscriptions may be considered by the Company on a case-by-case basis and granted at the sole discretion of the Company. If the Subscription has been made through an In-app purchase, the Application Store's refund policy will apply. If You wish to request a refund, You may do so by contacting the Application Store directly. ### Free Trial The Company may, at its sole discretion, offer a Subscription with a Free Trial for a limited period of time. You may be required to enter Your billing information in order to sign up for the Free Trial. If You do enter Your billing information when signing up for a Free Trial, You will not be charged by the Company until the Free Trial has expired. On the last day of the Free Trial period, unless You canceled Your Subscription, You will be automatically charged the applicable Subscription fees for the type of Subscription You have selected. At any time and without notice, the Company reserves the right to (i) modify the terms and conditions of the Free Trial offer, or (ii) cancel such Free Trial offer. ## In-app Purchases The Application may include In-app Purchases that allow you to buy products, services or Subscriptions. More information about how you may be able to manage In-app Purchases using your Device may be set out in the Application Store's own terms and conditions or in your Device's Help settings. In-app Purchases can only be consumed within the Application. If you make a In-app Purchase, that In-app Purchase cannot be cancelled after you have initiated its download. In-app Purchases cannot be redeemed for cash or other consideration or otherwise transferred. If any In-app Purchase is not successfully downloaded or does not work once it has been successfully downloaded, we will, after becoming aware of the fault or being notified to the fault by You, investigate the reason for the fault. We will act reasonably in deciding whether to provide You with a replacement In-app Purchase or issue You with a patch to repair the fault. In no event will We charge You to replace or repair the In-app Purchase. In the unlikely event that we are unable to replace or repair the relevant In-app Purchase or are unable to do so within a reasonable period of time and without significant inconvenience to You, We will authorize the Application Store to refund You an amount up to the cost of the relevant In-app Purchase. Alternatively, if You wish to request a refund, You may do so by contacting the Application Store directly. You acknowledge and agree that all billing and transaction processes are handled by the Application Store from where you downloaded the Application and are governed by that Application Store's own terms and conditions. If you have any payment related issues with In-app Purchases, then you need to contact the Application Store directly. ## User Accounts When You create an account with Us, You must provide Us information that is accurate, complete, and current at all times. Failure to do so constitutes a breach of the Terms, which may result in immediate termination of Your account on Our Service. You are responsible for safeguarding the password that You use to access the Service and for any activities or actions under Your password, whether Your password is with Our Service or a Third-Party Social Media Service. You agree not to disclose Your password to any third party. You must notify Us immediately upon becoming aware of any breach of security or unauthorized use of Your account. You may not use as a username the name of another person or entity or that is not lawfully available for use, a name or trademark that is subject to any rights of another person or entity other than You without appropriate authorization, or a name that is otherwise offensive, vulgar or obscene. ## Content ### Your Right to Post Content Our Service allows You to post Content. You are responsible for the Content that You post to the Service, including its legality, reliability, and appropriateness. By posting Content to the Service, You grant Us the right and license to use, modify, publicly perform, publicly display, reproduce, and distribute such Content on and through the Service. You retain any and all of Your rights to any Content You submit, post or display on or through the Service and You are responsible for protecting those rights. You agree that this license includes the right for Us to make Your Content available to other users of the Service, who may also use Your Content subject to these Terms. You represent and warrant that: (i) the Content is Yours (You own it) or You have the right to use it and grant Us the rights and license as provided in these Terms, and (ii) the posting of Your Content on or through the Service does not violate the privacy rights, publicity rights, copyrights, contract rights or any other rights of any person. ### Content Restrictions The Company is not responsible for the content of the Service's users. You expressly understand and agree that You are solely responsible for the Content and for all activity that occurs under your account, whether done so by You or any third person using Your account. You may not transmit any Content that is unlawful, offensive, upsetting, intended to disgust, threatening, libelous, defamatory, obscene or otherwise objectionable. Examples of such objectionable Content include, but are not limited to, the following: - Unlawful or promoting unlawful activity. - Defamatory, discriminatory, or mean-spirited content, including references or commentary about religion, race, sexual orientation, gender, national/ethnic origin, or other targeted groups. - Spam, machine – or randomly – generated, constituting unauthorized or unsolicited advertising, chain letters, any other form of unauthorized solicitation, or any form of lottery or gambling. - Containing or installing any viruses, worms, malware, trojan horses, or other content that is designed or intended to disrupt, damage, or limit the functioning of any software, hardware or telecommunications equipment or to damage or obtain unauthorized access to any data or other information of a third person. - Infringing on any proprietary rights of any party, including patent, trademark, trade secret, copyright, right of publicity or other rights. - Impersonating any person or entity including the Company and its employees or representatives. - Violating the privacy of any third person. - False information and features. The Company reserves the right, but not the obligation, to, in its sole discretion, determine whether or not any Content is appropriate and complies with these Terms, refuse or remove this Content. The Company further reserves the right to make formatting and edits and change the manner of any Content. The Company can also limit or revoke the use of the Service if You post such objectionable Content. As the Company cannot control all content posted by users and/or third parties on the Service, you agree to use the Service at your own risk. You understand that by using the Service You may be exposed to content that You may find offensive, indecent, incorrect or objectionable, and You agree that under no circumstances will the Company be liable in any way for any content, including any errors or omissions in any content, or any loss or damage of any kind incurred as a result of your use of any content. ### Content Backups Although regular backups of Content are performed, the Company does not guarantee there will be no loss or corruption of data. Corrupt or invalid backup points may be caused by, without limitation, Content that is corrupted prior to being backed up or that changes during the time a backup is performed. The Company will provide support and attempt to troubleshoot any known or discovered issues that may affect the backups of Content. But You acknowledge that the Company has no liability related to the integrity of Content or the failure to successfully restore Content to a usable state. You agree to maintain a complete and accurate copy of any Content in a location independent of the Service. ## Copyright Policy ### Intellectual Property Infringement We respect the intellectual property rights of others. It is Our policy to respond to any claim that Content posted on the Service infringes a copyright or other intellectual property infringement of any person. If You are a copyright owner, or authorized on behalf of one, and You believe that the copyrighted work has been copied in a way that constitutes copyright infringement that is taking place through the Service, You must submit Your notice in writing to the attention of our copyright agent via email at [legal@altimate.ai](mailto:legal@altimate.ai) and include in Your notice a detailed description of the alleged infringement. You may be held accountable for damages (including costs and attorneys' fees) for misrepresenting that any Content is infringing Your copyright. ### DMCA Notice and DMCA Procedure for Copyright Infringement Claims You may submit a notification pursuant to the Digital Millennium Copyright Act (DMCA) by providing our Copyright Agent with the following information in writing (see 17 U.S.C 512(c)(3) for further detail): - An electronic or physical signature of the person authorized to act on behalf of the owner of the copyright's interest. - A description of the copyrighted work that You claim has been infringed, including the URL (i.e., web page address) of the location where the copyrighted work exists or a copy of the copyrighted work. - Identification of the URL or other specific location on the Service where the material that You claim is infringing is located. - Your address, telephone number, and email address. - A statement by You that You have a good faith belief that the disputed use is not authorized by the copyright owner, its agent, or the law. - A statement by You, made under penalty of perjury, that the above information in Your notice is accurate and that You are the copyright owner or authorized to act on the copyright owner's behalf. You can contact our copyright agent via email at [legal@altimate.ai](mailto:legal@altimate.ai). Upon receipt of a notification, the Company will take whatever action, in its sole discretion, it deems appropriate, including removal of the challenged content from the Service. ## Intellectual Property The Service and its original content (excluding Content provided by You or other users), features and functionality are and will remain the exclusive property of the Company and its licensors. The Service is protected by copyright, trademark, and other laws of both the Country and foreign countries. Our trademarks and trade dress may not be used in connection with any product or service without the prior written consent of the Company. ## Your Feedback to Us You assign all rights, title and interest in any Feedback You provide the Company. If for any reason such assignment is ineffective, You agree to grant the Company a non-exclusive, perpetual, irrevocable, royalty free, worldwide right and license to use, reproduce, disclose, sub-license, distribute, modify and exploit such Feedback without restriction. ## Links to Other Websites Our Service may contain links to third-party web sites or services that are not owned or controlled by the Company. The Company has no control over, and assumes no responsibility for, the content, privacy policies, or practices of any third party web sites or services. You further acknowledge and agree that the Company shall not be responsible or liable, directly or indirectly, for any damage or loss caused or alleged to be caused by or in connection with the use of or reliance on any such content, goods or services available on or through any such web sites or services. We strongly advise You to read the terms and conditions and privacy policies of any third-party web sites or services that You visit. ## Termination We may terminate or suspend Your Account immediately, without prior notice or liability, for any reason whatsoever, including without limitation if You breach these Terms and Conditions. Upon termination, Your right to use the Service will cease immediately. If You wish to terminate Your Account, You may simply discontinue using the Service. ## Limitation of Liability Notwithstanding any damages that You might incur, the entire liability of the Company and any of its suppliers under any provision of this Terms and Your exclusive remedy for all of the foregoing shall be limited to the amount actually paid by You through the Service or 100 USD if You haven't purchased anything through the Service. To the maximum extent permitted by applicable law, in no event shall the Company or its suppliers be liable for any special, incidental, indirect, or consequential damages whatsoever (including, but not limited to, damages for loss of profits, loss of data or other information, for business interruption, for personal injury, loss of privacy arising out of or in any way related to the use of or inability to use the Service, third-party software and/or third-party hardware used with the Service, or otherwise in connection with any provision of this Terms), even if the Company or any supplier has been advised of the possibility of such damages and even if the remedy fails of its essential purpose. Some states do not allow the exclusion of implied warranties or limitation of liability for incidental or consequential damages, which means that some of the above limitations may not apply. In these states, each party's liability will be limited to the greatest extent permitted by law. ## "AS IS" and "AS AVAILABLE" Disclaimer The Service is provided to You "AS IS" and "AS AVAILABLE" and with all faults and defects without warranty of any kind. To the maximum extent permitted under applicable law, the Company, on its own behalf and on behalf of its Affiliates and its and their respective licensors and service providers, expressly disclaims all warranties, whether express, implied, statutory or otherwise, with respect to the Service, including all implied warranties of merchantability, fitness for a particular purpose, title and non-infringement, and warranties that may arise out of course of dealing, course of performance, usage or trade practice. Without limitation to the foregoing, the Company provides no warranty or undertaking, and makes no representation of any kind that the Service will meet Your requirements, achieve any intended results, be compatible or work with any other software, applications, systems or services, operate without interruption, meet any performance or reliability standards or be error free or that any errors or defects can or will be corrected. Without limiting the foregoing, neither the Company nor any of the company's provider makes any representation or warranty of any kind, express or implied: (i) as to the operation or availability of the Service, or the information, content, and materials or products included thereon; (ii) that the Service will be uninterrupted or error-free; (iii) as to the accuracy, reliability, or currency of any information or content provided through the Service; or (iv) that the Service, its servers, the content, or e-mails sent from or on behalf of the Company are free of viruses, scripts, trojan horses, worms, malware, timebombs or other harmful components. Some jurisdictions do not allow the exclusion of certain types of warranties or limitations on applicable statutory rights of a consumer, so some or all of the above exclusions and limitations may not apply to You. But in such a case the exclusions and limitations set forth in this section shall be applied to the greatest extent enforceable under applicable law. ## Governing Law The laws of the Country, excluding its conflicts of law rules, shall govern this Terms and Your use of the Service. Your use of the Application may also be subject to other local, state, national, or international laws. ## Disputes Resolution If You have any concern or dispute about the Service, You agree to first try to resolve the dispute informally by contacting the Company. ## For European Union (EU) Users If You are a European Union consumer, you will benefit from any mandatory provisions of the law of the country in which you are resident in. ## United States Federal Government End Use Provisions If You are a U.S. federal government end user, our Service is a "Commercial Item" as that term is defined at 48 C.F.R. §2.101. ## United States Legal Compliance You represent and warrant that (i) You are not located in a country that is subject to the United States government embargo, or that has been designated by the United States government as a "terrorist supporting" country, and (ii) You are not listed on any United States government list of prohibited or restricted parties. ## Severability and Waiver ### Severability If any provision of these Terms is held to be unenforceable or invalid, such provision will be changed and interpreted to accomplish the objectives of such provision to the greatest extent possible under applicable law and the remaining provisions will continue in full force and effect. ### Waiver Except as provided herein, the failure to exercise a right or to require performance of an obligation under these Terms shall not effect a party's ability to exercise such right or require such performance at any time thereafter nor shall the waiver of a breach constitute a waiver of any subsequent breach. ## Translation Interpretation These Terms and Conditions may have been translated if We have made them available to You on our Service. You agree that the original English text shall prevail in the case of a dispute. ## Changes to These Terms and Conditions We reserve the right, at Our sole discretion, to modify or replace these Terms at any time. If a revision is material We will make reasonable efforts to provide at least 30 days' notice prior to any new terms taking effect. What constitutes a material change will be determined at Our sole discretion. By continuing to access or use Our Service after those revisions become effective, You agree to be bound by the revised terms. If You do not agree to the new terms, in whole or in part, please stop using the website and the Service. ## Contact Us If you have any questions about these Terms and Conditions, You can contact us: - By email: [legal@altimate.ai](mailto:legal@altimate.ai) --- # Altimate on Snowflake URL: https://altimate.ai/altimate-on-snowflake Discover how Altimate helps you manage your Snowflake instances with AI-powered cost optimization, query analysis, and autonomous operations. ## Altimate logo Altimate on Snowflake AI-powered cost optimization, query analysis, and autonomous operations for your Snowflake environment. Cut costs by 30% while improving performance. Platform Features Hidden Cost Leaks Feature Deep-Dives Quick Demos ## Optimize for Cost and Performance [Altimate cost savings showcase](https://app.supademo.com/showcase/cmcwgwwob01m94s0ixjvi2a88?demo=1&step=1) ### Snowflake compute, managed autonomously. Just switch on Altimate and immediately realize 10% - 20% savings on your Snowflake compute bills with no human work required. Altimate goes far beyond other solutions, automatically adjusting warehouse sizing, autoscaling, and suspension settings thousands of times per day to perfectly match workload requirements. Autonomous warehouse sizing and suspension optimization dashboard ### Make platform cost savings a team sport. Altimate continuously identifies opportunities for savings and efficiency in your Snowflake environment, and provides tailored views to each of your internal teams. With just a click, an opportunity can be assigned to the right person with exact change instructions generated by the AI, and followed up until it's done. Team cost savings assignment and tracking interface ### Workload-level understanding. Plenty of solutions track cost, efficiency, and performance at the virtual warehouse level, or at the level of an individual query. Altimate looks deeper, with an understanding of entire query workloads, and correlates workload run metrics with a multitude of other events — warehouse configuration changes, code changes, task failures — pinpointing the exact causes of shifts in performance and cost of your workloads over time. Workload-level performance and cost analysis dashboard ### The hidden cost of your tables, exposed. The true cost of a table goes far beyond its size on disk. Altimate digs below the surface to uncover the real cost of a table, including the compute cost of the pipelines that build it, and Snowflake's hidden 'failsafe' storage that can balloon under certain usage patterns. Then it goes a step further — is the table ever read? Can it be dropped and the pipeline jobs stopped entirely? Hidden table cost breakdown showing storage, compute, and failsafe costs ### Expert query advice, with just a click. Altimate analyzes every query along dozens of dimensions, applying deep knowledge of cost and performance improvement techniques to identify where queries can be improved, delivering precise change instructions to your platform team and Snowflake consumers. Quickly target and refactor the most frequent and most expensive queries running in your environment. AI-generated query optimization recommendations ### Needles from haystacks, in seconds. Altimate dives into individual queries, quickly parsing and understanding thousands of lines of SQL code, calling out the few lines that matter and making recommendations on exactly what should be changed to tune and optimize the query. SQL code analysis highlighting key lines for optimization ### Prevent costly mistakes during development. Altimate works side-by-side with your SQL developers, understanding the queries they are building, detecting anti-patterns and guiding on best practices as queries are being written. Immediate feedback is provided in the IDEs your team is already using, as well as within your existing Git workflows and CI/CD pipelines. IDE integration showing real-time SQL anti-pattern detection ## Deep-Dive Videos [5 Hidden Snowflake Cost Leaks Only AI Can Find](https://www.youtube-nocookie.com/embed/Fn5yLAoeR3Y) ### 5 Hidden Snowflake Cost Leaks Only AI Can Find The comprehensive story of what Altimate does and why it matters for your Snowflake operations. [AI Services Cost Breakdown: Finally Understand Where Your Cortex Spend Goes](https://www.youtube-nocookie.com/embed/ILmjhm13PS8) ### AI Services Cost Breakdown: Finally Understand Where Your Cortex Spend Goes See how Altimate breaks down opaque Snowflake AI Services costs into actionable line items. ## Quick Demos Real Snowflake problems solved by AI agents in seconds. [Finding Snowflake Cost Spikes in Seconds with AI Agents](https://www.youtube-nocookie.com/embed/cZLSM5N2gB0) ### Finding Snowflake Cost Spikes in Seconds with AI Agents See how a single question to an AI agent instantly surfaces where your Snowflake spend is going. [Stop Wasting Compute on Failed Queries with AI Analysis](https://www.youtube-nocookie.com/embed/do3uqbLtgC0) ### Stop Wasting Compute on Failed Queries with AI Analysis AI analysis prioritizes failed queries by business impact so your team fixes what matters first. [Dashboard Went Slow? How AI Found the Root Cause in Minutes](https://www.youtube-nocookie.com/embed/F_bV3K5q0JQ) ### Dashboard Went Slow? How AI Found the Root Cause in Minutes Platform-level intelligence traces performance degradation across history, indexes, and dependencies. [2-Hour Pipeline? AI Found the Bottleneck in 10 Seconds](https://www.youtube-nocookie.com/embed/G8a7YbKaH1w) ### 2-Hour Pipeline? AI Found the Bottleneck in 10 Seconds AI understands data flow end-to-end to pinpoint exactly where your pipeline is losing time. ## Start Optimizing Your Snowflake Today Join hundreds of data teams already saving on Snowflake costs with Altimate AI. [Book a Demo](https://calendly.com/d/cxsb-8h7-5z9/altimate-ai-for-snowflake-overview) [Start for Free](https://app.myaltimate.com/register) See Altimate AI Teammates in Action! [Book a Product Demo](https://calendly.com/d/cxsb-8h7-5z9/altimate-ai-for-snowflake-overview) --- # Brand Kit URL: https://altimate.ai/brand Download official Altimate AI logos, including icon-only and white versions, with the brand colors and the rules for placing and linking the logo. Brand Kit ## Altimate Brand Assets By using our brand assets, you accept our usage guidelines. Violations result in termination of permission to use our assets. Logos ## Official Logo Files Available in SVG and PNG. Use the white variants on dark backgrounds. Altimate AI Primary Logo Primary Logo [↓ SVG](https://altimate.ai/brand/svg/primary-logo.svg) [↓ PNG](https://altimate.ai/brand/png/primary-logo.png) Altimate AI Icon Only Icon Only [↓ SVG](https://altimate.ai/brand/svg/altimate-icon-only.svg) [↓ PNG](https://altimate.ai/brand/png/altimate-icon-only.png) Altimate AI White Logo White Logo [↓ SVG](https://altimate.ai/brand/svg/primary-logo-white.svg) [↓ PNG](https://altimate.ai/brand/png/primary-logo-white.png) Altimate AI White Icon White Icon [↓ SVG](https://altimate.ai/brand/svg/altimate-icon-white.svg) [↓ PNG](https://altimate.ai/brand/png/altimate-icon-white.png) Altimate AI White Logo (70% Opacity) White Logo (70% Opacity) [↓ SVG](https://altimate.ai/brand/svg/logo-white-70-opacity.svg) [↓ PNG](https://altimate.ai/brand/png/logo-white-70-opacity.png) Color Palette ## Brand Colors Click any color to copy its hex code. Orange #F07030 Primary accent — CTAs, highlights Background #09090B Page background Panel #141418 Cards, elevated surfaces Surface #111113 Alternate section background Teal #00CCAA Secondary accent White #FAFAFA Primary text Muted #C8C8D0 Secondary text Dim #9090A0 Labels and meta Body #DDDDDD Body text Guidelines ## Usage Guidelines ### Do - ✓ Use the official logo files provided on this page without modification. - ✓ Maintain adequate clear space around the logo — at minimum the height of the logomark on all sides. - ✓ Use the reversed (white) logo on dark backgrounds where the primary logo lacks contrast. - ✓ Link the logo back to altimate.ai when used digitally. ### Don't - ✕ Alter, rotate, or distort the logo in any way. - ✕ Change the logo colors or apply effects such as shadows or gradients. - ✕ Use the logo at sizes that make it illegible or difficult to recognize. - ✕ Place the logo on busy backgrounds that reduce its visibility. - ✕ Use our brand assets in a way that implies sponsorship or endorsement without prior written consent. - ✕ Use our brand assets in a way that harms our reputation or misleads consumers. Attribution When referencing Altimate AI in text, always capitalize "Altimate" and write "AI" in uppercase. The correct full name is "Altimate AI" — not "altimate", "ALTIMATE", or "Altimate.ai". --- # Support URL: https://altimate.ai/support Get help with Altimate AI products. Contact our support team for issues, feature requests, or questions about Altimate MCP and Power User for dbt. SUPPORT ## Get in Touch With Us for Issues or Feature Requests Community & Pro Enterprise Power User for dbt Slack support Power User for dbt Slack Get help from users and maintainers of "Power User for dbt" extension in the dbtTM Slack community. Get Help from the Community [Join](https://getdbt.slack.com/archives/C05KPDGRMDW) Altimate Code Slack support Altimate Code Slack Get help from users and maintainers in the Altimate Code Slack community. Get Help from the Community [Join](https://altimate.studio/join-agentic-data-engineering-slack) Github support Github Interested in reporting a bug or suggesting a feature? Join Us on GitHub. Create a Github Issue [Create](https://github.com/AltimateAI/vscode-dbt-power-user/issues) --- # Tutorials URL: https://altimate.ai/videos Watch tutorials and video guides for Altimate AI products. Explore popular use cases to accelerate your data engineering and dbt development. Tutorial ## Explore popular use cases to accelerate your development ## Tutorials and walkthroughs All development query explanation dbt model update gen AI data marts dbt documentation dbt models testing preview query results sql validator column lineage project health check collaboration defer to prod governance bookmark share queries notebooks ad-hoc query datapilot code visibility query store Query Store for dbt Query Store for SQL dbt power user extension dbt cloud CI/CD VSCode dbt tests query translation dbt Cloud tutorial\_31 dbt · dbt Cloud · dbt models · ad-hoc query · datapilot ### Simplify Ad Hoc Analysis in dbt with Data Visualization Discover how to streamline ad hoc analysis with DBT and Python, solving common challenges like working with raw data, unnesting JSON arrays, and creating clear visualizations. This video shows you how to transform complex datasets into actionable insights quickly and efficiently, making data analysis more intuitive and impactful. Perfect for tackling real-world problems and improving your workflow! Watch → tutorial\_30 dbt · dbt Cloud · dbt models · dbt tests · VSCode · datapilot ### Use dbt Cloud in VSCode with the Power User Extension This video show steps to enable development for dbt Cloud projects in VSCode with the help of Power User extension. Watch → tutorial\_29 dbt · dbt models · query explanation · datapilot · dbt tests ### SQL Query / dbt Model explanation Query explanation is invaluable to understanding a complex piece of dbt or SQL code (especially written by others!). You can ask follow-up questions in the chat window, like - "Explain this function with some sample data" as well. Watch → tutorial\_28 dbt · dbt models · datapilot · dbt tests ### Update dbt Models using DataPilot Many times, you need to add additional columns or change transformation in your dbt Model. Just ask for the changes in natural language, and DataPilot can generate the dbt model update code in VSCode itself. Watch → tutorial\_27 query translation · dbt · dbt models · datapilot ### Query translation demo Do you want to translate SQL queries from one dialect (SQL Server) to another (Snowflake)? Look no further; this video will show you how to do it easily. There are 15 different SQL dialects supported, and the DataPilot provides a translation explanation. Watch → tutorial\_26 dbt · dbt tests · dbt models · datapilot ### Generate and edit dbt tests Generate code for your dbt tests, and edit and delete those tests quickly in VSCode UI without touching the YAML files in the dbt project. Watch → tutorial\_25 column lineage · dbt · dbt models ### dbt Column Lineage Use dbt Column Lineage to understand the impact of your changes and troubleshoot dbt Models. Watch → tutorial\_24 defer to prod · datapilot · dbt · dbt models · VSCode ### defer to prod in dbt with VSCode Deferring to prod functionality in dbt saves data teams a lot of time and money. The Power User extension makes it easy to use this functionality in local and SaaS modes. Watch → tutorial\_23 dbt · dbt models · dbt documentation · datapilot ### dbt Documentation Generation Generate descriptions for your dbt models and columns in different languages and as per personas. Watch → tutorial\_22 dbt documentation · dbt · dbt models · collaboration · datapilot ### Collaborate on dbt documentation and code via IDE and UI Collab functionality enables you to discuss code and documentation easily with the stakeholders via VSCode and UI. Many stakeholders are not comfortable using IDE directly; this functionality enables them also to have a discussion with technical users. Watch → tutorial\_21 dbt · dbt models · CI/CD · datapilot ### Check dbt best practices in IDE, Git, CI/CD Project governance functionality lets you swiftly scan all dbt projects against dbt best practices and organizational guidelines in IDE, Git, CI/CD. You can also create different check configurations based on projects, environments, and teams. Watch → tutorial\_20 column lineage · dbt models · datapilot · dbt documentation ### DataPilot - View dbt Documentation and Column Lineage with transformations in SaaS UI Now, you can view all your dbt documentation and column lineage with transformation info in DataPilot SaaS UI. Watch → 1 2 3 GET STARTED ## Ready to Get Started? You are only few clicks away from experiencing your own autopilot for data. What are you waiting for?? [Get Started for Free](https://app.myaltimate.com/register) --- # Altimate Code Benchmarks: ADE-Bench and DAB Bench Results URL: https://altimate.ai/benchmarks altimate-code scores 78.0% on ADE-Bench and 71.7% Pass@1 on DAB Bench, published across Snowflake, DuckDB, PostgreSQL, MongoDB, and SQLite. ## Benchmark Results altimate-code publishes a score on every agentic data benchmark it has entered, across five database platforms, beating agents on more expensive models. ## Benchmarked Across the Whole Data Stack Platform coverage Snowflake DuckDB PostgreSQL MongoDB SQLite Tested and top-ranked across five database systems. Most data agents are built for a single platform. altimate-code is evaluated on all of them, with every score published. ADE-Bench results As of June 2026 altimate-code 78% DeepSeek V4 Pro Cortex Code CLI 65% Opus 4.6 dbt Labs 59% Sonnet 4.5 Claude Code 40% Sonnet 4.6 · baseline View full results → https://altimate.ai/benchmarks/ade-bench dbt analytics-engineering tasks Snowflake · DuckDB [Source →](https://github.com/dbt-labs/ade-bench) DAB Bench results As of June 2026 altimate-code 71.7% GPT-5.5 + Sonnet 4.6 Spacedock (Recce) 67.2% Opus 4.8 MinusX 65.2% Sonnet 4.6 + GPT-5.5-mini + Haiku 4.5 Pi Coding Agent 61% Opus 4.6 View full results → https://altimate.ai/benchmarks/dab-bench heterogeneous multi-database data tasks PostgreSQL · MongoDB · SQLite · DuckDB [Source →](https://github.com/ucbepic/DataAgentBench) Note > Both benchmarks are independent, third-party evaluations — ADE-Bench from dbt Labs, DAB from UC Berkeley's EPIC Data Lab. altimate-code ships 100+ purpose-built, deterministic tools for data agents — SQL analysis, column-level lineage, dbt, FinOps, and warehouse connectivity — which is why it leads across platforms regardless of the backbone model. --- # ADE-Bench Results URL: https://altimate.ai/benchmarks/ade-bench ADE-Bench: altimate-code hits 74.4% on Snowflake with Sonnet 4.6 and 78.0% on DuckDB with DeepSeek V4 Pro, ahead of Cortex Code CLI, dbt Labs, and Claude Code. [Benchmarks](https://altimate.ai/benchmarks) / ADE-Bench ## ADE-Bench Benchmark Results altimate-code + DeepSeek V4 Pro achieves 78.0% pass rate (32/41 tasks on DuckDB) — matching Sonnet 4.6 at a fraction of the cost. Model Sonnet 4.6 DeepSeek V4 Pro Kimi K2 soon Database Snowflake DuckDB ## About ADE-Bench ADE-Bench is a benchmark created by Benn Stancil (founder of Mode) in collaboration with dbt Labs. It evaluates AI agents on real-world analytics and data engineering tasks using actual dbt projects and databases. Each task runs in a Docker container sandbox; the agent attempts to resolve the task, and success is measured by whether all dbt tests pass afterward. Tasks include realistic data problems: vague requests like "it’s broken," debugging, schema issues, and complex analytics queries. ## Test Configuration | Harness and LLM | altimate-code (DeepSeek V4 Pro) | | --- | --- | | Model source | OpenRouter (deepseek/deepseek-chat-v4-pro) | | Database | DuckDB (local) | | Total Tasks | 41 | | Max Retries on failures | 3 | | Best Run (pass@3) | 32/41 (78.0%) | | Single-run range | 26–28/41 (63–68%) | ## Benchmark Comparison Agents evaluated on ADE-Bench with DuckDB. altimate-code (DeepSeek V4 Pro · DuckDB) — 32/41 78% altimate-code (Sonnet 4.6 · DuckDB) — 32/41 78% dbt Labs (Sonnet 4.5 · DuckDB) — ~25/43 59% [Source →](https://www.getdbt.com/blog/ade-bench-dbt-data-benchmarking) Claude Code (Sonnet 4.6 · baseline · DuckDB) — ~17/43 40% ### Key insight: the harness matters more than the model Across both benchmarks, altimate-code on Sonnet 4.6 beats competitors running Opus 4.6 — a more capable, more expensive model. Purpose-built tooling and deterministic operations outperform raw model capability alone. The harness — not the model — is the differentiator. ## Per-Task Results — DuckDB Best Run — 32 passed, 9 failed out of 41 tasks | # | Task | Result | Score | Pass Rate | | --- | --- | --- | --- | --- | | 1 | airbnb001 | ✓ | 10/10 | 100% | | 2 | airbnb002 | ✓ | 11/11 | 100% | | 3 | airbnb003 | ✓ | 7/7 | 100% | | 4 | airbnb004 | ✓ | 2/2 | 100% | | 5 | airbnb005 | ✓ | 4/4 | 100% | | 6 | airbnb006 | ✓ | 7/7 | 100% | | 7 | airbnb007 | ✓ | 11/11 | 100% | | 8 | airbnb008 | ✓ | 4/4 | 100% | | 9 | airbnb009 | ✗ | 0/1 | 0% | | 10 | analytics\_engineering001 | ✓ | 1/1 | 100% | | 11 | analytics\_engineering002 | ✓ | 2/2 | 100% | | 12 | analytics\_engineering003 | ✓ | 2/2 | 100% | | 13 | analytics\_engineering004 | ✗ | 1/2 | 50% | | 14 | analytics\_engineering005 | ✓ | 3/3 | 100% | | 15 | analytics\_engineering006 | ✗ | 4/7 | 57% | | 16 | analytics\_engineering007 | ✓ | 10/10 | 100% | | 17 | analytics\_engineering008 | ✓ | 1/1 | 100% | | 18 | asana001 | ✓ | 2/2 | 100% | | 19 | asana002 | ✓ | 3/3 | 100% | | 20 | asana003 | ✗ | 16/17 | 94% | | 21 | asana004 | ✗ | 5/6 | 83% | | 22 | asana005 | ✗ | 7/8 | 87% | | 23 | f1001 | ✓ | 6/6 | 100% | | 24 | f1002 | ✗ | 9/10 | 90% | | 25 | f1003 | ✓ | 4/4 | 100% | | 26 | f1004 | ✓ | 2/2 | 100% | | 27 | f1005 | ✓ | 4/4 | 100% | | 28 | f1006 | ✓ | 4/4 | 100% | | 29 | f1007 | ✓ | 6/6 | 100% | | 30 | f1009 | ✓ | 1/1 | 100% | | 31 | f1010 | ✓ | 2/2 | 100% | | 32 | f1011 | ✗ | 5/6 | 83% | | 33 | intercom001 | ✓ | 2/2 | 100% | | 34 | intercom002 | ✓ | 4/4 | 100% | | 35 | intercom003 | ✓ | 2/2 | 100% | | 36 | quickbooks001 | ✓ | 12/12 | 100% | | 37 | quickbooks002 | ✓ | 8/8 | 100% | | 38 | quickbooks003 | ✗ | 5/14 | 35% | | 39 | quickbooks004 | ✓ | 48/48 | 100% | | 40 | simple001 | ✓ | 1/1 | 100% | | 41 | simple002 | ✓ | 1/1 | 100% | ## Sources - [ADE-Bench on GitHub (dbt-labs)](https://github.com/dbt-labs/ade-bench) — Benchmark repository and methodology - [Snowflake Blog: Cortex Code CLI Expands Support](https://www.snowflake.com/en/blog/cortex-code-cli-expands-support/) — Cortex Code benchmark results (65%, Opus 4.6) - [dbt Labs Blog: Introducing ADE-Bench](https://www.getdbt.com/blog/ade-bench-dbt-data-benchmarking) — dbt Labs benchmark results (59%, Sonnet 4.5) ## ADE-Bench FAQs What is ADE-Bench? ADE-Bench is a benchmark created by Benn Stancil (founder of Mode) in collaboration with dbt Labs. It evaluates AI agents on real-world analytics engineering tasks using actual dbt projects running in Docker containers. Success is measured by whether all dbt tests pass after the agent attempts the task. It covers 43 tasks across Snowflake and DuckDB. What score did Altimate Code achieve on ADE-Bench? Altimate Code achieved 74.4% (32/43 tasks) on ADE-Bench using Claude Sonnet 4.6 on Snowflake. On DuckDB with DeepSeek V4 Pro, it scored 78.0% (32/41 tasks). Cortex Code CLI scored 65%, dbt Labs scored 59%, and the Claude Code baseline scored 40%. Why does Altimate Code outperform agents running more expensive models? Altimate Code on Sonnet 4.6 beats competitors running Opus 4.6 — a more capable, more expensive model. The reason is the harness: purpose-built, deterministic tools for data engineering outperform raw model capability. SQL analysis, column-level lineage, and dbt integration are handled by specialized tools, not general-purpose code generation. How does ADE-Bench measure agent performance? Each ADE-Bench task runs in an isolated Docker container with a real dbt project and database. The agent receives a task description (sometimes intentionally vague, like "it's broken") and must resolve it. Success is binary — all dbt tests must pass. Tasks include debugging, schema issues, complex analytics queries, and data engineering work across 43 scenarios. Can I run ADE-Bench myself to verify the results? Yes — ADE-Bench is open source on GitHub at github.com/dbt-labs/ade-bench. All benchmark tasks, scoring methodology, and submission instructions are public. Altimate Code's submission is PR #44 on the DataAgentBench repository. What databases does ADE-Bench cover? ADE-Bench currently covers Snowflake and DuckDB. Altimate Code achieved 74.4% on Snowflake (Sonnet 4.6) and 78.0% on DuckDB (DeepSeek V4 Pro). Altimate Code also supports BigQuery, Databricks, Redshift, Postgres, MySQL, and SQLite in production. --- # DAB Bench Results URL: https://altimate.ai/benchmarks/dab-bench DAB (Data Agent Benchmark): altimate-code scores 71.7% Pass@1 across PostgreSQL, MongoDB, SQLite, and DuckDB. [Benchmarks](https://altimate.ai/benchmarks) / DAB Bench ## DAB Benchmark Results altimate-code scores 71.7% Pass@1 on DAB, ahead of agents running Claude Opus 4.6. ## About DAB DAB (Data Agent Benchmark) is the first benchmark for evaluating data agents on realistic, complex, data-oriented tasks — a collaboration between the EPIC Data Lab at UC Berkeley and Hasura PromptQL. Unlike prior SQL-only or single-database benchmarks, DAB stresses agents under real enterprise data complexity: multi-database integration, ill-formatted key joins, unstructured-text transformation, and domain knowledge. It spans 54 queries across 12 datasets, 9 domains, and 4 database systems — PostgreSQL, MongoDB, SQLite, and DuckDB. ## Test Configuration | Agent | altimate-code | | --- | --- | | Backbone LLM | GPT-5.5 + Claude Sonnet 4.6 | | Result | Pass@1 0.717 | | Trials | 5 per query (270 trials across 54 queries) | | Datasets | 12 datasets · 9 domains | | Databases | PostgreSQL, MongoDB, SQLite, DuckDB | | Dataset hints | Yes (db\_description\_withhint.txt) | | Submission | PR #53 | ## Leaderboard Official DAB leaderboard, Pass@1, ranked, as published June 2026. altimate-code scores 0.717. #1 Altimate Code + GPT-5.5 + Claude Sonnet 4.6 (GPT-5.5 + Sonnet 4.6) 71.7% [Submission →](https://github.com/ucbepic/DataAgentBench/pull/53) #2 Altimate Code + Claude Sonnet 4.6 (Claude Sonnet 4.6) 68.2% [Submission →](https://github.com/ucbepic/DataAgentBench/pull/44) #3 Spacedock (Recce) + Claude Opus 4.8 (Claude Opus 4.8) 67.2% [Submission →](https://github.com/ucbepic/DataAgentBench/pull/55) #4 MinusX + Claude Sonnet 4.6 + GPT-5.5-mini + Haiku 4.5 (Sonnet 4.6 + GPT-5.5-mini + Haiku 4.5) 65.2% [Submission →](https://github.com/ucbepic/DataAgentBench/pull/50) #5 Pi Coding Agent + Claude Opus 4.6 (Claude Opus 4.6) 61.0% [Submission →](https://github.com/ucbepic/DataAgentBench/pull/31) #6 PromptQL + Gemini 3.1 Pro (Gemini 3.1 Pro) 60.0% [Submission →](https://github.com/ucbepic/DataAgentBench/pull/24) ▼ Show 11 more entries ## Benchmark Composition 12 datasets spanning 4 database systems — every task requires the agent to work across heterogeneous stores. | Dataset | Databases | DBs | Tables | Queries | | --- | --- | --- | --- | --- | | agnews | MongoDB, SQLite | 2 | 3 | 4 | | bookreview | PostgreSQL, SQLite | 2 | 2 | 3 | | crmarenapro | DuckDB, PostgreSQL, SQLite | 6 | 27 | 13 | | deps\_dev\_v1 | DuckDB, SQLite | 2 | 3 | 2 | | github\_repos | DuckDB, SQLite | 2 | 6 | 4 | | googlelocal | PostgreSQL, SQLite | 2 | 2 | 4 | | music\_brainz\_20k | DuckDB, SQLite | 2 | 2 | 3 | | pancancer\_atlas | DuckDB, PostgreSQL | 2 | 3 | 3 | | patents | PostgreSQL, SQLite | 2 | 2 | 3 | | stockindex | DuckDB, SQLite | 2 | 2 | 3 | | stockmarket | DuckDB, SQLite | 2 | 2754 | 5 | | yelp | DuckDB, MongoDB | 2 | 5 | 7 | | Total | PostgreSQL · MongoDB · SQLite · DuckDB | 28 | 2,811 | 54 | ## Sources - [DAB Leaderboard (UC Berkeley EPIC Data Lab)](https://ucbepic.github.io/DataAgentBench/) — Official leaderboard and benchmark overview - [DataAgentBench on GitHub](https://github.com/ucbepic/DataAgentBench) — Benchmark repository, datasets, and submission guide - [altimate-code submission (PR #44)](https://github.com/ucbepic/DataAgentBench/pull/44) — Our leaderboard submission - [DAB paper (arXiv)](https://arxiv.org/abs/2603.20576) — Methodology and analysis ## DAB Bench FAQs What is DAB Bench? DAB (Data Agent Benchmark) is the first benchmark for evaluating data agents on realistic, complex, multi-database tasks — a collaboration between the EPIC Data Lab at UC Berkeley and Hasura PromptQL. Unlike SQL-only or single-database benchmarks, DAB tests agents on real enterprise data complexity: multi-database integration, ill-formatted key joins, unstructured-text transformation, and domain knowledge. It spans 54 queries across 12 datasets, 9 domains, and 4 database systems — PostgreSQL, MongoDB, SQLite, and DuckDB. What score did Altimate Code achieve on DAB Bench? Altimate Code scored 71.7% Pass@1 on DAB Bench using GPT-5.5 + Claude Sonnet 4.6, ahead of Spacedock/Recce (67.2% on Opus 4.8), MinusX (65.2%), and Pi Coding Agent (61.0% on Opus 4.6). Altimate Code's submission is PR #53 on the DataAgentBench repository. Why does Altimate Code outperform agents running more expensive models on DAB Bench? Altimate Code on GPT-5.5 + Sonnet 4.6 beats competitors running Opus 4.8 — a more capable, more expensive model. The reason is the harness: purpose-built, deterministic tools for data engineering outperform raw model capability. DAB requires multi-database reasoning across PostgreSQL, MongoDB, SQLite, and DuckDB simultaneously — exactly the kind of task that specialized tooling handles better than general-purpose code generation. How does DAB Bench differ from ADE-Bench? ADE-Bench (by dbt Labs) focuses on analytics engineering tasks using dbt projects on Snowflake and DuckDB. DAB Bench (by UC Berkeley EPIC Data Lab) tests agents on heterogeneous multi-database tasks — requiring agents to query and reason across PostgreSQL, MongoDB, SQLite, and DuckDB simultaneously. DAB spans 54 queries across 12 datasets and 9 domains, stressing agents on real enterprise data complexity rather than single-warehouse analytics work. Can I verify the DAB Bench results myself? Yes — DAB Bench is open source on GitHub at github.com/ucbepic/DataAgentBench. The official leaderboard is at ucbepic.github.io/DataAgentBench. Altimate Code's submission is PR #44 on the DataAgentBench repository. The benchmark methodology is also published in a paper on arXiv (arxiv.org/abs/2603.20576). What databases does DAB Bench cover? DAB Bench covers 4 database systems: PostgreSQL, MongoDB, SQLite, and DuckDB — spanning 12 datasets and 28 individual databases across 54 queries. Every task requires the agent to work across heterogeneous stores simultaneously. Altimate Code also supports Snowflake, BigQuery, Databricks, Redshift, MySQL, and additional warehouses in production. --- # Skills & Tools URL: https://altimate.ai/skills Altimate Code gives your AI agent purpose-built tools for SQL, dbt, lineage, testing, and warehouse work. Skills are the workflows it runs. Altimate Code capabilities ## Skills and tools for the AI data engineer Altimate Code gives your AI agent purpose-built tools for SQL, dbt, lineage, testing, and warehouse work. Skills are the workflows it can run. Tools are what powers them. ## Skills and tools catalog Skills 19 Tools 79 What Altimate can do for your data engineering work. All Setup 1 SQL Quality 3 dbt 4 Testing 2 Lineage 1 Validation 1 Migrations 1 Governance 1 FinOps 1 Visualization 1 Team Learning 3 19 skills Setup ### Altimate Setup Set up the AI data engineer before the work starts. Connect Altimate to your warehouse and dbt project so the agent has context from day one. datamate\_manager project\_scan +2 more https://altimate.ai/skills/altimate-setup SQL Quality ### SQL Review AI code review, but for SQL. Catch SQL anti-patterns, performance risks, and safety problems before code ships. Altimate Code grades queries and explains every finding with a fix. sql\_analyze altimate\_core\_check +1 more https://altimate.ai/skills/sql-review SQL Quality ### Query Optimize Make slow or expensive SQL easier to fix. Find what's making a query slow or expensive and get specific, explainable suggestions. sql\_optimize sql\_analyze +2 more https://altimate.ai/skills/query-optimize dbt ### dbt Troubleshoot Debug broken dbt builds with project context. Diagnose dbt compilation failures, runtime errors, failing tests, wrong results, and performance issues. altimate\_core\_semantics altimate\_core\_column\_lineage +3 more https://altimate.ai/skills/dbt-troubleshoot Testing ### dbt Unit Tests Generate dbt unit test scaffolds without starting from blank YAML. Analyze dbt model logic and generate unit test scenarios with mock inputs and expected outputs. dbt\_unit\_test\_gen dbt\_manifest +3 more https://altimate.ai/skills/dbt-unit-tests Lineage ### Lineage Diff Show how data flow changed, not just how SQL text changed. Compare column-level lineage between two SQL versions to see how data flow changed, not just which lines changed. Built for dbt model reviews. lineage\_check sql\_diff https://altimate.ai/skills/lineage-diff Validation ### Data Parity Prove two datasets match after a migration, refactor, or rewrite. Validate that two tables or query results are identical or diagnose exactly how they differ. data\_diff schema\_inspect +1 more https://altimate.ai/skills/data-parity FinOps ### Cost Report Find where warehouse spend is leaking. Find where Snowflake spend is leaking. Altimate Code analyzes query costs, ranks top spenders, and recommends warehouse tuning and query fixes. finops\_query\_history finops\_analyze\_credits +3 more https://altimate.ai/skills/cost-report Governance ### PII Audit Find sensitive data before it spreads. Classify schema columns for PII and check whether SQL or dbt models expose sensitive data before it reaches production or external consumers. altimate\_core\_classify\_pii altimate\_core\_query\_pii +2 more https://altimate.ai/skills/pii-audit dbt ### dbt Analyze Understand downstream impact before changing a model. Analyze the blast radius of dbt model changes using dependency information and column-level lineage. dbt\_manifest dbt\_lineage +3 more https://altimate.ai/skills/dbt-analyze Migrations ### Schema Migration Catch breaking schema changes before production. Analyze DDL and schema changes for data loss, compatibility, and downstream risk. altimate\_core\_migration altimate\_core\_schema\_diff +1 more https://altimate.ai/skills/schema-migration Testing ### dbt Test Add the right data quality tests to dbt models. Add schema tests, unit tests, and data quality checks that match model risk. Altimate Code inspects columns and relationships and suggests the right tests. dbt\_manifest altimate\_core\_testgen +1 more https://altimate.ai/skills/dbt-test dbt ### dbt Develop Build dbt models that fit the project. Build dbt models that fit your project structure: correct layers, naming conventions, refs, sources, YAML, and validation. Not just SQL generation. schema\_search dbt\_profiles +2 more https://altimate.ai/skills/dbt-develop dbt ### dbt Docs Turn empty dbt docs into useful business context. Write dbt model and column descriptions that explain meaning, not just names. Altimate Code reads model SQL and drafts docs that are actually useful. dbt\_manifest dbt\_lineage https://altimate.ai/skills/dbt-docs SQL Quality ### SQL Translate Translate SQL across warehouse dialects. Convert SQL between Snowflake, BigQuery, Postgres, MySQL, Databricks, Redshift, DuckDB, and other dialects. sql\_translate altimate\_core\_validate https://altimate.ai/skills/sql-translate Visualization ### Data Viz Turn query results into charts and dashboards. Build charts, dashboards, and interactive views from query results. sql\_execute schema\_search https://altimate.ai/skills/data-viz Team Learning ### Teach Teach the AI your team's patterns from one good example. Show Altimate a strong example file and save the reusable pattern for future work. training\_save training\_list https://altimate.ai/skills/teach Team Learning ### Train Turn team standards into reusable AI behavior. Load your style guides, review checklists, glossary docs, and standards so the AI can apply them consistently. training\_import training\_save +1 more https://altimate.ai/skills/train Team Learning ### Training Status See what the AI teammate has learned. Display saved patterns, rules, glossary terms, standards, and training metadata. training\_list training\_remove https://altimate.ai/skills/training-status --- # Altimate Setup URL: https://altimate.ai/skills/altimate-setup Connect Altimate Code to your warehouse and dbt project before the first task. The skill walks you through credentials and checks that the agent can reach them. Setup ## Connect Altimate Code to Your Warehouse and dbt Project Set up the AI data engineer before the work starts. Connect Altimate to your warehouse and dbt project so the agent has context from day one. [Connect your warehouse first. →](https://docs.altimate.sh/getting-started) The problem it solves General AI coding tools often start from a blank prompt. For data engineering, that means the assistant does not know the warehouse, dbt project, schema context, or team setup until the user explains it manually. ## What it does Guides the user through Altimate platform credential setup. Validates that the agent can access configured Altimate resources. Supports the setup flow that makes later skills more useful. Pairs naturally with environment discovery and warehouse configuration. ## When to use it WHEN setting up Altimate Code for the first time WHEN reconnecting or updating Altimate platform credentials BEFORE running workflows that depend on platform or metadata access ## How it works Collect credentials, write local configuration, validate platform access, then hand off to discovery and warehouse setup workflows. Tools used datamate\_manager Manage Altimate platform datamate resources and validate platform access. project\_scan Scan the local data engineering environment for dbt projects, warehouse connections, tooling, and config. warehouse\_discover Discover local database containers and connection candidates. warehouse\_test Validate that a configured warehouse connection works. Related skills Team Learning Training Status https://altimate.ai/skills/training-status Team Learning Teach https://altimate.ai/skills/teach Team Learning Train https://altimate.ai/skills/train [← All skills & tools](https://altimate.ai/skills) --- # SQL Review URL: https://altimate.ai/skills/sql-review Catch SQL anti-patterns, performance risks, and safety problems before code ships. Altimate Code grades queries and explains every finding with a fix. SQL Quality ## SQL Code Review: Catch Issues Before They Ship AI code review, but for SQL. Catch SQL anti-patterns, performance risks, and safety problems before code ships. Altimate Code grades queries and explains every finding with a fix. [Start reviewing SQL with Altimate Code. →](https://docs.altimate.sh/getting-started) The problem it solves SQL review is slow because reviewers have to check syntax, joins, filters, performance, readability, and data risk manually. Generic AI can comment on SQL, but it does not follow a repeatable data engineering review process. ## What it does Detects SQL anti-patterns such as SELECT \*, cartesian joins, missing limits, and function filters. Flags readability, maintainability, and performance issues. Grades or summarizes query quality for review. Explains why each issue matters and suggests concrete fixes. ## When to use it BEFORE merging SQL or dbt model changes DURING pull request review BEFORE running exploratory SQL that may be expensive WHEN standardizing SQL quality across a team ## How it works Pass SQL to the skill. It runs static analysis and review rules, then returns each finding with a severity level, why it matters, and what to fix. Tools used sql\_analyze Analyze SQL for anti-patterns, performance issues, and safety risks without executing it. altimate\_core\_check Run core SQL checks for quality, safety, and review workflows. altimate\_core\_grade Grade SQL quality or review output through Altimate Core. Related skills SQL Quality Query Optimize https://altimate.ai/skills/query-optimize Migrations Schema Migration https://altimate.ai/skills/schema-migration Governance PII Audit https://altimate.ai/skills/pii-audit [← All skills & tools](https://altimate.ai/skills) --- # Query Optimize URL: https://altimate.ai/skills/query-optimize Find the SQL anti-patterns that make a query slow or expensive, such as SELECT * and missing partition pruning, and get a rewrite with the reason for each fix. SQL Quality ## Find What Is Making Your SQL Slow or Expensive Make slow or expensive SQL easier to fix. Find what's making a query slow or expensive and get specific, explainable suggestions. [Use Altimate Code to tune SQL with evidence. →](https://docs.altimate.sh/getting-started) The problem it solves Data teams often know a query is slow or expensive, but identifying the cause takes experience: broad scans, non-sargable filters, SELECT \*, bad joins, unnecessary sorts, or missing partition pruning. ## What it does Finds performance anti-patterns in SQL. Suggests rewritten SQL or targeted optimization steps. Uses schema context and execution plans when available. Can pair optimization suggestions with equivalence checks. ## When to use it WHEN a query is slow, expensive, or scanning too much data BEFORE productionizing exploratory SQL WHEN tuning warehouse workloads DURING cost reduction or performance review work ## How it works The skill reviews the SQL, gathers schema or execution context when available, identifies optimization opportunities, and explains what changed and why it helps. Tools used sql\_optimize Find query performance issues and suggest safer rewrites or optimization strategies. sql\_analyze Analyze SQL for anti-patterns, performance issues, and safety risks without executing it. sql\_explain Generate warehouse execution plans so the agent can reason about query cost and performance. altimate\_core\_equivalence Check whether two SQL queries are intended to produce equivalent results. Related skills SQL Quality SQL Review https://altimate.ai/skills/sql-review FinOps Cost Report https://altimate.ai/skills/cost-report SQL Quality SQL Translate https://altimate.ai/skills/sql-translate [← All skills & tools](https://altimate.ai/skills) --- # dbt Troubleshoot URL: https://altimate.ai/skills/dbt-troubleshoot Diagnose dbt compilation failures, runtime errors, failing tests, wrong results, and performance issues. dbt ## Debug Broken dbt Builds With Full Project Context Debug broken dbt builds with project context. Diagnose dbt compilation failures, runtime errors, failing tests, wrong results, and performance issues. [Debug dbt with an agent that knows your project. →](https://docs.altimate.sh/getting-started) The problem it solves When dbt fails, pasting the error into a chatbot is usually not enough. The fix depends on model SQL, compiled SQL, upstream dependencies, tests, configuration, and warehouse-specific errors. ## What it does Follows a structured dbt debugging workflow. Inspects the failing model, compiled context, and likely upstream causes. Explains the root cause rather than only patching the symptom. Recommends or applies fixes when file edits are appropriate. ## When to use it WHEN a dbt build, run, compile, or test command fails WHEN a model returns wrong or unexpected data WHEN a dbt model is slow or timing out WHEN a warehouse error needs to be mapped back to dbt logic ## How it works The skill checks project health, inspects the failing artifact, follows dependencies, diagnoses the likely cause, proposes a fix path, and validates the result. Tools used altimate\_core\_semantics Analyze SQL or model semantics for correctness-oriented workflows. altimate\_core\_column\_lineage Compute column-level lineage through Altimate Core. altimate\_core\_correct Suggest correctness-oriented repairs or improvements through Altimate Core. altimate\_core\_fix Suggest deterministic fixes for SQL or data-engineering issues. sql\_fix Diagnose SQL errors and suggest corrected SQL or next debugging steps. Related skills dbt dbt Develop https://altimate.ai/skills/dbt-develop Testing dbt Test https://altimate.ai/skills/dbt-test dbt dbt Analyze https://altimate.ai/skills/dbt-analyze [← All skills & tools](https://altimate.ai/skills) --- # dbt Unit Tests URL: https://altimate.ai/skills/dbt-unit-tests Generate dbt unit test YAML from your model logic. Altimate Code reads joins, CASE statements, and null handling, then writes mock inputs and expected outputs. Testing ## Generate dbt Unit Test Scaffolds Without Starting From Blank YAML Generate dbt unit test scaffolds without starting from blank YAML. Analyze dbt model logic and generate unit test scenarios with mock inputs and expected outputs. [Add meaningful dbt unit tests faster with Altimate Code. →](https://docs.altimate.sh/getting-started) The problem it solves dbt unit tests are powerful but often skipped because writing mock inputs, expected outputs, and edge cases takes time and careful model understanding. ## What it does Analyzes SQL logic such as joins, CASE statements, null handling, windows, and incremental behavior. Finds upstream dependencies that need mock inputs. Generates type-aware dbt unit test YAML scaffolds. Supports iterative refinement and validation of generated tests. ## When to use it WHEN adding coverage for a business-critical dbt model WHEN a model contains logic that could silently regress DURING test-driven development in dbt BEFORE refactoring model SQL ## How it works The skill reads model SQL and manifest context, identifies testable logic, generates scenarios, refines expected outputs, and validates the tests when possible. Tools used dbt\_unit\_test\_gen Generate dbt unit test scaffolds from model SQL, manifest metadata, and upstream dependencies. dbt\_manifest Parse dbt manifest metadata to understand models, sources, tests, and dependency structure. dbt\_lineage Extract dbt model dependency and lineage context. altimate\_core\_validate Run deterministic SQL or data-engineering validation through Altimate Core. altimate\_core\_testgen Generate testing suggestions and scenarios through Altimate Core. Related skills Testing dbt Test https://altimate.ai/skills/dbt-test dbt dbt Troubleshoot https://altimate.ai/skills/dbt-troubleshoot Validation Data Parity https://altimate.ai/skills/data-parity [← All skills & tools](https://altimate.ai/skills) --- # Lineage Diff URL: https://altimate.ai/skills/lineage-diff Compare column-level lineage between two SQL versions to see how data flow changed, not just which lines changed. Built for dbt model reviews. Lineage ## See How Data Flow Changed, Not Just Which Lines Changed Show how data flow changed, not just how SQL text changed. Compare column-level lineage between two SQL versions to see how data flow changed, not just which lines changed. Built for dbt model reviews. [See what actually changed in your data flow, not just which lines changed. →](https://docs.altimate.sh/getting-started) The problem it solves Code diffs show line changes, but data teams need to understand semantic changes: new sources, removed relationships, changed transformations, or columns flowing from different places. ## What it does Runs column-level lineage on the before and after versions of SQL. Compares source-to-output relationships. Highlights added, removed, and changed lineage edges. Improves code review for refactors and model changes. ## When to use it DURING SQL or dbt model review BEFORE merging a refactor WHEN changing source fields or transformation logic WHEN governance teams need to understand data flow impact ## How it works The skill obtains the original and modified SQL, traces lineage for both, computes the diff, and reports how data flow changed. Tools used lineage\_check Trace column-level lineage through SQL transformations. sql\_diff Compare two SQL versions and summarize structural changes. Related skills dbt dbt Analyze https://altimate.ai/skills/dbt-analyze Migrations Schema Migration https://altimate.ai/skills/schema-migration Validation Data Parity https://altimate.ai/skills/data-parity [← All skills & tools](https://altimate.ai/skills) --- # Data Parity URL: https://altimate.ai/skills/data-parity Compare two tables or query results after a migration or refactor. Altimate Code reports row count, schema, key, and value differences as sign-off evidence. Validation ## Prove Two Datasets Match After a Migration or Refactor Prove two datasets match after a migration, refactor, or rewrite. Validate that two tables or query results are identical or diagnose exactly how they differ. [Validate data changes with evidence, not hope. →](https://docs.altimate.sh/getting-started) The problem it solves Most migration sign-offs are vibes. You need row counts, schema comparison, key coverage, and targeted diffs before you can actually trust the new result. ## What it does Compares two tables or query results. Supports safer profile-based comparison for sensitive or large data. Identifies row count, schema, key, and value differences. Produces evidence that helps teams validate migrations. ## When to use it AFTER a warehouse migration AFTER an ETL or dbt refactor WHEN replacing one table or query with another BEFORE signing off on data parity in critical workflows ## How it works The skill inspects both sides, confirms keys and exclusions, profiles cheaply first, and runs targeted row-level diff only when appropriate. Tools used data\_diff Compare two tables or queries with profile or row-level diff strategies. schema\_inspect Inspect a table's columns, types, nullability, and constraints. sql\_execute Run SQL against a configured warehouse and return a formatted result set. Related skills Migrations Schema Migration https://altimate.ai/skills/schema-migration Lineage Lineage Diff https://altimate.ai/skills/lineage-diff Testing dbt Unit Tests https://altimate.ai/skills/dbt-unit-tests [← All skills & tools](https://altimate.ai/skills) --- # Cost Report URL: https://altimate.ai/skills/cost-report Find where Snowflake spend is leaking. Altimate Code analyzes query costs, ranks top spenders, and recommends warehouse tuning and query fixes. FinOps ## Find Where Snowflake Spend Is Leaking Find where warehouse spend is leaking. Find where Snowflake spend is leaking. Altimate Code analyzes query costs, ranks top spenders, and recommends warehouse tuning and query fixes. [Find what's costing money and do something about it. →](https://docs.altimate.sh/getting-started) The problem it solves Warehouse spend often grows quietly. A few expensive queries, oversized warehouses, idle resources, or repeated scans can drive cost without a clear owner. ## What it does Finds expensive queries and cost trends. Groups spend by warehouse, user, or workload. Identifies optimization opportunities. Recommends practical next actions such as query tuning, auto-suspend changes, or warehouse resizing. ## When to use it DURING Snowflake cost reviews WHEN investigating a cost spike WHEN looking for query tuning opportunities WHEN right-sizing warehouses ## How it works The skill gathers query history and usage metadata, ranks cost drivers, analyzes patterns, and turns findings into recommended actions. Tools used finops\_query\_history Fetch recent query execution history from a warehouse. finops\_analyze\_credits Break down warehouse credit consumption by time, user, and warehouse. finops\_expensive\_queries Find the most expensive queries and identify optimization opportunities. finops\_warehouse\_advice Recommend warehouse sizing, suspend settings, and cost controls from usage patterns. finops\_unused\_resources Find unused tables and idle warehouse resources. Related skills SQL Quality Query Optimize https://altimate.ai/skills/query-optimize SQL Quality SQL Review https://altimate.ai/skills/sql-review Governance PII Audit https://altimate.ai/skills/pii-audit [← All skills & tools](https://altimate.ai/skills) --- # PII Audit URL: https://altimate.ai/skills/pii-audit Classify schema columns for PII and check whether SQL or dbt models expose sensitive data before it reaches production or external consumers. Governance ## Find Sensitive Data Before It Reaches Production Find sensitive data before it spreads. Classify schema columns for PII and check whether SQL or dbt models expose sensitive data before it reaches production or external consumers. [Catch privacy risk while the model is still being built. →](https://docs.altimate.sh/getting-started) The problem it solves Sensitive data often enters analytics through ordinary columns: email, phone, IP address, name, address, or customer identifiers. Teams need to catch exposure early. ## What it does Classifies columns that may contain personally identifiable information. Checks whether SQL or dbt models expose sensitive fields. Reports categories and confidence. Supports privacy and governance workflows. ## When to use it BEFORE productionizing a model that touches customer data WHEN auditing a schema for sensitive columns WHEN checking whether a query exposes PII DURING GDPR, CCPA, HIPAA, or internal governance reviews ## How it works The skill scans column names, types, and query lineage to find columns likely to contain PII and flag where they're being used. Tools used altimate\_core\_classify\_pii Classify potential PII columns through Altimate Core. altimate\_core\_query\_pii Check whether a query exposes potential PII. schema\_detect\_pii Scan warehouse metadata for columns that may contain personally identifiable information. schema\_inspect Inspect a table's columns, types, nullability, and constraints. Related skills SQL Quality SQL Review https://altimate.ai/skills/sql-review Migrations Schema Migration https://altimate.ai/skills/schema-migration FinOps Cost Report https://altimate.ai/skills/cost-report [← All skills & tools](https://altimate.ai/skills) --- # dbt Analyze URL: https://altimate.ai/skills/dbt-analyze Analyze the blast radius of dbt model changes using dependency information and column-level lineage. dbt ## Understand Downstream Impact Before Changing a dbt Model Understand downstream impact before changing a model. Analyze the blast radius of dbt model changes using dependency information and column-level lineage. [Know the blast radius before changing dbt. →](https://docs.altimate.sh/getting-started) The problem it solves A small dbt change can affect downstream marts, dashboards, and business processes. Teams need to know what depends on a model or column before merging changes. ## What it does Identifies downstream models and consumers. Uses dbt graph context and column-level lineage. Summarizes affected columns and risky dependencies. Helps reviewers understand blast radius before merge. ## When to use it BEFORE changing a shared staging or intermediate model WHEN renaming or removing columns DURING dbt refactors WHEN assessing change risk in a pull request ## How it works The skill locates the changed model, maps the dbt dependency graph, traces lineage, and reports downstream impact. Tools used dbt\_manifest Parse dbt manifest metadata to understand models, sources, tests, and dependency structure. dbt\_lineage Extract dbt model dependency and lineage context. lineage\_check Trace column-level lineage through SQL transformations. impact\_analysis Assess downstream impact for data model or schema changes. altimate\_core\_extract\_metadata Extract structured metadata from SQL, dbt, or warehouse context. Related skills Lineage Lineage Diff https://altimate.ai/skills/lineage-diff Migrations Schema Migration https://altimate.ai/skills/schema-migration dbt dbt Troubleshoot https://altimate.ai/skills/dbt-troubleshoot [← All skills & tools](https://altimate.ai/skills) --- # Schema Migration URL: https://altimate.ai/skills/schema-migration Compare old and new schema definitions before you run a DDL migration. Altimate Code flags dropped columns, type narrowing, and changes that break consumers. Migrations ## Catch Breaking Schema Changes Before They Hit Production Catch breaking schema changes before production. Analyze DDL and schema changes for data loss, compatibility, and downstream risk. [Review schema changes before they surprise production. →](https://docs.altimate.sh/getting-started) The problem it solves Schema changes can look harmless while still breaking consumers: dropped columns, type narrowing, nullability changes, renamed fields, missing defaults, or removed constraints. ## What it does Compares old and new schema definitions. Flags breaking changes and data loss risks. Highlights type narrowing, dropped columns, and compatibility issues. Supports safer production migration review. ## When to use it BEFORE applying DDL migrations BEFORE merging schema YAML or dbt contract changes DURING warehouse migrations WHEN reviewing pull requests that alter table shape ## How it works The skill compares the two schemas, runs migration analysis, and flags each risk with severity and why it matters. Tools used altimate\_core\_migration Analyze migration risk through Altimate Core. altimate\_core\_schema\_diff Compare schema versions through Altimate Core. schema\_diff Compare schema changes and highlight compatibility or migration risks. Related skills Validation Data Parity https://altimate.ai/skills/data-parity Lineage Lineage Diff https://altimate.ai/skills/lineage-diff dbt dbt Analyze https://altimate.ai/skills/dbt-analyze [← All skills & tools](https://altimate.ai/skills) --- # dbt Test URL: https://altimate.ai/skills/dbt-test Add schema tests, unit tests, and data quality checks that match model risk. Altimate Code inspects columns and relationships and suggests the right tests. Testing ## Add the Right Data Quality Tests to dbt Models Add the right data quality tests to dbt models. Add schema tests, unit tests, and data quality checks that match model risk. Altimate Code inspects columns and relationships and suggests the right tests. [Turn dbt testing from someday work into part of the workflow. →](https://docs.altimate.sh/getting-started) The problem it solves Teams know they should add tests, but deciding which columns need not\_null, unique, relationships, accepted values, ranges, or custom tests is repetitive and easy to postpone. ## What it does Inspects model columns and relationships. Suggests schema tests and data quality checks. Supports unit test and generic test workflows. Keeps test changes tied to model behavior and risk. ## When to use it WHEN adding coverage to a new dbt model WHEN improving test coverage on an existing model WHEN practicing test-driven development in dbt WHEN a failing test needs investigation rather than weakening ## How it works The skill reads your columns and relationships, proposes tests that fit the model's behavior, applies YAML when you approve, and validates with dbt. Tools used dbt\_manifest Parse dbt manifest metadata to understand models, sources, tests, and dependency structure. altimate\_core\_testgen Generate testing suggestions and scenarios through Altimate Core. altimate\_core\_validate Run deterministic SQL or data-engineering validation through Altimate Core. Related skills Testing dbt Unit Tests https://altimate.ai/skills/dbt-unit-tests dbt dbt Troubleshoot https://altimate.ai/skills/dbt-troubleshoot dbt dbt Develop https://altimate.ai/skills/dbt-develop [← All skills & tools](https://altimate.ai/skills) --- # dbt Develop URL: https://altimate.ai/skills/dbt-develop Build dbt models that fit your project structure: correct layers, naming conventions, refs, sources, YAML, and validation. Not just SQL generation. dbt ## Build dbt Models That Fit Your Project Structure Build dbt models that fit the project. Build dbt models that fit your project structure: correct layers, naming conventions, refs, sources, YAML, and validation. Not just SQL generation. [Build dbt models that actually fit your project. →](https://docs.altimate.sh/getting-started) The problem it solves A generic AI tool can write SQL, but dbt development also requires project architecture: staging/intermediate/marts, refs, sources, materializations, naming conventions, and validation. ## What it does Discovers project structure and existing patterns. Creates or modifies dbt model SQL. Wires refs, sources, configs, and YAML when appropriate. Validates generated work using dbt and Altimate analysis tools. ## When to use it WHEN creating a new staging, intermediate, mart, or incremental model WHEN modifying an existing model WHEN scaffolding model YAML or sources WHEN reorganizing dbt project structure ## How it works The skill plans the model, discovers dependencies and conventions, writes changes, and validates the result against project expectations. Tools used schema\_search Search indexed schemas, tables, and columns by keyword. dbt\_profiles Resolve dbt profiles and connection context for a project. altimate\_core\_validate Run deterministic SQL or data-engineering validation through Altimate Core. altimate\_core\_column\_lineage Compute column-level lineage through Altimate Core. Related skills Testing dbt Test https://altimate.ai/skills/dbt-test dbt dbt Docs https://altimate.ai/skills/dbt-docs dbt dbt Troubleshoot https://altimate.ai/skills/dbt-troubleshoot [← All skills & tools](https://altimate.ai/skills) --- # dbt Docs URL: https://altimate.ai/skills/dbt-docs Write dbt model and column descriptions that explain meaning, not just names. Altimate Code reads model SQL and drafts docs that are actually useful. dbt ## Write dbt Docs That Actually Explain What Models Do Turn empty dbt docs into useful business context. Write dbt model and column descriptions that explain meaning, not just names. Altimate Code reads model SQL and drafts docs that are actually useful. [Make dbt docs useful with Altimate Code. →](https://docs.altimate.sh/getting-started) The problem it solves dbt docs usually stay blank or just repeat the column name. That's useless for anyone who didn't write the model. ## What it does Reads model SQL and dbt context. Drafts model and column descriptions. Supports reusable documentation blocks. Improves discoverability for analysts and business users. ## When to use it WHEN adding docs to a new model WHEN improving a dbt docs site WHEN filling sparse schema YAML WHEN preparing data products for broader use ## How it works The skill reads the model SQL and dependencies, then writes descriptions that explain what each model and column actually does, not just what it's named. Tools used dbt\_manifest Parse dbt manifest metadata to understand models, sources, tests, and dependency structure. dbt\_lineage Extract dbt model dependency and lineage context. Related skills dbt dbt Develop https://altimate.ai/skills/dbt-develop Testing dbt Test https://altimate.ai/skills/dbt-test Team Learning Teach https://altimate.ai/skills/teach [← All skills & tools](https://altimate.ai/skills) --- # SQL Translate URL: https://altimate.ai/skills/sql-translate Convert SQL between Snowflake, BigQuery, Postgres, MySQL, Databricks, Redshift, DuckDB, and other dialects. SQL Quality ## Translate SQL Across Warehouse Dialects Without Manual Syntax Traps Translate SQL across warehouse dialects. Convert SQL between Snowflake, BigQuery, Postgres, MySQL, Databricks, Redshift, DuckDB, and other dialects. [Move SQL across warehouses with fewer manual syntax traps. →](https://docs.altimate.sh/getting-started) The problem it solves Warehouse migrations and multi-platform support are full of small SQL syntax traps: date functions, casts, arrays, QUALIFY, window behavior, null handling, and dialect-specific functions. ## What it does Translates SQL between source and target dialects. Flags lossy or review-needed translations. Supports migration and portability workflows. Can validate translated SQL when supported. ## When to use it DURING warehouse migration WHEN supporting customers on different warehouses WHEN porting Snowflake SQL to BigQuery, Postgres, Databricks, or similar platforms WHEN checking whether syntax is portable ## How it works The skill identifies source and target dialects, translates the SQL, reviews warnings, and optionally validates the translated query. Tools used sql\_translate Translate SQL between warehouse dialects such as Snowflake, BigQuery, Postgres, Databricks, and Redshift. altimate\_core\_validate Run deterministic SQL or data-engineering validation through Altimate Core. Related skills SQL Quality Query Optimize https://altimate.ai/skills/query-optimize SQL Quality SQL Review https://altimate.ai/skills/sql-review Validation Data Parity https://altimate.ai/skills/data-parity [← All skills & tools](https://altimate.ai/skills) --- # Data Viz URL: https://altimate.ai/skills/data-viz Build charts, dashboards, and interactive views straight from your query results, so you can spot trends and share findings without leaving your editor. Visualization ## Turn Query Results Into Charts and Dashboards Turn query results into charts and dashboards. Build charts, dashboards, and interactive views from query results. [Turn analysis into something stakeholders can see. →](https://docs.altimate.sh/getting-started) The problem it solves Most analysis ends with a table that nobody outside the data team can read. Stakeholders want a chart, a filter, something they can actually share. ## What it does Turns raw data or query output into charts and dashboards. Supports modern code-based visualization libraries. Creates interactive reporting prototypes. Helps turn analysis into something easier to understand. ## When to use it WHEN a query result needs to become a dashboard WHEN prototyping internal analytics views WHEN telling a data story visually WHEN replacing a static table with an interactive interface ## How it works The skill picks a chart type, builds the components, and checks that the output actually communicates what the data shows. Tools used sql\_execute Run SQL against a configured warehouse and return a formatted result set. schema\_search Search indexed schemas, tables, and columns by keyword. Related skills FinOps Cost Report https://altimate.ai/skills/cost-report Validation Data Parity https://altimate.ai/skills/data-parity Team Learning Teach https://altimate.ai/skills/teach [← All skills & tools](https://altimate.ai/skills) --- # Teach URL: https://altimate.ai/skills/teach Show Altimate Code one good example file. It extracts the reusable pattern, asks you to confirm, and saves the pattern as training for future sessions. Team Learning ## Teach Altimate Code Your Team's Patterns From One Good Example Teach the AI your team's patterns from one good example. Show Altimate a strong example file and save the reusable pattern for future work. [Help the AI learn how your team already works. →](https://docs.altimate.sh/getting-started) The problem it solves Generic AI tools don't know your team's patterns: how you structure staging models, name CTEs, order columns, or format YAML. They have to learn them somehow. ## What it does Reads a good example file from the codebase. Extracts reusable patterns rather than copying content. Asks for confirmation before saving. Stores the pattern as training for future sessions. ## When to use it WHEN onboarding Altimate Code to a project WHEN a file demonstrates the style the team wants repeated WHEN replacing long instructions with concrete examples WHEN team conventions are implicit rather than documented ## How it works The skill reads the file, pulls out the structural patterns, shows you what it found for confirmation, and saves them so future sessions can apply them. Tools used training\_save Save learned patterns, rules, glossary terms, standards, context, or playbooks. training\_list List saved training entries with metadata and usage information. Related skills Team Learning Train https://altimate.ai/skills/train Team Learning Training Status https://altimate.ai/skills/training-status dbt dbt Docs https://altimate.ai/skills/dbt-docs [← All skills & tools](https://altimate.ai/skills) --- # Train URL: https://altimate.ai/skills/train Load your style guides, review checklists, glossary docs, and standards so the AI can apply them consistently. Team Learning ## Load Team Standards So the AI Applies Them Consistently Turn team standards into reusable AI behavior. Load your style guides, review checklists, glossary docs, and standards so the AI can apply them consistently. [Make team standards show up during real work. →](https://docs.altimate.sh/getting-started) The problem it solves Team standards often live in documents that people forget to apply. Pasting a guide into one chat does not make that knowledge persistent or reusable. ## What it does Reads standards documents or pasted guidance. Extracts actionable rules, glossary terms, patterns, and playbooks. Saves training entries for future sessions. Makes team knowledge easier for the AI to apply consistently. ## When to use it WHEN loading a SQL style guide WHEN importing a review checklist WHEN teaching glossary terms or business definitions AFTER incidents or corrections that should become reusable rules ## How it works The skill reads the document, groups what it finds by type (rules, glossary terms, patterns), shows you a preview, and saves after you confirm. Tools used training\_import Import team standards or documentation into training entries. training\_save Save learned patterns, rules, glossary terms, standards, context, or playbooks. training\_list List saved training entries with metadata and usage information. Related skills Team Learning Teach https://altimate.ai/skills/teach Team Learning Training Status https://altimate.ai/skills/training-status SQL Quality SQL Review https://altimate.ai/skills/sql-review [← All skills & tools](https://altimate.ai/skills) --- # Training Status URL: https://altimate.ai/skills/training-status List what Altimate Code has learned about your project, grouped into patterns, rules, glossary terms, and standards, so you can review or remove stale entries. Team Learning ## See Everything Altimate Code Has Learned About Your Project See what the AI teammate has learned. Display saved patterns, rules, glossary terms, standards, and training metadata. [Keep AI team knowledge visible and inspectable. →](https://docs.altimate.sh/getting-started) The problem it solves Once an AI starts learning team behavior, you need to see what it knows. Without a status view, saved knowledge is invisible — and invisible knowledge goes stale. ## What it does Lists saved training entries. Groups knowledge by patterns, rules, glossary, standards, context, and playbooks. Shows recent training and usage metadata. Helps teams review, remove, or refresh learned knowledge. ## When to use it AFTER using teach or train WHEN auditing what the AI has learned BEFORE onboarding teammates to an existing project WHEN cleaning up stale training entries ## How it works The skill fetches all saved training entries and shows them grouped by type: patterns, rules, glossary terms, standards, and playbooks. Tools used training\_list List saved training entries with metadata and usage information. training\_remove Remove outdated or incorrect training entries. Related skills Team Learning Teach https://altimate.ai/skills/teach Team Learning Train https://altimate.ai/skills/train Setup Altimate Setup https://altimate.ai/skills/altimate-setup [← All skills & tools](https://altimate.ai/skills) --- # Use Cases URL: https://altimate.ai/use-cases Six use cases, one platform. Build pipelines, migrate workloads, ship data apps, manage infrastructure, optimize costs, and enforce governance. PROOF POINTS ## The Altimate Platform at Work. Most teams start with one urgent problem — runaway Snowflake costs, a migration that's dragging, a pipeline that broke at 2am. What's yours? Build Data Pipelines [Cost Optimization on Snowflake](https://altimate.ai/use-cases/altimate-for-snowflake) [Databricks Cost Optimization](https://altimate.ai/use-cases/altimate-for-databricks) Migrate Workloads Build Data Apps Optimize Workloads BUILD DATA PIPELINES ## Describe the Pipeline You Need. Altimate Builds, Tests, and Documents It. Reliably build entire pipelines from a plain-language spec. Altimate generates SQL with the correct grain and joins, writes targeted tests, and produces documentation from context. Every pipeline passes blast radius analysis before your data engineers review and approve. MIGRATE WORKLOADS ## 400+ Informatica Pipelines in Under Three Weeks, with the Business Logic Intact and Contracts Validated. Altimate extracts business logic before migration begins. Decision Memory pulls Git history, PR comments, and architectural context into ADR-formatted traces. From these we produce dbt models that preserve compliance rules, business semantics, and data contracts. One customer migrated 400+ Informatica pipelines in under three weeks, with the business logic intact and contracts validated. [Learn more →](https://altimate.ai/use-cases/workload-migration) 01 ### Extract business logic Decision Memory pulls 3yr of Git history into ADR traces before migration begins. 02 ### Map transformations Agent maps Informatica/SSIS logic to dbt models preserving semantics and contracts. 03 ### Validate blast radius Every generated model checked against downstream dependencies before deployment. 04 ### Deploy with audit trail Full compliance-ready documentation generated automatically. BUILD DATA APPS ## Describe Your Data App. Altimate Generates the Backend, Frontend, and Data Layer. Altimate agents generate the full stack (pipeline, backend, frontend) from a plain-language spec. They scan your existing semantic layer to reuse models, generate the API layer, and produce frontend components wired to the correct data sources. Auth and IAM configuration is inherited from your existing setup. Every app passes blast radius analysis prior to your review. OPTIMIZE WORKLOADS ## 30% a Year in Average Savings, Out of the Gate. 01 ### Problem: runaway costs 3,000 queries, with no ownership traced. 02 ### Automated analysis Altimate scans your warehouse, identifies top savings opportunities. 03 ### Solution: auto-tuning Changes applied with blast radius cleared. Built on 3 years of Fortune 500 query patterns, Altimate scans your warehouse, identifies optimization opportunities, calculates blast radius for every proposed change, and implements with a full audit trail. Auto Tune agents right-size your job clusters and SQL warehouses; everything else surfaces as sized, owned opportunities. One customer achieved $2.8M in annual savings across 168 queries. Another deployed Auto-tune in under 3 weeks and saved $1.8M a year. Find out what it can do for you. [Cost Optimization on Snowflake →](https://altimate.ai/use-cases/altimate-for-snowflake) [Databricks Cost Optimization →](https://altimate.ai/use-cases/altimate-for-databricks) GET STARTED ## Pick the Use Case That Fits Today. Build the Platform That Lasts. The AI readiness scan shows you which use case has the highest immediate ROI for your stack — in under 5 minutes. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) --- # Databricks Cost Optimization & Pricing Model URL: https://altimate.ai/use-cases/altimate-for-databricks Cut Databricks costs 20–35% by right-sizing job clusters, SQL warehouses, and all-purpose compute automatically. No serverless migration required. Use Case ## Databricks Cost Optimization Job clusters are typically the largest line on a Databricks bill. Altimate's Auto Tune agents handle Databricks cost optimization by right-sizing them automatically, along with your SQL warehouses and compute. Every change is checked, approved by you, and rolls back automatically if anything slows down. A co-pilot identifies savings opportunities and routes them to the right owner to approve. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) 20–35% typical cost reduction 100+ saving opportunities surfaced <30 days to proven value StubHub Booking.com Peloton Workday Slack Siemens Healthineers 01 · Auto Tune ## Autonomous Compute Savings, SLA Safe Auto Tune reads how your job clusters, SQL warehouses, and all-purpose clusters actually run, then moves each to a cheaper setup on its own. The waste creeps in through default cluster templates that nobody revisits, and at hundreds or thousands of jobs, nobody can revisit them by hand. Auto Tune revisits them continuously. Approve it once; it applies between runs, monitors continuously, and rolls back the moment anything slows down. Jobs, warehouses, and all-purpose compute Saves both DBU and cloud costs SLA-safe, approval-gated, auto-reverts Every automated change runs the same loop. No exceptions, no surprises. The Auto Tune loop: analyze, recommend, gate, apply, watch, with an automatic rollback from watch back to apply 01 **Approval-gated** — you choose the jobs, warehouses, and clusters it may touch. 02 **Policy-checked** — changes clear your SLA rules before they apply. 03 **Atomic** — one snapshot, one change, applied only between runs. 04 **Watched** — continuous monitoring against the prior baseline. 05 **Reversible** — failure or degradation triggers automatic rollback. Watch a real pass: analysis, gate checks, the applied change, the watch window that follows, and the audit trail every event lands in. [Auto Tune demo](https://app.supademo.com/embed/cmrxkdpcx3y27qmblhlpoylfb?embed_v=2) ## Get Serverless Efficiency at Classic Compute Pricing. Serverless is often pitched as a better solution where compute requirements are unpredictable, but it trades over-provisioning for a premium rate under Databricks pricing. Altimate's Auto Tune for Databricks fixes the sizing directly, on the classic compute you already own, and shows you which workloads are genuinely ad hoc enough to run on serverless. Already running serverless SQL warehouses? Auto Tune tunes those too. Right-size job clusters, SQL warehouses, and all-purpose compute in place. Typical result: 20–35% lower spend. See which workloads are stable enough to stay on classic compute and which are ad hoc enough to justify serverless. Serverless SQL warehouses get auto-tuned as well, so the savings follow you either way. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) 02 · Spark & job intelligence ## Spark Waste, Found and Fixed Slow shuffles, oversized executors, steps that run one after another when they could run together. Altimate reads each run, prices it, and recommends the fix — so a job creeping up on cost is caught weeks before month-end, with a concrete change already in hand. Finds hidden Spark waste Alerts when cost or runtime jumps Recommends the fix, not just the flag A Spark run flagged with a 1.0 GB spill and a 6.4x skew ratio, above three recommendations: eliminate the disk spill in the sort-merge join, reduce data skew on the join key, and cut GC pressure in the hash aggregate — each with its cost and time savings Altimate reads each Spark run — here flagging a 1.0 GB spill and a 6.4× skew — then hands back concrete fixes: eliminate the disk spill, rebalance the skewed join key, ease GC pressure. Each with the cost and time it saves. 03 · AI/ML spend intelligence ## The Fastest-Growing Line on the Bill Is the Least Watched. Model serving, vector search, AI gateway, and Genie — the fastest-growing spend on the platform. Altimate attributes every dollar by service, team, and day, and flags endpoints serving nobody. AI/ML Services cost dashboard showing Model Serving, Vector Search, and AI Gateway spend over 28 days Every dollar of AI/ML spend attributed by service and by day across all workspaces. Teams finally know where the budget is going, instead of stitching it together from billing exports after the fact. 3 idle model-serving endpoints. 30 days. Zero requests. **$19,800/month** in invisible waste. **We flagged it. They fixed it. $240K saved annually. Zero disruption.** Opportunity card recommending decommissioning three idle model serving endpoints for roughly twenty thousand dollars per month $20K/month. Low effort. First detected 5 days ago. Altimate doesn't wait for a quarterly review to find invisible spend. It surfaces the opportunity, sizes the saving, and hands it to the right engineer to action. 04 · Discover ## Every Opportunity Sized, Owned, and Tracked to Done. Spot-instance misconfigurations, wrong instance families, notebooks parked on all-purpose clusters, resources idling for days. A co-pilot does the heavy lifting, turning each into an opportunity with a dollar value, an effort score, and a name next to it. Links straight to the resource Filter, group, and export Tracked until it's done Discover page listing Databricks cost opportunities with money savings, time savings, effort, status, and owner columns The Discover queue: each row carries money savings, time savings, effort, status, and an owner. From finding to fix: how an opportunity gets explained, assigned, and closed. [Discover opportunities demo](https://app.supademo.com/embed/cmqfy89me4wquqmgjdkf4u3z2?embed_v=2) 05 · Chargeback & forecast ## Know Which Team Spent What. See Next Quarter Early. Spend split by team first, then by workspace, SKU, product, and user, and projected forward. Each SKU carries its own rate in the Databricks pricing model, so the split shows which rates drive the bill. Every dollar rolls up to an owning team, so budget conversations happen before the invoice arrives, not after. Spend by team and user Broken down by SKU and product See this quarter and next Breakdown dashboard: total cost of $603K across six workspaces, with cost broken down by product — SQL, Jobs, all-purpose, model serving, DLT, and vector search — each with DBUs and percent of total Total spend split by workspace and by product — SQL, Jobs, all-purpose, model serving, and the rest — each ranked by share of the bill. Users dashboard ranking the top Databricks users and service principals by query cost, with DBUs, query counts, success rate, and average duration per user Top users and service principals ranked by query cost, with DBUs, query volume, and success rate. Chargeback stops being a spreadsheet exercise. Team sport ## Cost Optimization Is a Team Sport. Autonomous savings and the opportunities your team closes land on one Summary page — measured in dollars and engineer-hours, each with an owner. Cost optimization stops being only the platform team's job. AUTONOMOUS Auto Tune agents apply the change themselves. Gated, monitored, reversible. ASSISTED A co-pilot does the heavy lifting — sizing each opportunity in dollars, scoring the effort, and naming an owner. Your team gives the final go-ahead. Altimate Summary page showing autonomous and assisted savings totals with time savings for a Databricks account The Summary page ledger: autonomous and assisted savings tracked side by side, with time savings in engineer-hours. FAQ ## Questions, Answered. Doesn't Databricks already optimize costs? Native tools give you visibility and storage upkeep. Cost dashboards and budgets show Databricks cost, and Predictive Optimization maintains tables. None of them change a cluster or warehouse config. Altimate acts on compute, the part that dominates the bill, and every action carries a snapshot and an automatic rollback. What access does Altimate need? A scoped service principal with read-only metadata access for analysis. Write permission exists only on the specific jobs, warehouses, and clusters you enable Auto Tune for, and it changes configuration only: never code, schedules, or data. Unity Catalog is required. How are savings measured? Per run, from your billing data, at your contracted rate rather than list price. Realized savings appear next to each tuned job, warehouse, and cluster, and roll up to the Summary page ledger, which tracks the cost of Databricks next to what Auto Tune saved. What about serverless? Serverless removes cluster management, but Databricks serverless pricing carries a higher rate, so migrating over-provisioned workloads to serverless replaces one cost problem with another. The cheaper first step is to right-size the compute you already run: Auto Tune resizes job clusters, SQL warehouses, and all-purpose compute automatically, with typical reductions of 20 to 35%. Altimate then shows which workloads are spiky or ad hoc enough that serverless genuinely fits. And if you already use serverless SQL warehouses, Auto Tune optimizes those too. Why do Databricks job clusters end up over-provisioned? Engineers rarely know what a job needs before it runs, so they pick a default small, medium, or large template and move on. Those defaults are sized for the worst case, they are almost never revisited, and at hundreds or thousands of jobs no team has time to revisit them manually. Auto Tune reads how each job actually runs and resizes it between runs, approval-gated and rolled back automatically if runtime degrades past 1.5x its baseline. What happens if a change slows something down? Auto Tune watches every touched job, warehouse, and cluster continuously. A failed run, or runtime beyond 1.5 times the recent baseline, triggers Backoff: the prior configuration is restored automatically and the event lands in the history. How is Databricks DBU pricing calculated? A Databricks Unit (DBU) is the unit Databricks uses to meter compute. Your DBU cost is the number of DBUs a workload consumes multiplied by the per-DBU rate for its SKU, cloud, and pricing tier. A cluster that is larger than the job needs consumes more DBUs on every run, so right-sizing it lowers the DBU cost directly. How does Azure Databricks pricing work? On classic compute, Azure Databricks bills in two parts: DBUs at the Azure Databricks rate for the workload type, plus the Azure virtual machines, storage, and networking underneath. Both parts grow with cluster size, so an over-provisioned cluster raises Azure Databricks cost twice. Is there a Databricks cost calculator? Yes. Databricks publishes a pricing calculator, and the Azure pricing calculator covers Azure Databricks. Each one estimates cost from the configuration you plan to run. Altimate measures what each job, warehouse, and cluster actually cost per run, from your billing data, so you can compare the estimate with the real figure. SOC 2 Type II certifiedMetadata-only analysisScoped service principalNo training on your dataPer-account pricing, no per-seat fee > "Significant money savings on warehouses that were already optimized by us." > > Aniket, VP of Data, ThredUp ## Ready to Cut Your Databricks Bill? Run a 30-day proof of value. Watch the savings land on your own Summary page. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) Also running Snowflake? See our [Snowflake cost optimization overview](https://altimate.ai/use-cases/altimate-for-snowflake). --- # Snowflake Query & Cost Optimization URL: https://altimate.ai/use-cases/altimate-for-snowflake Optimize Snowflake costs and queries with AI-powered tools for adaptive compute, warehouse efficiency, and smarter data workloads while maintaining performance. Use Case ## Snowflake Cost Optimization AI agents that optimize every layer of your Snowflake environment, around the clock, without breaking your SLAs. Snowflake cost optimization spans warehouses, queries, tables, and the spend behind each team. [Book a Demo](https://calendly.com/d/cxsb-8h7-5z9/altimate-ai-for-snowflake-overview) [Start Free in Snowflake](https://altimate.ai/products/altimate-lite) 30% average annual savings $2.82M saved by one company, live in 3 weeks 3,562 engineer-hours reclaimed StubHub Booking.com Peloton Workday Slack Siemens Healthineers **Our autonomous agents offer trust with validation and prevention.** 01 · Warehouse auto-tune ## Warehouse Auto-Tune A deterministic engine analyzes your warehouse metrics thousands of times a day. AI models pick the right action, policy checks confirm it's SLA-safe, and the system executes, watches the outcome, and rolls back if needed. **Autonomously changes warehouse sizing, clustering, and shutdowns** without violating your SLAs. Autonomous warehouse sizing and suspension optimization dashboard 02 · Query optimization ## Query Optimization Altimate pinpoints the exact lines driving query cost, rewrites the SQL, and **validates the results against your data**. Snowflake query optimization comes with context from your pipelines and BI layer, and the fixes land as ready-to-merge PRs. ### Pinpoint problematic lines ### SQL optimized, data validated ### Context across pipelines & BI ### PR creation with fixes SQL code analysis highlighting key lines for optimization Cartesian joins, missing filter pushdowns, inefficient CTEs: flagged down to the exact line. AI-generated query optimization recommendations 03 · Hidden table costs ## Hidden Table Costs Storage is cheap. **The pipeline that builds and refreshes a table isn't.** Altimate's Snowflake cost optimization prices every table in full: storage, the compute behind it, and additional costs like failsafe and time-travel retention. Plus, agents analyze lineage to evaluate multiple related tables and pipelines together. Then the harder question: **is the table even read?** If not, stop the pipeline and drop the table. **Whole pipelines and table clusters disappear this way.** ### Clustering key recommendations ### Materialization suggestions ### Failsafe / time travel analysis Hidden table cost breakdown showing storage, compute, and failsafe costs 04 · Workload intelligence ## Workload Intelligence **Altimate tracks cost per workload**, not just per query. Workloads are defined by query tags, and our package automates the tagging. You get **automated RCA via comparison of multiple workload runs**, pinpointing exactly what changed: warehouse size, code, task failures, or data volume. Workload-level performance and cost analysis dashboard 05 · Shift-left prevention ## Shift-Left Prevention Our agents **identify and fix costly mistakes during development itself**: in your IDE, CLI, Git, and CI/CD. **The cheapest fix is the one that never ships.** IDE integration showing real-time SQL anti-pattern detection 06 · Team sport & chargeback ## Team Sport & Chargeback Each team sees its open savings opportunities. One click **assigns one to the right person**, with agent-generated code changes and commands attached, and follows up until it's done. **Showback and chargeback** are built in: Snowflake team cost attribution maps schemas, query tags, and infrastructure cleanly to each team, so every team owns its spend. **Cost optimization stops being only the platform team's job.** Team cost savings assignment and tracking interface Start self-serve with Altimate's Snowflake Native App ## Altimate Lite Is Free for Your First 21 Days [Explore Altimate Lite](https://altimate.ai/products/altimate-lite) [Install from Snowflake Marketplace](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) Altimate Lite auto-tunes your warehouses and itemizes your Cortex AI spend. It installs in five minutes and never sends anything outside your Snowflake account. Private-preview customers saved **19% on average**. Altimate Lite Warehouses dashboard showing 30-day total cost, potential annual savings, Auto Tune eligible and enabled counts, and a daily spend and savings chart. Altimate Lite reports spend, recoverable idle, and realized savings for every warehouse, inside Snowsight. Need auto-resize, Studio, assisted optimization, and chargeback? That's the enterprise platform above. FAQ ## Questions, Answered. How much can I save on Snowflake costs with Altimate? Altimate customers see 30% average annual savings on Snowflake compute. One company saved $2.82M and reclaimed 3,562 engineer-hours, live within 3 weeks of deployment. Snowflake team cost attribution maps that spend back to the teams and workloads that caused it, so each team sees its own savings opportunities. Does Snowflake cost optimization with Altimate require engineering effort? No. With Altimate's Snowflake cost optimization, a deterministic engine analyzes warehouse metrics thousands of times a day, AI models pick the right action, policy checks confirm SLA safety, and the system executes and monitors the outcome autonomously, rolling back if needed. How does Warehouse Auto-Tune relate to Snowflake Adaptive Compute? They optimize different things, so they are not substitutes. Snowflake Adaptive Compute removes the settings: an Adaptive Warehouse points at a shared compute pool, and Snowflake selects the cluster size, the number of clusters, and the auto-suspend duration for you. It is positioned on simplicity and performance rather than savings, and it leaves one cost lever, the credit limit, so you can no longer route a low-priority report onto smaller compute. Warehouse Auto-Tune keeps your standard warehouses and those levers, then moves them against predicted workload with SLA policy checks, backoff thresholds, and rollback, logging the reason for every decision and the saving it produced. Because an Adaptive Warehouse is Snowflake-managed, Auto Tune does not tune it: there is no size, cluster, or auto-suspend setting left to move. Most teams run both, with Adaptive on the warehouses whose usage is unpredictable and Auto Tune on the ones that have a cost target to hit. We compare the two in detail in [Adaptive Compute vs. Auto Tune](https://altimate.ai/blog/adaptive-compute-vs-auto-tune-a-practical-guide-to-optimizing-snowflake-warehouses). How does Altimate optimize Snowflake queries? Altimate pinpoints the exact lines driving query cost, rewrites the SQL, validates the results against your data, and opens the fix as a ready-to-merge pull request with context from your pipelines and BI layer. Before a query runs, the Snowflake query cost estimate shows which queries will drive the most compute spend. For AI-assisted SQL work, see how [Altimate compares with Snowflake Cortex Code](https://altimate.ai/comparisons/snowflake-cortex). What are hidden table costs and how does Altimate find them? Hidden table costs include the compute behind building and refreshing a table, plus failsafe and time-travel retention, not just storage. Altimate prices every table in full, checks whether it's actually read, and identifies whole pipelines and table clusters that can be dropped. The same per-table cost feeds Snowflake team cost attribution, so each team sees what its data assets cost. Can Altimate prevent costly Snowflake mistakes before they ship? Yes. Altimate's agents identify and fix costly SQL anti-patterns during development itself, inside your IDE, CLI, Git, and CI/CD, before the cost ever hits production. Can Altimate estimate what a Snowflake query will cost before it runs? Yes. The Snowflake query cost estimate arrives before execution rather than after it: the input is the query or dbt model as written, and the output is a dollar figure with a risk level attached, which can be checked against a threshold and blocked in CI. It is priced by the same cost model as your invoice, which meters each query hourly, merges the warehouse's real credits for that hour, and splits them by execution time, so idle warehouse credits land on the queries that held the warehouse open. Published accuracy is 85% on cost optimization, which the [Enterprise Platform](https://altimate.ai/platform) page carries, and 100% F1 on 1,077 queries for anti-pattern classification. Can Altimate optimize costs across both Snowflake and Databricks? Yes. Altimate provides unified cost optimization across both platforms — see the [Databricks cost optimization page](https://altimate.ai/use-cases/altimate-for-databricks) for Databricks-specific coverage. How does Altimate Lite, our Snowflake Native App, differ? [Altimate Lite](https://altimate.ai/products/altimate-lite) covers two of the capabilities on this page: warehouse Auto Tune, which suspends idle warehouses and scales clusters down, and AI cost visibility, which itemizes Cortex spend by service, user, and model. It runs entirely inside your Snowflake account, so nothing leaves it. The enterprise platform adds the rest — warehouse auto-resize, query and table optimization, workload intelligence, shift-left prevention in your IDE and CI, Altimate Studio, and showback and chargeback — and covers Databricks alongside Snowflake. Can I start on Snowflake without a sales call? Yes. You install [Altimate Lite](https://altimate.ai/products/altimate-lite) from the Snowflake Marketplace in under five minutes with ACCOUNTADMIN access. It is free for 21 days, then $100/month plus 2.5% of the daily cost, after savings, of each warehouse you enable Auto Tune on. You only pay for the warehouses you turn on, and billing runs through Snowflake, so it can be paid with pre-purchased credits. > "Altimate's Query Insights agent serves as an expert for SQL queries across our data and business teams. Our AI co-pilot has directly enabled individual users to write better queries, resulting in real cost savings in Snowflake." > > Ben Goswami, Director of Engineering – Data/ML, Heyday ## Ready to Cut Your Snowflake Bill? 30-day Proof of Value. See your savings live. [Book a Demo](https://calendly.com/d/cxsb-8h7-5z9/altimate-ai-for-snowflake-overview) [Start Free in Snowflake](https://altimate.ai/products/altimate-lite) Also running Databricks? See our [Databricks cost optimization overview](https://altimate.ai/use-cases/altimate-for-databricks). --- # Analytics Engineering URL: https://altimate.ai/use-cases/analytics-engineering The dbt development app for analytics engineers. Generate dbt models, write docs, create tests, and enforce best practices from your IDE. 1M+ installs. USE CASE ## Analytics Engineering The dbt Development Acceleration app brings agent-powered dbt™ development to analytics engineers. Generate dbt™ models from SQL, write documentation, create tests, visualize lineage, and enforce best practices — all without leaving your IDE. 1M+ installs · Rated 5.0/5.0 on the VS Code Marketplace [Install Power User for dbt™](https://marketplace.visualstudio.com/items?itemName=innoverio.vscode-dbt-power-user) [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) "I would lose countless hours at my job without this tool." IA Isaak Anderson VS Code Marketplace "Has become an absolutely indispensable tool for developing." CM Carl Muller VS Code Marketplace "Improve a lot of the productivity." GS Guilherme da Silva VS Code Marketplace "Query / compile preview are great QOL features that I use often." MW Michael Weikman VS Code Marketplace "Viewing upstream / downstream models provides a lot of context." DL Daniel Ladd VS Code Marketplace "Love the functionality to drill into model references." AA Anthony Alvarez VS Code Marketplace "Run parent / child models with a click of a button." GC Graham Carman VS Code Marketplace "Best tool I have used for VS Code." BG Boris Gracevic VS Code Marketplace "Makes development so much easier." JR Juan Ramos VS Code Marketplace "Lots of great QOL features." JM Jacon Matson VS Code Marketplace "I would lose countless hours at my job without this tool." IA Isaak Anderson VS Code Marketplace "Has become an absolutely indispensable tool for developing." CM Carl Muller VS Code Marketplace "Improve a lot of the productivity." GS Guilherme da Silva VS Code Marketplace "Query / compile preview are great QOL features that I use often." MW Michael Weikman VS Code Marketplace "Viewing upstream / downstream models provides a lot of context." DL Daniel Ladd VS Code Marketplace "Love the functionality to drill into model references." AA Anthony Alvarez VS Code Marketplace "Run parent / child models with a click of a button." GC Graham Carman VS Code Marketplace "Best tool I have used for VS Code." BG Boris Gracevic VS Code Marketplace "Makes development so much easier." JR Juan Ramos VS Code Marketplace "Lots of great QOL features." JM Jacon Matson VS Code Marketplace RESOURCES Documentation [Open](https://docs.myaltimate.com) Tutorials Learn and practice [Open](https://www.youtube.com/@altimateai) GitHub Interested in contributing or reporting a bug? Join us on GitHub. [Open](https://github.com/AltimateAI/vscode-dbt-power-user) Slack community Get help from users in the "tools-dbt-power-user" channel in the dbt™ Slack community. [Join](https://bit.ly/agentic-data-engineering-slack) Model generation ## Generate dbt™ Models and Translate SQL. - Generate dbt™ models from SQL or source files - Update dbt™ models in natural language - Translate queries from one SQL dialect to another Generating dbt models from SQL in VS Code Documentation ## Document Tables, Columns, and dbt™ Models. - Generate descriptions for tables, columns, and dbt™ models - Support for different personas and languages - Enable AI-enhanced collaboration for documentation Auto-generating documentation for dbt models and columns Testing & lineage ## Generate Tests, Column Lineage, and Query Explanations. - Create and update tests seamlessly - Get query explanations and visualization - Impact analysis with real-time column lineage Generating dbt tests and viewing column lineage Governance ## Advise on Best Practices and Organizational Governance. - Define custom guidelines in SaaS - Get assistance and validations in IDE, Git, and CI/CD - Smart complete Jinja and SQL code in the IDE Best practices enforcement and governance in VS Code GET STARTED ## dbt Development Acceleration Join hundreds of data teams using Altimate AI to move faster, write better dbt™ code, and ship more reliable pipelines. [Install Free Extension](https://marketplace.visualstudio.com/items?itemName=innoverio.vscode-dbt-power-user) [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) --- # Workload Migration URL: https://altimate.ai/use-cases/workload-migration Migrate workloads 10 to 15 times faster with zero business logic lost. The Data Pipelines Migration app handles SQL transpilation, rebuilds, and validation. USE CASE ## Workload Migration Migrate workloads 10–15× faster. With zero business logic left behind. The Data Pipelines Migration app handles the hard parts of platform migration — SQL transpilation, business logic preservation, pipeline rebuilding, and validation — so your team ships in weeks, not months. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) 10–15× faster migration projects 100% business logic preserved 1M+ total installs Days not months, to first query Trusted by leading data teams StubHub Coinbase Booking.com Peloton Workday Capgemini Slack Siemens Healthineers Oodrive Explorium Outreach Elastic Chatbooks Ooni World of Books Clicksign Qover Carta DAZN Ritchie Bros. StubHub Coinbase Booking.com Peloton Workday Capgemini Slack Siemens Healthineers Oodrive Explorium Outreach Elastic Chatbooks Ooni World of Books Clicksign Qover Carta DAZN Ritchie Bros. SQL Transpilation ## SQL Transpiled Automatically, Dialect by Dialect. Altimate understands the SQL dialect of your source platform, whether moving off Redshift, Teradata, BigQuery, or legacy on-prem warehouses. It produces syntactically correct, semantically equivalent SQL in your target environment's dialect. Unlike generic LLM rewrites, Altimate maintains a Knowledge Graph which understands the upstream and downstream dependencies of every query before touching a single line of code. Source → Target Dialect Teradata SQL → Snowflake SQL Redshift → Databricks SQL BigQuery → Snowflake SQL Validated translation, not a best-guess rewrite Business Logic ## Preserve Business Logic: Every Transformation, Every Edge Case. The biggest complexity in any migration is the business logic buried in stored procedures, UDFs, and bespoke transformation patterns that took your team years to build. Altimate extracts, documents, and faithfully recreates that logic in the target environment, surfacing every non-trivial translation for engineer review before it goes anywhere near production. Every decision is stored in Decision Memory: what was migrated, how it was translated, and why — so your team has a full audit trail from day one. Logic Migration Coverage Standard SQL queries 98% Stored procedures & UDFs 91% Complex transformation patterns 85% Scheduling & orchestration logic 88% All non-trivial translations surfaced for engineer review Pipeline Modernization ## Don't Just Migrate Pipelines. Modernize Them. Migration is the perfect moment to retire technical debt. Altimate translates existing pipelines, and rebuilds them as clean, tested, documented dbt models, turning years of accumulated spaghetti SQL into a structured, maintainable data layer your whole team can work in. Source-to-target mappings, lineage graphs, and data contracts are generated automatically and kept in sync throughout the migration, so nothing falls through the cracks as you cut over workload by workload. Migration + Modernization, together 1 Ingest — scan all source queries, pipelines, and schedules 2 Translate — SQL transpiled and logic extracted automatically 3 Rebuild — restructured as clean dbt™ models with docs & tests 4 Validate — row-count and output parity checks before cutover Migration Planning ## A Migration Plan Built from Your Actual Workloads, Not a Spreadsheet. Most migration projects stall because no one has a clear picture of what exists, what it depends on, and what order things need to move. Altimate automatically inventories every query, pipeline, table, and job in your source environment, maps their dependencies, and produces a sequenced migration plan that respects the order in which things need to move. Business-critical workloads are prioritized. Dormant or unused pipelines are flagged for decommissioning. Automated Workload Inventory Ready to migrate 62% Needs review 21% Decommission (never used) 17% Don't pay to migrate your technical debt Validation ## Zero Surprises on Cutover Day. Altimate runs automated parity checks between source and target outputs throughout the migration so discrepancies are caught and resolved days before cutover, not the morning after. For each workload that passes validation, a confidence score is recorded in the Knowledge Graph. Your team has a clear, evidence-based view of exactly which workloads are ready to cut over and which still need attention. Migration Readiness Dashboard Row count parity ✓ 100% Column distribution parity ✓ 100% Business metric parity 94% Performance baseline met 87% Evidence-based cutover readiness, not gut feel Team Collaboration ## Migration Progress Your Whole Team Can See and Act On. Large migrations involve dozens of engineers, pipeline owners, and data consumers, each responsible for different pieces of the environment. Altimate provides tailored views to every role: platform engineers see infrastructure readiness, SQL developers see their query migration queue, and data consumers see which of their dashboards and reports will be affected. With a single click, any migration task can be assigned to the right person with change instructions already prepared. Tasks are tracked to completion, and nothing falls through the cracks between sprints. Role-Based Migration Views Platform Engineer SQL Developer Data Consumer ✓ Assigned — orders\_daily pipeline → Sarah K. ✓ In review — revenue\_summary UDF → James T. ! Needs owner — 3 pipelines unassigned Post-Migration ## Migrate to Lower Costs, Not Just a New Tech. Most teams discover, months after cutover, that they've recreated the same inefficient patterns on their new platform. They now are paying new prices for old problems. Altimate prevents this by applying warehouse cost optimization automatically as migrated workloads land in production. Query patterns are analyzed from the moment they begin running on the new platform. Warehouse configurations are auto-tuned to the actual workload profile. And cost anomalies are caught before they appear on the next invoice. Migrate and optimize in a single continuous motion. Cost trajectory after migration Week 1 — baseline established 100% Week 2 — auto-tune active 82% Month 1 — query optimization 70% Month 3 — steady state 58% Arrive at lower costs, not just a new platform GET STARTED ## Data Pipelines Migration The Data Pipelines Migration app is ready to get to work — migrating your workloads faster, cleaner, and at lower cost than any approach you've tried before. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) [Get Started Free](https://app.myaltimate.com/register) --- # The AI-Ready Data Engineer URL: https://altimate.ai/ai-guide A practical guide for data engineers adopting AI — covering core concepts, infrastructure decisions, and hands-on strategies to build AI-ready data platforms. ## The AI-Ready Data Engineer A practical guide for data engineers adopting AI — covering core concepts, infrastructure decisions, and hands-on strategies to build AI-ready data platforms. 1 ## Why AI Matters for Data Engineers Data engineering is becoming the foundational layer every AI initiative depends on. Understand the shift from executor to orchestrator, and why it matters now. 10 min read https://altimate.ai/ai-guide/why-ai-matters 2 ## The 48-Hour Crash Course Learn AI essentials in a weekend. Build your first RAG app and get to grips with LLMs, prompt engineering, AI agents, vector databases, and MCP. Weekend https://altimate.ai/ai-guide/48-hour-crash-course 3 ## The One-Month Program Build production AI skills in four weeks: advanced RAG, multi-agent frameworks with LangGraph and CrewAI, LLMOps, and fine-tuning with LoRA and QLoRA. 1 Month https://altimate.ai/ai-guide/one-month-program 4 ## The Three-Month Advanced Track For the ambitious data engineer. Cover research papers, multimodal systems, deployment patterns, evaluation frameworks, and reasoning models like DeepSeek-R1. 3 Months https://altimate.ai/ai-guide/three-month-advanced-track 5 ## Resources and Milestones Curated AI resources for data engineers: top courses from DeepLearning.AI and Fast.ai, essential GitHub repos, YouTube channels, communities, and docs. Reference https://altimate.ai/ai-guide/resources ·· ## Closing thoughts Where to start, what to build in the first week, and the infrastructure question your team will be answering by 2027. 5 min read https://altimate.ai/ai-guide/final-thoughts --- # Chapter 1: Why AI Matters for Data Engineers URL: https://altimate.ai/ai-guide/why-ai-matters Data engineering is becoming the foundational layer every AI initiative depends on. Understand the shift from executor to orchestrator, and why it matters now. [All chapters](https://altimate.ai/ai-guide) The AI-Ready Data Engineer ## Chapter 1: Why AI Matters for Data Engineers Data engineering is becoming the foundational layer every AI initiative depends on. Understand the shift from executor to orchestrator, and why it matters now. ## A practical path This is a collection of resources gathered over the past year: things that have been actually read, experimented with, and found genuinely useful. No fluff, just stuff that works. But before diving into the resources, it's worth taking a moment to understand why this matters, not just for your career, but for the role data engineering plays in the AI era. ## Why data engineers should care Here's the real story: data engineering isn't adapting to AI. It's becoming the foundation that AI depends on. Every domain in the enterprise is rebuilding its operating model around AI: finance, customer service, HR, operations, legal. Every single one of those initiatives runs through the data team. The data engineer is no longer the person who keeps the pipelines running so the BI team can build dashboards. They're the prerequisite that determines whether the company's AI strategy succeeds or fails. The numbers confirm how fast this is moving. Databricks [reported in early 2026](https://www.databricks.com/resources/ebook/state-of-ai-agents) that more than 80% of new databases on their platform are now being created by AI agents rather than human engineers. The shift from executor to orchestrator isn't a 2027 prediction. It's already happening. The job is shifting from *executor* to *orchestrator*. The data engineer of 2027 doesn't just build pipelines. They design the systems and governance models that let agents do that work safely, and make the judgment calls agents can't. Teams that recognize this moment can become the accelerators of their organization's AI strategy. Teams that don't will become bottlenecks and rapidly replaced. ## Why AI agents for data fail in production Here's the problem most teams don't see coming. You build a capable agent, connect it to your data estate, and it starts producing confident, well-structured outputs that are quietly wrong. Not wrong because the model is bad. Wrong because the agent is filling gaps in documentation with plausible inference and acting on those inferences at machine speed. This isn't a fringe problem. [MIT's 2025 GenAI Divide study](https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/) found that 95% of enterprise GenAI pilots are failing to deliver measurable business impact, with data quality cited as a primary cause. Gartner puts it more bluntly: 63% of organizations don't have the data management practices needed for AI, and they predict 60% of AI projects will be abandoned through 2026 if unsupported by AI-ready data. A human engineer navigating an unfamiliar part of the codebase slows down. They ask questions. They check Slack history. An agent does none of that. It fills the gap and moves on. The failure mode isn't caution. It's confident hallucination. This is the problem of **context debt**: the accumulated gap between what your data estate actually means (the decisions that shaped it, the rules it encodes, the dependencies it carries) and what has been explicitly documented and made machine-readable. It builds up from every PR that described *what* changed without saying *why*, every engineer who left without transferring their knowledge, every convention that was just "the way we do things around here" but never got written down. Building for agents requires a fundamentally higher standard of contextual completeness than building for humans. What was good enough for your team, like a doc that's 18 months out of date or a model with no lineage documentation, is a live failure mode for an agent. Recognising this early is the difference between AI initiatives that compound and ones that quietly stall. This is exactly the problem that purpose-built agentic harnesses aim to solve. [Altimate Code](https://github.com/AltimateAI/altimate-code) is an open-source example: it ships with 100+ tools across 10 warehouses, built specifically so that agents operate with the organizational context, governance guardrails, and data stack awareness that generic agent frameworks lack. It's worth exploring as a reference for what "context-aware" agent infrastructure looks like in practice. ## What this guide covers This covers all the hot topics: AI agents, RAG, MCP, vector databases, and more. Everything here is from 2023–2025, so it's all current. This won't make you an expert overnight (nothing will), but it'll get you from "what's RAG?" to building real production systems. The 48-hour section gets you started, and the 3-month path takes you pretty far if you stick with it. The field of AI is evolving in dog years, and new developments are landing every week. With that in mind, we'll do our best to keep this guide current, so check back regularly and expect things to change. ## How to use this guide It's organized by timeframe, but honestly, jump around based on what interests you. The 48-hour section is great for getting started quickly, while the deeper stuff can wait until you're ready. - [Chapter 2: The 48-Hour Crash Course](https://altimate.ai/ai-guide/48-hour-crash-course) — A weekend crash course to get your bearings - [Chapter 3: The One-Month Program](https://altimate.ai/ai-guide/one-month-program) — Building real production skills - [Chapter 4: The Three-Month Advanced Track](https://altimate.ai/ai-guide/three-month-advanced-track) — For the truly ambitious - [Chapter 5: Resources and Milestones](https://altimate.ai/ai-guide/resources) — All the links in one place [Back to Guide Overview](https://altimate.ai/ai-guide) [Next 2\. The 48-Hour Crash Course](https://altimate.ai/ai-guide/48-hour-crash-course) --- # Chapter 2: The 48-Hour Crash Course URL: https://altimate.ai/ai-guide/48-hour-crash-course Learn AI essentials in a weekend. Build your first RAG app and get to grips with LLMs, prompt engineering, AI agents, vector databases, and MCP. [All chapters](https://altimate.ai/ai-guide) The AI-Ready Data Engineer ## Chapter 2: The 48-Hour Crash Course Learn AI essentials in a weekend. Build your first RAG app and get to grips with LLMs, prompt engineering, AI agents, vector databases, and MCP. ## Day 1: Morning — Core concepts Get your bearings (4 hours) Start with Chip Huyen's ["LLM Engineering in Production"](https://huyenchip.com/2023/04/11/llm-engineering.html) blog. It's a reality check on what actually matters in production. Then watch Andrej Karpathy's ["Deep Dive into LLMs like ChatGPT"](https://youtube.com/watch?v=7xTGNNLPyMI) on YouTube (3h31m, February 2025). It covers the full modern stack including RLHF, reasoning models, and tool use. His follow-up ["How I Use LLMs"](https://youtube.com/watch?v=EWvNQjAaOHw) is worth watching after, for the practical workflows. For prompt engineering, Google has a surprisingly good [guide](https://developers.google.com/machine-learning/resources/prompt-eng). The [DAIR.AI Prompt Engineering Guide](https://github.com/dair-ai/Prompt-Engineering-Guide) on GitHub is also solid. Just skim the quick start. ## Day 1: Afternoon — RAG basics Retrieval-Augmented Generation basics (4 hours) AWS has a decent ["What is RAG?"](https://aws.amazon.com/what-is/retrieval-augmented-generation/) explainer. After that, check out the free [DeepLearning.AI RAG overview video](https://www.deeplearning.ai/courses/retrieval-augmented-generation-rag/) It's actually pretty good. For hands-on stuff, Pixegami has an excellent [tutorial on GitHub](https://github.com/pixegami/rag-tutorial-v2). You'll build your first RAG app end-to-end. Good learning experience. ## Day 1: Evening — AI agents Understanding AI Agents (4 hours) LangChain's [agent tutorial](https://docs.langchain.com/oss/python/langchain/agents) is the best starting point. You'll build a ReAct agent that can actually use tools. Pretty cool when you see it work for the first time. IBM has a good [explainer on ReAct agents](https://www.ibm.com/think/topics/react-agent) if you want the theory. Then watch Sam Witteveen's "Master CrewAI Tutorial" on YouTube. He's good at explaining multi-agent concepts and CrewAI is useful for multi-agent systems. If you want something more production-oriented from day one, the [OpenAI Agents SDK](https://openai.github.io/openai-agents-python/) is worth a look alongside LangChain. It's more opinionated about structure, which turns out to be a feature when you're productionizing. ## Day 2: Morning — Vector databases Vector databases (4 hours) Shakudo published a [March 2026 comparison of the top 9 vector databases](https://www.shakudo.io/blog/top-9-vector-databases), including pgvector, which is worth reading first — it covers the tradeoffs honestly and includes the "just use Postgres" option. Then just pick one (Pinecone or Qdrant are good starting points) and do their quickstart: [Pinecone](https://docs.pinecone.io/docs/quickstart), [Qdrant](https://qdrant.tech/documentation/quick-start/). [Daily Dose of DS](https://www.dailydoseofds.com/) has a practical RAG course that's worth checking out too. ## Day 2: Afternoon — Model Context Protocol Model Context Protocol (4 hours) Model Context Protocol is Anthropic's standard for how AI agents connect to external tools and data sources. The "USB-C port for AI" analogy is apt: one protocol, any tool. Check out the [introduction](https://modelcontextprotocol.io/docs/getting-started/intro) and DataCamp's [tutorial on using it with Claude Desktop](https://www.datacamp.com/tutorial/mcp-model-context-protocol). The [MCP servers repository](https://github.com/modelcontextprotocol/servers) has pre-built integrations for most tools you're already using. For a hands-on example of MCP applied to data engineering, explore [Altimate Code](https://github.com/AltimateAI/altimate-code), an open-source agentic harness that uses MCP to connect AI agents to dbt, SQL, and cloud warehouses with 100+ pre-built tools. MCP solves the connectivity layer: with 10,000+ public servers, any agent can now reach any tool through a standardized interface. What it doesn't solve is the organizational context layer. The agent can call the function, but it doesn't know why your team made the architectural decisions it did, which downstream models depend on the output, or what a schema change will cost against your specific query patterns. Every MCP harness hits this wall eventually. It's not a protocol gap. It's an organizational knowledge gap, and it compounds as your data estate grows. ## Day 2: Evening — Production basics Production basics (4 hours) ZenML compiled [1,200 LLMOps production deployments](https://www.zenml.io/blog/what-1200-production-deployments-reveal-about-llmops-in-2025) and published the patterns. Skim the summary for what's actually working in production. Then set up [Helicone](https://www.helicone.ai/) for monitoring (literally one line of code for cost tracking). Then deploy your RAG app with a [Streamlit](https://streamlit.io/) UI for demos. ## Resources to bookmark - [Ben's Bites newsletter](https://www.bensbites.co/) — best daily AI roundup - [r/LocalLLaMA subreddit](https://reddit.com/r/LocalLLaMA) — where the real discussions happen - [DataTalks.Club Slack](https://datatalks.club/slack.html) — great community, super helpful - [Hugging Face](https://huggingface.co/) — model hub and datasets - [Anthropic Claude docs](https://platform.claude.com/docs/en/home) — excellent MCP documentation - [Ollama](https://ollama.com/) — run LLMs locally on your machine [Previous 1\. Why AI Matters for Data Engineers](https://altimate.ai/ai-guide/why-ai-matters) [Next 3\. The One-Month Program](https://altimate.ai/ai-guide/one-month-program) --- # Chapter 3: The One-Month Program URL: https://altimate.ai/ai-guide/one-month-program Build production AI skills in four weeks: advanced RAG, multi-agent frameworks with LangGraph and CrewAI, LLMOps, and fine-tuning with LoRA and QLoRA. [All chapters](https://altimate.ai/ai-guide) The AI-Ready Data Engineer ## Chapter 3: The One-Month Program Build production AI skills in four weeks: advanced RAG, multi-agent frameworks with LangGraph and CrewAI, LLMOps, and fine-tuning with LoRA and QLoRA. ## Week 1: Advanced RAG techniques Check out NirDiamant's [RAG\_Techniques repo](https://github.com/NirDiamant/RAG_Techniques) on GitHub. It's a goldmine. The guy implemented every RAG technique that matters: HyDE, Multi-Query, Reranking, all with working code. [DeepLearning.AI's RAG course](https://www.deeplearning.ai/courses/retrieval-augmented-generation-rag/) is worth the time. They use real datasets and show you how to use [Phoenix](https://phoenix.arize.com/) for monitoring (important for cost management). For vector databases, actually benchmark them with YOUR data. Pinecone vs Weaviate vs Qdrant: they all have trade-offs. Spend time on chunking strategies. This makes or breaks your RAG system. [RAGAS](https://github.com/explodinggradients/ragas) is the go-to for evaluation. Set it up early, thank me later. ## Week 2: Agent frameworks and systems LangGraph fundamentals Harrison Chase (LangChain founder) teaches the [DeepLearning.AI course on LangGraph](https://www.deeplearning.ai/short-courses/ai-agents-in-langgraph/). The [LangChain Academy](https://academy.langchain.com/) has free courses too. LangGraph lets you build agents with actual memory and state, a significant improvement over stateless approaches. LangGraph 1.0 shipped in October 2025 and is now production-ready: durable execution (agents resume after server restarts), first-class streaming, human-in-the-loop pausing, and persistent memory management are all stable. Multi-agent systems [CrewAI](https://github.com/crewAIInc/crewAI) is interesting for multi-agent systems. You create agent teams with roles like "researcher" and "writer" and they actually collaborate. Microsoft's [AutoGen](https://github.com/microsoft/autogen) is more enterprise-y but worth exploring. Note that AutoGen has split: Microsoft shipped a full architectural rewrite (v0.4, actor model) in January 2025, while the original creators forked it as [AG2](https://github.com/ag2ai/ag2), which maintains backward compatibility with 0.2. Pick based on whether you need stability or the newer distributed architecture. For data engineering specifically, [Altimate Code](https://github.com/AltimateAI/altimate-code) takes a different approach: rather than a general-purpose agent framework, it's a domain-specific harness with pre-built tools for dbt, SQL, and warehouse operations. Worth comparing against the general frameworks to see the trade-offs between flexibility and out-of-the-box data awareness. NirDiamant's [GenAI\_Agents repo](https://github.com/NirDiamant/GenAI_Agents) has production examples with proper error handling and state management. Study these before you build anything serious. As you start building agents against real data estates, you'll hit a wall that has nothing to do with the framework you chose. Agents produce confident but wrong outputs when they lack the organizational context behind your data: why a model was built a certain way, which downstream dashboards depend on it, what compliance constraints apply. This is context debt in action. The frameworks here teach you how to build capable agents; the hard work is making sure those agents have the context they need to act safely. ## Week 3: LLMOps and production deployment Monitoring and observability tools LangSmith vs Helicone vs LangFuse: try them all. [Helicone](https://www.helicone.ai/) is easiest to start with (one line integration), [LangSmith](https://www.langchain.com/langsmith) has the best debugging tools, [LangFuse](https://langfuse.com/) is open source. If evaluation is your primary concern, [Braintrust](https://www.braintrust.dev/) has emerged as a strong dedicated eval platform — worth a look if you want something purpose-built for testing rather than tracing. [MLflow for LLMs](https://mlflow.org/) — experiment tracking and model registry. Prompt optimization [DSPy](https://github.com/stanfordnlp/dspy) from Stanford automates prompt optimization. It's like hyperparameter tuning but for prompts. Innovative approach to prompt engineering. [Learn Prompting](https://learnprompting.org) has interactive modules that are actually fun. Cost optimization This is where you save your company thousands. Implement caching (often 50% cost reduction), use smaller models where possible, batch requests. [Neptune.ai](https://neptune.ai/blog/llmops) has a solid LLMOps guide. ## Week 4: Fine-tuning on a budget LoRA and QLoRA techniques These techniques let you fine-tune huge models on normal GPUs. The [QLoRA paper implementation](https://github.com/artidoro/qlora) is the reference. [Unsloth](https://github.com/unslothai/unsloth) has become the default starting point for efficient fine-tuning: 2x faster training, 70% less VRAM, works with LoRA and QLoRA across Llama, DeepSeek, and Qwen models. Start here before reaching for the heavier tools. [Axolotl](https://github.com/axolotl-ai-cloud/axolotl) makes fine-tuning almost point-and-click. [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) has a web UI for those who prefer graphical interfaces. What you'll have built - Multiple RAG system implementations - Multi-agent systems with proper error handling - A fine-tuned model for your specific use case - Production monitoring and debugging setup [Previous 2\. The 48-Hour Crash Course](https://altimate.ai/ai-guide/48-hour-crash-course) [Next 4\. The Three-Month Advanced Track](https://altimate.ai/ai-guide/three-month-advanced-track) --- # Chapter 4: The Three-Month Advanced Track URL: https://altimate.ai/ai-guide/three-month-advanced-track For the ambitious data engineer. Cover research papers, multimodal systems, deployment patterns, evaluation frameworks, and reasoning models like DeepSeek-R1. [All chapters](https://altimate.ai/ai-guide) The AI-Ready Data Engineer ## Chapter 4: The Three-Month Advanced Track For the ambitious data engineer. Cover research papers, multimodal systems, deployment patterns, evaluation frameworks, and reasoning models like DeepSeek-R1. ## Month 1: Deep technical learning Seminal papers: the ones to actually read - [Scaling LLM Test-Time Compute](https://openai.com/index/learning-to-reason-with-llms/) (OpenAI's o1 approach) — introduced the reasoning model paradigm that defined 2025 - [The Prompt Report](https://arxiv.org/abs/2406.06608) — the most comprehensive survey of prompting techniques; still the reference - [BitNet b1.58](https://arxiv.org/abs/2402.17764) — seminal for the 1-bit quantization and on-device efficiency direction For everything published since, Sebastian Raschka maintains the best curated reading list: [2025 H1](https://magazine.sebastianraschka.com/p/llm-research-papers-2025-list-one) and [2025 H2](https://magazine.sebastianraschka.com/p/llm-research-papers-2025-part2), organized by reasoning, inference-time scaling, architectures, and training efficiency. The [Latent.Space 2025 AI Engineering Reading List](https://www.latent.space/p/2025-papers) is a good practitioner-oriented companion. Don't just read. Implement. [GraphRAG from Microsoft](https://github.com/microsoft/graphrag) is a great starting point. BitNet 1.58-bit quantization is worth exploring (1-bit models that actually work). Multimodal systems NVLM and Gemini 2.0 architectures are where things are heading. Build an image + text RAG system. Add voice if you're feeling adventurous. [Fast.ai's Part 2 course](https://course.fast.ai/Lessons/part2.html) has you implementing Stable Diffusion from scratch. Challenging but educational. Also check out Karpathy's ["Neural Networks: Zero to Hero"](https://karpathy.ai/zero-to-hero.html) course. ## Month 2: Production excellence Production deployment patterns NirDiamant's [agents-towards-production repo](https://github.com/NirDiamant/agents-towards-production) covers everything: Docker, FastAPI, GPU scaling, security. This is the difference between a demo and something that actually ships. Evaluation and testing frameworks [OpenAI's Evals framework](https://github.com/openai/evals) is the gold standard. [DeepEval](https://github.com/confident-ai/deepeval) and [LM-Evaluation-Harness](https://github.com/EleutherAI/lm-evaluation-harness) for comprehensive testing. For adversarial and security testing, [PromptFoo](https://github.com/promptfoo/promptfoo) has become the standard: YAML-configured prompt A/B testing and red-teaming in one tool. If you need rigorous, reproducible offline evaluation where auditability matters, [Inspect](https://inspect.ai-safety-institute.org.uk/) from the UK AI Safety Institute is worth knowing. Set up red teaming before someone else does it for you. Designing agents that know when *not* to act By Month 2 you've wired up tool-calling, retrieval, and code generation. The gap between that and something you'd run against a production dbt project is not more tooling. It's a decision layer that doesn't exist in most frameworks out of the box. The architecture that closes that gap is a three-step decision loop: **Understand → Evaluate → Execute**. **Understand** means the agent retrieves structured context before executing. Specifically: decision history (why was this model built this way?), lineage (what does it touch downstream?), and cost attribution (what did the last run cost, and what will this change cost?). An agent without this step reasons from the SQL alone. An agent with it reasons from your data estate's actual history. **Evaluate** means the agent runs two checks before acting: a confidence score against its retrieved context, and a blast radius calculation: how many downstream models, dashboards, and pipelines does this change touch? Those two numbers, combined, determine whether the agent proceeds, scopes down, or routes decisions to human review. The threshold is a design decision you make explicitly. Most teams discover during Month 2 that human-in-the-loop isn't a degraded mode. It's what makes the system trustworthy enough to automate more over time. **Execute** means the agent acts within the scope the Evaluate step established, then writes the outcome back to the context layer as a structured decision trace. That trace becomes input to the next Understand step. The compound effect is real: an agent that has seen your team approve or reject 200 similar changes will calibrate its confidence scores differently than one seeing your data estate for the first time. Most frameworks give you the tools to call functions and run queries. They don't give you the decision logic: what to check before acting, what counts as too risky to proceed autonomously, and how to capture the outcome so the next run starts smarter. That's the layer you have to build. The **Understand → Evaluate → Execute** loop is one concrete implementation of it, and the reason it's baked into every agent on the [Altimate platform](https://altimate.ai/). ## Month 3: Pushing boundaries Reasoning models deep dive In 2024, everyone was trying to replicate o1. In 2026, reasoning models are a mature category with distinct tradeoffs: o3, Gemini 2.5 Pro, DeepSeek-R1 and its distilled variants, Qwen3's hybrid thinking/non-thinking modes. The question has shifted from "can we build this?" to "when does the compute cost of chain-of-thought justify itself, and when does a smaller direct model win?" The [DeepSeek-R1 paper](https://github.com/deepseek-ai/DeepSeek-R1) remains the clearest technical account of how reinforcement learning produces emergent reasoning. Sebastian Raschka's [Understanding Reasoning LLMs](https://magazine.sebastianraschka.com/p/understanding-reasoning-llms) is the best single primer on the category: what they are, how they're trained, and where they break down. Capstone projects that matter 1. **AI-First Data Platform** — Redesign your entire data architecture with AI at the center 2. **Domain Expert System** — Fine-tune multiple models for your industry, build evaluation frameworks 3. **Open Source Contribution** — Pick a major repo and contribute. Instant credibility. [Previous 3\. The One-Month Program](https://altimate.ai/ai-guide/one-month-program) [Next 5\. Resources and Milestones](https://altimate.ai/ai-guide/resources) --- # Chapter 5: Resources and Milestones URL: https://altimate.ai/ai-guide/resources Curated AI resources for data engineers: top courses from DeepLearning.AI and Fast.ai, essential GitHub repos, YouTube channels, communities, and docs. [All chapters](https://altimate.ai/ai-guide) The AI-Ready Data Engineer ## Chapter 5: Resources and Milestones Curated AI resources for data engineers: top courses from DeepLearning.AI and Fast.ai, essential GitHub repos, YouTube channels, communities, and docs. ## Courses worth your time - [Building and Evaluating Advanced RAG](https://www.deeplearning.ai/short-courses/building-evaluating-advanced-rag/) — DeepLearning.AI - [LangChain for LLM Application Development](https://www.deeplearning.ai/short-courses/langchain-for-llm-application-development/) - [Functions, Tools and Agents with LangChain](https://www.deeplearning.ai/short-courses/functions-tools-agents-langchain/) - [Knowledge Graphs for RAG](https://www.deeplearning.ai/short-courses/knowledge-graphs-rag/) - [Preprocessing Unstructured Data for LLM Applications](https://www.deeplearning.ai/short-courses/preprocessing-unstructured-data-for-llm-applications/) - [DeepLearning.AI](https://www.deeplearning.ai/courses/) — Andrew Ng's team, consistently excellent - [Fast.ai](https://course.fast.ai/) — Free and better than most paid courses - [Complete Agentic AI Bootcamp](https://www.udemy.com/course/complete-agentic-ai-bootcamp-with-langgraph-and-langchain/) — Udemy, solid content - [Coursera AI courses](https://www.coursera.org/courses?query=artificial+intelligence) — IBM and Stanford courses are legit - [DataCamp](https://www.datacamp.com/) — Good for beginners, nice UI - [LangChain Academy](https://academy.langchain.com/) — Free and directly from the source - [Hugging Face AI Agents Course](https://huggingface.co/learn/agents-course) — Free, certified, covers smolagents, LlamaIndex, and LangGraph - [Hugging Face LLM Course](https://huggingface.co/learn/llm-course/chapter1/1) — Free, covers Transformers, fine-tuning, and reasoning models ## GitHub repos to star immediately - [AltimateAI/altimate-code](https://github.com/AltimateAI/altimate-code) — Open-source agentic data engineering harness - [artidoro/qlora](https://github.com/artidoro/qlora) — Efficient fine-tuning - [axolotl-ai-cloud/axolotl](https://github.com/axolotl-ai-cloud/axolotl) — Fine-tuning made easy - [confident-ai/deepeval](https://github.com/confident-ai/deepeval) — LLM testing framework - [crewAIInc/crewAI](https://github.com/crewAIInc/crewAI) — Multi-agent orchestration - [dair-ai/Prompt-Engineering-Guide](https://github.com/dair-ai/Prompt-Engineering-Guide) — Comprehensive prompting - [explodinggradients/ragas](https://github.com/explodinggradients/ragas) — RAG evaluation - [langchain-ai/langgraph](https://github.com/langchain-ai/langgraph) — Stateful agents - [langchain-ai/rag-from-scratch](https://github.com/langchain-ai/rag-from-scratch) — RAG fundamentals - [microsoft/AI-For-Beginners](https://github.com/microsoft/AI-For-Beginners) — Surprisingly comprehensive - [modelcontextprotocol/servers](https://github.com/modelcontextprotocol/servers) — MCP integrations - [NirDiamant/GenAI\_Agents](https://github.com/NirDiamant/GenAI_Agents) — Production agent examples - [NirDiamant/RAG\_Techniques](https://github.com/NirDiamant/RAG_Techniques) — Every RAG pattern you need - [openai/evals](https://github.com/openai/evals) — Evaluation framework - [openai/openai-agents-python](https://github.com/openai/openai-agents-python) — OpenAI Agents SDK - [promptfoo/promptfoo](https://github.com/promptfoo/promptfoo) — Prompt testing and red-teaming - [stanfordnlp/dspy](https://github.com/stanfordnlp/dspy) — Automated prompt engineering - [unslothai/unsloth](https://github.com/unslothai/unsloth) — Fast, memory-efficient fine-tuning ## YouTube channels that don't waste your time - [Andrej Karpathy](https://youtube.com/@AndrejKarpathy) — Ex-OpenAI, pure technical content - [Sam Witteveen](https://youtube.com/@samwitteveenai) — LangChain specialist, great energy - [Matthew Berman](https://youtube.com/@matthew_berman) — Open source focus, daily videos - [Yannic Kilcher](https://youtube.com/@YannicKilcher) — Best paper explanations - [Krish Naik](https://youtube.com/@krishnaik06) — Practical implementations - [DeepLearning.AI](https://youtube.com/@Deeplearningai) — Official channel with free courses - [FreeCodeCamp](https://youtube.com/@freecodecamp) — Long-form tutorials - [3Blue1Brown](https://youtube.com/@3blue1brown) — Best conceptual explanation of how neural networks actually work ## Communities where real learning happens - [DataTalks.Club Slack](https://datatalks.club/slack.html) — 13k+ members, super active - [r/LocalLLaMA](https://reddit.com/r/LocalLLaMA) — Where the open source community lives - [r/MachineLearning](https://reddit.com/r/MachineLearning) — Academic discussions - [Hugging Face Discord](https://discord.com/invite/hugging-face-879548962464493619) — The most active open-source ML community; directly tied to the tools you're using - Twitter/X — Follow [@karpathy](https://x.com/karpathy), [@sama](https://x.com/sama), [@ylecun](https://x.com/ylecun), [@emollick](https://x.com/emollick) ## Additional documentation and guides - [OpenAI Cookbook](https://cookbook.openai.com/) — Practical examples - [Anthropic Claude docs](https://platform.claude.com/docs/en/home) — Excellent MCP documentation - [Altimate Code docs](https://docs.altimate.sh/) — Agentic data engineering harness - [Hugging Face documentation](https://huggingface.co/docs) — Models and datasets - [Google AI documentation](https://ai.google.dev/) — Gemini and more - [AWS AI/ML resources](https://aws.amazon.com/ai/) — Comprehensive guides - [Microsoft AI docs](https://learn.microsoft.com/en-us/ai/) — Azure AI services - [Towards Data Science](https://towardsdatascience.com/) — Community articles - [Ben's Bites](https://www.bensbites.co/) — Best daily AI newsletter, period (according to Anand!) - [The Rundown AI](https://www.therundown.ai/) — Good alternative to Ben's - [Chip Huyen's blog](https://huyenchip.com/) — Production ML wisdom - [Papers with Code](https://paperswithcode.com/) — Implementation paradise - [The Gradient](https://thegradient.pub/) — Thoughtful long-form content - [Latent Space](https://www.latent.space/) — AI engineering focused - [Sebastian Raschka](https://magazine.sebastianraschka.com/) — Best paper summaries - [Alpha Signal](https://alphasignal.ai/) — Daily digest of top repos, papers, and models; strong signal-to-noise for staying current ## After a weekend - You've built a working RAG app - You understand agents and can build basic ones - You can contribute meaningfully to AI discussions - You know the landscape and key concepts ## After a month - You can architect AI systems - You understand multiple frameworks well - You can evaluate and debug AI applications effectively ## After three months - You're comfortable leading AI initiatives - You're contributing to open source projects - You understand production deployment deeply - You can design and implement complex AI systems [Previous 4\. The Three-Month Advanced Track](https://altimate.ai/ai-guide/three-month-advanced-track) [Next Closing thoughts](https://altimate.ai/ai-guide/final-thoughts) --- # Closing thoughts URL: https://altimate.ai/ai-guide/final-thoughts Where to start, what to build in the first week, and the infrastructure question your team will be answering by 2027. [All chapters](https://altimate.ai/ai-guide) The AI-Ready Data Engineer ## Closing thoughts Where to start, what to build in the first week, and the infrastructure question your team will be answering by 2027. By Anand Gupta, CTO at Altimate.ai - March 2026 I hope you found this guide valuable. If you have any feedback, please [reach out to me on LinkedIn](https://www.linkedin.com/in/anandguptas/). In the meantime, here is some advice from personal experience. Start with RAG. It has the highest return on investment for a data engineer right now. Every organization needs it, the tooling is mature, and you can have something working in a day. Join [DataTalks.Club Slack](https://datatalks.club/slack.html) today. Not next week. The density of practitioners working through real production problems in that community is hard to replicate elsewhere. The mistakes being caught there right now will save you weeks. Build in public. Put the first project on GitHub. Write about what broke and why. The feedback loop accelerates learning in ways that private tinkering doesn't. If you want to see what building in public looks like for agentic data engineering, check out [Altimate Code](https://github.com/AltimateAI/altimate-code) — it's open source and a good example of the kind of project that benefits from community contribution. Don't wait for the right moment. The 48-hour program is designed to start this weekend. By Monday you'll have built something real, with enough context to ask the right questions at work. Focus on what maps to your job: RAG and vector databases if you work with unstructured data, agent frameworks if you need automation, fine-tuning if you need custom model behavior. The question that defines the next two years: in 2025, teams are asking how to get a first agent into production safely. By 2027, it becomes how to run 50 agents safely, consistently, across a data estate that's still changing. The second question is answered with the same components covered in this guide. The RAG system you build in week one becomes the retrieval layer every downstream agent queries before acting. The lineage you map becomes the blast radius calculator when something goes wrong. The governance model you design for one agent becomes the operating system for fifty. The teams running those 50 agents won't be the ones that moved fastest in 2025. They'll be the ones that built the right foundations: data estates where decision history is captured, lineage is complete, and governance is designed for machine-speed operations rather than weekly review cycles. The transition is already underway. In 2026 and 2027, agents handle documentation, testing, cost optimization, and routine transformations. Data engineers focus on architecture, business logic, governance, and the judgment calls agents can't make. The role doesn't disappear. It becomes the person who defines how agents operate, reviews what they produce, and decides when agent confidence is sufficient to trust. The gap between teams that get this right and teams that don't will not be about talent. It will be about infrastructure. That's the work. It starts this weekend. P.S. Save this guide. The resources are current as of early 2026 and get updated regularly. The GitHub repos especially are worth bookmarking — they keep getting better. [Previous 5\. Resources and Milestones](https://altimate.ai/ai-guide/resources) [Back to Guide Overview](https://altimate.ai/ai-guide) --- # Blog URL: https://altimate.ai/blog Insights on AI agents, agentic data engineering, Snowflake, Databricks, and analytics engineering from the Altimate AI team. [RSS Feed](https://altimate.ai/blog.rss.xml) FROM THE BUILD LOG ## The Altimate Blog Insights on AI agents, agentic data engineering, Snowflake, Databricks, and analytics engineering. Sign up for future editions ## Latest posts How the Databricks semantic layer and Genie Ontology work Sep 18, 2026 ### How the Databricks semantic layer and Genie Ontology work The Databricks semantic layer has two halves: what your team models in Unity Catalog, and what Genie infers from how people use the estate. Genie Ontology is how Databricks joins them at runtime. Read more → https://altimate.ai/blog/databricks-semantic-layer-genie-ontology Right-sizing Databricks job clusters, automatically Sep 16, 2026 ### Right-sizing Databricks job clusters, automatically Auto Tune learns from each Databricks job’s own runs, right-sizes its cluster inside the bounds you set, and reverts… Databricks databricks cost databricks compute Read more → https://altimate.ai/blog/right-sizing-databricks-job-clusters-automatically The $2,000/Month Claude Bill Your Data Team Doesn't Need Sep 16, 2026 10 min read ### The $2,000/Month Claude Bill Your Data Team Doesn't Need Claude Code cost for a data team reaches $2,000 a month at Anthropic's published averages. See the arithmetic and five levers… Dev Tools dbt Snowflake Read more → https://altimate.ai/blog/the-2000-month-claude-bill-your-data-team-doesnt-need Beyond the Databricks Cost Calculator: 6 Cost Levers, Including Auto Termination Sep 11, 2026 ### Beyond the Databricks Cost Calculator: 6 Cost Levers, Including Auto Termination The official Databricks price calculator provides you information on the DBU rate times the hours you plan to run, the instance… Databricks Data Engineering Cost Optimization Read more → https://altimate.ai/blog/beyond-the-databricks-cost-calculator-6-levers-that-actually-cut-your-bill The Hidden Cost of AI Coding Tools: Why Your Token Bill Lies Sep 10, 2026 10 min read ### The Hidden Cost of AI Coding Tools: Why Your Token Bill Lies The cost of AI coding tools depends on tokens per finished task. See how context re-reads, cache writes, thinking and retries… Dev Tools AI dbt Read more → https://altimate.ai/blog/the-hidden-cost-of-ai-coding-tools-why-your-token-bill-lies Databricks Serverless vs Classic Compute: Which One Is Actually Cheaper? Sep 4, 2026 ### Databricks Serverless vs Classic Compute: Which One Is Actually Cheaper? Whenever somebody mentions serverless, two things come to mind: it should be cheaper than normal resources and no infrastructure… databricks cost databricks serverless databricks compute Read more → https://altimate.ai/blog/databricks-serverless-vs-classic-compute-which-one-is-actually-cheaper Why Your Snowflake Bill Grows by 30% a Quarter Sep 4, 2026 10 min read ### Why Your Snowflake Bill Grows by 30% a Quarter A Snowflake bill that grows 30% a quarter nearly triples in a year. Six drivers behind that growth, each with the ACCOUNT\_USAGE… Cost Optimization Data Engineering Snowflake Read more → https://altimate.ai/blog/why-your-snowflake-bill-grows-by-30-percent-a-quarter Schema Validation Costs 0 Tokens: How Deterministic Tools Cut LLM Spend Sep 1, 2026 10 min read ### Schema Validation Costs 0 Tokens: How Deterministic Tools Cut LLM Spend Schema validation, lineage and lint can run as compiled code with no model call. See how deterministic tools cut LLM spend on one… Data Engineering Dev Tools Analytics Read more → https://altimate.ai/blog/schema-validation-costs-0-tokens-how-deterministic-tools-cut-llm-spend Understanding Databricks DBU and Compute Costs Aug 19, 2026 ### Understanding Databricks DBU and Compute Costs There is so much around Databricks cost optimization techniques, strategies, approaches, what to do, what not to do, and many… Databricks Altimate finops Read more → https://altimate.ai/blog/understanding-databricks-compute-costs Cost-Per-Task vs. Dollars-Per-Million-Tokens Aug 14, 2026 10 min read ### Cost-Per-Task vs. Dollars-Per-Million-Tokens Cost per task divides all agent spend, failed runs included, by the tasks that pass your rule. A method, a formula and worked… Dev Tools Data Engineering dbt Read more → https://altimate.ai/blog/cost-per-task-vs-dollars-per-million-tokens Why AI Agents Fail on Production Data Stacks Aug 5, 2026 10 min read ### Why AI Agents Fail on Production Data Stacks AI agents fail on production data stacks in ways a dev sandbox hides. Seven failures, from inherited role grants to backfills… AI Data Engineering Cost Optimization Read more → https://altimate.ai/blog/why-ai-agents-fail-on-production-data-stacks Power User for dbt Can Now Tell You Why Your Model Is Slow Aug 2, 2026 ### Power User for dbt Can Now Tell You Why Your Model Is Slow The queries that cost you money usually aren't the ones anyone complains about. A dbt model ships to prod looking fine, then… dbt SQL sql optimize Read more → https://altimate.ai/blog/power-user-for-dbt-can-now-tell-you-why-your-model-is-slow Autonomous Agents for Snowflake Cost Optimization: Announcing Altimate Lite Jul 28, 2026 ### Autonomous Agents for Snowflake Cost Optimization: Announcing Altimate Lite TL;DR: it saves you money on Snowflake Altimate Lite is a Snowflake native app that automatically tunes warehouse configuration… Snowflake AI costs AI Read more → https://altimate.ai/blog/altimate-lite-autonomous-agents-for-snowflake-cost-optimization Adaptive Compute vs Auto-Tune: Optimizing Snowflake Warehouses Jul 24, 2026 11 min read ### Adaptive Compute vs Auto-Tune: Optimizing Snowflake Warehouses Snowflake Adaptive Compute is now GA. Let's explore what its two parameters and per-query billing change, and what auto-tuning a… Cost Optimization Data Engineering Snowflake Read more → https://altimate.ai/blog/adaptive-compute-vs-auto-tune-optimizing-snowflake-warehouses What AI Agents Are Really Changing in Data Engineering Jul 22, 2026 ### What AI Agents Are Really Changing in Data Engineering Now that job candidates write their resumes and rehearse their interview answers with AI, every application reads the same. "Now… AI Data Engineering Read more → https://altimate.ai/blog/what-ai-agents-are-really-changing-in-data-engineering Altimate Code vs Claude Code vs Cursor vs Cortex Code CLI Jul 22, 2026 11 min read Guide ### Altimate Code vs Claude Code vs Cursor vs Cortex Code CLI Altimate Code vs Claude Code vs Cursor vs Cortex Code CLI. What each agent does for a data team, what it costs, and which one… Guides Data Engineering Dev Tools Read more → https://altimate.ai/blog/altimate-code-vs-claude-code-vs-cursor Why AI Agents Break in Production: The Missing Harness in Your Data Stack Jul 15, 2026 ### Why AI Agents Break in Production: The Missing Harness in Your Data Stack Listen to the full conversation on DataScienceWithSam, available on Apple Podcasts and Spotify. Most data teams have lived some… Data Science Data Engineering AI Read more → https://altimate.ai/blog/why-ai-agents-break-in-production-the-missing-harness-in-your-data-stack A 6-Step Playbook for dbt Migration With an AI Agent Jul 15, 2026 11 min read Guide ### A 6-Step Playbook for dbt Migration With an AI Agent A six-step playbook for dbt migration with an AI agent that treats business-logic parity as the deliverable. Guides Data Engineering Analytics Read more → https://altimate.ai/blog/dbt-migration-with-an-ai-agent-playbook How I Stopped Running dbt Compile 50 Times a Day Jul 14, 2026 9 min read ### How I Stopped Running dbt Compile 50 Times a Day dbt compile renders Jinja into SQL and writes it to target/. How I moved the edit, compile and check loop into a dbt VS Code… Dev Tools Analytics Data Engineering Read more → https://altimate.ai/blog/how-i-stopped-running-dbt-compile-50-times-a-day 5 Reasons Deterministic Tooling for AI Agents Beats an LLM-Only Stack Jul 8, 2026 10 min read Guide ### 5 Reasons Deterministic Tooling for AI Agents Beats an LLM-Only Stack Some questions about your data have exact answers. Five reasons deterministic tooling for AI agents beats a model on those, and… Guides Data Engineering AI Read more → https://altimate.ai/blog/deterministic-tooling-vs-llm-only-data-agents 4 Blind Spots of General Coding Agents in Data Engineering Jul 6, 2026 ### 4 Blind Spots of General Coding Agents in Data Engineering A coding agent enters the data warehouse A data engineer drops a one-liner into Claude Code: "Add patient\_visit\_count to… Data Engineering AI llm Read more → https://altimate.ai/blog/4-blind-spots-of-general-coding-agents-in-data-engineering The Correctness Layer: Deterministic Validation Outside the LLM Loop Jul 3, 2026 10 min read ### The Correctness Layer: Deterministic Validation Outside the LLM Loop Deterministic validation can check a data agent's work at three points in its loop. See what each point checks, what it returns… Data Engineering AI dbt Read more → https://altimate.ai/blog/the-correctness-layer-deterministic-validation-outside-the-llm-loop 7 dbt Anti-Patterns AI Should Catch Before Your PR Merges Jul 1, 2026 10 min read Guide ### 7 dbt Anti-Patterns AI Should Catch Before Your PR Merges Seven dbt anti-patterns that pass review and cost you later, why each one survives a human read, and the check that catches it… Guides Data Engineering Dev Tools Read more → https://altimate.ai/blog/dbt-anti-patterns-ai-should-catch Where AI Agents Belong in Data Engineering: The Correctness Layer Jun 29, 2026 ### Where AI Agents Belong in Data Engineering: The Correctness Layer With ever-changing models, new and better ones coming out every few months, it's great if we don't have to rely on them too… Altimate harnessengineering Data Engineering Read more → https://altimate.ai/blog/where-ai-agents-belong-in-data-engineering-the-correctness-layer 8 Column-Level Lineage Use Cases That Change How Teams Ship Jun 17, 2026 10 min read Guide ### 8 Column-Level Lineage Use Cases That Change How Teams Ship Eight column-level lineage use cases where knowing which field, not just which table, changes the work rather than the reporting. Guides Analytics Data Engineering Read more → https://altimate.ai/blog/column-level-lineage-use-cases You Are the Trust Layer: Why AI Code Review Is Breaking Down Jun 15, 2026 ### You Are the Trust Layer: Why AI Code Review Is Breaking Down Your Job Just Became QA for an AI Pipeline You Didn't Design Sometime this year your job changed, and nobody sent the memo. You… AI Software Engineering AI Tools Read more → https://altimate.ai/blog/you-are-the-trust-layer-managing-data-engineering-ai-agents-at-scale 9 Best Agentic Data Engineering Tools for VS Code Jun 11, 2026 11 min read Guide ### 9 Best Agentic Data Engineering Tools for VS Code Nine agentic data engineering tools for VS Code, scored on warehouse context, dbt awareness, cost visibility and license, with… Guides Dev Tools Data Engineering Read more → https://altimate.ai/blog/best-agentic-data-engineering-tools-vs-code DeepSeek Cost 62% Less Than Claude. The Surprising Part Wasn't the Savings… Jun 10, 2026 ### DeepSeek Cost 62% Less Than Claude. The Surprising Part Wasn't the Savings… If you're running an AI agent on data (pipeline diagnostics, schema exploration, ad-hoc SQL, lineage queries), you've looked at… AI llm Deepseek Read more → https://altimate.ai/blog/deepseek-vs-claude-sonnet-ai-agent-benchmark 10 Best MCP Servers for Data Engineering Teams Jun 4, 2026 11 min read Guide ### 10 Best MCP Servers for Data Engineering Teams MCP lets an agent look things up instead of guessing. Ten best MCP servers for data engineering, what each exposes, and the… Guides AI Tools Dev Tools Read more → https://altimate.ai/blog/best-mcp-servers-for-data-engineers The Great Token Heist of ‘26 Jun 1, 2026 ### The Great Token Heist of ‘26 Your LLM provider charges by the token, but a token is not a portable unit of work, a unit of meaning, or any unit you can… Altimate AI Tokenomics Read more → https://altimate.ai/blog/the-great-token-heist-of-26 12 Snowflake Cost Optimization Techniques That Actually Work May 21, 2026 12 min read Guide ### 12 Snowflake Cost Optimization Techniques That Actually Work Twelve Snowflake cost optimization techniques ranked by what they save against what they cost you to run, from free config… Guides Cost Optimization Dev Tools Read more → https://altimate.ai/blog/snowflake-cost-optimization-techniques The Correctness Layer May 20, 2026 ### The Correctness Layer We started altimate-code with one goal: build a harness that data engineers could actually trust with their pipelines and… AI Data Engineering Altimate Read more → https://altimate.ai/blog/the-correctness-layer-in-ade Blast Radius Analysis Using Altimate Code May 18, 2026 ### Blast Radius Analysis Using Altimate Code What is Blast Radius Blast radius refers to the potential extent of damage. For example, before you knock down a wall in your… altimate-code Data pipelines dbt Read more → https://altimate.ai/blog/blast-radius-analysis-using-altimate-code 9 Signs You Need an Agent-First Data Platform May 13, 2026 12 min read Guide ### 9 Signs You Need an Agent-First Data Platform Most AI agent pilots stall on the platform underneath them, not the model. Nine diagnostics for whether you need an agent-first… Guides Data Engineering AI Tools Read more → https://altimate.ai/blog/signs-data-platform-needs-agent-first-overhaul We Created Data Engineering Skills for Claude Code May 11, 2026 ### We Created Data Engineering Skills for Claude Code Data engineering work spreads beyond SQL into lineage, tests, docs, cost, schema changes, and PII. We built skills that bring… Read more → https://altimate.ai/blog/we-created-data-engineering-skills-for-claude-code Best Data Conferences 2026: The Complete Guide May 11, 2026 ### Best Data Conferences 2026: The Complete Guide If you're trying to figure out which remaining data conferences in 2026 are worth your time and travel budget, this is the list… Read more → https://altimate.ai/blog/best-data-conferences-2026-the-complete-guide 10 Reasons AI Coding Agents Fail at Data Engineering May 5, 2026 11 min read Guide ### 10 Reasons AI Coding Agents Fail at Data Engineering AI coding agents fail at data engineering for one structural reason. Ten specific failure modes, why each one happens, and the… Guides Data Engineering AI Tools Read more → https://altimate.ai/blog/why-ai-coding-agents-fail-at-data-engineering Claude Code or Altimate Code for Data Engineering? Apr 23, 2026 ### Claude Code or Altimate Code for Data Engineering? Claude Code is fast, writes good software code, and can even handle dbt models and other data engineering tasks to certain… Dev Tools altimate-code Altimate Read more → https://altimate.ai/blog/claude-code-or-altimate-code-for-data-engineering We built AI that actually works for data engineering. We crushed the ADE benchmark. Mar 19, 2026 ### We built AI that actually works for data engineering. We crushed the ADE benchmark. tl;dr: Today, we are launching Altimate Code, an open-source agentic data engineering harness that far exceeds generic LLMs (and… AI Tool Data Engineering Altimate Read more → https://altimate.ai/blog/introducing-altimate-code Teaching Claude Code the Art of Data Engineering: Introducing Altimate Skills Jan 22, 2026 ### Teaching Claude Code the Art of Data Engineering: Introducing Altimate Skills Today, we're open-sourcing Altimate Skills — a collection of Claude Code skills specifically designed for analytics engineers. We… Altimate Data Engineering AI Tools Read more → https://altimate.ai/blog/teaching-claude-code-the-art-of-data-engineering-introducing-altimate-skills Adaptive Compute vs. Auto Tune: A Practical Guide to Optimizing Snowflake Warehouses Jul 30, 2025 ### Adaptive Compute vs. Auto Tune: A Practical Guide to Optimizing Snowflake Warehouses Managing Snowflake warehouses efficiently is a constant balancing act. Set them too large, and you're burning money on idle… Altimate Data Engineering Snowflake Read more → https://altimate.ai/blog/adaptive-compute-vs-auto-tune-a-practical-guide-to-optimizing-snowflake-warehouses Supercharging Cursor IDE: How the dbt Power User Extension’s Embedded MCP Server Unlocks AI-Driven dbt Development Mar 21, 2025 ### Supercharging Cursor IDE: How the dbt Power User Extension’s Embedded MCP Server Unlocks AI-Driven dbt Development Introduction We’re excited to announce a major new capability in the dbt Power User VSCode extension: an embedded Model Context… Dev Tools dbt Data Engineering Read more → https://altimate.ai/blog/supercharging-cursor-ide-how-the-dbt-power-user-extensions-embedded-mcp-server-unlocks-ai-driven-dbt-development Could AI Teammates Be the Secret to Efficient Snowflake Cost Management? Nov 13, 2024 ### Could AI Teammates Be the Secret to Efficient Snowflake Cost Management? In the past five years, Snowflake has disrupted the database, data storage, and analytics industry by massively simplifying the… Snowflake AI AI Tools Read more → https://altimate.ai/blog/could-ai-teammates-be-the-secret-to-efficient-snowflake-cost-management Write Better Data Docs Faster Sep 19, 2024 ### Write Better Data Docs Faster Once upon a time, an analyst in the trouble Imagine Leila, an analyst, asked to build a monitoring dashboard for finance - given… dbt Altimate Analytics Read more → https://altimate.ai/blog/write-better-data-docs-faster Can AI Agents reduce Snowflake Cost using Dynamic Tables? Sep 7, 2024 ### Can AI Agents reduce Snowflake Cost using Dynamic Tables? In my previous article on Dynamic Tables, I explained their purpose, to help simplify building transformation pipelines for… Snowflake AI Tools AI Read more → https://altimate.ai/blog/can-ai-agents-reduce-the-cost-of-snowflake-dynamic-tables How I Cut dbt Development Time by 20% with the Power User Vscode Extension Apr 14, 2024 ### How I Cut dbt Development Time by 20% with the Power User Vscode Extension Working with data in any capacity—be it as a data/analytics engineer, data scientist, or data analyst—inevitably dealing with the… dbt Altimate analytics tools Read more → https://altimate.ai/blog/how-i-cut-dbt-development-time-by-20-with-the-power-user-vscode-extension Optimizing Snowflake Queries: Expert Strategies for Data Engineers and Analysts Jan 28, 2024 ### Optimizing Snowflake Queries: Expert Strategies for Data Engineers and Analysts For data teams, the clock is always ticking. In a business landscape defined by rapidly expanding datasets, slow queries can… Snowflake Data Engineering AI Read more → https://altimate.ai/blog/optimizing-snowflake-queries-expert-strategies-for-data-engineers-and-analysts --- # Altimate AI Comparisons URL: https://altimate.ai/comparisons Side-by-side Altimate AI comparisons against Snowflake Cortex Code, dbt Wizard, Claude Code, Keebo, Revefi and more. Capability tables and ADE-Bench scores. Comparisons ## Altimate AI Comparisons Every comparison is a straight capability count against one product: what Altimate AI leads on, what they lead on, and the benchmark numbers or real customer stories behind it. ## Agents for Data Engineering ### vs Snowflake Cortex Code Altimate Code scores higher on the benchmark, works across multiple platforms and tools, and lets you use other LLMs for significant cost savings. ADE-Bench score78% vs 65% See full comparison https://altimate.ai/comparisons/altimate-vs-snowflake-cortex ### vs dbt Wizard Altimate Code beat dbt's agent on dbt's own benchmark, works across multiple platforms and tools, and is open source with free tokens to get started. ADE-Bench score78% vs 59% See full comparison https://altimate.ai/comparisons/altimate-vs-dbt-wizard ### vs Databricks Genie Code Altimate Code scores higher on the benchmark, works across a dozen and more data platforms and multiple tools rather than only inside the Databricks workspace, and lets you bring your own LLM for significant cost savings. See full comparison https://altimate.ai/comparisons/altimate-vs-databricks-genie-code ### vs Claude Code Claude Code is the general agent. Altimate is the specialized data engineering harness and can be used as the \[Altimate Claude Code plugin\](https://github.com/AltimateAI/altimate-claude-plugin). ADE-Bench score78% vs 40% See full comparison https://altimate.ai/comparisons/altimate-vs-claude-code The scores are public. Run the agent. MIT-licensed and free to install. [Try Altimate Code Free →](https://help.altimate.ai/code/getting-started/) ## Cost Optimization & FinOps Products ### vs SELECT (select.dev) SELECT is primarily a cost visibility and reporting tool. Altimate AI provides agents that run custom cost analysis, implement optimization recommendations, and prevent costly issues earlier in the development lifecycle. See full comparison https://altimate.ai/comparisons/altimate-vs-select-dev ### vs Capital One Slingshot Slingshot focuses on Snowflake cost optimization, predominantly at the warehouse and query level, many times resulting in severe SLA violations. Altimate AI supports Snowflake, Databricks, and other platforms. It delivers autonomous optimizations with explanations, prevents costly mistakes in development, and extends optimization to storage, data pipelines, and the BI layer. See full comparison https://altimate.ai/comparisons/altimate-vs-capital-one-slingshot ### vs Revefi Revefi began in data observability and has since expanded into cost observability with some basic optimization capabilities. Altimate AI, by contrast, prevents costly mistakes in development workflows, enables team-level collaboration with assignable savings opportunities, and, above all, autonomously optimizes warehouse sizing and shutdowns for significantly higher savings. See full comparison https://altimate.ai/comparisons/altimate-vs-revefi ### vs Seemore Data Seemore Data is a Snowflake-focused cost observability and optimization tool. Altimate AI prevents costly mistakes in development, works across Snowflake, Databricks, and other platforms, and gives teams an AI Studio and local IDE/CLI development for SQL and dbt. See full comparison https://altimate.ai/comparisons/altimate-vs-seemore-data ### vs Espresso AI Espresso AI provides automatic warehouse and query optimization, but its query-routing proxy sits inline in the critical query path — a component every routed query passes through, with no published latency SLO. Altimate AI takes a less intrusive architecture for autonomous optimization, and provides team-level showback and chargeback, coverage across the entire data stack (storage, data pipelines, and BI reports), and prevention of costly issues in development. See full comparison https://altimate.ai/comparisons/altimate-vs-espresso-ai ### vs Keebo Keebo autonomously optimizes Snowflake warehouses, but its aggressive shutdowns can trigger SLA violations. Altimate AI runs autonomous optimization SLA-safe, prevents costly mistakes in development, and optimizes across the full stack — storage, dbt pipelines, and the BI layer, not warehouses alone. See full comparison https://altimate.ai/comparisons/altimate-vs-keebo Ready to cut the bill? 30-day POV before you commit. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) [Try Altimate Lite Free Snowflake Native App](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) --- # Product Highlights URL: https://altimate.ai/product-highlights Real use cases for Altimate Code, the dbt Power User plugin, Altimate MCP, and the Enterprise platform. One page each, grounded in the help docs. Snowflake Databricks BigQuery dbt Postgres Amazon Redshift Apache Airflow ## PRODUCT HIGHLIGHTS Real things Altimate can do, one use case at a time. ## Browse by product 9 highlights across 2 products. Step into a card to filter by topic and tech stack, including Snowflake and Databricks. 8 highlights ## Enterprise Platform Cost, performance, and governance across Snowflake and Databricks, plus Altimate Lite for Snowflake. Filter by platform in the tech stack. Browse https://altimate.ai/product-highlights/category/enterprise-platform 1 highlight ## PowerUser Plugin for dbt Lineage, docs, tests, and query help inside VS Code for dbt projects. Browse https://altimate.ai/product-highlights/category/poweruser-plugin-for-dbt --- # Video Demos URL: https://altimate.ai/demos-walkthroughs Watch Altimate Code in action. Demos and walkthroughs covering dbt documentation, testing, SQL peer review, PII masking, Airflow debugging, and more. DEMOS & WALKTHROUGHS ## See Altimate Code in Action. Real workflows. Real codebases. No staged demos. Watch how data teams use Altimate Code to debug pipelines, write tests, mask PII, review PRs, and migrate workloads — with transcripts and summaries for every video. Altimate Code Launch: SQL Server to Snowflake Migration with dbt — Live Demo 35:03 Featured Migration · SQL Server · Snowflake ## Altimate Code Launch: SQL Server to Snowflake Migration with dbt — Live Demo Altimate Code migrated SQL Server stored procedures to dbt on Snowflake, validated the output with data diff, and generated a dashboard to prove it worked. Watch now → https://altimate.ai/demos-walkthroughs/sql-server-snowflake-migration-dbt Before You Rename That dbt Column, Check the Blast Radius 10:02 dbt · Column Lineage · Refactoring ## Before You Rename That dbt Column, Check the Blast Radius Renaming total\_cents to gross\_revenue\_cents sounded like a one-line change. Blast radius analysis showed it touched models, metrics, YAML docs, and dashboards. Watch → https://altimate.ai/demos-walkthroughs/dbt-column-rename-blast-radius Migrate SQL Server to Snowflake with dbt and Altimate Code 18:49 Migration · SQL Server · Snowflake ## Migrate SQL Server to Snowflake with dbt and Altimate Code Stored procedure logic in, validated dbt models out — with row count checks, schema diffs, and a migration dashboard stakeholders can actually trust. Watch → https://altimate.ai/demos-walkthroughs/sql-server-snowflake-dbt-migration How an AI Agent Adds Airflow DAGs That Actually Match Your Project 4:16 Airflow · Context-Aware Coding · DuckDB ## How an AI Agent Adds Airflow DAGs That Actually Match Your Project One prompt added a new DAG to an existing Airflow + DuckDB project. It read the codebase, matched the patterns, and ran clean on the first try. Watch → https://altimate.ai/demos-walkthroughs/airflow-dag-pattern-matching Snowflake AI Services Cost Breakdown: Finally Understand Where Your Cortex Spend Goes 10:47 Snowflake · Cost Optimization · Cortex AI ## Snowflake AI Services Cost Breakdown: Finally Understand Where Your Cortex Spend Goes Your AI Services bill shows one number. This breaks it down by service type, model, user, and team — so you can answer when the CFO asks why costs spiked 40%. Watch → https://altimate.ai/demos-walkthroughs/snowflake-ai-services-cost-breakdown 5 Hidden Snowflake Cost Leaks Only AI Can Find 16:49 Snowflake · Cost Optimization · Analytics ## 5 Hidden Snowflake Cost Leaks Only AI Can Find Five cost savings opportunities Snowflake's native tools won't surface — and how they added up to 32% off the bill. Watch → https://altimate.ai/demos-walkthroughs/snowflake-cost-leaks-ai Can an AI Agent Peer Review dbt SQL Before It Hits Production? 6:37 dbt · SQL Quality · Code Review ## Can an AI Agent Peer Review dbt SQL Before It Hits Production? One prompt drives the full peer review. Seven mart models come out graded, fixed, and re-validated without a senior engineer in the loop. Watch → https://altimate.ai/demos-walkthroughs/ai-sql-peer-review-dbt This Airflow Pipeline Was Broken. Here's How an AI Agent Helped Me Fix It. 6:37 Airflow · dbt · Debugging ## This Airflow Pipeline Was Broken. Here's How an AI Agent Helped Me Fix It. One AI session traced, diagnosed, and fixed a failing e-commerce ETL pipeline with 5 broken dbt models. Watch → https://altimate.ai/demos-walkthroughs/fix-broken-airflow-pipeline I Used an AI Agent to Review This dbt PR: Here's What It Found 6:55 dbt · Pull Requests · Column Lineage ## I Used an AI Agent to Review This dbt PR: Here's What It Found The code looked reasonable. Column-level lineage revealed 4 breaking downstream changes before merge. Watch → https://altimate.ai/demos-walkthroughs/dbt-pr-review-column-lineage New Data Project, Zero Context: Can AI Get Me Productive Fast? 7:35 Productivity · Data Analysis · Onboarding ## New Data Project, Zero Context: Can AI Get Me Productive Fast? A PM needs data in 20 minutes. You have no context on the project. Can an AI agent get you there? Watch → https://altimate.ai/demos-walkthroughs/new-data-project-zero-context Getting Productive on a New Data Team with an AI Agent 8:13 Airflow · Context-Aware Coding · Onboarding ## Getting Productive on a New Data Team with an AI Agent Starting from a new codebase with zero familiarity, one prompt added a fifth Airflow DAG that matched the team's patterns perfectly. Watch → https://altimate.ai/demos-walkthroughs/airflow-dag-context-aware-coding Masking PII with an AI Agent 12:44 PII · GDPR · dbt ## Masking PII with an AI Agent Most teams don't know where their PII lives. An AI agent found it, masked it, and validated everything, all for $3. Watch → https://altimate.ai/demos-walkthroughs/masking-pii-ai-agent-dbt Building dbt Docs with an AI Agent 11:40 dbt · Documentation · Productivity ## Building dbt Docs with an AI Agent The project had 50% model coverage and 20% column coverage. The AI rewrote all of it in 17 minutes for $1. Watch → https://altimate.ai/demos-walkthroughs/generate-dbt-documentation-ai Building dbt Tests with an AI Agent 14:33 dbt · Testing · Data Quality ## Building dbt Tests with an AI Agent The project went from 24 tests to 53 in one session. The agent wrote 30 new tests, ran them, and confirmed all passing, for $1.51 total. Watch → https://altimate.ai/demos-walkthroughs/generate-dbt-tests-ai-agent TRY IT YOURSELF ## 10 Million Tokens. No Credit Card. [Sign Up Free →](https://app.myaltimate.com/register) --- # Enterprise Platform highlights URL: https://altimate.ai/product-highlights/category/enterprise-platform Cost, performance, and governance across Snowflake and Databricks, plus Altimate Lite for Snowflake. Filter by platform in the tech stack. ## Enterprise Platform Cost, performance, and governance across Snowflake and Databricks, plus Altimate Lite for Snowflake. Filter by platform in the tech stack. Topic Warehouse cost 8 Data quality 0 Migration 0 Lineage 0 Governance 8 Productivity 0 AI safety 0 Context graph 0 Tech stack dbt 1 Snowflake 7 Databricks 3 BigQuery 0 Postgres 0 Tableau 0 VS Code 0 Cursor / Copilot 0 CI/CD 0 Altimate Lite 2 ## Enterprise Platform highlights Showing 8 of 8 Grid List Enterprise Platform ### Auto Tune Right-Sizes Snowflake Warehouses Without Slowing Queries Warehouse cost GovernanceMay 2026 · 6 min read https://altimate.ai/product-highlights/snowflake-warehouse-auto-tune-right-sizing Enterprise Platform ### Snowflake Cost Analysis in Plain English, with AI That Cites Every Number Warehouse cost GovernanceMay 2026 · 5 min read https://altimate.ai/product-highlights/snowflake-cost-analysis-ai-cited-answers Enterprise Platform ### Databricks Cost Breakdown, from One Bill Down to One Warehouse Warehouse cost GovernanceMay 2026 · 4 min read https://altimate.ai/product-highlights/databricks-cost-breakdown-workspace-sku-cluster Enterprise Platform ### Attribute Snowflake Cost to the Team That Owns It Warehouse cost GovernanceJun 2026 · 4 min read https://altimate.ai/product-highlights/snowflake-cost-alerts-by-team Enterprise Platform ### Root Cause Analysis for a dbt, Tableau, or Snowflake Workload Spike Warehouse cost GovernanceJun 2026 · 6 min read https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis Enterprise Platform ### Track Snowflake Query Cost Over Time, Run by Run Warehouse cost GovernanceJul 2026 · 6 min read https://altimate.ai/product-highlights/snowflake-query-cost-over-time Enterprise Platform ### Reduce Snowflake Idle Warehouse Spend with Altimate Lite Warehouse cost GovernanceJul 2026 · 6 min read https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost Enterprise Platform ### Snowflake Cortex AI Cost Breakdown by Service, User, and Model Warehouse cost GovernanceJul 2026 · 6 min read https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown --- # PowerUser Plugin for dbt highlights URL: https://altimate.ai/product-highlights/category/poweruser-plugin-for-dbt Lineage, docs, tests, and query help inside VS Code for dbt projects. ## PowerUser Plugin for dbt Lineage, docs, tests, and query help inside VS Code for dbt projects. Topic Warehouse cost 1 Data quality 0 Migration 0 Lineage 0 Governance 0 Productivity 1 AI safety 0 Context graph 0 Tech stack dbt 1 Snowflake 1 Databricks 0 BigQuery 1 Postgres 1 Tableau 0 VS Code 1 Cursor / Copilot 0 CI/CD 0 Altimate Lite 0 ## PowerUser Plugin for dbt highlights Showing 1 of 1 Grid List PowerUser Plugin for dbt ### Profile a Slow dbt Query and Read Its Execution Plan in Plain English Productivity Warehouse costJul 2026 · 4 min read https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan --- # Snowflake Auto Tune modes and safe backoff URL: https://altimate.ai/product-highlights/snowflake-warehouse-auto-tune-right-sizing Altimate Auto Tune right-sizes and suspends Snowflake warehouses automatically, and reverts to the default size if queue time or latency crosses a threshold. [.md](https://altimate.ai/product-highlights/snowflake-warehouse-auto-tune-right-sizing.md) Copy as MD Enterprise Platform ## Auto Tune Right-Sizes Snowflake Warehouses Without Slowing Queries Altimate Auto Tune right-sizes and suspends Snowflake warehouses automatically, and reverts to the default size if queue time or latency crosses a threshold. Pradnesh PatilMay 24, 2026Aug 11, 20266 min read Auto Tune right-sizes Snowflake warehouses without slowing queries TL;DR - **What it is.** Altimate AI's Auto Tune agents manage your Snowflake warehouse size, idle time, and clustering configuration. It cuts costs while staying within the performance bounds you set. Once enabled, your team doesn't need to do anything further. - **Who it is for.** Data platform leads and engineering managers who own the Snowflake bill and the query SLAs those warehouses have to meet. - **What you get.** Altimate manages your warehouses based on their workload patterns, inside your performance bounds. Customers save up to 20% of warehouse costs, and the savings are reported in a form both engineering and finance can use. ## Auto Tune Runs Autonomous Agents with Human Guardrails Auto Tune lowers your Snowflake warehouse cost. It runs autonomous agents that decide the optimum warehouse size and idle times. You turn it on warehouse by warehouse. | Agent | What it does | | --- | --- | | **Warehouse Shutdown and Scaling** | Scales clusters back and suspends the warehouse before Snowflake's own auto-suspend would fire. | | **Warehouse Sizing** | Drops the warehouse one level below its default during low-demand hours, reading the pattern from the same hours in earlier weeks. | Warehouse Sizing runs in one of two modes: 1. **Approval mode.** You get next week's downsizing schedule and approve the blocks you're happy with. Nothing resizes until you say so. Approved blocks run as Snowflake tasks, so Altimate needs task permissions on your account. 2. **Real-Time mode.** The agent decides on its own, every 15 minutes, without asking. It carries a built-in backoff that watches performance after each resize and reverts the change if performance degrades. Both modes take custom schedule blocks, so the hours you care about stay yours. A warehouse that idles from 2am to 4am on Tuesdays can sit one size down for those two hours. A warehouse running month-end close stays at its default size for the whole run, whatever the agent would otherwise decide. You can check the numbers before turning anything on. With the toggle still off, the warehouse page shows what Auto Tune would save you on any warehouse. The COMPUTE\_WH Snowflake warehouse detail page with the Auto Tune slider still off, showing scaling policy, cluster range, Small size and a 60 second auto-suspend. One red circle marks the off slider and a second marks the savings above it, realized savings of $945.13 to $1004.61 and $12,276.13 of estimated annual savings. ## Auto Tune Rolls Back Its Changes If Performance Degrades Real-Time mode measures two things after every resize: queue time and query execution time. Its baseline is the peak from the same day the week before, so a busy Monday is judged against last Monday. | Metric | What triggers a rollback | | --- | --- | | Queue time | Queue time passes 150% of the baseline | | Query latency | Execution time passes 150% of the baseline | When either threshold is crossed, Auto Tune restores the default size and leaves that warehouse alone for the next hour. After the cool-off, it can trigger again. Every agent change goes into an audit trail: the new size, the new config, and the reasoning behind it. You can dig in to understand why a particular change was made. The Auto Tune history timeline for one warehouse, listing resize-to-Medium, resize-to-Large and suspend events by DataPilot with timestamps, one row expanded to explain a 55.82 GB local spill on a Medium warehouse, plus an event filter across scaling, suspension, size changes, backoff and config changes. ## Auto Tune Reports Savings as a Range Snowflake checks for suspendable warehouses roughly every 30 seconds, so after your auto-suspend timer fires, the warehouse actually stops sometime in the next 30 seconds. Nobody can tell in advance when. Auto Tune accounts for this by giving you two numbers: - **The best case.** The warehouse stopped the instant the timer fired. - **The likely case.** It stopped 15 seconds later, halfway through that check interval. A single number would have to assume the best case, which inflates the savings. The second number is the one that holds up in a finance review. The savings chart tracks cost and savings together, by day, week, or month. Yellow is the spend; green stacked on top is what Auto Tune took off it. Hover a bar and it shows how much of that saving came from each agent. The Auto Tune Savings chart for one warehouse across a month, with the granularity dropdown set to Daily. Spend is yellow, Auto Tune savings stack in green on top, and a tooltip splits one day's saving into a range for Warehouse Suspension plus Scaling and a smaller range for Warehouse Sizing. Across the whole account, the same numbers tell you where to start. The Infra page filters warehouses by whether Auto Tune is resizing them. Sort by annual saving and the biggest wins come to the top. The Infra warehouses list with the three Auto Tune quick filters highlighted at the top, Eligible for Resizing, Resizing Real-Time and Resizing Approval, above a table of warehouses carrying cost, size, 90% execution time, opportunities, insights and auto-suspend setting. ## What You Get with Auto Tune Left On - **Warehouse Shutdown and Scaling.** Suspends a warehouse sooner than [Snowflake's own auto-suspend](https://altimate.ai/blog/snowflake-cost-optimization-techniques), with nobody filing a ticket for it. - **Warehouse Sizing.** Drops a warehouse one size during its quiet hours. You choose whether it asks you first. - **Automatic rollback.** If queue time or query time crosses your threshold, Auto Tune puts the default size back and leaves that warehouse alone for the next hour. - **Custom schedule blocks.** Force a smaller size on a window you know is quiet, or hold the default through a window you can't risk. - **Two savings numbers.** The best case and the case that usually happens. Proof Auto Tune cuts warehouse cost by **up to 20%** across our customers, without giving up the latency budget those warehouses run under. The backoff is why that holds. Cross your queue-time or latency threshold and the default size comes straight back. You do not have to take the 20% on trust. With both agents still off, the warehouse page already shows Possible Savings for that warehouse in dollars. That number is measured on your own workload. Read it and compare it against our 20% before you switch anything on. Auto Tune is part of the Enterprise Platform, and [Altimate for Snowflake](https://altimate.ai/use-cases/altimate-for-snowflake) is where a Snowflake team starts. ### Talk to us about your warehouse [Book a 30-min demo →](https://calendly.com/d/ctnv-rbc-yr3/altimate-ai-overview) ## Frequently Asked Questions What are the two Auto Tune agents on a Snowflake warehouse? Warehouse Shutdown and Scaling scales clusters back and suspends a warehouse before Snowflake's native auto-suspend fires. Warehouse Sizing downsizes it one level below the default during low-demand periods. You turn each one on separately from the Auto Tune slider on the warehouse page, and the same slider turns both off again. How much does Auto Tune actually save? Auto Tune cuts warehouse cost by up to 20% across our customers, and the backoff keeps that inside your latency budget. Your own number depends on how much idle time and how much predictable low-demand time your warehouses have. With the agents still off, the warehouse page shows Possible Savings in dollars for that warehouse. Check your own figure against our 20%. What makes the real-time backoff revert a resize? It tracks queue time and query execution time against a baseline, the peak on the same day the previous week. Cross the threshold you set, 150% by default, and Auto Tune restores the default size. It then makes no further size change for an hour. What thresholds suit a latency-sensitive warehouse? We suggest tightening to 120% or 140% for customer-facing dashboards, interactive queries and anything under an SLA, where a delay is felt straight away. Batch ETL and reporting warehouses can run at 180% or 200% and trade some latency for more savings. The 150% default sits between the two. Why is Warehouse Sizing missing on some of my warehouses? Auto Tune re-checks every warehouse daily, and only offers resizing where it should pay off without hurting queries. It looks for three things. Stretches of low load. Patterns that repeat daily or weekly. And enough compute cost to make the saving worth having. The toggle only appears on a warehouse that clears all three. The settings panel says why one does not. What is the difference between Approval mode and Real-Time mode? Approval mode gives you a recommended downsizing schedule for the week ahead, and the agent resizes only on the blocks you approve. Real-Time mode re-evaluates every 15 minutes against historical patterns and current load, with no input from you. The backoff protects this mode. Why are auto-suspend savings shown as a range? Snowflake checks for suspendable warehouses roughly every 30 seconds. A warehouse therefore stops between 0 and 30 seconds after its auto\_suspend timer fires. Auto Tune shows you both ends. The best case assumes it stopped instantly. The usual case assumes it stopped halfway through those 30 seconds. Take the second number to finance and it will hold up. Where can my team see Auto Tune on our own warehouses? Auto Tune is part of the Enterprise Platform, whose product documentation sits behind a partner login. The [Enterprise Platform overview](https://altimate.ai/platform) describes what is in it. A walkthrough on your own Snowflake account is the quickest way to find out which of your warehouses qualify for resizing. On this page - Auto Tune Runs Autonomous Agents with Human Guardrails - Auto Tune Rolls Back Its Changes If Performance Degrades - Auto Tune Reports Savings as a Range - What You Get with Auto Tune Left On ## Related Resources Altimate Lite for Snowflake Reduce Snowflake Idle Warehouse Spend with Altimate Lite https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost Enterprise Platform Snowflake Cost Analysis in Plain English, with AI That Cites Every Number https://altimate.ai/product-highlights/snowflake-cost-analysis-ai-cited-answers Help Doc AI Cost Analysis (DataPilot) https://help.altimate.ai/platform/summary/datapilot-analysis/ Enterprise Platform Attribute Snowflake Cost to the Team That Owns It https://altimate.ai/product-highlights/snowflake-cost-alerts-by-team --- # Studio answers cost questions with citations URL: https://altimate.ai/product-highlights/snowflake-cost-analysis-ai-cited-answers Snowflake cost analysis in plain English. Ask Studio about spend across AI Services, Serverless and warehouses, and every number traces to its source. [.md](https://altimate.ai/product-highlights/snowflake-cost-analysis-ai-cited-answers.md) Copy as MD Enterprise Platform ## Snowflake Cost Analysis in Plain English, with AI That Cites Every Number Snowflake cost analysis in plain English. Ask Studio about spend across AI Services, Serverless and warehouses, and every number traces to its source. Pradnesh PatilMay 25, 2026Aug 11, 20265 min read Snowflake cost analysis in plain English, with AI that cites every number TL;DR - **What it is.** Studio is a specialized agent for cost and performance questions across your data stack. It answers on Snowflake, Databricks, dbt, Tableau and the other tools Altimate connects to. Ask in plain English and you get the cause and the fix, not the number on its own. Every figure in the answer links back to the table and the query behind it. - **Who it is for.** Data platform leads, engineering managers and the finance partner who has to accept the number. - **What you get.** You get one place to ask why the bill moved, which workload moved it, and what to change. This article follows one Snowflake cost question the whole way. ## AI Cost Analysis on the Snowflake Page You Have Open Studio is Altimate AI's specialized agent for your data stack. You ask in plain English about cost, performance and what to change. The answer spans Snowflake, Databricks, dbt, Tableau and more. Every major page in the Enterprise Platform has a chat button that opens it. This article stays on the Snowflake cost question, because that is where most teams start. Open it from the cost Summary page and your cost breakdown stays on screen while you type. The questions Studio suggests there are all about spend: - What drove this month's cost. - Which five warehouses cost the most. - What next quarter is projected to cost. The Studio AI panel running a Snowflake cost analysis over the platform's Summary page. Behind it, Current State shows total costs of $117.78 split into compute, storage, serverless, cloud services and AI Services, while the panel offers suggested prompts about this month's cost drivers, the top five warehouses and projected spend for next quarter. Studio already reads whatever Altimate is connected to. You add more context on top of that, in three forms: - **Attach a file.** - **Pull in a Knowledge Base doc.** - **Start from a prompt** your team already saved. The full Studio page with chat history down the left rail, each past question marked Completed, and the context menu open beside the prompt box offering Add photos or files, Knowledge Base and View Prompt Library. ## Studio Answers Across Surfaces, Then Names the Cause and the Fix Visibility is the first step, and only the first step. Studio reads every cost surface in one thread, so the answer does not stop at which number moved. It runs root cause analysis on that number, then gives you an optimization plan with the saving quantified. On Snowflake, these are the surfaces it draws from: | Ask about | What the answer breaks down by | | --- | --- | | **Warehouses** | per-warehouse credits and query cost | | **AI Services** | each function, split across four usage types: AI services, AI inference, and the overage version of each | | **Serverless** | more than 19 serverless usage types, down to per-table clustering and dynamic-table refresh | | **Workloads** | workload tag, matching the Workloads page | Ask from a notebook page or a stored procedure page and Studio starts from that workload. You never have to name it in the question. Studio answers [the same questions for Databricks](https://altimate.ai/product-highlights/databricks-cost-breakdown-workspace-sku-cluster), counted in DBUs instead of Snowflake credits. ## Every Number in a Studio Answer Links to Its Source A number nobody can trace is a number finance will not accept. Studio puts a small marker next to the numbers in an answer. Click a marker and you get three things: - **The table** the number came from. - **The query** that produced it. - **The steps** it took to get from one to the other. Read a missing marker as a signal. A sentence with no marker is Studio's own commentary. A number with a marker has a query behind it. You can then share the answer or put it on a schedule: 1. **Share conversation.** Makes a read-only link to the answer. You choose whether the link expires after 7, 30 or 90 days, or never. 2. **Schedule report.** Re-runs the same conversation daily, weekly or monthly at 09:00 UTC, against fresh data each time. ## What You Get Once Studio Is Switched On - **Answers in plain English.** You ask about cost, performance or a slow workload, on the page you already have open. - **A source behind every number.** Click the marker and you get the table, the query and the steps. - **A link the finance partner can open.** It is read-only, and you decide when it expires. - **A report nobody has to rebuild.** The same conversation re-runs on a schedule against fresh data. Proof Studio adds the markers in a separate step, after the answer is already written. The wording never changes to fit a citation. A marker points at exactly what Studio computed. If it cannot trace a number, that number gets no marker at all. That is why an unmarked figure is a real warning rather than a gap in the tooling. Studio is in Beta today, and [Altimate for Snowflake](https://altimate.ai/use-cases/altimate-for-snowflake) is where a Snowflake team starts. ### Talk to us about your warehouse [Book a 30-min demo →](https://calendly.com/d/ctnv-rbc-yr3/altimate-ai-overview) ## Frequently Asked Questions What does a citation marker actually open? Three things: the source table, the SQL that produced the number, and the steps in between. When an answer mentions a Snowflake object, it also gives you a badge you can click through to that object's page. It lists everything it referenced in a Sources section. Why do some numbers in an answer have no marker? Because Studio could not trace them. Studio adds the markers in a separate step, after it writes the answer. Anything it wrote as commentary rather than computed from your data gets no marker. Read the absence as a signal. A sentence with no marker is opinion, and a number with a marker has a query behind it. Which kinds of spend can I ask about? Four. Warehouses gives you per-warehouse credits and query cost. AI Services splits each function across the four usage types, which is what you want when you are [reading a Cortex bill](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown). Serverless goes down to more than 19 usage types, including per-table clustering and [dynamic-table refresh](https://altimate.ai/blog/can-ai-agents-reduce-the-cost-of-snowflake-dynamic-tables). Workloads splits by workload tag, the same tag the Workloads page uses. Who can open a shared Studio conversation? You pick one of three modes when you create the link. Anyone who has the link, with no login needed. Named people who already have an account in your workspace. Or a connected Slack channel. Links expire after 7, 30 or 90 days, or never. A shared conversation is read-only, so whoever opens it sees a frozen answer and cannot keep asking questions. How often can a Studio conversation re-run itself? Daily at 09:00 UTC, every Monday at 09:00 UTC, or on the first of the month. Every run queries fresh data instead of replaying a cached snapshot. From the schedule view you can pause a report, edit it, run it by hand, or look at its run history. Does Studio tell me when it is unsure? Yes. Every answer comes with a confidence indicator. If it reads Low Confidence at 50%, Studio is telling you to narrow the question before you act on the answer. Together with the markers, that gives you two ways to check an answer before it goes into a finance review. How do I get citations, share links and scheduled reports turned on? None of the three is on by default. Each one is switched on per workspace, so ask your Altimate account team to turn them on for yours. Studio itself is in Beta. The [Enterprise Platform overview](https://altimate.ai/platform) covers what else is in the platform, and the product documentation sits behind a partner login. On this page - AI Cost Analysis on the Snowflake Page You Have Open - Studio Answers Across Surfaces, Then Names the Cause and the Fix - Every Number in a Studio Answer Links to Its Source - What You Get Once Studio Is Switched On ## Related Resources Enterprise Platform Auto Tune Right-Sizes Snowflake Warehouses Without Slowing Queries https://altimate.ai/product-highlights/snowflake-warehouse-auto-tune-right-sizing Enterprise Platform Root Cause Analysis for a dbt, Tableau, or Snowflake Workload Spike https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis Enterprise Platform Track Snowflake Query Cost Over Time, Run by Run https://altimate.ai/product-highlights/snowflake-query-cost-over-time Help Doc AI Cost Analysis (DataPilot) https://help.altimate.ai/platform/summary/datapilot-analysis/ --- # Databricks cost breakdown by SKU, warehouse URL: https://altimate.ai/product-highlights/databricks-cost-breakdown-workspace-sku-cluster A Databricks cost breakdown you can drill. Five categories on the Summary page, the SKU behind the largest, then the named warehouses that spend it. [.md](https://altimate.ai/product-highlights/databricks-cost-breakdown-workspace-sku-cluster.md) Copy as MD Enterprise Platform ## Databricks Cost Breakdown, from One Bill Down to One Warehouse A Databricks cost breakdown you can drill. Five categories on the Summary page, the SKU behind the largest, then the named warehouses that spend it. Saurabh AroraMay 27, 2026Aug 20, 20264 min read Databricks cost breakdown, from one bill down to one warehouse TL;DR - **What it is.** A Databricks cost breakdown in three steps. Find the category that costs most, then the SKU inside it, then the warehouse or cluster spending the money. - **Who it is for.** Data platform leads and managers who answer for Databricks spend across more than one workspace. - **What you get.** One bill turned into a short list of resources to fix. Setup is read-only. Nothing runs on your clusters. ## Start the Databricks Cost Breakdown at the Costliest Category Your invoice is one number. It names nothing you can act on. The Summary page splits the same spend into five categories: 1. **Clusters.** All-purpose and job compute. 2. **SQL Warehouse.** Classic, Pro and Serverless. 3. **AI/ML.** Model serving and vector search. 4. **Lakehouse.** Platform and storage. 5. **Platform.** Overhead and other SKUs. One category is always the most costly. That one decides where your month goes. For most enterprises the most costly category is Clusters. Job clusters run every scheduled batch load. All-purpose clusters stay open all day for analysts. Smaller accounts spend most on SQL Warehouse instead. Here is an example. A week costs about $12K. SQL Warehouse is roughly $6K of it and Clusters about $5.6K. SQL Warehouse costs most, so that is where you can start diagnosing. The Databricks Current State summary. A Total Costs card lists the five categories with a dollar figure each, beside a daily stacked cost chart with a tab per category. Each category splits again into the types Databricks bills for. Clusters splits into Jobs, All Purpose, DLT and Interactive. SQL Warehouse splits into Classic, Pro and Serverless. Follow the costliest type down. Here, most of the SQL Warehouse spend is Serverless. The SQL Warehouse tab of the same chart, split into Classic, Pro and Serverless bands by day, with Serverless the largest band. ## Take That Category Down to the SKU You Pay For A category tells you where to look. A SKU tells you what you pay for. The SKU is the billing unit in Databricks. The Breakdown page lists every SKU with its DBUs, its cost and its share of the bill. It sorts them by cost, so the top row is your answer. Keep the worked example going. The month costs about $49K. The top SKU is a serverless SQL compute line at roughly $18K. That one row is more than a third of the whole bill. The same page also splits the bill by workspace. One production workspace might hold about 80% of the spend. That per-workspace number is what Databricks chargeback runs on. The Databricks Breakdown page. Header cards show total cost, total DBUs and active workspaces. A Workspaces table splits cost by workspace, and a By SKU tab below lists each billing unit with DBUs, cost and share. ## Name the Warehouses and Clusters That Spend the Money The last step is the resource itself. The SQL Warehouses page gives every warehouse a row, sorted by cost. Say fifteen warehouses share the bill. The top one is near $6K and the next two about $4.5K each. Everything below them is small. That is three owners to talk to, not fifteen. The Clusters page does the same for job and all-purpose compute. The Databricks SQL Warehouses page, its table sorted by cost, with each row carrying type, t-shirt size, auto-stop, scaling range, queue time and idle time. Switch the chart between daily and weekly to give a spike a date. Serverless spend might sit near $900 a day, drop for a few days, then jump back over $900. Now you stop asking why the month rose. You ask what changed that day. A Total Warehouse Cost chart stacking Serverless against Classic spend by day across about a month, with one day tallest. One thing these pages will not tell you. They cannot say whether a warehouse is the right size. No column decides that. Average CPU and average memory do not decide it either. Sizing is Auto Tune's job, and this page tells Auto Tune where to start. [Altimate for Databricks](https://altimate.ai/use-cases/altimate-for-databricks) is where a Databricks team begins. ### Talk to us about your warehouse [Book a 30-min demo →](https://calendly.com/d/ctnv-rbc-yr3/altimate-ai-overview) ## Frequently Asked Questions What are the five Databricks cost categories? Clusters covers all-purpose and job compute. SQL Warehouse covers Classic, Pro and Serverless warehouse compute. AI/ML covers model serving, vector search and other ML workloads. Lakehouse covers platform and storage. Platform covers overhead and miscellaneous SKUs. The Summary page charts all five across the period you choose. What is a Databricks SKU? The SKU is the unit Databricks bills you in, so it is the most precise name your spend has. Premium Serverless SQL Compute and Premium JOBS Compute are separate SKUs at [separate DBU rates](https://altimate.ai/blog/understanding-databricks-compute-costs). Serverless SKUs also carry a region in the name, such as US EAST N Virginia. The same compute costs differently by region. Which cluster types does the Clusters view separate? There are four: Jobs, All Purpose, DLT and Interactive. Job clusters run scheduled and batch workloads, then spin down. All-purpose clusters stay available for ad hoc work from analysts and data scientists. That is why they often cost the most. DLT and Interactive are usually the smaller pair. Which SQL warehouse types does Databricks bill separately? Databricks charges for Classic, Pro and Serverless separately. The Summary chart plots them as three bands, so you can see which one moved. The split matters because [serverless bills at a different rate for a different thing](https://altimate.ai/blog/databricks-serverless-vs-classic-compute-which-one-is-actually-cheaper). A rising serverless band and a rising classic band need two different fixes. How do I see what one Databricks workspace costs? The Breakdown page carries a Workspaces Cost Details table with one row per workspace. Each row gives the name, the total cost for the window, the DBUs consumed and the share of overall spend. The page header adds the total cost, the total DBUs and the count of workspaces with any activity. What is the difference between the SKU and Product views? By SKU goes down to the billing unit, with DBUs, cost and share on every row. One such row is Premium Serverless SQL Compute US EAST N Virginia. By Product rolls the same spend up into SQL, JOBS, ALL Purpose, Model Serving and DLT. Use SKU to find the charge and Product to brief someone quickly. How do I find the day a cost spike started? Widen the window, then switch the aggregate between daily and weekly. Weekly tells you which week broke pattern, and daily tells you which day inside it. Once you have the date, set the resource table to that same window. It names the warehouse or cluster carrying the cost that day. Can this tell me whether a warehouse or cluster is sized correctly? No, and it should not pretend to. No column, ratio or usage figure on these pages decides whether a resource is right-sized. Average CPU and average memory in particular do not settle it, because a hot cluster can be retrying failed jobs or spilling data to disk. That is a reason to look harder, not a reason to leave it alone. Can I use these totals for Databricks chargeback? Yes, that is what the per-workspace and per-product views are for. Use them to allocate cost by workspace, to charge back or show back, to compare SKU choices and to plan capacity. The [Enterprise Platform overview](https://altimate.ai/platform) describes the rest of the product, and the detailed documentation sits behind a partner login. On this page - Start the Databricks Cost Breakdown at the Costliest Category - Take That Category Down to the SKU You Pay For - Name the Warehouses and Clusters That Spend the Money ## Related Resources Enterprise Platform Attribute Snowflake Cost to the Team That Owns It https://altimate.ai/product-highlights/snowflake-cost-alerts-by-team Enterprise Platform Snowflake Cost Analysis in Plain English, with AI That Cites Every Number https://altimate.ai/product-highlights/snowflake-cost-analysis-ai-cited-answers Help Doc AI Cost Analysis (DataPilot) https://help.altimate.ai/platform/summary/datapilot-analysis/ Enterprise Platform Auto Tune Right-Sizes Snowflake Warehouses Without Slowing Queries https://altimate.ai/product-highlights/snowflake-warehouse-auto-tune-right-sizing --- # Snowflake cost attribution by ownership rule URL: https://altimate.ai/product-highlights/snowflake-cost-alerts-by-team Give every Snowflake dollar an owner. Attribute cost to teams by warehouse or by query with ownership rules, then read each team's share and the spend no team owns. [.md](https://altimate.ai/product-highlights/snowflake-cost-alerts-by-team.md) Copy as MD Enterprise Platform ## Attribute Snowflake Cost to the Team That Owns It Give every Snowflake dollar an owner. Attribute cost to teams by warehouse or by query with ownership rules, then read each team's share and the spend no team owns. Pradnesh PatilJun 19, 2026Aug 20, 20264 min read Attribute Snowflake cost to the team that owns it TL;DR - **What it is.** Snowflake cost attribution that puts each team's name on its share of the bill. You match assets to a team with ownership rules, by warehouse or by query. It works on Snowflake and Databricks. - **Who it is for.** Data platform leads and engineering managers who answer for a team's cloud spend, not for one warehouse. - **What you get.** A per-team breakdown of the bill, the queries behind each team's number, and one figure for the spend no team owns yet. ## Attribute Snowflake Cost by Warehouse or by Query Your bill is one number, but the work behind it belongs to different teams. Cost attribution puts each team's name on its share. Altimate does this two ways, and you pick the one that fits how your teams share compute. - **By warehouse.** Each team runs its own warehouses, and the cost of each follows whoever owns it. If your team uses five warehouses and mine uses ten, we each pay for ours, and no query tags are needed. - **By query.** Several teams share one warehouse. Most teams already tag their queries with an owner and an application, and attribution reads those tags, so one warehouse splits cleanly across the teams that use it. | Method | Use it when | What it needs | | --- | --- | --- | | **By warehouse** | Each team has its own dedicated compute | Nothing extra, ownership follows the warehouse | | **By query** | Several teams share one warehouse | The owner and application tags most teams already set | Pick one and hold to it. Splitting by warehouse is simpler when compute is dedicated. Splitting by query is what you want when several teams share a warehouse and you need cost per project. ## Ownership Rules Decide Which Assets a Team Pays For An ownership rule is a pattern that says which warehouses, queries or tables belong to a team. You write it once. Anything that matches it later, a warehouse someone spins up next month included, joins the team on its own. Nobody edits the rule again. | A rule can match | Example | | --- | --- | | **A warehouse**, by name or tag | Every warehouse whose name starts `ANALYTICS_` | | **A query**, by role, user or tag | Every query run under the dbt service role | | **A table**, by schema or database | Every table in one team's schema | You can stack several rules on one team. As you add each rule, Altimate shows the cost it would pull in, so you are never guessing what a rule catches. The rules group those assets and attribute their combined cost to the team. The ownership rule builder for a team, matching a warehouse by name, with a panel above it showing the cost this rule would attribute to the team before you save it. That combined cost is what the Team Spends page reports. Each team's total is the sum of what its ownership rules own, and nothing was tagged by hand to get there. Open a team and you see the individual queries attributed to it, each with its own cost. A spike then [traces back to the query that caused it](https://altimate.ai/product-highlights/snowflake-query-cost-over-time), not to a team in the abstract. A team's attributed queries, each row a single query with its cost, execution time and the warehouse it ran on, the drill-down behind that team's total on the Team Spends page. ## See the Spend No Team Owns Yet Two numbers tell you how complete your attribution is, and you read them together. | Number | What it means | How you fix it | | --- | --- | --- | | **Unaccounted spend** | Bill that matched no ownership rule at all | Add rules until they cover every asset | | **Uncategorized** | Bill that matched two teams at once | Resolve the rules that overlap | Altimate never guesses a winner. When two teams claim the same asset it marks the asset Uncategorized, names both teams, and flags the team so you can settle it. You fix both numbers in the ownership rules, never in the bill. The target is a small Uncategorized figure, which means almost every dollar has a team that answers for it. The Team Spends dashboard, a stacked bar per day split by team, with total spend, active teams and unaccounted spend across the top and Uncategorized called out as its own band. A team's Ownership section warning that its rules overlap other teams on a set of entities, with the affected teams flagged in the list until the overlap is resolved. Access rules are the other half of a team. They control what the team can see, not what it pays for, so attribution and visibility stay separate. The [Enterprise Platform overview](https://altimate.ai/platform) covers the rest. ### Talk to us about your warehouse [Book a 30-min demo →](https://calendly.com/d/ctnv-rbc-yr3/altimate-ai-overview) ## Frequently Asked Questions What is cost attribution in Altimate? It is the mapping from your cloud bill to the teams that spent it. Ownership rules match every warehouse, query and table to a team, so a bill that arrives as one number leaves as a per-team breakdown. It runs on both Snowflake and Databricks. Should I attribute by warehouse or by query? Pick one and stick to it. By warehouse applies only the warehouse ownership rules. Every query inherits the owner of the warehouse it ran on, which suits teams with dedicated compute. By query looks at each query on its own, which is what you want when several teams share one warehouse and you need cost per project. What can an ownership rule match on? Warehouses match on name or tag. Queries match on warehouse name, user, role or tag. Tables match on name, database or schema. Which operators a field offers, such as equals or startsWith, is set per field, so read the builder rather than assume. Rules inside one group join with AND, and the groups join with OR, so one team can own several unrelated parts of the account. Do I have to tag my queries for this to work? Only for query-based attribution. If you attribute by warehouse, the team owns whatever runs on its warehouses and no tags are needed. If you attribute by query, attribution reads the owner and application tags most teams already set, so there is nothing new to add. What happens when two teams claim the same asset? Altimate does not pick a winner. The asset is marked Uncategorized, the reason names both competing teams, and the team is flagged in the list. Before you save a new rule you get a preview of what it would have owned over the last 28 days, so you catch an overlap before it lands. Why is my Uncategorized spend high? Three things usually cause it. Your ownership rules may not cover every asset yet. Two rules may claim the same assets and cancel into a conflict. Or your names and tags may not match the patterns the rules look for. You fix all three in the ownership rules, and the Team Spends page tells you which one you have. How are access rules different from ownership rules? They use the same kind of pattern but do a different job. Ownership rules decide which assets a team pays for, which is attribution. Access rules decide which assets a team can see, which is role-based access control. A rule that gives one team its own view does not change who the cost belongs to. Can I use this for chargeback? That is what the per-team totals are for. The Team Spends page gives you the figure a chargeback or showback conversation runs on. It comes from ownership rules, not hand tagging, so it stays current as the account changes. On this page - Attribute Snowflake Cost by Warehouse or by Query - Ownership Rules Decide Which Assets a Team Pays For - See the Spend No Team Owns Yet ## Related Resources Enterprise Platform Databricks Cost Breakdown, from One Bill Down to One Warehouse https://altimate.ai/product-highlights/databricks-cost-breakdown-workspace-sku-cluster Enterprise Platform Root Cause Analysis for a dbt, Tableau, or Snowflake Workload Spike https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis Enterprise Platform Auto Tune Right-Sizes Snowflake Warehouses Without Slowing Queries https://altimate.ai/product-highlights/snowflake-warehouse-auto-tune-right-sizing Help Doc AI Cost Analysis (DataPilot) https://help.altimate.ai/platform/summary/datapilot-analysis/ --- # Compare two workload runs to find the cause URL: https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis Compare two runs of a dbt, Tableau, or Snowflake workload and get the root cause named. The warehouse, the task count, or the code. [.md](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis.md) Copy as MD Enterprise Platform ## Root Cause Analysis for a dbt, Tableau, or Snowflake Workload Spike Compare two runs of a dbt, Tableau, or Snowflake workload and get the root cause named. The warehouse, the task count, or the code. Anas FarooquiJun 26, 2026Jul 31, 20266 min read Root cause analysis for a dbt, Tableau, or Snowflake workload spike TL;DR - **What it is.** Root cause analysis for any Snowflake workload. It covers every run of your dbt models, Tableau workloads, notebooks, stored procedures and Streamlit apps. Compare two runs and the AI names the cause: the warehouse, the task count, or the code. dbt and Tableau come in through connections you already made, and Snowflake queries need a tag so their runs group into a workload. - **Who it is for.** Analytics engineers, data platform leads and whoever is on call when the nightly job doubles. - **What you get.** You find out why the nightly job doubled, without reading three sets of logs. ## Root Cause Analysis Starts with Every Snowflake Workload Run Your nightly job took eighty minutes last night instead of forty. Three tools each hold part of the answer, and none of them holds all of it. So you read the dbt logs, then query history, then the Tableau admin views. By the time you have an answer, the next run has already started. The Code section reads all of it in one place. You get a run history for each of these: - **dbt models**, through the dbt connection you already made. - **Tableau workloads**, once the refresh queries carry a tag. - **Notebooks.** - **Stored procedures.** - **Streamlit apps.** Under any single run you also get the child queries that ran inside it. Cost and run time sit on the same chart. A run that got expensive and a run that got slow are one investigation instead of two. Here is one workload over 28 days. The blue line is what each run cost you. The purple line is how long each run took. Across 31 runs it averaged 22 minutes 54 seconds and $2.27, and its worst run took 1 hour 6 minutes. That worst run is the one you go and compare. Root cause analysis for one Snowflake workload: a Runs Cost chart over a 28 day window with the workload name blurred out. Header stats read 22m 54s average runtime, 1h 6m max, 31 total runs, 0.97 success ratio and $2.27 average cost, and cost in blue is plotted against run time in purple on the same axes. Each point on this chart is one run. When the run's makeup changes, the chart flags that point as changed. That flag is the task-count cause showing up before you even compare two runs. The same Runs Cost chart with 50 runs plotted, cost in blue against run time in purple. A hover card on the 18 Aug run shows its cost at $7.37 and its run time at 51m 37s, and a red-outlined Run has been changed flag marks that the run's definition shifted since the run before it. ## Compare Two Runs to Find Out What Changed The platform's AI compares two runs of the same workload and names the root cause of a spike or a drop. It comes back with one of three things: | Cause | What it means | | --- | --- | | **The warehouse** | The warehouse this workload ran on changed. | | **The task count** | The workload ran a different number of tasks this time. | | **The code** | The code behind the workload changed. | Three causes will not explain every incident. They do cover the ones your runbook can act on, and the rest usually trace back to one of them. Every workload type goes through the same three checks, whether it is a dbt model or a Streamlit app. When the warehouse is the cause, you get more than a direction to look in. You get the query's lineage, and a written cause. Here that cause is 202.23 GB spilled to remote storage because [the warehouse was sized Small](https://altimate.ai/product-highlights/snowflake-warehouse-auto-tune-right-sizing). The recommendation comes with it. Run the query on a larger size, or split it into smaller chunks. A query lineage graph with two source steps feeding an inner join and a filter into a downstream model, beside a query\_with\_remote\_spillage panel reporting 202.23 GB spilled to remote storage, attributing it to a warehouse configured as Small, and recommending a larger size or splitting the query into smaller chunks. ## Send the Cause to Jira, Linear, or an Agentic IDE Once the cause has a name, it stops being an investigation and becomes work. From the opportunity you send it out as a Jira or Linear ticket, or open it in an agentic IDE with the prompt already written. The Code tab of a query page showing a CREATE OR REPLACE TABLE statement, with the Opportunities panel below it. On the create\_or\_replace\_table opportunity the Actions menu is open, listing Create Ticket and Open in IDE. You choose where the work lands and who owns it. The platform files against Jira or Linear through the Datamate that holds that integration. The ticket reaches the team that already runs on it. Here only Jira is connected. The Create Ticket dialog for the create\_or\_replace\_table opportunity. The Platform field is set to Jira, and a Select Datamate dropdown lists the account's Datamates, including a Jira integration Datamate. What matters is what travels with the ticket. It carries the opportunity, the resource and the recommended fix. Whoever picks it up does not have to start by working out what went wrong. If you would rather send it yourself, Studio opens with the prompt already written, and the Datamate already selected. The Studio assistant opened from Create Ticket, its message box pre-populated with a prompt that reads Help me create a Jira ticket with the below details, followed by the create\_or\_replace\_table optimization and its resource id, with the Jira integration Datamate already selected below the box. ## What You Get Instead of an Hour of Log Reading - **One place for every workload.** dbt models, Tableau workloads, notebooks, stored procedures and Streamlit apps. Each keeps its own run history. - **A named cause instead of a theory.** The warehouse, the task count, or the code. - **The query that got expensive.** Every run lists the child queries that ran inside it, so the finding points at one query and not at the whole job. - **A ticket someone can pick up.** Jira, Linear, or an agentic IDE. The opportunity, the resource and the fix are already written into the prompt. Proof On the workload above, one run took 1 hour 6 minutes against a 22 minute 54 second average. The AI named one cause: **202.23 GB spilled to remote storage** on a Small warehouse. You get one query, one reason and one recommendation, rather than a cost line that went up and a shortlist of suspects. For the wider Snowflake cost picture around this, see [Altimate for Snowflake](https://altimate.ai/use-cases/altimate-for-snowflake). ### Talk to us about your warehouse [Book a 30-min demo →](https://calendly.com/d/ctnv-rbc-yr3/altimate-ai-overview) ## Frequently Asked Questions Which workloads does the root cause analysis read? Everything under the Code section. dbt jobs, models and runs. Tableau dashboard refreshes and tagged queries. Notebooks, stored procedures and Streamlit apps. Each type keeps its own run history, and every run lists the child queries that ran inside it. The comparison between two runs is built from those. What exactly does the AI compare between two runs? Warehouse changes, task-count changes and code changes. The Workloads documentation describes it as comparing different runs of a workload to find the root cause of a spike or a drop. It names those three as the reasons behind a shift in cost or run time. Why not just do this in each tool separately? Because the tools do not share a vocabulary. The dbt runner has its own log format, Tableau has its own admin views, and Snowflake has query history. At two in the morning you are the one stitching them together by hand. Here one page reads all of them. What does this need from me to work on dbt? The dbt connection you already made. The Code section reads your dbt jobs, models and their runs with the costs attached. So you compare a model that ran longer against that same model's earlier runs. There is no log file to read. Model documentation and column lineage sit in the same section. What do I have to do before I can compare Tableau refreshes? Tag the queries the refresh issues, through the Edit Initial SQL flow on the data source. Untagged, a dashboard refresh tells you nothing. All you see is a bigger warehouse bill that no team owns. The tag turns that refresh into a query set with its own history. A refresh that got expensive then names the query that did it. What if the spike is inside a notebook or a stored procedure? They work like every other workload type. You get a run history, and the child queries under each run. The comparison reads both. The finding points at the one child query that got more expensive, not at the whole notebook. How does this relate to tracking one query over time? This is the zoomed-out version. Here you compare a workload run against its own earlier runs, and the cause comes back as one of three things. Tracking a single query across its executions is the same idea at query level instead of workload level, on the same account. Where is this documented? The Workloads, dbt Models and Tableau Workloads pages. Those sit behind a partner login, because the platform documentation is gated. The [Enterprise Platform overview](https://altimate.ai/platform) is the public description, and your account team can walk this through on your own workloads. On this page - Root Cause Analysis Starts with Every Snowflake Workload Run - Compare Two Runs to Find Out What Changed - Send the Cause to Jira, Linear, or an Agentic IDE - What You Get Instead of an Hour of Log Reading ## Related Resources Enterprise Platform Track Snowflake Query Cost Over Time, Run by Run https://altimate.ai/product-highlights/snowflake-query-cost-over-time Enterprise Platform Snowflake Cost Analysis in Plain English, with AI That Cites Every Number https://altimate.ai/product-highlights/snowflake-cost-analysis-ai-cited-answers Enterprise Platform Attribute Snowflake Cost to the Team That Owns It https://altimate.ai/product-highlights/snowflake-cost-alerts-by-team Help Doc AI Cost Analysis (DataPilot) https://help.altimate.ai/platform/summary/datapilot-analysis/ --- # Spot the run where a query got expensive URL: https://altimate.ai/product-highlights/snowflake-query-cost-over-time Track one Snowflake query's cost over time. Every run sits under one row, so you see which run started costing more and what the platform found wrong. [.md](https://altimate.ai/product-highlights/snowflake-query-cost-over-time.md) Copy as MD Enterprise Platform ## Track Snowflake Query Cost Over Time, Run by Run Track one Snowflake query's cost over time. Every run sits under one row, so you see which run started costing more and what the platform found wrong. Saurabh AroraJul 5, 2026Jul 31, 20266 min read Track Snowflake query cost over time, run by run TL;DR - **What it is.** Snowflake query cost over time, with every run of the same query under one row on the Code page. Each run carries what the platform found wrong with it, so you can compare one run to the next. Nothing to configure. - **Who it is for.** Analytics engineers and data platform leads who own long-lived queries behind dashboards and nightly jobs. - **What you get.** You find the exact run where a query started costing more, and what changed on it. ## Snowflake Query Cost Over Time Sits Under One Row A query took twelve minutes six months ago. Today it takes forty. It never tripped an alert, because each week only added a minute or two as the upstream table grew. Nobody checks one query week by week. The Groups tab on the Code page does that checking for you. One row is one query. Every run of it sits under that row, going back as far as your history does. The platform writes the group name, so you can tell what a row is before you open it. The problem is that the same query rarely looks identical twice. A nightly job filters on a rolling date window, so its text and its hash change every run. Match on that hash and you see thousands of one-off queries that never repeat. The platform reads past the changing dates and files every run of that query under one group. That gives one query a stable identity across months of runs. A dashboard firing the same SELECT every night becomes one row, with all of those runs under it. You are no longer matching up thousands of separate executions by hand. ## See What the Platform Found Wrong on Each Run The platform checks every run against a list of [known anti-patterns](https://altimate.ai/blog/dbt-anti-patterns-ai-should-catch), and the finding it attaches to that run is called an insight. Insights have names like `warehouse_resize_prospect`, `query_with_local_spillage` and `query_timed_out`. An insight applies at one of two levels: | Level | Scope | Example | | --- | --- | --- | | **Group Insight** | The query and every run of it | `zero_impact`, `warehouse_resize_prospect` | | **Per-run insight** | One single execution | `query_timed_out` on last night's run but not the night before | Say a query used to come back `warehouse_resize_prospect` and now comes back `query_timed_out`. That tells you two things. The platform first thought the warehouse was too small for that query. Later, the same query stopped finishing at all. You also get a date out of it. The insight changed on one specific run. It did not drift a little at a time across all of them. You know which day to look at. A rising cost line on its own can never tell you that. The diagram below follows one query across five runs, with the run where the insight changed marked in red. A diagram titled Same query hash, changing insight. Runs 1 to 3 carry the insight warehouse\_resize\_prospect at $0.42, $0.44 and $0.51, a dashed red line marks the transition, and runs 4 and 5 carry query\_timed\_out at $1.20 and $1.85. A Groups panel on the right lists two query groups with sparkline cost trends and a current insight column. ## Rank Every Query by What It Cost You The Groups tab is the list you audit from. Every query is one row. You sort those rows by total cost or by cost trend, and each row also shows the insight that applies to it right now. Here one group ran 120 times in four weeks, at a 55 minute average and $3.89K total across those runs. The Groups tab on the Code page, listing parameterized query groups. One group is highlighted, with its total run count of 120, a 55 minute average execution time, a $32.42 average cost, a $3.89K total cost and a $50.77K annualized cost, beside columns for warehouses, users and insights. Sort by cost trend and the query that has been getting more expensive comes to the top of the list. You did not have to know that query existed to find it. Each row rolls up from the Queries feed. Every single execution in that feed already carries its own estimated cost, how long it ran, and its insight tags. The Queries page filtered to Snowflake, listing individual executions with estimated cost, execution time, and insight tags including create\_or\_replace\_table, query\_with\_local\_spillage, select\_star, agg\_before\_join, exploding\_join and long\_running\_query, alongside the warehouse and user for each run. You can click a tag. Click `redundant_filter_condition` and the detail view highlights the exact lines of the WHERE clause behind it. Below that, it restates the filter in plain words. A redundant\_filter\_condition detail. The upper pane highlights lines 495 to 504 of a WHERE clause in the query source, and below it a Query card and a Problem card sit side by side, restating the same filter as a domain list plus a year test plus a quote-stage and booked test. ## What You Get from One Query Group A group gives you more than a list of runs. It tells you what is wrong across every run of a query at once. Then you open the one run that proves it. The opportunities on a group apply to the whole group, not a single run. Every run of this query spills 26.05 GB to local storage. So one fix, resize the warehouse, pays off across all 120 runs at once. The Opportunities tab for a query group, showing two query\_with\_local\_spillage findings, each reporting 26.05 GB spilled to local storage during execution, with an effort rating and a time-savings estimate. From there you open any single run, each with its own cost, run time and timestamp. That is where a group-wide finding turns into the one run you go and read. The Queries tab inside a group: a Query Executions table where every row shares one query hash but carries its own estimated cost, execution time and timestamp, the individual runs that roll up into the group. - **A cost history per query.** Every run of it, with what each run cost you. - **The run where it changed.** The one execution where the insight went from one problem to another. - **The clause to fix.** The lines of SQL behind that insight, highlighted for you. Proof A run's cost is not the warehouse bill divided by the number of queries. The platform reads everything else running on that warehouse at the same time and splits the cost on that basis. That is what makes one run comparable to the run before it. Without it, a cost trend would only be telling you how busy the warehouse was that night. Query Insights is part of the Enterprise Platform, and the [Enterprise Platform overview](https://altimate.ai/platform) covers the rest of what it tracks on your Snowflake spend. ### Talk to us about your warehouse [Book a 30-min demo →](https://calendly.com/d/ctnv-rbc-yr3/altimate-ai-overview) ## Frequently Asked Questions How does the platform know two runs are the same query? The Groups tab matches queries by their pattern, not their exact text. A rolling date window changes the text and the hash on every run, but the pattern stays the same. The platform reads past those dates and files every run under one group. One row is one query, every run sits underneath it, and you configure nothing to get this. What is the difference between a Group Insight and a per-run insight? A Group Insight applies to the query and all of its runs, for example `zero_impact` or `warehouse_resize_prospect`. A per-run insight applies to one execution, for example `query_timed_out` on last night's run but not the night before. When a per-run insight shows up where it never used to, that is your regression. What kind of regression does this actually catch? Silent ones. A query that got slower because an upstream table doubled. A query that started spilling to remote storage after someone reshaped a warehouse. A query that began retrying under contention. All three change the insight between runs. A plain cost chart would show the line going up and tell you nothing about why. Can I see which clause is causing the problem? Yes, for the insights that point at a piece of SQL. Click the tag and the detail view highlights the exact code behind it, next to a plain restatement of what that code does. A redundant filter condition gives you the predicate itself, plus the reason the extra test buys you nothing. How do queries get grouped into a job? A job is a set of queries that ran together, and the platform spots those sets and names them for you. Open any query in a group and you get the insights for that one execution. Use See Group Insights for the findings that apply to the whole group. Where does the cost figure on a run come from? The platform looks at everything else running on that warehouse at the same time and splits the cost on that basis. It does not divide the bill flatly across queries. That is what makes one run comparable to the run before it. Without it, a cost trend would only tell you how busy the warehouse was that night. How does this differ from workload root-cause analysis? The size of the thing you compare. Workload root-cause analysis compares two runs of a whole workload. It names one of three causes: the warehouse, the task count, or the code. This is the same idea one level down, on the same account. The unit is a single query, and the evidence is the insight on each of its runs. Where can I see this on my own account? The Query Insights documentation sits behind a partner login. It covers the Groups tab, See Group Insights, Summarize Query and the Query Explanation agent. [Altimate for Snowflake](https://altimate.ai/use-cases/altimate-for-snowflake) is the public description, and your account team can run the Groups tab against your own query history. On this page - Snowflake Query Cost Over Time Sits Under One Row - See What the Platform Found Wrong on Each Run - Rank Every Query by What It Cost You - What You Get from One Query Group ## Related Resources Enterprise Platform Root Cause Analysis for a dbt, Tableau, or Snowflake Workload Spike https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis Enterprise Platform Snowflake Cost Analysis in Plain English, with AI That Cites Every Number https://altimate.ai/product-highlights/snowflake-cost-analysis-ai-cited-answers Enterprise Platform Auto Tune Right-Sizes Snowflake Warehouses Without Slowing Queries https://altimate.ai/product-highlights/snowflake-warehouse-auto-tune-right-sizing Help Doc AI Cost Analysis (DataPilot) https://help.altimate.ai/platform/summary/datapilot-analysis/ --- # Auto Tune for Snowflake compute savings URL: https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost Altimate Lite is a Snowflake Native App that cuts warehouse cost in your own account. It copies nothing out. Turn Auto Tune on and idle time stops billing. [.md](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost.md) Copy as MD Altimate Lite for Snowflake ## Reduce Snowflake Idle Warehouse Spend with Altimate Lite Altimate Lite is a Snowflake Native App that cuts warehouse cost in your own account. It copies nothing out. Turn Auto Tune on and idle time stops billing. Atharva ShahJul 28, 2026Jul 31, 20266 min read Reduce Snowflake idle warehouse spend with Altimate Lite TL;DR - **What it is.** Altimate Lite for Snowflake is a Snowflake Native App you install straight from the [Snowflake Marketplace](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) in under five minutes. From there, you flip an Auto Tune toggle on any eligible warehouse to let it run. - **Who it is for.** Data platform leads who want to cut wasted compute, and FinOps managers holding a Snowflake budget. - **What you get.** Altimate Lite suspends a warehouse as soon as it goes idle, and scales its clusters back when demand drops. It checks far more often than Snowflake's own settings do. ## Altimate Lite Cuts Snowflake Warehouse Cost When Idle [Altimate Lite is a Snowflake Native App](https://altimate.ai/blog/altimate-lite-autonomous-agents-for-snowflake-cost-optimization). The whole app installs into your own Snowflake account and runs there, on your own compute. Altimate never copies anything out. You open no vendor connection and file no data-residency exception. The [Altimate Lite overview](https://help.altimate.ai/snowflake-native-app/) has the privileges it asks for and the uninstall path. Auto Tune is the feature that moves the bill. It is a toggle on the Warehouses page. You turn it on per warehouse, so you pick which ones it manages. Turn it on and that warehouse gets tuned against real usage from then on: - **Suspended when it goes idle.** Within about a minute of the last query finishing. - **Clusters scaled down when demand drops.** Back up again when demand returns. Two things it never does. It never changes a warehouse's size, and it never touches a warehouse you did not toggle on. Suspend and cluster scale-down are the only two settings it writes. ### Why this beats Snowflake's built-in auto-suspend Snowflake already ships [auto-suspend and auto-resume](https://altimate.ai/blog/snowflake-cost-optimization-techniques). They work off fixed settings that somebody picked once and rarely revisits. | | Snowflake auto-suspend | Altimate Auto Tune | | --- | --- | --- | | **How often it looks** | A fixed probe interval | Far more often, on an interval you configure | | **What it acts on** | The timeout in the warehouse definition | Real usage, read continuously | | **When it adapts** | Only when somebody edits the setting | As the workload changes, on its own | Auto Tune throttles its own checks back while a warehouse stays quiet. Watching closely costs you almost nothing. The suspend itself still lands within about a minute of the last query finishing. ## The Warehouses Page Lists Cost, Savings, and a Toggle The page gives you two counts, and together they are your to-do list: - **Eligible.** Warehouses Auto Tune could take on today, but nobody has turned on yet. This is the work left. - **Enabled.** Warehouses it already manages. This is the work done. The Altimate Lite Warehouses page, showing Snowflake warehouse cost for an account with 63 warehouses: $653.40 of 30-day cost, $1,547 per year in potential savings, 3 warehouses eligible but not yet tuned and 9 with Auto Tune already on, a Daily Spend and Savings chart, and COMPUTE\_WH showing Saved $70.59 with an estimated annual $859 beside its Auto Tune toggle. Each day in the Daily Spend & Savings chart stacks three colors: | Band | Colour | What it is | | --- | --- | --- | | **Spend** | Yellow | What that warehouse actually cost that day | | **Possible Savings** | Orange | Idle spend Auto Tune could still recover, where it is off | | **Auto Tune Savings** | Green | Idle spend Auto Tune already recovered | A tall yellow bar with a thick orange cap is uncaptured money. A mostly green bar is a warehouse Auto Tune has already caught up with. Each row reads either **Saved $X**, money already captured, or **Potential $X**, an estimate of what turning it on would recover. Open any warehouse to see its two savings figures together and every decision Auto Tune made on it, timestamped and exportable as CSV. That is the audit trail you hand to whoever asks where the savings came from. COMPUTE\_WH detail page: Key Information, an enabled Auto Tune card reading Saved last 30 days $70.59, Realized Savings of $71 for the window beside Projected Savings of $859 per year annualized, an Agent Decisions chart, and an Auto Tune history of timestamped suspend events. Realized Savings is actual dollars recovered in the window you picked. Projected Savings is always annualized. The two use different time bases deliberately, so a 30-day number never gets read as a year. ## Auto Tune Reports What It Saved, Warehouse by Warehouse - **It runs where your data already is.** Altimate never copies your usage data out, so nobody has to review where it went. - **Per-warehouse opt-in.** Auto Tune acts only on what you toggled on, and one click reverses it. - **Every savings number is backed by a log.** Realized dollars for the window you picked, Projected annualized, and a timestamped record of every decision behind both. Proof Across private preview, customers averaged 19% off warehouse cost. Query performance and queue times held steady. The single best day reached 57.5%, usually a weekend, when workloads are least consistent. Altimate Lite prices two halves of your Snowflake bill. This page covers warehouse compute. The same install also [splits your Cortex AI bill by service, user, and model](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown). The [Warehouses guide](https://help.altimate.ai/snowflake-native-app/warehouses/) covers which warehouse classes Auto Tune can manage and the grant a warehouse sometimes needs first. ### Talk to us about your warehouse [Book a 30-min demo →](https://calendly.com/d/ctnv-rbc-yr3/altimate-ai-overview) ## Frequently Asked Questions Does any warehouse data leave my Snowflake account? No. Altimate Lite is a Snowflake Native App, so it runs inside your Snowflake account and makes zero outbound network calls. The Warehouses and AI Services pages are built from a copy of your own usage data that stays in your account and refreshes on a schedule. Data residency follows your account, in every region the Marketplace listing is published in. What does Auto Tune actually change on a warehouse? The docs name two adjustments. It suspends the warehouse when it goes idle. It scales clusters down when demand drops. So spend tracks real workload, not a setting somebody picked once. Nothing else about the warehouse changes, and it only ever acts on warehouses you toggled on yourself. Where do I find what Auto Tune saved on my own account? On the Warehouses page, as Realized Savings for the window you picked, with Projected Savings annualized beside it. Open any warehouse and you get every decision Auto Tune made on it, timestamped, and Export CSV takes the whole table out. Quote your own figure internally rather than ours. The decision log sitting underneath it is what makes it hold up. How fast does it react, and can I turn it off? Auto Tune reacts within about a minute of an opted-in warehouse going idle. It checks far more often than Snowflake's own auto-suspend probe does, on an interval you can configure. While a warehouse stays quiet, it throttles those checks back. Turning it off is the same toggle you turned on, and it takes effect without uninstalling the app. The Agent Decisions chart on the detail page shows how often it has acted over the window you picked. Can Auto Tune manage every warehouse in my account? No, and the app tells you which ones it can. Some Snowflake-managed warehouse classes are not eligible at all, and on an eligible warehouse the toggle sometimes asks for one grant before it takes effect. The Warehouses guide names both cases, so approving the install dialog is not always the only grant step. Why does one warehouse row say Saved and another say Potential? A row shows Saved when Auto Tune is on and the money has already been captured. It shows Potential when Auto Tune is off, and the figure is then an estimate of what turning it on would recover. Both carry an annualized figure underneath. Export CSV next to the search box downloads the full table. What does the app itself cost to run? Auto Tune and the web UI share a single compute pool. The app's own NATIVE\_APP\_WH warehouse only spins up when the UI runs a usage query. The app's FAQ states that net savings outweigh that runtime cost. What happens to my warehouses if I uninstall the app? A single DROP APPLICATION removes the application, the services it owns, and the dedicated NATIVE\_APP\_WH warehouse. It also removes every row of app state, including savings logs and Auto Tune events. It does not undo the idle-timeout changes already applied. Those warehouses keep running with whatever setting Auto Tune last gave them. On this page - Altimate Lite Cuts Snowflake Warehouse Cost When Idle - The Warehouses Page Lists Cost, Savings, and a Toggle - Auto Tune Reports What It Saved, Warehouse by Warehouse ## Related Resources Altimate Lite for Snowflake Snowflake Cortex AI Cost Breakdown by Service, User, and Model https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown Enterprise Platform Auto Tune Right-Sizes Snowflake Warehouses Without Slowing Queries https://altimate.ai/product-highlights/snowflake-warehouse-auto-tune-right-sizing Help Doc AI Cost Analysis (DataPilot) https://help.altimate.ai/platform/summary/datapilot-analysis/ Enterprise Platform Snowflake Cost Analysis in Plain English, with AI That Cites Every Number https://altimate.ai/product-highlights/snowflake-cost-analysis-ai-cited-answers --- # Where your Snowflake AI credits actually go URL: https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown Altimate Lite gives you a Snowflake Cortex AI cost breakdown by service, user, and model, then drills into single calls. Usage data never leaves the account. [.md](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown.md) Copy as MD Altimate Lite for Snowflake ## Snowflake Cortex AI Cost Breakdown by Service, User, and Model Altimate Lite gives you a Snowflake Cortex AI cost breakdown by service, user, and model, then drills into single calls. Usage data never leaves the account. Atharva ShahJul 28, 2026Jul 31, 20266 min read Snowflake Cortex AI cost breakdown by service, user, and model TL;DR - **What it is.** Altimate Lite for Snowflake, an app from the [Snowflake Marketplace](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) that prices every Cortex feature you use from the usage data Snowflake already records. It installs in under five minutes. No extra setup needed. - **Who it is for.** Data platform leads, FinOps managers and AI engineers. Anyone who answers for a Cortex line nobody can currently break apart. - **What you get.** Your Cortex Cost Breakdown split by service, by user, and by model. ## See What Each Cortex Service Costs You Altimate Lite runs entirely inside your Snowflake account, and its AI Services page is where the Cortex line comes apart. The app reports each feature on its own, because they bill for different things. Cortex is one half of what the app prices, and the [Altimate Lite overview](https://help.altimate.ai/snowflake-native-app/) has the warehouse half. | Service | What it bills for | | --- | --- | | **AI SQL Functions** | LLM tasks run as SQL functions, directly on your data | | **Cortex Analyst** | plain-English questions answered against a semantic model | | **Cortex Search** | search over your own documents and unstructured data | | **Cortex Agents** | an orchestrated agent combining Analyst and Search | | **Snowflake Intelligence** | the natural-language interface for exploring data | | **Cortex Code** | the AI coding assistant for SQL and Python | | **Fine-tuning** | customizing a base model on your own data | For the window you pick, the page gives you five things at a glance: - **Total AI cost**, in dollars. - **Token count**, split into input and output. - **Active users**, how many people used Cortex at all. - **Services used**, how many of the seven you actually touched. - **A daily cost trend**, stacked by service. Under that, the page ranks every service by spend and gives its share of the total. The Snowflake Cortex AI cost breakdown on the AI Services page, showing $2.67 of total AI cost, 416.0K total tokens with the input and output split, 4 active users, an AI Services Used tile, a daily cost chart stacked by service, and a Spend by service table ranking Cortex Analyst at $2.21 and 76.1% of spend above Cortex Code, AI SQL Functions, Fine-tuning, and Cortex Search. Start with the share column before the cost column. On the account above, Cortex Analyst is 76.1% of all Cortex spend. One service is nearly the whole Cortex line. Fixing Analyst usage is the only work worth doing. The other four services together are the remaining quarter, and they can wait. If instead the top service came in at 30%, spend is spread out and there is no single thing to fix. ## Filter the Snowflake Cortex AI Cost Breakdown by Service Services, Users, and Models are three tabs over the same window, each a ranked table with a share-of-total column. The Service, User, and Model filters above them combine. One service plus one model is a single slice, and every figure on the page redraws around it. Export CSV then pulls every matching row for the current filter, not just the rows on screen. That turns "which model is driving the Analyst bill for the growth team" into three clicks. Click any row and the drill-down reaches the individual calls behind the total. AI SQL Functions drill-down showing $0.09 of cost over 30 days, 48.9K tokens, 190 queries, 0.88 credits per million tokens, a daily spend chart scoped to that slice, and an Activity table of 190 rows carrying query ID, user, function, model, role, tokens, pages, cost, and completion time. How deep that table goes depends on what Snowflake reports for the service: | Service | What one row is | What it carries | | --- | --- | --- | | **AI SQL Functions, Cortex Code, Agents, Intelligence** | One call | Query ID or Request ID, plus user, function, model, tokens | | **Cortex Analyst** | One user | A Requests count in place of tokens, because Snowflake reports hourly request counts per user | | **Cortex Search** | One named search service | A Tag marking query-time or indexing-time credits, because no user or request id exists | ## Trace Snowflake Cortex Cost to Each Service, User, or Model - **A service split.** Every Cortex feature the app tracks on its own line, with its share of total spend beside it. - **A user and model split.** Two more ranked views over the same window, filterable together with the first. - **Call-level evidence.** A drill-down that stops exactly where Snowflake's own reporting stops. Where the dollar figure comes from [Snowflake bills Cortex in credits](https://altimate.ai/blog/snowflake-cortex-ai-cost-catalog-billing-quotas). Every dollar on this page is a conversion, and the page names the rate it used. Most features draw on the **AI Credit pool**, priced from your account's own invoiced AI spend. Where that history does not exist, the app falls back to Snowflake's published **$2.00 per credit** for global routing and **$2.20** for regional. Fine-tuning and Cortex Analyst bill **regular compute credits** at your account's actual rate. When no reliable rate exists, the page shows credits and refuses to guess at dollars. Cortex is one half of what Altimate Lite prices. The other half is compute, and [Auto Tune cuts idle Snowflake warehouse spend](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) from the same install. How the rate is set for each credit pool is in the [AI Services guide](https://help.altimate.ai/snowflake-native-app/ai-services/). ### Talk to us about your warehouse [Book a 30-min demo →](https://calendly.com/d/ctnv-rbc-yr3/altimate-ai-overview) ## Frequently Asked Questions Does my Cortex usage data leave the account to be analyzed? No. Altimate Lite runs entirely inside your Snowflake account, with zero outbound network calls. The AI Services page is built from a copy of your own usage data, stored inside your account. The Cortex calls the app makes for its own features also stay in-account and bill your existing Cortex credits. Which Cortex services show up on the page? Seven of them. AI SQL Functions, Cortex Analyst, Cortex Search and Cortex Agents. Then Snowflake Intelligence, Cortex Code and Fine-tuning. Each gets its own color on the daily cost chart and its own row in the Spend by service table. Services you have never used carry no spend. The adoption tile reports how many are in use. Why do some services report requests instead of tokens? Snowflake reports them differently. The app does not invent a number to fill the gap. AI SQL Functions, Cortex Code, Agents and Intelligence each expose one row per call, with a Query ID or Request ID. Cortex Analyst only reports hourly request counts per user. Cortex Search carries no user or request id at all. How is the dollar figure calculated when Snowflake bills in credits? It depends on which pool the feature bills. Most Cortex features bill a separate AI Credit pool. Those are priced from your account's own invoiced AI spend where that history exists. Where it does not, the app falls back to Snowflake's published flat rate of $2.00 per credit for global routing and $2.20 for regional. Fine-tuning and Cortex Analyst bill regular compute credits instead, priced at your account's actual compute rate. What happens when no reliable rate is available? The page shows credits rather than guessing at dollars. Hovering the Total AI Cost info icon names the exact rate in use for the figure on screen. So a number you take into a budget review traces back to the rate that produced it. Can I narrow the whole page to one team's usage? Yes. The Service, User, and Model filters are searchable. They also combine. You can hold one service, one user, one model, or any mix of the three. Every number recalculates around that filter, including the tiles, the daily chart, and all three tables. Export CSV then downloads every matching row. How current are the numbers, and can I force a refresh? A refresh runs automatically once a day. The first one happens the moment you install the app, and pulls thirty days of history so the page has something to show straight away. Every refresh after that is incremental. Run now on the Data Ingestion page pulls the latest figures without waiting for the schedule. Do I need Cortex enabled before I install? The app installs on Snowflake Standard edition or higher, and the SNOWFLAKE.CORTEX\_USER database role is one of the privileges surfaced in the install dialog. If your account is not using any Cortex AI features yet, the AI Services page has nothing to show. The Warehouses page works either way. On this page - See What Each Cortex Service Costs You - Filter the Snowflake Cortex AI Cost Breakdown by Service - Trace Snowflake Cortex Cost to Each Service, User, or Model ## Related Resources Altimate Lite for Snowflake Reduce Snowflake Idle Warehouse Spend with Altimate Lite https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost Enterprise Platform Snowflake Cost Analysis in Plain English, with AI That Cites Every Number https://altimate.ai/product-highlights/snowflake-cost-analysis-ai-cited-answers Enterprise Platform Auto Tune Right-Sizes Snowflake Warehouses Without Slowing Queries https://altimate.ai/product-highlights/snowflake-warehouse-auto-tune-right-sizing Help Doc AI Cost Analysis (DataPilot) https://help.altimate.ai/platform/summary/datapilot-analysis/ --- # Why is my dbt model slow? Ask the warehouse URL: https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan Run a dbt model, click Profile Query, and Altimate Code runs EXPLAIN on your own warehouse to tell you in plain English why that query is slow. [.md](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan.md) Copy as MD PowerUser Plugin for dbt ## Profile a Slow dbt Query and Read Its Execution Plan in Plain English Run a dbt model, click Profile Query, and Altimate Code runs EXPLAIN on your own warehouse to tell you in plain English why that query is slow. Atharva ShahJul 31, 2026Aug 21, 20264 min read Profile a slow dbt query and read its execution plan in plain English TL;DR - **What it is.** A Profile Query button in the Query Results panel that costs one click and no new setup. It hands your compiled SQL to Altimate Code, which runs your warehouse's own EXPLAIN and reads the plan back to you. - **Who it is for.** Analytics engineers who own a dbt model that got slower and cannot read a query plan. - **What you get.** The reason a slow dbt query is slow, in words, without learning your warehouse's plan format. Your warehouse already knows [why your model is slow](https://altimate.ai/blog/power-user-for-dbt-can-now-tell-you-why-your-model-is-slow). Before it runs anything, it works out a plan. The plan names the tables to read, the join strategy to use and the data each step scans. That plan comes back as hundreds of lines of nested operators written for a database engineer, so almost nobody asks for it. So you end up guessing why a model is slow. Rewrite a CTE, move a filter earlier, try an incremental. When that fails, you still do not know why. ## Click Profile Query When Your dbt Model Runs Slow Run the query the way you already do, with Cmd+Enter on a Mac or Control+Enter elsewhere. Once results come back, a **Profile Query** button becomes available in the Query Results toolbar. It appears only when there is data to profile. Clicking it opens an Altimate Code chat titled after the file you were in, carrying your SQL and a fixed instruction. That instruction asks for three things: - **Performance bottlenecks.** Which step your query spends its time in. - **Data distribution issues.** Skew or volume the plan did not expect. - **Optimization opportunities.** What to change, and where. The SQL it sends is the **compiled** query. It falls back to the raw text only when no compiled version exists. A `ref()` says nothing about how much data gets read. The compiled query is what your warehouse actually ran. The button becomes available on a customers.sql query that returned 500 rows in 4.6 seconds. The Query Results panel in VS Code, mid-click on the Profile Query button that profiles a slow dbt query, on the customers.sql model in the jaffle-shop-1 project. The toolbar reads Preview 500 rows in 4.6s next to a 227 credits chip, and the result grid below reads 500 x 7. ## Altimate Code Runs EXPLAIN Against Your Own Warehouse Profiling answers questions a chatbot cannot, because [Altimate Code](https://help.altimate.ai/dbt-power-user/teammates/altimate-code/) holds a live warehouse connection. Its `sql_explain` tool runs EXPLAIN on a query and returns the execution plan. Use it to [diagnose slow queries](https://altimate.ai/blog/optimizing-snowflake-queries-expert-strategies-for-data-engineers-and-analysts), find full table scans and work out join strategies. EXPLAIN returns the plan the warehouse intends to follow, and its estimate of how many rows each step produces. EXPLAIN ANALYZE goes further and runs the query for real. That is how you catch a step that expected a hundred rows and got four million. **Snowflake does not support EXPLAIN ANALYZE.** On Snowflake you read the plan and its estimates instead. So the numbers you get there are a forecast rather than a measurement. Two limits come from the tool itself, so they apply wherever you run it: - **A warehouse connection is required.** Profiling reads a real plan from a real database. With nothing configured there is nothing to read. - **Values have to be inlined.** Profiling cannot explain a query that still holds a placeholder like `?`, `:name` or `$1`. Nothing supplies the value. Compiled dbt SQL normally satisfies this already. ## Pick Explain When You Have Not Run the Query Yet Two Altimate Code buttons answer different questions. | Button | How you reach it | What it answers | | --- | --- | --- | | Explain with Altimate Code | The SQL tab | What does this query do? It reads the code in your editor, or the part you selected, and describes the logic | | Profile Query | The Query Results toolbar, after a run | How does this query perform here? It reads diagnostic output from your warehouse | Explain never touches your data, which makes it the right call on a teammate's model you are reviewing. Profile needs a run and a connection, and pays you back with facts about your data. On `customers.sql`, Explain maps all seven columns back to their source staging model. It also places the model at the end of the DAG, behind three staging models. The Altimate Code chat titled Explain: customers.sql. A table maps each of the seven columns to its source model and a one-line description, and a DAG Position block below it shows stg\_customers, stg\_orders and stg\_payments feeding customers (mart). Both leave the context loaded in the chat, so a follow-up is one sentence. Ask why a step is expensive, or what changes if you filter earlier. A profile does not promise a faster query and does not rewrite your model. It tells you what to try next. So you get a named cause instead of a guess, read off the compiled query that actually ran. For the rest of the panel, see the [Query Results panel guide](https://help.altimate.ai/dbt-power-user/test/queryResults/). ### Install the dbt VS Code extension [Install from Marketplace](https://marketplace.visualstudio.com/items?itemName=innoverio.vscode-dbt-power-user) [Book a 30-min demo →](https://calendly.com/d/ctnv-rbc-yr3/altimate-ai-overview) [Read the docs →](https://help.altimate.ai/dbt-power-user/) ## Frequently Asked Questions Why is my dbt model slow? Usually because of something the SQL text does not show. Your filter may be written so the warehouse cannot prune on it. Your join may explode before its filter runs. Your row estimate may be wrong by orders of magnitude. The warehouse names the real cause in its execution plan, and Profile Query reads that plan back to you in plain English. Where is the Profile Query button? In the Query Results panel toolbar. It only appears once a query has returned results, so run the query first. If you have results and no button, the panel is showing a view other than the default query result. Does it profile my Jinja or my compiled SQL? The compiled SQL, and it falls back to the raw query text only when no compiled version is available. This is the right choice for performance work, because the compiled query is what your warehouse executed. Do I need a warehouse connection? Yes. Profiling works by running your warehouse's own EXPLAIN and reading what comes back, so with no connection configured there is no plan to read. This is the main way Profile differs from Explain, which only reads your code. Does this work on Snowflake, BigQuery and Postgres? Yes. The tool runs the EXPLAIN of whichever warehouse you connected it to, so it is not built for one database. How deep the answer goes does depend on what your database reports. Snowflake does not support EXPLAIN ANALYZE, so there you get the plan and its estimates rather than measured runtime figures. What is the difference between EXPLAIN and EXPLAIN ANALYZE? EXPLAIN returns the plan and the warehouse's estimates without running the query. EXPLAIN ANALYZE actually executes it and reports real numbers, so it is slower and costs a real run. The profiling tool defaults to plain EXPLAIN. Why did profiling fail on my query? The most common cause is bind placeholders. A query still carrying `?`, `:name` or `$1` cannot be explained. The tool says so rather than passing a broken statement to your warehouse. Inline the values and run it again. How is this different from pasting my SQL into a chatbot? A chatbot can only reason about the text you gave it. It has no way to know your table sizes or your partitioning. It cannot know that the optimizer's row estimate was wrong by four orders of magnitude. Those facts live in your warehouse, and profiling is what brings them into the conversation. Will it make my query faster on its own? No. It names the likely cause and suggests what to try. Applying a fix stays with you, and so does deciding whether the tradeoff is right for your model. On this page - Click Profile Query When Your dbt Model Runs Slow - Altimate Code Runs EXPLAIN Against Your Own Warehouse - Pick Explain When You Have Not Run the Query Yet ## Related Resources Enterprise Platform Track Snowflake Query Cost Over Time, Run by Run https://altimate.ai/product-highlights/snowflake-query-cost-over-time Enterprise Platform Root Cause Analysis for a dbt, Tableau, or Snowflake Workload Spike https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis Enterprise Platform Databricks Cost Breakdown, from One Bill Down to One Warehouse https://altimate.ai/product-highlights/databricks-cost-breakdown-workspace-sku-cluster Help Doc BigQuery Cost Estimator https://help.altimate.ai/dbt-power-user/test/bigquerycost/ --- # 10 Reasons AI Coding Agents Fail at Data Engineering URL: https://altimate.ai/blog/why-ai-coding-agents-fail-at-data-engineering AI coding agents fail at data engineering for one structural reason. Ten specific failure modes, why each one happens, and the fix that holds. ## 10 Reasons AI Coding Agents Fail at Data Engineering AI coding agents fail at data engineering for one structural reason. Ten specific failure modes, why each one happens, and the fix that holds. Atharva Shah 2026-05-05 2026-09-04 11 min read On this page 12 sections 1. Failure 1. The Agent Guesses Your Schema 2. Failure 2. SQL That Compiles but Joins Wrong 3. Failure 3. Model Edits That Ignore the DAG 4. Failure 4. Generated Queries That Leak Personal Data 5. Failure 5. No Idea What the Query Will Cost 6. Failure 6. Every Session Starts From Zero 7. Failure 7. The Agent Recommends Anti-Patterns Cheerfully 8. Failure 8. Tests That Never Assert Anything 9. Failure 9. Correctness Judged by Another Model 10. Failure 10. Nothing Between the Agent and Production 11. Our Runs Put the Failure Rate Near One Task in Four 12. What Makes AI Coding Agents Fail at Data Engineering tl;dr AI coding agents fail at data engineering because the agent cannot see most of what it is changing: the warehouse schema, the dependency graph, the rows and the bill all live outside the file it edits. The ten failure modes below are ranked by what they cost. Nine of the ten trace to a missing fact or a missing check, and a stronger model fixes none of them. The expensive failures return a wrong number instead of an error, so the SQL compiles, runs and goes unquestioned. The worst failure is having nothing between the agent and production, and it is also the cheapest one to prevent. Reliability, not model capability, is the constraint. Even our best ADE-Bench run left a quarter unfinished, 32 of 43, and the same setup swung to 29 on a worse run. Build for that failure rate, not the run that goes well. An AI coding agent that refactors a React component cleanly will still write SQL that is wrong. The wrong SQL does not fail in any visible way. It compiles, it runs, and it returns a set of numbers that look reasonable. A wrong join raises no exception, so nothing tells you the numbers are wrong. The difference between refactoring a React component and editing a dbt model is what the agent can see. A React component carries most of its own context inside the file. A dbt model carries almost none of it. When the agent edits a dbt model, it receives that single file, and four things it needs sit somewhere else: - the warehouse schema, - the dependency graph between models, - the rows the query will run against, and - the compute bill the query will produce. The agent is working from one piece of the problem out of five. That gap is why AI coding agents fail at data engineering. Below are ten failure modes ordered by what they cost, each with the information or check that removes it. A split diagram. On the left, a self-contained application file with all its context inside the box. On the right, a dbt model whose box is surrounded by four external boxes labeled warehouse schema, dependency graph, row-level data and the monthly bill, with dashed lines showing the agent can only see the model file itself *Application code carries its own context, while a data pipeline depends on four things outside the file.* ## Failure 1. The Agent Guesses Your Schema An agent asked to write a query has to name columns. If nobody told it the names, it guesses the name a typical schema would use. You get `user_id` where the column is `usr_key`, and `created_at` where it is `ingest_ts`. A missing-column error at run time is the good outcome, because the warehouse refuses the query. The expensive outcome is a guessed name that does exist, in two tables, with a different meaning in each. The query runs, the numbers look reasonable, and nothing flags a problem. Take `orders.status` and `payments.status`. `orders.status` records fulfillment and `payments.status` records authorization. Both columns hold the value `complete`. An agent that joins the two tables and filters on `p.status = 'complete'` gets a count of authorized payments. The person who asked wanted a count of shipped orders. ```sql -- orders.status: pending | shipped | complete (fulfillment) -- payments.status: pending | failed | complete (authorization) select count(*) from orders o join payments p on p.order_id = o.order_id where p.status = 'complete'; -- not the question anybody asked ``` Prompting does not fix this. A warehouse with a thousand tables does not fit in a system prompt. The schema also changes after you write the prompt. The fix is a lookup that runs at the moment the agent needs a column name. A Model Context Protocol (MCP) server is the component that does that lookup. Our [MCP server configuration reference](https://help.altimate.ai/code/configure/mcp-servers/) shows how to set one up. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## Failure 2. SQL That Compiles but Joins Wrong A join on the wrong key still parses and still runs. The query returns a row count that looks plausible. The error shows only when somebody reconciles that count against a source system. The classic version is a fan-out. A fan-out is a join on a key that is unique in one table and repeated in the other. Every row from the unique side is copied once per matching row on the repeated side. Join `orders` to `order_items` on `order_id`, where each order has four line items. Every order row now appears four times, so summing `orders.total` returns four times the true total. ```sql -- orders: 10,000 rows, one per order_id, total sums to $1,000,000 -- order_items: 40,000 rows, four per order_id select sum(o.total) from orders o join order_items i on i.order_id = o.order_id; -- returns $4,000,000 ``` Both tables are correct, the join key is correct, and the number is wrong by 300%. Catching a fan-out needs the key's cardinality, which means how many times each `order_id` value appears in each table. Nothing in the SQL says whether `order_id` is unique in either table, so an agent that reads only the SQL cannot answer the question. The check has to run against the data. ## Failure 3. Model Edits That Ignore the DAG A dbt project is a graph of models, and each model reads from the models upstream of it. An agent editing one model sees only that model's file. It does not see the models that read from the file it is changing. Rename `customer_id` to `customer_key` in `stg_customers` and the diff is four lines. Each of those four lines is correct on its own. Every downstream model that writes `select customer_id` now fails to compile. The dashboard at the end of that chain fails with them, on the next scheduled run. Column-level lineage prevents this failure. It maps which downstream columns read from each upstream column, so an agent that holds the map before the change lands can list every model the rename touches. That list is the blast radius. The agent then either edits those models in the same pull request, or it stops and asks you. The same lineage query run after the merge produces an incident report instead of a fix. Our write-up of [blast radius analysis](https://altimate.ai/blog/blast-radius-analysis-using-altimate-code) covers the mechanics. ## Failure 4. Generated Queries That Leak Personal Data Ask an agent for a debugging query and it will `SELECT *` from a customer table. That result set may hold email addresses, phone numbers and payment identifiers. It then lands in a log, a terminal buffer or a shared notebook. The agent is not behaving badly. It answered with the most direct query available. Nobody told it which columns carry personal data, so it had no reason to leave any column out. A column called `email_address` announces what it holds, and a column called `contact_2` announces nothing. The agent cannot classify `contact_2` from its name, so the classification has to come from you. The fix is a classifier that reads the query before it runs. Pattern matching for personal data runs offline and costs no tokens. Sending the same check to a model is worse on privacy, because the check itself ships your sensitive column names to a third party. You declare the rules in our [governance configuration](https://help.altimate.ai/code/configure/governance/). A dependency graph of a dbt project with one model highlighted in the middle. Twelve downstream models are shaded red and labeled breaks tomorrow, while the pull request diff shown beside it is only four lines long *A four-line diff can break twelve downstream models, so diff size does not measure risk.* ## Failure 5. No Idea What the Query Will Cost An agent has no price list. Asked a question about yesterday, it scans a year of history, because nothing told it that the year costs more than the day. A one-day scan returns the same answer. Only the year-long scan shows up on the bill. Snowflake bills warehouse compute by warehouse size, and an X-Large warehouse costs [16 credits an hour](https://docs.snowflake.com/en/user-guide/warehouses-overview). | Warehouse size | Credits per hour | One 10-minute scan | The same scan, 20 times | | --- | --- | --- | --- | | Small | 2 | 0.3 credits | 6.7 credits | | Large | 8 | 1.3 credits | 26.7 credits | | X-Large | 16 | 2.7 credits | 53.3 credits | An agent iterating toward an answer reruns that scan on every attempt, and the table above prices that loop at twenty attempts. Nothing in the loop reads what the last attempt cost, so the agent has no reason to make the twentieth scan any cheaper than the first. The bill arrives at month end. It is attributed to a warehouse rather than to the session that ran the loop, so nobody connects the cost to the agent. A cost estimate only changes what the agent does if the estimate arrives before the query runs. Estimating the cost means reading the table statistics and the warehouse configuration, because the query text alone does not say how many rows it will scan. Altimate publishes 85% accuracy on its cost analysis. Wire a pre-execution estimate into the agent loop and the twentieth scan never runs. ## Failure 6. Every Session Starts From Zero You explain on Monday that the `orders` table has duplicate rows before 2021 and everyone filters them out. On Tuesday the agent writes a query without the filter. Chat history is a transcript, and a transcript ends when you close the window. The next session starts with a model that knows the general shape of a dbt project and nothing about yours. The model cannot tell those two kinds of knowledge apart, so it writes the query with the same confidence either way. You can restate the rule at the start of every session. That works until one person forgets to. What helps is a written store that the agent reads on every run. That store holds the project's rules rather than its conversations. The duplicate-row rule and the project's definition of revenue each fit on one line. ```yaml # rules.yml, read on every run rules: - "orders holds duplicate rows before 2021-01-01. Filter on ingest_ts." - "Revenue means net revenue. Read fct_revenue_net, never orders.total." ``` A human reads that file in a pull request and corrects a wrong rule in one line, dated and attributed. The alternative is an embedded memory that nobody can open and read. A wrong rule in there produces confident errors, and nobody can trace an error back to the sentence that caused it. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## Failure 7. The Agent Recommends Anti-Patterns Cheerfully Ask for a query and you may get `SELECT *` in a production model. You may get a cross join the agent did not mean, or an `ORDER BY` inside a subquery the optimizer discards. None of these are unusual mistakes. The model learned SQL from public code, and public SQL records what people wrote under deadline rather than what they knew was correct. The model repeats the common habit with the same confidence it gives a correct answer. Altimate publishes 19 anti-pattern rules scoring 100% accuracy across 1,077 benchmark queries. That accuracy holds because each rule is compiled code that reads the parse tree of the query. A compiled rule either finds `SELECT *` or it does not, and it returns the same answer on the thousandth run as on the first. The [validator reference](https://help.altimate.ai/code/data-engineering/validators/) lists each rule. ## Failure 8. Tests That Never Assert Anything Ask for tests and you get tests. Read them and you find assertions that only check that rows exist. The test suite grows and the coverage number improves, while nothing is verified. A test that fails gets fixed. A test that can never fail gets trusted, which is the more dangerous outcome. The coverage number only counts test entries in a YAML file. A generated test earns its place by failing when the thing it describes breaks. A `not_null` test on a column the warehouse already declares `NOT NULL` can never fail, so it adds a row to the coverage count and tells you nothing. ```yaml # fct_orders.order_id is declared NOT NULL in the warehouse models: - name: fct_orders columns: - name: order_id tests: - not_null # green forever, asserts nothing - unique # can fail, so it is worth running ``` Point every generated test at a corrupted copy of the table. Delete the ones that stay green. ## Failure 9. Correctness Judged by Another Model The common pattern for validating agent output is to ask a second model whether the first one was right. For prose that pattern is defensible, because prose has no ground truth to compare against. For a query rewrite the same pattern throws away the ground truth you already have. Whether a rewritten query returns the same rows has an exact answer. You get it by running both queries against the same snapshot and comparing row by row. A model asked the same question reads the two queries and produces an opinion. The opinion is often right and never checkable, because the model never ran either query. We made this point on a DataCamp panel about [what AI agents are really changing](https://altimate.ai/blog/what-ai-agents-are-really-changing-in-data-engineering). A data diff runs both queries and lists every row that differs. Use a data diff rather than a second opinion on anything where rows can settle the argument. ## Failure 10. Nothing Between the Agent and Production Give an agent live credentials with no policy in between and one bad session can end the whole rollout. Every session is one more chance to run a statement nobody reviews. One of those sessions eventually drops something you cannot get back. Atlan's write-up on agent failures in production names a concrete case. In July 2025 a Replit AI coding agent deleted a production database. The agent held standing privileges to that database. The only instruction not to touch production was verbal, which is a social control rather than a technical one. The credentials allowed the delete, so the delete ran. Three controls stop an agent session from doing unrecoverable damage: - Permissions cover only the task in front of the agent. - A confidence threshold makes the agent hand a low-confidence action to a human instead of executing it. - A log records every action, with the policy that allowed it. None of the three controls is a better model, and none of them asks the agent to behave. A permission the agent does not hold cannot be misused. Our [permissions model](https://help.altimate.ai/code/configure/permissions/) covers how you declare one. ## Our Runs Put the Failure Rate Near One Task in Four A good agent on a good model still fails often enough to matter. Two models ran the same 270 DataAgentBench trials through one harness, recorded on [our benchmarks page](https://altimate.ai/benchmarks/dab-bench). One harness means the same agent code, the same prompts and the same tools. Only the model changed between the two columns. | DataAgentBench, 270 trials each | Claude Sonnet 4.6 | DeepSeek v4 pro | | --- | --- | --- | | Stratified Pass@1 (the share of tasks solved on the first attempt, balanced across task types) | 60.4% | 56.9% | | Cost per trial | $0.76 | $0.29 | | Trials that produced nothing | 32 | 79 | | Empty-run rate | 12% | 29% | On the first row the two models look interchangeable, three and a half points apart. On the last row the cheaper one wrote nothing at all more than twice as often. A Pass@1 score records only whether the task was solved. A trial that produced nothing and a trial that produced a wrong answer count the same. ADE-Bench measures data engineering task completion. Our best run against Snowflake is [74.4%, at 32 of 43 tasks](https://altimate.ai/benchmarks/ade-bench). One task in four still does not complete. Snowflake's Cortex Code CLI reports [65% on the same 43 tasks](https://www.snowflake.com/en/blog/cortex-code-cli-expands-support/). We ran the identical 43 tasks repeatedly on the same harness and the same model, and the runs spanned 32 tasks down to 29. A seven-point swing on a fixed task set means one passing run proves very little. Assume the failure rate is high and build for it. The compiled checks that catch these failures run at 0.48 ms per query, so skipping them saves nothing. A team that assumes a 95% success rate reviews as though the agent is usually right. That habit is what lets a fan-out join through, because nobody reconciles a number they expect to be right. Two bar groups. The first shows Pass@1 scores of 60.4% and 56.9% for two models, looking similar. The second shows empty-output runs of 32 of 270 and 79 of 270 for the same two models, a much wider gap, annotated the number that does not appear in a benchmark score *Two models that look close on Pass@1 and are not close at all on how often they produce nothing.* ## What Makes AI Coding Agents Fail at Data Engineering | Failure | What the agent lacked | What fixes it | | --- | --- | --- | | Guessed schema | live catalog at call time | MCP server, not a longer prompt | | Wrong join | key cardinality | deterministic check on the data | | DAG break | downstream graph | column-level lineage before merge | | Data leak | column sensitivity | offline pattern classifier | | Cost blindness | table stats and warehouse config | pre-execution estimate | | No memory | the project's written rules | reviewable memory file | | Anti-patterns | a rule set | compiled rules, not inference | | Empty tests | a failing case | run the test against wrong data | | Model-judged correctness | the actual rows | data diff | | No guardrail | a policy | scoped permissions plus audit log | Atlan's write-up of why agents fail in production reaches the same conclusion and names fragmented context rather than weak models. None of the ten fixes is a better model. Nine of the ten are information the agent lacked, or a check with an exact answer. The tenth, the guardrail, is a policy that limits what the agent can do at all. You build the information and the checks once, and they hold for every model you swap in afterwards. [Deterministic tooling alongside the model](https://altimate.ai/blog/deterministic-tooling-vs-llm-only-data-agents) works through why the tooling under the agent decides more than the model you pick. [Nine signs your data platform needs an agent-first overhaul](https://altimate.ai/blog/signs-data-platform-needs-agent-first-overhaul) is the same diagnosis one layer up, at the platform level. ## Frequently Asked Questions Will a smarter model fix these failures? Most of them do not improve with the model. Nine of the ten fixes in the table above are information the agent lacked or a check with an exact answer, and neither changes when the model gets better. A better model guesses a schema more plausibly, and a plausible wrong guess is harder to spot than an obvious one. Do these SQL automation failures apply to AI assistants in the editor as well as to agents? They apply to both. An assistant that completes a query in the editor and an agent that runs one both write against a schema they cannot read. The assistant fails less often only because a human reads its output before anything executes. Failure 10, the missing guardrail, is the one failure that only an agent has, because an editor assistant holds no credentials of its own. How many teams actually run agents in production? Almost none do. Cleanlab's 2025 survey of 1,837 engineering and AI leaders found 95 with agents live in production. That is about 5%. The constraint is reliability rather than model capability. Do these failures apply to warehouse-native agents too? Only one goes away outright. An agent running inside Snowflake or BigQuery starts with the warehouse schema already loaded, which removes Failure 1. [Google's BigQuery agent](https://cloud.google.com/blog/products/data-analytics/exploring-the-data-engineering-agent-in-bigquery) is a good example. The other nine do not depend on where the agent runs. The DAG, cost, memory and governance failures all stay, because none of that context lives in the warehouse's schema either. Which of these ten should I fix first? Fix the one that has already cost you something. If nothing has broken yet, fix the missing guardrail between an agent and production. It is the only one here where a single incident can be unrecoverable rather than merely expensive. On this page 1. Failure 1. The Agent Guesses Your Schema 2. Failure 2. SQL That Compiles but Joins Wrong 3. Failure 3. Model Edits That Ignore the DAG 4. Failure 4. Generated Queries That Leak Personal Data 5. Failure 5. No Idea What the Query Will Cost 6. Failure 6. Every Session Starts From Zero 7. Failure 7. The Agent Recommends Anti-Patterns Cheerfully 8. Failure 8. Tests That Never Assert Anything 9. Failure 9. Correctness Judged by Another Model 10. Failure 10. Nothing Between the Agent and Production 11. Our Runs Put the Failure Rate Near One Task in Four 12. What Makes AI Coding Agents Fail at Data Engineering From the product - [PowerUser Plugin for dbt Profile a slow dbt query and read its execution plan in plain English 4 min read](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan) - [Enterprise Platform Root cause analysis for a dbt, Tableau, or Snowflake workload spike 6 min read](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis) - [Enterprise Platform Track Snowflake query cost over time, run by run 6 min read](https://altimate.ai/product-highlights/snowflake-query-cost-over-time) - [Enterprise Platform Reduce Snowflake idle warehouse spend with Altimate Lite 6 min read](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) - [Enterprise Platform Snowflake Cortex AI cost breakdown by service, user, and model 6 min read](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # 9 Agentic Data Engineering Tools for VS Code URL: https://altimate.ai/blog/best-agentic-data-engineering-tools-vs-code Nine agentic data engineering tools for VS Code, scored on warehouse context, dbt awareness, cost visibility and license, with where each one stops. ## 9 Best Agentic Data Engineering Tools for VS Code Nine agentic data engineering tools for VS Code, scored on warehouse context, dbt awareness, cost visibility and license, with where each one stops. Atharva Shah 2026-06-11 2026-09-04 11 min read On this page 13 sections 1. How to Score Agentic Data Engineering Tools for VS Code 2. 1\. Altimate Code 3. 2\. GitHub Copilot 4. 3\. Cline 5. 4\. Continue.Dev 6. 5\. Snowflake Cortex Code CLI 7. 6\. dbt MCP Server 8. 7\. dbt Labs VS Code Extension 9. 8\. SQLTools and Database Clients 10. 9\. Notebook Extensions 11. How the Nine Tools Compare 12. How to Run a Two-Week Trial 13. How Most Teams Combine These Tools tl;dr Nine agentic data engineering tools for VS Code follow, scored on four criteria: warehouse schema, the dbt DAG, query cost and license. Most general coding assistants score zero on the first three, because they read your repository well and never open your warehouse. Nothing that writes code covers all four, so most working setups pair a general assistant with a data layer. The published ADE-Bench scores help you narrow the field, not pick from it. Altimate Code posts 74.4% in our own unaudited run and Snowflake reports 65% for Cortex Code CLI, but the two come from separate experiments, and most published figures do not compare directly. So treat a score as a starting signal and settle it with a short trial on your own work, where the number that matters is how often you had to correct a column the tool guessed wrong, not the minutes it saved. You open a `.sql` file in VS Code and the assistant autocompletes a column name. The name came from your other models, and nothing checked it against the warehouse. Most AI coding extensions were built for application developers. They read your programming language well and know nothing about the tables your queries run against. The gap between the language and the tables decides whether a tool saves you time or hands you a query that returns the wrong number. Four criteria rank these nine agentic data engineering tools for VS Code: live warehouse schema, dbt dependency-graph awareness, query cost visibility, and license. Each entry says: - what the tool is, - who it fits, - what it costs, and - where it falls short. A capability matrix with five tools and four columns labeled warehouse context, dbt DAG, cost visibility and license. GitHub Copilot shows a cross in all three capability columns *Four criteria separate a data tool from a coding assistant that also opens.sql files.* ## How to Score Agentic Data Engineering Tools for VS Code Four criteria rank the tools, listed from the most expensive miss to the cheapest. - **Warehouse context.** Does the tool read live schema when it writes SQL, or guess from the open file? - **dbt DAG awareness.** The DAG is the dependency graph between your dbt models. Does the tool know what sits downstream of the model you are editing? - **Cost visibility.** Can the tool price a query before you run it? - **License.** Your security team accepts or rejects the tool on its license: open source, free tier or paid seat. Warehouse context is the criterion where a mistake costs money. A misspelled column name fails at compile time, so you find it immediately. A column name that exists and is the wrong one runs, and the query returns a wrong number with no error. Note **Worked example.** Your source table carries both `customer_id` and `cust_id`, and only `cust_id` joins to the orders feed. - A tool completing from the open file writes `customer_id`, because every other model uses it. - The query compiles and runs, because `customer_id` is a real column. - The join returns a third of the rows, so the report undercounts and nothing flags it. A tool can score zero on three of the four criteria and still be a good tool. The criteria measure what context a tool can see. They say nothing about how well it writes code. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## 1\. Altimate Code - **What it is.** A CLI and VS Code extension built to check SQL rather than to generate it. It lints and reviews what you or an agent wrote. - **Best for.** Teams who spend more time proving a change is safe than writing it, on any warehouse. - **Key features.** The SQL lint at 0.48 ms per query is fast enough to run on every keystroke rather than on save. Its 19 anti-pattern rules hold 100% accuracy across 1,077 benchmark queries, with zero false positives. It ships 21 skills. A skill is a workflow playbook the agent loads when the task calls for it, and the 21 cover SQL review, lineage diff, PII audits and dialect translation. - **In the editor.** One way to reach these capabilities in VS Code is [Power User for dbt](https://altimate.ai/blog/how-i-cut-dbt-development-time-by-20-with-the-power-user-vscode-extension), Altimate's extension for dbt projects. It has 1M+ installs. It reads the DAG from the manifest that dbt compiles, and it adds compiled SQL preview, click-to-run for one model, and [column-level lineage in the editor](https://altimate.ai/blog/column-level-lineage-use-cases). The lineage panel draws from the last compile, so recompile after an edit before you trust it. - **Pricing.** [Open source](https://github.com/AltimateAI/altimate-code) under MIT. A free Community tier grants 10 million tokens once. Pro is $29 per seat per month with 20 million tokens included, then $5 per million after that. Power User for dbt is free. - **Limits.** It reviews and lints SQL. It does not write your next model from a Jira ticket. Most of its skills assume a dbt project, so a repository of plain `.sql` files gets the lint and not the dbt-specific skills. A VS Code layout in two panes. The left holds fct\_orders.sql. The right holds a column lineage graph for order\_total, with rpt\_revenue, customer\_ltv and finance\_recon marked downstream *One column at the cursor resolves to three named downstream models, in the Power User for dbt lineage panel.* ADE-Bench is an agentic data-engineering benchmark that scores an agent pass or fail on a fixed set of tasks. [Altimate Code scores 74.4% on ADE-Bench](https://altimate.ai/blog/introducing-altimate-code) with 32 of 43 tasks. We ran that benchmark ourselves, on Sonnet 4.6 against Snowflake, and nobody outside Altimate has checked the run. Read the method on [our benchmarks page](https://altimate.ai/benchmarks/ade-bench). ## 2\. GitHub Copilot - **What it is.** GitHub Copilot has the widest reach of the nine tools on this list. It also knows the least about your data. - **Best for.** Boilerplate SQL and Jinja macros. On a long `case when` ladder the next branch is predictable, so inline completion saves minutes. - **Key features.** It completes inline from the open file and the repository, across every language in your project. - **Pricing.** A paid seat per user. Check GitHub's pricing page for the current rate. - **Limits.** It completes from your files, so a column name it offers is one your other files use, and nothing checks that name against the warehouse. Pair it with a tool that reads live schema. ## 3\. Cline - **What it is.** An open-source agent extension that runs multi-step tasks inside VS Code and asks permission before each action. - **Best for.** Agent work that touches a warehouse. Every command pauses for your approval, which slows the agent down and stops it from running a command you never reviewed. - **Key features.** It adds a human-in-the-loop approval step to every file write and command. Beyond that, what Cline can do depends on the Model Context Protocol (MCP) servers you connect. MCP is the open protocol an agent uses to call outside tools, and an MCP server is one such tool. A warehouse MCP server gives Cline live schema, and the dbt MCP server gives it the dependency graph. - **Pricing.** Open source and free. You pay only for the model API it calls. - **Limits.** Cline ships with no warehouse knowledge of its own. With no MCP server connected it is a general coding agent with a terminal, and its SQL is guessed from your files like Copilot's. ## 4\. Continue.Dev - **What it is.** An open-source assistant extension whose main appeal is model choice, including a model you host yourself. - **Best for.** Teams that cannot send code to a third-party API, such as healthcare and defense. - **Key features.** Point it at any model, local or hosted. - **Pricing.** Open source and free. - **Limits.** It knows nothing about your warehouse. Pair it with a self-hosted MCP server that connects on a read-only warehouse role. The agent then reads live schema, and no token leaves your network, because the model and the server both run inside it. ## 5\. Snowflake Cortex Code CLI - **What it is.** Snowflake's own agent. It is the only tool on this list that runs inside the warehouse, so it starts with your schema, your query history and your role grants already loaded. - **Best for.** Teams whose data all sits in one Snowflake account and who want no context setup. - **Key features.** It has native access to Snowflake schema, query history and grants, with nothing to connect or keep in sync. - **Pricing.** It bills against Snowflake credits, so it has no seat price. - **Limits.** Its context ends at the Snowflake account boundary. If a Databricks job feeds a Snowflake mart, the agent sees the mart and none of the job that built it. It has no way to know the upstream half exists, so it cannot warn you that its answer is incomplete. Its ADE-Bench result is 65%, at 28 of 43 tasks, sourced to [Snowflake's own blog](https://www.snowflake.com/en/blog/cortex-code-cli-expands-support/). Snowflake ran Claude Code on the same 43 tasks, and it finished 25. Both agents ran on Claude Opus 4.6, so the model does not explain the three-task gap. The one difference left is the warehouse context Cortex Code starts with. ## 6\. dbt MCP Server - **What it is.** An MCP server from dbt Labs that gives any MCP-capable agent your dbt project graph. - **Best for.** Any MCP-capable extension that needs dbt DAG awareness without a purchase. - **Key features.** Its tools expose the model graph, compiled SQL and test results. Run it yourself with `uvx dbt-mcp`, or reach the remote server over HTTP on the dbt platform. - **Pricing.** It is free to self-host under Apache 2.0. - **Limits.** Your dbt account has a quota of dbt Copilot actions. When the account runs out, the remote server blocks every tool that runs through it. A self-hosted server that forwards some calls to the remote one loses those calls too. dbt Labs ran ADE-Bench with third-party models under two dbt setups. dbt Core alone passed 50% of the tests in those runs, and dbt Core plus the MCP server passed 54%. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## 7\. dbt Labs VS Code Extension - **What it is.** dbt Labs now ships its own VS Code extension, and it is the only officially supported one. It shipped on 27 May 2025. It runs on the dbt Fusion engine, which is dbt Labs' rewrite of the dbt engine in Rust. - **Best for.** dbt teams that want the editor to understand their SQL. - **Key features.** dbt Core treats a model as a Jinja template and renders it into a string of SQL. Fusion parses the SQL itself, so it can check a column reference against the model upstream. Reference a column that does not exist upstream, and the extension underlines it while you type. dbt Labs reports [up to 30x faster parsing and 2x quicker full-project compilation](https://docs.getdbt.com/blog/dbt-fusion-engine), so a parse that took a minute finishes in two seconds. - **Pricing.** Free, but you have to run Fusion, which is a separate free binary with proprietary parts. dbt Core v2.0, the Apache 2.0 release of the Rust engine, reached [first alpha on 1 June 2026](https://docs.getdbt.com/blog/dbt-core-v2-is-here). - **Limits.** It is not an agent, so it writes no code. dbt Labs does not support the extension on dbt Core, so a team on dbt Core has to move to Fusion first. Two code panels. The left, labeled raw Jinja, holds a dbt model with a misspelled ref call to stg\_ordrs under a red squiggle. The right, labeled compiled SQL, resolves it to analytics.stg\_orders *Power User for dbt and the dbt Labs extension both show compiled SQL, which is never quite what you wrote.* ## 8\. SQLTools and Database Clients - **What it is.** These are database client extensions rather than agents. They connect VS Code straight to the warehouse, so the live schema sits in a panel beside your file. - **Best for.** Teams that keep a general coding assistant and want to close its schema gap by hand. - **Key features.** You read the column name off the schema tree and paste it into the prompt, so the assistant no longer has to guess it. They run on your machine, so they clear security review in locked-down environments. - **Pricing.** Free. - **Limits.** They show you the live schema and nothing about the dbt DAG or query cost. ## 9\. Notebook Extensions - **What it is.** Jupyter and SQL notebook extensions that cover the exploratory half of data work. - **Best for.** Ad-hoc investigation. Most data work starts as a question, and the question gets asked in a notebook long before any line reaches a dbt file. - **Key features.** They run interactive SQL and Python against the warehouse. A wrong assumption usually starts in an exploratory query, before any model is committed. An assistant that reads only your committed models never sees that query, and an assistant with the notebook open does. - **Pricing.** Free. - **Limits.** They show you nothing about the dbt DAG or query cost. A notebook stops at the exploratory query, and every other tool on this list starts there. ## How the Nine Tools Compare | Tool | Warehouse | dbt DAG | Cost | License | | --- | --- | --- | --- | --- | | Altimate Code | ✓ | ✓ | ✓ | open source, MIT | | GitHub Copilot | ✗ | ✗ | ✗ | paid seat | | Cline | ≈ via MCP | ✗ | ✗ | open source | | Continue.dev | ≈ via MCP | ✗ | ✗ | open source | | Cortex Code CLI | ≈ Snowflake only | ≈ partial | ≈ Snowflake only | Snowflake account | | dbt MCP server | ✗ | ✓ | ✗ | Apache 2.0 | | dbt Labs extension | ≈ via Fusion | ✓ | ≈ partial | free, Fusion required | | SQLTools and clients | ≈ manual | ✗ | ✗ | free | | Notebook extensions | ≈ manual | ✗ | ✗ | free | Only Altimate Code scores on all four criteria, and it writes no code, so a complete setup still pairs a writing assistant with a data layer. The published ADE-Bench figures come from three separate experiments rather than one leaderboard. A row compares cleanly only with another row from the same experiment. | Agent or configuration | Model | Pass rate | Source | | --- | --- | --- | --- | | Altimate Code on Snowflake | Sonnet 4.6 | 74.4%, 32 of 43 | us, unaudited | | Snowflake Cortex Code CLI | Opus 4.6 | 65%, 28 of 43 | Snowflake | | Claude Code on Snowflake | Opus 4.6 | 58%, 25 of 43 | Snowflake | | Codex | GPT-5.1 | 56% | dbt Labs | | dbt Core plus the dbt MCP server | not stated | 54% | dbt Labs | | dbt Core alone | not stated | 50% | dbt Labs | | Claude Haiku 3 | Haiku 3 | 10% | dbt Labs | Two pairs in the table isolate one change each. Snowflake's two rows share a model and a platform, so the three-task gap between them comes from the agent. The two dbt Core rows come from one dbt Labs experiment and differ only by the MCP server, so the four points between them are what the server added. Our row carries no independent check. The Codex and Claude Haiku 3 rows change the model and the configuration at once, so neither compares with any other row. ## How to Run a Two-Week Trial Run a two-week trial on your own work before you pick from this list. Pick three tasks you do monthly. - Build a new staging model on a source nobody has modeled yet. - Change a model that half the project depends on. - Investigate a query somebody says is slow. Run each twice, with the tool and without. Log the wall-clock time, then count how often you corrected something the tool claimed. The correction count matters more than the time saved. A tool that saves ten minutes and invents one wrong column name costs you more than it saved, because somebody has to find that column name in review. Then ask the tool three questions about a project it has never seen. - Name the schema of a table nobody has touched in a year. - List what depends on a model buried in the middle of the DAG. - Price this query before it runs. A tool that reads live schema answers all three questions. A tool completing from your repository invents all three answers. A tool that fails visibly costs you a retry. A tool that returns a wrong answer with no error costs a reviewer the time to find it. ## How Most Teams Combine These Tools Two arrangements cover the working setups. The first is a general assistant with a data layer underneath, where the data layer is whatever supplies the warehouse schema and the dbt DAG. Keep Copilot or Cline for writing speed and add Altimate Code, which supplies the live schema, the DAG and the checks. You replace nothing, and the two pieces together cover all four criteria. The second arrangement is a warehouse-native agent, and it works only for a team on one platform. The agent's context stops at the platform boundary, and in exchange that context costs nothing to set up. On an all-Snowflake team Cortex Code is a reasonable default, until somebody stands up a second warehouse. [The MCP servers worth knowing about](https://altimate.ai/blog/best-mcp-servers-for-data-engineers) covers the context layer. Ask the tool you already run for the schema of a table nobody touched this year. The answer tells you whether it reads live schema or guesses from your open files. ## Frequently Asked Questions What makes agentic data engineering tools for VS Code different from general coding tools? The difference is warehouse context and dependency-graph awareness. A general assistant completes from the files in your repository. The column names it offers are the ones your other files use, and nothing checks them against the warehouse. A data tool reads the live schema, so its column names are checked. It also reads the dbt DAG, so it knows which models break downstream when you change one. Can I use GitHub Copilot for dbt work? It works well as a typing accelerator for boilerplate SQL and Jinja macros. It does not know your schema, so treat every column name it offers as a guess. Which AI tools suit analytics engineers who do not use dbt? Most of this list assumes dbt. SQLTools plus a general assistant gives you live schema with no dbt project. Altimate Code's SQL lint runs on plain `.sql` files, so the anti-pattern checks work without dbt too. Which of these tools is free? The dbt Labs extension is free, and so is Power User for dbt. Altimate Code is [open source](https://github.com/AltimateAI/altimate-code) under MIT, the dbt MCP server carries Apache 2.0, and Cline and Continue.dev are open source too. How much does ADE-Bench matter when choosing? It is a useful signal and a narrow one. A higher score means the agent finishes more of a fixed task set. It says nothing about your stack, your license rules, or your editor. On this page 1. How to Score Agentic Data Engineering Tools for VS Code 2. 1\. Altimate Code 3. 2\. GitHub Copilot 4. 3\. Cline 5. 4\. Continue.Dev 6. 5\. Snowflake Cortex Code CLI 7. 6\. dbt MCP Server 8. 7\. dbt Labs VS Code Extension 9. 8\. SQLTools and Database Clients 10. 9\. Notebook Extensions 11. How the Nine Tools Compare 12. How to Run a Two-Week Trial 13. How Most Teams Combine These Tools From the product - [PowerUser Plugin for dbt Profile a slow dbt query and read its execution plan in plain English 4 min read](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan) - [Enterprise Platform Snowflake cost analysis in plain English, with AI that cites every number 5 min read](https://altimate.ai/product-highlights/snowflake-cost-analysis-ai-cited-answers) - [Enterprise Platform Root cause analysis for a dbt, Tableau, or Snowflake workload spike 6 min read](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis) - [Enterprise Platform Reduce Snowflake idle warehouse spend with Altimate Lite 6 min read](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) - [Enterprise Platform Snowflake Cortex AI cost breakdown by service, user, and model 6 min read](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # 12 Snowflake Cost Optimization Techniques for 2026 URL: https://altimate.ai/blog/snowflake-cost-optimization-techniques Twelve Snowflake cost optimization techniques ranked by what they save against what they cost you to run, from free config changes to continuous tuning. ## 12 Snowflake Cost Optimization Techniques That Actually Work Twelve Snowflake cost optimization techniques ranked by what they save against what they cost you to run, from free config changes to continuous tuning. Atharva Shah 2026-05-21 2026-09-04 12 min read On this page 14 sections 1. 1\. Cut Auto-Suspend to Sixty Seconds 2. 2\. Right-Size Down, Then Measure 3. 3\. Kill the Warehouses Nobody Owns 4. 4\. Set Resource Monitors Before You Need Them 5. 5\. Find the Queries Doing the Most Damage 6. 6\. Stop Scanning What You Do Not Need 7. 7\. Cluster the Tables That Earn It 8. 8\. Move Small Frequent Work Off Big Warehouses 9. 9\. Watch the Serverless Features 10. 10\. Itemize AI and Cortex Spend 11. 11\. Attribute Cost to Teams, Not Warehouses 12. 12\. Tune Continuously Instead of Quarterly 13. What Gen2 Warehouses and Adaptive Compute Now Handle 14. Where to Start With Snowflake Cost Optimization tl;dr Snowflake cost optimization has three targets: stop paying for compute nobody uses, stop running work nobody asked for, and make the bill explainable. The twelve techniques here each serve one of those three, ordered by payoff against effort. Start with the four free config changes: cut auto-suspend, right-size down, kill the warehouses nobody owns, and set resource monitors. They cost nothing, take an afternoon, and usually surface the biggest single block of savings. Then rank your queries by total credits rather than duration before you touch anything else, because that ranking tells you whether query-level work can move the bill at all or whether the answer is warehouse configuration. Change one thing at a time, and normalize on credits per query so a quiet month does not read as a saving. Gen2 warehouses run faster but cost more per hour, so test the break-even before you switch. What stays yours, query shape, scan volume, clustering, serverless sprawl and cost attribution, needs business context a platform cannot supply. Continuous tuning is the one lever a human cannot work by hand; in our private preview it saved 19% on average and never less than 12.4% on any warehouse on any day, with no change to query performance or queue time. The Snowflake bill goes up, and the invoice does not say why. The extra credits could come from an idle warehouse, from a query nobody needed, or from a serverless feature that no per-warehouse view shows. Snowflake cost optimization has three targets, one for each of those causes. You stop paying for compute nobody uses, you stop running work nobody asked for, and you make the bill explainable. Each of the twelve techniques below serves one of those three targets. They are ordered by effort against payoff. The first four are free configuration changes you can make this afternoon. The last eight need tooling or a real project. A bar chart of twelve techniques ordered left to right by effort, with bar height showing typical saving. The four leftmost bars are labeled free config change and are the tallest on the chart ## 1\. Cut Auto-Suspend to Sixty Seconds A warehouse bills for every second it runs, whether or not a query is running. Auto-suspend is the idle time a warehouse waits before it shuts itself off. The default is 600 seconds, so after every query the warehouse sits for ten minutes and bills every one of them. Most workloads need far less than ten minutes. On a warehouse that serves ad-hoc analyst queries, the idle time between queries can exceed the time spent running them. On that warehouse, more than half the line item bought you nothing. One `ALTER WAREHOUSE` statement drops auto-suspend to sixty seconds. Pick sixty rather than forty-five. [Snowflake's create-warehouse reference](https://docs.snowflake.com/en/sql-reference/sql/create-warehouse) puts the background suspend process at about every 30 seconds, so a 45-second setting suspends at 60 anyway. The stronger argument against a short auto-suspend is Snowflake's minimum billing. Snowflake bills a full minute on every start or resume, and [that minimum restarts each time](https://docs.snowflake.com/en/user-guide/cost-understanding-compute). On a bursty workload, an aggressive auto-suspend can therefore cost more than a longer one, because the warehouse resumes more often and pays the minimum each time. Do the math on your own workload. A warehouse that wakes 200 times a day for twenty-second queries does 67 minutes of work and bills 200 minutes, because each wake pays the one-minute minimum. The 133-minute gap costs about 36 credits a day on an X-Large at 16 credits an hour. The weaker argument is cache warmth. Suspending clears the local SSD cache, so the next query reads from remote storage. The cache argument holds for a BI workload that queries the same tables all day. It does not hold for a nightly batch that reads different partitions every run. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## 2\. Right-Size Down, Then Measure Each step up in warehouse size doubles the cost per hour. Snowflake's [published rates](https://docs.snowflake.com/en/user-guide/warehouses-overview) run 1 credit an hour at X-Small, 8 at Large and 16 at X-Large. A 2X-Large costs four times a Large. The test is cheap. Drop the warehouse one size, rerun the same workload, and compare wall-clock time against the previous run. A Large at 8 credits an hour costs less than an X-Large at 16 unless the workload takes more than twice as long on the Large. Doubling the size only halves runtime for work that parallelizes across the extra nodes. A query bound by one slow single-threaded step gets no faster. The same holds for a large sort that spills to disk, or a UDF running row by row. Each one costs exactly twice as much on the bigger warehouse. Teams size up during an incident and nobody sizes back down, because the person who did it has moved on. Check your five most expensive warehouses for one that never came back down. ## 3\. Kill the Warehouses Nobody Owns Warehouses accumulate from a migration that finished, a proof of concept nobody adopted, or an analyst who left. Query `SNOWFLAKE.ACCOUNT_USAGE` for warehouses that served no queries in the last thirty days. A suspended warehouse costs nothing, so most of what you find is clutter rather than spend. A few still burn credits on a schedule nobody remembers setting, usually because a scheduled task still points at them. Ownership is the harder half. You cannot drop a warehouse with no named owner, because nobody can confirm what breaks, and so it survives every cleanup. ## 4\. Set Resource Monitors Before You Need Them A resource monitor caps credit spend on a warehouse and suspends it at a threshold. It is free and takes minutes to set up. A runaway query, a loop in a script, or an agent with no cost awareness can spend a month of budget in a day. A resource monitor turns that into a stopped warehouse and an alert. Set the threshold above your normal peak and treat a trigger as a signal that something changed. A monitor that fires routinely teaches people to ignore it. A resource monitor covers warehouses only, so the serverless features in technique 9 need a different control. ## 5\. Find the Queries Doing the Most Damage The query somebody complained about is the slow one. The expensive one finishes in four seconds and nobody notices it, because a short query draws no complaint. Rank queries by total credits consumed, never by duration. Total cost is duration multiplied by how many times the query runs. A three-second query that runs fifty thousand times a day consumes far more compute than a twenty-minute report that runs once. A list sorted by duration puts the report at the top and the three-second query near the bottom, so the expensive pattern never appears. Ranking queries by credits tells you whether the other eleven techniques are worth your time. If your top ten queries are 4% of spend, no query-level work moves the bill and the answer is warehouse configuration. If they are 60%, query-level work is where the saving is. Altimate [tracks query cost over time, run by run](https://altimate.ai/product-highlights/snowflake-query-cost-over-time). ## 6\. Stop Scanning What You Do Not Need You pay for the data a query reads, and [partition pruning](https://altimate.ai/blog/optimizing-snowflake-queries-expert-strategies-for-data-engineers-and-analysts) is what keeps a query from reading everything. Two query patterns defeat pruning and account for most avoidable scanning. The first is `SELECT *` in a model that needs four columns. Every unread column is compute you paid for and threw away, and the cost scales with table width rather than with your logic. The second is a filter on a function of a column. `WHERE date_trunc('month', order_ts) = '2026-08-01'` forces a full scan, and `WHERE order_ts >= '2026-08-01' AND order_ts < '2026-09-01'` prunes to one month while returning the same rows. Both are mechanical to detect in the parse tree, and both are what a generated query writes by default. A lint before execution catches both, cheaper than an audit on next month's invoice. ## 7\. Cluster the Tables That Earn It A clustering key improves pruning on a large table and bills you continuously. Automatic reclustering is serverless compute that runs whenever the table changes. A clustering key is the only change on this list that can cost more than it saves. Clustering pays only when the table is large enough that a full scan hurts and the queries filter on the same predicate nearly every time. A table filtered on twelve columns by twelve consumers has no single good key, so picking one optimizes for whichever consumer complained most recently. Measure partitions scanned before and after rather than trusting the theory. A key on the wrong column produces a permanent reclustering bill and no reduction in scan volume. ## 8\. Move Small Frequent Work Off Big Warehouses A dashboard firing hundreds of small queries at an X-Large warehouse pays X-Large rates for work an X-Small would serve at the same latency. It sits there because somebody pointed it there during a launch, and moving it requires knowing who owns it. The fix is workload separation rather than a smaller warehouse, because the heavy batch on that warehouse really does need the size. Put the small frequent queries on their own warehouse and leave the batch where it is. An interactive warehouse costs less than a standard warehouse at the same size. The [Service Consumption Table](https://www.snowflake.com/legal-files/CreditConsumptionTable.pdf) lists both rates. | Warehouse size | Interactive credits per hour | Standard credits per hour | | --- | --- | --- | | X-Small | 0.6 | 1 | | 4X-Large | 76.8 | 128 | That is 40% off at the same size, before you rewrite a single query. Check availability in your region and edition first. Snowflake does not offer interactive warehouses everywhere. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## 9\. Watch the Serverless Features Four Snowflake features bill you without a warehouse attached: - **Snowpipe** loads files as they land in your stage. - **Materialized view maintenance** keeps a view current as its source changes. - **Automatic clustering** reclusters a table in the background. - **Search optimization** builds and maintains a search access path. These four features bill as serverless compute, so neither the per-warehouse view nor a resource monitor catches them. [Budgets](https://docs.snowflake.com/en/user-guide/budgets) do reach them. Snowflake's supported-services list for Budgets names `AUTO_CLUSTERING`, `MATERIALIZED_VIEW`, `PIPE` and `SEARCH_OPTIMIZATION`, which covers all four. Look for a feature switched on during an evaluation and never switched off. Search optimization stays enabled on a table nobody queries and rebuilds its access path on every load. Pull serverless credit consumption as its own report and check each entry against a consumer that still exists. Check that queries still filter a clustered table on its clustering key, and that somebody still reads a materialized view rather than the base table. ## 10\. Itemize AI and Cortex Spend AI features are the newest line on the bill and the least understood. Spend concentrates in a few users and a few models, and the bill total shows neither. Three questions locate the spend: - Which service spends the credits? - Which model runs, and which user calls it? - Is the expensive model doing work a cheap one would do? A summarization task running on a frontier model is easy to miss, because it works and it returns on time. You see it only in a credit line that grew last month. Know the unit before you build a spend model. Warehouse compute burns Platform Credits, and [Snowflake quotes most Cortex rates in AI Credits](https://altimate.ai/blog/snowflake-cortex-ai-cost-catalog-billing-quotas). [A single per-user quota](https://docs.snowflake.com/en/user-guide/budgets/per-user-quotas) monitors one or the other, never a mix. Capping one person's whole spend therefore takes two quota objects. Altimate breaks this out as [Cortex AI cost by service, user, and model](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown). Any tool that itemizes it will do. ## 11\. Attribute Cost to Teams, Not Warehouses A bill organized by warehouse has no clear owner, so it lands in a shared channel and nobody reads it. A bill organized by team reaches the person who can change the query. Costs fall when that person sees the number. Attribution is a mapping problem more than a tooling one. A query tag set by dbt names the model, the role names the service account, and only the combination names the team. Altimate [attributes cost to the team that owns it](https://altimate.ai/product-highlights/snowflake-cost-alerts-by-team) with ownership rules, and the [cost optimization guide](https://help.altimate.ai/code/data-engineering/guides/cost-optimization/) covers the mapping. Attribution has to match your org chart, or the number lands on no one. ## 12\. Tune Continuously Instead of Quarterly Every technique above is a decision you make once against a workload that never stops changing. That gap between a static setting and a moving workload holds the remaining saving. Vendors in this category publish the same diagnosis. Keebo's guide puts it flatly. > Observability without execution is just a dashboard and saves you nothing. Revefi describes its own product as continuously optimizing warehouses for efficiency. Continuous tuning is the one item here a human cannot do, because nobody adjusts configuration hundreds of times a day. Altimate's [Auto Tune](https://altimate.ai/product-highlights/snowflake-warehouse-auto-tune-right-sizing) and [Altimate Lite](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) adjust configuration every five to six seconds, against a suspend process that polls every 30 seconds. Across the private preview: - The average saving was 19%, across every customer, warehouse and day. - The floor was 12.4%, on any warehouse on any day. - The peak was 57.5%, on weekends, when workloads are least consistent. - One warehouse logged 118 tuning decisions in a single day. We measured no change in query performance or queue time across the preview. A saving that comes from slower queries only moves the cost onto whoever waits for the dashboard. ## What Gen2 Warehouses and Adaptive Compute Now Handle The twelve techniques above assume you set warehouse configuration yourself. Snowflake now sets some of that configuration for you, and that changes which techniques you still need. Standard Warehouse Generation 2 (Gen2) reached general availability, and Snowflake reports [2.1x faster performance for core analytics workloads](https://www.snowflake.com/en/blog/adaptive-compute-smarter-warehouses/). Faster is not automatically cheaper, because Gen2 costs more per hour at every size. The saving depends on whether the speed-up on your workload outruns the price increase. | Warehouse size | Gen1 credits per hour | Gen2 on AWS and GCP | Gen2 on Azure | | --- | --- | --- | --- | | X-Small | 1 | 1.35 | 1.25 | | Large | 8 | 10.8 | 10 | | 2X-Large | 32 | 43.2 | 40 | Run the break-even before you switch. A one-hour job on a Gen1 Large costs 8 credits. The same 8 credits buy 44 minutes on a Gen2 Large at 10.8 credits an hour, so Gen2 pays only when the job finishes at least 26% faster. Snowflake's 2.1x figure is a 52% cut, so the workloads Snowflake measured clear that bar. Your own slowest query might not. [Snowflake's own Gen2 page](https://docs.snowflake.com/en/user-guide/warehouses-gen2) tells you to test rather than assume. The larger shift is [Adaptive Compute](https://altimate.ai/blog/adaptive-compute-vs-auto-tune-a-practical-guide-to-optimizing-snowflake-warehouses), [generally available](https://docs.snowflake.com/en/user-guide/warehouses-adaptive) on Enterprise Edition in select regions. Snowflake describes Adaptive Warehouses this way. > Minimal configuration is required as Snowflake will automatically select the appropriate cluster size(s), the number of clusters, and auto-suspend/resume duration for jobs. The docs name four things you stop managing: warehouse size, multi-cluster settings, Query Acceleration Service settings, and suspend and resume policies. Technique 1 sets auto-suspend, technique 2 right-sizes the warehouse, and technique 8 separates workloads. Adaptive Compute now does all three for you. You cannot convert an X5Large, an X6Large, a Snowpark-optimized warehouse or an interactive one. A spreadsheet that maps workloads to warehouse sizes is obsolete under Adaptive Compute, so stop building one. The techniques that stay yours get more valuable instead. Query shape, scan volume, clustering choices, serverless sprawl and cost attribution all depend on what your queries mean. A vendor can size a warehouse. It cannot know that your dashboard reads a year of history to display last week. A timeline split into two bands. The upper band, labeled going to the platform, holds auto-suspend timing, warehouse sizing and workload separation. The lower band, labeled staying yours, holds query shape, scan volume, clustering and cost attribution *What the platform absorbs, and what stays yours because it needs business context.* ## Where to Start With Snowflake Cost Optimization Do the free four first, then measure before you touch anything else. Cost work goes wrong when somebody makes ten changes, the bill drops, and nobody knows which change did it. Measure three things first: total credits per day over ninety days, credits by warehouse, and credits for your ten most expensive queries. Then change one thing at a time and give it a week. Normalize against credits per query, because raw spend falling during a quiet month is not a saving. Credits per query survives a change in how much work you did. When a vendor reports a saving, ask whether it was measured this way or projected. Take one step today. Set auto-suspend to sixty seconds on your most expensive warehouse, then read its credits in a week. If the bill is a symptom of something wider, [nine signs your data platform needs an agent-first overhaul](https://altimate.ai/blog/signs-data-platform-needs-agent-first-overhaul) covers the rest. ## Frequently Asked Questions Which Snowflake cost optimization technique pays off fastest? Cut auto-suspend on your busiest warehouses and drop your most expensive warehouse one size. Both are single statements, both are reversible, and neither needs a purchase. Measure wall-clock time either side so you can put the size back if the workload needed it. Does a Gen2 warehouse cost less than a Gen1 warehouse? It costs more per hour. Gen2 lists at 1.35 credits at X-Small on AWS and GCP, against 1 for Gen1. It saves money only when the same work finishes at least 26% faster. Test your own workload, because Gen2 is not the default on a new warehouse. Where does serverless spend hide? Serverless spend hides in four features: Snowpipe loading files, materialized views refreshing, automatic clustering re-sorting tables, and search optimization rebuilding its access path. None of those four runs on a warehouse, so none of them reaches a per-warehouse view and a resource monitor cannot see them. Snowflake's Budgets cover all four, so put the cap there. Do the automated cost vendors actually reduce spend? Several vendors execute changes rather than only reporting them. Ask any vendor whether its reported savings are measured or projected. Check the claim against [Snowflake's warehouse docs](https://docs.snowflake.com/en/user-guide/warehouses-overview), which describe what each lever does. Who should own Snowflake cost optimization? The workload team should own it. A central FinOps group can measure and rank. It cannot judge whether a nightly refresh earns its credits. Keep Snowflake cost optimization decisions with the people who know what the query is for. On this page 1. 1\. Cut Auto-Suspend to Sixty Seconds 2. 2\. Right-Size Down, Then Measure 3. 3\. Kill the Warehouses Nobody Owns 4. 4\. Set Resource Monitors Before You Need Them 5. 5\. Find the Queries Doing the Most Damage 6. 6\. Stop Scanning What You Do Not Need 7. 7\. Cluster the Tables That Earn It 8. 8\. Move Small Frequent Work Off Big Warehouses 9. 9\. Watch the Serverless Features 10. 10\. Itemize AI and Cortex Spend 11. 11\. Attribute Cost to Teams, Not Warehouses 12. 12\. Tune Continuously Instead of Quarterly 13. What Gen2 Warehouses and Adaptive Compute Now Handle 14. Where to Start With Snowflake Cost Optimization From the product - [Enterprise Platform Auto Tune right-sizes Snowflake warehouses without slowing queries 6 min read](https://altimate.ai/product-highlights/snowflake-warehouse-auto-tune-right-sizing) - [Enterprise Platform Reduce Snowflake idle warehouse spend with Altimate Lite 6 min read](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) - [Enterprise Platform Snowflake cost analysis in plain English, with AI that cites every number 5 min read](https://altimate.ai/product-highlights/snowflake-cost-analysis-ai-cited-answers) - [Enterprise Platform Attribute Snowflake cost to the team that owns it 4 min read](https://altimate.ai/product-highlights/snowflake-cost-alerts-by-team) - [Enterprise Platform Snowflake Cortex AI cost breakdown by service, user, and model 6 min read](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # 8 Column-Level Lineage Use Cases for Data Teams URL: https://altimate.ai/blog/column-level-lineage-use-cases Eight column-level lineage use cases where knowing which field, not just which table, changes the work rather than the reporting. ## 8 Column-Level Lineage Use Cases That Change How Teams Ship Eight column-level lineage use cases where knowing which field, not just which table, changes the work rather than the reporting. Atharva Shah 2026-06-17 2026-09-04 10 min read On this page 11 sections 1. 1\. Deleting a Column Without Breaking a Dashboard 2. 2\. Sizing a Pull Request Before Review 3. 3\. Triaging an Incident in Minutes Instead of Days 4. 4\. Proving Where Personal Data Ends Up 5. 5\. Giving an AI Agent Something to Check Before It Acts 6. 6\. Finding the Column That Costs You Money 7. 7\. Retiring a Legacy Pipeline With Evidence 8. 8\. Making a Data Contract Mean Something 9. Where Column Lineage Comes From 10. Column Lineage Got Cheaper to Build In 2026 11. These Column-Level Lineage Use Cases All Replace a Search With a Traversal tl;dr Table lineage answers "what reads this table". Column lineage answers "what reads this field", through every rename and expression on the way. The difference decides the work: dropping one column in this project means checking forty tables with the first and two with the second. The eight use cases below run from dropping a column safely to making a data contract enforceable, and each one replaces a manual search with a graph traversal. A column graph needs two inputs. Static parsing reads the code before it runs, and warehouse query history reads what already ran. The graph also has to cover data pipelines, the warehouse and the BI tool. A graph that stops at the warehouse edge misses the dashboards where wrong numbers get noticed. You want to drop `legacy_segment_code` from `dim_customer`. First you need to know whether anything still reads that column. Table-level lineage gives you a list of every table downstream of `dim_customer`, forty tables in this project. The list does not say which of the forty read `legacy_segment_code`, because table-level lineage records that a table reads a table and nothing finer. So you open all forty models by hand and search each one for the column name. Column-level lineage records a finer edge, which field in one model feeds which field in the next, through every rename and expression on the way. Ask it the same question and it names the two models that read `legacy_segment_code`. You check two files instead of forty. Every one of the eight column-level lineage use cases below makes that same trade. Table lineage gives you a list you still have to search. Column lineage gives you the names. Two side-by-side dependency graphs of the same project. The left graph shows table-level lineage with one thick edge between each pair of tables, forty nodes highlighted downstream. The right graph shows column-level lineage with thin per-column edges, and only two nodes highlighted downstream of the same column *Same project, same question. Table lineage highlights forty tables. Column lineage highlights two.* The model names below follow the standard dbt convention, and the prefix marks each model's layer. A `source` is a raw table from a system like a CRM. A `stg_` (staging) model cleans and renames one source. An `int_` (intermediate) model joins staging models together. The `fct_` and `dim_` models at the end are what people query. A fact table such as `fct_orders` holds one row per order. A dimension such as `dim_customer` holds what those orders describe, like customers. A `mart_` model is built for one report. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## 1\. Deleting a Column Without Breaking a Dashboard A column is safe to drop when nothing downstream reads it. Proving that nothing reads it is the hard part, because the proof has to cover every model in the project. Column-level lineage turns that proof into a list of consumer names you can check in minutes. Take `fct_orders`, the fact table with one row per order, which carries 58 columns, one per attribute an order can have. Table lineage reports that 41 models read `fct_orders`. It cannot say which of those 41 models read which of the 58 columns, so a careful review of one column means opening all 41 files. The column graph answers the same question in one traversal, which means following the edges out from one column to every field that reads it. Run that traversal for all 58 columns and nine come back with zero consumers. Seven of those nine are `_v1` leftovers from a rename nobody finished. Without column edges, proving that a column is unused means reading every downstream model by hand. The safe answer to every delete request is then no. Dead columns then accumulate, because nobody can afford the proof. The table gets wider with each one, and every `select *` an analyst writes reads all of them. Delete the nine dead columns and every downstream scan reads less data at once. ## 2\. Sizing a Pull Request Before Review The review effort a pull request deserves should track how far the change reaches into the project. Diff size measures how many lines changed, which says nothing about reach. A column graph measures reach directly, by counting the fields downstream of the change. Say two pull requests land the same morning. The first rewrites 180 lines inside `mart_marketing_touch`, a reporting mart at the end of its chain. That model is a leaf, so no other model reads it, and one dashboard is its only consumer. The second changes four lines in `stg_orders`, the staging model that cleans the raw orders feed. One of those four lines changes the data type of `order_status`, the field that records the state each order is in. By diff size, the 180-line change looks like the risky one. Count downstream consumers instead and the ranking flips. The field `order_status` feeds 63 models and the finance revenue report. The 180-line diff feeds one dashboard tile. A reviewer who ranks by diff size spends the hour on the wrong pull request. Put the downstream count on the pull request itself and the reviewer reads it before opening a file. Somebody can then give the four-line diff an hour and the 180-line diff five minutes. Our write-up of [blast radius analysis](https://altimate.ai/blog/blast-radius-analysis-using-altimate-code) covers the mechanics. ## 3\. Triaging an Incident in Minutes Instead of Days Tracing a wrong number on a dashboard back to its cause is a graph traversal when you hold column edges. Without column edges the same trace is a manual search through every model between the dashboard and the source. Monday's revenue tile reads 12% under Friday's. The column graph shows the value reaching the tile through four hops: 1. The tile shows `net_revenue` from a Tableau extract. 2. The extract reads `mart_finance_daily.net_revenue`. 3. That reads `fct_orders.net_amount`. 4. That value is `gross_amount` minus `discount_amount`. The last commit that touched `discount_amount` is the first suspect. Done by hand, you open the dashboard's query, then the view that query reads, then the models sitting behind that view. That walk crosses three tools, the BI tool, the warehouse console and the dbt repo, and each one needs its own login. Most of the elapsed time in an incident goes on the walk rather than on the fix. A column graph that spans all three tools answers all three steps from one place. Atlan makes the same point about agent-driven triage: > If lineage stops at the warehouse boundary, the agent finds the broken table and stops there. Altimate's version of the workflow is [root cause analysis for a workload spike](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis). ## 4\. Proving Where Personal Data Ends Up A privacy review has to trace one field through every rename and every expression it passes through. Table-level lineage cannot follow a rename, because it records that a table reads a table and never which field became which. An auditor asks where this customer's email address goes. Table-level lineage can say that the source table sits upstream of thirty tables. It cannot say which of those thirty still carry the address, and the address is what the auditor asked about. In this project `stg_customers` renames the raw CRM fields into cleaner names, and `dim_customer` builds the customer dimension the reports read from those staged fields. The manual version of the trace is a text search, and renames are what break it: ```sql -- models/staging/stg_customers.sql select id as customer_id, email as contact_email, first_name from {{ source('crm', 'customers') }} -- models/marts/dim_customer.sql select customer_id, contact_email as primary_contact, concat(first_name, ' (', contact_email, ')') as display_label from {{ ref('stg_customers') }} ``` Search the project for `email` and the matches stop at `dim_customer`. Every model downstream of it reads `primary_contact` or `display_label`, and neither name contains the word email. The address is in both. A text search follows a name, so it loses the value at the first rename. The column graph follows the value through the rename into `primary_contact` and through the `concat()` call into `display_label`. The review then ends with a list of every field that holds the address. A single column traced through five hops. A source field named email is renamed to contact\_email in staging, aliased again in an intermediate model, concatenated into a display string in a mart, and finally read by a dashboard tile. Each hop is labeled with the transformation applied *One field, five hops, two renames. Table-level lineage shows the five tables and none of the renames.* ## 5\. Giving an AI Agent Something to Check Before It Acts An AI agent that renames or drops a field needs the exact set of downstream consumers before it acts. A column graph returns that set. Table lineage returns every downstream model, which is too much for the agent to check. Hand an agent the rename of `order_status`, the order-state field, to `status_code` with no lineage attached. The agent edits `stg_orders`, and the project still parses. dbt's `ref()` resolves a model name to a table and never checks which columns the referencing model selects. A downstream model that still selects `order_status` therefore raises no error at parse time. The agent reads that silence as an all-clear. The error arrives at run time, in whichever downstream model runs first and asks the warehouse for a column that no longer exists. Attach the column graph and the agent computes the 63 consumers of `order_status` before it writes a line. It then edits all 63 models in the same change, or it stops and escalates the list to a human. Either way a human sees the full downstream set before the change merges. That pre-flight check is what makes an agent safe to run unattended, because the check has one exact answer. An agent without the graph guesses at the consumers, and a stronger model only guesses better. The same argument runs across a wider set of decisions in [deterministic tooling beating LLM-only agents](https://altimate.ai/blog/deterministic-tooling-vs-llm-only-data-agents). ## 6\. Finding the Column That Costs You Money Scan cost comes from the fields a query reads. Snowflake and Databricks both store data by column, so a query pays for the columns it names and skips the rest of the row. A cost report that works at the query level blames whole queries. Trace one expensive field backwards through the column graph instead and you arrive at the upstream column that every reader downstream is paying to scan. The table `fct_events`, the fact table with one row per tracked event, holds 500 million rows. One of its columns, `raw_payload`, stores the whole event as a block of JSON text and averages 2 KB per row. Any model that selects `raw_payload` therefore reads roughly 1 TB. Three hourly aggregations select it. None of those three needs more than two keys out of that JSON. Promote those two keys to their own columns in the upstream model and the three readers select the keys instead of `raw_payload`. Nobody flagged the cost before, because a query-level report shows three hourly queries that each look reasonable alone. The column graph shows the same three queries as three readers of one 2 KB column. Reading the execution plan for one of those queries confirms what the scan pulled, and our write-up of [profiling a slow dbt query](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan) walks through that step. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## 7\. Retiring a Legacy Pipeline With Evidence [Turning off a legacy pipeline](https://altimate.ai/blog/dbt-migration-with-an-ai-agent-playbook) is safe once you hold a list of its live consumers, field by field. Column lineage that spans both the old path and its replacement produces that list. Without the list, the decision rests on whoever has been at the company longest. One question decides whether a legacy path can go: does anything still read it? The code alone cannot answer it, so the fallback is to turn the path off and wait for complaints. That works, and it costs an incident each time a consumer nobody knew about breaks. Column lineage answers the question with numbers. The legacy model `legacy_orders_daily`, an older daily orders rollup being retired, exposes 34 columns. Its replacement, `fct_orders`, the new orders fact table, already serves 31 of those 34 under the same names. The column graph across both paths shows that only three of the 34 fields still have live consumers on the legacy path. All three feed one finance extract. Send that one team a cutover date for those three fields and turn the rest of the legacy path off the same week. ## 8\. Making a Data Contract Mean Something A data contract is a promise about the shape of a model, the columns it returns and their types. The contract binds only when something detects that a change touched a field it covers. That detection is a lineage query, because the change that breaks a contract usually happens upstream of the model the contract names. At build time, dbt enforces the shape of a model: ```yaml # models/marts/schema.yml models: - name: dim_customer config: contract: enforced: true columns: - name: primary_contact data_type: varchar ``` With `enforced: true`, dbt checks before the build that `dim_customer` returns `primary_contact`, the customer's contact field, as a varchar. That check sees the model's output and nothing else. The upstream rename in `stg_customers` that produced the field sits outside the check, and so does every model downstream that depends on it. [dbt's model contracts documentation](https://docs.getdbt.com/docs/collaborate/govern/model-contracts) states that a contract defines the shape of the returned dataset. The column graph supplies the other half. When a pull request changes the expression behind `primary_contact`, the graph reports that the change reaches a field under contract. The reviewer then knows the change needs a version bump. Contracts fail because nobody noticed that a change touched one, and the column graph notices. ## Where Column Lineage Comes From A column graph comes from two sources, static parsing of the SQL and the warehouse's own query history. Each source covers the cases the other misses. Static parsing reads your SQL and follows the field references. It is fast, it needs no warehouse access, and it works before a model has ever run. Static parsing struggles with dynamic SQL, heavy macro use and `SELECT *`: ```sql -- models/staging/stg_orders.sql select * from {{ source('shop', 'orders') }} -- models/intermediate/int_orders_enriched.sql select o.*, o.gross_amount - o.discount_amount as net_amount from {{ ref('stg_orders') }} o ``` A parser reading those two files knows that `int_orders_enriched` depends on `stg_orders`. The parser can name exactly one field edge, the one that builds `net_amount`. Every other column in `orders` reaches `int_orders_enriched` through two stars, and a star carries no names until something resolves the source schema. Warehouse query history is the other source. Snowflake records column-level access in [ACCESS\_HISTORY](https://docs.snowflake.com/en/sql-reference/account-usage/access_history), and Databricks captures [Unity Catalog lineage](https://docs.databricks.com/aws/en/data-governance/unity-catalog/data-lineage) down to the column. These edges come from queries that ran, so the warehouse already expanded every star and every macro before it recorded the columns. That covers the dynamic cases a parser cannot read. Query history has no record of a model that has not run yet, or of the branch on your laptop. | | Static SQL parsing | Warehouse query history | | --- | --- | --- | | Reads | model code in the repo | queries the platform already executed | | A model that never ran | ✓ resolved | ✗ invisible | | `SELECT *` | ≈ resolved only with the source schema | ✓ expanded, because the warehouse expanded it at run time | | Dynamic SQL and macro output | ✗ usually missed | ✓ recorded once it runs | | An ad-hoc query outside dbt | ✗ invisible | ✓ recorded | | Freshness | current with the working branch | Snowflake documents ACCESS\_HISTORY latency of up to 180 minutes | | History depth | whatever the repo holds | 365 days on ACCESS\_HISTORY, a one-year rolling window on the Databricks lineage system tables | | Documented blind spot | unresolved stars, macro-heavy models | intermediate views between the base table and the direct object on Snowflake, file-path sources and user-defined functions on Databricks | A tool that reads both sources covers what each one misses. Our [column lineage reference](https://help.altimate.ai/dbt-power-user/test/lineage/) covers the editor-side version and names which edges the lineage panel resolves statically and which ones need a warehouse connection. Freshness decides how much of the graph you can act on. A graph rebuilt nightly describes yesterday's project, and the change you are reviewing belongs to today. An impact count from a nightly graph therefore answers a question about a codebase that no longer exists. Rebuild on every change, and treat a nightly graph as documentation rather than a check. Check the coverage number before you trust a graph. A tool that parses 90% of your models sounds good until you find that the missing 10% are the macro-heavy models everything depends on. A useful coverage report names the failed models, because a percentage says nothing about which models fell out. A coverage report listing model counts with three bars: parsed successfully, parsed with warnings, and unparseable. A callout points at the unparseable bar noting these are the macro-heavy models with the highest downstream fan-out *The unparseable models are the macro-heavy ones with the most downstream readers.* ## Column Lineage Got Cheaper to Build In 2026 Building column-level lineage used to mean writing a SQL parser first, so a vendor was the only practical source. That changed when the dbt engine started parsing SQL. dbt's Fusion engine, the rewrite behind [dbt Core v2.0](https://docs.getdbt.com/blog/dbt-core-v2-is-here), reads each model's SQL as SQL, where earlier versions treated it as a text template. An engine that resolves every field reference already knows which column feeds which, so column-level edges fall out as a by-product. A parser added afterwards has to reconstruct those edges. Today the parser ships in the engine you already run, and what is left to build is the graph and the coverage reporting on top of it. Whichever side of the build-versus-buy line you land on, ask the same three questions. What percentage of models parse cleanly, which models fail, and how many downstream models sit behind those failures? ## These Column-Level Lineage Use Cases All Replace a Search With a Traversal Each of the eight column-level lineage use cases above comes down to one question that column edges answer with a name or a count. | Question | Table-level answers | Column-level answers | | --- | --- | --- | | Can I drop this column? | ✗ check everything downstream | ✓ two consumers | | How risky is this pull request? | ≈ by diff size | ✓ by downstream column count | | What broke this dashboard number? | ≈ a candidate list of tables | ✓ a path to the field | | Where does this email address go? | ≈ tables that might carry it | ✓ the field, through renames | | What will this agent's change break? | ✗ unknown | ✓ an exact set, before it acts | | Which column is driving up scan cost? | ✗ blames the whole query | ✓ names the field | | Is the legacy path still used? | ≈ probably | ✓ a list of live consumers | | Did a change break a data contract? | ✗ nothing flags it | ✓ flags the version bump | Column lineage is one of the nine checks in [nine signs your data platform needs an agent-first overhaul](https://altimate.ai/blog/signs-data-platform-needs-agent-first-overhaul), which is the broader diagnostic. Pick the field your team argues about most. Ask your lineage tool for that field's downstream consumers, then open the models and count the consumers by hand. The gap between the two counts is your coverage number. ## Frequently Asked Questions What is the difference between table-level and column-level lineage? Table-level lineage records that one table reads from another. Column-level lineage records which field flows into which field, including through renames and expressions. Only column-level lineage can answer whether a given column has any live consumers. Do I need column-level lineage if I already have a data catalog? Most catalogs give you table-level lineage and a list of each table's columns. Neither one is a column-level edge, because a column list says what a table holds and never who reads each column. Check whether your catalog can answer "what reads this specific column" rather than "what reads this table". Can column-level lineage cross tool boundaries? Column-level lineage has to cross tool boundaries for incident triage to run as one traversal. A wrong number is usually noticed on a dashboard, so the trace starts in the BI tool. A graph that stops at the warehouse edge has no record of the dashboard tile, so the trace has to start one hop in, by hand. What does SELECT \* do to column lineage? A star carries no field names, so a static parser reading `select *` records the table edge and no column edges under it. Give the parser the source schema so it can expand the star. The other fix is warehouse query history, where the star was already expanded when the query ran. Do these column-level lineage use cases matter on a small project? On a small project where one engineer can name every consumer of every column, the benefit is modest. Every one of the column-level lineage use cases above starts to pay off once nobody on the team can do that without looking. On this page 1. 1\. Deleting a Column Without Breaking a Dashboard 2. 2\. Sizing a Pull Request Before Review 3. 3\. Triaging an Incident in Minutes Instead of Days 4. 4\. Proving Where Personal Data Ends Up 5. 5\. Giving an AI Agent Something to Check Before It Acts 6. 6\. Finding the Column That Costs You Money 7. 7\. Retiring a Legacy Pipeline With Evidence 8. 8\. Making a Data Contract Mean Something 9. Where Column Lineage Comes From 10. Column Lineage Got Cheaper to Build In 2026 11. These Column-Level Lineage Use Cases All Replace a Search With a Traversal From the product - [Enterprise Platform Root cause analysis for a dbt, Tableau, or Snowflake workload spike 6 min read](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis) - [PowerUser Plugin for dbt Profile a slow dbt query and read its execution plan in plain English 4 min read](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan) - [Enterprise Platform Reduce Snowflake idle warehouse spend with Altimate Lite 6 min read](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) - [Enterprise Platform Snowflake Cortex AI cost breakdown by service, user, and model 6 min read](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown) - [Enterprise Platform Attribute Snowflake cost to the team that owns it 4 min read](https://altimate.ai/product-highlights/snowflake-cost-alerts-by-team) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # 7 dbt Anti-Patterns to Catch Before Merge URL: https://altimate.ai/blog/dbt-anti-patterns-ai-should-catch Seven dbt anti-patterns that pass review and cost you later, why each one survives a human read, and the check that catches it automatically. ## 7 dbt Anti-Patterns AI Should Catch Before Your PR Merges Seven dbt anti-patterns that pass review and cost you later, why each one survives a human read, and the check that catches it automatically. Atharva Shah 2026-07-01 2026-09-04 10 min read On this page 11 sections 1. 1\. SELECT \* in a Model Anything Depends On 2. 2\. Joins That Fan Out Without Anyone Noticing 3. 3\. Filters That Defeat Pruning 4. 4\. Non-Deterministic Ordering Treated as Stable 5. 5\. Incremental Models That Drop Rows or Duplicate Them 6. 6\. Tests That Cannot Fail 7. 7\. Undocumented Columns in a Model Others Build On 8. How Each dbt Anti-Pattern Shows Up and What Catches It 9. Where to Catch These dbt Anti-Patterns 10. Dbt's Fusion Engine Changes Where Checks Belong In 2026 11. Which Anti-Patterns a Machine Should Own tl;dr Seven dbt anti-patterns follow, each with the pattern in code, the fix in code, and the check that catches it before merge. All seven pass code review, build without an error, and return a number that looks right, so the cost arrives weeks later. The fan-out join costs the most, because its inflated total still reads as a plausible revenue figure that nobody reconciles against the source system. The checks run in three places: - the editor catches a pattern while you type, - CI catches one that arrived from somewhere else, and - the warehouse catches the two that need the data rather than the SQL. Altimate publishes 19 anti-pattern rules at 100% accuracy and zero false positives across 1,077 benchmark queries. The false-positive rate matters more than the rule count, because a check that fires wrongly once gets the whole set turned off. A fan-out join is a join where the key repeats on one side, so each row from the other side comes back several times. Put one in a revenue model and the sum of `gross_amount` reports four times the real revenue when every order has four line items. The model still builds. The inflated total still looks like a plausible dollar figure, so the reviewer approves it and the pull request merges. All seven dbt anti-patterns below work the same way. The code compiles, the model runs, and the result looks right. The cost arrives weeks later as a wrong figure, a slow model, or a warehouse bill nobody can explain. A machine check should own these seven problems. A reviewer reading a long diff checks the logic against the ticket, so a reviewer looks for intent errors. None of the seven is an intent error. Each one has an exact definition, and a compiled rule evaluates that definition the same way on every run. A human applies it consistently on the first model of the day and less consistently by the last. A pull request review screen with a diff beside an automated check panel that lists seven named checks, one of them expanded to show the offending line, the rule name and the downstream models affected *What a reviewer should see next to the diff.* First, the dbt terms these examples lean on. A dbt project is a set of SQL files called models, each building one table or view. A model reads from raw tables through `source()` and from other models through `ref()`. Those links form the project's dependency graph. A model's prefix marks its layer. A `stg_` staging model cleans one raw source. The `fct_` and `dim_` models are the marts people query. A model is "downstream" of another when it reads from it. A change to a shared upstream model [reaches everything downstream of it](https://altimate.ai/blog/blast-radius-analysis-using-altimate-code). Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## 1\. `SELECT *` in a Model Anything Depends On `SELECT *` in a staging model reads as tidy code and behaves badly. The star couples the model to whatever columns the source holds today. When somebody adds a column upstream, your staging table widens on the next run. ```sql -- models/staging/stg_orders.sql -- Anti-pattern: the column list is whatever the source happens to hold today. select * from {{ source('shop', 'orders') }} ``` The first cost is a wider scan, because every downstream model that also selects a star reads the new columns too. The worse cost is a name collision. A new upstream column arrives with a name a downstream model already uses. That downstream model then breaks, and it breaks in a file nobody edited. ```sql -- models/staging/stg_orders.sql -- Fix: name the columns. A new source column now changes nothing here. select order_id, customer_id, order_status, gross_amount, discount_amount, ordered_at from {{ source('shop', 'orders') }} ``` The check is a text match on the star, so any linter can run it in the editor. A reviewer sees the same three characters and cannot see the columns they will pull in after the source grows. Scope the rule to models that have dependents. A star in a one-off exploratory model harms nobody, and a rule that fires on that model gets switched off for the whole project. A [dbt model contract](https://docs.getdbt.com/docs/collaborate/govern/model-contracts) enforces the declared column list at build time. ## 2\. Joins That Fan Out Without Anyone Noticing A join fans out when the join key is unique in one table and repeats in the other. Each row on the unique side comes back once per match on the repeating side. The model still builds, and every sum over the result is multiplied by the number of matches. Here `stg_orders` holds one row per order and `stg_order_items` holds one row per line item. Joining them on `order_id` returns each order once for every item it has. ```sql -- models/marts/fct_order_revenue.sql -- Anti-pattern: order_items repeats order_id, so each order row multiplies. select o.order_id, o.gross_amount, i.sku from {{ ref('stg_orders') }} o join {{ ref('stg_order_items') }} i using (order_id) ``` With four line items per order, every order row appears four times, and a sum of `gross_amount` reports four times the real revenue. The fan-out join is the most expensive of the seven anti-patterns. The inflated total still reads as a plausible dollar figure, so nobody questions the total. Nobody reconciles a revenue total against the source system every week, so the wrong figure can survive for months. ```sql -- Fix: collapse the many side to one row per key before the join. with items as ( select order_id, count(*) as item_count from {{ ref('stg_order_items') }} group by 1 ) select o.order_id, o.gross_amount, items.item_count from {{ ref('stg_orders') }} o left join items using (order_id) ``` Catching a fan-out join needs the cardinality of the join key, and cardinality is a property of the data. Nothing in the query text says whether `order_id` repeats in `stg_order_items`. A parser, a linter and a language model reading the file all share that blind spot. The check has to run against the warehouse. It is two counts on the many side of the join, and the key repeats whenever the two counts differ. ```sql -- The check: the key repeats, so this join fans out. select count(*) as rows_in, count(distinct order_id) as keys_in from {{ ref('stg_order_items') }} ``` ## 3\. Filters That Defeat Pruning A filter that wraps a column in a function stops the warehouse from [skipping partitions](https://altimate.ai/blog/optimizing-snowflake-queries-expert-strategies-for-data-engineers-and-analysts). The warehouse skips a partition by comparing the filter value against the range of values it stores for that partition. A column wrapped in `date_trunc` is a computed value with no stored range, so the warehouse reads every partition. Both queries below return the same rows, and one of them reads the whole table. ```sql -- Anti-pattern: the filter wraps the column, so the pruner cannot use it. select order_id, gross_amount from {{ ref('fct_orders') }} where date_trunc('month', ordered_at) = '2026-03-01' ``` ```sql -- Fix: a half-open range on the raw column. Same rows, three partitions. select order_id, gross_amount from {{ ref('fct_orders') }} where ordered_at >= '2026-03-01' and ordered_at < '2026-04-01' ``` A reviewer reads this query as ordinary. The SQL is short, the logic is right, and the result set is small. The full-table scan sits in the plan, and nobody opens the plan during a review. A lint rule finds the wrapped column from the text alone, so it fires in the editor while the author still has the file open. Our write-up of [profiling a slow dbt query](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan) covers reading the plan. Two query plans for the same filter. The left plan, on a function of the column, scans the full table across 480 partitions. The right plan, on a range over the raw column, reads 3 partitions *The same rows, two ways. The left plan reads 480 partitions and the right reads three.* ## 4\. Non-Deterministic Ordering Treated as Stable A `LIMIT` with no `ORDER BY` returns whichever rows the engine reaches first. An `ORDER BY` on a column with duplicate values has the same problem, because the engine breaks each tie however it likes. Both look stable in testing, because the data has not moved yet. ```sql -- Anti-pattern: two events share one updated_at, so rn = 1 picks either row. select * from ( select *, row_number() over ( partition by customer_id order by updated_at desc ) as rn from {{ ref('stg_customer_events') }} ) where rn = 1 ``` The window above numbers each customer's events newest first and keeps number one, which should leave one row per customer. Downstream logic then depends on an order the engine never promised. After the next reload the engine breaks the tie on `updated_at` the other way, so the model keeps a different row for the same customer. The dashboard number moves with no commit behind it, and somebody goes looking for a diff that does not exist. ```sql -- Fix: break the tie on a column that is unique inside the partition. select * from ( select *, row_number() over ( partition by customer_id order by updated_at desc, event_id desc ) as rn from {{ ref('stg_customer_events') }} ) where rn = 1 ``` The check reads the SQL alone, so it runs in the editor. It flags any `LIMIT` with no total ordering, and any window function whose partition and order cannot pin down one row. A reviewer misses both, because a `LIMIT 1` and a `row_number()` window both look like finished code. ## 5\. Incremental Models That Drop Rows or Duplicate Them An incremental model adds only new rows on each run instead of rebuilding the whole table. That is what keeps a large model cheap to run. It has two defaults that cost you rows in opposite directions. A filter with no lookback window drops rows that arrive late. A config with no `unique_key` duplicates rows the table already holds. Neither default ever fails a run, so the model stays green while the row count drifts. ```sql -- models/marts/fct_events.sql -- Anti-pattern: no lookback window, and no unique_key on the config. {{ config(materialized='incremental') }} select * from {{ ref('stg_events') }} {% if is_incremental() %} where event_at > (select max(event_at) from {{ this }}) {% endif %} ``` The filter above loads only rows with a timestamp above the stored maximum. A row that lands later with an earlier timestamp fails that filter on every run, so it never enters the table. The model runs green and a slice of the data is absent. The gap is invisible, because you cannot see a row that never arrived. The second default adds rows instead of hiding them. On most adapters a model with no `unique_key` appends every row the query returns, whether or not the table already holds it. [dbt's incremental models documentation](https://docs.getdbt.com/docs/build/incremental-models) states that plainly. Widen the lookback on a model with no key, and every run appends three days of history the table already has. ```sql -- Fix: a lookback window, plus a key so late rows merge instead of appending. {{ config(materialized='incremental', unique_key='event_id') }} select * from {{ ref('stg_events') }} {% if is_incremental() %} where event_at >= (select max(event_at) - interval '3 days' from {{ this }}) {% endif %} ``` A check finds both defaults from the file alone. The first finding is an incremental filter that reads strictly greater than the stored maximum, with no lookback window. The second is a config with no `unique_key`, which is a single grep over the config blocks. A reviewer misses both, because each one is an absence, and a diff shows what was written rather than what was left out. How wide the lookback should be depends on how late your source delivers rows, and nobody can automate that judgment. Whether the model has a lookback at all is a yes-or-no question, and the check answers it. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## 6\. Tests That Cannot Fail A dbt test is an assertion on a column, such as `not_null` or `unique`. It fails the build when the data breaks that rule. One config key produces the worst version of a test that cannot fail. Set `severity: warn` at the project level, and every test still runs and reports a result. None of them can block a merge again, because a warning does not fail the build. ```yaml # dbt_project.yml # Anti-pattern: one line, and no test in the project can fail a build again. tests: +severity: warn ``` The default severity is `error`. Once the key reads `warn`, a failing test becomes an error only when somebody passes `--warn-error`. [dbt's severity reference](https://docs.getdbt.com/reference/resource-configs/severity) spells that out. An audit that looks for missing tests finds nothing wrong, because every test is still there. Two quieter versions do the same damage one test at a time. A `not_null` test on a column the database already declares `NOT NULL` can never fail, because the database rejects the null first. A `unique` test on a surrogate key the model generates tests your own hash function. Both pass forever and both count toward coverage, so the coverage number overstates what the suite protects. The check for `severity: warn` is a grep over `dbt_project.yml` and every schema file. The check for the two quieter versions asks whether a given test has ever failed and whether it can. That needs test history, and dbt does not keep test results between runs. [The data tests documentation](https://docs.getdbt.com/docs/build/data-tests) says `store_failures` replaces the previous failures for the same test, so the store holds only the latest run. Collect `run_results.json` from each CI run, because that file is the only history you will have. ## 7\. Undocumented Columns in a Model Others Build On [An undocumented column](https://altimate.ai/blog/write-better-data-docs-faster) looks harmless, because nothing fails when you skip the description. It becomes expensive the moment somebody downstream has to decide what the column means with nothing to read. A human asks in Slack and waits for an answer. An agent picks the most plausible reading and reports a number. ```yaml # models/marts/schema.yml # Anti-pattern: the column is declared, and nothing says what it counts. models: - name: fct_orders columns: - name: net_revenue ``` The label `net_revenue` could exclude tax, refunds, canceled orders or all three. Every reading produces a defensible number. A human building on the model picks one reading, and it differs from the one the author had in mind. Two reports then disagree by the size of the refunds. An agent guesses faster, and it gives no sign that it guessed. ```yaml # Fix: one description an agent and a new analyst both read the same way. models: - name: fct_orders columns: - name: net_revenue description: > Gross amount minus discounts and refunds, in USD, excluding tax. Canceled orders are excluded. Finance reports on this column. ``` The check compares the columns a model exposes against the columns its schema file describes, and it flags any column with no description on a model that has dependents. That is a text comparison, so it runs in CI, where the project graph already exists. A revenue analysis agent could not answer a single question in one Atlan-documented deployment at Workday, because nothing mapped the term revenue to authoritative columns. ## How Each dbt Anti-Pattern Shows Up and What Catches It | Anti-pattern | How it shows up | The check that catches it | | --- | --- | --- | | `SELECT *` in a shared model | A downstream model breaks on a name nobody added | Reject a star in any model with dependents | | Fan-out join | A revenue total is a clean multiple of the truth | Count rows against distinct keys on the many side | | Pruning-defeating filter | One query dominates the bill and looks ordinary | Lint for a column wrapped in a function inside `WHERE` | | Non-deterministic ordering | A number moves with no commit behind it | Flag `LIMIT` with no total order, and ties inside a window | | Incremental with bad defaults | Late rows never arrive, or row counts grow faster than the source does, and nothing errors either way | Read the incremental filter for a lookback window, and grep the config block for a `unique_key` | | Test with `severity: warn` | Every test runs and no test blocks | Grep `dbt_project.yml` and every schema file | | Undocumented column | An agent answers a revenue question wrongly | Compare the model's columns against its descriptions | Every symptom in the middle column shows up weeks after the commit that caused it. Review misses all seven because of that delay. A reviewer connects a diff to a problem they can see, and at review time the problem has not happened yet. A check reads the diff for the pattern, so it does not need the symptom to appear first. ## Where to Catch These dbt Anti-Patterns Run these checks in three places: the editor, CI, and the warehouse. Each place catches something the other two miss. The editor catches a pattern while the person who wrote it is still looking at it, which is when a fix costs seconds. A slow check cannot run on every keystroke, so an editor check works from the text of the open file alone. CI catches whatever arrived from somewhere else, including code an agent wrote while nobody watched. CI is the only place where a check is never optional, so it owns the rules you block a merge on. A gate that fires on style opinions gets bypassed, and a bypassed gate protects nothing. The warehouse catches what neither the editor nor CI can see. Key cardinality, real row counts and test history are properties of the data, so you get them from a query against the warehouse and never from parsing a file. Cost history belongs in the same layer, and [tracking query cost over time](https://altimate.ai/product-highlights/snowflake-query-cost-over-time) is where a pruning regression shows up as money. One rule decides where each check goes. If the SQL text alone answers the question, put the check in the editor. If the check needs the project graph, put it in CI, because CI builds the graph anyway. If the check needs the data, run it against the warehouse on a schedule, and accept that some problems surface after they merge. A three-column diagram labeled editor, CI and warehouse, each column listing which of the seven anti-patterns it can catch, with two patterns in the warehouse column only and marked needs the data, not the SQL *Which of the seven each layer can catch.* ## Dbt's Fusion Engine Changes Where Checks Belong In 2026 Until 2026, catching these anti-patterns meant a separate linter with its own SQL parser. Then dbt's Fusion engine shipped. Fusion is a Rust rewrite of dbt that parses and resolves SQL itself. dbt Labs publishes it as [up to 30x faster than dbt Core v1](https://docs.getdbt.com/blog/dbt-fusion-engine), with real-time error detection in the editor. An engine that understands the query catches a class of structural problem with no second tool reading the same files. The SQL comprehension, the linting and the column-level lineage ship in the Fusion binary only. [dbt Core v2.0](https://docs.getdbt.com/blog/dbt-core-v2-is-here), the Apache-licensed build that reached its first alpha on 1 June 2026, does not carry them. Fusion covers part of the list. Reference errors, type mismatches and column resolution are properties of the SQL, so they improve once the engine comprehends the query. Anything that needs the data stays out of reach. The fan-out join needs cardinality and the test that cannot fail needs test history, so no engine catches either one from a parse. Our own rule layer sits between the engine and the warehouse query. It catches the pattern-level anti-patterns the engine has no opinion about, such as `SELECT *` in a shared model or a pruning-defeating filter. Each check runs in about 0.48 ms, so it fits between two keystrokes rather than waiting for a save. The [Validators reference](https://help.altimate.ai/code/data-engineering/validators/) lists what each rule checks, and [dbt PR Review](https://help.altimate.ai/code/usage/dbt-pr-review/) covers running the same rules at merge time. A team that puts everything into one custom linter rewrites SQL parsing badly, then maintains it through every dialect change. A team that expects the engine to catch everything ships fan-out joins, because no engine reads cardinality out of a query it has not run. ## Which Anti-Patterns a Machine Should Own Six of the seven anti-patterns have an exact definition, which is what makes them automatable. The exception is the incremental model, where the presence of a lookback is exact and its width is a judgment. Altimate publishes 19 anti-pattern rules at 100% accuracy across 1,077 benchmark queries with zero false positives. The false-positive rate matters more than the rule count, because a check that fires wrongly once gets the whole set turned off. The general form of this argument is [why deterministic tooling beats LLM-only data agents](https://altimate.ai/blog/deterministic-tooling-vs-llm-only-data-agents). For sizing a change before review, [column-level lineage use cases](https://altimate.ai/blog/column-level-lineage-use-cases) covers the other half. Start with two greps. Search your project for `severity: warn`, then for incremental configs with no `unique_key`. Neither needs any tooling you do not already have. ## Frequently Asked Questions Why not just catch these in code review? A reviewer reads a diff for intent, and none of the seven is an intent error. A `SELECT *` is three characters, and a join that fans out looks identical to a join that does not fan out. Nothing in the diff gives a reviewer something to react to. Which of the seven costs the most? The fan-out join costs the most. Its output is an inflated total that still looks like a plausible revenue figure. Nobody reconciles revenue against the source system every week, so the wrong total can survive for months. Every decision taken against that total inherits the error. The pruning filter costs more in warehouse credits, but it announces itself on the bill. Can an LLM catch dbt anti-patterns instead of a rule set? An LLM catches most of them most of the time, and for a merge gate that is the wrong kind of reliability. A gate that fires on the same code one day and passes it the next loses trust, and a gate nobody trusts gets skipped. These seven patterns have exact definitions, so a compiled rule evaluates each one the same way every time and can reach zero false positives. How do I check whether my dbt tests actually test anything? Start with a grep for `severity: warn`, because that one config makes every test under it unable to fail a build. Then collect `run_results.json` from your CI runs and list the tests that have never failed. For each one, ask whether it could fail. A `not_null` test on a column the database already constrains cannot fail, so it protects nothing. Do dbt's built-in tests cover these seven? dbt's built-in tests cover the assertions you write yourself, and together with model contracts they handle the documented cases well. [dbt's own test and contract features](https://docs.getdbt.com) do not detect a fan-out join, a pruning-defeating filter, or a test somebody set to warn. Each of those needs a parse of the SQL or a query against the data. A dbt test is an assertion about a column, which is neither. On this page 1. 1\. SELECT \* in a Model Anything Depends On 2. 2\. Joins That Fan Out Without Anyone Noticing 3. 3\. Filters That Defeat Pruning 4. 4\. Non-Deterministic Ordering Treated as Stable 5. 5\. Incremental Models That Drop Rows or Duplicate Them 6. 6\. Tests That Cannot Fail 7. 7\. Undocumented Columns in a Model Others Build On 8. How Each dbt Anti-Pattern Shows Up and What Catches It 9. Where to Catch These dbt Anti-Patterns 10. Dbt's Fusion Engine Changes Where Checks Belong In 2026 11. Which Anti-Patterns a Machine Should Own From the product - [PowerUser Plugin for dbt Profile a slow dbt query and read its execution plan in plain English 4 min read](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan) - [Enterprise Platform Track Snowflake query cost over time, run by run 6 min read](https://altimate.ai/product-highlights/snowflake-query-cost-over-time) - [Enterprise Platform Root cause analysis for a dbt, Tableau, or Snowflake workload spike 6 min read](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis) - [Enterprise Platform Reduce Snowflake idle warehouse spend with Altimate Lite 6 min read](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) - [Enterprise Platform Snowflake Cortex AI cost breakdown by service, user, and model 6 min read](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # 10 Best MCP Servers for Data Engineering URL: https://altimate.ai/blog/best-mcp-servers-for-data-engineers MCP lets an agent look things up instead of guessing. Ten best MCP servers for data engineering, what each exposes, and the security question first. ## 10 Best MCP Servers for Data Engineering Teams MCP lets an agent look things up instead of guessing. Ten best MCP servers for data engineering, what each exposes, and the security question first. Atharva Shah 2026-06-04 2026-09-04 11 min read On this page 14 sections 1. MCP Replaces Per-Agent Wiring With One Server 2. 1\. Warehouse Servers for Snowflake and BigQuery 3. 2\. Postgres and Operational Database Servers 4. 3\. dbt and Transformation-Layer Servers 5. 4\. Lineage and Catalog Servers 6. 5\. Altimate MCP Server for Warehouse and Cost Context 7. 6\. Orchestrator Servers for Airflow and Dagster 8. 7\. Git and Code-Host Servers 9. 8\. Documentation and Knowledge-Base Servers 10. 9\. Ticketing Servers for Jira and Linear 11. 10\. Filesystem and Local Servers 12. Official Servers Arrived in 2026 as Reference Servers Were Archived 13. The Security Questions to Ask Before You Install a Server 14. How to Pick the Best MCP Servers for Data Engineering tl;dr An agent that guesses your schema writes plausible SQL that does not run. MCP lets the agent look the schema up instead. This list of the best MCP servers for data engineering covers ten categories, ranked from warehouse servers down to the local filesystem. Check maintenance status before you install. The MCP steering group archived its reference PostgreSQL and GitHub servers, and config examples in 2025 tutorials still name them. Copying one of those examples installs a package nobody patches. The spec tells a client to treat a server's own tool labels as untrusted. A tool that labels itself read-only has therefore proved nothing. Start read-only, scope a filesystem server to one directory, and widen access only when you choose to. You point an AI agent at your dbt project and ask it to write a model. The agent has no way to read your warehouse, so it [guesses the schema](https://altimate.ai/blog/why-ai-coding-agents-fail-at-data-engineering). The SQL it writes looks plausible, then fails on the first column name the agent invented. Model Context Protocol, or MCP, is the standard that lets the agent look the schema up instead of guessing. [The spec](https://modelcontextprotocol.io) defines how a server exposes tools any compatible agent can call. Pasting the schema into the system prompt does not work either. A production warehouse holds too many tables for one prompt. The schema also changes while you write the prompt, so the pasted copy goes stale. This guide ranks ten categories of MCP server by what they expose, covering: - what each server is, - who it fits, - what it costs, and - where it falls short. A hub diagram. An MCP client labeled your agent sits in the center, ringed by ten labeled server boxes for warehouse, transformation, orchestration and knowledge ## MCP Replaces Per-Agent Wiring With One Server Before MCP you wired each agent to each data system by hand. Three agents against one lineage store meant three integrations to write and maintain. MCP replaces that wiring with one server. The server declares a set of tools, and every client that speaks MCP calls the same tools. One lineage lookup then serves a CLI agent, an editor agent and a chat assistant. The spec defines two transports. A stdio server runs as a subprocess on your own machine, and a Streamable HTTP server answers at a URL. ## 1\. Warehouse Servers for Snowflake and BigQuery A guessed schema is the first thing that breaks agent-written SQL, so a warehouse server is the first one to install. - **What it is:** A server exposing list-tables, describe-table and run-query against Snowflake or BigQuery. - **Best for:** Teams whose agent keeps naming columns that do not exist, such as `customer_email` for a field the table calls `email_address`. - **Key features:** The three tools work together. The agent lists the tables, describes the one it needs and runs its query against the real column names, so it no longer guesses a column. Snowflake also runs a managed server inside your account. That server answers over HTTPS and authenticates with OAuth. No long-lived key sits in a config file. - **Pricing:** Free to run. You pay the warehouse only for what run-query executes. - **Limits:** run-query is the tool to watch. Given a role with write grants, it can change data. Given a large table, one query can scan enough of it to cost money. Read the vendor's permission guidance before you grant the role. Snowflake grants the managed server one primary role and no secondary roles, so the grants on that one role are the whole permission set. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## 2\. Postgres and Operational Database Servers Your pipeline's source of truth usually lives in an application database another team owns. A Postgres server reads that schema live. A staging model written against a stale copy of the schema breaks on the next load. - **What it is:** A server that reads an operational database such as Postgres. - **Best for:** Teams whose staging models sit on a schema that drifts. A column turned nullable and the team's doc still calls it required. - **Key features:** It reads the live schema, including constraints, indexes and nullability. It also runs read-only queries against the source. - **Pricing:** Free and open source. - **Limits:** The MCP steering group archived its reference PostgreSQL server. The archived package still installs, and config examples in older tutorials still show `@modelcontextprotocol/server-postgres`. Copying one of those examples installs a package nobody patches. Take the vendor's server or an active replacement from the MCP Registry. ## 3\. dbt and Transformation-Layer Servers A dbt server hands the agent your project's dependency graph, so the agent asks the server what depends on a model. Without the server it infers the graph by reading `ref()` calls across every file. - **What it is:** A server that hands an agent your dbt project graph, compiled SQL and test results. dbt Labs maintains the official one under Apache 2.0, in a self-hosted form and a remote form. - **Best for:** Any team on dbt whose agent needs compiled SQL and test results rather than the raw model files in the repo. - **Key features:** The self-hosted form runs with `uvx dbt-mcp` on your own machine over stdio. The remote form runs on the dbt platform and answers over HTTP. Nine tool groups sit behind the dbt server, including the dbt CLI, the Semantic Layer and the dbt Fusion engine tools. - **Pricing:** Free and open source. The remote form needs a dbt platform account. - **Limits:** An account that runs out of dbt Copilot actions loses every tool on the [remote server](https://docs.getdbt.com/docs/dbt-ai/about-mcp). That includes the self-hosted tools the remote server proxies. The remote server also shares the dbt platform rate limit of 5,000 requests per minute per IP address. Wire the self-hosted form into a client as a local stdio server: ```json { "mcpServers": { "dbt": { "command": "uvx", "args": ["dbt-mcp"] } } } ``` ## 4\. Lineage and Catalog Servers A catalog server exposes metadata the warehouse does not store, including glossary definitions, lineage edges and quality-check results. - **What it is:** Atlan's catalog server exposes its metadata to an agent as function calls. - **Best for:** Teams with a populated catalog. Their agents need the definition of a metric that the finance team agreed on, and a column name alone cannot supply that definition. - **Key features:** It searches the catalog, traverses lineage in either direction and retrieves an asset by name. It also looks up a glossary term, returns quality-check results and updates metadata. - **Pricing:** Free to install. It requires a catalog account, which is a paid product. - **Limits:** Updating metadata is the only write in that list. The glossary definition is what every downstream reader trusts, and an agent holding the update tool can change it. Grant that one tool last. Atlan states the problem its server is built for: > Agents fail in production because their context is fragmented: lineage, definitions, and policies live elsewhere. ## 5\. Altimate MCP Server for Warehouse and Cost Context Altimate MCP adds the cost context that catalog and warehouse servers leave out. - **What it is:** An official Altimate server. It answers cost questions, and lineage questions that cross from dbt into the warehouse. - **Best for:** Teams whose agent must price a query before running it and trace lineage across dbt and the warehouse. - **Key features:** It reports [which queries drove last week's spend](https://altimate.ai/product-highlights/snowflake-query-cost-over-time). It also traces lineage across dbt and the warehouse together. - **Pricing:** Free to install through [the Altimate extension](https://altimate.ai/blog/supercharging-cursor-ide-how-the-dbt-power-user-extensions-embedded-mcp-server-unlocks-ai-driven-dbt-development) or npm. - **Limits:** Altimate's Guardrails classify sensitive fields and set each one to block, mask or allow. A Guardrail decides what data reaches the language model. It has no say over whether the agent may write. A masked email column does nothing to stop an `UPDATE` against the table that holds it. Write permission stays a decision you make in the role grant. Our [Guardrails docs](https://help.altimate.ai/workspaces/user-guide/components/guardrails/) list the block, mask and allow choices. ## 6\. Orchestrator Servers for Airflow and Dagster An orchestrator server tells you whether a job failed or never ran, which is the first thing you need in an incident. - **What it is:** A server that exposes run history, task state and failure logs from Airflow or Dagster. - **Best for:** On-call data engineers who want an agent to read pipeline state during an incident. - **Key features:** It returns run history and failure logs. It reports state per task rather than only for the pipeline as a whole. - **Pricing:** Free and open source where a server exists. - **Limits:** Few maintained servers exist in this category. Astronomer archived its Airflow MCP server. The last push to the community replacement was months ago. No first-party Dagster server exists. Writing a small read-only server against your orchestrator's REST API is a reasonable option. You need only three endpoints, which list runs, fetch task instances and return logs. ## 7\. Git and Code-Host Servers A git server lets an agent read a model's history, which is often the only record of why a filter exists. - **What it is:** A server that exposes git history, so `git log` and `git blame` work from the agent. - **Best for:** Teams where a `WHERE country!= 'XX'` clause looks like a bug until the commit message names the source system that used to emit `XX` for missing country data. - **Key features:** It exposes `git log`, `git blame` and file history. It installs with `uvx mcp-server-git`. - **Pricing:** Free and open source. - **Limits:** The MCP steering group archived its GitHub reference server, so that package receives no patches. Take GitHub's own server instead. ## 8\. Documentation and Knowledge-Base Servers A knowledge-base server gives an agent the write-ups nobody would paste into a prompt. - **What it is:** A server that reads Confluence, Notion or an internal wiki. - **Best for:** Teams whose context lives in write-ups. One incident note explains why the fact table is one row per line. - **Key features:** It searches and reads pages from Confluence, Notion or a wiki, so an agent pulls a write-up instead of guessing. - **Pricing:** Free to install. It reads a knowledge base you already pay for. - **Limits:** A knowledge base holding three competing definitions of active user gives an agent three confident answers. The agent cannot tell which one the finance team uses. ## 9\. Ticketing Servers for Jira and Linear A ticketing server turns a problem an agent found into a tracked issue somebody owns. - **What it is:** A server that files and reads issues in Jira or Linear. - **Best for:** Teams whose agent files a problem with the query text, cost and affected models attached. - **Key features:** It creates issues and reads existing ones, so an agent can check what is already filed before it files another. - **Pricing:** Free to install. It reads a tracker you already pay for. - **Limits:** Keep the write permission narrow, and allow create before you allow close. Creating a ticket is reversible. Closing a ticket asserts that the problem is gone. That is a judgment an agent should not make alone. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## 10\. Filesystem and Local Servers A filesystem server lets an agent read and write project files under the directories you name on the command line. - **What it is:** The reference server that reads and writes files under directories you name. - **Best for:** Any editor agent that needs project files. Scope it to one directory, which is the simplest form of least privilege. - **Key features:** The allowed directories are positional arguments on the command line. The scope is visible in the command you ran, instead of in a settings file edited long ago. A path outside those directories returns an access error. A symlink inside the tree that points outside the tree returns the same error. - **Pricing:** Free and open source. - **Limits:** Write access is on inside those directories, so the agent can overwrite a project file. Widen the scope only when somebody decides to, never by default. ```bash # the reference filesystem server, scoped to one project directory npx -y @modelcontextprotocol/server-filesystem ~/projects/analytics ``` ## Official Servers Arrived in 2026 as Reference Servers Were Archived Every server in this list started as a community project, and most 2025 tutorials still describe them that way. Since then, vendors shipped official servers of their own. dbt Labs shipped [its own MCP server](https://docs.getdbt.com/docs/dbt-ai/about-mcp) alongside the Fusion engine, so that server tracks dbt's object model directly. The MCP steering group went the other way and archived thirteen of its reference servers. For anything outside its maintained list, the reference repository now points you at the MCP Registry. An archived server keeps working on the day it is archived. It stops working the day the upstream API changes, because nobody ships the fix. Until that day nothing warns you that the server is unmaintained. | Reference server | Status in 2026 | Maintained version | | --- | --- | --- | | Filesystem | active | `@modelcontextprotocol/server-filesystem` | | Git | active | `mcp-server-git` | | Fetch, Memory, Time, Sequential Thinking | active | the reference repo | | PostgreSQL, SQLite, Redis | archived | your platform vendor, or the MCP Registry | | GitHub, GitLab | archived | the code host's own server | | Slack | archived | Zencoder took it over | | Sentry, Google Drive, Google Maps, Puppeteer | archived | the MCP Registry | Before you swap to a vendor's official server, confirm it exposes the tools your agents call. Then check whether it assumes a paid tier, because several official servers authenticate against a platform account that your open-source users do not have. An official badge means the vendor wrote the code. It says nothing about how you configured that code. An official server left on default settings still hands the agent its write tools. ## The Security Questions to Ask Before You Install a Server Every server here is a set of capabilities you hand to an agent that acts on its own. Three questions decide the risk. - **What can this write?** A read-only server sits in a different risk class from one that executes SQL or triggers a pipeline. - **Whose credentials does it use?** A server that runs under an admin role gives the agent admin rights. The task the agent was given does not narrow those rights. - **What gets logged?** If you cannot reconstruct what the agent did, you cannot debug it or defend it. The MCP spec settles part of this for you, in three places. The first is the tool annotation. A tool annotation is a label a server attaches to its own tool, such as a flag that says the tool only reads. The spec says a client **MUST** treat those annotations as untrusted unless they come from a server the client already trusts. So a tool that labels itself read-only has proved nothing, because the same server wrote both the tool and the label. The second is token passthrough. Token passthrough means the server takes the credential the client sent and forwards it unchanged to the downstream service, such as your warehouse. The spec names that an anti-pattern. It is the credentials question above answered the wrong way, because the agent then acts with whatever the forwarded credential can do. The third is the human in the loop. The spec says there **SHOULD** always be a human who can deny a tool call before it runs. In July 2025 a Replit AI coding agent deleted a production database during an active code freeze. The agent held standing privileges on the production database. The code freeze was a verbal instruction, with no technical control behind it. The tool worked exactly as configured. | Server type | The write tool to watch | Sensible default | | --- | --- | --- | | Warehouse | none, until you add one | read-only, non-prod role | | Transformation | run models | read-only to start | | Catalog and lineage | update metadata | read-only | | Orchestrator | trigger runs | read-only | | Git | commit | read-only | | Ticketing | create issues | create only | | Filesystem | write files | scoped to one directory | Create a dedicated role before you install anything, because a server borrowing your personal credentials inherits every grant you hold. That usually includes production write access somebody gave you during a past incident and never revoked. Start with a role that reads metadata and nothing else. Point that role at a non-production environment for the first week. A dev schema carries the same table and column names, so the agent learns the same schema. A wrong write there damages a dev table instead of a production one. Turn write tools off explicitly rather than assuming they are off. Snowflake Labs' retired self-hosted server ships them on. Its sample configuration sets `Delete`, `Drop`, `TruncateTable` and `Update` to `True`, leaving only `Unknown` at `False`. Copying that sample configuration lets your agent drop a table. A hardened configuration refuses every destructive statement type: ```yaml # Snowflake MCP service configuration, destructive statements refused other_services: object_manager: false # no CREATE, ALTER or DROP against objects query_manager: true # the agent may still run a SELECT semantic_manager: true sql_statement_permissions: - Select: true - Describe: true - Delete: false - Drop: false - TruncateTable: false - Update: false - Unknown: false # anything sqlglot cannot classify is refused ``` Each key under `sql_statement_permissions` is an expression type from the SQL parser sqlglot. The server parses every statement the agent sends, classifies it, and refuses any type you set to false. The `Unknown: false` entry refuses any statement sqlglot cannot classify. Then watch which tools the agent actually calls for a week. Install the tool the agent tries to call and cannot find. Remove the tools that never fire. Logging is the third question, and [the MCP spec](https://modelcontextprotocol.io) leaves it to you. It defines no audit trail, so record the tool name, the arguments and the calling identity for every call. Without that record you cannot say what the agent did last Tuesday. ## How to Pick the Best MCP Servers for Data Engineering Add a warehouse server first, read-only, on a non-production role. Add a transformation server second for the dependency graph. Then stop and watch for a week before you install anything else from this list. Every server you add is a surface somebody has to review. Check the maintenance status of anything a tutorial recommends. Checking the archive banner and the last commit date catches the archived PostgreSQL server that 2025 tutorials still name. [Nine agentic data engineering tools for VS Code](https://altimate.ai/blog/best-agentic-data-engineering-tools-vs-code) covers the tools that consume these servers. [Nine signs your data platform needs an agent-first overhaul](https://altimate.ai/blog/signs-data-platform-needs-agent-first-overhaul) covers the platform-level picture. Open the config of an MCP server you already run and read its write tools. If you cannot say why each one is on, turn it off. ## Frequently Asked Questions What is Model Context Protocol in one sentence? Model Context Protocol is a standard interface that lets an AI agent call tools on an external system instead of guessing what it holds. Which of the best MCP servers for data engineering should you install first? Install a warehouse server first, read-only, under a non-production role. That removes the guessed-column failure, which is the most common way agent-written SQL breaks. Add a dbt or transformation server next, for the dependency graph. Which MCP servers are archived, and what do you install instead? The MCP steering group archived its reference PostgreSQL, SQLite, Redis, GitHub, GitLab, Slack and Sentry servers. Configuration examples in that repository still name some of them. Filesystem and Git stay active. For anything else, take the vendor's own server or a project on the MCP Registry. Does MCP work with Cursor, Claude Code and VS Code? Yes. [The spec](https://modelcontextprotocol.io) defines two transports, stdio and Streamable HTTP, and any compatible client calls any compliant server over one of them. What should an MCP server never be allowed to do? It should never execute arbitrary SQL against production, as an admin role, with no logging. Each of those three is survivable alone. The combination of all three is how incidents happen. On this page 1. MCP Replaces Per-Agent Wiring With One Server 2. 1\. Warehouse Servers for Snowflake and BigQuery 3. 2\. Postgres and Operational Database Servers 4. 3\. dbt and Transformation-Layer Servers 5. 4\. Lineage and Catalog Servers 6. 5\. Altimate MCP Server for Warehouse and Cost Context 7. 6\. Orchestrator Servers for Airflow and Dagster 8. 7\. Git and Code-Host Servers 9. 8\. Documentation and Knowledge-Base Servers 10. 9\. Ticketing Servers for Jira and Linear 11. 10\. Filesystem and Local Servers 12. Official Servers Arrived in 2026 as Reference Servers Were Archived 13. The Security Questions to Ask Before You Install a Server 14. How to Pick the Best MCP Servers for Data Engineering From the product - [Enterprise Platform Snowflake cost analysis in plain English, with AI that cites every number 5 min read](https://altimate.ai/product-highlights/snowflake-cost-analysis-ai-cited-answers) - [Enterprise Platform Root cause analysis for a dbt, Tableau, or Snowflake workload spike 6 min read](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis) - [PowerUser Plugin for dbt Profile a slow dbt query and read its execution plan in plain English 4 min read](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan) - [Enterprise Platform Reduce Snowflake idle warehouse spend with Altimate Lite 6 min read](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) - [Enterprise Platform Snowflake Cortex AI cost breakdown by service, user, and model 6 min read](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # 6-Step dbt Migration Playbook With an AI Agent URL: https://altimate.ai/blog/dbt-migration-with-an-ai-agent-playbook A six-step playbook for dbt migration with an AI agent that treats business-logic parity as the deliverable. ## A 6-Step Playbook for dbt Migration With an AI Agent A six-step playbook for dbt migration with an AI agent that treats business-logic parity as the deliverable. Atharva Shah 2026-07-15 2026-09-04 11 min read On this page 10 sections 1. Step 1. Inventory Before You Translate Anything 2. Step 2. Decide What Not to Migrate 3. Step 3. Translate the Syntax 4. Step 4. Prove Parity With a Data Diff, Not a Review 5. Step 5. Cut Over One Consumer at a Time 6. Step 6. Decommission the Legacy Pipeline on Evidence, Not Silence 7. Each dbt Migration Step and How to Know It Worked 8. Four Semantic Differences Cause Most Parity Failures 9. Choose dbt Core V1 or V2 10. Where a dbt Migration With an AI Agent Helps and Where It Does Not tl;dr This is a six-step playbook for migrating to dbt with an AI agent. Every step has a command to run and an exit condition that tells you it is finished. An agent earns its place on the two mechanical steps, inventorying every legacy object and translating the SQL dialect, because reading and rewriting SQL is what a model does well. The rest of the migration is not mechanical, and that is where the schedule goes. The step that decides the migration is parity: proving the new pipeline returns the same numbers as the old one. Prove it with a data diff, not a code review, because SQL that compiles only shows the target engine accepts the statement, not that the rows match. Diff a full month, and when the two pipelines sit on different platforms, compare them with bisection hashing so no row leaves its engine. Four causes produce most mismatches: integer division, null handling in aggregates, timezone semantics, and undocumented business filters. Roll out one consumer at a time, internal dashboards first, so a gap surfaces quietly instead of on a customer-facing report. Decommission on evidence, not silence: suspend the old objects for one more business cycle rather than dropping them, so a job nobody documented fails with an error you can undo instead of destroying data you cannot. Target dbt Core v1 today, and run `dbt parse --use-v2-parser` before you treat v2 as an option. A dbt migration plan that budgets the schedule for translating SQL finishes the translation early. The rest of the schedule then goes to parity testing, which means proving the new pipeline returns the same numbers as the old one. That proof is slow because the old pipeline holds years of business decisions nobody wrote down. Each decision has to be found in the old code, then reproduced in the new pipeline or explained away. Start parity testing in week one, against whatever you have translated so far. A horizontal timeline of six steps with a widening risk band above it. Steps one to three are narrow and labeled cheap to redo. Steps four to six are wide and labeled expensive to unwind *Risk concentrates between steps four and six, not at the start.* ## Step 1. Inventory Before You Translate Anything Start a dbt migration with an AI agent by building an inventory of every database object you are moving: tables, views, stored procedures and jobs. Build that list from recollection and you miss the ones nobody thinks of, which are the ones that break the migration later. The inventory has four columns: - The object name. - What reads that object. - The last-read date, meaning when a query last touched it. - Who owns it. The last-read date saves the most work, and you do not have to fill it in by hand. Your data platform already logs it. Snowflake records every object a query read in [ACCESS\_HISTORY](https://docs.snowflake.com/en/sql-reference/account-usage/access_history), which retains 365 days. ```sql -- Snowflake. Every base object, with the last time a query read it. -- An object that returns no row here has not been read in a year. select f.value:objectName::string as object_name, max(a.query_start_time) as last_read, count(*) as reads from snowflake.account_usage.access_history a, lateral flatten(input => a.base_objects_accessed) f group by 1 order by last_read asc; ``` An object missing from this result, or carrying a `last_read` a year old, is a first candidate to cut. Step 2 decides which objects actually go. Hand this step to an agent. Reading hundreds of stored procedures is slow for a person and fast for a model. A misreading here is cheap, because you correct one row of the inventory and carry on. Altimate right-sizes Databricks job clusters, SQL warehouses and all-purpose compute automatically, with no serverless migration required. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) ## Step 2. Decide What Not to Migrate Deciding what not to migrate is the only step that makes the project smaller. Proving that nothing reads an object is harder than migrating it anyway, so scope stays larger than it needs to be. Evidence shrinks the scope back. An opinion about what looks unused does not survive the first objection. Column-level lineage supplies that evidence. Traced at table level, lineage reports every downstream model and tells you to migrate. Traced at column level, lineage names only the reports that actually read the field. [Column-level lineage use cases](https://altimate.ai/blog/column-level-lineage-use-cases) shows how to trace a single field to its downstream consumers. dbt answers the table half of the question on its own: ```bash # Every model downstream of one legacy staging model, as a countable list. dbt ls --select stg_legacy_orders+ --resource-type model --output name ``` `dbt ls` follows the `ref()` links between models, so it lists whole models and never individual columns. Cut an object only when both checks agree: - The last-read date from step 1 shows no reads inside the retention window. - Column-level lineage shows nothing still reading it. Every object you cut is one fewer to translate in step 3 and one fewer to diff in step 4. ## Step 3. Translate the Syntax An agent translates SQL from one dialect to another quickly, so step 3 is one of the shorter steps in a migration. Migration plans still give translation the most room, because it is visible progress while the risk sits later in step 4. Dialect differences are mechanical and most of them have deterministic mappings: | Dialect difference | Why it needs a rewrite | | --- | --- | | Date functions | Names and argument order differ between platforms. | | String concatenation | The concatenation operator differs, so `CONCAT` is the portable form. | | Window syntax | Frame clauses and default frames vary between engines. | | Null handling in aggregates | Nulls sort first on one engine and last on another. | | Division | Integer division truncates on one engine and keeps decimals on another. | Division is the difference most likely to change a number without anyone noticing: ```sql -- Redshift divides two integers and truncates the result. select 7 / 2 as ratio; -- 3 -- Snowflake divides the same two integers and scales the result. select 7 / 2 as ratio; -- 3.500000 ``` Both statements are valid SQL on both platforms. Each platform returns a different value, and neither raises an error. Open-source tooling already handles cross-database SQL translation. [SQLGlot](https://github.com/tobymao/sqlglot) translates between more than 30 dialects, and its README states what that does not cover. > SQLGlot is a transpiler, not a validator. A query that parses successfully may still fail at execution time. A transpiler rewrites a query from one dialect into another. A transpiler answers whether the target engine accepts the statement. It does not answer whether the rows the new query returns match the rows the old query returned. [Altimate Code connects to over a dozen data platforms](https://altimate.ai/blog/introducing-altimate-code): Snowflake, Databricks, BigQuery, Redshift, Postgres, DuckDB, Trino, ClickHouse, MongoDB, MySQL, SQL Server, Oracle and SQLite. Dialect translation covers fewer of those platforms. The documented cross-dialect translation covers Snowflake, BigQuery, Databricks, Redshift, PostgreSQL, MySQL, SQL Server and DuckDB. Our [migration guide](https://help.altimate.ai/code/data-engineering/guides/migration/) publishes one 47-model project end to end. 38 models translated cleanly, 6 needed manual review for `VARIANT` columns, and 3 used Snowflake-only features. That is 81% automatic. Snowflake `STREAMS` map to change data capture and `TASKS` to scheduled queries, which are architecture calls rather than syntax. A funnel showing the object set narrowing across the six steps. It starts at all legacy objects, narrows after the cutting step, then splits at the parity step into rows that match and rows that differ *How the object set narrows across the six steps. Step 4 is where the differences surface.* ## Step 4. Prove Parity With a Data Diff, Not a Review Parity testing runs both pipelines against the same input and compares the results to prove they agree. Scheduled as final QA, after every model is translated, parity testing is the step that slips the schedule. The number of mismatches decides how long it takes, and nobody knows that number until the first diff runs. Leave parity to the end and you learn it with the deadline already fixed. Parity has an exact answer, which is why a review is the wrong tool. A reviewer reads two queries and says they look equivalent. [A data diff](https://altimate.ai/blog/the-correctness-layer-in-ade) is a query that compares the rows the two pipelines produced and reports where they differ. Inside one platform the diff is two set operations: ```sql -- Row-level parity inside one platform. A pass returns zero rows. select 'legacy_only' as side, d.* from ( select * from legacy.fct_orders except select * from target.fct_orders ) d union all select 'target_only' as side, d.* from ( select * from target.fct_orders except select * from legacy.fct_orders ) d; ``` Run an aggregate diff as well. The row-level diff above uses `EXCEPT`, which removes duplicate rows before it compares the two sides. A row that appears twice on one side and once on the other passes that diff, and a `SUM` over the table drifts by one row's value: ```sql -- The check a row-level diff passes straight through. with legacy_monthly as ( select date_trunc('month', order_date) as month, sum(net_amount) as total from legacy.fct_orders group by 1 ), target_monthly as ( select date_trunc('month', order_date) as month, sum(net_amount) as total from target.fct_orders group by 1 ) select coalesce(l.month, t.month) as month, l.total as legacy_total, t.total as target_total, l.total - t.total as delta from legacy_monthly l full outer join target_monthly t on l.month = t.month where l.total is distinct from t.total; ``` `EXCEPT` only works when both tables sit on one engine. In most migrations they do not, because the old pipeline runs on the platform you are leaving and the new one on the platform you are moving to. Bisection hashing is the diff for that case, and it compares checksums instead of rows. It splits both tables into blocks and hashes each block on its own engine, so no row leaves its platform. It then repeats on only the blocks whose hashes disagree. You compare a 100M-row table by moving checksums, and the rows stay where they are. Note Altimate Code's diff covers twelve data platforms in any combination, so a table on one engine compares against a table on another. The diff can print up to 5 sample rows into its output, and your model provider sees those rows. On regulated tables, use profile mode instead, which compares column statistics and moves no row values. Our [data parity guide](https://help.altimate.ai/code/data-engineering/guides/data-parity/) names the partition modes. A high mismatch count does not always mean the migration is wrong, so read what the differences are before you act on the count. One published SQL Server to Microsoft Fabric run shows why: - **What ran.** A row-level diff ran on the `fact_sales` table. - **What came back.** The diff returned 92 differing rows. - **Why it passed.** All 92 differed by trailing decimal zeros alone, because the staging layer cast to a fixed scale. The values matched, so the table passed. When an agent rewrites a query, [a second model reviewing that query](https://altimate.ai/blog/why-ai-coding-agents-fail-at-data-engineering) is no substitute for a diff. We covered that on our DataCamp panel about [what AI agents are really changing in data engineering](https://altimate.ai/blog/what-ai-agents-are-really-changing-in-data-engineering). Altimate right-sizes Databricks job clusters, SQL warehouses and all-purpose compute automatically, with no serverless migration required. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) ## Step 5. Cut Over One Consumer at a Time Cutting over means pointing each consumer at the new pipeline. A consumer is any dashboard, report or job that reads your data. Move the consumers across one at a time. Start with the consumers whose owners sit close to the data team, because those owners notice a wrong number quickly and report it privately. In practice that means internal dashboards before customer-facing reports. Warning Switching every consumer on the same day turns a small parity gap into a public one. A gap in one revenue model then surfaces as a wrong number on the board's dashboard. Caught one consumer at a time, the same gap is a quiet fix on an internal dashboard, before anyone outside the data team sees it. Keep the diff running on a schedule while both pipelines are live, rather than as a one-time gate. A model that matched once stops matching when a source changes shape. While both pipelines run, you can see which side moved. Once the old pipeline is off, nothing is left to compare against. ## Step 6. Decommission the Legacy Pipeline on Evidence, Not Silence Turn the legacy pipeline off on evidence that nothing depends on it. A quiet week proves nothing, because a quarterly job does not run in a quiet week. Three things have to hold before you decommission: - Lineage shows nothing reads the legacy objects. - The diff has come back clean through a full business cycle. - The last-read date shows no unexpected recent activity. Then suspend the objects rather than dropping them, for one more cycle: ```sql -- Reversible. A surprise consumer raises an error instead of losing data. alter task legacy_db.public.load_orders_daily suspend; alter view legacy_db.public.v_orders rename to v_orders_retired; ``` A drop destroys the data. A suspend leaves the data in place, so the extra cycle costs only calendar time. A quarterly job nobody documented then fails with a clear error instead of quietly returning nothing. You undo a suspend by resuming the task and renaming the view back. Once a full cycle passes with no surprises, drop the objects and archive the step 1 inventory. ## Each dbt Migration Step and How to Know It Worked | Step | Run this | What proves it worked | Agent value | Human decision | | --- | --- | --- | --- | --- | | 1\. Inventory | the `ACCESS_HISTORY` query above | every object has an owner or an out-of-scope note | ✓ reads and summarizes hundreds of objects | scope and ownership | | 2\. Decide what to cut | `dbt ls --select +`, then a lineage traversal | each cut object shows zero live consumers | ≈ runs the lineage queries | the risk appetite for deleting | | 3\. Translate | `dbt parse`, then `dbt compile` | every model in scope compiles and runs | ✓ this is the mechanical part | the target platform and version | | 4\. Prove parity | the row diff and the aggregate diff, on a full month | zero rows back, or one signed-off reason per mismatch | ≈ explains a mismatch once found | whether a mismatch is acceptable | | 5\. Cut over | both pipelines live, diff scheduled daily | the diff stays clean through a month end | ✗ this is coordination | the order of consumers | | 6\. Decommission | `alter task... suspend` | one full cycle of silence | ✗ the agent does not make this call | the decision to drop | ## Four Semantic Differences Cause Most Parity Failures Four causes account for most parity failures, so you can work step 4 as a checklist rather than an open investigation. | Cause | Why it drifts | How you catch it | | --- | --- | --- | | Implicit type coercion | One engine truncates integer division and another keeps decimals, so a ratio changes value and shows up as a rounding difference three models downstream. | The aggregate diff on any `SUM` that depends on the ratio. | | Null handling in aggregates | Nulls sort first on one engine and last on another, so a query that keeps one row per group keeps a different row. | The row-level diff, which returns different rows when the sort order flips. | | Timezone and timestamp semantics | A timestamp with no timezone means different things per platform, and a session timezone setting shifts results with no code change. | The aggregate diff run across a month end, where the boundary rows move. | | Undocumented business filters | The legacy pipeline hides filters nobody remembers, and cleaning up the translation drops them. | The row-level diff, which returns the extra rows the cleanup let through. | Null handling drifts on sort order. Engines disagree on whether a null sorts before or after every other value. That disagreement matters in a query that keeps one row per group. A common shape is `row_number() over (partition by customer_id order by updated_at)` where `updated_at` holds nulls. Each engine keeps a different row for those groups, and the row-level diff catches it. Timezone drift starts with a timestamp column that carries no timezone. Each platform decides for itself which clock that value is on. One platform reads it as UTC and another reads it in the session timezone, and a session setting shifts every result with no change to the code. Take an order placed late on the last day of the month. A `date_trunc('month', order_date)` moves that order into the next month on one platform and leaves it in place on the other. The two monthly totals differ by exactly the orders in that boundary window. The aggregate diff run across a month end catches it, because those boundary rows move. Undocumented filters are the hardest of the four causes to spot, because nothing marks them. A legacy pipeline hides three kinds: - A `WHERE` clause that excludes test accounts. - A hardcoded date cutoff nobody remembers setting. - A customer ID excluded after an incident. An agent translating faithfully keeps all three filters, which is correct. The numbers break when somebody later tidies the translation and removes them. Diff a full month rather than a sample, because timezone boundaries and hidden date cutoffs only show up across a month end. [Workload migration](https://altimate.ai/use-cases/workload-migration) covers the same ground as a use case. A pie chart of parity mismatches by cause, with four labeled wedges for implicit type coercion, null handling in aggregates, timezone semantics and undocumented business filters *The four causes of parity failures. Each one changes a number while both engines run the query without an error.* ## Choose dbt Core V1 or V2 As of August 2026, target dbt Core v1. dbt Labs published the first alpha of dbt Core v2.0 on 1 June 2026. The [launch post](https://docs.getdbt.com/blog/dbt-core-v2-is-here) describes two free distributions of the v2 engine. One is Fusion, a binary that contains some proprietary code, and the other is the Apache 2.0 code in the `dbt-core` repo. Advanced SQL comprehension, linting and column-level lineage ship in Fusion only. The [v2 roadmap](https://github.com/dbt-labs/dbt-core/blob/main/docs/roadmap/2026-06-announcing-v2.md) says Core v1 "is not going away tomorrow, or any time soon." dbt Core v1.12 ships the v2 parser behind a flag: ```bash # Run from dbt Core v1.12. If the project parses, v2 becomes a real option. dbt parse --use-v2-parser ``` If your project parses under that flag, v2 is a realistic target. If it fails, the errors name the stricter rules you would have to fix first, so you learn the size of the job early. The [`dbt-autofix`](https://github.com/dbt-labs/dbt-autofix) package handles much of the mechanical part. Migrating to v1 now and v2 later is two migrations, so pick one and write down why. ## Where a dbt Migration With an AI Agent Helps and Where It Does Not An agent earns its place in the step 1 inventory and the step 3 syntax translation. Steps 4 through 6 are where migrations fail. Those steps need a diff and a human decision, and more generated code does not help there. Run the inventory and the translation unattended, then gate parity on a diff before anything moves. The broader argument for that split is [why deterministic tooling beats LLM-only data agents](https://altimate.ai/blog/deterministic-tooling-vs-llm-only-data-agents). Run an aggregate diff against the models you have already translated, before you translate the next one. ## Frequently Asked Questions How long does a dbt migration with an AI agent take? The translation work is fast, because reading and rewriting SQL is what a model is good at. Proving parity takes nearly as long as it did before agents. You wait for the comparisons to run and for people to sign off on the differences. Plan for the second half to dominate the schedule. Why does a dbt migration stall? Business-logic parity takes the time. The old pipeline encodes years of decisions nobody documented. Proving that the new pipeline reproduces each one is slow work. Teams that run a diff from week one of a dbt migration with an AI agent find the mismatches earlier. How do I run data migration parity testing across two different platforms? `EXCEPT` and `FULL OUTER JOIN` only work when both sides live on one engine. Bisection hashing works across two engines. It splits both tables into blocks, hashes each block on its own engine, and repeats only where the hashes disagree. Only the checksums move, so the rows stay put. Can an AI agent handle stored procedure to dbt conversion? An agent handles the mechanical translation well, including control flow rewritten as models and tests. An agent cannot tell you whether a procedure's behavior was intentional or a bug somebody worked around downstream. That question needs a diff and a person. Do we still need a data diff if the translated SQL compiles? Yes. Compiling proves the target engine accepts the syntax. It says nothing about whether the numbers match. Skipping the diff is the most common false economy in a migration. [dbt's built-in tests](https://docs.getdbt.com) check for nulls, uniqueness and accepted values, and none of them compares the new numbers with the old ones. On this page 1. Step 1. Inventory Before You Translate Anything 2. Step 2. Decide What Not to Migrate 3. Step 3. Translate the Syntax 4. Step 4. Prove Parity With a Data Diff, Not a Review 5. Step 5. Cut Over One Consumer at a Time 6. Step 6. Decommission the Legacy Pipeline on Evidence, Not Silence 7. Each dbt Migration Step and How to Know It Worked 8. Four Semantic Differences Cause Most Parity Failures 9. Choose dbt Core V1 or V2 10. Where a dbt Migration With an AI Agent Helps and Where It Does Not From the product - [PowerUser Plugin for dbt Profile a slow dbt query and read its execution plan in plain English 4 min read](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan) - [Enterprise Platform Root cause analysis for a dbt, Tableau, or Snowflake workload spike 6 min read](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis) - [Enterprise Platform Attribute Snowflake cost to the team that owns it 4 min read](https://altimate.ai/product-highlights/snowflake-cost-alerts-by-team) - [Enterprise Platform Snowflake cost analysis in plain English, with AI that cites every number 5 min read](https://altimate.ai/product-highlights/snowflake-cost-analysis-ai-cited-answers) - [Enterprise Platform Reduce Snowflake idle warehouse spend with Altimate Lite 6 min read](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Deterministic Tooling for AI Agents vs LLM-Only URL: https://altimate.ai/blog/deterministic-tooling-vs-llm-only-data-agents Some questions about your data have exact answers. Five reasons deterministic tooling for AI agents beats a model on those, and where the model still wins. ## 5 Reasons Deterministic Tooling for AI Agents Beats an LLM-Only Stack Some questions about your data have exact answers. Five reasons deterministic tooling for AI agents beats a model on those, and where the model still wins. Atharva Shah 2026-07-08 2026-09-04 10 min read On this page 8 sections 1. The Two Layers Behind Deterministic Tooling for AI Agents 2. Reason 1. Compiled Checks Cost Nothing Per Run 3. Reason 2. Models Are Right on Average and Wrong in Ways You Cannot Predict 4. Reason 3. Sub-Millisecond Checks Run While You Type 5. Reason 4. Deterministic Checks Produce a Trace You Can Defend 6. Reason 5. Tools Compose Across Teams and Prompts Do Not 7. Ask Whether Two Engineers Could Disagree 8. How to Wire the Layers in the Right Order tl;dr Deterministic tooling for AI agents splits a data agent into two layers. Compiled checks answer every question that has one correct answer, and the model handles every question that needs judgment. Ask a model a question of the first kind and you get a confident estimate instead of the answer. A check that is right 95% of the time wrongly passes one change in twenty, and a reviewer who catches that stops trusting the check. The benchmark evidence points the same way. On DataAgentBench, Claude Sonnet 4.6 and DeepSeek v4 pro scored 3.5 points apart on Pass@1, but the score hides the trials where the model wrote nothing: 32 of 270 for Sonnet and 79 of 270 for DeepSeek. Wire the compiled checks underneath the model and let the model call them, so the agent catches errors it would otherwise ship. An AI agent writes you a dbt model. The dbt model compiles, it runs and it returns numbers. None of those three steps tells you whether the numbers are right. Some parts of that question have exactly one right answer. Three of those parts already have a tool that answers them exactly: - A row-level diff tells you whether a rewritten query returns the same rows as the query it replaced. - A pattern scan tells you whether a column holds email addresses. - [Column-level lineage](https://altimate.ai/blog/column-level-lineage-use-cases) names every model that breaks when you rename a field. Each of those three tools gives the same answer on every run. A language model can answer those same three questions, and its answer moves. Change the phrasing of the request and the answer can change. Deploy a new model version and the answer can change again. On a question with one fixed answer, an answer that moves is a defect, because you cannot tell which run to trust. Deterministic tooling for AI agents means compiled code that returns the same answer for the same input, on every run. A data agent built on that code has two layers. Compiled checks own every question with one correct answer. The model owns every question where judgment decides. A two-layer diagram. The lower layer is a solid box labeled compiled checks, holding lint, schema validation, data diff and PII scan. The upper layer is a dashed cloud labeled model, holding intent, ambiguity and cross-domain reasoning. Arrows show the model calling into the lower layer, never the reverse *The two layers of a data agent. Compiled logic runs the checks underneath, and the model supplies judgment on top.* Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## The Two Layers Behind Deterministic Tooling for AI Agents A working data agent runs two layers, and each layer fails differently. The lower layer is compiled code. It: - parses SQL, - validates a schema, - compares two result sets, - scans for personal data, and - walks a dependency graph. Each of those five jobs has a correct answer that does not change with the phrasing, the temperature or the model version you deployed. The upper layer is the model. Its job starts where the exact answers run out. It reads a request like "check whether my refactor broke anything" and works out which tools answer that request. Then it calls those tools in order and explains the result. No fixed rule could cover those steps, because the right tools depend on what the person meant. A model asked to do the lower layer's job produces answers that are usually right. On a question with one correct answer, usually right still means wrong some of the time. | Check | Deterministic layer | LLM layer | What happens when it is wrong | | --- | --- | --- | --- | | Do these two queries return the same rows? | a row-level diff, exact | a summary of a diff it never ran | a migration ships a silent 0.4% row loss | | Does this column hold email addresses? | a pattern scan over sampled values | a guess from the column name | personal data lands in an ungoverned table | | What breaks if I drop this field? | a column-level lineage traversal | a list built from remembered names | a dashboard breaks downstream | | Is this SQL a known anti-pattern? | 19 named rules, each with a version | a confident paragraph | the reviewer stops trusting the gate | | Is this dbt model name clear? | no rule decides it | the model reads the surrounding project | a rule encodes one person's taste as law | Every failure in the last column happens silently. Your pipeline already catches the loud failures, such as a query that does not compile, and a loud failure costs one retry. A silent failure looks like success. A model that summarizes a diff it never ran returns a clean report, and the migration ships with the row loss inside it. ## Reason 1. Compiled Checks Cost Nothing Per Run A model check costs tokens on every call, so the bill scales with how often you check. A compiled check costs the same on the thousandth run as on the first, and that cost is close to zero. The cost per run decides how often you can afford to check. How often you check decides how early you catch a mistake. A lint you pay for per call runs once, at the end, against the final diff. That is the only placement a budget justifies. A free lint runs on every keystroke, on every save and inside the agent loop. It catches the problem while the author still remembers what they meant. Altimate's Rust core runs fast enough that placement stops being a decision: - **Linting is free per run.** A SQL lint costs 0.48 ms per query, so it runs on every keystroke without a budget conversation. - **Validation is nearly free too.** A schema check costs 2 ms per model, so a full pass over a 600-model project finishes in 1.2 seconds. ## Reason 2. Models Are Right on Average and Wrong in Ways You Cannot Predict An accuracy number tells you how often a system is right. It does not tell you which cases the system gets wrong. Suppose a model classifies SQL anti-patterns correctly 95% of the time. That model is wrong on one query in twenty. You cannot predict which queries get the wrong answer, and the errors do not spread evenly across easy queries and hard ones. Run that rate across a 600-model project and one pass misjudges thirty models. Neither the false alarms nor the silent passes announce themselves, so the reviewer has to check the checker. A reviewer who finds bad SQL behind a green check stops trusting the check. A check nobody trusts gets skipped, and a skipped check catches nothing. Altimate publishes [19 anti-pattern rules](https://altimate.ai/blog/dbt-anti-patterns-ai-should-catch) at 100% accuracy across 1,077 benchmark queries, with zero false positives. A rule set can reach 100% on a bounded problem because the rule is the definition of the pattern it looks for. A query either matches the definition or misses it. A model has no definition to match against. It estimates whether the query looks like the pattern, and an estimate is wrong some of the time. The same unpredictability shows up when a model does the whole job. One agent ran 270 [DataAgentBench](https://altimate.ai/benchmarks/dab-bench) trials on each of two models, [Claude Sonnet 4.6 and DeepSeek v4 pro](https://altimate.ai/blog/deepseek-vs-claude-sonnet-ai-agent-benchmark). The run reported stratified Pass@1, which is the share of tasks solved on the first attempt, balanced across task types. | Measure | Claude Sonnet 4.6 | DeepSeek v4 pro | | --- | --- | --- | | Stratified Pass@1 | 60.4% | 56.9% | | Trials that returned nothing | 32 of 270 | 79 of 270 | | Cost per trial | $0.76 | $0.29 | The two Pass@1 scores sit 3.5 points apart, which reads as a close race. The score hides a second number, the count of trials where the model wrote nothing at all. DeepSeek v4 pro returned nothing on 79 of 270 trials. Claude Sonnet 4.6, the better model on the headline score, still returned nothing on 32 of 270 trials, which is an 11.9% miss. A headline score folds those empty runs into the failures and never reports them on their own. For a gate, an empty run means the change went through unchecked. A gate built on the model layer alone would leave roughly one dbt model in eight unchecked. On the next run it would leave a different one in eight unchecked, so rerunning the gate does not close the gap. A second benchmark, ADE-Bench, scores the whole agent against real dbt projects, with the compiled tools and the model together. That combined run reaches [74.4%](https://altimate.ai/benchmarks/ade-bench). The model layer decides that ceiling. The compiled layer's job is to catch the quarter the model missed and report it. Two panels. The left panel plots compiled-layer results as a flat line at 100% across 1,077 cases with a zero-variance band. The right panel plots model-layer results as a wide scatter, marking 60.4% and 56.9%, with bars for empty runs at 32 of 270 and 79 of 270 *The same system measured at both layers. The left panel shows no variance because the compiled layer has none.* Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## Reason 3. Sub-Millisecond Checks Run While You Type Latency decides where a check can run. A check taking two seconds cannot run while somebody types, so it moves to save, then commit, then CI. Each move puts distance between the mistake and the feedback. Compiled validation at half a millisecond runs inside the editor while you write, so the underline appears at the cursor and the fix costs one keystroke. The same check through a model call returns after you move to the next file. An agent that validates its own work between steps needs each check cheap in wall-clock time as well as in money. A two-second check after each of twenty steps adds forty seconds of waiting to one task, and nobody waits for that. An execution plan is the same kind of exact answer, and our write-up on [profiling a slow dbt query](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan) walks through reading one. ## Reason 4. Deterministic Checks Produce a Trace You Can Defend A regulated team has to answer one question after the fact. Why was this change allowed? A rule answers that question with a name and a version. Rule 12 passed and rule 14 failed, on this commit. A model answers the same question with a different paragraph each time, because nothing pins the wording. A different paragraph is fine in a summary somebody reads once. It is useless as evidence. An auditor compares the record from today with an earlier one, and the two have to match. ## Reason 5. Tools Compose Across Teams and Prompts Do Not A prompt that works is a local artifact. Its behavior depends on the model version it was tuned against. It also depends on the exact phrasing somebody reached after repeated attempts, and often on examples from one team's project. Copy that prompt to another team and it keeps producing output. The output gets worse, and nothing reports the change. A tool has a signature instead of a phrasing. It accepts typed arguments, it returns a structured result, and it behaves the same for every caller. That signature turns a check into shared infrastructure rather than one engineer's private prompt. [The Model Context Protocol](https://altimate.ai/blog/best-mcp-servers-for-data-engineers) gives tools a standard interface. One lineage lookup written against that interface serves a CLI agent, an editor agent and a chat assistant, with one implementation instead of three. Altimate ships 21 skills in [its open-source repo](https://github.com/AltimateAI/altimate-code) behind that interface. A standard interface does not make a tool trustworthy on its own. An annotation is a server's own description of what its tool does. The protocol tells a client not to believe an annotation, in its section on [tool annotations](https://modelcontextprotocol.io/specification/2026-07-28/server/tools): > For trust & safety and security, clients **MUST** consider tool annotations to be untrusted unless they come from trusted servers. What a client can rely on instead is the tool's behavior. Run the tool against a known test input and compare the result with the answer you expected. ## Ask Whether Two Engineers Could Disagree One question sorts almost every case: could two competent engineers, given the same data, disagree? If they could not disagree, the question belongs in the compiled layer. The row diff, the email scan and the lineage lookup from the table above all pass that test. Two engineers who disagree on any of those three are reading different data. If they could reasonably disagree, the question belongs to the model or to a person. dbt model naming, grain and where a transformation should live are all questions of that kind. A rule that picks one side encodes somebody's preference as law. People argue with that rule, then disable it. Three jobs belong to the model alone. Atlan's published analysis of agent failures reaches the same division of labor between the model and the compiled checks. - The model reads a vague request and decides what the person actually wants. - The model weighs a tradeoff where two valid answers exist and the choice depends on context. - The model explains a technical result to somebody who did not ask a technical question. A third case sits between the two. Some questions have an exact answer that costs too much to compute on every run. A full row-level diff of a billion-row table is one example. Those questions still belong in the compiled layer, run on a schedule or against a sample. ## How to Wire the Layers in the Right Order | Layer | Job | Right answer exists | Cost per run | | --- | --- | --- | --- | | Compiled core | parse, validate, diff, scan, walk the graph | ✓ | none | | Tool interface | expose the core to any agent | n/a | none | | Model | interpret intent, choose tools, explain | ✗ | per token | | Human gate | decide when the model is unsure | ✗ | attention | The order these layers run in decides the outcome more than the choice of components does. When the model calls the compiled checks before it answers, it sees its own errors and can fix them. When the compiled checks run after the model has produced its output, they can only report the errors. Our [agent modes reference](https://help.altimate.ai/code/data-engineering/agent-modes/) is one implementation of that ordering. A team that already owns a general-purpose agent is missing the compiled core and the tool interface underneath it. [Ten ways AI coding agents fail at data engineering](https://altimate.ai/blog/why-ai-coding-agents-fail-at-data-engineering) walks through that gap one failure at a time. [Nine signs your data platform needs an agent-first overhaul](https://altimate.ai/blog/signs-data-platform-needs-agent-first-overhaul) covers the same gap at the platform level. Start with one change. Pick the check your team runs by hand most often, and move it under the agent. ## Frequently Asked Questions What does deterministic tooling for AI agents mean? A deterministic check gives the same answer every time for the same input, because it is compiled logic rather than inference. Parsing SQL, comparing two result sets and walking a dependency graph are all deterministic. Whether a dbt model name is clear depends on who you ask, so no compiled check settles it. Does deterministic tooling beat an LLM for SQL? On any SQL question with one right answer, yes. A compiled check returns the same answer every time. A model estimates, so it is wrong on a small set of cases you cannot predict. Keep the model for judgment calls, such as naming and grain. Should I stop using an LLM agent for data work? Keep using one, and put compiled checks underneath it. The model is good at reading a vague request and choosing what to do. It is unreliable at questions with exact answers. Those questions are most of what decides whether a pipeline is correct. Do I get SQL comprehension free with dbt Core v2.0? The Apache 2.0 build in the dbt-core repo leaves SQL comprehension out. Advanced SQL comprehension, linting and column-level lineage ship only in the Fusion binary, which is free and holds some proprietary code. How do I add deterministic tooling for AI agents without rebuilding my stack? Add one deterministic check to the path your agent already uses. Pick the check whose failure has cost you money. Usually that is a data diff on rewritten queries or a lineage check before merge. On this page 1. The Two Layers Behind Deterministic Tooling for AI Agents 2. Reason 1. Compiled Checks Cost Nothing Per Run 3. Reason 2. Models Are Right on Average and Wrong in Ways You Cannot Predict 4. Reason 3. Sub-Millisecond Checks Run While You Type 5. Reason 4. Deterministic Checks Produce a Trace You Can Defend 6. Reason 5. Tools Compose Across Teams and Prompts Do Not 7. Ask Whether Two Engineers Could Disagree 8. How to Wire the Layers in the Right Order From the product - [PowerUser Plugin for dbt Profile a slow dbt query and read its execution plan in plain English 4 min read](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan) - [Enterprise Platform Track Snowflake query cost over time, run by run 6 min read](https://altimate.ai/product-highlights/snowflake-query-cost-over-time) - [Enterprise Platform Root cause analysis for a dbt, Tableau, or Snowflake workload spike 6 min read](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis) - [Enterprise Platform Reduce Snowflake idle warehouse spend with Altimate Lite 6 min read](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) - [Enterprise Platform Snowflake Cortex AI cost breakdown by service, user, and model 6 min read](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Altimate Code vs Claude Code vs Cursor Compared URL: https://altimate.ai/blog/altimate-code-vs-claude-code-vs-cursor Altimate Code vs Claude Code vs Cursor vs Cortex Code CLI. What each agent does for a data team, what it costs, and which one fits your stack. ## Altimate Code vs Claude Code vs Cursor vs Cortex Code CLI Altimate Code vs Claude Code vs Cursor vs Cortex Code CLI. What each agent does for a data team, what it costs, and which one fits your stack. Atharva Shah 2026-07-22 2026-09-04 11 min read On this page 9 sections 1. Altimate Code 2. Claude Code 3. Cursor 4. Cortex Code CLI 5. Altimate Code vs Claude Code vs Cursor vs Cortex Code CLI, Feature by Feature 6. How Each Agent Handles a Column Rename 7. The Three Costs That Sit Above the License Price 8. Your Model Choice Moves the Score More Than Your Agent Choice 9. Which Agent to Pick for Your Stack tl;dr Claude Code and Cursor are general coding agents, and Cortex Code CLI is Snowflake's agent that starts inside the warehouse. Altimate Code is not a fourth agent. It is an open-source data layer that: - runs beside whichever agent you already use, - reads live schema from over a dozen data platforms, - [traces column-level lineage](https://altimate.ai/blog/column-level-lineage-use-cases), and - blocks 19 SQL anti-patterns before a merge. The four differ less in the models they run than in what each one reads before it writes SQL, so pick the agent by the shape of your stack and add the data layer where your agent cannot see. Cursor remains the best editor of the four, which is why most teams keep it and add Altimate Code beside it. On benchmarks, your model choice moves the score more than your agent choice does: Snowflake held its agent fixed, swapped only the model, and measured a 17-point spread in first-attempt pass rate. Your dbt project is more than a folder of SQL files. It has a DAG, a test suite, column-level dependencies and a live warehouse behind it. A general coding agent reads the one SQL file it was asked to edit and none of the rest. The SQL it writes from that one file looks plausible, and a column name or a join in it can still be wrong. Wrong SQL of that kind produces no visible failure. It compiles, it runs, and it returns numbers that look reasonable. Not one step in that chain raises an error, so you find out the numbers are wrong only when somebody reads them. Altimate Code vs Claude Code comes down to what each agent reads before it writes, and the same test separates all four tools. Each one runs a frontier model, so the model behind the agent does not separate them. What separates them is the context each one holds when it writes SQL. Altimate Code and Cortex Code CLI read your warehouse schema before they write. Claude Code and Cursor write from the repository first and leave the checking to you. The Altimate Code panel beside a customers.sql dbt model in VS Code, with CodeLens actions on each CTE and a four-step agent plan showing completed read, glob and bash tool calls *Altimate Code running beside a dbt model in VS Code. The agent reads the project and the warehouse before it proposes an edit.* A five-row comparison grid of the four agents, marked with ticks, crosses and partial marks, with the Altimate Code column highlighted *Four agents against the work a data engineer actually does. Only Altimate Code and Cortex Code CLI reach the data rows, and Cursor leads on the editing row.* Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## Altimate Code Altimate Code adds a data layer under whichever editor or terminal agent you already pay for. It does not replace that agent. **What it is.** Altimate Code is an open-source set of data engineering tools built for coding agents. It ships as a CLI, a terminal UI and a VS Code extension. Its checks are compiled Rust code rather than model calls, so they spend no tokens and return the same result every time. **Who it is for.** It suits data engineers and analytics engineers who spend more time proving a change is safe than writing it. It is stack-agnostic, so one install connects to every data platform and database you run. **Who uses it today.** Altimate runs in production at Elastic, Booking.com, Carta and Coinbase. **What Altimate Code does well** - **Reads live warehouse metadata at call time.** Column names and types come from the warehouse rather than from a string search over the repository. - **Connects to any data platform or cloud database.** Native drivers cover Snowflake, Databricks, BigQuery, Redshift, Postgres, DuckDB, Trino, ClickHouse, MongoDB, MySQL, SQL Server, Oracle and SQLite. - **Blocks 19 SQL anti-patterns before a merge.** The rules scored 100% accuracy across 1,077 benchmark queries with zero false positives. The lint runs in about 0.48 ms per query. - **Traces column-level lineage.** An edge is one link from a column to a column that reads from it. Altimate Code matched 100% of edges across 500 queries containing joins, CTEs and subqueries. It can name every consumer of a column before you rename it. - **Diffs data across different engines.** You can compare a 100M-row table on one platform against its replica on another. Neither table leaves its platform. Any two connected platforms work. - **Reviews dbt pull requests deterministically.** Only the compiled checks can block a merge. Anything the model suggests is advisory. - **Runs on any model you already buy.** Altimate Code works with the major LLM providers, including Anthropic, OpenAI, Google, Amazon Bedrock, Azure, Mistral, Cohere, Groq and Snowflake Cortex. - **Records every session as a replayable trace.** The recap shows the prompt, the files changed, the commands run and the cost. The Altimate Recap trace viewer, showing the session model and status, a 4m44s duration, 43 tool calls, 37 LLM calls, the prompt and the file that changed **Where it falls short.** Altimate Code edits data code only. A change that also touches your application services still needs Cursor or Claude Code beside it. Cursor also applies a multi-file edit faster than Altimate Code does. **What it costs.** - The Community tier is free and grants 10 million tokens once. - Pro costs $29 per seat per month and includes 20 million tokens, with overage at $5 per million. - The code is MIT licensed, so with your own model key the tooling costs nothing. See [the pricing page](https://altimate.ai/pricing) and [the quickstart](https://help.altimate.ai/code/getting-started/quickstart-new/). ## Claude Code Claude Code handles long multi-step work better than the other three tools here. It becomes useful on data work once you wire data tools into it. **What it is.** Claude Code is Anthropic's coding agent. It runs in the terminal or in an IDE, and it reads a large codebase before it acts. **Who it is for.** It suits software teams whose work spans application code, infrastructure and data. An MCP server gives an agent one tool, such as a schema lookup, over a standard interface. A data team can assemble its own tooling around Claude Code from those servers. **What Claude Code does well** - **Handles long multi-step tasks.** It plans, executes and revises across many files in a single session. - **Ships as part of the Anthropic ecosystem.** Anthropic builds the agent and the Claude models it runs, so new models and agent features land here first. The agent is also included in Anthropic's paid Claude plans. - **Hosts MCP servers.** Give it a warehouse server and a dbt server, and it chains a schema lookup into a lineage query unprompted. - **Pairs with Altimate Code directly.** The `/configure-claude` command registers an `/altimate` command inside Claude Code, so Claude Code can call Altimate Code as a tool. **Where it falls short.** Claude Code ships without a data layer. Out of the box it cannot read a dbt DAG, track a warehouse cost or apply an anti-pattern rule. Each of those needs an MCP server that you connect. Every MCP server you add is a moving part you own and maintain. Our head-to-head detail is in [Claude Code or Altimate Code for data engineering](https://altimate.ai/blog/claude-code-or-altimate-code-for-data-engineering). **What it costs.** Anthropic includes Claude Code with its paid Claude plans, and it also runs on pay-as-you-go API billing. Your bill scales with tokens rather than with seats. ## Cursor Cursor has the strongest editing experience of the four. The editing lead matters most when an engineer reads the generated code and edits it by hand. Many teams now let the model generate the code and let the tests judge it, which narrows Cursor's lead. **What it is.** Cursor is an AI-first editor built on VS Code. It does inline editing, multi-file changes and strong autocomplete. **What Cursor does well** - **Edits across many files fastest.** Inline diffs and multi-file rewrites land ahead of the other three tools. - **Completes from your repository.** It learns your naming conventions and matches them closely. - **Accepts MCP servers.** Cursor does not read your warehouse on its own. A schema MCP server that you wire in closes part of that gap. **Where it falls short.** Cursor completes from the text in your repository, so a column name in its generated SQL is plausible rather than checked against the warehouse. It does not trace column lineage, apply anti-pattern rules, or estimate query cost before it runs. **What it costs.** Cursor lists a free Hobby tier, Pro at $20 per month and Teams at $40 per user per month. On seat price, Cursor is the cheapest of the four. ## Cortex Code CLI Cortex Code CLI is Snowflake's own agent, and it is the only one of the four that starts inside the warehouse. **What it is.** Cortex Code CLI is a coding agent that starts with your Snowflake schema, query history and role-based permissions already loaded. **Who it is for.** It suits teams whose estate is entirely or mostly Snowflake and who want the least assembly work. **What Cortex Code CLI does well** - **Reads Snowflake natively.** Schema, query history and grants need no connector. - **Executes SQL inside your Snowflake account.** [Snowflake reports](https://www.snowflake.com/en/blog/cortex-code-cli-expands-support/) that running inside the account cut total calls by nearly half against Claude Code on the same tasks, with 2x fewer file reads and 4x fewer bash commands. - **Reads dbt and Apache Airflow.** In 2026 Snowflake began expanding it beyond the warehouse, starting with dbt and Airflow support. **Where it falls short.** Its deepest context is still Snowflake. On a mixed estate it cannot compare a Postgres table against a Databricks table. It publishes no anti-pattern rule set. **What it costs.** Teams already on Snowflake pay in credits, so the agent's cost lands inside the Snowflake bill with no separate line to control. Everyone else buys a self-service monthly subscription. A two-band stack diagram with the four coding agents above and the data domain layer below *Claude Code and Cursor sit in the upper band only. Cortex Code CLI reaches part of the lower band, and Altimate Code covers all of it.* Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## Altimate Code vs Claude Code vs Cursor vs Cortex Code CLI, Feature by Feature All four tools run comparable frontier models, so the differences that matter are in what each one can reach inside your stack. A skill is a workflow playbook the agent loads when it is relevant. Altimate Code ships 21 skills as open source. Snowflake supplies a skill set for Cortex Code CLI, and on Claude Code and Cursor you assemble your own. | | Altimate Code | Claude Code | Cursor | Cortex Code CLI | | --- | --- | --- | --- | --- | | Category | data-domain layer | general agent | general agent plus editor | warehouse-native | | Warehouse context | ✓ multi-platform | ≈ via MCP | ≈ via MCP | ✓ Snowflake | | dbt DAG awareness | ✓ | ✗ | ✗ | ✓ | | Airflow awareness | ✗ | ✗ | ✗ | ✓ | | Column-level lineage | ✓ | ✗ | ✗ | ✗ | | Anti-pattern rules | ✓ 19 | ✗ | ✗ | ✗ | | Cost estimate before a query runs | ✓ | ✗ | ✗ | ✓ Snowflake only | | Cross-platform data diff | ✓ multi-platform | ✗ | ✗ | ✗ | | Deterministic PR gate | ✓ | ✗ | ✗ | ✗ | | Replayable session trace | ✓ | ✗ | ✗ | ✗ | | Skill library | ✓ 21, open source | ≈ you assemble it | ≈ you assemble it | ✓ Snowflake-supplied | | Best editing experience | ✗ | ✗ | ✓ | ✗ | | License | open source, MIT | commercial | commercial | subscription | ## How Each Agent Handles a Column Rename A column rename separates these four tools more clearly than any benchmark score does. The task is to rename a column in a staging model and keep every downstream model working. The rename compiles even when you update none of the downstream models, because dbt resolves a model reference to a table and does not check the columns inside that table. The first complaint comes at run time, when a downstream model selects a name that no longer exists. | Step in the rename | Altimate Code | Cortex Code CLI | Claude Code | Cursor | | --- | --- | --- | --- | --- | | Read the column's real name and type | warehouse metadata at call time | native SQL inside the account | warehouse query, if you wired MCP | string match over the repo | | Name every downstream model that reads it | ✓ column-level lineage | ✓ dbt project context | ✗ | ✗ | | Flag an anti-pattern the rewrite introduces | ✓ 19 rules | ✗ | ✗ | ✗ | | Apply the edit across many files | ≈ data code | ✓ | ✓ | ✓ best of the four | An agent that cannot list a column's downstream consumers falls back to a string search over the repository. A string search misses any reference that a Jinja template builds at compile time, because that name never appears in the source as plain text. Those built references are where a rename breaks in a real project. A dbt MCP server wired into Cursor or Claude Code lets either agent read a column's real type and list its downstream consumers. Your team sets that server up and keeps it running. ## The Three Costs That Sit Above the License Price License price is the smallest of the four costs of running any of these agents. The other three follow from how each tool is built. - **Assembly cost.** A general agent, the MCP servers under it and a CI check are all moving parts you own, and keeping them working is an ongoing job. - **Token cost.** A domain-native agent already holds the schema, so it reads fewer files to answer the same question, and fewer files read means fewer tokens bought. Pass@1 is the share of tasks an agent solves on the first attempt. Snowflake measured Cortex Code CLI at a 4-point higher Pass@1 at 3.9x lower cost than Claude Code on its own benchmark. - **Review cost.** An agent with no [correctness layer](https://altimate.ai/blog/the-correctness-layer-in-ade) shifts your effort from writing SQL to reviewing it. When the SQL is plausible and wrong, that review can cost more time than the generation saved. The [case for deterministic tooling over an LLM-only stack](https://altimate.ai/blog/deterministic-tooling-vs-llm-only-data-agents) sets out why that check has to be code rather than a second model, and [Altimate Code](https://github.com/AltimateAI/altimate-code) ships its checks in the open. ## Your Model Choice Moves the Score More Than Your Agent Choice Changing the model moves the benchmark score more than changing the agent does. We tested that on DataAgentBench by holding the agent and the task set fixed and changing only the model. Stratified Pass@1 in the table is Pass@1 balanced across task types, so no single task type dominates the score. | Measure | Claude Sonnet 4.6 | DeepSeek v4 pro | | --- | --- | --- | | Stratified Pass@1 | 60.4% | 56.9% | | Cost per trial | $0.76 | $0.29 | | Median runtime | 4 minutes | 9 minutes | | Trials returning no output | 32 of 270 | 79 of 270 | On the first two rows [DeepSeek v4 pro](https://altimate.ai/blog/deepseek-vs-claude-sonnet-ai-agent-benchmark) looks like the better buy. It costs $0.29 per trial against $0.76, and it trails Claude Sonnet 4.6 on Pass@1 by 3.5 points. The last row changes the decision. DeepSeek v4 pro returned nothing on 79 of 270 trials, which is 29% of attempts, against 32 of 270 for Claude Sonnet 4.6. Each empty run costs a retry and the time you waited for it. The Snowflake team came to the same conclusion with its own benchmark, and the spread it measured was wider: > Holding the harness fixed at CoCo, Pass@1 varies drastically across models: from 73.8% with Opus 5 to 64.1% with GPT 5.6 Sol to 56.6% with Sonnet 5, a 17-point spread. The harness in that quote is the agent, so Snowflake swapped only the model and the first-attempt pass rate moved by 17 points. In the same report, Snowflake's agent-versus-agent comparison separates Cortex Code CLI from Claude Code by 4 points. On Snowflake's own two numbers, the model swap moved the score about four times as far as the agent swap did. Your model choice therefore deserves at least as much attention as your agent choice. Still, the choice of agent matters. A score follows the model, and a model is a configuration value you can change at any time. What a model swap cannot change is what each tool reaches: - which schema it reads, - whether it traces column lineage, and - whether a deterministic check gates the merge. Pick the model for the score, and pick the tool for the reach. The model swap is also the cheaper change, because an agent is a tool every engineer has to move to. A data layer's checks run without a model, so a model swap changes nothing about what those checks return. You can reproduce both DataAgentBench runs from [the benchmarks page](https://altimate.ai/benchmarks). ## Which Agent to Pick for Your Stack There is no single best AI agent for data engineering, because these four tools do not compete on one layer. Claude Code, Cursor and Cortex Code CLI are agents, and an agent hosts tools. Altimate Code is a set of tools that an agent calls. Pick the agent by the shape of your stack, then decide whether to add the tool set. - **Mixed estate, and you already like your editor agent.** Keep it and add Altimate Code. This is the most common situation and the smallest change. - **Entirely on Snowflake, and you want the least assembly.** Pick Cortex Code CLI. The context comes already loaded, and Snowflake publishes call-efficiency numbers for it. - **On dbt or Airflow but not on Snowflake.** Cortex Code CLI now deserves a trial, because the standalone subscription removes the need for a Snowflake account. - **One general agent for everything, including non-data code.** Run Claude Code or Cursor. Wire in MCP servers for schema, and put Altimate Code in CI as the check. - **Editing speed matters most.** Run Cursor with a data layer beside it. Run a column rename through whichever agent you already pay for, and watch whether it names the downstream models before it edits a file. An agent that cannot name them is missing the data layer. For another example of what that layer returns, see our [cited answers on Snowflake spend](https://altimate.ai/product-highlights/snowflake-cost-analysis-ai-cited-answers). ## Frequently Asked Questions Is Altimate Code a replacement for Claude Code or Cursor? Altimate Code replaces neither one. It runs beside your editor agent and adds the data layer that Claude Code and Cursor lack. Which of the four agents reads my warehouse schema without extra setup? Altimate Code and Cortex Code CLI both do. Altimate Code reads schema through native drivers for any data platform or cloud database. Cortex Code CLI starts with your Snowflake schema and query history already loaded. Claude Code and Cursor need an MCP server that you configure and maintain. Do you need a Snowflake account to run Cortex Code CLI? You do not need one. Snowflake sells Cortex Code CLI as a self-service monthly subscription for teams that do not run on Snowflake, and on that subscription the agent reads dbt and Apache Airflow projects. Its deepest context is still Snowflake's own schema and query history. Can I run Altimate Code inside Cursor or VS Code? You can run Altimate Code in both. It ships as a VS Code extension that also installs into Cursor. The `/configure-claude` command wires it into Claude Code as well. Which of the four agents is open source? Altimate Code is open source under the MIT license, with 21 skills in the public repository. Claude Code and Cursor are commercial products. Cortex Code CLI is a Snowflake product sold on a subscription or consumed against Snowflake credits. On this page 1. Altimate Code 2. Claude Code 3. Cursor 4. Cortex Code CLI 5. Altimate Code vs Claude Code vs Cursor vs Cortex Code CLI, Feature by Feature 6. How Each Agent Handles a Column Rename 7. The Three Costs That Sit Above the License Price 8. Your Model Choice Moves the Score More Than Your Agent Choice 9. Which Agent to Pick for Your Stack From the product - [PowerUser Plugin for dbt Profile a slow dbt query and read its execution plan in plain English 4 min read](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan) - [Enterprise Platform Snowflake cost analysis in plain English, with AI that cites every number 5 min read](https://altimate.ai/product-highlights/snowflake-cost-analysis-ai-cited-answers) - [Enterprise Platform Root cause analysis for a dbt, Tableau, or Snowflake workload spike 6 min read](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis) - [Enterprise Platform Reduce Snowflake idle warehouse spend with Altimate Lite 6 min read](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) - [Enterprise Platform Snowflake Cortex AI cost breakdown by service, user, and model 6 min read](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # 9 Signs You Need an Agent-First Data Platform URL: https://altimate.ai/blog/signs-data-platform-needs-agent-first-overhaul Most AI agent pilots stall on the platform underneath them, not the model. Nine diagnostics for whether you need an agent-first data platform. ## 9 Signs You Need an Agent-First Data Platform Most AI agent pilots stall on the platform underneath them, not the model. Nine diagnostics for whether you need an agent-first data platform. Atharva Shah 2026-05-13 2026-09-04 12 min read On this page 12 sections 1. What an Agent-First Data Platform Means 2. Sign 1. Your Agents Invent Columns That Do Not Exist 3. Sign 2. Your Warehouse Bill Grew Faster Than Your Data 4. Sign 3. Pull Requests Sit for Days Because Nobody Trusts the Diff 5. Sign 4. Your Cost Tooling Reports and Never Acts 6. Sign 5. Every Migration Loses Months to Business Logic Parity 7. Sign 6. Nobody Can Say What a Column Means 8. Sign 7. Incidents Are Measured in Days 9. Sign 8. Your Agent Works in One Warehouse and Nowhere Else 10. Sign 9. You Have No Answer for "Who Approved This" 11. How to Score Your Own Platform 12. Which Sign to Fix First tl;dr Most AI agent pilots stall on the platform underneath them, not on the model. In Cleanlab's 2025 survey, one team in nineteen had agents live with real users, and observability and guardrails come from the platform around the model, so a stronger model does not move that number. What does move it is context the agent can query: governed metadata in the prompt took text-to-SQL accuracy from a 10-31% range to 94-99% with no change to the model. Each of the nine signs below has a check you can run today and a threshold that tells you whether it applies to your stack. Four or more failing signs means the platform blocks your next agent pilot. Start with the sign that costs you now: if the invoice is what you notice, attribute spend to teams first, and if pull requests sit for days, add lineage and diffing to CI first. Your team gave an AI agent access to write SQL or review dbt pull requests, and the pilot looked promising. The agent still is not running unsupervised, because a human checks every output before it ships. The model is rarely the reason. The reason is usually the platform under the agent, which was built for people to read and which software cannot query. Cleanlab surveyed 1,837 engineering and AI leaders in 2025 and found that 95 of them had agents live with real users. That is one in nineteen, and fewer than a third of those were happy with their observability and guardrails. A stronger model does not move that figure, because observability and guardrails come from the platform around the model. An agent-first data platform supplies both, along with the context an agent needs to decide correctly. A three-column diagram labeled Context, Guardrails and Record. Under each column sits the human version of it (a wiki page, a reviewer's judgment, a Slack thread) and the agent version (a queryable catalog, a policy check, an immutable log), with arrows showing the human column cannot be read by software *The three things an agent-first data platform exposes to software rather than only to people.* ## What an Agent-First Data Platform Means An agent-first data platform makes three things available to software. A conventional platform makes the same three things available only to people. - The context an agent needs for a correct decision is queryable at the moment the agent decides. - The guardrails decide whether a decision may execute, and they run before it executes. - The record of what happened is written by the platform, so nobody has to write it up afterwards. A conventional stack gives people all three and agents none. Column meanings live in a wiki, which an agent cannot query. Approval lives in a reviewer's head, so it applies only to the changes that reviewer sees. The audit trail is a Slack thread, which software cannot read. A person works around those three gaps by asking a colleague. An agent cannot ask, so it guesses. A guess that compiles returns a number and raises no error, so the mistake ships without anyone seeing it. An agent-first overhaul is narrower than a replatform. You modernize the three parts an agent has to read and leave the rest of the stack alone. Altimate right-sizes Databricks job clusters, SQL warehouses and all-purpose compute automatically, with no serverless migration required. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) ## Sign 1. Your Agents Invent Columns That Do Not Exist An agent writes SQL against a table it has never seen and fills the gaps with plausible names. You get `customer_id` where the column is `cust_key`, or a join on a field that has since been renamed. Your query history settles this. Pull the last twenty statements an agent wrote against production. Run each one through `EXPLAIN`, which validates every identifier and executes nothing. ```sql -- Snowflake. The columns an agent is allowed to believe in. select column_name, data_type, comment from analytics.information_schema.columns where table_name = 'ORDERS'; -- And the cheap validity check on what the agent wrote. explain using text select cust_key, order_total from analytics.orders; ``` More than one failure in twenty means [the agent is guessing at your schema](https://altimate.ai/blog/why-ai-coding-agents-fail-at-data-engineering). A wrong column that compiles costs more than one that fails. Read the column choices that did compile and check them against the definition your finance team uses. An agent reads `order_total` when the reported figure comes from `order_total_usd`, and the query returns a number that is wrong. Snowflake and Atlan measured what governed metadata in the prompt does to text-to-SQL accuracy. It moved from a range of 10% to 31% up to a range of 94% to 99%, and the models did not change. Give the agent a path to live schema, column types and definitions at call time. [Model Context Protocol](https://modelcontextprotocol.io) servers turn the catalog into something an agent queries mid-task rather than something a human reads first. Prompt engineering does not substitute for that path. A warehouse with 1,200 tables averaging 18 columns is 21,600 column names to carry in every prompt. Every one of them has to stay current, and a schema change lands while you paste it. ## Sign 2. Your Warehouse Bill Grew Faster Than Your Data A line chart over eighteen months with two lines. Storage volume rises in a gentle straight line while compute spend curves upward roughly twice as steeply, and the widening gap between them is shaded and labeled unexplained spend *When compute spend grows faster than storage, growth is not what is driving the bill.* Compare storage volume against compute spend over the last eighteen months. Two queries settle it, and both series sit in `SNOWFLAKE.ACCOUNT_USAGE`. ```sql -- Compute, by month, for the last eighteen months. select date_trunc('month', start_time) as month, sum(credits_used) as credits from snowflake.account_usage.warehouse_metering_history where start_time >= dateadd('month', -18, current_timestamp()) group by 1 order by 1; -- Storage, by month, over the same window. select date_trunc('month', usage_date) as month, avg(storage_bytes) as bytes from snowflake.account_usage.storage_usage where usage_date >= dateadd('month', -18, current_date()) group by 1 order by 1; ``` Divide the last month by the first month, once for credits and once for bytes. This sign applies if the compute multiple is more than twice the storage multiple. Three things drive that gap, and the invoice breaks out none of them. - AI workloads were free to try and are not free to run. - [Warehouses sit sized for a peak](https://altimate.ai/blog/snowflake-cost-optimization-techniques) that happens twice a week. - Queries scan everything, because narrowing them was nobody's job. Idle time is the easiest of those three drivers to price. [Snowflake's docs](https://docs.snowflake.com/en/user-guide/warehouses-overview) put a Large warehouse at 8 credits an hour with auto-suspend at 600 seconds. Suppose forty query bursts a day, each followed by ten idle minutes. They leave 400 idle minutes running, which is 53 credits a day, or roughly 1,600 a month. Agents move that number in both directions. An unsupervised agent can run up spend in an afternoon, and nobody sees it until the invoice. An agent pointed at warehouse configuration tunes that configuration continuously instead, because no person adjusts a warehouse a hundred times a day. Altimate's Auto Tune [right-sizes warehouses without slowing queries](https://altimate.ai/product-highlights/snowflake-warehouse-auto-tune-right-sizing). ## Sign 3. Pull Requests Sit for Days Because Nobody Trusts the Diff AI made writing code cheaper, and reviewing that code still costs what it always did. A reviewer who cannot tell whether a 400-line model change is correct defers it, the queue grows, and the team concludes AI made them slower. Measure the queue rather than trusting the feeling. Ask an engineer how long reviews take, and they report the time they spend reviewing, a much smaller number than the time a pull request spends waiting. ```bash # Open pull requests touching models/ that have waited more than five days. gh pr list --state open --limit 100 --json number,createdAt,files --jq \ '[ .[] | select(any(.files[]; .path | startswith("models/"))) | select(now - (.createdAt | fromdate) > 432000) ] | length' ``` Divide that count by the number of people who actually review model changes, which is usually smaller than the team. More than one stale review per reviewer means the queue is stuck on verifying rather than writing, so buying more capacity to write makes it worse. The team may be right about the slowdown. A 2025 study put sixteen experienced developers on real tasks in repositories they knew well, with and without AI tools. They took longer with the tools while believing they had been faster. Writing got faster and the work moved into verifying, which nobody instrumented or staffed. A team that measures only the writing half records a gain while the total time per change goes up. Evidence attached to the change clears the queue. Show a reviewer three things: - Which downstream models the diff touches, as a count rather than a diagram. - That the rewritten query returns identical rows. - That no new anti-pattern appeared. A reviewer checking those three claims is not reading 400 lines of SQL. The platform produces those three claims, so the team does not change how it runs review. [Why AI agents break in production](https://altimate.ai/blog/why-ai-agents-break-in-production-the-missing-harness-in-your-data-stack) makes the longer argument. ## Sign 4. Your Cost Tooling Reports and Never Acts Open your cost tool and export last quarter's recommendations, then count how many became a shipped change. The tool records what it advised and not what you did, so the evidence sits in git history rather than in the tool. Two hundred recommendations and six applied changes is an applied rate of 3%. Under 10% means you bought a dashboard. Keebo sells autonomous Snowflake optimization, and its own guide puts it flatly. > Observability without execution is just a dashboard and saves you nothing. Keebo is naming the failure mode of the category it competes in. A reporting tool describes a state. An execution tool alters that state, under a policy, with a rollback. A team with more recommendations than bandwidth needs an execution tool. Four categories compete for this budget, and each one answers a different question, so a team can own several and still fail this sign. | Category | Examples | Reads | Acts | Crosses platforms | | --- | --- | --- | --- | --- | | Catalog-first | Atlan, Secoda | metadata, lineage, glossary | ✗ | ✓ | | Observability | Monte Carlo, Elementary | freshness, volume, schema drift | ✗ alerts only | ✓ | | Post-hoc FinOps | Keebo, Espresso AI, Revefi, Unravel | query and warehouse history | ≈ on config | ✗ usually one platform | | Warehouse-native agents | Snowflake Cortex, Databricks Lakeflow, BigQuery Data Engineering Agent | everything inside their own platform | ✓ | ✗ | Ask of each tool you own which of the nine signs it closes. Altimate right-sizes Databricks job clusters, SQL warehouses and all-purpose compute automatically, with no serverless migration required. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) ## Sign 5. Every Migration Loses Months to Business Logic Parity Syntax translation is the part that goes well. A Redshift dialect becomes a Snowflake dialect, a stored procedure becomes a dbt model, and the first eighty percent lands in weeks. Then the project stops. Somebody has to prove the new pipeline produces the same numbers as the old one, and that old pipeline encodes a decade of undocumented decisions. A `WHERE` clause excludes test accounts by a rule that exists nowhere except in that clause. Score this as a coverage ratio rather than a schedule. Count the objects in migration scope. Then count the ones with an automated row-level and column-level diff running today, and divide the second count by the first. For an object with no diff, parity rests on somebody's reading of two queries. Suppose a legacy warehouse holds 340 views and 120 stored procedures. A diff covering the views and skipping the procedures reports 74% coverage. The missing 26% holds the business logic finance will question, so treat that 74% as zero coverage. [Row-level and column-level diffing between two platforms](https://altimate.ai/blog/dbt-migration-with-an-ai-agent-playbook) turns an assumption of parity into a report a stakeholder signs. Teams that run the diff in CI from week one do not lose the quarter, and teams that treat parity as final QA usually do. ## Sign 6. Nobody Can Say What a Column Means Pick a table your business runs on and ask two analysts to define its fourth column without looking it up. If the answers differ, the definition lives in people's heads rather than the platform. An agent reading the schema has less to work with than either analyst. Then measure the whole schema. Description coverage is a single query on any Snowflake account. ```sql -- What share of one database's columns carries a description? select count(*) as columns_total, count(comment) as columns_described from snowflake.account_usage.columns where deleted is null and table_catalog = 'ANALYTICS'; ``` Ten thousand columns with 2,100 descriptions is 21% coverage. The threshold is 60% on the schemas your agents query. Below it, the agent infers the meaning of most columns from the column name alone. Atlan's write-up of a Workday deployment names the same failure. Its revenue analysis agent could not answer a single question. Nothing mapped the word "revenue" to the authoritative tables and columns. Documentation used to help whichever colleague read it. Now it is the input to every agent that touches the table. ## Sign 7. Incidents Are Measured in Days A dashboard is wrong on Monday, someone notices on Tuesday, and you track down the owner on Wednesday. If that sequence is familiar, the incident itself is the smaller cost. The larger one is the distrust that follows, because every stakeholder starts rechecking numbers they used to accept. Your last ten incidents settle it. Record the hours between the first report and the named root cause, then take the median. A median above eight working hours means triage is a search rather than a lookup. Then count the tools somebody opens during one of those triages. The person debugging reconstructs a dependency chain across a warehouse, an orchestrator and a BI tool that share no graph. Atlan's own diagnosis of agentic triage lands on the same constraint. > If lineage stops at the warehouse boundary, the agent finds the broken table and stops there. [Column-level lineage that crosses tool boundaries](https://altimate.ai/blog/column-level-lineage-use-cases) turns that reconstruction into a lookup. An agent that maps the downstream impact before acting can escalate instead of proceeding. Altimate documents that workflow in [root cause analysis for a dbt, Tableau, or Snowflake workload spike](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis). ## Sign 8. Your Agent Works in One Warehouse and Nowhere Else Warehouse-native agents work well inside their own platform. Google's BigQuery Data Engineering Agent reached general availability in April 2026, per [the Google Cloud blog](https://cloud.google.com/blog/products/data-analytics/exploring-the-data-engineering-agent-in-bigquery). Databricks ships Lakeflow with Agent Bricks, and Snowflake has Cortex. The check is a coverage fraction. List every platform holding a table that something downstream depends on. List the platforms your agent layer can read, and divide the second count by the first. Suppose you run Snowflake, a Postgres database behind the product, and a BigQuery project you inherited. An agent confined to one platform covers 33% of the estate, and the surprises come from the two platforms it cannot read. Atlan states the tradeoff plainly, that warehouse-native agents see only their own platform. A portable agent layer connects to each platform through that platform's own driver rather than through one vendor's runtime. Our [warehouse configuration reference](https://help.altimate.ai/code/configure/warehouses/) describes how. ## Sign 9. You Have No Answer for "Who Approved This" Ask what happens when an agent proposes a schema change to a table holding regulated data. Suppose the answer routes through one person's judgment, with nothing written down. Then you have no governance layer. Start with permissions. That check is the fastest of the nine checks and the only one where a single bad answer is unrecoverable. The other eight cost time and money. This one can destroy data. ```sql -- Everything the agent's role may touch, regardless of today's task. show grants to role AGENT_SERVICE_ROLE; ``` Read that list against what the agent's task needs today. Any grant beyond that is a standing privilege. In July 2025 a Replit AI coding agent deleted a production database, and the agent had standing privileges to it. Nothing exotic happened. A capable agent held permissions broader than its task and used them. Then check the record. Pick one agent action from last week and name: - what changed, - the policy that allowed it, - what else it could have touched, and - who was accountable. If you cannot answer one of those four questions, the governance layer is missing rather than the paperwork. A governance layer here covers less ground than the word suggests. It has three parts, and none of them requires a written policy framework. - Scoped permissions give an agent the minimum access its current task needs. - A confidence threshold routes a low-certainty action to a human instead of executing it. - An immutable record names what changed, under which policy, and why. Our [governance configuration reference](https://help.altimate.ai/code/configure/governance/) is one worked example of the three parts written down as configuration. Regulated teams need the immutable record to deploy at all. Every other team needs it the first time somebody asks what happened. [You are the trust layer](https://altimate.ai/blog/you-are-the-trust-layer-managing-data-engineering-ai-agents-at-scale) covers how that responsibility distributes across a team. ## How to Score Your Own Platform You can score your own platform against these nine checks in an afternoon. | Sign | The check | Passing | | --- | --- | --- | | 1\. Invented columns | Last 20 agent queries through `EXPLAIN` | No unknown identifiers | | 2\. Bill outruns data | 18 months of credits against stored bytes | Compute grows no faster | | 3\. Stalled review | Open `models/` PRs over five days old | Under one per reviewer | | 4\. Reports, never acts | Applied changes over recommendations | Over half ship | | 5\. Migration parity | Objects diffed, over objects in scope | Full, before cutover | | 6\. Undefined columns | Share of columns with a description | Over 60% | | 7\. Slow triage | Median hours from report to root cause | Under eight | | 8\. One platform only | Platforms read, over platforms run | All of them | | 9\. No approval record | `SHOW GRANTS` on the agent role | No standing grant | Four or more failing rows means the platform blocks your next agent pilot, and a stronger model changes none of those rows. The rows are independent, so each failing one is a project a single person can own. Once execution and evidence live in one system, you can attribute a saving to the change that produced it rather than estimate it after the fact. Altimate's own platform customers average $840K a year on that basis. ## Which Sign to Fix First Start with the sign that costs most this quarter rather than the one easiest to fix. When the invoice is what you notice first, attribute spend to teams and models before you tune anything. When pull requests sit for days, add lineage and diffing to CI first. When nothing is urgent, fix Sign 6 first. Column meanings take longest to build because they need the people who know the answers, and once those definitions exist they feed every other sign here. ## Frequently Asked Questions What does agent-first mean for a data platform? The context, guardrails and record an agent needs are available to software rather than only to people. Three things are true in that state: How do we test our AI readiness without buying anything? Every one of the nine checks runs on systems you already have. Three are single queries against `SNOWFLAKE.ACCOUNT_USAGE`, two are counts out of git, and one is a `SHOW GRANTS`. Score the nine rows before you talk to a vendor. Is this a replatform? No item on this list requires replacing a platform you already run. Each one adds a layer to the stack you have. Start with one of the three layers, context, guardrails or record, rather than all three. Why do so few teams have agents in production? Cleanlab's 2025 survey of 1,837 engineering and AI leaders found 95 with agents live in production. Fewer than a third of those were satisfied with their guardrail tooling. Observability and guardrails come from the platform around the model, so a stronger model does not close that gap. Do warehouse-native agents solve this? Inside one platform they work well, and BigQuery, Databricks and Snowflake all ship capable ones. Each one stops at its own platform edge. A mixed estate needs lineage and policy that cross vendor boundaries. On this page 1. What an Agent-First Data Platform Means 2. Sign 1. Your Agents Invent Columns That Do Not Exist 3. Sign 2. Your Warehouse Bill Grew Faster Than Your Data 4. Sign 3. Pull Requests Sit for Days Because Nobody Trusts the Diff 5. Sign 4. Your Cost Tooling Reports and Never Acts 6. Sign 5. Every Migration Loses Months to Business Logic Parity 7. Sign 6. Nobody Can Say What a Column Means 8. Sign 7. Incidents Are Measured in Days 9. Sign 8. Your Agent Works in One Warehouse and Nowhere Else 10. Sign 9. You Have No Answer for "Who Approved This" 11. How to Score Your Own Platform 12. Which Sign to Fix First From the product - [Enterprise Platform Root cause analysis for a dbt, Tableau, or Snowflake workload spike 6 min read](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis) - [Enterprise Platform Attribute Snowflake cost to the team that owns it 4 min read](https://altimate.ai/product-highlights/snowflake-cost-alerts-by-team) - [Enterprise Platform Auto Tune right-sizes Snowflake warehouses without slowing queries 6 min read](https://altimate.ai/product-highlights/snowflake-warehouse-auto-tune-right-sizing) - [Enterprise Platform Databricks cost breakdown, from one bill down to one warehouse 4 min read](https://altimate.ai/product-highlights/databricks-cost-breakdown-workspace-sku-cluster) - [Enterprise Platform Snowflake cost analysis in plain English, with AI that cites every number 5 min read](https://altimate.ai/product-highlights/snowflake-cost-analysis-ai-cited-answers) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # The Hidden Cost of AI Coding Tools and Your Token Bill URL: https://altimate.ai/blog/the-hidden-cost-of-ai-coding-tools-why-your-token-bill-lies The cost of AI coding tools depends on tokens per finished task. See how context re-reads, cache writes, thinking and retries grow a Claude Code bill. ## The Hidden Cost of AI Coding Tools: Why Your Token Bill Lies The cost of AI coding tools depends on tokens per finished task. See how context re-reads, cache writes, thinking and retries grow a Claude Code bill. Atharva Shah 2026-09-10 2026-09-16 10 min read On this page 9 sections 1. The Cost of AI Coding Tools Depends on Tokens per Finished Task 2. Claude Code Sends the Whole Conversation With Every Request 3. Prompt Caching Cuts Re-Read Cost to One Tenth of the Input Price 4. Thinking Tokens and Compaction Add Requests You Never Typed 5. Failed and Retried Runs Are Billed Like Successful Ones 6. One Sonnet 4.6 Trial Spent 72% of Its Cost on the Prompt Cache 7. Our Benchmark Runs Show Cost per Trial Moving in Both Directions 8. Deterministic Tools Remove LLM Calls From Validation 9. Measure Cost per Merged Pull Request Before You Switch Models tl;dr The per-token price on a pricing page is exact, but it prices one request. An agent task sends many requests, and each request carries the whole conversation again. Cache writes, thinking tokens, compaction and failed runs add more requests that nobody typed. In one of our benchmark runs, a Claude Sonnet 4.6 trial read 1.22 million tokens from the prompt cache. By our arithmetic, the cache accounted for about 72% of that trial's $0.76 cost. To see the cost of AI coding tools on your team, divide weekly spend by merged pull requests. A data team rolls out Claude Code, sets a budget from the price list, and gets a bill that does not match the budget. The price list was correct. The cost of AI coding tools depends on how many tokens one finished task consumes, and the price list says nothing about that number. As of September 2026, Anthropic lists Claude Sonnet 4.6 at $3 per million input tokens and $15 per million output tokens. Those rates apply to each API request. One agent task, such as "add a not-null test to this dbt model", sends dozens of requests. Each request carries the full conversation so far, plus every tool result the agent has collected. Anthropic's own [Claude Code costs documentation](https://code.claude.com/docs/en/costs) puts the average at about $13 per developer per active day. It also gives a range of $150 to $250 per developer per month, across enterprise deployments. For a five-person data team that works 20 days a month, $13 a day comes to $1,300 a month. That average is a planning number, and your own mix of models and session habits moves it in either direction. Spend grows past the list price in five places inside one task. Each place has a habit or a setting that controls it, and a weekly query shows which one moved your bill. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## The Cost of AI Coding Tools Depends on Tokens per Finished Task A token price is an input to your bill. The count of tokens per finished task is the multiplier. Two teams on the same plan, with the same price per token, can pay very different amounts for the same work. Five mechanisms raise the token count of one task: - The agent sends the whole conversation again with every request. - The prompt cache charges extra to store that conversation. - Thinking tokens bill at the output rate. - Compaction reads the whole conversation and writes a shorter version of it. - A failed or retried run bills the same as a run that worked. None of the five shows up as a line item on the price list. All five show up on the invoice. A sixth cause sits with the vendor, in tokenizers, fees and model changes. [Why token counts differ between vendors](https://altimate.ai/blog/the-great-token-heist-of-26) covers that cause. ## Claude Code Sends the Whole Conversation With Every Request Claude Code sends your full conversation with every request, says its cost guide. Each time the agent uses a tool, Claude Code sends another request that carries the tool results. So a one-line question late in a long session pays for everything above it. A data engineering session grows fast. The agent reads a schema listing, a `manifest.json` excerpt, a compiled SQL file and a `dbt build` log. All four stay in context. The next request carries all of them again, whether the agent still needs them or not. Anthropic's doc names the usual cause of high spend on an API plan. The doc traces high spend to long sessions nobody cleared, and to Opus left as the default model. Run `/clear` between unrelated tasks. The doc says a fresh start with `/clear` costs nothing. ## Prompt Caching Cuts Re-Read Cost to One Tenth of the Input Price Prompt caching makes the repeated context cheaper, and it adds a charge of its own. On [Anthropic's price list](https://platform.claude.com/docs/en/about-claude/pricing), a cache read costs 0.1x the base input price. A cache write costs 1.25x the base input price for a 5-minute lifetime, and 2x for a 1-hour lifetime. For Sonnet 4.6, that means $0.30 per million tokens read from cache and $3.75 per million written with the 5-minute lifetime. Anthropic's pricing page says caching "pays off after one cache read for the 5-minute duration". A cache that expires before the next request only adds the write premium. A short cache lifetime can erase the saving. The costs doc says the first message after a break longer than the cache lifetime misses the cache and reprocesses the full context. On an API key, that lifetime is five minutes by default. A developer who reads a query plan for ten minutes and then asks a follow-up pays to process the whole session again. ## Thinking Tokens and Compaction Add Requests You Never Typed Anthropic bills thinking tokens at the output rate. The costs doc says the default budget "can be tens of thousands of tokens per request". At Sonnet 4.6's $15 output rate, 20,000 thinking tokens cost $0.30 before the agent writes a line of SQL. The same doc says you can lower the budget on models with a fixed thinking budget: ```bash # Lower the thinking budget for simple tasks on fixed-budget models export MAX_THINKING_TOKENS=8000 ``` Compaction is the second request you never typed. The doc warns that compacting "a large context is itself a large request." Auto-compaction runs when a session nears its context limit. At that point, the conversation is at its largest. Agent teams multiply both effects. In plan mode, agent teams use about 7x more tokens than standard sessions, the doc says. Each teammate keeps its own context window. For simple subagent jobs, the doc recommends Haiku in the subagent file: ```yaml # .claude/agents/dbt-test-runner.md frontmatter name: dbt-test-runner description: Runs dbt tests and returns only the failures model: haiku ``` ## Failed and Retried Runs Are Billed Like Successful Ones A run that produces wrong SQL is billed at the same rates as a run that produces correct SQL. The provider bills tokens, and it has no view of whether your `dbt build` passed. So every retry adds a full run to the cost of one finished task. We measured this in a plugin experiment on Claude Sonnet 4.6. On a cross-warehouse migration task, bare Claude produced zero working output in 5 attempts. Five runs were billed, and none of them produced a model you could merge. Our DataAgentBench run shows the same effect at a larger scale. Claude Sonnet 4.6 ran 270 trials at an average of $0.76 each, so the run cost about $205. Of those 270 trials, 100 failed. In 32 trials, the agent never wrote an answer. Those 100 failed trials were billed at the same token rates as the 170 that passed. The per-trial price never shows how much of the spend bought a wrong answer. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## One Sonnet 4.6 Trial Spent 72% of Its Cost on the Prompt Cache Our [540-trial DeepSeek and Sonnet cost test](https://altimate.ai/blog/deepseek-vs-claude-sonnet-ai-agent-benchmark) published the token mix of an average Claude Sonnet 4.6 trial. Each trial wrote 49,000 tokens to Anthropic's prompt cache and read 1.22 million tokens back. It spent zero reasoning tokens, and it cost $0.76. The arithmetic below is ours. It uses the list prices above and the 5-minute cache lifetime. The post does not state which lifetime the run used. | Line | Tokens per trial | Rate per million | Cost | | --- | --- | --- | --- | | Cache reads | 1,220,000 | $0.30 | $0.37 | | Cache writes | 49,000 | $3.75 | $0.18 | | Uncached input and output | not published | $3 in, $15 out | $0.21, the remainder | | Total per trial | | | $0.76 | A stacked bar splits one Claude Sonnet 4.6 DataAgentBench trial costing $0.76 into three parts: cache reads of 1.22 million tokens at $0.37, cache writes of 49,000 tokens at $0.18, and uncached input and output at $0.21 *One average Sonnet 4.6 trial from our DataAgentBench run, split by our arithmetic at list prices.* Cache reads and writes together come to $0.55, or about 72% of the $0.76. Without the cache, the same 1.22 million tokens at the $3 input rate would cost $3.66 on their own. The average trial made 36 tool calls, so the agent re-read roughly 34,000 tokens for each call. With the 1-hour lifetime, the writes cost $0.29 and the cache share rises. For this trial, context size moved the cost more than the price per token did. If every re-read had missed the cache, the input alone would have cost $3.66, nearly five times the trial. A trimmed schema payload saves money on every one of those 36 re-reads. ## Our Benchmark Runs Show Cost per Trial Moving in Both Directions A cheaper model per token can cost less per task, and a more expensive setup can still be worth paying for. Two of our own runs show both cases. | Run | Setup A | Setup B | What changed | | --- | --- | --- | --- | | DataAgentBench, 270 trials each | Claude Sonnet 4.6, $0.76 per trial | DeepSeek v4 pro, $0.29 per trial | The model, with the same agent and tools | | ADE-Bench, 43 tasks, Claude Sonnet 4.5 | No skills, $0.33 per task, 20 of 43 solved | With skills, $0.40 per task, 23 of 43 solved | Skills added, with the same model | Two panels. The left panel compares DataAgentBench cost per trial, $0.76 for Claude Sonnet 4.6 and $0.29 for DeepSeek v4 pro. The right panel compares ADE-Bench cost per task on Claude Sonnet 4.5, $0.33 without skills and $0.40 with skills *Cost per attempt from two of our benchmark runs. A model swap cut it, and adding skills raised it.* On DataAgentBench, DeepSeek v4 pro ran at 38% of Sonnet's per-trial cost and scored 3.5 points lower on stratified Pass@1. It also left 79 of 270 trials with no answer, against 32 for Sonnet. On ADE-Bench, skills raised the cost of each attempt by 21% and solved three more tasks. Neither per-attempt number tells you which setup is cheaper per finished task. To compare the setups, divide total spend by the tasks that passed, so the failures count against each setup. [Cost per completed task](https://altimate.ai/blog/cost-per-task-vs-dollars-per-million-tokens) shows that calculation as a SQL query over logged attempts. The costs that sit on top of tokens, such as review time and rework, belong in the same calculation. [The costs that sit above the license price](https://altimate.ai/blog/altimate-code-vs-claude-code-vs-cursor) lists them for three agents. ## Deterministic Tools Remove LLM Calls From Validation Some checks have exactly one right answer, and a compiled tool answers them without calling a model. Altimate Code, which is [an AI data engineering agent](https://altimate.ai/), ships such tools. The `altimate_core_validate` tool checks SQL syntax and schema references in a Rust library, with no LLM call. The cost difference comes from the loop. Without the tool, the agent learns about a wrong table name from a warehouse error, then re-reads its context and tries again. Our launch post describes a wrong table name that the tool catches in 2 ms, where Snowflake returns the error after 30 seconds. Five fix cycles took 10 ms against 2.5 minutes of warehouse round-trips. [The zero-token schema check](https://altimate.ai/blog/schema-validation-costs-0-tokens-how-deterministic-tools-cut-llm-spend) follows one column rename through that loop. [Compiled checks that cost nothing per run](https://altimate.ai/blog/deterministic-tooling-vs-llm-only-data-agents) covers the wider argument. The `altimate-code check` command runs lint, validate, safety, policy, PII, semantic and grade checks without a model provider or API key. This approach has limits, and you should plan for them: - The agent still spends tokens to call the tool and to read its result, so a tool call is never free. - Altimate does not lower the price per token. With your own LLM key, you pay the provider at its own rates. - In our plugin experiment on Claude Sonnet 4.6, the plugin cut aggregate cost by 18.6% on three medium ADE-Bench tasks that both setups passed. - On the simplest model-creation tasks, the same plugin added about 25% cost with no gain in correctness. We ran each setup once or twice, so treat these percentages as a direction only. ## Measure Cost per Merged Pull Request Before You Switch Models Start with one number your team can track each week: AI spend divided by merged pull requests. The number is rough. It still counts retries and failed sessions, and a price list counts neither. Take spend from the most authoritative source you have. The costs doc names the Usage page in the Claude Console for API billing, and the `/usage` figure is only an estimate. Teams plans export a spend report CSV per user and model. OpenTelemetry export streams per-user token and cost metrics on every setup. ```sql -- ai_spend_daily: one row per engineer per day, loaded from your billing export -- merged_prs: one row per merged pull request, loaded from your git host with spend as ( select date_trunc('week', usage_date) as week, sum(usd) as usd from ai_spend_daily group by 1 ), merged as ( select date_trunc('week', merged_at) as week, count(*) as merged_prs from merged_prs group by 1 ) select s.week, s.usd, m.merged_prs, round(s.usd / nullif(m.merged_prs, 0), 2) as usd_per_merged_pr from spend as s left join merged as m on m.week = s.week order by s.week; ``` Watch the trend for four weeks before you change anything. A rise with flat pull request counts points at context growth, cache misses or retries, in the order this post covers them. Then change one thing, such as a smaller thinking budget or a Haiku subagent. Compare the next four weeks against the first four. To take validation calls out of the model loop, install [Altimate Code](https://altimate.ai/products/altimate-code) with `npm install -g altimate-code` and run `altimate-code check` on one dbt project. ## Frequently Asked Questions What is the hidden cost of AI coding tools? The hidden cost is the number of tokens one finished task consumes. Each agent request carries the whole conversation. Cache writes, thinking tokens, compaction and failed runs add cost on top. The price per token stays the same while the tokens per task grow. How much does Claude Code cost per developer? Anthropic's costs documentation gives an average of about $13 per developer per active day. It also gives $150 to $250 per developer per month across enterprise deployments. It says 90% of users stay below $30 per active day. Does prompt caching lower my Claude bill? Prompt caching lowers the cost of repeated context to 0.1x the base input price on Anthropic's price list. A cache write costs 1.25x for a 5-minute lifetime or 2x for a 1-hour lifetime. A cache that expires between requests only adds the write premium. How do I check Claude Code token usage? Run `/usage` in Claude Code to see the session's token counts and an estimated cost. The doc calls that figure an estimate. For authoritative API billing, use the Usage page in the Claude Console. Do deterministic tools make an AI agent free to run? Deterministic tools remove the model call from a check, such as schema validation. The agent still spends tokens to call the tool and read the result. They cut cost on checks with one right answer, and they do not change the price per token. On this page 1. The Cost of AI Coding Tools Depends on Tokens per Finished Task 2. Claude Code Sends the Whole Conversation With Every Request 3. Prompt Caching Cuts Re-Read Cost to One Tenth of the Input Price 4. Thinking Tokens and Compaction Add Requests You Never Typed 5. Failed and Retried Runs Are Billed Like Successful Ones 6. One Sonnet 4.6 Trial Spent 72% of Its Cost on the Prompt Cache 7. Our Benchmark Runs Show Cost per Trial Moving in Both Directions 8. Deterministic Tools Remove LLM Calls From Validation 9. Measure Cost per Merged Pull Request Before You Switch Models From the product - [PowerUser Plugin for dbt Profile a slow dbt query and read its execution plan in plain English 4 min read](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan) - [Enterprise Platform Root cause analysis for a dbt, Tableau, or Snowflake workload spike 6 min read](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis) - [Enterprise Platform Reduce Snowflake idle warehouse spend with Altimate Lite 6 min read](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) - [Enterprise Platform Snowflake Cortex AI cost breakdown by service, user, and model 6 min read](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown) - [Enterprise Platform Attribute Snowflake cost to the team that owns it 4 min read](https://altimate.ai/product-highlights/snowflake-cost-alerts-by-team) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Claude Code Cost for Data Teams and How to Cut It URL: https://altimate.ai/blog/the-2000-month-claude-bill-your-data-team-doesnt-need Claude Code cost for a data team reaches $2,000 a month at Anthropic's published averages. See the arithmetic and five levers that cut tokens on dbt work. ## The $2,000/Month Claude Bill Your Data Team Doesn't Need Claude Code cost for a data team reaches $2,000 a month at Anthropic's published averages. See the arithmetic and five levers that cut tokens on dbt work. Atharva Shah 2026-09-16 2026-09-16 10 min read On this page 9 sections 1. How a Data Team's Claude Code Cost Reaches $2,000 a Month 2. Every Schema and Model File the Agent Reads Adds to the Bill 3. Deterministic Tools Answer Schema, Lineage and Lint Questions Without Model Tokens 4. Haiku Costs One-Fifth of Opus 5 per Token for Simple Subagent Work 5. Prompt Caching Cuts a Repeated 50,000-Token Prefix By 78.5% 6. A Project Manifest Hands the Agent the Graph Without Reading Every Model 7. Check /Usage Every Session and the Console Every Week 8. What Altimate Code Changes in the Bill and What It Leaves Alone 9. Measure One Week of /Usage, Then Move One Check Off the Model tl;dr Anthropic puts the average Claude Code cost at about $13 per developer per active day. At that rate, a data team of eight to ten engineers spends about $2,000 a month. A team of two can reach the same bill if both engineers run agent teams on half their working days. On dbt and SQL work, much of that spend pays for context. The agent reads schemas, upstream models and test files. You can cut it without giving up the agent. Answer exact questions with deterministic tools, and pick the model per task. Keep prompt caching effective, give the agent a dbt manifest, and check `/usage` in every session. Your data team runs Claude Code every day, and its invoice now comes up in budget reviews. Anthropic publishes the rates behind that invoice on its [Claude Code costs page](https://code.claude.com/docs/en/costs). It also says 90% of users stay below $30 per active day. Multiply those rates by a team and a $2,000 month appears with nobody misusing the tool. Eight engineers at $250 a month reach it. Two engineers who run agent teams on half their working days reach it too. The Claude Code cost you can remove sits inside those sessions, in the tokens each one spends. Data work pushes that cost up for a specific reason. Anthropic's costs page says token costs scale with context size. A dbt change pulls in the schema, the upstream models and the tests. Each file the agent reads adds context. So the best first fix removes context the model does not need. Your team keeps the agent it already trusts. ## How a Data Team's Claude Code Cost Reaches $2,000 a Month A data team's Claude Code cost reaches $2,000 a month at eight to ten engineers on Anthropic's averages. Every row below starts from Anthropic's figures. The active-day counts are our assumptions, so replace them with your own. | Team shape | Rate per engineer | Arithmetic | Monthly total | | --- | --- | --- | --- | | 10 engineers at the average daily cost | $13 per active day, 15 days | 10 × $13 × 15 | $1,950 | | 8 engineers at the top of the monthly range | $250 per month | 8 × $250 | $2,000 | | 4 engineers at the 90th-percentile ceiling | $30 per active day, 17 days | 4 × $30 × 17 | $2,040 | | 2 engineers using agent teams on 10 of 20 days | $13 on standard days, about $91 on agent-team days | 2 × (10 × $13 + 10 × $91) | $2,080 | The last row carries the largest multiplier. Anthropic says agent teams use about 7x more tokens than standard sessions when teammates run in plan mode. The row assumes that cost follows tokens, so a $13 day becomes about $91. Treat that figure as rough, because the 7x describes tokens and not dollars. Horizontal bar chart of four team shapes that each reach about $2,000 a month in Claude Code cost. Ten engineers at $13 per active day for 15 days reach $1,950. Eight engineers at $250 a month reach $2,000. Four engineers at $30 per active day for 17 days reach $2,040. Two engineers who use agent teams on half their days reach $2,080 *Four team shapes that reach about $2,000 a month. The rates come from Anthropic's Claude Code costs page, and the active-day counts are assumptions.* Two more cost drivers sit outside the table. Thinking tokens bill as output tokens, and the default thinking budget can reach tens of thousands of tokens per request. Anthropic names the usual cause of high spend as a long session nobody cleared, or Opus left as the default. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## Every Schema and Model File the Agent Reads Adds to the Bill A dbt change needs more than the one file you asked the agent to edit. To rename a column safely, the agent needs the model, its sources, its downstream models and their tests. Claude Code finds those by reading files and running commands. Every result it reads becomes context, and context is what you pay for. Context also carries forward inside a session. Claude Code manages that history with prompt caching and with auto-compaction near the context limit. Compaction has its own price, because `/compact` reads the whole conversation it summarizes. Our own benchmark traces show the scale of that repeated context. In our DataAgentBench runs, Claude Sonnet 4.6 wrote about 49,000 tokens to the prompt cache per trial. It read 1.22 million tokens back from that cache in the same trial. A token count also changes when the model changes. Anthropic's migration guide puts that change at roughly 1x to 1.35x for the same text. Our write-up on [token inflation between vendors](https://altimate.ai/blog/the-great-token-heist-of-26) collects the measured ratios. Your usage baseline needs a recheck after every model upgrade. ## Deterministic Tools Answer Schema, Lineage and Lint Questions Without Model Tokens A question with one right answer does not need a model to answer it. "Does this column exist?" and "which models read this column?" both have exact answers. Compiled code returns those answers for the same input on every run, and it calls no model to do so. We covered the wider argument in [deterministic tooling for AI agents](https://altimate.ai/blog/deterministic-tooling-vs-llm-only-data-agents). Altimate Code ships these tools in a compiled Rust engine. Four of them answer schema, lineage and lint questions: - `altimate_core_validate` checks SQL syntax and schema references. - `altimate_core_column_lineage` traces column lineage offline, with no API key or account. - `sql_analyze` flags SQL anti-patterns through the engine's lint and semantic checks. - `dbt_manifest` parses the `manifest.json` file that dbt writes for your project. The same checks also run headless, with no model provider and no API key. Run them in CI or a pre-commit hook and they spend no tokens at all. The commands come from the [Altimate Code check reference](https://help.altimate.ai/code/usage/check/): ```bash npm install -g altimate-code # Lint two dbt models for anti-patterns and unsafe SQL altimate-code check models/staging/stg_orders.sql models/marts/fct_revenue.sql # Validate column references against a schema file, as JSON altimate-code check --checks validate --schema schema.yml --format json ``` These tools have one limit inside an agent session. An agent that calls a tool still pays tokens for the call and for reading the result. The saving is the gap between that result and the files the agent would read instead. We have not published a measurement of that gap for Claude Code sessions. ## Haiku Costs One-Fifth of Opus 5 per Token for Simple Subagent Work Claude Haiku 4.5 lists at $1 per million input tokens, and Claude Opus 5 lists at $5. The prices below come from [Anthropic's pricing page](https://platform.claude.com/docs/en/about-claude/pricing) as of 16 September 2026. MTok means one million tokens. | Model | Input per MTok | Output per MTok | Cache read per MTok | | --- | --- | --- | --- | | Claude Haiku 4.5 | $1 | $5 | $0.10 | | Claude Sonnet 5 | $2 | $10 | $0.20 | | Claude Opus 5 | $5 | $25 | $0.50 | The costs page says Sonnet handles most coding tasks well and costs less than Opus. For simple subagent tasks, it says to specify `model: haiku`. Set Sonnet as the project default in `.claude/settings.json`, so Opus runs only when somebody picks it: ```json { "model": "sonnet" } ``` Then give narrow, repetitive dbt work to a Haiku subagent. Writing column descriptions into `schema.yml` is one example, because the task reads two files and follows a fixed format. Save this file as `.claude/agents/dbt-yaml-writer.md`: ```markdown --- name: dbt-yaml-writer description: Writes column descriptions into dbt schema.yml files tools: Read, Edit model: haiku --- Write one description per column. Read only the model file and its schema.yml. ``` A lower price per token does not prove Haiku passes your tasks. On SWE-bench Verified, Haiku 4.5 resolved 66.6% of tasks at $0.33 each. Opus 4.5 resolved 76.8% at $0.75 each. Those are software tasks, not dbt tasks, so test Haiku on your own work before you route to it. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## Prompt Caching Cuts a Repeated 50,000-Token Prefix By 78.5% Anthropic bills a cache read at 0.1x the base input price. A 5-minute cache write costs 1.25x the base price, and a 1-hour write costs 2x the base price. Caching pays off after one read for the 5-minute cache, or after two reads for the 1-hour cache. Here is the arithmetic for one session on Claude Sonnet 5, at $2 per million input tokens. The session sends the same 50,000-token prefix on 10 turns, each within five minutes of the last. The prefix holds project instructions and a model list. | Setup | Arithmetic | Input cost for the prefix | | --- | --- | --- | | No cache | 10 × 50,000 × $2 per MTok | $1.00 | | 5-minute cache | 1 write at $2.50 per MTok, then 9 reads at $0.20 per MTok | $0.215 | Bar chart comparing the input cost of a 50,000-token prefix sent on 10 turns with Claude Sonnet 5. Without a cache the prefix costs $1.00. With a 5-minute cache it costs $0.215, split into a $0.125 cache write and $0.09 of cache reads *The same 10 turns with and without a 5-minute prompt cache, at Anthropic's September 2026 list price for Claude Sonnet 5.* Claude Code applies prompt caching automatically, so you do not switch it on. Your part is to keep the cached prefix stable. A cache matches a repeated prefix, so avoid editing project instructions in the middle of a session. The 1-hour cache costs more to write, and it suits sessions with long pauses between turns. ## A Project Manifest Hands the Agent the Graph Without Reading Every Model dbt already knows your dependency graph, so the agent does not need to rebuild it by reading files. [dbt parse](https://docs.getdbt.com/reference/commands/parse) reads and validates your project without a warehouse connection. `dbt ls` then prints only the models a change touches: ```bash dbt parse dbt ls --select +fct_orders+ --resource-type model --output name ``` Paste that list into the prompt and ask the agent to read only those models. Altimate Code reads the same graph through its `dbt_manifest` and `dbt_lineage` tools, and neither tool calls a model. Two Claude Code habits keep context small after that. The costs page recommends subagents for verbose work, because only a summary returns to the main conversation. Run `/clear` when you switch tasks, because a session nobody cleared keeps billing for old context. ## Check /Usage Every Session and the Console Every Week `/usage` shows the current token usage of a Claude Code session. Anthropic calls the figure an estimate and points to the Usage page in the Claude Console for authoritative billing. Background processes add a small amount, typically under $0.04 per session. Compare each engineer's daily figure with the $13 average and the $30 mark that 90% of users stay below. Then match the pattern to a fix: | What the usage shows | Likely cause | What to change | | --- | --- | --- | | A day above $30 | A long session that nobody cleared | Run `/clear` between tasks | | Opus on most requests | Opus left as the default model | Set Sonnet as the default, and use Haiku subagents | | A day at several times the average | Agent teams in plan mode | Save agent teams for tasks that need them | | Repeated schema and model reads | No lookup tool in the loop | Add deterministic tools and a dbt manifest | ## What Altimate Code Changes in the Bill and What It Leaves Alone Altimate Code is an open-source harness for data engineering, released under the MIT license. It runs inside Claude Code, because its `/configure-claude` command registers an `/altimate` command there. It adds the compiled SQL, lineage and dbt tools described above for [Claude Code data engineering](https://altimate.ai/products/altimate-code) work. [Altimate Code and Claude Code side by side](https://altimate.ai/comparisons/altimate-vs-claude-code) shows where each one fits. Altimate Code is model-agnostic. You can bring your own key for Anthropic, OpenAI, Amazon Bedrock, Azure OpenAI or Google Vertex AI. You can also run a local model through Ollama or LM Studio. The [Altimate pricing page](https://altimate.ai/pricing) listed three options on 16 September 2026: - **Community.** The plan costs $0 and includes a one-time grant of 10M tokens on Altimate's hosted model access. - **Pro.** The plan costs $29 per seat per month and includes 20M tokens per seat per month. - **Bring your own LLM.** Every plan can connect your own model key, and those calls use no Altimate tokens. With your own key, you pay your model provider directly. The CLI itself carries no license fee. Altimate Code also has limits on what it saves. Our [four blind spots experiment](https://altimate.ai/blog/4-blind-spots-of-general-coding-agents-in-data-engineering) ran every setup on Claude Sonnet 4.6, with one or two runs each: - **It does not lower token prices.** A call with your own key bills at your provider's rate. - **It does not make tool calls free inside a session.** The model still reads every tool result. - **It does not save on every task.** On the simplest model-creation tasks, the tools added about 25% to cost and fixed nothing. - **It saved 18.6% in one measured case.** That cut covers three ADE-Bench medium tasks that both setups passed. - **It can cost more per run and less per correct answer.** On a cross-warehouse migration, it cost about 6x a bare run. Bare Claude produced no working output in 5 attempts. We have not published a team-level saving for Claude Code sessions. [The hidden cost of AI coding tools](https://altimate.ai/blog/the-hidden-cost-of-ai-coding-tools-why-your-token-bill-lies) shows how context re-reads, cache writes and retries grow a session bill. To compare two tools, measure [cost per task instead of cost per token](https://altimate.ai/blog/cost-per-task-vs-dollars-per-million-tokens). ## Measure One Week of /Usage, Then Move One Check Off the Model Record `/usage` for each engineer for one week, and mark the days above $30. For those days, find the lookup the agent repeated most often. It is usually a schema read, a lineage question or a lint pass. Move that lookup to a deterministic tool or a dbt manifest, then compare the next week. To try the compiled tools inside Claude Code, run `npm install -g altimate-code` and then `/configure-claude`. The [Altimate Code product page](https://altimate.ai/products/altimate-code) lists the tools and the supported models. ## Frequently Asked Questions How much does Claude Code cost per developer? Anthropic's costs page puts the average at about $13 per developer per active day. That comes to $150 to $250 per developer per month. The same page says 90% of users stay below $30 per active day. Is Claude Code included in a Claude subscription? Yes. Anthropic says Claude Code is included in all paid plans and shares each plan's usage limits. Enterprise seats cost $20 per month, plus usage billed at API rates. Which Claude model should a data team use by default? Anthropic's costs page says Sonnet handles most coding tasks well and costs less than Opus. It recommends Haiku for simple subagent tasks. Pick Opus for a task only after a cheaper model fails it. Does prompt caching need any setup in Claude Code? No. Claude Code applies prompt caching automatically. Keep your project instructions stable during a session, because the cache matches a repeated prefix. Does Altimate Code replace Claude Code? No. The `/configure-claude` command registers an `/altimate` command inside Claude Code, so the two run together. Altimate Code adds compiled SQL, lineage and dbt tools to the session. How do I see what Claude Code spent today? Run `/usage` in the session for an estimate of current token usage. The Usage page in the Claude Console holds the authoritative billing figures. On this page 1. How a Data Team's Claude Code Cost Reaches $2,000 a Month 2. Every Schema and Model File the Agent Reads Adds to the Bill 3. Deterministic Tools Answer Schema, Lineage and Lint Questions Without Model Tokens 4. Haiku Costs One-Fifth of Opus 5 per Token for Simple Subagent Work 5. Prompt Caching Cuts a Repeated 50,000-Token Prefix By 78.5% 6. A Project Manifest Hands the Agent the Graph Without Reading Every Model 7. Check /Usage Every Session and the Console Every Week 8. What Altimate Code Changes in the Bill and What It Leaves Alone 9. Measure One Week of /Usage, Then Move One Check Off the Model From the product - [PowerUser Plugin for dbt Profile a slow dbt query and read its execution plan in plain English 4 min read](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan) - [Enterprise Platform Root cause analysis for a dbt, Tableau, or Snowflake workload spike 6 min read](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis) - [Enterprise Platform Reduce Snowflake idle warehouse spend with Altimate Lite 6 min read](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) - [Enterprise Platform Snowflake Cortex AI cost breakdown by service, user, and model 6 min read](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown) - [Enterprise Platform Attribute Snowflake cost to the team that owns it 4 min read](https://altimate.ai/product-highlights/snowflake-cost-alerts-by-team) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # How Deterministic Tools Cut LLM Spend on dbt Changes URL: https://altimate.ai/blog/schema-validation-costs-0-tokens-how-deterministic-tools-cut-llm-spend Schema validation, lineage and lint can run as compiled code with no model call. See how deterministic tools cut LLM spend on one dbt change, and what remains. ## Schema Validation Costs 0 Tokens: How Deterministic Tools Cut LLM Spend Schema validation, lineage and lint can run as compiled code with no model call. See how deterministic tools cut LLM spend on one dbt change, and what remains. Atharva Shah 2026-09-01 2026-09-16 10 min read On this page 7 sections 1. One Column Rename Shows Where Deterministic Tools Cut LLM Spend 2. A Model Pays for Schema Context on Every Turn 3. Six Checks on the Rename Run Without a Model Call 4. The Validator Returns a Short Result the Model Can Act On 5. Zero Tokens Means Zero LLM Tokens, and Other Costs Remain 6. Four Questions About the Rename Still Need the Model 7. Move Your Most Repeated Check Out of the Prompt First tl;dr A data agent can answer many questions about a dbt change without asking the model. A parser, a schema validator, a lineage walker and a linter each return the exact answer. None of them spends LLM tokens to compute it. That is how deterministic tools cut LLM spend: the model stops carrying the schema and the manifest in its context just to answer yes or no. The saving has limits. The model still pays tokens to call a tool and read its result. The tools still use CPU and warehouse metadata queries. No tool can tell you whether the change was the right one. You rename one column in a dbt staging model, and your coding agent has to confirm that nothing downstream breaks. The model can answer that question if you give it enough text. That text is the warehouse schema, the dbt manifest and every downstream SQL file. You pay for all of it as input tokens, again on each turn that keeps it in context. A compiled tool answers the same question from the same files and sends the model a few lines back. That swap is the most direct way to cut LLM spend on data work. The model keeps the parts of the job that need judgment, and the checks with one right answer move to code. This post follows one real change through six checks. For each check, it names the Altimate Code tool that runs it. It also names the text the model would need in context to answer without that tool. Prices are Anthropic list prices as of September 2026. Two columns. The left column lists what a model needs in context to check a column rename: the 684,396-byte manifest.json, the schema of each referenced table, and every downstream model file. The right column shows the result the validator returns: a title reading Validate INVALID and one finding with category missing\_column *The model needs the whole project in context to answer by reading. The validator sends back a few lines.* Power User for dbt is an open source VS Code extension that adds column lineage, autocomplete, AI docs, query previews and health checks to your dbt project. [Explore Power User for dbt](https://altimate.ai/products/dbt-power-user) ## One Column Rename Shows Where Deterministic Tools Cut LLM Spend The example project is the jaffle-shop sample that ships in the [altimate-code repo](https://github.com/AltimateAI/altimate-code/tree/024e80004be3f113b8136fb691474ce1cbca4cf9/packages/opencode/sample-projects/jaffle-shop-duckdb). It runs on DuckDB and holds four models, two seeds and 13 tests. The mart model `customers` sums `o.amount` from `stg_orders`. The change renames `amount` to `amount_usd` in `stg_orders`: ```sql -- models/staging/stg_orders.sql, after the change select id as order_id, customer_id, order_date, amount as amount_usd from {{ ref('raw_orders') }} ``` The `customers` model still reads `o.amount`, so the project is now broken. Six questions decide whether the change is safe to merge: - The agent has to know which models depend on `stg_orders`. - The agent has to confirm that `customers` still references columns that exist. - The agent has to trace which output columns read the renamed column. - The agent has to check the edited SQL for known anti-patterns. - The agent has to prove that the rows did not change. - The agent has to check whether the change exposes personal data. Each of these questions has one correct answer. None of them needs a model to compute it. ## A Model Pays for Schema Context on Every Turn Anthropic's [Claude Code cost guide](https://code.claude.com/docs/en/costs) says that token costs "scale with context size" on every request. A model that answers a schema question by reading needs the schema in its context window. Anthropic bills that text as input on each request that carries it, until the session drops or compacts it. The table below prices 100,000 tokens of context for one turn. The rates come from [Anthropic's pricing page](https://platform.claude.com/docs/en/about-claude/pricing) as of September 2026. A cache read costs 0.1x the base input price, and the first write to the 5-minute cache costs 1.25x. | Model | Input per million tokens | 100,000 tokens, uncached | 100,000 tokens, cache read | | --- | --- | --- | --- | | Claude Haiku 4.5 | $1 | $0.10 | $0.01 | | Claude Sonnet 5 | $2 | $0.20 | $0.02 | | Claude Opus 5 | $5 | $0.50 | $0.05 | Multiply the per-turn figure by the number of turns that keep the context loaded. Caching lowers the rate, but the cost still grows with every turn. The same guide warns that compaction has a cost too: "/compact reads the conversation it summarizes." A bar chart of the cost of 100,000 tokens of context for one turn. Claude Haiku 4.5 costs $0.10 uncached and $0.01 as a cache read. Claude Sonnet 5 costs $0.20 and $0.02. Claude Opus 5 costs $0.50 and $0.05 *List prices as of September 2026. A tool that answers from files keeps this text out of the context window.* ## Six Checks on the Rename Run Without a Model Call Each subsection below names the check, the tool, and the text the model would need to answer without that tool. The tool names come from the Altimate Code tool registry at commit `024e800`. ### 1\. Parse the Project Instead of Reading the Manifest `dbt_manifest` parses `target/manifest.json` and returns a table of models, dependencies and columns. For the sample project, that table has four model rows. A model that answers by reading needs the manifest itself. The sample's `manifest.json` is 684,396 bytes for four models. Most of that size is macro definitions: the file carries 475 macros. You can check a larger project yourself with `wc -c target/manifest.json`. If you only need to confirm that the project still parses, [`dbt parse`](https://docs.getdbt.com/reference/commands/parse) does that without a warehouse connection. It "parses and validates the contents of your dbt project." ### 2\. Validate the SQL Against the Schema The `altimate_core_validate` tool checks SQL syntax and schema references. It runs in the Rust engine, `altimate-core`. It takes the schema as a file path or as an inline table map. The inline shape is `{ table: { column: TYPE } }`. For `customers`, the check needs only the columns of `stg_customers` and `stg_orders`. The model alternative needs those same column lists in context, plus the compiled SQL. It then has to reason over them without error. On a larger project, the agent must first find the referenced tables. Until it finds them, it loads more schemas than the check needs. ### 3\. Trace Column Lineage From the Parsed SQL The `altimate_core_column_lineage` tool maps how columns flow from source tables to the query output. Its description says it runs fully offline in the native engine. For the rename, the question is which output columns of `customers` read `stg_orders.amount`. In the sample SQL, that column is `total_amount`. A model needs every downstream SQL file in context to answer the same question. It then has to follow each CTE and alias by reading. For column-level lineage across more models, see [the column-level lineage use cases](https://altimate.ai/blog/column-level-lineage-use-cases). ### 4\. Lint for Known Anti-Patterns The `sql_analyze` tool checks SQL for anti-patterns and safety issues. It does not run the query. The headless form is the `altimate-code check` command, which runs `lint` and `safety` by default. The [check command docs](https://help.altimate.ai/code/usage/check/) say it needs no model provider and no API key. The model alternative needs the rule definitions in the prompt, plus the SQL. It also returns a paragraph where the linter returns a rule ID and a line number. ### 5\. Diff the Rows Before and After `data_diff` compares two tables or two queries row by row. For a rename, pass two queries keyed on `order_id`. The first selects `amount` from the old model, and the second selects `amount_usd as amount` from the new one. List `amount` in `extra_columns`, because a query diff compares only key columns without it. A model has no way to answer this question by reading. It has to run queries, read the results into context and compare them. That path costs tokens in proportion to the rows it reads. Power User for dbt is an open source VS Code extension that adds column lineage, autocomplete, AI docs, query previews and health checks to your dbt project. [Explore Power User for dbt](https://altimate.ai/products/dbt-power-user) ### 6\. Scan for Personal Data The `altimate_core_classify_pii` tool classifies schema columns by name patterns and data types. The `schema_detect_pii` tool scans column names across an indexed warehouse. The model alternative needs every column name in context. A rename can move a personal field under a new name, so run the scan on each schema change. ## The Validator Returns a Short Result the Model Can Act On The validator's reply decides how many tokens the next turn costs. `altimate_core_validate` returns a title, a small metadata object and a few lines of text. The shape below comes from [`altimate-core-validate.ts`](https://github.com/AltimateAI/altimate-code/blob/024e80004be3f113b8136fb691474ce1cbca4cf9/packages/opencode/src/altimate/tools/altimate-core-validate.ts). The engine writes the message text, and its exact wording varies by version. ```json { "title": "Validate: INVALID", "metadata": { "success": true, "valid": false, "has_schema": true, "findings": [{ "category": "missing_column" }] }, "output": "Validation failed:\n\n • \n at line 6" } ``` Two fields matter to you. The `success: true` field means the engine ran, so an invalid query counts as a finding. The `has_schema` field tells the model whether table and column checks ran at all. With no schema, the tool still checks syntax and dialect. Its output then says that it skipped the existence checks. The headless CLI reports the same finding as structured JSON. Each finding carries `file`, `line`, `rule`, `severity`, `message` and an optional `suggestion`. An agent that reads this result knows to edit line 6 of `customers.sql`. ## Zero Tokens Means Zero LLM Tokens, and Other Costs Remain The "0 tokens" in the title covers one thing only: the check itself makes no LLM call. Four costs remain, and you should plan for each. - **The tool call costs tokens.** The model writes the SQL into the call as output tokens, and it reads the result back as input. - **Tool definitions cost context.** The Altimate Code docs say sending about 78 tool definitions on every turn "floods the context window." The opt-in setting `ALTIMATE_TOOL_RETRIEVAL=1` trims that set per turn. - **The engine uses CPU.** [The Correctness Layer post](https://altimate.ai/blog/the-correctness-layer-in-ade) reports that the benchmark battery "validates 1,000 SQL queries in 30 ms." - **Warehouse checks use the warehouse.** The `schema_index` tool calls `listTables` and `describeTable` on your connection. It stores the result in `~/.altimate-code/schema-cache.db`. The `data_diff` tool runs its comparison queries in the warehouse too. One more cost is easy to miss. The `data_diff` tool description warns that up to 5 sample diff rows appear in its output. Those rows go to the LLM provider. For regulated data, the description says to use `algorithm='profile'`. That mode returns statistics only. ## Four Questions About the Rename Still Need the Model A compiled check answers whether the SQL is consistent with the schema. It cannot answer whether the change is right. These four questions stay with the model or with you: - The model can judge whether `amount_usd` is the right name for your team's conventions. - The model can decide whether `customers` should follow the rename or keep its output column name. - The model can explain the change in the pull request so a reviewer understands the intent. - You decide whether the values in `amount` really are US dollars. That last question is a data question that no validator settles. The validator checks references and syntax. It does not check whether a join key is unique. A `unique` test in dbt, or a query against the data, answers that. Two more limits apply. The `schema_detect_pii` tool scans column names, so it misses personal data stored under a neutral name. And the lint rules have a published benchmark only on synthetic Snowflake queries. The split still saves money, because the model now reads a short finding instead of the project. [Deterministic tooling for AI agents](https://altimate.ai/blog/deterministic-tooling-vs-llm-only-data-agents) covers the wider argument for that split. [The correctness layer](https://altimate.ai/blog/the-correctness-layer-deterministic-validation-outside-the-llm-loop) shows the three points in an agent loop where these checks run. [The hidden cost of AI coding tools](https://altimate.ai/blog/the-hidden-cost-of-ai-coding-tools-why-your-token-bill-lies) shows how re-sent context grows a Claude Code bill. ## Move Your Most Repeated Check Out of the Prompt First Pick the check your agent runs most often by reading files. For most dbt teams, that check asks whether a column exists. Give it to a validator with an explicit schema. Then compare the input token count on your next session. You can run the same checks headless before you change any agent setup. Compile first, because the checks parse SQL and a raw model still holds Jinja. `dbt compile` needs a warehouse connection. ```bash npm install -g altimate-code dbt compile altimate-code check "target/compiled/**/*.sql" --checks lint,validate,safety --schema schema.yml --format json ``` The [sql-review skill](https://altimate.ai/skills/sql-review) wraps these checks in a review workflow. [Altimate Code](https://altimate.ai/products/altimate-code) runs them inside the agent loop. For the team-wide version, see the [agentic data engineering platform](https://altimate.ai/platform). ## Frequently Asked Questions Does schema validation really cost 0 tokens? The check itself makes no LLM call, so it costs 0 LLM tokens. The agent still spends tokens to write the tool call and read the result. The engine uses CPU, and a schema lookup may query your warehouse. How do deterministic tools cut LLM spend? They answer questions with one correct answer from files, so the model does not carry those files in context. The model reads a short result instead. Your provider bills context on every turn that carries it. So the saving grows with session length. Can I validate SQL against a schema without an LLM? Yes. Run `altimate-code check --checks validate --schema schema.yml`. The command needs no model provider or API key. Without a schema, it still checks syntax, but it skips table and column checks. What can a schema validator not catch? It cannot tell you whether a rename matches your conventions or whether values are correct. It does not check whether a join key is unique. A dbt test or a data diff covers those questions. Does prompt caching make the schema-in-context approach cheap enough? Caching lowers the rate to 0.1x the base input price for a cache read. The cost still grows with each turn. The first cache write also costs 1.25x. The model must still reason over the schema correctly. On this page 1. One Column Rename Shows Where Deterministic Tools Cut LLM Spend 2. A Model Pays for Schema Context on Every Turn 3. Six Checks on the Rename Run Without a Model Call 4. The Validator Returns a Short Result the Model Can Act On 5. Zero Tokens Means Zero LLM Tokens, and Other Costs Remain 6. Four Questions About the Rename Still Need the Model 7. Move Your Most Repeated Check Out of the Prompt First From the product - [PowerUser Plugin for dbt Profile a slow dbt query and read its execution plan in plain English 4 min read](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan) - [Enterprise Platform Root cause analysis for a dbt, Tableau, or Snowflake workload spike 6 min read](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Deterministic Validation Outside the LLM Loop URL: https://altimate.ai/blog/the-correctness-layer-deterministic-validation-outside-the-llm-loop Deterministic validation can check a data agent's work at three points in its loop. See what each point checks, what it returns, how the loop ends, and a CI job. ## The Correctness Layer: Deterministic Validation Outside the LLM Loop Deterministic validation can check a data agent's work at three points in its loop. See what each point checks, what it returns, how the loop ends, and a CI job. Atharva Shah 2026-07-03 2026-09-16 10 min read On this page 8 sections 1. Deterministic Validation Sits at Three Points in an Agent Loop 2. The Harness Classifies Every Statement Before It Reaches the Warehouse 3. A Completion Gate Blocks the Agent's Done Until Tests Pass 4. The Loop Ends on a Pass or on a Spent Retry Budget 5. CI Runs the Same Checks With No Model in the Job 6. The Result Format Decides Whether the Model Can Fix the Error 7. Deterministic Validation Has Five Limits You Should Plan Around 8. Add the CI Check Before You Enable the Completion Gate tl;dr Deterministic validation works best at three points in a data agent's loop. The first point is inside the loop, before any SQL reaches the warehouse. The second point is at completion, when the agent says it is done. The third point is in CI, before a person merges the change. All three points return a structured finding that the model can act on. The loop ends when the checks pass or when a fixed retry budget runs out. CI then applies the same checks again with no model in the job. You build or buy a data agent, and you have to decide where its checks run. A check the model calls is one design. A check the harness runs without asking the model is another. A check in CI that no model can skip is a third. Each position catches a different failure. A complete correctness layer uses all three positions. Deterministic validation means code that returns the same result for the same input on every run. A parser, a schema validator and a dbt test all qualify. A second model that reviews the first model's SQL does not qualify, because its answer can change between runs. This post is for the engineer who wires the loop in an agentic data engineering stack. It shows the three positions, what each one returns to the model, and how the loop terminates. The examples use Altimate Code. Its source is public, so you can check each behavior in the code. Three panels in a row, one per checkpoint, below a box reading model drafts SQL and calls tools. Panel one, before execution, classifies each sql\_execute call and refuses DROP DATABASE, DROP SCHEMA and TRUNCATE. Panel two, at completion, runs 7 dbt validators with up to 3 retries. Panel three, before merge, runs dbt parse, dbt compile and altimate-code check in CI with no model. A note below says the loop ends when validators pass or the retry budget runs out *Three checkpoints for one agent session. The first two sit inside the loop, and the third runs after it.* ## Deterministic Validation Sits at Three Points in an Agent Loop Three positions cover the path from a draft to a merged change. The three positions answer different questions. | Position | Question it answers | Altimate Code mechanism | Model involved | | --- | --- | --- | --- | | Before execution | Is this statement safe and valid to run? | `sql_execute` classifier, `altimate_core_validate` | The model calls the tool | | At completion | Did the agent really finish the job? | Completion validators | The harness runs them | | Before merge | Does the changed SQL pass the team's rules? | `altimate-code check` in CI | None | [The correctness layer inside Altimate Code](https://altimate.ai/blog/the-correctness-layer-in-ade) explains the three software layers behind these checks. That post covers how the dispatcher routes a call to compiled Rust. This post covers when each check fires and what happens next. Power User for dbt is an open source VS Code extension that adds column lineage, autocomplete, AI docs, query previews and health checks to your dbt project. [Explore Power User for dbt](https://altimate.ai/products/dbt-power-user) ## The Harness Classifies Every Statement Before It Reaches the Warehouse The first checkpoint runs on each tool call that touches data. In Altimate Code, the `sql_execute` tool classifies the SQL before it runs. The classifier in [`sql-classify.ts`](https://github.com/AltimateAI/altimate-code/blob/024e80004be3f113b8136fb691474ce1cbca4cf9/packages/opencode/src/altimate/tools/sql-classify.ts) uses the engine's AST-based statement types. The classifier applies three rules: - It treats only a plain query as a known safe read. - It routes any write through the `sql_execute_write` permission, which asks you first. - It refuses `DROP DATABASE`, `DROP SCHEMA` and `TRUNCATE` in every case. For those three statements, the tool throws a fixed error that no permission setting lifts. No prompt changes that result, because the model never makes the decision. If the native engine fails to load, a regex fallback treats every statement it cannot prove is a read as a write. Agent modes add a second boundary. The [agent modes reference](https://help.altimate.ai/code/data-engineering/agent-modes/) lists Analyst as a read-only mode. In Analyst mode, the classifier denies INSERT, UPDATE, DELETE and DROP outright, with no approval prompt. At the same checkpoint, the model can call `altimate_core_validate` on a draft before it runs anything. That tool checks syntax and schema references against a schema file or an inline table map. A draft that names a missing column fails in the engine, before the warehouse bills a single second. [A worked example of a zero-token check](https://altimate.ai/blog/schema-validation-costs-0-tokens-how-deterministic-tools-cut-llm-spend) runs this validator on one column rename. The model decides when to call the validator. So this checkpoint catches errors only when the agent's instructions or skills tell it to validate. The second checkpoint does not depend on that choice. ## A Completion Gate Blocks the Agent's Done Until Tests Pass The second checkpoint fires when the model declares a clean stop. Altimate Code calls these checks [completion validators](https://help.altimate.ai/code/data-engineering/validators/). The docs state that the validators "are not visible to the agent." The harness runs them after the model's last message ends with a stop. At commit `024e800`, the harness registers seven dbt validators. They run in this order, cheapest first: 1. `dbt-nothing-built` 2. `dbt-build-green` 3. `dbt-deliverable-names` 4. `dbt-incremental-config` 5. `dbt-dialect-guard` 6. `dbt-schema-verify` 7. `dbt-tests-pass` The docs describe the last two in detail. The `dbt-schema-verify` validator compares each modified model's columns with its `schema.yml` spec. The `dbt-tests-pass` validator runs the dbt tests of each modified model. The docs say it "refuses to terminate if any model's tests fail or error." A failed validator does not end the session. The harness writes one synthetic user message that lists every failure, and the model gets another turn. The message format comes from [`prompt.ts`](https://github.com/AltimateAI/altimate-code/blob/024e80004be3f113b8136fb691474ce1cbca4cf9/packages/opencode/src/session/prompt.ts): ```text [altimate-validator: dbt-schema-verify] ``` The docs give an example reason: "2 of 3 models you edited have a column-shape mismatch against schema.yml: foo, bar". The model reads the model names and the fix hint, then edits those files. ## The Loop Ends on a Pass or on a Spent Retry Budget A correctness layer needs an exit rule, or a model that cannot fix a failure loops forever. Altimate Code bounds the loop with `ALTIMATE_VALIDATORS_MAX_RETRIES`, which defaults to 3. The simplified loop below follows `prompt.ts` and `sql-execute.ts`: ```text retries = 0 loop: step = model.next(messages) # draft SQL, pick tools for call in step.tool_calls: if call.tool == "sql_execute": kind = classify(call.sql) # AST statement types if kind is DROP DATABASE, DROP SCHEMA or TRUNCATE: refuse if kind is write: ask for sql_execute_write permission messages.append(run_tool(call)) # native handler, no model call if step.finish == "stop" and no tool calls are outstanding: if validators_on: # ENABLED=1 or SHADOW=1 failures = [v for v in validators if v.applies() and not v.check().ok] if failures and enforcement_on and retries < MAX_RETRIES: messages.append(user_turn(format(failures))) retries = retries + 1 continue break ``` The loop has three exits: - **All validators pass.** The session ends, and the agent's claim of done holds for every applied check. - **The retry budget runs out.** The harness emits a `validator_retries_exhausted` event and marks the session completed with unresolved failures. - **Validators are off.** The session ends when the model stops, and only the first and third checkpoints protect you. The second exit is the one to plan for. A spent budget means the agent could not fix its own work. The session record says so, and a person has to take over. Power User for dbt is an open source VS Code extension that adds column lineage, autocomplete, AI docs, query previews and health checks to your dbt project. [Explore Power User for dbt](https://altimate.ai/products/dbt-power-user) ## CI Runs the Same Checks With No Model in the Job The third checkpoint runs after the session, on the pull request. Nothing in the job calls a model, so the result depends only on the files. The [check command docs](https://help.altimate.ai/code/usage/check/) say `altimate-code check` needs no model provider and no API key. It exits 1 when findings reach the `--fail-on` level. The GitHub Actions job below runs three checks in order: parse, compile, then the Altimate Code check. It uses the DuckDB adapter, like the Altimate Code sample project. Swap in your own adapter and credentials. ```yaml name: dbt deterministic checks on: [pull_request] jobs: checks: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Install dbt and Altimate Code run: | pip install dbt-duckdb npm install -g altimate-code - name: Parse the project run: dbt parse - name: Compile models to plain SQL run: dbt compile - name: Run deterministic checks run: | altimate-code check "target/compiled/**/*.sql" \ --checks lint,validate,safety \ --schema schema.yml \ --format json --fail-on error ``` The steps run in this order for a reason. The [`dbt parse`](https://docs.getdbt.com/reference/commands/parse) command validates the project without a warehouse connection. So a broken `ref` fails first, before any step connects to the warehouse. The `dbt compile` command then renders Jinja into plain SQL, which the engine can parse. The last step lints, validates and scans the compiled files. The JSON output ends with a `summary` object that holds `errors`, `warnings` and `pass`. A later step can read `summary.pass` with `jq` and post the counts on the pull request. The check docs show that pattern with `actions/github-script`. For a faster loop on your own machine, the docs show a pre-commit hook: ```yaml repos: - repo: local hooks: - id: altimate-sql-check name: SQL Check entry: altimate-code check --fail-on warning language: system types: [sql] pass_filenames: true ``` Point the hook at compiled SQL if your models use Jinja, because the engine parses SQL text. ## The Result Format Decides Whether the Model Can Fix the Error A check helps the loop only if the model can act on its result. Each checkpoint in Altimate Code returns a short, structured result. - **The validator tool** returns `valid`, `has_schema` and a list of findings with a category such as `missing_column`. - **The completion gate** returns `ok`, a `reason` that names the failing models, and a `fixHint`. - **The CI check** returns JSON findings with `file`, `line`, `rule`, `severity`, `message` and an optional `suggestion`. Each result names a location and a rule. A model that reads "line 6, missing column" edits line 6. A model that reads a reviewer's paragraph has to guess which line the paragraph meant. The CI result is the same for the model and for a person, so a reviewer can read the finding the agent read. [Five reasons deterministic tooling beats an LLM-only stack](https://altimate.ai/blog/deterministic-tooling-vs-llm-only-data-agents) covers why a fixed answer matters for trust. [Why human review stops at scale](https://altimate.ai/blog/you-are-the-trust-layer-managing-data-engineering-ai-agents-at-scale) covers the reviewer's side of the same problem. [Why agents fail after merge](https://altimate.ai/blog/why-ai-agents-fail-on-production-data-stacks) covers the failures that appear only once a change reaches production. ## Deterministic Validation Has Five Limits You Should Plan Around A correctness layer catches only what its rules describe. These five limits come from the Altimate Code source and docs. **The completion gate is off by default.** Enforcement needs `ALTIMATE_VALIDATORS_ENABLED=1`, and shadow mode needs `ALTIMATE_VALIDATORS_SHADOW=1`. A comment in `prompt.ts` warns against enforcement beyond a shadow soak. It says duplicated heuristics already caused a blocking false positive. **The completion gate costs time.** The docs put each per-model subprocess at 5 to 30 seconds. Five touched models take about 1 to 2 minutes after the agent says done. **The completion gate misses some models.** It scans only `.sql` files under a `models/` folder. Python models and custom `model-paths` are outside its scan. **The anti-pattern benchmark is synthetic.** The [benchmark file](https://github.com/AltimateAI/altimate-code/blob/024e80004be3f113b8136fb691474ce1cbca4cf9/experiments/BENCHMARKS.md) reports F1 of 1.00 on 19 rules across 1,077 queries, with 0 false positives. A script generated those queries from a fixed seed, in the Snowflake dialect only. The file says 100% on synthetic queries "does not guarantee 100% on production SQL." **Rules check shape, not intent.** The docs show policy rules as regex patterns over SQL text. A validator confirms that a column exists, not that the join returns the right grain. A dbt test on your real data covers that gap, which is why `dbt-tests-pass` runs last. The CI job also reads files, not data. It validates against the schema file you pass, so a stale `schema.yml` gives a stale result. ## Add the CI Check Before You Enable the Completion Gate Start with the checkpoint that has no model in it. Add the CI job above to one dbt repository, with `--fail-on error`. Read the findings on several pull requests before you tighten the threshold. Then run the completion gate in shadow mode with `ALTIMATE_VALIDATORS_SHADOW=1`. Shadow mode runs every validator and records whether it would have fired, without blocking the session. Each run emits a `validator_check` event with the validator name, the `ok` result and the retry count. Turn on enforcement only when those events show few false positives. For a whole team, [the agentic data engineering platform](https://altimate.ai/platform) adds shared context and governance. To try the checks today, install [Altimate Code](https://altimate.ai/products/altimate-code) and run `altimate-code check` on your compiled models. Add [a lineage diff before merge](https://altimate.ai/skills/lineage-diff) when column-level changes matter to your reviewers. ## Frequently Asked Questions What is deterministic validation for a data agent? It is code that checks the agent's SQL and returns the same result for the same input on every run. A parser, a schema validator and a dbt test all count. A second model that reviews the SQL does not count, because its answer can change between runs. Where should the correctness layer sit in an agent loop? Put it at three points. The first is before any statement reaches the warehouse, and the second is when the agent declares done. The third is CI, before a person merges the change. How does the validation loop end? It ends when every validator passes or when the retry budget runs out. Altimate Code defaults that budget to 3 retries. A spent budget marks the session completed with unresolved failures, so a person takes over. Can I run these checks in CI without an LLM? Yes. The `altimate-code check` command needs no model provider or API key. It writes JSON findings and exits 1 when findings reach the `--fail-on` level. Is a 100% anti-pattern benchmark proof of production accuracy? No. The published result covers 19 rules on 1,077 synthetic queries in the Snowflake dialect. The benchmark file itself says that result does not guarantee the same accuracy on production SQL. On this page 1. Deterministic Validation Sits at Three Points in an Agent Loop 2. The Harness Classifies Every Statement Before It Reaches the Warehouse 3. A Completion Gate Blocks the Agent's Done Until Tests Pass 4. The Loop Ends on a Pass or on a Spent Retry Budget 5. CI Runs the Same Checks With No Model in the Job 6. The Result Format Decides Whether the Model Can Fix the Error 7. Deterministic Validation Has Five Limits You Should Plan Around 8. Add the CI Check Before You Enable the Completion Gate From the product - [PowerUser Plugin for dbt Profile a slow dbt query and read its execution plan in plain English 4 min read](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan) - [Enterprise Platform Root cause analysis for a dbt, Tableau, or Snowflake workload spike 6 min read](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis) - [Enterprise Platform Reduce Snowflake idle warehouse spend with Altimate Lite 6 min read](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) - [Enterprise Platform Databricks cost breakdown, from one bill down to one warehouse 4 min read](https://altimate.ai/product-highlights/databricks-cost-breakdown-workspace-sku-cluster) - [Enterprise Platform Snowflake Cortex AI cost breakdown by service, user, and model 6 min read](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Snowflake Adaptive Compute vs Auto-Tune After GA URL: https://altimate.ai/blog/adaptive-compute-vs-auto-tune-optimizing-snowflake-warehouses Snowflake Adaptive Compute is now GA. Let's explore what its two parameters and per-query billing change, and what auto-tuning a standard warehouse still does. ## Adaptive Compute vs Auto-Tune: Optimizing Snowflake Warehouses Snowflake Adaptive Compute is now GA. Let's explore what its two parameters and per-query billing change, and what auto-tuning a standard warehouse still does. Atharva Shah 2026-07-24 2026-09-16 11 min read On this page 10 sections 1. Snowflake Adaptive Compute Replaces Four Settings With Two Parameters 2. Conversion Is Online but Bills Twice While Old Queries Finish 3. Adaptive Warehouses Bill per Query in QUERY\_METERING\_HISTORY 4. Auto-Tuning a Standard Warehouse Means Moving Suspend and Cluster Settings 5. Altimate Lite Suspends Idle Warehouses and Lowers the Cluster Ceiling 6. The Enterprise Platform Adds Downsizing, Query Cost and Rewrite Proposals 7. The Comparison Table Rebuilt From the Docs and the Code 8. Neither Path Fixes a Query That Reads Too Much 9. Which Warehouses to Convert and Which to Keep and Tune 10. Convert One Warehouse and Measure Credits per Query tl;dr Snowflake Adaptive Compute became generally available on AWS on 16 June 2026. An Adaptive Warehouse has no size, no cluster counts and no suspend policy. You set a per-query performance ceiling and a throughput multiplier, and Snowflake bills each query. Auto-tuning is the other path. You keep a standard Gen1 or Gen2 warehouse and let software adjust its suspend timing and cluster counts as the workload moves. Altimate Lite writes two settings on a standard warehouse. It suspends the warehouse when it goes idle, and it lowers the maximum cluster count when demand drops. It never changes the size or your SQL. Convert the unpredictable and fragmented warehouses to Adaptive. Keep and tune the ones Snowflake cannot convert, the mostly-HTAP ones, and the ones where you want a fixed size. Neither path fixes a query that scans a table it does not need. If you run Snowflake warehouses, you have heard the pitch: "Convert to Snowflake Adaptive Compute and stop choosing between Small and X-Large." The pitch is mostly accurate. It also leaves out which warehouses Snowflake will not convert and what the bill looks like afterwards. Snowflake Adaptive Compute replaces the fixed-size warehouse with an engine that sizes each query on its own. Snowflake's release note of 16 June 2026 made it generally available on AWS. The user guide now lists select AWS, Azure and GCP regions. We found no Azure or GCP release note, so check your own region before you plan a migration. The GA release changes the comparison with auto-tuning a standard warehouse. It brings two new parameters, per-query billing and a list of warehouses that cannot convert. Our [2025 preview-era take](https://altimate.ai/blog/adaptive-compute-vs-auto-tune-a-practical-guide-to-optimizing-snowflake-warehouses) described settings that the GA release renamed. Three columns compare which warehouse settings each layer writes. Snowflake Adaptive Compute owns size, cluster count, query acceleration and suspend. Altimate Lite writes only auto-suspend and max cluster count on a standard warehouse. The Enterprise Platform adds a one-step downsize and per-query cost. A bottom band marks query shape as owned by none of the three *Each layer writes a different set of warehouse settings, and none of the three rewrites your SQL on its own.* Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## Snowflake Adaptive Compute Replaces Four Settings With Two Parameters An Adaptive Warehouse takes away four settings you used to manage by hand. [Snowflake's Adaptive Compute guide](https://docs.snowflake.com/en/user-guide/warehouses-adaptive) names them: warehouse size, multi-cluster settings, Query Acceleration Service settings, and suspend and resume policies. All jobs on all Adaptive Warehouses in one account run on a shared pool of compute. Two parameters take their place: - **MAX\_QUERY\_PERFORMANCE\_LEVEL** sets the upper bound of performance for any single query. The default is XLARGE, and valid values run from XSMALL to X4LARGE. Snowflake says the value does not map to a specific hardware configuration. - **QUERY\_THROUGHPUT\_MULTIPLIER** is a non-negative integer that controls burst capacity. The default is 2. A value of 0 means unlimited throughput with no cap. You create one directly, or you change the type of an existing warehouse: ```sql -- A new Adaptive Warehouse with explicit limits CREATE ADAPTIVE WAREHOUSE analytics_adaptive_wh WITH MAX_QUERY_PERFORMANCE_LEVEL = LARGE QUERY_THROUGHPUT_MULTIPLIER = 2; -- Convert an existing standard warehouse ALTER WAREHOUSE adhoc_wh SET WAREHOUSE_TYPE = 'ADAPTIVE'; -- Lower the per-query ceiling later ALTER WAREHOUSE adhoc_wh SET MAX_QUERY_PERFORMANCE_LEVEL = MEDIUM; -- Convert back if the result disappoints you ALTER WAREHOUSE adhoc_wh SET WAREHOUSE_TYPE = 'STANDARD'; ``` Standard properties stop working after the change. You cannot set WAREHOUSE\_SIZE, MIN\_CLUSTER\_COUNT, MAX\_CLUSTER\_COUNT or SCALING\_POLICY on an Adaptive Warehouse. A script that resizes warehouses on a schedule will fail against a converted one, so find those scripts first. The 2025 preview described a Warehouse Credit Limit and a Target Statement Size. Neither name appears in the GA guide. The preview used `SET TYPE` to convert a warehouse. The GA release uses `SET WAREHOUSE_TYPE`. ## Conversion Is Online but Bills Twice While Old Queries Finish Snowflake runs a conversion without downtime. The guide calls it an online operation. Queries that were already running finish on the old compute, and new queries go to the Adaptive engine. You pay for both sets of compute while the old queries run. Convert a warehouse during a quiet window, not while a three-hour batch job is running on it. Snowflake computes the two new parameters for you at conversion. It derives them from four inputs: the old size, MAX\_CLUSTER\_COUNT, the Query Acceleration Service scale factor and the warehouse generation. Read the derived values after the change, because they become your new ceiling. Four kinds of warehouse cannot convert yet. Snowflake blocks conversion to or from X5Large and X6Large, and to or from Snowpark-optimized and interactive warehouses. The account also needs Enterprise Edition or higher. Suspend behaves differently too. You disable an Adaptive Warehouse, and new jobs sent to it are rejected. A resource monitor with a suspend action disables an Adaptive Warehouse immediately, per the [resource monitor docs](https://docs.snowflake.com/en/user-guide/resource-monitors). ## Adaptive Warehouses Bill per Query in QUERY\_METERING\_HISTORY Adaptive Warehouses use a query-based billing model. Snowflake charges nothing to create one, and charges start when the first query runs. The usage appears under COMPUTE in your usage statements, in virtual warehouse credits. Snowflake's cost claim is that Adaptive Warehouses run significantly more queries at a similar cost to Gen2. The guide gives no number for that claim. Measure your own warehouse before and after. Per-query cost comes from a new view. [QUERY\_METERING\_HISTORY](https://docs.snowflake.com/en/sql-reference/account-usage/query_metering_history) returns per-query credits for Adaptive Warehouses over the last 365 days. It writes one row per query per metering hour, with up to one hour of latency. This query ranks last week's most expensive query shapes on one converted warehouse: ```sql SELECT query_parameterized_hash, ANY_VALUE(query_id) AS sample_query_id, COUNT(DISTINCT query_id) AS runs, SUM(credits_used_compute) AS compute_credits, SUM(credits_used_cloud_services) AS cloud_services_credits, SUM(credits_used) AS total_credits FROM SNOWFLAKE.ACCOUNT_USAGE.QUERY_METERING_HISTORY WHERE warehouse_name = 'ADHOC_WH' AND query_start_time >= DATEADD(day, -7, CURRENT_DATE()) GROUP BY query_parameterized_hash ORDER BY total_credits DESC LIMIT 20; ``` Group by `query_parameterized_hash`, because one dashboard query with different filter values then counts as one shape. The docs warn that queries with very low or zero credit usage might not appear in the view. For the before-and-after comparison, sum daily CREDITS\_USED\_COMPUTE from WAREHOUSE\_METERING\_HISTORY for 30 days on each side. Its CREDITS\_ATTRIBUTED\_COMPUTE\_QUERIES column is NULL for Adaptive Warehouses. So Snowflake's idle-cost query for standard warehouses stops working after the conversion. Run it before you convert, and keep the result. Divide daily credits by daily query count before you compare the two periods. Without that step, a quiet month looks like a saving. ## Auto-Tuning a Standard Warehouse Means Moving Suspend and Cluster Settings Auto-tuning means software changes a standard warehouse's settings while the workload runs. Two settings carry most of the idle cost. The first is AUTO\_SUSPEND. The [CREATE WAREHOUSE reference](https://docs.snowflake.com/en/sql-reference/sql/create-warehouse) sets the default at 600 seconds. The background suspend process runs about every 30 seconds. Snowflake [bills per second with a 60-second minimum](https://docs.snowflake.com/en/user-guide/cost-understanding-compute), and each resume starts that minimum again. A setting that is too long pays for idle minutes. A setting that is too short pays the minimum on every resume. The second is the cluster range on a multi-cluster warehouse. MIN\_CLUSTER\_COUNT and MAX\_CLUSTER\_COUNT bound how many clusters run, and SCALING\_POLICY is STANDARD or ECONOMY. Multi-cluster warehouses need Enterprise Edition. Each running cluster bills at the full rate for its size. Gen2 changes the price of both mistakes. New standard warehouses default to Gen2, per the [Gen2 guide](https://docs.snowflake.com/en/user-guide/warehouses-gen2). Gen2 costs 1.35 credits an hour at X-Small on AWS and GCP, against 1 credit for Gen1. That 1.35x ratio holds at every size, so each idle minute on Gen2 costs 35% more on AWS. Run `SHOW WAREHOUSES` and read the `auto_suspend`, `max_cluster_count` and `generation` columns. A NULL or 0 in `auto_suspend` means the warehouse never suspends, so fix those rows first. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## Altimate Lite Suspends Idle Warehouses and Lowers the Cluster Ceiling [Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) is a Snowflake Native App that auto-tunes the standard warehouses you enable. It writes two settings and no others. We read the daemon code to confirm the exact behavior. - **Idle suspend.** When a warehouse goes idle, Altimate Lite sets AUTO\_SUSPEND to 1 second, then restores your original value. Snowflake suspends the warehouse in between. - **Cluster scale-down.** When demand drops, it sets MAX\_CLUSTER\_COUNT one lower, then restores the original count. Snowflake shuts down one cluster in between. The app checks each enabled warehouse every five to six seconds. That is about ten checks inside Snowflake's 30-second suspend cycle. It runs inside your Snowflake account and makes no outbound network calls. The limits are specific. Altimate Lite never changes a warehouse's size. It does not rewrite SQL and it does not show per-query warehouse cost. It does not touch Adaptive Warehouses, because Snowflake manages those. It covers Snowflake only. The [Altimate Lite launch post](https://altimate.ai/blog/altimate-lite-autonomous-agents-for-snowflake-cost-optimization) reports private preview results. It gives a 19% average saving across all customers, warehouses and days. The lowest saving on any warehouse on any day was 12.4%. The highest single-day saving was 57.5%, typically on weekends. One warehouse logged 118 tuning decisions in one day. The post publishes no customer count, time window or method, so treat these as the vendor's own preview figures. Setup details are in the [Altimate Lite docs](https://help.altimate.ai/snowflake-native-app/auto-tune-cost-savings/). The published write-up on [idle warehouse spend](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) shows the per-warehouse view. ## The Enterprise Platform Adds Downsizing, Query Cost and Rewrite Proposals The Altimate Enterprise Platform adds three things on a standard warehouse. The first is size. A warehouse sizing agent downsizes a warehouse one level during low-demand periods. Approval Mode builds a weekly schedule that a person approves. Real-Time Mode acts on its own and evaluates every 15 minutes. If queue time or latency crosses a threshold, the platform reverts to the default size and pauses resizing for one hour. The highlight on how [Auto Tune right-sizes Snowflake warehouses](https://altimate.ai/product-highlights/snowflake-warehouse-auto-tune-right-sizing) walks through both modes. The second is cost per query. The platform splits warehouse credits across queries by concurrent workload, and it includes idle time. Snowflake's own QUERY\_ATTRIBUTION\_HISTORY view leaves idle time out. Query groups show total, average and annualized cost. The third is SQL. The platform can rewrite a query, validate the results against your data and open the fix as a pull request. A person merges it. Auto Tune itself never changes SQL. ## The Comparison Table Rebuilt From the Docs and the Code | Capability | Adaptive Compute | Altimate Lite | Enterprise Platform | | --- | --- | --- | --- | | Chooses compute size | ✓ per query, under a ceiling | ✗ never changes size | ✓ one level down in low demand | | Suspends idle compute | ✓ Snowflake manages it | ✓ enabled warehouses only | ✓ | | Adjusts cluster count | ✓ Snowflake manages it | ✓ lowers the maximum by one | ✓ | | Per-query cost | ✓ QUERY\_METERING\_HISTORY | ✗ | ✓ idle time included | | Rewrites SQL | ✗ | ✗ | ✓ as a pull request a person merges | | Works on Adaptive Warehouses | ✓ | ✗ not eligible | \- | | Platforms | Snowflake | Snowflake | Snowflake and Databricks | No row in this table covers BigQuery. Altimate documents Auto Tune for Snowflake and Databricks only. ## Neither Path Fixes a Query That Reads Too Much Adaptive Compute and auto-tuning both change the compute under a query. Neither one changes what the query reads. A `SELECT *` in a model that needs four columns reads every column on any warehouse type. A filter on `DATE_TRUNC('month', order_ts)` still blocks partition pruning. Adaptive Compute can give that query more compute, up to your ceiling, and you pay for it per query. Altimate Lite only decides when the warehouse stops running. Find the worst readers with QUERY\_HISTORY. Sort by partitions scanned against partitions total, and look at the top of the list. Technique 6 in our [Snowflake cost optimization techniques](https://altimate.ai/blog/snowflake-cost-optimization-techniques) covers both fixes. Our page on [Snowflake query optimization](https://altimate.ai/use-cases/altimate-for-snowflake) shows how query fixes and warehouse tuning fit together. ## Which Warehouses to Convert and Which to Keep and Tune | Warehouse | Choice | Reason | | --- | --- | --- | | Ad hoc or sandbox with unpredictable queries | Convert to Adaptive | Snowflake sizes each query, and you stop guessing a size | | Several small team warehouses with low use | Convert to Adaptive | The names stay for accounting, and the compute pool is shared | | Mostly HTAP (hybrid table) traffic | Keep on Gen2 standard | Snowflake's guide says to use Gen2 for these | | X5Large, X6Large, Snowpark-optimized or interactive | Keep and tune | Conversion is not supported yet | | Any warehouse on Standard Edition | Keep and tune | Adaptive needs Enterprise Edition | | Batch ETL where you want a fixed size | Keep and tune | Adaptive has a ceiling, but no fixed size | | A region Snowflake does not list for Adaptive | Keep and tune | Check the region list in the guide | | A warehouse with scripts that set size or clusters | Keep until the scripts change | Those properties fail on an Adaptive Warehouse | Many accounts end up with both types. That mix means two cost views: QUERY\_METERING\_HISTORY for the Adaptive side, and WAREHOUSE\_METERING\_HISTORY plus idle cost for the standard side. [Why your Snowflake bill grows each quarter](https://altimate.ai/blog/why-your-snowflake-bill-grows-by-30-percent-a-quarter) lists the cost drivers that each view shows. ## Convert One Warehouse and Measure Credits per Query Pick your least predictable ad hoc warehouse. Save 30 days of its daily credits, query counts and idle cost. Convert it during a quiet hour, then compare credits per query after 30 days. Keep the warehouses in the "keep and tune" rows on standard compute. Set their AUTO\_SUSPEND from measured idle gaps, not from the 600-second default. To automate suspend and cluster scale-down on those warehouses, install [Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) from the Snowflake Marketplace. ## Frequently Asked Questions Is Snowflake Adaptive Compute generally available? Yes. Adaptive Compute reached general availability on AWS on 16 June 2026. Snowflake's guide also lists select Azure and GCP regions as generally available. It requires Enterprise Edition or higher. What replaces warehouse size on an Adaptive Warehouse? Two parameters replace it. MAX\_QUERY\_PERFORMANCE\_LEVEL caps the performance of one query, with XLARGE as the default. QUERY\_THROUGHPUT\_MULTIPLIER controls burst capacity, with 2 as the default and 0 as unlimited. How do I see what one query cost on an Adaptive Warehouse? Query SNOWFLAKE.ACCOUNT\_USAGE.QUERY\_METERING\_HISTORY. It returns per-query credits for the last 365 days, one row per metering hour. Latency is up to one hour. Does Altimate Lite work on Adaptive Warehouses? No. Snowflake manages Adaptive Warehouses, so Altimate Lite does not tune them. It tunes standard warehouses that you enable, and it only changes auto-suspend and maximum cluster count. Does Adaptive Compute reduce my Snowflake bill? Snowflake says it runs significantly more queries at a similar cost to Gen2, and gives no number. Measure credits per query for 30 days before and after one conversion. On this page 1. Snowflake Adaptive Compute Replaces Four Settings With Two Parameters 2. Conversion Is Online but Bills Twice While Old Queries Finish 3. Adaptive Warehouses Bill per Query in QUERY\_METERING\_HISTORY 4. Auto-Tuning a Standard Warehouse Means Moving Suspend and Cluster Settings 5. Altimate Lite Suspends Idle Warehouses and Lowers the Cluster Ceiling 6. The Enterprise Platform Adds Downsizing, Query Cost and Rewrite Proposals 7. The Comparison Table Rebuilt From the Docs and the Code 8. Neither Path Fixes a Query That Reads Too Much 9. Which Warehouses to Convert and Which to Keep and Tune 10. Convert One Warehouse and Measure Credits per Query From the product - [Enterprise Platform Auto Tune right-sizes Snowflake warehouses without slowing queries 6 min read](https://altimate.ai/product-highlights/snowflake-warehouse-auto-tune-right-sizing) - [Enterprise Platform Reduce Snowflake idle warehouse spend with Altimate Lite 6 min read](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) - [Enterprise Platform Track Snowflake query cost over time, run by run 6 min read](https://altimate.ai/product-highlights/snowflake-query-cost-over-time) - [Enterprise Platform Snowflake Cortex AI cost breakdown by service, user, and model 6 min read](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown) - [Enterprise Platform Attribute Snowflake cost to the team that owns it 4 min read](https://altimate.ai/product-highlights/snowflake-cost-alerts-by-team) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Why Your Snowflake Bill Grows 30% a Quarter URL: https://altimate.ai/blog/why-your-snowflake-bill-grows-by-30-percent-a-quarter A Snowflake bill that grows 30% a quarter nearly triples in a year. Six drivers behind that growth, each with the ACCOUNT_USAGE query that finds it. ## Why Your Snowflake Bill Grows by 30% a Quarter A Snowflake bill that grows 30% a quarter nearly triples in a year. Six drivers behind that growth, each with the ACCOUNT\_USAGE query that finds it. Atharva Shah 2026-09-04 2026-09-16 10 min read On this page 10 sections 1. A Snowflake Bill Growing 30% a Quarter Outpaces Snowflake's Own Customer Base 2. Split the Snowflake Bill by Service Type Before You Blame a Warehouse 3. Idle Warehouses Bill Every Second Until AUTO\_SUSPEND Runs 4. Oversized Warehouses Stay Oversized After the Incident Ends 5. Full Table Rebuilds Repeat Last Night's Work on Every Run 6. Repeated Scans Pay for the Same Answer Many Times 7. Serverless and AI Service Lines Grow With No Warehouse Attached 8. Gen2 Defaults Raise the Price of Each Warehouse Hour 9. Match Each Driver to the View That Shows It 10. Run the Service Split This Week and Give Each Line an Owner tl;dr A Snowflake bill that grows 30% a quarter ends the year at 2.86 times its starting size. Snowflake's own filing gives a baseline for comparison. Existing customers spend about 26% more over a full year, which is about 6% a quarter. Growth at five times that pace usually comes from several small drivers at once. Check six drivers. The first two are idle warehouses and oversized warehouses. The next two are dbt models rebuilt in full and repeated scans. The last two are serverless and AI service lines, and Gen2 pricing. Each one shows up in a SNOWFLAKE.ACCOUNT\_USAGE view, and each section below gives the query that finds it. Your Snowflake bill went up again this quarter, and finance wants a reason. Suppose it grew 30%. We found no source that says a typical account grows at that rate, so treat 30% as your scenario. The point of this post is to find which part of the bill moved. Start with the arithmetic, because 30% a quarter compounds fast. A $100,000 quarter becomes $130,000, then $169,000, then $219,700, then $285,610. The bill passes double its starting size in the third quarter. After four quarters it is 2.86 times the start. Now compare that with a measured baseline. Snowflake reported a net revenue retention rate of 126% for the second quarter of fiscal 2027, in its [SEC earnings exhibit](https://www.sec.gov/Archives/edgar/data/1640147/000164014726000033/fy2027q2earnings.htm). Net revenue retention compares what the same customers spend now with what they spent a year earlier. So existing customers spend about 26% more per year, which compounds to about 6% a quarter. Snowflake counts only customers on capacity contracts in that rate, so treat it as a proxy. Some of that growth is new work that somebody approved. The job is to split the approved growth from the drift, and the six sections below each find one kind of drift. A grouped bar chart indexed to 100 compares two paths over four quarters. The 30% a quarter path reaches 130, 169, 219.7 and 285.6. The path at Snowflake's 126% net revenue retention pace reaches about 106, 112, 119 and 126 *A Snowflake bill growing 30% a quarter pulls away from the pace Snowflake's existing customers show, and the gap widens every quarter.* Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## A Snowflake Bill Growing 30% a Quarter Outpaces Snowflake's Own Customer Base | Quarter | Your bill at 30% a quarter | Existing-customer pace, 126% a year | Snowflake product revenue pace, 37% a year | | --- | --- | --- | --- | | Start | 100 | 100 | 100 | | Q1 | 130 | 106.0 | 108.2 | | Q2 | 169 | 112.2 | 117.0 | | Q3 | 219.7 | 118.9 | 126.6 | | Q4 | 285.6 | 126.0 | 137.0 | The same filing reports product revenue of $1.49 billion, up 37% year over year. That figure includes new customers, so it sits above the retention pace. It also reports 828 customers with more than $1 million in trailing 12-month product revenue. The two Snowflake columns are our own quarterly conversions of the annual figures. Snowflake publishes only the annual rates. A bill on the 30% path is not normal for the customer base, so it is worth an hour of diagnosis. ## Split the Snowflake Bill by Service Type Before You Blame a Warehouse Warehouse compute is one line of the Snowflake bill among many. The [METERING\_HISTORY view](https://docs.snowflake.com/en/sql-reference/account-usage/metering_history) splits credits by SERVICE\_TYPE. The values include WAREHOUSE\_METERING, AUTO\_CLUSTERING, SEARCH\_OPTIMIZATION, MATERIALIZED\_VIEW, PIPE, SERVERLESS\_TASK, QUERY\_ACCELERATION and AI\_SERVICES. Run this first. It shows which line grew, month by month: ```sql SELECT DATE_TRUNC('month', start_time) AS usage_month, service_type, ROUND(SUM(credits_used), 1) AS credits FROM SNOWFLAKE.ACCOUNT_USAGE.METERING_HISTORY WHERE start_time >= DATEADD(month, -6, DATE_TRUNC('month', CURRENT_DATE())) GROUP BY usage_month, service_type ORDER BY usage_month, credits DESC; ``` The view has up to three hours of latency. Its cloud services column can lag by up to six hours. Cloud services credits need one more step. Snowflake [charges cloud services only](https://docs.snowflake.com/en/user-guide/cost-understanding-compute) when daily cloud services use exceeds 10% of daily warehouse use. The raw CREDITS\_USED\_CLOUD\_SERVICES number is therefore larger than what you pay on most days. If WAREHOUSE\_METERING grew fastest, read the next four sections. If a serverless line or AI\_SERVICES grew fastest, skip to the serverless section. ## Idle Warehouses Bill Every Second Until AUTO\_SUSPEND Runs An idle warehouse bills as if it were working. The default AUTO\_SUSPEND is 600 seconds, per the [CREATE WAREHOUSE reference](https://docs.snowflake.com/en/sql-reference/sql/create-warehouse). A background process checks for idle warehouses about every 30 seconds. A setting of 0 or NULL means the warehouse never suspends. Short settings have their own cost. Snowflake bills per second with a 60-second minimum, and every resume starts the minimum again. The worked example in our [12 Snowflake cost optimization techniques](https://altimate.ai/blog/snowflake-cost-optimization-techniques) shows the effect. A warehouse that wakes 200 times a day for 20-second queries does 67 minutes of work and bills 200 minutes. Snowflake documents an idle-cost query. This version groups idle cost by month to show the trend: ```sql SELECT DATE_TRUNC('month', start_time) AS usage_month, warehouse_name, ROUND(SUM(credits_used_compute), 1) AS compute_credits, ROUND(SUM(credits_used_compute) - SUM(credits_attributed_compute_queries), 1) AS idle_credits FROM SNOWFLAKE.ACCOUNT_USAGE.WAREHOUSE_METERING_HISTORY WHERE start_time >= DATEADD(month, -3, DATE_TRUNC('month', CURRENT_DATE())) AND end_time < CURRENT_DATE() GROUP BY usage_month, warehouse_name ORDER BY idle_credits DESC; ``` Idle credits that grow faster than compute credits point at new warehouses, or at warehouses with more idle gaps. Then check the setting with `SHOW WAREHOUSES` and read the `auto_suspend` column. Altimate Lite for Snowflake suspends enabled warehouses when they go idle, and the write-up on [idle warehouse spend with Altimate Lite](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) shows the per-warehouse view. ## Oversized Warehouses Stay Oversized After the Incident Ends Each size step doubles the credit rate. The [warehouse overview](https://docs.snowflake.com/en/user-guide/warehouses-overview) lists Gen1 rates of 1 credit an hour at X-Small, 8 at Large and 16 at X-Large. A team that sizes up for a migration or an incident rarely sizes back down. A warehouse is a candidate to size down when its queries do not queue, do not spill and finish fast. QUERY\_HISTORY carries all three signals: ```sql SELECT warehouse_name, warehouse_size, COUNT(*) AS queries, APPROX_PERCENTILE(execution_time, 0.95) / 1000 AS p95_exec_seconds, SUM(queued_overload_time) / 1000 AS queued_seconds, SUM(bytes_spilled_to_local_storage) / POWER(1024, 3) AS spilled_gb FROM SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY WHERE start_time >= DATEADD(day, -14, CURRENT_DATE()) AND warehouse_name IS NOT NULL GROUP BY warehouse_name, warehouse_size ORDER BY queries DESC; ``` Look for a large warehouse with zero queued seconds, zero spill and a short p95. Drop it one size and rerun the same workload. The size-down test is technique 2 in the same techniques post. ## Full Table Rebuilds Repeat Last Night's Work on Every Run A dbt model with the `table` materialization rebuilds the whole table on every run. The table grows every day, so the rebuild costs a little more every day. That growth is invisible until someone lists the rebuilds. dbt adds a JSON comment to every query it runs, per the [dbt query-comment reference](https://docs.getdbt.com/reference/project-configs/query-comment). The comment carries the `node_id` of the model. On Snowflake, dbt puts the comment at the end of the query. This query ranks rebuilt models by credits over the last 30 days: ```sql SELECT REGEXP_SUBSTR(q.query_text, '"node_id": "([^"]+)"', 1, 1, 'e') AS dbt_node, COUNT(*) AS rebuilds, ROUND(SUM(a.credits_attributed_compute), 2) AS credits, ROUND(AVG(q.bytes_scanned) / POWER(1024, 3), 1) AS avg_gb_scanned FROM SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY q JOIN SNOWFLAKE.ACCOUNT_USAGE.QUERY_ATTRIBUTION_HISTORY a ON a.query_id = q.query_id WHERE q.start_time >= DATEADD(day, -30, CURRENT_DATE()) AND q.query_type = 'CREATE_TABLE_AS_SELECT' AND q.query_text ILIKE '%"app": "dbt"%' GROUP BY dbt_node ORDER BY credits DESC LIMIT 25; ``` A model near the top that appends new rows each day is a candidate for the `incremental` materialization. QUERY\_ATTRIBUTION\_HISTORY has up to eight hours of latency. It leaves out idle time and queries of about 100 milliseconds or less, so these credits are a lower bound. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## Repeated Scans Pay for the Same Answer Many Times A short query that runs thousands of times a day can cost more than a slow report that runs once. A list sorted by duration hides that query. Group by `query_parameterized_hash` instead, so one query with different filter values counts as one shape: ```sql SELECT q.query_parameterized_hash, ANY_VALUE(q.warehouse_name) AS warehouse, COUNT(*) AS runs, ROUND(SUM(a.credits_attributed_compute), 2) AS credits, ROUND(SUM(q.partitions_scanned) / NULLIF(SUM(q.partitions_total), 0), 2) AS share_of_partitions_read FROM SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY q JOIN SNOWFLAKE.ACCOUNT_USAGE.QUERY_ATTRIBUTION_HISTORY a ON a.query_id = q.query_id WHERE q.start_time >= DATEADD(day, -7, CURRENT_DATE()) GROUP BY q.query_parameterized_hash ORDER BY credits DESC LIMIT 25; ``` Two patterns show up at the top. The first is a shape with many runs, which often means a dashboard refreshes more often than anyone reads it. The second is a share of partitions read close to 1.0, which means the filter does not prune. The Enterprise Platform tracks [query cost run by run](https://altimate.ai/product-highlights/snowflake-query-cost-over-time), with idle time included. ## Serverless and AI Service Lines Grow With No Warehouse Attached Several Snowflake features bill without a warehouse, so the warehouse queries above miss them. [Resource monitors work for warehouses only](https://docs.snowflake.com/en/user-guide/resource-monitors), so they miss these lines too. Three lines are worth a check each month: - **Query acceleration.** New Gen2 warehouses have the Query Acceleration Service turned on by default, with a scale factor of 2. The service bills per second while it runs. Snowflake's own example puts a Medium warehouse at scale factor 5 at up to 20 extra credits an hour. - **Clustering and search optimization.** Both run in the background as the table changes. A key set during an evaluation keeps billing after the evaluation ends. - **AI services.** Cortex functions bill on the AI\_SERVICES line. The breakdown of [Cortex AI cost by service](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown) splits that line by service, user and model. This query lists the serverless objects that cost the most last month: ```sql SELECT service_type, database_name, schema_name, name, ROUND(SUM(credits_used), 2) AS credits FROM SNOWFLAKE.ACCOUNT_USAGE.METERING_HISTORY WHERE service_type NOT IN ('WAREHOUSE_METERING', 'WAREHOUSE_METERING_READER') AND start_time >= DATEADD(month, -1, CURRENT_DATE()) GROUP BY service_type, database_name, schema_name, name ORDER BY credits DESC LIMIT 25; ``` Check each row against a consumer that still exists. Snowflake Budgets reach serverless features that resource monitors miss, and technique 9 in the techniques post covers them. ## Gen2 Defaults Raise the Price of Each Warehouse Hour A new standard warehouse is Gen2 unless you say otherwise. The [Gen2 guide](https://docs.snowflake.com/en/user-guide/warehouses-gen2) says Snowflake defaults to `GENERATION = '2'` when the statement omits the clause. A warehouse created by a script last quarter may be Gen2 without anyone choosing it. Gen2 costs more per hour. The [Snowflake Service Consumption Table](https://www.snowflake.com/legal-files/CreditConsumptionTable.pdf) lists 1.35 credits an hour at X-Small on AWS and GCP, and 1.25 on Azure. A Gen1 Large at 8 credits becomes 10.8 credits on AWS. Gen2 saves money only when the same work finishes at least 26% faster. Find the Gen2 warehouses and their creation dates: ```sql SHOW WAREHOUSES; SELECT "name", "size", "generation", "created_on", "enable_query_acceleration" FROM TABLE(RESULT_SCAN(LAST_QUERY_ID())) WHERE "generation" = '2' ORDER BY "created_on" DESC; ``` Compare credits per query before and after each switch. For the choice between tuning a standard warehouse and converting it, read the comparison of [Snowflake Adaptive Compute and auto-tuning](https://altimate.ai/blog/adaptive-compute-vs-auto-tune-optimizing-snowflake-warehouses). ## Match Each Driver to the View That Shows It | Driver | Where it shows | Signal to look for | Fix in the techniques post | | --- | --- | --- | --- | | Idle warehouses | WAREHOUSE\_METERING\_HISTORY | idle credits growing month over month | technique 1 | | Oversized warehouses | QUERY\_HISTORY | no queueing, no spill, short p95 | technique 2 | | dbt full refreshes | QUERY\_HISTORY plus QUERY\_ATTRIBUTION\_HISTORY | the same node rebuilt every run | technique 6 | | Repeated scans | QUERY\_HISTORY plus QUERY\_ATTRIBUTION\_HISTORY | many runs, partitions read near 1.0 | techniques 5 and 6 | | Serverless and AI lines | METERING\_HISTORY | a non-warehouse service growing fastest | techniques 9 and 10 | | Gen2 pricing | SHOW WAREHOUSES | generation 2 on recently created warehouses | the Gen2 section | Each driver needs a person who can act on it. A bill split by warehouse reaches nobody in particular. A bill split by team reaches the person who owns the query. The Enterprise Platform shows [the team that owns each credit](https://altimate.ai/product-highlights/snowflake-cost-alerts-by-team). ## Run the Service Split This Week and Give Each Line an Owner Run the METERING\_HISTORY query for the last six months. Name the one service line that grew fastest, and run the matching query from this post. Write down one owner for each of the top five rows. Some fixes are one statement. Setting AUTO\_SUSPEND from measured idle gaps is one of them. Others need tooling. Altimate Lite for Snowflake suspends idle warehouses and lowers the maximum cluster count on the warehouses you enable. It does not resize warehouses and it does not show per-query cost. A spike that the queries above cannot explain calls for [the root cause of a workload spike](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis). Our page on [Snowflake query optimization](https://altimate.ai/use-cases/altimate-for-snowflake) lists what Altimate does for Snowflake cost optimization. ## Frequently Asked Questions Is 30% quarterly growth normal for a Snowflake bill? We found no source that says so. Snowflake reports a 126% net revenue retention rate, which is about 6% a quarter for existing customers. A bill growing 30% a quarter moves about five times faster. Which ACCOUNT\_USAGE view shows where a Snowflake bill grew? Start with METERING\_HISTORY, grouped by month and SERVICE\_TYPE. It separates warehouse compute from clustering, search optimization, query acceleration, serverless tasks and AI services. Then use WAREHOUSE\_METERING\_HISTORY and QUERY\_HISTORY for the warehouse line. How do I measure idle warehouse cost in Snowflake? Subtract CREDITS\_ATTRIBUTED\_COMPUTE\_QUERIES from CREDITS\_USED\_COMPUTE in WAREHOUSE\_METERING\_HISTORY. Snowflake's docs give this query. The column is NULL for Adaptive Warehouses, so the method works on standard warehouses only. Why do dbt models increase Snowflake costs over time? A model with the table materialization rebuilds the whole table on every run. As the table grows, each rebuild reads and writes more data. The incremental materialization processes only new rows. Do resource monitors stop serverless spend? No. Snowflake says resource monitors work for warehouses only. Use Snowflake Budgets for serverless features, and check the METERING\_HISTORY service lines each month. On this page 1. A Snowflake Bill Growing 30% a Quarter Outpaces Snowflake's Own Customer Base 2. Split the Snowflake Bill by Service Type Before You Blame a Warehouse 3. Idle Warehouses Bill Every Second Until AUTO\_SUSPEND Runs 4. Oversized Warehouses Stay Oversized After the Incident Ends 5. Full Table Rebuilds Repeat Last Night's Work on Every Run 6. Repeated Scans Pay for the Same Answer Many Times 7. Serverless and AI Service Lines Grow With No Warehouse Attached 8. Gen2 Defaults Raise the Price of Each Warehouse Hour 9. Match Each Driver to the View That Shows It 10. Run the Service Split This Week and Give Each Line an Owner From the product - [Enterprise Platform Reduce Snowflake idle warehouse spend with Altimate Lite 6 min read](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) - [Enterprise Platform Attribute Snowflake cost to the team that owns it 4 min read](https://altimate.ai/product-highlights/snowflake-cost-alerts-by-team) - [Enterprise Platform Track Snowflake query cost over time, run by run 6 min read](https://altimate.ai/product-highlights/snowflake-query-cost-over-time) - [Enterprise Platform Root cause analysis for a dbt, Tableau, or Snowflake workload spike 6 min read](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis) - [Enterprise Platform Snowflake Cortex AI cost breakdown by service, user, and model 6 min read](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Why AI Agents Fail on Production Data Stacks URL: https://altimate.ai/blog/why-ai-agents-fail-on-production-data-stacks AI agents fail on production data stacks in ways a dev sandbox hides. Seven failures, from inherited role grants to backfills, and the control that stops each. ## Why AI Agents Fail on Production Data Stacks AI agents fail on production data stacks in ways a dev sandbox hides. Seven failures, from inherited role grants to backfills, and the control that stops each. Atharva Shah 2026-08-05 2026-09-16 10 min read On this page 9 sections 1. Seven Ways AI Agents Fail on Production Data That a Sandbox Never Shows 2. An Agent Role That Inherits Write Grants Can Change Production Tables 3. A Query That Is Cheap in Dev Can Burn Credits on a Production Warehouse 4. A Schema That Changed Since the Agent Last Read It Breaks Its SQL 5. Incremental Models and Backfills Fail on State the Dev Build Never Had 6. A Model Diff Hides the Dashboards and Syncs That Read the Model 7. Production Rows Carry Personal Data Into the Agent's Context 8. An Agent With Write Access Needs a Gate Between Its Branch and Production 9. Start With a Read-Only Role and a Statement Timeout for Agent Sessions tl;dr AI agents fail on production data stacks because production holds five things that a dev sandbox lacks: - role grants that the agent inherits, - a warehouse that bills by the second, - a schema that other teams change, - incremental tables with history, and - readers that a diff does not show. An agent can pass every check in its own schema and still break one of those five. Each failure has a control that works with any model. Give agent sessions a read-only role and a statement timeout. Build each changed incremental model twice in CI. Gate the merge on lineage and a data diff, and keep the final approval with a person. An agent usually tests its dbt change in a dev schema. That schema holds a sample of the data. The developer's own role owns every object in it. No dashboard reads from it. Production changes all three conditions on the day the change merges. That is why AI agents fail on production data in places the sandbox never tests. The failures appear after the merge, as a grant the agent should not hold, a credit spike or a broken sync. Cleanlab's 2025 survey found 95 of 1,837 engineering and AI leaders with agents live in production. That is about 5% of the leaders it asked. [Ten failure modes of AI coding agents](https://altimate.ai/blog/why-ai-coding-agents-fail-at-data-engineering) covers the mistakes an agent makes while it writes code. The seven failures below start after that code merges. Each one has a control in SQL, YAML or a CI job, plus what Altimate Code covers and where it stops. ## Seven Ways AI Agents Fail on Production Data That a Sandbox Never Shows A sandbox passes all seven failures because it lacks the condition that triggers each one. | Failure | What the sandbox lacks | Control that catches it | | --- | --- | --- | | Write access through role inheritance | inherited production roles | a read-only agent role and deny rules | | Runaway query cost | full-size tables and a bill | a statement timeout and a resource monitor | | Schema drift | changes made by other teams | `on_schema_change` set to fail, and a schema check | | Incremental and backfill errors | table history | a full build, then an incremental build, then a diff | | Hidden downstream readers | dashboards and syncs | exposures and column-level lineage | | Personal data in agent context | production rows | a masking policy on the agent role | | Unreviewed writes | a merge gate | CI checks and a person who approves | A two-column comparison. The left column, labeled dev sandbox, lists a developer-owned role, a data sample, a schema only you change, tables built from empty, no downstream readers and masked data. The right column, labeled production, lists inherited roles, full tables billed per second, schemas other teams change, incremental tables with history, dashboards and syncs, and real personal data *Six conditions differ between a dev schema and production. A sandbox test exercises none of them.* Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## An Agent Role That Inherits Write Grants Can Change Production Tables A Snowflake role inherits every privilege of the roles granted to it. So an agent that runs as a developer's role can hold more access than the developer expects. If that role inherits from a transform role, the agent can write to production schemas. The sandbox hides this, because the developer owns every object in the dev schema anyway. Give agent sessions their own role with read grants only. Future grants extend the read access to tables that other teams create later. ```sql CREATE ROLE agent_readonly; GRANT USAGE ON WAREHOUSE agent_wh TO ROLE agent_readonly; GRANT USAGE ON DATABASE analytics TO ROLE agent_readonly; GRANT USAGE ON ALL SCHEMAS IN DATABASE analytics TO ROLE agent_readonly; GRANT USAGE ON FUTURE SCHEMAS IN DATABASE analytics TO ROLE agent_readonly; GRANT SELECT ON ALL TABLES IN DATABASE analytics TO ROLE agent_readonly; GRANT SELECT ON FUTURE TABLES IN DATABASE analytics TO ROLE agent_readonly; ``` Altimate Code adds a second layer inside the agent. Analyst mode runs SELECT statements only. It blocks INSERT, UPDATE, DELETE and DROP without a prompt. Builder mode asks before each write and always blocks `DROP DATABASE`, `DROP SCHEMA` and `TRUNCATE`. The `--yolo` flag approves every prompt automatically. Explicit `deny` rules still apply under `--yolo`, because they fire before any prompt exists. So put production write denials in the config file: ```json { "permission": { "sql_execute_write": "deny", "bash": { "*": "ask", "DROP *": "deny" } } } ``` Altimate Code's `finops_role_grants` tool reports the grants that each role holds. On Snowflake, `finops_role_hierarchy` shows which roles inherit from which. Run both before you connect an agent. The role design itself stays with your platform team. ## A Query That Is Cheap in Dev Can Burn Credits on a Production Warehouse A query that scans a dev sample in two seconds can scan billions of rows in production. The SQL text is the same in both places. Only the table size and the warehouse change. Snowflake bills warehouse time per second, with a 60-second minimum each time a warehouse resumes. Put the agent on its own warehouse and cap each statement. An X-Small warehouse bills 1 credit an hour, so a 600-second cap stops one statement at about 0.17 credits. The same cap on an X-Large, at 16 credits an hour, stops it at about 2.7 credits. ```sql CREATE WAREHOUSE agent_wh WITH WAREHOUSE_SIZE = 'XSMALL' AUTO_SUSPEND = 60 STATEMENT_TIMEOUT_IN_SECONDS = 600; -- 50 credits is an example quota. Set your own. CREATE RESOURCE MONITOR agent_monthly WITH CREDIT_QUOTA = 50 TRIGGERS ON 100 PERCENT DO SUSPEND_IMMEDIATE; ALTER WAREHOUSE agent_wh SET RESOURCE_MONITOR = agent_monthly; ``` Only the ACCOUNTADMIN role can create a resource monitor. A monitor covers warehouses only, so serverless features need their own limits. Ask for the plan before a large query runs. `EXPLAIN` compiles the statement without executing it, and it needs no running warehouse. ```sql EXPLAIN USING TEXT SELECT region, SUM(amount) FROM analytics.fct_orders GROUP BY region; ``` Altimate Code's `finops_expensive_queries` and `finops_warehouse_advice` tools read warehouse history after queries run. Altimate has not published a method for its pre-run cost estimates. Treat the timeout and the monitor as your hard limits. When a spike does get through, [a workload root cause analysis](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis) traces it to the query that caused it. ## A Schema That Changed Since the Agent Last Read It Breaks Its SQL Another team renames a column on Tuesday. The agent's schema snapshot dates from Monday. So the agent writes SQL against a column that no longer exists, and the model fails on its next scheduled run. Two dbt settings turn schema drift into an early, loud failure. An incremental model with `on_schema_change='fail'` stops the run when the source and target columns diverge. The default value, `ignore`, keeps the run going. A [model contract](https://docs.getdbt.com/docs/collaborate/govern/model-contracts) fails the build when the output columns stop matching the declared list. ```sql {{ config( materialized='incremental', unique_key='order_id', on_schema_change='fail' ) }} ``` The `--empty` flag builds each selected model against zero input rows. dbt still runs the model SQL on the warehouse, which proves the model builds against today's schema. It skips the expensive reads of input data. ```bash dbt build --select state:modified+ --state prod-artifacts/ --empty ``` Altimate Code's `altimate_core_validate` checks table and column names against a supplied or cached schema. It makes no warehouse call, so a stale cache gives a stale answer. Run `schema_index` after a schema change to refresh the local cache. The `dbt-schema-verify` validator compares each modified model's columns against `schema.yml` after the agent reports that it is done. Validators are off by default. Set `ALTIMATE_VALIDATORS_ENABLED=1` to turn them on. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## Incremental Models and Backfills Fail on State the Dev Build Never Had A dev build usually creates an incremental model from an empty target. dbt then runs the full-refresh path of the model. The `is_incremental()` branch runs for the first time in production, so a bug inside that branch never met a test. Three ADE-Bench airbnb task failures in our runs trace to a broken `is_incremental()` filter in the benchmark's own models. The fix is pull request 145 in the `dbt-labs/ade-bench` repository. Null values in the key cause a second silent failure. dbt's documentation warns that nulls in a `unique_key` column can stop the merge from matching rows. The model then writes duplicate rows. Test the key columns directly: ```yaml models: - name: fct_orders columns: - name: order_id data_tests: - not_null - unique ``` Build each changed incremental model twice in CI, against a copy of production. The first build creates the table. The second build runs the incremental branch against that table. ```bash dbt build --select state:modified+,config.materialized:incremental --state prod-artifacts/ --full-refresh dbt build --select state:modified+,config.materialized:incremental --state prod-artifacts/ ``` A backfill carries the opposite risk. A `--full-refresh` on a large production table rebuilds the whole table and bills for the full scan. Compare the incremental result with a full rebuild over the same date range before you trust either one. Altimate Code's `data_diff` tool runs that comparison. On one database it uses a single FULL OUTER JOIN. Across two databases it compares checksums by bisection. The rows stay in their own databases. The tool needs a warehouse connection and a primary key that you confirm. ## A Model Diff Hides the Dashboards and Syncs That Read the Model A pull request shows the SQL that changed. It does not show the dashboard, the reverse ETL sync or the notebook that reads the table. dbt exposures declare those readers in YAML. A selector then lists every exposure that a change reaches. ```yaml exposures: - name: crm_account_sync label: CRM account sync type: application owner: name: Revenue Operations email: revops@example.com depends_on: - ref('dim_accounts') ``` ```bash dbt ls --select state:modified+ --resource-type exposure --state prod-artifacts/ ``` Altimate Code's `impact_analysis` tool combines the dbt manifest with column-level lineage. It lists the affected models, tests and exposures for one model or column change. In one blast radius run, a change to `stg_orders` had 5 dependent files. The report covered 8 files, with 5 that needed changes and 3 that were safe. The follow-up `altimate-dbt build` ran 34 models, with 36 passes and 0 errors. The limit sits in the manifest. A dashboard that no exposure declares is outside that lineage. Declare the readers you know about before you let an agent change the models they read. A four-stage flow from left to right. The stages are agent session, warehouse, CI and merge. Under agent session sit a read-only role, Analyst mode and deny rules. Under warehouse sit a statement timeout, a resource monitor and a masking policy. Under CI sit an empty build, a full then incremental build, and altimate-code check. Under merge sit exposures, impact analysis, a data diff and a person who approves *Each control sits at one stage, and a change has to pass all four before it reaches production.* ## Production Rows Carry Personal Data Into the Agent's Context A dev schema often holds masked or synthetic data. Production holds real email addresses and phone numbers. When an agent reads query results, those rows become part of its prompt. With a hosted model, the prompt goes to the model provider. Mask sensitive columns for the agent role inside the warehouse. Snowflake masking policies need Enterprise Edition or higher. ```sql CREATE OR REPLACE MASKING POLICY email_mask AS (val STRING) RETURNS STRING -> CASE WHEN CURRENT_ROLE() = 'AGENT_READONLY' THEN '*********' ELSE val END; ALTER TABLE analytics.dim_customers MODIFY COLUMN email SET MASKING POLICY email_mask; ``` Altimate Code's `altimate_core_classify_pii` flags likely personal-data columns from their names and data types. It runs offline, with no model call. `altimate-code check --checks pii` runs the same class of check in CI. These checks read names and SQL, and they cannot see the values. The masking policy is the control that acts on the rows. ## An Agent With Write Access Needs a Gate Between Its Branch and Production An agent that can open a pull request can ship a change that nobody read closely. The gate belongs in CI, where it runs on every change, whatever wrote it. ```yaml name: agent-pr-gate on: [pull_request] jobs: sql-check: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - run: pip install dbt-snowflake && npm install -g altimate-code - run: dbt compile --target ci - run: altimate-code check "target/compiled/**/*.sql" --checks lint,safety,pii --fail-on error ``` `altimate-code check` needs no model provider and no API key. For dbt pull requests, `dbt_pr_review` adds a verdict of `APPROVE`, `COMMENT` or `REQUEST_CHANGES`. Each blocking finding comes from the deterministic engine. The LLM reviewer adds advisory comments, and those comments never block. The review bot always posts a COMMENT review event on GitHub. So the bot alone cannot satisfy branch protection, and a person still approves the merge. [Where each deterministic check sits in the agent loop](https://altimate.ai/blog/the-correctness-layer-deterministic-validation-outside-the-llm-loop) covers the checks that run inside the session, before CI. Altimate's [Agentic Data Engineering Platform](https://altimate.ai/platform) applies the same pattern to agents that change compute settings. Its Databricks SQL warehouse Auto Tune runs each change as snapshot, apply and verify. It rolls a change back when p95 query latency passes 1.5 times its baseline-week level and also rises by a couple of seconds. ## Start With a Read-Only Role and a Statement Timeout for Agent Sessions Set up the two warehouse controls first, because neither one needs a new tool. Create the `agent_readonly` role and the timed `agent_wh` warehouse from the SQL above. Then add the CI gate, so every agent branch passes the same checks as a human branch. [Altimate Code](https://altimate.ai/products/altimate-code) is an [AI Data Engineering Agent](https://altimate.ai/) that installs with `npm install -g altimate-code`. The same package ships Analyst mode, deny rules and the `check` command. [The missing harness in your data stack](https://altimate.ai/blog/why-ai-agents-break-in-production-the-missing-harness-in-your-data-stack) covers the wider design for data agents. [An agent-first data platform](https://altimate.ai/blog/signs-data-platform-needs-agent-first-overhaul) covers the same problem at the platform level. ## Frequently Asked Questions Why do AI agents fail on production data when they pass in dev? A dev schema lacks the conditions that trigger most production failures. Inherited roles, full-size tables, schema changes from other teams and downstream readers all exist only in production. Should an AI agent hold write access to a production warehouse? Start without it. Give agent sessions a read-only role, and let the agent write through pull requests that CI checks. How do I cap what an agent query can cost on Snowflake? Run the agent on its own warehouse with `STATEMENT_TIMEOUT_IN_SECONDS` set. Attach a resource monitor that suspends the warehouse at a credit quota. Only the ACCOUNTADMIN role can create the monitor. Does Altimate Code stop an agent from dropping a production table? Analyst mode blocks DROP outright. Builder mode always blocks `DROP DATABASE`, `DROP SCHEMA` and `TRUNCATE`, and it asks before other writes. A `deny` rule in the config file still applies when `--yolo` approves every other prompt. Which control should a data team add first? Add the read-only agent role first. A write that the role cannot perform cannot reach production, whatever the agent decides to do. On this page 1. Seven Ways AI Agents Fail on Production Data That a Sandbox Never Shows 2. An Agent Role That Inherits Write Grants Can Change Production Tables 3. A Query That Is Cheap in Dev Can Burn Credits on a Production Warehouse 4. A Schema That Changed Since the Agent Last Read It Breaks Its SQL 5. Incremental Models and Backfills Fail on State the Dev Build Never Had 6. A Model Diff Hides the Dashboards and Syncs That Read the Model 7. Production Rows Carry Personal Data Into the Agent's Context 8. An Agent With Write Access Needs a Gate Between Its Branch and Production 9. Start With a Read-Only Role and a Statement Timeout for Agent Sessions From the product - [Enterprise Platform Root cause analysis for a dbt, Tableau, or Snowflake workload spike 6 min read](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis) - [Enterprise Platform Reduce Snowflake idle warehouse spend with Altimate Lite 6 min read](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) - [Enterprise Platform Snowflake Cortex AI cost breakdown by service, user, and model 6 min read](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown) - [Enterprise Platform Attribute Snowflake cost to the team that owns it 4 min read](https://altimate.ai/product-highlights/snowflake-cost-alerts-by-team) - [Enterprise Platform Snowflake cost analysis in plain English, with AI that cites every number 5 min read](https://altimate.ai/product-highlights/snowflake-cost-analysis-ai-cited-answers) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # How I Stopped Running dbt Compile 50 Times a Day URL: https://altimate.ai/blog/how-i-stopped-running-dbt-compile-50-times-a-day dbt compile renders Jinja into SQL and writes it to target/. How I moved the edit, compile and check loop into a dbt VS Code extension, feature by feature. ## How I Stopped Running dbt Compile 50 Times a Day dbt compile renders Jinja into SQL and writes it to target/. How I moved the edit, compile and check loop into a dbt VS Code extension, feature by feature. Atharva Shah 2026-07-14 2026-09-16 9 min read On this page 8 sections 1. What dbt compile Does and Where It Writes the SQL 2. Jinja, ref() and Macros Are Why the Compile Loop Exists 3. A Compiled SQL Preview Replaces the Terminal Round Trip 4. Query Results and CTE Previews Show the Rows in the Editor 5. SQL Validation Catches the Column Names That Compile Cleanly 6. Lineage and Defer Remove the Last Two Terminal Trips 7. The Official VS Code Extension Covers Part of the Same Loop 8. Install the Extension and Check Your Next Model in the Editor tl;dr `dbt compile` renders the Jinja in a model into plain SQL and writes the result to the `target/` directory. A model file is a template. Rendering it is the only sure way to see what a `ref()`, a macro or an `is_incremental()` branch becomes. That need creates the loop of edit, compile, open the rendered file and check. I moved most of those checks into the editor with Power User for dbt, a free VS Code extension. Six of its features each remove one reason to open a terminal. They are the compiled SQL preview, query results, CTE preview, SQL validation, lineage and defer to production. Column lineage and the AI features need a free API key, and everything else needs no account. A dbt model is a Jinja template, and the SQL that runs is whatever dbt renders from it. So a day of model work turns into a loop. You edit the model and run `dbt compile`. You open the rendered file under `target/` and read it. Then you go back to the editor. The 50 in the title is a round number for a day of those loops, not a count I kept. No single pass through the loop takes long. The cost is the switch from editor to terminal and back. You also read a file that is not the one you edit. Most of my compile loops answered one of four questions: - What SQL did this `ref()` or macro render to? - What rows does the rendered query return? - Does every column in the query exist in the warehouse? - What breaks downstream if I change this model? A dbt VS Code extension can answer all four inside the editor. I work at Altimate, which makes Power User for dbt. I use that extension for each answer below. Two loops side by side. The left loop has five steps, which are edit the model, switch to the terminal, run dbt compile, open the rendered file under target, and read it. The right loop has two steps inside VS Code, which are edit the model and read the compiled SQL preview beside it *The terminal loop has five steps. The preview loop has two, and both stay in the editor.* Power User for dbt is an open source VS Code extension that adds column lineage, autocomplete, AI docs, query previews and health checks to your dbt project. [Explore Power User for dbt](https://altimate.ai/products/dbt-power-user) ## What `dbt compile` Does and Where It Writes the SQL `dbt compile` generates executable SQL from your project files. [dbt's command reference](https://docs.getdbt.com/reference/commands/compile) lists models, data tests, analysis files, functions and snapshots as its inputs. The compiled SQL files land in the `target/` directory of the project. Running that SQL is a separate step, and the docs suggest you run the compiled `select` yourself while you debug. The same page corrects two common assumptions: - `dbt compile` is not a prerequisite of `dbt run` or other build commands, because those commands compile on their own. - To read and validate a project without connecting to the warehouse, dbt points you to `dbt parse`. Since dbt v1.5, compile also works interactively. The `--select` flag compiles one node by name, and `--inline` compiles an arbitrary dbt-SQL query. Both print the compiled SQL to the terminal, and dbt still writes it to `target/`. ```bash dbt compile --select "stg_orders" dbt compile --inline "select * from {{ ref('raw_orders') }}" ``` In the docs' example, the inline query renders the `ref()` call to a fully qualified name, `"jaffle_shop"."main"."raw_orders"`. That output answers one question per command, and it answers it in a terminal. ## Jinja, `ref()` and Macros Are Why the Compile Loop Exists The `.sql` file in your editor is a template, and dbt renders it into SQL at compile time. A `ref()` call becomes a relation name for the current target. A macro call becomes whatever SQL the macro returns. You can read the template and predict the output. That prediction can still miss a macro's default argument, a variable or a branch. One branch depends on the state of the warehouse rather than on the file. [dbt's incremental models docs](https://docs.getdbt.com/docs/build/incremental-models) say `is_incremental()` returns true only when three things hold: - the model's table already exists in the database, - the run does not pass the `--full-refresh` flag, and - the model config sets `materialized='incremental'`. So one file compiles to two statements. A first run or a full refresh leaves the filter out, and a later run keeps it. The docs' own example shows the pattern: ```sql {{ config(materialized='incremental') }} select *, my_slow_function(my_column) from {{ ref('app_data_events') }} {% if is_incremental() %} where event_time >= (select coalesce(max(event_time), '1900-01-01') from {{ this }}) {% endif %} ``` The docs also say the SQL must be valid whether `is_incremental()` evaluates to true or false. To check both versions, you render the model in both states, and each state costs another trip through the loop. One incremental dbt model on the left compiles to two SQL statements on the right. The top statement is for a first run or a full refresh and has no where clause. The bottom statement is for a later run and adds a where filter on event\_time against the existing table *One template, two compiled statements. The state of the table decides which one runs.* ## A Compiled SQL Preview Replaces the Terminal Round Trip Power User for dbt is a [dbt VS Code extension](https://altimate.ai/products/dbt-power-user) with 1M+ installs across VS Code and Open VSX. Its compiled SQL preview shows the rendered SQL for the open model. The [compiled SQL preview docs](https://help.altimate.ai/dbt-power-user/develop/compiledCode/) place it as a toolbar action at the top right of VS Code. The product page describes the preview as available while you write. The docs do not say the preview re-renders on every keystroke. Open it after an edit, and read it beside the model. The rendered `ref()` names, macro output and join conditions sit next to the template that produced them. Set two things before you use the preview. Associate `.sql` files with `jinja-sql`, and set `dbt.dbtIntegration` to match how you run dbt: ```json { "files.associations": { "*.sql": "jinja-sql", "*.yml": "jinja-yaml" }, "dbt.dbtIntegration": "core" } ``` The [configuration reference](https://help.altimate.ai/dbt-power-user/setup/configuration/) shows that `dbt.dbtIntegration` accepts `core`, `cloud`, `fusion` or `corecommand`. The default is `core`. The extension supports dbt Core, dbt Cloud and dbt Fusion, on dbt 1.0 and above. The compiled SQL preview is free and needs no account. ## Query Results and CTE Previews Show the Rows in the Editor The compiled SQL tells you what will run. The rows tell you whether it is right. Checking them used to mean pasting the compiled `select` into a SQL client. The [query results docs](https://help.altimate.ai/dbt-power-user/test/queryResults/) describe the in-editor version: - Select the whole model or part of it, then press Cmd+Enter on Mac or Control+Enter on Windows and Linux. - Read the rows in the Query Results panel, and open its SQL tab to see the exact query that ran. - Save a result in a tab, change the query, and compare the old result with the new one. The preview returns 500 rows by default. The `dbt.queryLimit` setting changes that number. The docs say no adapter needs a custom `dbt.queryTemplate` anymore, including Oracle and MS SQL. ```json { "dbt.queryLimit": 500 } ``` When a model chains several CTEs, an Execute CTE action above each one runs that CTE on its own. You can then see which step changes the numbers. After a query returns, a Profile this query button starts an Altimate Code session that looks for performance problems. The published walkthrough shows how to [read a query plan in plain English](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan). Power User for dbt is an open source VS Code extension that adds column lineage, autocomplete, AI docs, query previews and health checks to your dbt project. [Explore Power User for dbt](https://altimate.ai/products/dbt-power-user) ## SQL Validation Catches the Column Names That Compile Cleanly `dbt compile` renders the template, and a wrong column name renders as cleanly as a right one. In one Altimate experiment, an agent shipped a dbt model with two hallucinated column names. That model still passed `dbt compile`, and it would have failed on its first run. The experiment is written up as one of the [four blind spots of general coding agents](https://altimate.ai/blog/4-blind-spots-of-general-coding-agents-in-data-engineering). The extension's Validate SQL command checks three things without running the query: - It flags columns that do not exist. - It flags typos in SQL keywords. - It highlights missing or extra parentheses. The command writes its findings to the Problems panel. It also reports four failures that used to pass silently. In the first three, dbt has no manifest loaded, or the manifest lacks the model or its parent entry. The fourth is a compile error, such as a broken `ref()`. The message for a missing manifest tells you to run `dbt parse`. Each error carries a Fix with Altimate Code button, which sends the compiled SQL and the errors to Altimate Code. That button is an AI feature, so it needs an API key. ## Lineage and Defer Remove the Last Two Terminal Trips The fourth question, what breaks downstream, needs lineage. Model lineage in the extension shows sources, seeds, models, tests, metrics and exposures, and it needs no API key. Column lineage needs a free Altimate AI API key, set in `dbt.altimateAiKey`. In the column view, a solid line means data flows through a `select`. A dotted line means the column appears in a `where`, `join` or `having` clause. The [lineage docs](https://help.altimate.ai/dbt-power-user/test/lineage/) list the gaps. Column lineage does not support snapshots yet. Unnest, lateral view flatten and JSON flatten can leave the lineage incomplete. Defer to production is the second fix. It runs a subset of models or tests without building their upstream parents first. You turn it on in the Actions panel, then point it at a production `manifest.json`. Local mode reads that manifest from your machine and needs no API key. SaaS mode stores the manifest for the team, and it needs an API key. It also needs `pip install altimate-datapilot`, which uploads the manifest. The `dbt.deferConfigPerProject` setting holds three properties: `deferToProduction`, `manifestPathForDeferral` and `favorState`. The AI features send query context to Altimate's backend. The only stored data is feedback you choose to give, and it is kept for 30 days. Telemetry follows VS Code's standard framework, and you can turn it off in VS Code settings. ## The Official VS Code Extension Covers Part of the Same Loop dbt Labs publishes its own extension, `dbtLabsInc.dbt`, in the VS Code Marketplace. The [dbt extension docs](https://docs.getdbt.com/docs/about-dbt-extension) call it a public preview release and say its behavior may change before general availability. The same page says dbt v1 and v2 both support it. [dbt v2](https://docs.getdbt.com/docs/introduction) is Rust-based and is now the default when you install dbt. It builds on an Apache 2.0 runtime. dbt Labs' May 2026 product update claims 30x faster parsing and real-time SQL feedback as you type. Both are vendor claims, and I have not measured either one. Power User for dbt also runs on Fusion, with `dbt.dbtIntegration` set to `fusion`. [Agentic data engineering tools for VS Code](https://altimate.ai/blog/best-agentic-data-engineering-tools-vs-code) compares the dbt Labs extension with eight other tools. In Cursor, install Power User for dbt from the Open VSX Registry. Some Cursor installs freeze and show "Failed to fetch", and the install docs give a workaround. The extension also embeds an MCP server that runs on localhost, covered in [dbt Power User inside Cursor](https://altimate.ai/blog/supercharging-cursor-ide-how-the-dbt-power-user-extensions-embedded-mcp-server-unlocks-ai-driven-dbt-development). ## Install the Extension and Check Your Next Model in the Editor Install Power User for dbt from the VS Code Marketplace, or from the command line: ```bash code --install-extension innoverio.vscode-dbt-power-user ``` Then click the dbt status icon in the bottom status bar to start the setup wizard. The wizard walks you through the Python interpreter, the dbt install, `dbt deps` and a project check. Open a model that calls a macro, and read the compiled SQL preview before you open a terminal. If the preview answers your question, you skipped one `dbt compile` loop. The [Power User for dbt page](https://altimate.ai/products/dbt-power-user) lists the full feature set. The page on [analytics engineering workflows](https://altimate.ai/use-cases/analytics-engineering) shows where the extension fits in a team's day. ## Frequently Asked Questions Does dbt compile run my models? `dbt compile` generates executable SQL and writes it to the `target/` directory, but it does not build your models. `dbt run` and `dbt build` compile on their own, so you do not need to run compile first. Where does dbt compile write the compiled SQL? It writes the files to the `target/` directory of your dbt project. It also prints the compiled SQL to the terminal when you pass `--select` or `--inline`, which dbt added in v1.5. Why does my incremental model compile to different SQL on different runs? `is_incremental()` returns true only when three conditions hold. The table exists, the run skips `--full-refresh`, and the model is incremental. A first run or a full refresh renders the model without the incremental filter. Is Power User for dbt free? The core extension is open source under the MIT license. Autocomplete, model lineage, compiled SQL preview, click-to-run, defer to production, SQL validation and query preview need no account. Column lineage and the other AI features need a free Altimate AI API key. Does Power User for dbt work with dbt Fusion? It works with dbt Fusion when you set `dbt.dbtIntegration` to `fusion`. It also supports dbt Core and dbt Cloud, on dbt 1.0 and above. On this page 1. What dbt compile Does and Where It Writes the SQL 2. Jinja, ref() and Macros Are Why the Compile Loop Exists 3. A Compiled SQL Preview Replaces the Terminal Round Trip 4. Query Results and CTE Previews Show the Rows in the Editor 5. SQL Validation Catches the Column Names That Compile Cleanly 6. Lineage and Defer Remove the Last Two Terminal Trips 7. The Official VS Code Extension Covers Part of the Same Loop 8. Install the Extension and Check Your Next Model in the Editor From the product - [PowerUser Plugin for dbt Profile a slow dbt query and read its execution plan in plain English 4 min read](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan) - [Enterprise Platform Root cause analysis for a dbt, Tableau, or Snowflake workload spike 6 min read](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Cost per Task vs Dollars per Million Tokens for AI Agents URL: https://altimate.ai/blog/cost-per-task-vs-dollars-per-million-tokens Cost per task divides all agent spend, failed runs included, by the tasks that pass your rule. A method, a formula and worked numbers for data engineering. ## Cost-Per-Task vs. Dollars-Per-Million-Tokens Cost per task divides all agent spend, failed runs included, by the tasks that pass your rule. A method, a formula and worked numbers for data engineering. Atharva Shah 2026-08-14 2026-09-16 10 min read On this page 9 sections 1. Cost per Task Divides All Agent Spend by Completed Tasks 2. Write the Completion Rule Before You Count a Dollar 3. The Formula Counts Every Attempt and Only Completed Tasks 4. SWE-Bench Prices Opus 4.6 at 1.5X Haiku 4.5 per Resolved Task 5. Empty Runs Need Their Own Column Next to Pass Rate 6. Skills Raised ADE-Bench Cost per Attempt by 21% and per Solved Task By 5% 7. Add Reviewer Time Only When You Log It 8. Log Each Attempt in One Table to Get Cost per Completed Task 9. Rerun Your Model Choice on a Fixed Set of Your Own Tasks tl;dr Dollars per million tokens is the price of an input. Cost per task is the price of a result. To get it, add the spend on every attempt, including failed and empty runs. Then divide by the tasks that met a completion rule you wrote first. On the SWE-bench Verified leaderboard, Claude Opus 4.6 costs $0.55 per task and Claude Haiku 4.5 costs $0.33. Per resolved task, the gap narrows to $0.73 against $0.50. In our DataAgentBench run, DeepSeek v4 pro cost less per passing trial than Claude Sonnet 4.6. It also left 79 of 270 trials empty. Log the empty runs next to the pass rate, because pass rate hides them. Your team wants to know which model to run its data engineering agent on. [Anthropic's pricing page](https://platform.claude.com/docs/en/about-claude/pricing) and its peers answer in dollars per million tokens. Your finance lead asks a different question: what does one finished dbt change cost? Cost per task answers the finance question. It is total agent spend divided by the number of tasks that passed a completion rule. The spend includes every retry, every failed run and every run that returned nothing. A price per token feeds into that number, but tokens per task and the pass rate decide most of it. The method has four steps. Write the completion rule, log every attempt, divide spend by completed tasks, and count empty runs in their own column. Add reviewer time only when you record it, because an estimated minute makes the number look precise when it is a guess. A million tokens is also a weak unit across models. The same prompt can tokenize to a different count on a newer model. Our post on why [a token is not a portable unit of work](https://altimate.ai/blog/the-great-token-heist-of-26) quotes Anthropic's migration guide at "roughly 1x to 1.35x" for one model change. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## Cost per Task Divides All Agent Spend by Completed Tasks Cost per task is the number that stays comparable when you swap the model, the agent or the prompt. A price per token changes meaning with each tokenizer. A pass rate hides what each attempt cost. Cost per task puts both in one figure. Two runs can show why. A model at a low token price can retry three times and still fail. A model at a higher token price can pass on the first attempt. The per-token price favors the first model, and cost per task favors the second. ## Write the Completion Rule Before You Count a Dollar A task counts as completed only when it passes a rule you wrote before the run. Without a written rule, a generous reviewer marks more tasks as done, and the cost per task drops for no reason. For dbt work, a practical rule has three parts: - The change compiles, which `dbt compile` or `dbt build` confirms. - The model and its tests pass with `dbt build --select `. - A person reviewed the diff and approved it. ADE-Bench uses a strict version of the second part. Each task runs in Docker against a dbt project and a database, and a task passes only when all of its dbt tests pass. Use the same binary rule, so a half-working change counts as a failure. Keep the rule fixed for the whole comparison. If you tighten it halfway, the early and late runs measure different things. ## The Formula Counts Every Attempt and Only Completed Tasks The numerator sums every attempt. The denominator counts only completed tasks. The two counts differ on purpose, because the tasks that passed carry the cost of each failed or empty run. ```text cost_per_completed_task = ( sum of model spend over all attempts, including failed and empty runs + reviewer_minutes x cost_per_minute ) <- only if you log the minutes / number of tasks that passed the completion rule empty_run_rate = attempts that wrote no output / all attempts ``` Report `empty_run_rate` beside the result. Two setups can share a cost per task and still differ on how often they return nothing. An empty run needs a different response than a wrong one. To estimate cost per task before you run anything, our token post gives a planning formula: ```text cost_per_task = (price_per_token x tokens_per_request x (1 + reasoning_overhead) x (1 - cache_hit_rate x cache_discount) x (1 - batch_share x 0.5)) / task_success_rate ``` The estimate is useful for a budget. The measured formula above is what you compare setups with, because it uses your own tasks and your own failures. ## SWE-Bench Prices Opus 4.6 at 1.5X Haiku 4.5 per Resolved Task The [SWE-bench leaderboard](https://www.swebench.com/) publishes an average cost per task next to the resolve rate. Its default view runs each model through the same bash-only agent on SWE-bench Verified, a human-filtered set of 500 tasks. So the model is the only variable, and you can divide cost by resolve rate yourself. | Model on SWE-bench Verified | % resolved | Avg. $ per task | $ per resolved task, our arithmetic | | --- | --- | --- | --- | | Claude 4.5 Haiku (high) | 66.6% | $0.33 | $0.50 | | Claude 4.6 Opus | 75.6% | $0.55 | $0.73 | | Claude 4.5 Sonnet (high) | 71.4% | $0.66 | $0.92 | | Claude 4.5 Opus (high) | 76.8% | $0.75 | $0.98 | A bar chart of four Claude models on SWE-bench Verified. Each model shows two bars, average cost per task and cost per resolved task: Haiku 4.5 at $0.33 and $0.50, Opus 4.6 at $0.55 and $0.73, Sonnet 4.5 at $0.66 and $0.92, Opus 4.5 at $0.75 and $0.98 *SWE-bench Verified average cost per task, and the same cost divided by resolve rate.* Opus 4.6 lists at $5 per million input tokens, five times Haiku 4.5's $1. On this leaderboard, Opus 4.6 costs 1.67x more per task and about 1.47x more per resolved task. The price ratio overstates the task cost ratio by a factor of three. Opus 4.5 resolves 10.2 more points than Haiku 4.5 and costs almost twice as much per resolved task. Whether those 10.2 points are worth it depends on what an unresolved task costs you. A person who finishes the task by hand is part of that cost. These figures have limits. SWE-bench tasks are general software issues, with no dbt project and no warehouse. The harness also moves the score. Anthropic reports 73.3% for Haiku 4.5 on its own scaffold, against 66.6% on the leaderboard's bash-only agent. ## Empty Runs Need Their Own Column Next to Pass Rate A pass rate folds every failure into one number, and a failure that returns nothing behaves differently from a wrong answer. Our DataAgentBench A/B shows the gap. One agent ran 270 trials on each model, with the same prompts, tools and validators. | Measure | Claude Sonnet 4.6 | DeepSeek v4 pro | | --- | --- | --- | | Cost per trial | $0.76 | $0.29 | | Raw Pass@1 | 63.0% | 60.0% | | Cost per passing trial, our arithmetic | $1.21 | $0.48 | | Trials that never wrote an answer | 32 of 270 | 79 of 270 | | Median duration per trial | 4 min | 9 min | Two panels. The left panel shows cost per passing trial, $1.21 for Claude Sonnet 4.6 and $0.48 for DeepSeek v4 pro. The right panel shows trials that never wrote an answer, 32 of 270 for Sonnet and 79 of 270 for DeepSeek *Cost per passing trial and empty runs from the same 540 trials.* On cost per passing trial, DeepSeek v4 pro wins by about 2.5x. The empty-run column changes the decision for some teams. DeepSeek wrote nothing on 29% of trials and Sonnet on 12%. An empty run is easy to detect with a presence check, and it still costs you something. If your pipeline retries the task, the retry spend goes into the numerator. If a person picks it up, the reviewer term applies. At a 9-minute median, a retry also delays the result more than a Sonnet retry at 4 minutes. Our write-up on [cost per successful trial](https://altimate.ai/blog/deepseek-vs-claude-sonnet-ai-agent-benchmark) has the full failure breakdown. The two models failed a similar number of trials, and the failures took opposite shapes. Altimate Lite is a Snowflake Native App that auto-tunes your warehouses as workloads change and breaks down your AI costs by model, user and service. [Try Altimate Lite for Snowflake](https://altimate.ai/products/altimate-lite) ## Skills Raised ADE-Bench Cost per Attempt by 21% and per Solved Task By 5% In January 2026 we ran the 43 ADE-Bench tasks on Snowflake with Claude Sonnet 4.5, with and without our dbt skills. The [pass rate on ADE-Bench](https://altimate.ai/benchmarks/ade-bench) and the cost moved together. | Setup | Cost per task | Tasks solved | Run total, our arithmetic | Cost per solved task, our arithmetic | | --- | --- | --- | --- | --- | | No skills | $0.33 | 20 of 43 | $14.19 | $0.71 | | With skills | $0.40 | 23 of 43 | $17.20 | $0.75 | The skills run cost $3.01 more in total and solved three more tasks. So each additional solved task cost about $1.00 in model spend. That marginal figure is the one to compare against a person's time to finish a task by hand. The [43 tasks across five dbt projects](https://altimate.ai/blog/teaching-claude-code-the-art-of-data-engineering-introducing-altimate-skills) are small demo projects, so the dollar figures will be larger on a production project. The ratio between the two setups is the part to test on your own work. A plugin experiment on Claude Sonnet 4.6 gave a smaller example of the same arithmetic. On the ADE-Bench task `intercom003`, the plugin turned a fail into a pass for about $0.13 extra. ## Add Reviewer Time Only When You Log It Reviewer time is the easiest term in the formula to invent. Put it in the formula only when you record it per task. An estimated "ten minutes per review" turns a measured number into a guess. One low-effort way is a required field in the pull request template, such as `review_minutes`. A reviewer fills it in when they approve. After two weeks, you have a per-task figure from your own team. Keep the reviewer term in a separate column as well as in the total. A setup that cuts model spend and doubles review time looks cheaper on the model column alone. [What each agent costs a data team](https://altimate.ai/blog/altimate-code-vs-claude-code-vs-cursor) lists the other costs that sit above the license price. ## Log Each Attempt in One Table to Get Cost per Completed Task The measurement needs one row per attempt. [Claude Code](https://code.claude.com/docs/en/costs) resets its `/usage` session totals when `/clear` starts a new session. So one session per task gives you a per-task cost estimate. For billing-grade numbers, the costs doc points to the Usage page in the Claude Console and to OpenTelemetry export. ```sql -- agent_attempts: one row per attempt -- (task_id, model, attempt_no, usd, wrote_output, build_passed, review_approved) with per_task as ( select task_id, model, sum(usd) as usd_all_attempts, count(*) as attempts, sum(case when not wrote_output then 1 else 0 end) as empty_attempts, max(case when build_passed and review_approved then 1 else 0 end) as completed from agent_attempts group by task_id, model ) select model, count(*) as tasks, sum(completed) as completed_tasks, round(sum(usd_all_attempts), 2) as usd_total, round(sum(usd_all_attempts) / nullif(sum(completed), 0), 2) as usd_per_completed_task, round(avg(attempts), 2) as attempts_per_task, round(sum(empty_attempts) * 1.0 / sum(attempts), 3) as empty_run_rate from per_task group by model order by usd_per_completed_task; ``` Run the same task list through each setup. [An AI data engineering agent](https://altimate.ai/) such as Altimate Code validates SQL in compiled code before it answers. The query above shows whether that check lowers attempts per task on your project. Altimate has published no cost per task for Opus, Haiku or GPT models on dbt work. The SWE-bench and DataAgentBench rows above are the closest figures we can source. [Where the tokens in one task go](https://altimate.ai/blog/the-hidden-cost-of-ai-coding-tools-why-your-token-bill-lies) splits one Claude Sonnet 4.6 trial into cache reads, cache writes and the rest. ## Rerun Your Model Choice on a Fixed Set of Your Own Tasks Pick a fixed list of tasks from your own backlog, and write the completion rule first. Run each model or agent on the full list, and log every attempt in the table above. Compare cost per completed task and the empty-run rate together. Repeat the run when a vendor ships a new model or changes a price. A new tokenizer or a new thinking default can move cost per task while the price per token stays flat. Altimate Code swaps models without code changes. Install [Altimate Code](https://altimate.ai/products/altimate-code) with `npm install -g altimate-code`, then point it at your own LLM key. ## Frequently Asked Questions What is cost per task for an AI agent? Cost per task is total model spend across every attempt, divided by the number of tasks that passed your completion rule. Failed runs, empty runs and retries all count in the spend. Only passing tasks count in the divisor. Why is dollars per million tokens a poor way to compare AI coding tools? Dollars per million tokens prices one input. It says nothing about how many tokens a task uses, how often the model retries or how often it fails. Token counts also change between tokenizers, so the same work can bill a different number of tokens. Is a cheaper model always cheaper per task? A cheaper model is often cheaper per task, and the gap is smaller than the price list suggests. On SWE-bench Verified, Opus 4.6 lists at five times Haiku 4.5's input price and costs about 1.47x more per resolved task. How do I count retries in cost per task? Log each attempt as its own row with its own spend. Sum the spend for all attempts on a task, then divide by completed tasks. A task that took three attempts carries the cost of all three. Should I include engineer review time in cost per task? Include review time only when you record it per task, for example in a pull request template field. Keep it in its own column as well, so a drop in model spend does not hide a rise in review time. On this page 1. Cost per Task Divides All Agent Spend by Completed Tasks 2. Write the Completion Rule Before You Count a Dollar 3. The Formula Counts Every Attempt and Only Completed Tasks 4. SWE-Bench Prices Opus 4.6 at 1.5X Haiku 4.5 per Resolved Task 5. Empty Runs Need Their Own Column Next to Pass Rate 6. Skills Raised ADE-Bench Cost per Attempt by 21% and per Solved Task By 5% 7. Add Reviewer Time Only When You Log It 8. Log Each Attempt in One Table to Get Cost per Completed Task 9. Rerun Your Model Choice on a Fixed Set of Your Own Tasks From the product - [PowerUser Plugin for dbt Profile a slow dbt query and read its execution plan in plain English 4 min read](https://altimate.ai/product-highlights/profile-a-slow-dbt-query-execution-plan) - [Enterprise Platform Root cause analysis for a dbt, Tableau, or Snowflake workload spike 6 min read](https://altimate.ai/product-highlights/snowflake-workload-root-cause-analysis) - [Enterprise Platform Reduce Snowflake idle warehouse spend with Altimate Lite 6 min read](https://altimate.ai/product-highlights/altimate-lite-snowflake-warehouse-cost) - [Enterprise Platform Snowflake Cortex AI cost breakdown by service, user, and model 6 min read](https://altimate.ai/product-highlights/snowflake-cortex-ai-cost-breakdown) - [Enterprise Platform Attribute Snowflake cost to the team that owns it 4 min read](https://altimate.ai/product-highlights/snowflake-cost-alerts-by-team) [All product highlights](https://altimate.ai/product-highlights) Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Altimate AI vs Snowflake Cortex Code URL: https://altimate.ai/comparisons/altimate-vs-snowflake-cortex Altimate AI vs Snowflake Cortex Code (now CoCo) on the public ADE-Bench leaderboard, 78% vs 65%, plus warehouse coverage, BYO LLM, and deterministic guardrails. Warehouse-native ## Altimate AI vs Snowflake Cortex Code Altimate Code scores higher on the benchmark, works across multiple platforms and tools, and lets you use other LLMs for significant cost savings. [Try Altimate Code free →](https://help.altimate.ai/code/getting-started/) Updated August 2026 ADE-Bench score Altimate AI 78% Snowflake Cortex Code 65% TL;DR Snowflake Cortex Code, renamed CoCo in June 2026, scores 65% on the public ADE-Bench leaderboard. Altimate Code scores 78% on the same leaderboard. It connects across a dozen and more data platforms vs Snowflake-anchored, supports true BYO LLM across 10+ providers, and ships a deterministic Rust validation core alongside the Altimate Platform for cross-stack lineage Cortex Code cannot produce. Side by side ## Altimate AI vs Snowflake Cortex Code capability comparison Every capability, grouped by area. Capability comparison between Altimate AI and Snowflake Cortex Code, grouped by area. | Capability | Altimate AI | Snowflake Cortex Code | | --- | --- | --- | | Benchmark & Proof | | ADE-Bench score (public leaderboard) | 78% | 65% | | AI Coding Agent | | Multi-platform connectivity (over a dozen data platforms) | Yes | No | | True BYO LLM (10+ providers, local models) | Yes | No | | Deterministic Rust validation core | Yes | No | | MCP server and SDK for agent embedding | Yes | Yes | | Agent Teams (parallel sub-agent orchestration) | No | Yes | | Platform Features | | Air-gappable / self-hostable | Yes | No | | MIT-licensed open source | Yes | No | | dbt Development | | dbt-native project intelligence | Yes | Yes | | Security & Governance | | PII detection across schema PR CI | Yes | No | | Data residency within Snowflake perimeter | No | Yes | | Lineage & Catalog | | Cross-stack lineage (dbt → warehouse → BI) | Yes | No | | Developer Surfaces | | Headless CI execution | Yes | No | | Works in VS Code, Claude Code, Cursor | Yes | Yes | | Integrations & Execution | | Native Snowflake platform integration (RBAC, catalog) | No | Yes | 5 reasons ## Where Altimate wins ### 78% vs 65% on the public ADE-Bench leaderboard On the benchmark dbt Labs built, Altimate Code scores 78% while Cortex Code scores 65%, both verifiable at altimate.ai/benchmarks/ade-bench. ### over a dozen data platforms, no Snowflake gravity Cortex Code pulls toward Snowflake for residency and catalog, while Altimate Code runs the same harness across a dozen and more data platforms including BigQuery, Databricks, and Redshift. ### True BYO LLM, not a curated model menu Cortex Code offers only the models Snowflake hosts, while Altimate Code takes Anthropic, OpenAI, Google, Bedrock, Azure, Ollama, local models, and Cortex itself. ### Deterministic Rust core: LLM reasons, engines validate Altimate's Rust core validates schemas in 2ms, lints SQL at 0.48ms per query, and catches PII offline before any SQL reaches the warehouse. ### No consumption incentive conflict Snowflake earns more when you spend more on Snowflake, whereas Altimate's advice cuts warehouse spend no matter which warehouse you run. Straight answer ## Where Snowflake Cortex Code wins ### Deeply integrated into the Snowflake platform For all-Snowflake teams, Cortex Code's native Cortex Search, Analyst, Snowpark, and RBAC awareness need no extra install, while Altimate Code is a separate setup. ### Billing through the existing Snowflake account CoCo bills through Snowflake at $20/month plus tokens with a 30-day, $40-credit trial, an easy add for committed teams, while Altimate bills separately. What it costs ## Snowflake Cortex Code pricing vs Altimate AI Altimate Code Free MIT-licensed · BYO LLM · no lock-in Or use our Model via Model Gateway: Sign up free — 10M tokens $29/month — 20M tokens Snowflake Cortex Code $20/mo subscription + to… $20/mo subscription + token consumption via Snowflake; 30-day trial [See full Altimate pricing →](https://altimate.ai/pricing) Common questions ## Snowflake Cortex Code comparison FAQs How does Altimate Code compare to Cortex Code on ADE-Bench? Altimate AI's open-source data engineering harness, Altimate Code, scores 78% on the public ADE-Bench leaderboard (74.4% on the earlier cut). Snowflake Cortex Code scores 65% on the same leaderboard, running Claude Opus 4.6 on Snowflake. Both scores are verifiable at altimate.ai/benchmarks/ade-bench and github.com/dbt-labs/ade-bench. Can I use cheaper LLMs with Altimate Code to cut costs? Yes. Altimate Code is true BYO LLM across 10+ providers — Anthropic, OpenAI, Google, Bedrock, Azure, Ollama, local models, and Snowflake Cortex itself — so you can route to a cheaper or self-hosted model for significant savings, or use our managed Model Gateway. Cortex Code runs only the models Snowflake hosts. Does Altimate Code work with Snowflake? Yes. Snowflake is one of Altimate AI's primary supported warehouses. It connects directly via profiles.yml or environment credentials, runs queries, inspects schemas, analyses warehouse cost, and validates SQL against your Snowflake environment. You can also use Snowflake Cortex as the LLM provider inside Altimate AI. Should I use Cortex Code or Altimate Code if my entire stack is Snowflake? Both are viable. Cortex Code offers deep native integration, data residency within Snowflake's perimeter, and billing through your existing Snowflake account, a natural fit for teams fully committed to Snowflake. Altimate Code offers higher ADE-Bench performance, true BYO LLM, a deterministic offline validation layer, and no structural conflict between cost-reduction advice and platform revenue. Many Snowflake-native teams run both: Cortex Code for platform-native features, Altimate Code for the harness layer and cross-stack work. Is Cortex Code really warehouse-agnostic now? Cortex Code now touches dbt and Airflow, which Snowflake positioned as extending beyond Snowflake. But touching dbt and Airflow is different from being warehouse-neutral. The data residency, catalog dependency, and cross-region inference requirements still pull toward Snowflake. Try a Cortex Code workflow that lands in BigQuery or traces lineage to Tableau, and the architecture returns to Snowflake. Altimate runs the same harness across a dozen and more data platforms with no platform gravity. How does Cortex Code pricing work vs Altimate? CoCo starts with a 30-day trial that includes $40 in free credits, then a $20/month subscription plus token-consumption billing through your Snowflake account. Snowsight usage bills tokens to the Snowflake account, and Snowflake compute is billed separately on top. Altimate Code is MIT-licensed free with BYO keys. Pro at $29/seat/month includes 20M tokens via a managed gateway, separate from your Snowflake spend. Ready to try Altimate Code? MIT-licensed. Free to install. Install via npm in a few seconds. No credit card. No vendor lock-in. Bring your own LLM. [Install Altimate Code →](https://help.altimate.ai/code/getting-started/quickstart-new/) [← All comparisons](https://altimate.ai/comparisons) --- # Altimate AI vs SELECT (select.dev) URL: https://altimate.ai/comparisons/altimate-vs-select-dev Altimate AI vs SELECT for Snowflake cost optimization: autonomous intraday warehouse sizing, IDE prevention, Altimate Studio, spend forecasting, and pricing. ## Altimate AI vs SELECT (select.dev) SELECT is primarily a cost visibility and reporting tool. Altimate AI provides agents that run custom cost analysis, implement optimization recommendations, and prevent costly issues earlier in the development lifecycle. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) [Try Altimate Lite Free Snowflake Native App](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) Updated August 2026 Pricing Altimate AI Starting at $100/mo via Altimate Lite Snowflake Native App SELECT (select.dev) Starting at $1,499/mo TL;DR - Altimate AI changes warehouse sizing far more frequently within a day, dynamically based on live load, with detailed explanations and an audit history of every change. - Prevent costly mistakes during development of dbt and SQL code through IDE extensions, the altimate-code CLI, and Git integrations. - Altimate Studio offers AI agents that run custom analysis and reports across your entire data stack — warehouses, pipelines, and BI layers — using metadata such as query profiles, lineage, and dashboard and report access history. - SELECT's architecture runs entirely inside Snowflake, so all data processing for insights and recommendations happens on the customer's own Snowflake warehouses. Altimate AI's enterprise edition runs that processing in the Altimate AI SaaS instead, avoiding the higher cost of operation SELECT's approach imposes on the customer. Side by side ## Altimate AI vs SELECT (select.dev) capability comparison Every capability, grouped by area. Capability comparison between Altimate AI and SELECT (select.dev), grouped by area. | Capability | Altimate AI | SELECT (select.dev) | | --- | --- | --- | | Snowflake Cost Optimization | | Warehouse autotune with intraday resizing and decision explanations | Yes | No | | Autonomous changes checked against query SLAs | Yes | No | | Query timeout predictions | Yes | No | | Predictive spend forecasting and budget alignment | Yes | No | | Cost visibility and reporting | Yes | Yes | | dbt model and workload level cost visibility | Yes | Yes | | Databricks Cost Optimization | | Automated job and cluster optimization | Yes | Yes | | Databricks cost visibility and attribution | Yes | Yes | | Fixes & Prevention | | AI-generated code fixes for savings opportunities | Yes | No | | Assign saving opportunities to team members | Yes | Yes | | Prevention in the IDE and CI, before spend is committed | Yes | No | | AI agent for custom analysis and report creation | Yes | No | | BI & Lineage | | Tableau user license downgrade recommendations | Yes | No | | Tableau dashboard refresh and view frequency drifts | Yes | No | | Cross-stack lineage (dbt → warehouse → BI) | Yes | No | | Platform & Coverage | | AI agent for data pipeline development (BYO LLM) | Yes | No | | Pre-purchased Snowflake credit payment | Yes | Yes | 5 reasons ## Where Altimate wins ### Warehouse resizing that adapts through the day Altimate AI changes warehouse sizing far more frequently within a day, adjusting dynamically to live load, and records a detailed explanation and full audit history for every change. SELECT tunes on a coarser cadence and does not expose the per-change decision trail. ### Costly mistakes caught in development, not after the bill dbt and SQL cost problems are prevented while the code is still being written — through the IDE extensions, the altimate-code CLI, and Git integrations — instead of being reported after the workload has already run and billed. SELECT has no development-time surface. ### AI agents for custom analysis across the whole stack Altimate Studio offers AI agents that run custom analysis and build reports across your entire data stack — warehouses, pipelines, and BI layers — using metadata such as query profiles, lineage, and dashboard and report access history. SELECT's reporting is fixed to its own dashboards. ### Processing that doesn't run on your warehouse SELECT's architecture runs entirely inside Snowflake, so all processing for its insights and recommendations happens on your own Snowflake warehouses. Altimate AI's enterprise edition runs that processing in the Altimate AI SaaS, so the cost-optimization tool adds only minimal load to the warehouses it is meant to optimize. ### Tableau cost a warehouse view misses Altimate extends cost work into the BI layer: it recommends Tableau user license downgrades for underused seats, and detects dashboard refresh and view-frequency drift where extracts keep rebuilding data that is rarely opened. SELECT stops at the warehouse and does not see this spend. Straight answer ## Where SELECT (select.dev) wins ### You only need cost observability and reporting If the goal is a dedicated cost visibility and reporting surface — dashboards, per-query and per-user attribution, anomaly alerts — and there is no need for an AI-native approach with agents that analyze, fix, and prevent cost issues, SELECT covers that lane well. ### You need a specific integration SELECT already ships SELECT has built-in integrations with certain tools — for example Periscope or Opsgenie — that a given workflow may depend on. Where one of those is a hard requirement, SELECT's existing connector is the simpler path. Customer example ## Why an enterprise cloud security company chose Altimate over SELECT The team adopted Altimate to go past cost reporting: autotuning warehouse configurations, surfacing inefficient queries, and preventing costly mistakes from data consumers during development. The first phase returned more than $600K in annualized Snowflake savings, and the engagement then expanded from Snowflake cost savings into cost savings for dbt models and other data pipelines — an expansion path a reporting-only tool does not support. Common questions ## SELECT (select.dev) comparison FAQs Does Altimate Platform replace SELECT for Snowflake cost optimization? For most teams, yes. Both products execute warehouse changes autonomously, so the deciding factors are what sits around that execution. Altimate resizes warehouses intraday as load shifts, checks each autonomous change against your query SLAs, estimates what a new query or dbt model will cost before it runs, forecasts where spend is heading for budget planning, and generates the code changes for savings past the warehouse setting. SELECT's automated savings stop at warehouse-level actions, and query and code fixes stay assisted-only. For teams that want a cost observability surface on its own, SELECT's narrow lane is deep. For teams that want the code fixed too, Altimate covers both halves. Does running Altimate add to my Snowflake bill the way a monitoring tool can? With the enterprise edition, no. Only metadata — table names, schemas, query shapes — enters the Altimate AI SaaS, where the enterprise edition runs the analysis; your actual data stays in your environment. Because that processing happens in the Altimate AI SaaS rather than on your warehouses, the work of finding and fixing cost consumes only minimal compute on the warehouse it is measuring. SELECT's architecture runs inside Snowflake, where its cost monitoring executes as queries against your account\_usage data, so the observability itself draws more heavily on the warehouse it reports on. Does SELECT have an IDE surface or a coding agent? No. SELECT works from warehouse and query metadata after workloads run, with no VS Code extension, no CLI agent, and no coding surface. That makes every finding a post-mortem on spend already committed. Altimate shifts the same work left: Altimate Code sits at the IDE layer with dbt Power User at 1M+ installs, so an expensive model is flagged while it is still being written, and the same checks run again in CI. Does SELECT provide AI agents for custom analysis and report creation? No. SELECT is a cost visibility and reporting tool: it surfaces spend through fixed dashboards and attribution reports, with no AI agent to run custom analysis. Altimate Studio provides AI agents that build custom analyses and reports across your entire data stack — warehouses, pipelines, and BI layers — using metadata such as query profiles, lineage, and dashboard and report access history. How does SELECT pricing compare to Altimate? SELECT starts at $1,499/month and is custom from there, payable with pre-purchased Snowflake credits, and a quote is required for specifics. Altimate Platform starts at $100/month via the Altimate Lite Snowflake Native App, with custom enterprise pricing above that and a 30-day POV to prove the savings first. Full detail is on the Altimate pricing page. Ready to cut the bill? 30-day POV. We prove the savings before you commit. OR install the Altimate Lite Snowflake Native App with a free 21-day trial. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) [Try Altimate Lite Free Snowflake Native App](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) [← All comparisons](https://altimate.ai/comparisons) --- # Altimate AI vs dbt Wizard URL: https://altimate.ai/comparisons/altimate-vs-dbt-wizard Altimate AI vs dbt Wizard on the public ADE-Bench leaderboard, 78% vs 59%, plus GA status, VS Code support, BYO LLM, and the warehouse cost layer dbt lacks. dbt\-native ## Altimate AI vs dbt Wizard Altimate Code beat dbt's agent on dbt's own benchmark, works across multiple platforms and tools, and is open source with free tokens to get started. [Try Altimate Code free →](https://help.altimate.ai/code/getting-started/) Updated August 2026 ADE-Bench score Altimate AI 78% dbt Wizard 59% TL;DR dbt Wizard is dbt Labs' AI agent, still in preview or beta on every surface as of August 2026, with billing docs that say pricing and usage are subject to change. On the public ADE-Bench leaderboard, Altimate Code scores 78% while dbt Labs' agent posted 59%. Altimate Code is MIT-licensed, generally available, works with dbt Core and dbt Cloud without a paid-plan gate, and adds the warehouse cost layer Wizard does not have. Beyond dbt authoring, Altimate Code works across multiple platforms and tools — warehouses, BI, and cost — and stays free to start: it is open source with a free managed-token tier or your own LLM, where dbt Wizard's managed usage is plan-gated and its terminal bills tokens through your own AI provider. Side by side ## Altimate AI vs dbt Wizard capability comparison Every capability, grouped by area. Capability comparison between Altimate AI and dbt Wizard, grouped by area. | Capability | Altimate AI | dbt Wizard | | --- | --- | --- | | AI Coding Agent | | BYO LLM, including self-hosted / local models | Yes | Enterprise / CLI only | | Self-validation against the warehouse | Yes | Yes | | Deterministic Rust validation core | Yes | No | | Plan mode before applying changes | Yes | No | | Custom skills encoding team conventions | Yes | Yes | | Features | | ADE-Bench score (public leaderboard) | 78% | 59% (76% self-graded, different cut) | | General availability | GA | Preview / beta (Aug 2026) | | MIT-licensed open source | Yes | No | | Works without a paid platform plan | Yes | CLI only | | Works beyond dbt — Airflow, E/L tools, BI, stored procedures | Yes | No | | VS Code surface | Yes | No | | Terminal CLI | Yes | Yes | | Plugin for Claude Code and Codex | Yes | No | | Pre-execution cost estimate on new SQL | Yes | No | | Debug and fix dbt Cloud job runs | No | Yes | 5 reasons ## Where Altimate wins ### 78% vs 59% on the public ADE-Bench leaderboard On dbt's own benchmark, Altimate Code scores 78% while dbt's agent posted 59%; dbt's 76% is self-graded on a different, non-public set. ### GA today vs preview on every surface As of July 2026 Wizard is still preview or beta everywhere with pricing 'subject to change,' while Altimate Code is GA, MIT-licensed, and openly priced. ### VS Code, where the engineers already are Wizard has no VS Code surface, while Altimate owns it with dbt Power User at 460K+ installs, plus Altimate Code in the terminal, Cursor, and Copilot. ### BYO LLM without asterisks Wizard gates platform BYOK to Enterprise, blocks Claude subscriptions, and routes embeddings through its own OpenAI key, while Altimate's BYO LLM is unconditional across 10+ providers. ### The cost layer dbt does not have Wizard has no warehouse cost or FinOps capability, while Altimate estimates SQL cost at 85% accuracy and auto-tunes 10-20% off Snowflake compute. Straight answer ## Where dbt Wizard wins ### Native metadata engine inside the dbt platform Wizard grounds itself in dbt's lineage, tests, contracts, and run results and validates its own SQL, needing zero setup for teams deep in dbt Cloud. ### The default choice inside a dbt Cloud renewal For teams on paid dbt plans, Wizard arrives inside the existing contract, making it one renewal line item rather than a new vendor conversation. What it costs ## dbt Wizard pricing vs Altimate AI Altimate Code Free MIT-licensed · BYO LLM · no lock-in Or use our Model via Model Gateway: Sign up free — 10M tokens $29/month — 20M tokens dbt Wizard Paid dbt plans Managed usage metered by actions; the terminal is bring-your-own-key. dbt Wizard's managed usage requires a paid dbt plan (not the Developer plan) and is metered by actions; the Wizard terminal is free but bring-your-own-key, so you pay your AI provider directly. Altimate Code is MIT-licensed and free with BYO LLM keys, or the Model Gateway with a free 10M-token tier and $29/month for 20M tokens. [See full Altimate pricing →](https://altimate.ai/pricing) Common questions ## dbt Wizard comparison FAQs Is dbt Wizard generally available? No. As of July 2026, the Studio IDE surface is in public preview and the platform tab and terminal CLI are in public beta, all launched June 1, 2026 at Snowflake Summit. dbt Copilot stays available until Wizard goes GA, with a migration path and billing bridge. Altimate Code is GA and MIT-licensed today. How do Altimate Code and dbt Wizard compare on ADE-Bench? On the public ADE-Bench leaderboard, Altimate Code scores 78% and dbt Labs' agent posted 59%. dbt's June 2026 blog quotes 76%, but that score is self-graded on a different 75-task set that is not the public leaderboard cut. The public numbers are verifiable at altimate.ai/benchmarks/ade-bench. Does dbt Wizard work in VS Code? No. Wizard runs in dbt's Studio IDE, the platform home tab, and a terminal CLI. The dbt VS Code extension's agentic story is a Fusion migration that runs in third-party agents like Copilot or Cursor. Altimate's dbt Power User extension has 460K+ VS Code installs, and Altimate Code runs in the terminal and embeds into Cursor, Copilot, and Claude Code. Can I bring my own LLM to dbt Wizard? With caveats. The CLI supports OpenAI, Anthropic, Azure, Bedrock, Gemini, Snowflake Cortex, and Databricks Unity AI Gateway, but platform BYOK is Enterprise-only, Claude subscription licenses are not supported, and embeddings still route through dbt's managed OpenAI key. Altimate's BYO LLM has no such conditions and includes self-hosted models. What does dbt Wizard cost? Platform access requires a paid Starter, Enterprise, or Enterprise+ plan, metered in dbt Copilot actions (100, 5,000, and 10,000 per month respectively, with access cut off until the next cycle when exhausted). Wizard gets separate metering after July 13, 2026, and dbt's docs say pricing and usage are subject to change. The CLI is free but you pay your AI provider directly. Altimate Code is MIT-licensed free; Pro is $29/seat/month with 20M tokens included. Does dbt Wizard help with warehouse costs? No. There is no warehouse spend, query cost, or FinOps capability anywhere in Wizard's docs or marketing. Altimate pairs the coding agent with pre-execution cost estimates at 85% accuracy, Auto Tune savings of 10-20% on Snowflake compute, and cost attribution across teams and workloads. Ready to try Altimate Code? MIT-licensed. Free to install. Install via npm in a few seconds. No credit card. No vendor lock-in. Bring your own LLM. [Install Altimate Code →](https://help.altimate.ai/code/getting-started/quickstart-new/) [← All comparisons](https://altimate.ai/comparisons) --- # Altimate AI vs Databricks Genie Code URL: https://altimate.ai/comparisons/altimate-vs-databricks-genie-code Altimate AI vs Databricks Genie Code: public ADE-Bench proof vs internal benchmarks, over a dozen data platforms vs workspace-only, and BYO LLM. Warehouse-native ## Altimate AI vs Databricks Genie Code Altimate Code scores higher on the benchmark, works across a dozen and more data platforms and multiple tools rather than only inside the Databricks workspace, and lets you bring your own LLM for significant cost savings. [Try Altimate Code free →](https://help.altimate.ai/code/getting-started/) Updated August 2026 Pricing Altimate AI Free MIT-licensed · BYO LLM · no lock-in Databricks Genie Code Pay as you go TL;DR Databricks Genie Code went GA in March 2026 and runs only inside the Databricks workspace, with no CLI, no VS Code surface, and no dbt awareness. Its free era ended July 8, 2026: usage is now pay-as-you-go past 150 LLM DBUs per user per month, with agent-launched compute billed separately. Altimate Code is MIT-licensed, scores 78% on the public ADE-Bench leaderboard, works across a dozen and more data platforms including Databricks, plus many other data tools, and supports true BYO LLM. Side by side ## Altimate AI vs Databricks Genie Code capability comparison Every capability, grouped by area. Capability comparison between Altimate AI and Databricks Genie Code, grouped by area. | Capability | Altimate AI | Databricks Genie Code | | --- | --- | --- | | Benchmark & Proof | | ADE-Bench score (public leaderboard) | 78% | No public entry | | AI Coding Agent | | Multi-platform connectivity (over a dozen data platforms) | Yes | No | | True BYO LLM (10+ providers, local models) | Yes | No | | Deterministic Rust validation core | Yes | No | | MCP support | Yes | Yes | | Agent skills and custom instructions | Yes | Yes | | Developer Surfaces | | Terminal CLI | Yes | No | | VS Code surface | Yes | No | | dbt Development | | dbt project awareness | Yes | No | | Platform Features | | MIT-licensed open source | Yes | No | | Cost Optimization | | Pre-execution cost estimate on new SQL | Yes | No | | Warehouse auto-tune (Snowflake + Databricks) | Yes | No | | Security & Governance | | Native Unity Catalog governance inheritance | No | Yes | | Integrations & Execution | | In-workspace notebooks, Lakeflow, dashboards, MLflow | No | Yes | | End-to-end ML workflows (train, deploy, monitor) | No | Yes | 5 reasons ## Where Altimate wins ### A public benchmark, not an internal one Databricks quotes an internal 77.1% with no methodology and no public entry, while Altimate Code scores 78% on the public, reproducible ADE-Bench. ### over a dozen data platforms vs one workspace Genie Code runs only inside Databricks, while Altimate Code connects to over a dozen data platforms including Databricks, following the team through whatever the stack becomes. ### dbt-native where Genie Code has nothing Genie Code has no dbt features, while Altimate is dbt-native end to end with dbt Power User at 460K+ installs and cross-stack lineage into BI. ### True BYO LLM vs Databricks-controlled routing Databricks controls Genie Code's model routing and strips agentic features if you disable partner models, while Altimate Code takes any LLM across 10+ providers. ### Cost you can see before it lands Genie Code has no pre-execution estimate and now meters LLM usage in DBUs, while Altimate forecasts query cost at 85% accuracy and attributes spend per team. Straight answer ## Where Databricks Genie Code wins ### Unity Catalog governance for free Genie Code inherits permissions, audit logging, lineage, and semantics from Unity Catalog with nothing to configure, while Altimate connects to that governance as an external tool. ### Default-on distribution Genie Code ships on by default with 150 free LLM DBUs per user monthly and no install step, a reach Altimate cannot match inside Databricks. What it costs ## Databricks Genie Code pricing vs Altimate AI Altimate Code Free MIT-licensed · BYO LLM · no lock-in Or use our Model via Model Gateway: Sign up free — 10M tokens $29/month — 20M tokens Databricks Genie Code Pay as you go Pay as you go since Jul 2026: 150 free LLM DBUs/user/mo, then metered; compute separate [Visit Databricks Genie Code →](https://www.databricks.com/product/genie/code) Genie Code is pay-as-you-go since July 8, 2026: 150 free DBUs of LLM usage per user per month (about $10.50), then metered by LLM consumption in DBUs, with agent-launched compute billed separately and one shared billing tag across all Genie products. Altimate Code is MIT-licensed free with BYO LLM keys; Pro at $29/seat/month includes 20M tokens via a managed gateway, with cost visible before queries run. [See full Altimate pricing →](https://altimate.ai/pricing) Common questions ## Databricks Genie Code comparison FAQs Is Databricks Genie Code still free? No. Genie Code launched at no additional charge in March 2026, but pricing changed on July 8, 2026 to pay-as-you-go: 150 free DBUs of LLM usage per user per month (roughly $10.50 at US East rates), then metered by LLM consumption in DBUs. The SQL warehouses the agent spins up are billed separately, and all three Genie products share one billing tag with no per-product budget separation. Does Genie Code work outside the Databricks workspace? No. There is no CLI, no VS Code extension, and no terminal surface in Databricks' docs or marketing. It cannot execute against Snowflake, BigQuery, or Redshift; Unity Catalog federation extends data access, not agent execution. Altimate Code runs in the terminal and VS Code against over a dozen data platforms, including Databricks. How does Genie Code compare to Altimate Code on benchmarks? Genie Code's only published number is internal: 77.1% of real-world data science tasks versus 32.1% for an unnamed leading coding agent plus Databricks MCP, with the methodology unpublished. Altimate Code scores 78% on the public ADE-Bench leaderboard, verifiable at altimate.ai/benchmarks/ade-bench. Does Genie Code understand dbt projects? No dbt capability appears in Genie Code's documentation or product pages. Teams running dbt on Databricks pair the workspace agent with nothing dbt-aware. Altimate is dbt-native: project intelligence, test generation, column-level lineage, and a VS Code extension with 460K+ installs. Can we run Genie Code and Altimate Code together? Yes, and Databricks-committed teams often should. Genie Code covers in-workspace notebooks, dashboards, and ML workflows under Unity Catalog governance. Altimate covers the dbt repo, the terminal workflow, BYO LLM policy, pre-execution cost estimates, and every warehouse that is not Databricks. Ready to try Altimate Code? MIT-licensed. Free to install. Install via npm in a few seconds. No credit card. No vendor lock-in. Bring your own LLM. [Install Altimate Code →](https://help.altimate.ai/code/getting-started/quickstart-new/) [← All comparisons](https://altimate.ai/comparisons) --- # Altimate AI vs Claude Code URL: https://altimate.ai/comparisons/altimate-vs-claude-code Altimate AI vs Claude Code. A strong general coding agent with no data-engineering surface, next to the data layer it calls: dbt, lineage, and cost. General agent ## Altimate AI vs Claude Code Claude Code is the general agent. Altimate is the specialized data engineering harness and can be used as the [Altimate Claude Code plugin](https://github.com/AltimateAI/altimate-claude-plugin). [Try Altimate Code free →](https://help.altimate.ai/code/getting-started/) Updated August 2026 ADE-Bench score Altimate AI 78% Claude Code 40% TL;DR Claude Code is Anthropic's general-purpose coding agent and a partner to Altimate, not a replacement. It has no native warehouse connectivity, no dbt integration, and no data lineage; databases arrive only via MCP servers. On ADE-Bench, Altimate Code scores 78% while the baseline Claude Code posts 40%. Altimate ships the [Altimate Claude Code plugin](https://github.com/AltimateAI/altimate-claude-plugin) along with published skills, so the developers who already trust Claude Code get a data-domain specialist without switching tools. Side by side ## Altimate AI vs Claude Code capability comparison Every capability, grouped by area. Capability comparison between Altimate AI and Claude Code, grouped by area. | Capability | Altimate AI | Claude Code | | --- | --- | --- | | Benchmark & Proof | | ADE-Bench score (public leaderboard) | 78% | 40% (baseline) | | AI Coding Agent | | Native platform connectivity (over a dozen data platforms) | Yes | Via MCP servers only | | Deterministic Rust validation core | Yes | No | | BYO LLM (10+ providers, local models) | Yes | No | | General-purpose application coding | No | Yes | | MCP connectivity | Yes | Yes | | dbt Development | | dbt project intelligence | Yes | No | | Lineage & Catalog | | Column-level lineage (dbt → warehouse → BI) | Yes | No | | Cost Optimization | | Pre-execution cost estimate on new SQL | Yes | No | | Security & Governance | | PII detection across schema, PR, CI | Yes | No | | Platform Features | | MIT-licensed open source | Yes | No | | Developer Surfaces | | Desktop, web, mobile, Slack surfaces | No | Yes | | Terminal CLI and VS Code | Yes | Yes | | GitHub Actions / CI execution | Yes | Yes | 5 reasons ## Where Altimate wins ### 78% vs 40% on ADE-Bench On the public ADE-Bench, Altimate Code scores 78% while baseline Claude Code posts 40%; the gap is the harness, not the model. ### Native warehouse connectivity, not MCP-only Claude Code reaches databases only through MCP servers, while Altimate Code ships native drivers for over a dozen data platforms, inherits dbt profiles, and prices SQL at 85% accuracy. ### A deterministic validation layer Claude Code cannot prove its SQL before it runs, while Altimate's Rust core validates schemas, lints SQL, and detects PII offline at zero token cost. ### Cross-stack lineage a general agent cannot see A general agent misses that a column rename broke a Tableau dashboard downstream, while Altimate traces dbt to warehouse to BI and shows blast radius before every change. ### Model neutrality Claude Code drives only Anthropic models, while Altimate Code supports 10+ providers including Anthropic, so teams keep their model policy and still get the data harness. Straight answer ## Where Claude Code wins ### Best-in-class general coding agent For general software work, Claude Code's agentic search, multi-file edits, and broad surface coverage set the standard, while Altimate Code is purpose-built for data engineering. What it costs ## Claude Code pricing vs Altimate AI Altimate Code Free MIT-licensed · BYO LLM · no lock-in Or use our Model via Model Gateway: Sign up free — 10M tokens $29/month — 20M tokens Claude Code Pro $20/mo, Max $100-200… Pro $20/mo, Max $100-200/mo, Team/Enterprise; API tokens [Visit Claude Code →](https://claude.com/product/claude-code) Claude Code is included in Claude Pro ($20/month) and Max plans ($100 to $200/month), with Team and Enterprise options and token-based API consumption. Altimate Code is MIT-licensed free with BYO LLM keys, and its skills plug into Claude Code directly. Pro at $29/seat/month includes 20M tokens via a managed gateway. [See full Altimate pricing →](https://altimate.ai/pricing) Common questions ## Claude Code comparison FAQs Should I replace Claude Code with Altimate Code? No. Claude Code is a partner, not a rival. Keep Claude Code for general software work and add Altimate for the data layer. Altimate publishes skills for Claude Code, so the same developers get dbt graph awareness, cross-stack lineage, and warehouse cost context inside the agent they already run. Why does a data team need a separate agent at all? A general agent edits your dbt project file by file. It will not catch that a column rename broke a Tableau dashboard three layers downstream, and it cannot price a query before it runs. Cross-stack lineage, pre-execution cost estimates, and deterministic SQL validation are data-domain capabilities, not coding features. That is why baseline Claude Code posts 40% on ADE-Bench while Altimate Code scores 78%. How does Altimate work inside Claude Code? Through the [Altimate Claude Code plugin](https://github.com/AltimateAI/altimate-claude-plugin), MCP, and published skills. The plugin exposes Altimate Code's warehouse connectivity, dbt intelligence, lineage, and cost tools to Claude Code, the same pattern Snowflake uses by shipping CoCo as a Claude Code plugin. The model stays Anthropic's; the harness around it becomes data-aware. Does Claude Code connect to my warehouse? Not natively. Databases are reachable only through MCP servers, which someone has to stand up and maintain per warehouse. Altimate Code ships native drivers for over a dozen data platforms, resolves your existing dbt profiles automatically, and indexes schemas so the agent works from real column names rather than repo guesses. How do the two compare on price? Claude Code is included in Claude Pro at $20/month or Max plans at $100 to $200/month, plus token-based API consumption for teams. Altimate Code is MIT-licensed free with BYO keys, including an Anthropic key, and Pro at $29/seat/month includes 20M tokens. Running both costs less than one bad warehouse query that neither a general agent nor a human reviewer priced in advance. Ready to try Altimate Code? MIT-licensed. Free to install. Install via npm in a few seconds. No credit card. No vendor lock-in. Bring your own LLM. [Install Altimate Code →](https://help.altimate.ai/code/getting-started/quickstart-new/) [← All comparisons](https://altimate.ai/comparisons) --- # Altimate AI vs Capital One Slingshot URL: https://altimate.ai/comparisons/altimate-vs-capital-one-slingshot Altimate AI vs Capital One Slingshot: autonomous savings vs manual approval, optimization below the warehouse, and an open self-serve AI-agent platform. ## Altimate AI vs Capital One Slingshot Slingshot focuses on Snowflake cost optimization, predominantly at the warehouse and query level, many times resulting in severe SLA violations. Altimate AI supports Snowflake, Databricks, and other platforms. It delivers autonomous optimizations with explanations, prevents costly mistakes in development, and extends optimization to storage, data pipelines, and the BI layer. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) [Try Altimate Lite Free Snowflake Native App](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) Updated August 2026 Pricing Altimate AI Starting at $100/mo via Altimate Lite Snowflake Native App Capital One Slingshot Not publicly listed TL;DR - Altimate AI resizes warehouses autonomously, as often as every few minutes, where Slingshot only offers sizing recommendations in large time blocks — a coarser cadence that frequently leads to significant SLA violations. - Altimate optimizes not just queries but storage (clustering, fail-safe), data pipelines (dbt), and BI layers (Tableau). - Prevent costly mistakes during development of dbt and SQL code through IDE extensions, the altimate-code CLI, and Git integrations. - Altimate AI offers AI Studio — AI agents for deeper custom analysis and report generation across your entire data stack. - Teams collaborate on savings opportunities inside Altimate and adopt it self-serve, where Slingshot has no collaborative model and, as a bank's product, a months-long procurement. Side by side ## Altimate AI vs Capital One Slingshot capability comparison Every capability, grouped by area. Capability comparison between Altimate AI and Capital One Slingshot, grouped by area. | Capability | Altimate AI | Capital One Slingshot | | --- | --- | --- | | Snowflake Cost Optimization | | Warehouse rightsizing with scheduling and rollback | Yes | Yes | | Autonomous changes checked against query SLAs | Yes | No | | Clustering key recommendations | Yes | No | | Storage-layer cost incl. fail-safe and time travel | Yes | Yes | | Cost visibility and reporting | Yes | Yes | | dbt model and workload level cost visibility | Yes | No | | Optimization beyond warehouses (tables, pipelines, dbt, Tableau, AI services) | Yes | No | | Fixes & Prevention | | AI writes and applies the SQL fix as a pull request | Yes | No | | Assign saving opportunities to team members | Yes | No | | Prevention in the IDE and CI, before spend is committed | Yes | No | | AI agent for custom analysis and report creation | Yes | No | | BI & Lineage | | Tableau user license downgrade recommendations | Yes | No | | Tableau dashboard refresh and view frequency drifts | Yes | No | | Cross-stack lineage (dbt → warehouse → BI) | Yes | No | | Platform & Coverage | | Data platforms supported | Snowflake, Databricks, and others | Snowflake | | AI agent for data pipeline development (BYO LLM) | Yes | No | 5 reasons ## Where Altimate wins ### Autonomous optimization without SLA violations Altimate resizes warehouses autonomously, as often as every few minutes, adapting to live load. Slingshot offers sizing only as recommendations in large time blocks — a coarser cadence that frequently leads to significant SLA violations. Altimate also checks every autonomous change to sizing, clustering, and shutdowns against your query SLAs before it applies. ### Optimization beyond the query Slingshot's opportunities stop at the warehouse and the individual query. Altimate optimizes storage (clustering keys, fail-safe and time-travel retention), data pipelines (dbt models), and the BI layer (Tableau) — not warehouses alone. ### Costly mistakes prevented in development Slingshot reports on spend after workloads run, with no development-time surface. Altimate prevents costly dbt and SQL patterns while the code is still being written — through IDE extensions, the altimate-code CLI, and Git integrations. ### AI Studio agents for custom analysis and reports Slingshot is a fixed console with set reports and dashboards. Altimate AI offers AI Studio — AI agents that run deeper custom analysis and generate reports across your entire data stack, from a single question. ### Built for teams, adopted self-serve Slingshot lacks a collaborative model for a team to work an opportunity together, and as a bank's product it carries a months-long procurement. Altimate lets teams collaborate on savings and optimization opportunities, and adopt it self-serve. Straight answer ## Where Capital One Slingshot wins ### Governance around warehouse creation Slingshot provides guardrails around warehouse creation — controlling who can spin up new Snowflake warehouses and how they are sized — so teams that want tight governance over warehouse provisioning and size management have that built in. ### Existing Capital One relationship and brand trust If you already work with Capital One for other software and value brand trust above all else, Slingshot comes from a vendor you already have a relationship with — meaningful for a regulated enterprise that prefers to standardize on a provider it knows. Customer example ## Why a global beverage company moved to Altimate from Slingshot A global beverage company moved from Slingshot to Altimate. The deciding factor was depth and autonomy: optimization below the warehouse — clustering, table storage, and fail-safe — applied autonomously and landed as a pull request, on an open self-serve platform their team could collaborate in, rather than recommendations routed through manual approval. Common questions ## Capital One Slingshot comparison FAQs Does Slingshot apply savings autonomously, and safely against SLAs? Slingshot surfaces idle-time and scaling opportunities, but applies them in large time blocks through manual review and an approval workflow — a coarse cadence that frequently leads to SLA violations. Altimate resizes warehouses autonomously, as often as every few minutes, and checks every change to sizing, clustering, and shutdowns against your query SLAs before it applies. What does Altimate optimize that Slingshot doesn't? Both cover warehouse rightsizing and storage-layer cost, including fail-safe and time travel. Altimate goes further: clustering key recommendations, dbt model and pipeline cost, and the BI layer — Tableau license downgrades and refresh waste — with cross-stack lineage from dbt through the warehouse to the dashboard. Slingshot stays at the warehouse and query level. Does Slingshot have a development surface, or write the fix itself? Neither. Slingshot's Query Advisor is a paste-in tool: the engineer edits the SQL by hand and copies it back, and per Slingshot's docs the edited text is not saved, with no IDE, CLI, or coding agent. Altimate prevents costly dbt and SQL patterns while the code is still being written — through IDE extensions, the altimate-code CLI, and Git integrations — and its agent writes the change, validates it, and opens the pull request. Does Slingshot provide AI agents for custom analysis and report creation? No. Slingshot is a cost console with fixed reports and dashboards. Altimate AI offers AI Studio — AI agents that run deeper custom analysis and generate reports across your entire data stack, from a single question. Which data platforms does each support? Slingshot focuses on Snowflake. Altimate supports Snowflake, Databricks, and other platforms, so a multi-platform setup stays on one product rather than one console per platform. How much does Capital One Slingshot cost? Slingshot does not publish pricing on its official pages. Access is via a demo request or a Snowflake Marketplace free trial and a free Health Check app. Altimate Platform starts at $100/month via the Altimate Lite Snowflake Native App, with custom enterprise pricing above that and a 30-day POV to prove the savings first. Ready to cut the bill? 30-day POV. We prove the savings before you commit. OR install the Altimate Lite Snowflake Native App with a free 21-day trial. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) [Try Altimate Lite Free Snowflake Native App](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) [← All comparisons](https://altimate.ai/comparisons) --- # Altimate AI vs Revefi URL: https://altimate.ai/comparisons/altimate-vs-revefi Altimate AI vs Revefi: continuous autonomous tuning with auto-revert guardrails, deterministic AST-based SQL rewrites, and dbt-repo fixes before merge. ## Altimate AI vs Revefi Revefi began in data observability and has since expanded into cost observability with some basic optimization capabilities. Altimate AI, by contrast, prevents costly mistakes in development workflows, enables team-level collaboration with assignable savings opportunities, and, above all, autonomously optimizes warehouse sizing and shutdowns for significantly higher savings. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) [Try Altimate Lite Free Snowflake Native App](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) Updated August 2026 Pricing Altimate AI Starting at $100/mo via Altimate Lite Snowflake Native App Revefi Custom pricing depending on spend amount TL;DR - Altimate prevents costly dbt and SQL mistakes in the development workflow — through IDE extensions, the altimate-code CLI, and Git — instead of reporting the cost after the workload has run. - Savings and optimization opportunities are assignable to team members and worked collaboratively in Altimate; Revefi has no collaborative model. - Above all, Altimate autonomously optimizes warehouse sizing and shutdowns for significantly higher savings — Revefi does not autonomously optimize warehouses. Side by side ## Altimate AI vs Revefi capability comparison Every capability, grouped by area. Capability comparison between Altimate AI and Revefi, grouped by area. | Capability | Altimate AI | Revefi | | --- | --- | --- | | Snowflake Cost Optimization | | Autonomous warehouse sizing optimization | Yes | No | | Autonomous warehouse shutdown optimization | Yes | No | | Workload-type visibility (Notebooks, SPs, Cortex) | Yes | No | | Cost visibility, attribution, and reporting | Yes | Yes | | Fixes & Prevention | | Deterministic AST-based SQL rewrite (semantically identical) | Yes | No | | AI writes and applies the SQL fix as a pull request | Yes | No | | Prevents costly mistakes in development (IDE extensions, altimate-code CLI, Git) | Yes | No | | Team collaboration via assignment of saving opportunities | Yes | No | | AI agent for custom analysis and report creation (Altimate Studio) | Yes | Yes | | Monitoring & Lineage | | Tableau dashboard refresh and view frequency | Yes | Yes | | Tableau license downgrade recommendations | Yes | No | | Auto-created anomaly monitors on every table | No | Yes | | BI lineage coverage | Tableau | 4 BI tools | | Platform & Coverage | | AI agent for data pipeline development (BYO LLM) | Yes | No | | AI token cost attribution (user → agent → model) | No | Yes | 3 reasons ## Where Altimate wins ### Costly mistakes prevented in development Revefi surfaces cost after the workload has run. Altimate prevents costly dbt and SQL patterns while the code is still being written — through IDE extensions, the altimate-code CLI, and Git — so the waste never ships in the first place. ### Built for teams to collaborate on savings Revefi lacks a collaborative model for a team to work an opportunity together. In Altimate, savings and optimization opportunities are assignable to team members and worked collaboratively, so the work does not stall on a single owner. ### Autonomous warehouse sizing and shutdown optimization This is where the savings come from. Revefi's optimization stays basic — it surfaces opportunities without autonomously optimizing warehouses. Altimate autonomously optimizes warehouse sizing and shutdowns, driving significantly higher savings. Straight answer ## Where Revefi wins ### You need data quality observability too Revefi began in data observability and covers data quality, freshness, and anomaly monitoring alongside cost. If you need that data-quality observability in the same tool, it is Revefi's core strength. ### You want observability and reporting, not autonomous optimization If you want primarily a cost observability and reporting surface — dashboards, trends, and opportunity notifications — rather than a tool that autonomously optimizes warehouses, Revefi is built for that shape. Customer example ## Why an enterprise data-security company moved to Altimate from Revefi An enterprise data-security company moved from Revefi to Altimate. The deciding factor was autonomous cost optimization that actually delivers the savings — rather than just a cost observability tool that gives visibility without actual autonomous cost optimization. Common questions ## Revefi comparison FAQs Does Revefi autonomously optimize cost, or mainly observe it? Revefi began in data observability and has expanded into cost observability with some basic optimization. It surfaces spend opportunities and notifications, but it does not autonomously optimize warehouse sizing or shutdowns. Altimate autonomously optimizes both, which is where the significant savings come from. Does Revefi prevent cost issues during development? No. Revefi reports on cost after the workload has run. Altimate prevents costly dbt and SQL patterns while the code is still being written — through IDE extensions, the altimate-code CLI, and Git — so the waste never ships in the first place. Can a team collaborate on savings in Revefi? Revefi surfaces savings as notifications, with no collaborative model for a team to work an opportunity together. In Altimate, savings and optimization opportunities are assignable to team members and worked collaboratively, so nothing stalls on a single owner. Are Altimate's SQL rewrites safe? Yes. Altimate rewrites SQL through a deterministic AST parser paired with an AI agent, which guarantees the rewritten query is semantically identical to the original — a provably equivalent optimization, not a best-effort suggestion you re-verify by hand. Does Revefi optimize BI and Tableau cost? Partly. Revefi maps BI lineage across Tableau, Power BI, Looker, and ThoughtSpot, and surfaces Tableau dashboard refresh-vs-read frequency opportunities. It does not optimize BI cost directly. Altimate audits unused Tableau licenses and extract refreshes that rebuild data nobody opened, recommending license downgrades and cutting refresh waste. What does Revefi cost? Revefi does not publish full pricing; the pricing page is gated and priced based on your data platform spend. Altimate Platform starts at $100/month via the Altimate Lite Snowflake Native App, with custom enterprise pricing above that and a 30-day POV to prove the savings first. Ready to cut the bill? 30-day POV. We prove the savings before you commit. OR install the Altimate Lite Snowflake Native App with a free 21-day trial. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) [Try Altimate Lite Free Snowflake Native App](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) [← All comparisons](https://altimate.ai/comparisons) --- # Altimate AI vs Seemore Data URL: https://altimate.ai/comparisons/altimate-vs-seemore-data Altimate AI vs Seemore Data: prevention in development, multi-cloud beyond Snowflake, an AI Studio, and automated query tagging for team chargebacks. ## Altimate AI vs Seemore Data Seemore Data is a Snowflake-focused cost observability and optimization tool. Altimate AI prevents costly mistakes in development, works across Snowflake, Databricks, and other platforms, and gives teams an AI Studio and local IDE/CLI development for SQL and dbt. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) [Try Altimate Lite Free Snowflake Native App](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) Updated August 2026 Pricing Altimate AI Starting at $100/mo via Altimate Lite Snowflake Native App Seemore Data No public pricing TL;DR - Altimate re-optimizes warehouses at a minutes-level cadence through the day, where Seemore's SmartPulse optimizes hourly — so idle time and spikes are caught between Seemore's cycles. - Altimate prevents costly SQL and dbt mistakes before they run — shift-left through IDE extensions and the altimate-code CLI — instead of flagging cost after the workload has run. - Altimate goes beyond Snowflake with multi-cloud support across Snowflake, Databricks, DuckDB, and more, where Seemore is Snowflake-focused. - Altimate's AI Studio is a flexible workspace for FinOps, cross-workload dependency checks, ad-hoc analysis, and SQL editing. - Savings and optimization opportunities are assignable to team members and worked collaboratively in Altimate, where Seemore surfaces cost without a collaborative model. - Granular team-level showback and chargeback in Altimate, where every team member gets their own view of costs and savings. Side by side ## Altimate AI vs Seemore Data capability comparison Every capability, grouped by area. Capability comparison between Altimate AI and Seemore Data, grouped by area. | Capability | Altimate AI | Seemore Data | | --- | --- | --- | | Snowflake Cost Optimization | | Autonomous warehouse optimization | Yes | Yes | | Intraday warehouse optimization (minutes-level cadence) | Yes | No | | Anomaly detection with spike alerts | Yes | Yes | | Per-project / per-warehouse budgets and autonomy modes | Yes | Yes | | Storage-layer and clustering cost optimization | Yes | No | | Cost visibility and reporting | Yes | Yes | | Fixes & Prevention | | Prevents costly mistakes in development (IDE extensions, altimate-code CLI, Git) | Yes | No | | SQL / dbt fixes in your editor and PRs | Yes | No | | AI agent for custom analysis and report creation (Altimate Studio) | Yes | No | | Team collaboration via assignment of saving opportunities | Yes | No | | Granular team-level showback and chargeback (per-member views) | Yes | No | | BI & Lineage | | Tableau license and refresh cost optimization | Yes | No | | Cross-stack lineage into BI | Yes | No | | Platform & Coverage | | Data platforms supported | Snowflake, Databricks, and others | Snowflake | | AI agent for data pipeline development (BYO LLM) | Yes | No | 6 reasons ## Where Altimate wins ### Minutes-level optimization, not hourly Seemore's SmartPulse optimizes Snowflake warehouses on an hourly cadence. Altimate re-optimizes at a minutes-level cadence through the day, adapting to load between Seemore's cycles so idle time and spikes are caught sooner. ### Costly mistakes prevented before they run Seemore flags cost once the workload has already run and billed. Altimate shifts left: it prevents costly SQL and dbt patterns while the code is still being written, through IDE extensions and the altimate-code CLI, so the expensive model never reaches the warehouse. ### Beyond Snowflake to a multi-cloud stack Seemore's optimization is Snowflake-focused. Altimate runs the same cost surface across Snowflake, Databricks, DuckDB, and more, so a multi-platform setup stays on one product rather than one console per platform. ### An AI Studio for FinOps and analysis Altimate's AI Studio is a flexible workspace for FinOps, cross-workload dependency checks, ad-hoc analysis, and SQL editing — a surface Seemore's dashboards do not provide. ### Built for teams to collaborate on savings Seemore surfaces cost and budgets by domain. Altimate makes savings and optimization opportunities assignable to team members and worked collaboratively, so a finding becomes owned work rather than a monitored line item. ### Granular showback and chargeback Altimate breaks cost and savings down to the team level and gives each member their own view, so every team sees exactly what it spends and what it can save. Straight answer ## Where Seemore Data wins ### Relationship-based sales engagement model Seemore has no self-serve free signup — the only way to try the product is to book a demo and reach out to their sales team. If you prefer a relationship-based, sales-led evaluation over a self-serve trial, that model fits. ### Just platform-team-level cost optimization If you're comfortable with the platform team driving the majority of optimization — without data consumers like analysts and scientists actively involved in the process — Seemore's platform-team-centric approach works. Common questions ## Seemore Data comparison FAQs Does Seemore prevent cost issues during development? No. Seemore works from query history and warehouse metadata, so its findings arrive after the workload has run and billed. Altimate shifts left: it prevents costly SQL and dbt patterns while the code is still being written — through IDE extensions and the altimate-code CLI — so the expensive model never reaches the warehouse. What happens when your stack isn't only Snowflake? Seemore is Snowflake-focused. Altimate runs the same cost surface across Snowflake, Databricks, DuckDB, and more, and traces lineage across dbt, the warehouse, and Tableau in one place, so a multi-platform setup stays on one product rather than one console per platform. Can Seemore write dbt models or fix code locally? Not really. Seemore surfaces rewrites and one-click fixes, but there is no local IDE or CLI authoring. Altimate integrates with your IDE extension or CLI to write, preview, and edit SQL and dbt on your local machine, then lands the change as a pull request and routes the saving to Git or Jira as assignable work. What is Altimate Studio? Altimate Studio is a flexible AI workspace for FinOps, cross-workload dependency checks, ad-hoc analysis, and SQL editing — plain-language questions about spend across your whole data stack, a surface Seemore's dashboards do not provide. How do team chargebacks work? Altimate ships a dedicated snowflake\_query\_tags package that automates Snowflake query tagging and metadata enrichment, so cost attributes cleanly to the team that ran it and chargebacks become effortless — without hand-maintained tagging conventions. Ready to cut the bill? 30-day POV. We prove the savings before you commit. OR install the Altimate Lite Snowflake Native App with a free 21-day trial. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) [Try Altimate Lite Free Snowflake Native App](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) [← All comparisons](https://altimate.ai/comparisons) --- # Altimate AI vs Espresso AI URL: https://altimate.ai/comparisons/altimate-vs-espresso-ai Altimate AI vs Espresso AI: prevention in development, team collaboration on savings, and metadata-driven optimization that never sits in the query path. ## Altimate AI vs Espresso AI Espresso AI provides automatic warehouse and query optimization, but its query-routing proxy sits inline in the critical query path — a component every routed query passes through, with no published latency SLO. Altimate AI takes a less intrusive architecture for autonomous optimization, and provides team-level showback and chargeback, coverage across the entire data stack (storage, data pipelines, and BI reports), and prevention of costly issues in development. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) [Try Altimate Lite Free Snowflake Native App](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) Updated August 2026 Pricing Altimate AI Starting at $100/mo via Altimate Lite Snowflake Native App Espresso AI 36-40% of savings TL;DR - Savings and optimization opportunities are assignable to team members and worked collaboratively in Altimate; Espresso has no collaborative model. - Altimate optimizes the entire data stack, not just warehouses and queries — it optimizes storage, dbt models, and BI dashboards, and even performs workload-level root-cause analysis and comparisons. - Espresso's query-routing proxy sits inline in the critical query path — every routed query passes through it, with no published latency SLO, adding a layer to rule out when a query lags or fails. Altimate reads SQL from metadata, so it never rides on a live query. - Prevent costly mistakes during development of dbt and SQL code through IDE extensions, the altimate-code CLI, and Git integrations. Side by side ## Altimate AI vs Espresso AI capability comparison Every capability, grouped by area. Capability comparison between Altimate AI and Espresso AI, grouped by area. | Capability | Altimate AI | Espresso AI | | --- | --- | --- | | Snowflake Cost Optimization | | Autonomous warehouse tuning | Yes | Yes | | Autonomous query rewriting with equivalence proof | Yes | Yes | | Validates the optimized query against actual data (row-level) | Yes | No | | Cross-warehouse query routing | No | Yes | | Non-intrusive architecture, never inline in the critical query path | Yes | No | | Storage-layer cost incl. fail-safe and clustering | Yes | No | | Cost visibility and reporting | Yes | Yes | | Fixes & Prevention | | Prevents costly mistakes in development (IDE extensions, altimate-code CLI, Git) | Yes | No | | Fixes the source in the dbt model, not only the query | Yes | No | | AI agent for custom analysis and report creation (Altimate Studio) | Yes | No | | Workload-level root-cause analysis and run comparisons | Yes | No | | Team collaboration via assignment of saving opportunities | Yes | No | | Team-level showback and chargeback | Yes | No | | BI & Lineage | | Tableau license and refresh cost optimization | Yes | No | | Cross-stack lineage (dbt → warehouse → BI) | Yes | No | | Platform & Coverage | | Snowflake Native App deployment | Yes | No | | Optimization beyond warehouses (tables, pipelines, dbt, Tableau, AI services) | Yes | No | | AI agent for data pipeline development (BYO LLM) | Yes | No | 4 reasons ## Where Altimate wins ### Costly mistakes prevented before they run Espresso intercepts a query on the wire and rewrites it after it has already been written. Altimate shifts left: it prevents costly SQL and dbt patterns while the code is still being authored — from the IDE, the altimate-code CLI, and Git — so the expensive query never reaches the warehouse. ### Built for teams to collaborate on savings Espresso has no collaborative model for a team to work an opportunity together. In Altimate, savings and optimization opportunities are assignable to team members and worked collaboratively, so nothing stalls on a single owner. ### Optimization across the full stack Espresso works on compute at the query and warehouse level. Altimate optimizes across tables, pipelines, dbt, Tableau, and AI services with team chargeback — including storage-layer cost and the BI layer — not warehouses and queries alone. ### Never in the query path Espresso's query router sits inline in the request path of every query and, when a query lags or fails, becomes one more layer to rule out in root-cause analysis. Altimate reads SQL from metadata in the information schema, so tuning and anomaly detection never ride on a live query and never complicate an incident. Straight answer ## Where Espresso AI wins ### You just need autonomous functionality, not collaboration or chargeback If all you want is autonomous cost optimization — without team-level collaboration, showback, or chargeback — Espresso stays focused on that lane. ### Cross-warehouse query routing Espresso routes queries across warehouses and scales clusters to raise utilization, a genuine capability Altimate does not offer. It comes with an intrusive architecture, though: the router sits inline in the critical query path that every routed query passes through. Customer example ## Why a large fintech company chose Altimate over building its own query router A large fintech company built an in-house query router but, faced with the operational complexity of keeping a router reliable in the query path, adopted Altimate for Snowflake optimization instead — cost optimization that works from metadata, with no extra layer to operate and debug. Common questions ## Espresso AI comparison FAQs How does Altimate's query optimization compare to Espresso's? Both rewrite SQL and prove the result is equivalent to the original, but Altimate goes a step further: it offers an option to validate the optimized query against your actual data, running the rewrite and comparing the results row-for-row rather than relying on a static proof alone. The other difference is architecture: Espresso rewrites inline, in the query path, on every execution, while Altimate works from metadata and fixes the source — the dbt model or query the pattern came from — so an expensive pattern is corrected once instead of rewritten on every run, and nothing sits between your clients and Snowflake. Does Espresso's inline query router complicate root-cause analysis? It can. Espresso's Scheduler and Query Agent sit in the request path of every routed query, with no published p99 latency or throughput SLO. When a query lags or fails, the router is one more layer to rule out. Altimate uses a non-intrusive architecture: it ingests SQL through metadata in the information schema and never sits in the query path, so tuning and anomaly detection never ride on a live query and never complicate an incident. Can a team collaborate on savings in Espresso, with showback and chargeback? No. Espresso has no collaborative model for a team to work an opportunity together, and no team-level showback or chargeback. In Altimate, savings and optimization opportunities are assignable to team members and worked collaboratively, with team-level showback and chargeback, so nothing stalls on a single owner. Does Espresso prevent cost issues during development? No. Espresso operates at the query and warehouse level, rewriting after the fact, with no dbt integration and no development-time surface. Altimate shifts left: it is dbt-native and prevents costly SQL and dbt patterns while the code is still being written — through IDE extensions, the altimate-code CLI, and Git — so an expensive pattern is caught before it ever runs. Does Espresso optimize beyond warehouses and queries? No. Espresso works on compute at the query and warehouse level. Altimate optimizes the entire data stack — storage (including fail-safe and clustering), dbt models, and BI dashboards — and performs workload-level root-cause analysis and run comparisons, so cost is addressed wherever it actually lives, not just at the warehouse. How does Espresso's pricing and deployment compare to Altimate? Espresso is outcome-based: roughly 40% of monthly verified savings on its guaranteed-ROI plan, or 36% of estimated annual savings billed upfront. Altimate takes a much smaller share and deploys as a Snowflake Native App: Altimate Platform starts at $100/month via the Altimate Lite Snowflake Native App, with custom enterprise pricing above that and a 30-day POV to prove the savings first. Ready to cut the bill? 30-day POV. We prove the savings before you commit. OR install the Altimate Lite Snowflake Native App with a free 21-day trial. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) [Try Altimate Lite Free Snowflake Native App](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) [← All comparisons](https://altimate.ai/comparisons) --- # Altimate AI vs Keebo URL: https://altimate.ai/comparisons/altimate-vs-keebo Altimate AI vs Keebo: SLA-safe autonomous optimization vs aggressive warehouse shutdowns, full-stack coverage beyond warehouses, and dev-time prevention. ## Altimate AI vs Keebo Keebo autonomously optimizes Snowflake warehouses, but its aggressive shutdowns can trigger SLA violations. Altimate AI runs autonomous optimization SLA-safe, prevents costly mistakes in development, and optimizes across the full stack — storage, dbt pipelines, and the BI layer, not warehouses alone. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) [Try Altimate Lite Free Snowflake Native App](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) Updated August 2026 Pricing Altimate AI Starting at $100/mo via Altimate Lite Snowflake Native App Keebo Custom pricing share of realized savings TL;DR - Altimate's autonomous optimization is SLA-safe — where Keebo's aggressive warehouse shutdowns can trigger SLA violations, Altimate checks every change against your query SLAs before it applies. - Altimate optimizes across the full stack — query code, table storage, fail-safe storage, clustering, dbt pipelines, and BI dashboards — not warehouses alone. - Altimate prevents costly SQL before it deploys, from the IDE, instead of catching it after the warehouse has already run. - Altimate's AI Studio is a flexible workspace for FinOps, cross-workload dependency checks, ad-hoc analysis, and SQL editing. Side by side ## Altimate AI vs Keebo capability comparison Every capability, grouped by area. Capability comparison between Altimate AI and Keebo, grouped by area. | Capability | Altimate AI | Keebo | | --- | --- | --- | | Snowflake Cost Optimization | | Autonomous warehouse optimization | Yes | Yes | | Warehouse auto-suspend and auto-resume tuning | Yes | Yes | | Agentless, read-only Snowflake connection | Yes | Yes | | Cost anomaly detection and spend alerting | Yes | Yes | | Autonomous changes checked against query SLAs | Yes | No | | Clustering key recommendations | Yes | No | | Storage-layer cost incl. fail-safe and time travel | Yes | No | | Cost visibility and reporting | Yes | Yes | | Fixes & Prevention | | Prevents costly mistakes in development (IDE extensions, altimate-code CLI, Git) | Yes | No | | AI writes and applies the SQL fix as a pull request | Yes | No | | AI agent for custom analysis and report creation (Altimate Studio) | Yes | No | | Team collaboration via assignment of saving opportunities | Yes | No | | BI & Lineage | | Tableau license downgrade recommendations | Yes | No | | Tableau dashboard refresh and view frequency | Yes | No | | Cross-stack lineage (dbt → warehouse → BI) | Yes | No | | Platform & Coverage | | Optimization beyond warehouses (storage, dbt, BI, AI services) | Yes | No | | AI agent for data pipeline development (BYO LLM) | Yes | No | 4 reasons ## Where Altimate wins ### SLA-safe autonomous optimization Keebo optimizes warehouses autonomously, but its aggressive shutdowns can leave a query waiting on a cold warehouse and breach an SLA. Altimate checks every autonomous change to sizing, clustering, and shutdowns against your query SLAs before it applies, so a saving never lands by missing a deadline. ### Optimization across the full stack Keebo works at the warehouse. Altimate optimizes across the whole stack — query code, table storage, fail-safe storage, clustering, dbt pipelines, and BI dashboards — with full visibility into dbt, Tableau, stored procedures, and Snowflake AI services. ### Costly SQL prevented before it deploys Keebo tunes the warehouse after the workload has run. Altimate prevents costly SQL while the code is still being written — from the IDE, the altimate-code CLI, and Git — so the expensive query never reaches the warehouse. ### An AI Studio for the whole data stack Altimate's AI Studio is a flexible workspace for FinOps, cross-workload dependency checks, ad-hoc analysis, and SQL editing — questions Keebo's warehouse optimizer has no surface to answer. Straight answer ## Where Keebo wins ### You just want fully hands-off warehouse optimization Keebo autonomously optimizes Snowflake warehouse sizing and suspension with minimal configuration. For non-SLA-critical workloads where you don't need coverage across the whole data stack — or where you already have an existing relationship with Keebo — that set-and-forget warehouse focus is its strength. ### You want a credits-based pricing model You like Keebo's credits-based pricing model and relationship-based sales model vs. Altimate AI's pricing model that starts at $100 per month without the need to talk to sales. Customer example ## Why a manufacturing software company moved to Altimate from Keebo A hardware manufacturing software company moved from Keebo to Altimate after warehouse shutdowns triggered repeated SLA violations. Altimate's autonomous optimization is SLA-safe — every change is checked against query SLAs before it applies — so the savings hold without missing a deadline. Common questions ## Keebo comparison FAQs Is Altimate's autonomous optimization SLA-safe? Yes. Keebo optimizes warehouses autonomously, but its aggressive shutdowns can leave a query waiting on a cold warehouse and breach an SLA. Altimate checks every autonomous change to sizing, clustering, and shutdowns against your query SLAs before it applies, so cost reduction never comes at the cost of a missed deadline. Does Keebo optimize beyond the warehouse? No. Keebo focuses on autonomous warehouse sizing and suspension. Altimate optimizes across the full stack — query code, table storage, fail-safe storage, clustering, dbt pipelines, and BI dashboards — with visibility into dbt, Tableau, stored procedures, and Snowflake AI services. Does Keebo prevent cost issues during development? No. Keebo tunes the warehouse after the workload has run. Altimate prevents costly SQL and dbt patterns while the code is still being written — from IDE extensions, the altimate-code CLI, and Git — so the expensive query never reaches the warehouse. What is Altimate Studio? Altimate Studio is a flexible AI workspace for FinOps, cross-workload dependency checks, ad-hoc analysis, and SQL editing. It answers plain-language questions about spend across your whole data stack — a surface Keebo's warehouse optimizer does not have. How does Keebo pricing compare to Altimate? Keebo prices as a share of the savings it produces, quoted per environment. Altimate Platform starts at $100/month via the Altimate Lite Snowflake Native App, with custom enterprise pricing above that and a 30-day POV to prove the savings first. Ready to cut the bill? 30-day POV. We prove the savings before you commit. OR install the Altimate Lite Snowflake Native App with a free 21-day trial. [Book a Demo](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings) [Try Altimate Lite Free Snowflake Native App](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) [← All comparisons](https://altimate.ai/comparisons) --- # dbt Column Rename: Check the Blast Radius URL: https://altimate.ai/demos-walkthroughs/dbt-column-rename-blast-radius Renaming total_cents to gross_revenue_cents sounded like a one-line change. Blast radius analysis showed it touched models, metrics, YAML docs, and dashboards. [← All videos](https://altimate.ai/demos-walkthroughs) dbt · Column Lineage · Refactoring ## Before You Rename That dbt Column, Check the Blast Radius Renaming total\_cents to gross\_revenue\_cents sounded like a one-line change. Blast radius analysis showed it touched models, metrics, YAML docs, and dashboards. [Before You Rename That dbt Column, Check the Blast Radius](https://www.youtube.com/embed/wV3LJrqLRds) Overview Renaming a column in dbt sounds harmless. One field, one model, done. Until it isn't — and finance is asking why their dashboard broke. This video shows Altimate Code running a full blast radius analysis before a single character is changed. The rename target: total\_cents → gross\_revenue\_cents in staging\_orders. The agent traces the impact through every downstream dbt model, semantic model, metric definition, YAML documentation file, and BI dashboard that references the column. Once the blast radius is mapped, the rename is applied and every affected reference is updated in one pass. Targeted dbt builds confirm each fix, and a full project build validates the complete change. The lesson: a column rename touches far more than the model you're editing. The right order is always blast radius first, fix everything, then validate. What you'll learn - → How to run a column-level blast radius analysis before making a dbt change - → How a single column rename propagates through models, metrics, YAML docs, and dashboards - → How to apply a rename and update all downstream references in one automated pass - → How to validate refactoring changes with targeted and full project dbt builds Key points - Traces column impact through dbt models, semantic models, metrics, YAML, and dashboards - Identifies every reference to total\_cents before touching a single file - Applies the rename and updates all affected downstream references automatically - Validates with targeted per-model builds before running the full project build - Avoids the classic mistake: changing production SQL without knowing the downstream blast radius Try it yourself — 10M free tokens, no credit card required. [Get started free →](https://app.myaltimate.com/register) [Watch on YouTube ↗](https://www.youtube.com/watch?v=wV3LJrqLRds) More videos Migrate SQL Server to Snowflake with dbt and Altimate Code Migrate SQL Server to Snowflake with dbt and Altimate Code Watch → https://altimate.ai/demos-walkthroughs/sql-server-snowflake-dbt-migration How an AI Agent Adds Airflow DAGs That Actually Match Your Project How an AI Agent Adds Airflow DAGs That Actually Match Your Project Watch → https://altimate.ai/demos-walkthroughs/airflow-dag-pattern-matching Snowflake AI Services Cost Breakdown: Finally Understand Where Your Cortex Spend Goes Snowflake AI Services Cost Breakdown: Finally Understand Where Your Cortex Spend Goes Watch → https://altimate.ai/demos-walkthroughs/snowflake-ai-services-cost-breakdown --- # Migrate SQL Server to Snowflake with dbt URL: https://altimate.ai/demos-walkthroughs/sql-server-snowflake-dbt-migration Stored procedure logic in, validated dbt models out — with row count checks, schema diffs, and a migration dashboard stakeholders can actually trust. [← All videos](https://altimate.ai/demos-walkthroughs) Migration · SQL Server · Snowflake ## Migrate SQL Server to Snowflake with dbt and Altimate Code Stored procedure logic in, validated dbt models out — with row count checks, schema diffs, and a migration dashboard stakeholders can actually trust. [Migrate SQL Server to Snowflake with dbt and Altimate Code](https://www.youtube.com/embed/7MtD0NJjZS4) Overview Migrating SQL Server stored procedures to Snowflake and dbt is one of the highest-risk data projects a team can run. It's not just about translating syntax. It's about proving the output is identical. This walkthrough covers the full migration lifecycle with Altimate Code: reading stored procedure business logic, converting it into dbt models structured across staging, silver, gold, and analytics layers, and validating every step of the way. Migration best practices are applied throughout to ensure output matches team standards and the target Snowflake architecture. Validation runs at multiple levels: schema diff to catch structural mismatches, row count comparison between SQL Server and Snowflake, and column-level data diff to confirm migrated outputs match the original source exactly. The session ends with a migration validation dashboard showing status, results, model mappings, lineage, and key metrics — the kind of artifact that gives stakeholders actual confidence, not just an engineer's word that it worked. What you'll learn - → How to convert SQL Server stored procedure logic into structured dbt models - → How to apply a medallion architecture (staging → silver → gold → analytics) during migration - → How to validate migrations with schema diff, row count comparison, and column-level data diff - → How to generate a migration validation dashboard for stakeholder sign-off Key points - Reads SQL Server stored procedure logic and converts it into layered dbt models - Applies migration best practices and medallion layering throughout - Validates with row count checks between SQL Server and Snowflake - Runs schema diff and column-level data diff to confirm output matches the source - Generates an interactive migration validation dashboard for stakeholder review Try it yourself — 10M free tokens, no credit card required. [Get started free →](https://app.myaltimate.com/register) [Watch on YouTube ↗](https://www.youtube.com/watch?v=7MtD0NJjZS4) More videos Before You Rename That dbt Column, Check the Blast Radius Before You Rename That dbt Column, Check the Blast Radius Watch → https://altimate.ai/demos-walkthroughs/dbt-column-rename-blast-radius How an AI Agent Adds Airflow DAGs That Actually Match Your Project How an AI Agent Adds Airflow DAGs That Actually Match Your Project Watch → https://altimate.ai/demos-walkthroughs/airflow-dag-pattern-matching Snowflake AI Services Cost Breakdown: Finally Understand Where Your Cortex Spend Goes Snowflake AI Services Cost Breakdown: Finally Understand Where Your Cortex Spend Goes Watch → https://altimate.ai/demos-walkthroughs/snowflake-ai-services-cost-breakdown --- # Add Airflow DAGs That Match Your Project URL: https://altimate.ai/demos-walkthroughs/airflow-dag-pattern-matching One prompt added a new DAG to an existing Airflow + DuckDB project. It read the codebase, matched the patterns, and ran clean on the first try. [← All videos](https://altimate.ai/demos-walkthroughs) Airflow · Context-Aware Coding · DuckDB ## How an AI Agent Adds Airflow DAGs That Actually Match Your Project One prompt added a new DAG to an existing Airflow + DuckDB project. It read the codebase, matched the patterns, and ran clean on the first try. [How an AI Agent Adds Airflow DAGs That Actually Match Your Project](https://www.youtube.com/embed/DxWNgBvSMjU) Overview Adding a new DAG to an existing Airflow project isn't just a coding problem. It's a codebase-fitting problem: right connection pattern, right pool handling, right style. Get any of it wrong and the code sticks out immediately — or worse, silently breaks something. This video shows Altimate Code adding a fifth DAG to an open-source Airflow and DuckDB example repo in a single prompt. The agent reads the existing four DAGs to understand the project's connection setup, pool serialization pattern, and coding conventions, then creates a feature branch, writes the DAG file, commits it, and runs it successfully in Airflow on the first try. No Stack Overflow copy-paste. No manual pattern-matching across files. The result looks exactly like it was written by someone who had been on the project for months — because the agent read the project the same way a good engineer would. What you'll learn - → How an AI agent reads existing code to infer a project's implicit conventions - → Why codebase-fitting matters more than just 'working code' for Airflow DAGs - → How to add new DAGs that match connection patterns, pool handling, and team style - → What a one-prompt, end-to-end DAG creation workflow looks like in practice Key points - Reads 4 existing DAGs to infer connection patterns, pool usage, and code style - Creates a feature branch, writes the DAG file, and commits — all from one prompt - Uses the DuckDB pool pattern correctly to serialize concurrent file access - New DAG runs clean in Airflow on the first try with no manual edits - Result matches the team's existing codebase style without explicit instructions Try it yourself — 10M free tokens, no credit card required. [Get started free →](https://app.myaltimate.com/register) [Watch on YouTube ↗](https://www.youtube.com/watch?v=DxWNgBvSMjU) More videos Before You Rename That dbt Column, Check the Blast Radius Before You Rename That dbt Column, Check the Blast Radius Watch → https://altimate.ai/demos-walkthroughs/dbt-column-rename-blast-radius Migrate SQL Server to Snowflake with dbt and Altimate Code Migrate SQL Server to Snowflake with dbt and Altimate Code Watch → https://altimate.ai/demos-walkthroughs/sql-server-snowflake-dbt-migration Snowflake AI Services Cost Breakdown: Finally Understand Where Your Cortex Spend Goes Snowflake AI Services Cost Breakdown: Finally Understand Where Your Cortex Spend Goes Watch → https://altimate.ai/demos-walkthroughs/snowflake-ai-services-cost-breakdown --- # Snowflake AI Services Cost Breakdown URL: https://altimate.ai/demos-walkthroughs/snowflake-ai-services-cost-breakdown Your AI Services bill shows one number. This breaks it down by service type, model, user, and team — so you can answer when the CFO asks why costs spiked 40%. [← All videos](https://altimate.ai/demos-walkthroughs) Snowflake · Cost Optimization · Cortex AI ## Snowflake AI Services Cost Breakdown: Finally Understand Where Your Cortex Spend Goes Your AI Services bill shows one number. This breaks it down by service type, model, user, and team — so you can answer when the CFO asks why costs spiked 40%. [Snowflake AI Services Cost Breakdown: Finally Understand Where Your Cortex Spend Goes](https://www.youtube.com/embed/ILmjhm13PS8) Overview Snowflake shows you what you spent on AI Services. It doesn't tell you why. Which models are being called? By which teams? How many tokens are being consumed by Cortex Functions versus Document AI versus Fine-tuning? This video shows how to get complete, granular visibility into Snowflake AI Services spend using Altimate. Starting from the single "AI Services" line item that Snowflake surfaces, the breakdown drills into service type (Functions, Search, Analyst, Document AI, Fine-tuning), then into individual Cortex models, then into usage by user and team for accurate chargeback. When a cost spike of 40% lands in your inbox, "AI Services" isn't a useful answer. Model-level visibility is. This is what cost transparency actually looks like for AI workloads: knowing exactly where every credit went and which teams and models are driving the bill. What you'll learn - → How to break down Snowflake AI Services spend by service type, model, user, and team - → What drives Cortex Functions costs at the individual model level - → How to set up accurate chargeback reporting for AI services across teams - → How to identify optimization opportunities hidden inside a single AI Services line item Key points - Breaks down AI Services by type: Functions, Search, Analyst, Document AI, Fine-tuning - Drills into Cortex Functions cost at the individual model level - Tracks token consumption per model for accurate cost attribution - Surfaces user- and team-level spend for chargeback and accountability - Turns a single line item into actionable, model-level cost intelligence Try it yourself — 10M free tokens, no credit card required. [Get started free →](https://app.myaltimate.com/register) [Watch on YouTube ↗](https://www.youtube.com/watch?v=ILmjhm13PS8) More videos Before You Rename That dbt Column, Check the Blast Radius Before You Rename That dbt Column, Check the Blast Radius Watch → https://altimate.ai/demos-walkthroughs/dbt-column-rename-blast-radius Migrate SQL Server to Snowflake with dbt and Altimate Code Migrate SQL Server to Snowflake with dbt and Altimate Code Watch → https://altimate.ai/demos-walkthroughs/sql-server-snowflake-dbt-migration How an AI Agent Adds Airflow DAGs That Actually Match Your Project How an AI Agent Adds Airflow DAGs That Actually Match Your Project Watch → https://altimate.ai/demos-walkthroughs/airflow-dag-pattern-matching --- # 5 Hidden Snowflake Cost Leaks Only AI Can Find URL: https://altimate.ai/demos-walkthroughs/snowflake-cost-leaks-ai Five cost savings opportunities Snowflake's native tools won't surface — and how they added up to 32% off the bill. [← All videos](https://altimate.ai/demos-walkthroughs) Snowflake · Cost Optimization · Analytics ## 5 Hidden Snowflake Cost Leaks Only AI Can Find Five cost savings opportunities Snowflake's native tools won't surface — and how they added up to 32% off the bill. [5 Hidden Snowflake Cost Leaks Only AI Can Find](https://www.youtube.com/embed/Fn5yLAoeR3Y) Overview Your Snowflake bill keeps growing. The native tooling shows spend totals. It doesn't show the patterns that are silently burning compute you didn't authorize. This video walks through five cost leak categories that only become visible with AI-powered analysis: unexpected cost spikes that emerge without a clear trigger, failed queries that consume compute without producing output, performance degradation patterns that gradually drive costs up over time, run-to-run cost variance on jobs that should be predictable, and unpredictable runtimes that make capacity planning impossible. For each category, the video shows what the signal looks like in Altimate, why Snowflake's native tools miss it, and how to fix or automatically remediate it. Taken together, identifying and addressing these five patterns produced 32% savings on the total Snowflake bill — without any changes to the underlying data models or pipelines. What you'll learn - → How to identify unexpected cost spikes that Snowflake's native monitoring won't flag - → Why failed queries are a significant and often overlooked compute waste category - → How to detect gradual performance degradation before it compounds into a major cost problem - → How AI-driven Autotune automatically optimizes infrastructure to prevent recurring leaks Key points - Identifies 5 hidden cost categories: spikes, failures, degradation, variance, unpredictability - Shows why each category is invisible to Snowflake's built-in cost tooling - Failed queries waste compute budget with zero data output — AI surfaces the pattern - Run-to-run variance analysis catches cost growth before it shows up on the monthly bill - Autotune automatically remediates infrastructure issues without manual intervention - Combined savings across all five categories: 32% off total Snowflake spend Try it yourself — 10M free tokens, no credit card required. [Get started free →](https://app.myaltimate.com/register) [Watch on YouTube ↗](https://www.youtube.com/watch?v=Fn5yLAoeR3Y) More videos Before You Rename That dbt Column, Check the Blast Radius Before You Rename That dbt Column, Check the Blast Radius Watch → https://altimate.ai/demos-walkthroughs/dbt-column-rename-blast-radius Migrate SQL Server to Snowflake with dbt and Altimate Code Migrate SQL Server to Snowflake with dbt and Altimate Code Watch → https://altimate.ai/demos-walkthroughs/sql-server-snowflake-dbt-migration How an AI Agent Adds Airflow DAGs That Actually Match Your Project How an AI Agent Adds Airflow DAGs That Actually Match Your Project Watch → https://altimate.ai/demos-walkthroughs/airflow-dag-pattern-matching --- # AI Agent Peer Review for dbt SQL URL: https://altimate.ai/demos-walkthroughs/ai-sql-peer-review-dbt One prompt drives the full peer review. Seven mart models come out graded, fixed, and re-validated without a senior engineer in the loop. [← All videos](https://altimate.ai/demos-walkthroughs) dbt · SQL Quality · Code Review ## Can an AI Agent Peer Review dbt SQL Before It Hits Production? One prompt drives the full peer review. Seven mart models come out graded, fixed, and re-validated without a senior engineer in the loop. [Can an AI Agent Peer Review dbt SQL Before It Hits Production?](https://www.youtube.com/embed/Vee87oFkqD8) Overview When a junior engineer pushes their first dbt models to production, the code might be green. Is the SQL actually good? Anti-patterns like SELECT \*, unnecessary UNIONs, and phantom joins don't break pipelines immediately, but they compound into expensive, hard-to-maintain technical debt. Altimate Code automates the entire SQL peer review process in a single prompt. The agent reads every mart model, applies quality checks in parallel, grades the SQL, applies fixes, and re-validates, producing a full audit report with before-and-after scores. Models that scored D and F grades (due to SELECT \* scanning billions of rows, UNION forcing expensive deduplication sorts, and phantom self-joins multiplying row counts) came out at A and B after automated fixes. The agent also flags patterns that aren't errors but warrant a conversation, making it a better teaching tool than a silent linter. What you'll learn - → How to run automated SQL quality checks across an entire dbt mart layer - → Which anti-patterns are most expensive in compute and maintainability - → How to generate before-and-after quality grades for code review - → How to use AI-generated audit reports to coach junior engineers Key points - Detects SELECT \*, UNION without ALL, phantom joins, and subquery wrappers - Grades every model before and after fixes (D/F → A/B) - Runs dbt build to validate all fixes compile and execute correctly - Generates a full recurring-pattern report across the mart layer - Works in one prompt, with no manual file-by-file review Try it yourself — 10M free tokens, no credit card required. [Get started free →](https://app.myaltimate.com/register) [Watch on YouTube ↗](https://www.youtube.com/watch?v=Vee87oFkqD8) More videos Before You Rename That dbt Column, Check the Blast Radius Before You Rename That dbt Column, Check the Blast Radius Watch → https://altimate.ai/demos-walkthroughs/dbt-column-rename-blast-radius Migrate SQL Server to Snowflake with dbt and Altimate Code Migrate SQL Server to Snowflake with dbt and Altimate Code Watch → https://altimate.ai/demos-walkthroughs/sql-server-snowflake-dbt-migration How an AI Agent Adds Airflow DAGs That Actually Match Your Project How an AI Agent Adds Airflow DAGs That Actually Match Your Project Watch → https://altimate.ai/demos-walkthroughs/airflow-dag-pattern-matching --- # Fix a Broken Airflow Pipeline with AI URL: https://altimate.ai/demos-walkthroughs/fix-broken-airflow-pipeline One AI session traced, diagnosed, and fixed a failing e-commerce ETL pipeline with 5 broken dbt models. [← All videos](https://altimate.ai/demos-walkthroughs) Airflow · dbt · Debugging ## This Airflow Pipeline Was Broken. Here's How an AI Agent Helped Me Fix It. One AI session traced, diagnosed, and fixed a failing e-commerce ETL pipeline with 5 broken dbt models. [This Airflow Pipeline Was Broken. Here's How an AI Agent Helped Me Fix It.](https://www.youtube.com/embed/clIs3wzOXjE) Overview A pipeline in red in Airflow is one of the most stressful sights in data engineering. The hard part isn't seeing the failure. It's tracing it through layers of staging models, intermediate transformations, and mart dependencies to find the actual root cause. This demo walks through a broken daily e-commerce ETL pipeline in Airflow. The pipeline runs dbt through staging, intermediate, and mart layers, and supports everything from reporting to analysis. Instead of manually chasing failures through Airflow logs and individual dbt model files, Altimate Code traces the full DAG structure, reads every model in the dependency graph, and surfaces all five root causes in one pass. The fixes cover Snowflake-to-PostgreSQL dialect translation (replacing:: cast syntax with standard CAST), a missing column reference caused by an upstream rename, a Cartesian join from a missing pre-aggregation step, a duplicate model reference, and a missing mart that was referenced in the README but never built. After the fixes, the pipeline runs to completion and comes back clean. What you'll learn - → How to trace Airflow pipeline failures through dbt lineage without manually reading every model - → How to identify dialect incompatibilities between Snowflake and other databases - → How to fix upstream column renames that silently break downstream models - → How to go from a red pipeline to a successful build in one session Key points - Agent reads the full DAG and surfaces 5 root causes before writing a single fix - Fixes Snowflake dialect:: cast → standard CAST for PostgreSQL compatibility - Rewrites a Cartesian fan-out by adding the missing pre-aggregation CTE - Creates the missing mart\_daily\_sales model from scratch based on the README spec - Reruns the full Airflow pipeline to a successful build Try it yourself — 10M free tokens, no credit card required. [Get started free →](https://app.myaltimate.com/register) [Watch on YouTube ↗](https://www.youtube.com/watch?v=clIs3wzOXjE) More videos Before You Rename That dbt Column, Check the Blast Radius Before You Rename That dbt Column, Check the Blast Radius Watch → https://altimate.ai/demos-walkthroughs/dbt-column-rename-blast-radius Migrate SQL Server to Snowflake with dbt and Altimate Code Migrate SQL Server to Snowflake with dbt and Altimate Code Watch → https://altimate.ai/demos-walkthroughs/sql-server-snowflake-dbt-migration How an AI Agent Adds Airflow DAGs That Actually Match Your Project How an AI Agent Adds Airflow DAGs That Actually Match Your Project Watch → https://altimate.ai/demos-walkthroughs/airflow-dag-pattern-matching --- # AI Agent dbt PR Review: Column Lineage URL: https://altimate.ai/demos-walkthroughs/dbt-pr-review-column-lineage The code looked reasonable. Column-level lineage revealed 4 breaking downstream changes before merge. [← All videos](https://altimate.ai/demos-walkthroughs) dbt · Pull Requests · Column Lineage ## I Used an AI Agent to Review This dbt PR: Here's What It Found The code looked reasonable. Column-level lineage revealed 4 breaking downstream changes before merge. [I Used an AI Agent to Review This dbt PR: Here's What It Found](https://www.youtube.com/embed/SpUWKq7x_T0) Overview A normal PR diff shows you what changed in the code. It doesn't show you what changed in the data flow: which columns were renamed, which downstream dashboards will break, or whether it's actually safe to merge. This demo shows a PR review on mart\_patient\_360, a core healthcare analytics model being refactored to fix a Cartesian explosion, mask SSN for HIPAA compliance, and add financial metrics. The PR touches lineage, sensitive fields, and grain, making it exactly the kind of change where a code review alone isn't enough. Altimate Code runs a column-level lineage diff between the main branch and the PR branch. It identifies 17 added columns, 3 renames, and 2 removals, along with the downstream blast radius for each change. The most important finding: the primary key (patient\_id) was completely missing from the old model and is now added. The most dangerous: renaming full\_name to patient\_name will silently break every BI dashboard that references that column. The agent surfaces both, along with the grain change from multi-row to one-row-per-patient, before anyone approves the merge. What you'll learn - → How to run a column-level lineage diff between branches of a dbt project - → Why code review alone is insufficient for critical analytics models - → How to assess blast radius on downstream dashboards before merging - → How to catch breaking changes that look intentional but aren't safe Key points - Compares column lineage between main and PR branch of mart\_patient\_360 - Identifies 17 added columns, 3 renames, 2 removals with downstream impact notes - Detects that the primary key (patient\_id) was missing from the original model - Flags SSN masking as a HIPAA fix that may break BI tools using the raw column - Surfaces grain change: multi-row → one-row-per-patient, with downstream implications Try it yourself — 10M free tokens, no credit card required. [Get started free →](https://app.myaltimate.com/register) [Watch on YouTube ↗](https://www.youtube.com/watch?v=SpUWKq7x_T0) More videos Before You Rename That dbt Column, Check the Blast Radius Before You Rename That dbt Column, Check the Blast Radius Watch → https://altimate.ai/demos-walkthroughs/dbt-column-rename-blast-radius Migrate SQL Server to Snowflake with dbt and Altimate Code Migrate SQL Server to Snowflake with dbt and Altimate Code Watch → https://altimate.ai/demos-walkthroughs/sql-server-snowflake-dbt-migration How an AI Agent Adds Airflow DAGs That Actually Match Your Project How an AI Agent Adds Airflow DAGs That Actually Match Your Project Watch → https://altimate.ai/demos-walkthroughs/airflow-dag-pattern-matching --- # Get Productive Fast on a New Data Project URL: https://altimate.ai/demos-walkthroughs/new-data-project-zero-context A product manager needs data in 20 minutes from a dbt project you never opened. Altimate Code reads the project, maps lineage, and writes the query. [← All videos](https://altimate.ai/demos-walkthroughs) Productivity · Data Analysis · Onboarding ## New Data Project, Zero Context: Can AI Get Me Productive Fast? A PM needs data in 20 minutes. You have no context on the project. Can an AI agent get you there? [New Data Project, Zero Context: Can AI Get Me Productive Fast?](https://www.youtube.com/embed/3RV6OKfuflU) Overview Every data engineer knows this feeling: you're 20 minutes from a meeting and a product manager needs data you've never seen before from a project you've never touched. The traditional answer involves frantic Slack messages, half-guessed SQL, and a lot of hedging in the meeting. This video explores a different approach: using Altimate Code to build context on an unfamiliar data project in real time. With no prior knowledge of the schema, models, or business logic, the agent reads the project structure, maps the relevant tables, and delivers the answer before the meeting starts. The key insight is that getting productive on a new project isn't about guessing. It's about having a tool that can read the codebase, understand what's upstream and downstream of the data you need, and tell you exactly what to query. Altimate Code does the context-building so you can focus on the answer. What you'll learn - → How to use an AI agent to get context on an unfamiliar dbt project quickly - → How to find the right table and model for a specific business question under time pressure - → What it looks like to go from zero familiarity to a working query in minutes Key points - Agent reads project structure and maps data lineage with no prior context - Identifies the correct models and tables for a specific business question - Generates the query based on actual schema, not assumptions - Gets from 'I don't know this project' to 'I have the answer' in under 20 minutes Try it yourself — 10M free tokens, no credit card required. [Get started free →](https://app.myaltimate.com/register) [Watch on YouTube ↗](https://www.youtube.com/watch?v=3RV6OKfuflU) More videos Before You Rename That dbt Column, Check the Blast Radius Before You Rename That dbt Column, Check the Blast Radius Watch → https://altimate.ai/demos-walkthroughs/dbt-column-rename-blast-radius Migrate SQL Server to Snowflake with dbt and Altimate Code Migrate SQL Server to Snowflake with dbt and Altimate Code Watch → https://altimate.ai/demos-walkthroughs/sql-server-snowflake-dbt-migration How an AI Agent Adds Airflow DAGs That Actually Match Your Project How an AI Agent Adds Airflow DAGs That Actually Match Your Project Watch → https://altimate.ai/demos-walkthroughs/airflow-dag-pattern-matching --- # Get Productive on a New Data Team with AI URL: https://altimate.ai/demos-walkthroughs/airflow-dag-context-aware-coding Starting from a new codebase with zero familiarity, one prompt added a fifth Airflow DAG that matched the team's patterns perfectly. [← All videos](https://altimate.ai/demos-walkthroughs) Airflow · Context-Aware Coding · Onboarding ## Getting Productive on a New Data Team with an AI Agent Starting from a new codebase with zero familiarity, one prompt added a fifth Airflow DAG that matched the team's patterns perfectly. [Getting Productive on a New Data Team with an AI Agent](https://www.youtube.com/embed/C0hCM0D3cMw) Overview Joining a new data team and making a meaningful contribution on day one is genuinely hard. Every project has its own connection patterns, pull patterns, DAG structures, and coding conventions. Copy-pasting from Stack Overflow doesn't cut it. You end up with code that technically runs but doesn't fit. This demo shows Altimate Code adding a new DAG to an open-source Airflow and DuckDB project that already has four established DAGs. The task: create a fifth DAG that loads a CSV into DuckDB, runs a regional aggregation, and uses the project's existing pool pattern to serialize DuckDB file access. The agent wasn't given the pool pattern spec. It read the existing DAGs and inferred it. The result: a new DAG that runs cleanly in Airflow on the first try, uses the correct serialization approach, and looks like it was written by someone who'd been on the project for months. What you'll learn - → How context-aware coding differs from generic LLM code generation - → How to add to a codebase so new code matches existing patterns exactly - → How an AI agent infers implicit conventions from reading existing code - → How to specify complex requirements in a single natural-language prompt Key points - Agent reads 4 existing DAGs to learn connection, pool pattern, and code style - Creates duck\_db\_csv\_ingest.py from a single natural-language prompt - Uses the DuckDB pool pattern correctly to serialize concurrent file access - New DAG runs successfully in Airflow on first try, no manual edits needed - Code style matches the existing project as if written by a team member Try it yourself — 10M free tokens, no credit card required. [Get started free →](https://app.myaltimate.com/register) [Watch on YouTube ↗](https://www.youtube.com/watch?v=C0hCM0D3cMw) More videos Before You Rename That dbt Column, Check the Blast Radius Before You Rename That dbt Column, Check the Blast Radius Watch → https://altimate.ai/demos-walkthroughs/dbt-column-rename-blast-radius Migrate SQL Server to Snowflake with dbt and Altimate Code Migrate SQL Server to Snowflake with dbt and Altimate Code Watch → https://altimate.ai/demos-walkthroughs/sql-server-snowflake-dbt-migration How an AI Agent Adds Airflow DAGs That Actually Match Your Project How an AI Agent Adds Airflow DAGs That Actually Match Your Project Watch → https://altimate.ai/demos-walkthroughs/airflow-dag-pattern-matching --- # Masking PII with an AI Agent URL: https://altimate.ai/demos-walkthroughs/masking-pii-ai-agent-dbt Most teams don't know where their PII lives. An AI agent found it, masked it, and validated everything, all for $3. [← All videos](https://altimate.ai/demos-walkthroughs) PII · GDPR · dbt ## Masking PII with an AI Agent Most teams don't know where their PII lives. An AI agent found it, masked it, and validated everything, all for $3. [Masking PII with an AI Agent](https://www.youtube.com/embed/KtWwjVIhlGI) Overview Personal identifiable information hiding in your data stack is one of the most common compliance risks in data engineering, and one of the hardest to find manually. Most teams don't have a complete picture of which columns contain names, emails, phone numbers, or other PII across their dbt models and raw source tables. This demo shows Altimate Code scanning a full Airbnb dbt project running on Snowflake to find PII, then building a GDPR-compliant masking implementation without any manual file editing. Before writing a single line of code, the agent asks about compliance requirements: whether names should be hashed or tokenized, whether review text should be scrubbed for embedded PII, and what to do with columns that are high-risk but ambiguous. The implementation creates a reusable mask\_pii\_text macro using regex to strip emails, phone numbers, and URLs from free text. Names are masked with SHA-256 hashing, deterministic, so joins and surrogate keys remain consistent. After changes, the agent runs a full dbt build with refresh and pulls a sample output to visually confirm the masking worked. Total cost: $3. What you'll learn - → How to discover PII across a dbt project and Snowflake schema automatically - → How to implement GDPR-compliant data masking using dbt macros - → How to hash identifiers while keeping surrogate keys consistent for joins - → How to validate masking with a full dbt build and visual data inspection Key points - Scans all dbt models and raw Snowflake tables for PII column patterns - Asks GDPR/CCPA compliance questions before acting: hashing vs. tokenizing vs. dropping - Creates a reusable mask\_pii\_text macro with regex for emails, phones, and URLs - Applies SHA-256 hashing to name columns to maintain join consistency - Runs dbt build with full refresh and validates output visually, $3 total Try it yourself — 10M free tokens, no credit card required. [Get started free →](https://app.myaltimate.com/register) [Watch on YouTube ↗](https://www.youtube.com/watch?v=KtWwjVIhlGI) More videos Before You Rename That dbt Column, Check the Blast Radius Before You Rename That dbt Column, Check the Blast Radius Watch → https://altimate.ai/demos-walkthroughs/dbt-column-rename-blast-radius Migrate SQL Server to Snowflake with dbt and Altimate Code Migrate SQL Server to Snowflake with dbt and Altimate Code Watch → https://altimate.ai/demos-walkthroughs/sql-server-snowflake-dbt-migration How an AI Agent Adds Airflow DAGs That Actually Match Your Project How an AI Agent Adds Airflow DAGs That Actually Match Your Project Watch → https://altimate.ai/demos-walkthroughs/airflow-dag-pattern-matching --- # Building dbt Docs with an AI Agent URL: https://altimate.ai/demos-walkthroughs/generate-dbt-documentation-ai The project had 50% model coverage and 20% column coverage. The AI rewrote all of it in 17 minutes for $1. [← All videos](https://altimate.ai/demos-walkthroughs) dbt · Documentation · Productivity ## Building dbt Docs with an AI Agent The project had 50% model coverage and 20% column coverage. The AI rewrote all of it in 17 minutes for $1. [Building dbt Docs with an AI Agent](https://www.youtube.com/embed/PY63_Eu3Si4) Overview dbt documentation is universally important and universally neglected. It's one of those things that everyone agrees should be done well, but few teams have the time or discipline to maintain properly. An overall score of 50% model coverage and 20% column coverage is more common than most data teams want to admit. This demo audits an Airbnb dbt project's documentation, finds the gaps, and auto-writes descriptions for every undocumented model and column. The agent doesn't just fill in placeholder text. It reads the underlying SQL transformations to understand what each model actually does, then writes descriptions that explain the business logic, the upstream sources, the transformations applied, and the downstream consumers. The generated descriptions are more complete than what most engineers write by hand. For a column like listing\_count\_classification, the agent reads the CASE statement in the SQL and documents the accepted values with their exact thresholds (inactive=0, single=1, small=2-5, medium=6-15, large=16+). For models, it includes where the data comes from, what joins are applied, and what feeds downstream, the kind of lineage context that takes real time to write manually. Total cost: $1. What you'll learn - → How to audit documentation coverage across an entire dbt project - → Why AI-generated descriptions are often more complete than manually-written ones - → How to document column accepted values from CASE statement logic in the SQL - → How to validate documentation with dbt docs generate and the catalog Key points - Audits project: finds 50% model coverage, 20% column coverage across all layers - Reads SQL transformations to write business-logic descriptions, not generic placeholders - Documents accepted values by extracting thresholds from CASE statements - Includes upstream/downstream lineage context in every model description - Validates with dbt docs generate, all 12 models and 3 sources in catalog, zero errors Try it yourself — 10M free tokens, no credit card required. [Get started free →](https://app.myaltimate.com/register) [Watch on YouTube ↗](https://www.youtube.com/watch?v=PY63_Eu3Si4) More videos Before You Rename That dbt Column, Check the Blast Radius Before You Rename That dbt Column, Check the Blast Radius Watch → https://altimate.ai/demos-walkthroughs/dbt-column-rename-blast-radius Migrate SQL Server to Snowflake with dbt and Altimate Code Migrate SQL Server to Snowflake with dbt and Altimate Code Watch → https://altimate.ai/demos-walkthroughs/sql-server-snowflake-dbt-migration How an AI Agent Adds Airflow DAGs That Actually Match Your Project How an AI Agent Adds Airflow DAGs That Actually Match Your Project Watch → https://altimate.ai/demos-walkthroughs/airflow-dag-pattern-matching --- # Building dbt Tests with an AI Agent URL: https://altimate.ai/demos-walkthroughs/generate-dbt-tests-ai-agent The project went from 24 tests to 53 in one session. The agent wrote 30 new tests, ran them, and confirmed all passing, for $1.51 total. [← All videos](https://altimate.ai/demos-walkthroughs) dbt · Testing · Data Quality ## Building dbt Tests with an AI Agent The project went from 24 tests to 53 in one session. The agent wrote 30 new tests, ran them, and confirmed all passing, for $1.51 total. [Building dbt Tests with an AI Agent](https://www.youtube.com/embed/O49qIMh2QPQ) Overview Writing dbt tests is one of those tasks that data engineers know they should do more of, and consistently don't do enough of. The result is models that look correct until they hit production. Then they start producing wrong answers that could have been caught by a not\_null or accepted\_values test that took 10 seconds to write. This demo audits a dbt project's existing 24 tests, identifies the gaps (zero coverage in intermediate and mart layers), plans the additions, and implements 30 new tests, including generic tests, accepted\_values with extracted thresholds, singular tests converted to proper dbt format, and a dbt unit test for price segment logic. The agent doesn't just blindly generate tests. When it discovers that 15 hosts in the source data have null names (a real data quality issue in the Airbnb dataset), it downgrades the test from error to warn severity, because a test that breaks the build for a known source data quirk is worse than no test at all. After all additions, the full dbt build runs: 53 tests, 2 expected warnings, zero errors. What you'll learn - → How to audit test coverage gaps across staging, intermediate, and mart layers - → How to write accepted\_values tests by reading CASE logic in existing models - → How to handle real-world data quality quirks without breaking the build - → How dbt unit tests work and when to use them for transformation logic Key points - Audits 24 existing tests, finds zero coverage in intermediate and mart layers - Adds not\_null, unique, accepted\_values, and unit tests per layer - Extracts accepted\_values thresholds directly from CASE statements in the SQL - Discovers real data quality issue (null host names) and correctly sets warn severity - Final build: 53 tests pass, 2 expected warnings, zero errors, $1.51 total Try it yourself — 10M free tokens, no credit card required. [Get started free →](https://app.myaltimate.com/register) [Watch on YouTube ↗](https://www.youtube.com/watch?v=O49qIMh2QPQ) More videos Before You Rename That dbt Column, Check the Blast Radius Before You Rename That dbt Column, Check the Blast Radius Watch → https://altimate.ai/demos-walkthroughs/dbt-column-rename-blast-radius Migrate SQL Server to Snowflake with dbt and Altimate Code Migrate SQL Server to Snowflake with dbt and Altimate Code Watch → https://altimate.ai/demos-walkthroughs/sql-server-snowflake-dbt-migration How an AI Agent Adds Airflow DAGs That Actually Match Your Project How an AI Agent Adds Airflow DAGs That Actually Match Your Project Watch → https://altimate.ai/demos-walkthroughs/airflow-dag-pattern-matching --- # SQL Server to Snowflake Migration with dbt URL: https://altimate.ai/demos-walkthroughs/sql-server-snowflake-migration-dbt Altimate Code migrated SQL Server stored procedures to dbt on Snowflake, validated the output with data diff, and generated a dashboard to prove it worked. [← All videos](https://altimate.ai/demos-walkthroughs) Migration · SQL Server · Snowflake ## Altimate Code Launch: SQL Server to Snowflake Migration with dbt — Live Demo Altimate Code migrated SQL Server stored procedures to dbt on Snowflake, validated the output with data diff, and generated a dashboard to prove it worked. [Altimate Code Launch: SQL Server to Snowflake Migration with dbt — Live Demo](https://www.youtube.com/embed/g-ACWwz9TGg) Overview Migrating stored procedures from SQL Server to a modern dbt and Snowflake stack is one of the most common (and most painful) projects in data engineering. It's not just a translation problem. It's a validation problem. How do you prove the migrated models produce the same results as the original procedures? This live demo from the Altimate Code launch event walks through the full migration lifecycle. Starting from SQL Server stored procedures, Altimate Code reads the business logic, generates dbt models across staging, silver, gold, and analytics layers, and follows a custom migration best-practices guide to ensure the output matches team standards. The raw layer is already in Snowflake via dbt seed. The focus is entirely on rebuilding the transformation layer. Validation happens at three levels: compilation, row-count comparison between SQL Server and Snowflake, and a column-level data diff using Altimate Code's built-in diff skill. When schema mismatches are found, they're fixed in place. When the migration is done, the agent generates an interactive HTML dashboard showing migration status, validation results, model lineage, and KPIs, the kind of artifact that makes stakeholders actually trust a migration instead of just taking the engineer's word for it. What you'll learn - → How to structure a SQL Server to Snowflake migration using dbt and a medallion architecture - → How to use data diff to validate that migrated models produce the same results - → How to provide migration best-practice guides to shape the agent's output - → How to generate a migration validation dashboard for stakeholder sign-off Key points - Reads SQL Server stored procedures and generates equivalent dbt models from scratch - Follows a custom migration best-practices markdown guide for model structure and style - Builds staging → silver → gold → analytics medallion layers in Snowflake - Validates with row-count comparison and column-level data diff between databases - Generates an interactive HTML dashboard: status, KPIs, lineage, and validation results Try it yourself — 10M free tokens, no credit card required. [Get started free →](https://app.myaltimate.com/register) [Watch on YouTube ↗](https://www.youtube.com/watch?v=g-ACWwz9TGg) More videos Before You Rename That dbt Column, Check the Blast Radius Before You Rename That dbt Column, Check the Blast Radius Watch → https://altimate.ai/demos-walkthroughs/dbt-column-rename-blast-radius Migrate SQL Server to Snowflake with dbt and Altimate Code Migrate SQL Server to Snowflake with dbt and Altimate Code Watch → https://altimate.ai/demos-walkthroughs/sql-server-snowflake-dbt-migration How an AI Agent Adds Airflow DAGs That Actually Match Your Project How an AI Agent Adds Airflow DAGs That Actually Match Your Project Watch → https://altimate.ai/demos-walkthroughs/airflow-dag-pattern-matching --- # Databricks Semantic Layer and Genie Ontology URL: https://altimate.ai/blog/databricks-semantic-layer-genie-ontology The Databricks semantic layer pairs modeled Unity Catalog semantics with context Genie infers from usage, ranked by authority and gated by permissions. [Back to blog](https://altimate.ai/blog) Databricks · Databricks Genie · ontology ## How the Databricks semantic layer and Genie Ontology work Altimate AI Engineering Sep 18, 2026 **ENTERPRISE CONTEXT · TECHNICAL DEEP DIVE** > The Databricks semantic layer has two halves: what your team models in Unity Catalog, and what Genie infers from how people use the estate. Genie Ontology is how Databricks joins them at runtime. ## What is the Databricks semantic layer? The Databricks semantic layer is the set of business meanings that sits between your tables and whoever queries them, whether that is a person or an agent. It names the metrics you report on, the dimensions you slice them by, the entities they belong to, and the joins that are valid between them. Unity Catalog holds the part your team defines deliberately, and Genie Ontology adds a second part that Databricks infers from how the estate is used. Say someone asks for net revenue last quarter. The semantic layer part is the definition Finance published, which is gross amount minus refunds, taken from the orders table and joined to customers on customer\_id. The ontology part is what Genie has noticed around that definition, such as which dashboard people actually open for revenue, that revenue questions should go to the curated Finance agent, and that a lead only counts once a demo is booked. Answering the question needs both, because the definition produces the number and the surrounding context decides whose definition to use and who may see the result. So, the two halves answer different questions. Modeled semantics say what your company has agreed a term means. Inferred context says what people do with that term in practice, which is often not the same answer. Genie Ontology keeps both and resolves between them at the moment an agent asks. ## What came before the Databricks semantic layer? In 1970, Edgar Codd proposed [the relational model](https://en.wikipedia.org/wiki/Relational_model) because users should not have to know how data is physically stored to work with it. The abstraction was simple and durable: represent facts as relations, expose a small set of operations, and let the database worry about the machinery underneath. That solved an enormous problem, but not the next one, which is what the data means to the business. A data row can tell you that the account Acme generated $1.2 million. It cannot tell you whether the company considers that account “enterprise,” whether Finance and Sales use the same definition of revenue, whether the churn policy applies to it, or whether a particular dashboard should be trusted over another. So the industry kept adding representations of the facts. A five-step timeline of how data representation evolved, each step labelled with the question it answered. The relational model in the 1970s handled rows, columns and tuples, and asked what facts exist. RDF and OWL across the 1990s and 2000s added triples, classes and relationships, and asked how facts relate. Semantic layers and knowledge graphs in the 2010s added metrics, dimensions and business logic, and asked what the business means. Retrieval and RAG in the 2020s added embeddings, chunks and search, and asked what is relevant right now. Runtime context for agents, the step labelled Now, covers plans, tools, permissions and change, and asks whether this agent should trust this now. RDF reduced knowledge to subject-predicate-object triples. OWL added formal classes, properties, and logical semantics. Knowledge graphs made connected entities practical at a large scale. Semantic layers narrowed the problem to business concepts such as metrics, dimensions, entities, and valid joins. More recently, retrieval systems stopped trying to model everything in advance and instead assembled relevant context when a question arrived. RDF, OWL, knowledge graphs, semantic layers and retrieval systems are not interchangeable, and they share a recurring ambition: put enough structure around raw facts that software can use them without carrying the entire organization inside the application. Capable AI agents change who consumes that structure, which is what turns a reporting semantic layer into a semantic layer for AI. ## Why does an AI agent need more than a metric definition? A dashboard needs a metric definition. An AI agent needs the metric, the right source, the valid join path, the meaning of company-specific terms, the user’s permissions, the relevant historical work, and some basis for deciding [what to trust when those sources disagree](https://altimate.ai/blog/where-ai-agents-belong-in-data-engineering-the-correctness-layer). Then it may need to act on that result. A semantic layer for AI has to carry all of that, not just the metric. That is why the market is converging on several different representations of roughly the same organizational problem. Five approaches to modelling business meaning, arranged on an axis running from mostly authored ahead of time on the left to more assembled at runtime on the right. Semantic layers from dbt, LookML and Snowflake carry explicit metrics, dimensions, and entities with joins. RDF and graph systems from W3C carry explicit entities, relationships and classes. Palantir's operational ontology adds objects, links, and actions with writeback. Retrieval and RAG systems retrieve documents, chunks and similarity context. Genie Ontology from Databricks sits furthest right, combining governed semantics, inferred snippets and authority with ACLs, and retrieving and ranking context when an agent asks. *Snowflake Semantic Views explicitly model logical tables, metrics, dimensions and relationships. dbt MetricFlow builds a semantic graph from models, entities and metrics. Palantir’s Ontology adds object types, link types and action types, including writeback. Genie’s current design is more hybrid: explicit Unity Catalog semantics plus inferred context assembled from platform activity.* This backdrop makes Databricks’ choice of the word *ontology* more interesting. The company is not merely renaming a semantic layer. It is trying to build a broader context system from the activity the data platform already records. ## How does Genie Ontology combine modeled and inferred context? Databricks describes Genie Ontology as a unified context layer with two inputs, which makes it the company’s answer to the Databricks semantic layer question. The first is **Unity Catalog semantics**: context people define and govern deliberately. Metric Views, Domains and Pages belong here. Pages can define a business concept, attach sources and related assets, and become the preferred context when Genie encounters that concept. The second is **inferred context**: snippets Genie automatically extracts and maintains from existing assets and usage, including Metric Views, dashboards, SQL queries, and Genie Agents. Databricks’ docs give [examples](https://docs.databricks.com/aws/en/genie/genie-ontology) such as an “active user” definition, a routing rule that revenue questions should use a curated finance agent, or a business rule that a lead only counts after a demo is booked. **Databricks Metric View** ```yaml version: 1.1 source: prod.finance.orders joins: - name: customer source: prod.crm.customers 'on': source.customer_id = customer.customer_id fields: - name: customer_segment expr: customer.segment measures: - name: net_revenue expr: SUM(source.gross_amount - source.refunds) comment: "Canonical Finance definition of net revenue" ``` *This example uses the documented Databricks Metric Views YAML shape. Databricks supports sources, joins, fields, measures, filters and additional metadata in Metric View definitions.* The modeled layer tells Genie, “this is what we mean.” The inferred layer tells it, “this is how people actually use the estate.” Those two signals do not always agree. Databricks assigns inferred snippets an authority score based on where they came from, usage and freshness. Its launch post describes a PageRank-like ranking process that can also consider the authority of the author and connections to certified or widely used assets. At query time, Genie ranks relevant snippets, resolves conflicts, filters them through Unity Catalog permissions and uses the selected context to answer. A left-to-right flow showing how Genie answers a question. A user or agent submits a business question. Two context sources feed ontology retrieval: human-modeled context covering Metric Views, Pages, Domains and certification, and inferred context covering SQL queries, dashboards, Genie Agents, and usage with freshness. Ontology retrieval assembles candidate snippets and filters them by relevance, authority score and a permission gate. The selected context then routes or plans, choosing the agent, data asset or tool that should answer; executes SQL, Python or a tool call to compute rather than guess; and returns an answer with citations, leaving the evidence visible to the user. *The exact internal algorithm is not public. This figure reflects the product behavior that Databricks documents: retrieve candidate context, rank snippets by authority, enforce permissions, route or execute work, then return citations.* In practice, Genie behaves less like a static knowledge graph and more like a context-selection system. ## Can you trace why an agent chose one definition over another? Data engineers already trace a number backward when it looks wrong: dashboard → semantic model → transformation → source table → upstream system. Each edge explains where the value came from. An agent introduces another chain above that [data lineage](https://altimate.ai/blog/column-level-lineage-use-cases). Why did it choose the Finance Page instead of the Sales dashboard? Why did it route the question to one Genie Agent? Which snippets were considered and rejected? Which join did it decide was valid? Which SQL did it generate? Which result changed the next step? Reasoning lineage is the name for that second chain. What you audit is not the model’s chain-of-thought but the trace around it: the evidence retrieved, the authority decisions, the policy checks, the tool calls, the generated code, the query results and the final citations. A Databricks semantic layer records what a term means, and none of that records why one meaning won. Two lineage chains stacked for comparison. Classic data lineage runs source to value: CRM orders, a dbt model transform, a Metric View as the semantic step, a dashboard as the consumer, and a final value of $12.4M. Agent or reasoning lineage runs above it with six steps: retrieve, finding 8 candidates across Pages, SQL and agents; rank, where Finance wins on authority and freshness; policy, where the ACL passes and the user may see the source; plan, which selects the metric view and chooses a tool and query; execute, which runs the SQL and returns $12.4M; and return, which hands back the answer with a citation and visible evidence. *“Reasoning lineage” is our term, not a Databricks one. It names what the system retrieved and did, which you can check, rather than the model’s account of its own thinking, which you cannot.* **What an auditable agent trace would need to preserve** ```json { "question": "Net revenue for enterprise EMEA last quarter", "context_candidates": ["finance.page", "sales.dashboard", "query_8472"], "selected_context": "finance.page", "selection_evidence": { "authority": "human-modeled / published", "permissions": "allowed" }, "tool": "sql_warehouse", "generated_sql_hash": "sha256:…", "result_artifact": "query_run_01928", "citations": ["finance.page", "metric_view.net_revenue"] } ``` *This is a conceptual trace format rather than a Databricks API schema.* ## Which Databricks tools read the same semantic layer? This is where Genie Ontology becomes more than an analytics feature. Genie One reads the ontology when a business user asks a question. Genie Code searches the same ontology, so a metric definition or join hint curated once can affect developer work. Databricks’ managed Genie One MCP server lets external [MCP clients](https://altimate.ai/blog/best-mcp-servers-for-data-engineers) send questions to Genie and receive answers grounded in the ontology, with Unity Catalog permissions enforced. The architecture starts to look like a shared context layer that agents read at runtime, which is a larger claim than a Databricks semantic layer usually makes. Reading is already broadly supported. Writing authoritative context is much more constrained. Genie automatically maintains inferred context as the underlying assets and usage change. Humans can publish and edit Pages. Genie Code can draft Pages from source material, but those drafts still go through a human publication flow. And the Genie Agent MCP server is explicitly read-only for data queries. So an agent cannot, today, simply decide that it learned a new business truth, write that fact into the shared ontology, and have every future agent inherit it as authoritative context. That is probably a good constraint. Genie Ontology, holding modeled and inferred context, sits at the centre of a loop with four surrounding blocks. The enterprise estate of tables, queries, dashboards, Metric Views, agents and usage generates new evidence as it changes. Consumers including Genie One, Genie Code, Genie Agents and MCP clients read that context at runtime. Their actions and outputs, meaning queries, documents and workflows, change external systems and can create more estate activity. The governed write path of Pages, certification and edits runs through human review and publication, which is the only route that explicitly establishes authority. *The important feedback loop exists indirectly: agents and humans create artifacts and usage, which can become new inferred evidence. Directly promoting an agent’s new claim to shared organizational truth is a much harder governance problem.* If Databricks eventually lets agents write to it, the product will need stronger provenance, approval, temporal validity and rollback. Otherwise the ontology risks learning from the consequences of its own previous outputs and amplifying mistakes. ## What breaks a semantic layer as the business changes? Ontologies look clean in architecture diagrams because the diagram freezes the organization in time. Companies do not stay still. They rename metrics. They acquire businesses. Teams create local definitions because the global one is inconvenient. A source system changes grain. A dashboard remains popular after the logic underneath it becomes stale. Permissions change. New agents generate more queries and documents, which themselves become evidence for future agents. The graph is not merely large. It is moving. Calling ontology maintenance “NP-hard” is tempting but imprecise. But the enterprise maintenance problem is difficult for an additional reason: much of the ambiguity is not computational. Whether “customer” means a billing account, legal entity, workspace or product tenant is an organizational decision. Whether the new definition should replace the old one retroactively is a policy decision. Whether Sales may override Finance for pipeline reporting is a governance decision. No graph algorithm settles those questions by itself. Six pressures arranged around a centre labelled ontology state, what the company currently means. Concept drift: definitions change over time, and active customer is not static. Source and schema change: tables and grains move, and old relationships stop being valid. Org change: mergers, teams and domains turn two vocabularies into one. Conflicting authority: usage popularity can conflict with certified definitions, where Finance does not equal Sales Operations. Permission drift: who may know what changes, so context is identity-dependent. Agent feedback: AI creates new evidence, and outputs can influence future context. *The maintenance challenge is both algorithmic and organizational. Automatic curation reduces setup cost, but it also makes change visibility, conflict handling and rollback more important.* Maintenance is where Genie Ontology will succeed or fail. Databricks has an unusually good position from which to infer context because it sees a large amount of enterprise data exhaust: catalog metadata, queries, dashboards, lineage, permissions, Metric Views, Genie Agents and developer activity. The product can start without a pristine manually designed ontology, then use explicit semantics as stronger anchors where correctness matters. The same activity already drives other automation on the platform, such as [tuning a job cluster from its own run history](https://altimate.ai/blog/right-sizing-databricks-job-clusters-automatically). The danger is that the same automation that makes the Databricks semantic layer cheap to start can make it difficult to reason about later. ## Is Genie Ontology ready for production use? Not yet. As of September 2026, Genie Ontology is in Public Preview. Pages are Beta. There is no public export or snapshot. Provenance is visible at answer level through citations, but there is no dedicated ontology lineage view. There is not yet a full reviewable changelog or diff of newly learned context. | Status | Capability | What it means today | | --- | --- | --- | | Available now | Hybrid context | Human-modeled Unity Catalog semantics plus automatically maintained inferred snippets. | | Available now | Authority and ACLs | Relevant snippets are ranked, and Unity Catalog permissions gate what the user can consume. | | Available now | Cross-surface reuse | Genie One and Genie Code search the same ontology, and Genie can be exposed through MCP. | | Still immature | Ontology-level auditability | No public snapshot or export, dedicated relationship lineage, or full change diff today. | | Still immature | Direct agent writeback | No public generic API for an agent to promote a newly learned claim into authoritative shared context. | | Open question | Long-term maintenance | Can automatic curation stay correct as definitions, schemas, teams, permissions and agents change? | Pages also carry a preview-era security caveat worth noticing. Databricks warns that Page data does not support customer-managed key encryption and is stored as plain text that may be replicated globally, so Pages should not contain PII, regulated data or other sensitive content. Databricks has also published internal benchmark [results](https://www.databricks.com/blog/introducing-genie-one-genie-ontology-and-genie-agents): 84.5% first-attempt correctness on a 28-question enterprise data-analysis suite, versus 52.4% for its strongest anonymized [general-purpose coding-agent](https://altimate.ai/blog/why-ai-coding-agents-fail-at-data-engineering) comparator, with roughly half the latency. It is an encouraging result, but still a small vendor-run benchmark. The same caveat applies to [ADE-Bench](https://altimate.ai/benchmarks/ade-bench), which we run ourselves. The more consequential test is what happens after eighteen months of production use, when the ontology has absorbed thousands of new assets, stale definitions, reorganizations, permission changes and agent-generated artifacts. Success will depend less on whether Genie can build this semantic layer than on whether it can keep it true as the organization changes. > Databricks’ bet is that the platform now contains enough behavioral evidence to keep an ontology current without requiring an enterprise to model everything manually. **Can Databricks make an enterprise ontology that is not only useful when it is created, but inspectable, governable and correct, in near real time, while the enterprise keeps changing?** ## Frequently asked questions ### Is the Databricks semantic layer the same as Unity Catalog Metric Views? Databricks Metric Views are one part of it. They are where a team writes a metric down deliberately, with its source, joins, fields and measures, in the YAML shape shown above. Genie Ontology also covers Domains and Pages, and adds the inferred half that Genie maintains from dashboards, SQL queries, Genie Agents and usage. Metric Views are the modeled anchor rather than the whole layer. ### Can an AI agent write new definitions into the Databricks semantic layer? Not into the authoritative part. Genie maintains inferred context automatically as assets and usage change, and Genie Code can draft Pages from source material, but those drafts still go through a human publication flow. The Genie Agent MCP server is read-only for data queries. An agent cannot decide it has learned a new business truth and have every future agent inherit it as authoritative context. ### How does it differ from dbt MetricFlow or Snowflake Semantic Views? Those two model meaning ahead of time. dbt MetricFlow builds a semantic graph from models, entities and metrics, and Snowflake Semantic Views model logical tables, metrics, dimensions and relationships. Genie Ontology is more hybrid, pairing explicit Unity Catalog semantics with context assembled from platform activity, then ranking the candidates by authority at the moment a question arrives. --- Altimate Code puts a [deterministic tool layer](https://altimate.ai/blog/deterministic-tooling-vs-llm-only-data-agents) under the model and returns a signed verdict with the evidence it used on every dbt pull request, so the retrieval, the checks and the generated SQL stay inspectable. Read what [Altimate Code](https://altimate.ai/products/altimate-code) does, or how it runs against [Databricks](https://altimate.ai/use-cases/altimate-for-databricks). Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [Databricks](https://altimate.ai/blog?tag=databricks) [Databricks Genie](https://altimate.ai/blog?tag=databricks-genie) [ontology](https://altimate.ai/blog?tag=ontology) [semantic layer](https://altimate.ai/blog?tag=semantic-layer) [AI Tools](https://altimate.ai/blog?tag=ai-tools) [Metric Views](https://altimate.ai/blog?tag=metric-views) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Right-sizing Databricks Job Clusters URL: https://altimate.ai/blog/right-sizing-databricks-job-clusters-automatically Auto Tune learns from each Databricks job’s runs, right-sizes its cluster inside the bounds you set, and reverts if a run regresses. Median job 33% cheaper. [Back to blog](https://altimate.ai/blog) Databricks · databricks cost · databricks compute ## Right-sizing Databricks job clusters, automatically Altimate AI Engineering Sep 16, 2026 > Altimate AI’s Auto Tune learns from each Databricks job’s metrics, converges its cluster toward its right-sized configuration within the bounds you set, and reverts and alerts whenever a run breaches them. ## Why do Databricks job clusters end up oversized? Most Databricks job clusters are sized once and rarely revisited. That first spec is typically a generous one, with more memory, more cores and a bigger node than the run turns out to need, chosen with a safety margin so the job simply works. Nothing forces a second look, so they keep running on that day-one config indefinitely. We see it in the telemetry across hundreds of our customers’ job clusters: most run far below the capacity they pay for, CPU and memory sitting largely idle through each run. Closing that gap by hand does not scale across hundreds of scheduled jobs, and the attempt carries real risk, because a wrong change can break a production pipeline at three in the morning. What we wanted to design for at Altimate is a system that keeps every job cluster right-sized, continuously learning from each job’s runs and watching each change for a breach of the configured bounds, so resources stay matched to what it actually needs. ## How does Auto Tune keep Databricks job clusters right-sized? Auto Tune works as a closed loop. It observes each job’s runs for resource utilization, run times and failures, then generates a right-sizing recommendation. It applies that recommendation, measures the runs that follow for any breach of the configured bounds, and decides the next change, tightening the fit each cycle. Four components carry it, and the sections below take them in this order: - **Metrics Store** collects the telemetry, run history and billing from every run. - **Recommendation engine** turns that evidence into a proposed change. - **Apply and watch** makes the change and measures what happened. - **Savings and audit history** records what was kept and why. Auto Tune targets classic job compute, the clusters whose node types and sizes you choose. Serverless is out of scope for now. Architecture diagram of a closed tuning loop across two columns, the customer’s Databricks environment and Altimate AI. Five numbered steps run clockwise: Unity Catalog system tables ingest into the Metrics Store, the Metrics Store feeds the recommendation engine, the engine feeds apply and watch, the change is applied to the job configuration, and the next runs on the job clusters feed the loop. A dotted arrow back to the Metrics Store measures each run against the job’s own history, a dashed arrow reverts the job configuration if a run breaches its bounds, and a savings and audit history panel records the outcome. *The system learns from each job’s own runs, makes one change, watches the runs that follow, and lets the result decide the next one.* ## What does the Metrics Store collect? *The Metrics Store holds the telemetry, billing and history behind every decision.* Everything starts with what the workspace already records. The Metrics Store ingests the customer’s own [Databricks system tables](https://docs.databricks.com/aws/en/admin/system-tables/jobs-cost): per-minute node telemetry for CPU, memory, disk and network, run history and outcomes, and billing. Alongside them it keeps configuration history, so the spec a job ran on last month is still available, and the audit trail of every change Auto Tune has made. ## How does the recommendation engine decide a new size? *Databricks cluster sizing runs from memory, CPU, disk and network, one change at a time.* The engine handles Databricks cluster sizing from the dimensions that decide whether a job fits its hardware: how much memory and CPU it needs, whether it needs a local disk attached and at what size and throughput, and how much network bandwidth. It turns those into concrete recommendations: the worker node family and size, the driver, and how the cluster scales, either a fixed worker count or an autoscale range. One change per cycle, priced on both cost components at the customer’s negotiated DBU rate and the cloud provider’s prices. A Lakeflow job whose tasks run on several clusters is tuned per cluster. Auto Tune groups the job’s runs by the cluster each set of tasks uses and recommends for each of them on its own, so a job with a small orchestration cluster and a large one for the heavy task gets the right change on each. The settings view below shows this, with a separate recommendation for each of the job’s task clusters. Before a workspace enables anything, a job owner can see what is proposed for every task cluster of a job, side by side with what runs today, and choose which tasks are in scope. An illustration of the Auto Tune settings page for a multi-task job, with task names and figures anonymized rather than a live product screen. Task scope shows an estimated saving of 35 to 45 percent for the selected tasks. Five tasks are grouped inside two dashed cluster outlines: tasks 1, 2 and 3 share cluster A and each show a current m5d.4xlarge worker proposed down to m5d.2xlarge with the driver unchanged and an estimated saving of 40 to 50 percent; tasks 4 and 5 share cluster B and each show an r5.2xlarge worker proposed down to r5.xlarge with an estimated saving of 25 to 35 percent. A bottom bar offers reset to default, cancel, and enable Auto Tune. *What a job owner sees before enabling: tasks are grouped by the cluster they share, and every task on a cluster gets the same recommendation. This is an illustration of the settings page rather than a live product screen, with task names and figures anonymised.* ## How does Auto Tune apply a change and watch the result? *Auto Tune applies a change, watches what happens, keeps the learning, and reverts if a run regresses beyond the configured bounds.* Flow diagram of the tuning loop. Recommend a change leads to apply, then to watching the next runs, measured against the job’s own history. The outcome branches three ways: a regression reverts to the previous configuration and captures the learning before recommending again, a change that held with more headroom makes the updated config the new baseline and returns to recommending a change, and a change that held on a well-sized job holds with no change this cycle while the job stays monitored. *Inside the loop: the next runs decide whether a change is kept, converged on, or reverted.* A proposal is applied to the job configuration, and the next runs are measured against the job’s own history. If the change holds, the updated configuration becomes the new baseline. If the job has converged, no change is made this cycle and it stays monitored. If a run regresses beyond the configured bounds, the previous configuration is restored automatically and the reason is recorded. The loop does not stop at the first win. A held configuration is monitored for a few days and confirmed as the new stable baseline only once it has proved itself across several runs, and each cycle then builds on the last. Automatic reversion is what makes unattended tuning acceptable to a platform team: a change that does not work out is reverted and recorded, not left to compound. Auto Tune applies a change only when no run is in flight, so an in-progress run is never cancelled, and it then monitors the job every ten minutes after the change lands. ## What does the savings and audit history record? *The history records the measured saving on every job, what changed, and the controls an owner keeps.* The four charts below are representative, each showing one job’s daily cost before and after enablement, measured against each job’s own history rather than a benchmark. Bar chart headed Job A, daily cluster cost before and after tuning, marked 47% lower. A dashed line labelled tuned splits the grey bars before from the green, visibly shorter bars after. Bar chart headed Job B, daily cluster cost before and after tuning, marked 22% lower. A dashed line labelled tuned splits the grey bars before from the green bars after, which drift gently downward. Bar chart headed Job C, daily cluster cost before and after tuning, marked 27% lower. A dashed line labelled tuned follows one tall spike, after which the green bars settle below the earlier grey ones. Bar chart headed Job D, daily cluster cost before and after tuning, marked 59% lower. A dashed line labelled tuned splits irregular grey bars with occasional tall spikes from green bars that step down and stay low. | Job | Daily cluster cost after tuning | | --- | --- | | Job A | 47% lower | | Job B | 22% lower | | Job C | 27% lower | | Job D | 59% lower | Each chart is one representative job’s daily cluster cost, before and after tuning. Heights are the real daily cost, and the dashed line marks when tuning was enabled. - **Hundreds of jobs** tuned every day. - The median actively tuned job runs **33% cheaper**, with no regressions. A tenant also configures the bounds it is willing to accept, and those bounds decide how aggressively Auto Tune trades runtime for cost. Tighter bounds keep it conservative and it will only propose changes that leave runtime almost untouched. Looser ones let it go further for a larger saving. ## Do Lakeflow jobs cover every Databricks workload? ### What about jobs Lakeflow doesn’t schedule? Plenty of Databricks work is not scheduled by Lakeflow at all. It is triggered by an external orchestrator such as Airflow or Azure Data Factory. Those jobs go through the same recommendation engine. What changes is the last step. Their cluster configuration lives in the customer’s own pipelines and repositories rather than in Lakeflow, so Auto Tune cannot apply a change live. Instead it produces the recommendation, can open a pull request that updates the configuration at its source, and raises an alert when a job breaches its expected cost or runtime. The engine and evidence are the same as for Lakeflow jobs. Only the last step changes, from a live edit to a pull request the team reviews and merges, after which Auto Tune resumes watching. ### Databricks job cluster vs all-purpose cluster, which should run scheduled work? Scheduled work belongs on a job cluster. The Databricks job cluster vs all-purpose cluster choice is not about capability, because both run the same code, but about how each is billed and how long it stays up. Interactive clusters get recommendations of their own: sizing the [auto-termination window](https://altimate.ai/blog/beyond-the-databricks-cost-calculator-6-levers-that-actually-cut-your-bill) to how the cluster is actually used, lowering the minimum worker count on an over-provisioned autoscaling cluster, and turning autoscaling on where a fixed-size cluster sits idle. More often, though, the finding is that scheduled work is living somewhere it should not. Recurring jobs, continuously running jobs and repeated one-off submissions end up on a shared all-purpose cluster because that is where someone first tried them, and they never moved. Auto Tune identifies them and recommends moving each onto its own job cluster, after which the tuning loop applies to it like any other. The all-purpose vs jobs compute split is what makes that move worth the trouble. It pays because Databricks prices all-purpose vs jobs compute very differently: the DBU component on all-purpose compute runs [roughly 3.5 to 3.7 times the jobs-compute rate](https://www.databricks.com/product/pricing/product-pricing/instance-types). ## What changes when Databricks job clusters stay right-sized? The jobs nobody goes back to no longer have to drift. Right-sizing stops being a periodic project and becomes a property of the platform, running quietly in the background. The real question isn’t whether your scheduled jobs have headroom. It’s how much you’d let a system take back for you. Point [Auto Tune](https://altimate.ai/use-cases/altimate-for-databricks) at a workspace and it will answer that in your own numbers. ## Frequently asked questions ### What happens if a tuning change makes a job slower? The configuration is restored automatically and the reason is recorded. Auto Tune measures the runs that follow each change against the job's own history, and a run that regresses beyond the bounds the workspace configured triggers the revert without anyone intervening. Those bounds are the control: tighter ones keep Auto Tune conservative and only allow changes that leave runtime almost untouched, looser ones let it trade more runtime for a larger saving. ### How much cheaper does a right-sized Databricks job cluster run? The median actively tuned job runs 33% cheaper, with no regressions, across hundreds of jobs tuned every day. Individual jobs vary more than the median suggests: the four representative jobs above came down by 22%, 27%, 47% and 59% on their daily cluster cost. Each figure is measured against that job's own cost before tuning rather than against a benchmark. ### Does Auto Tune right-size serverless compute? Not at present. Auto Tune targets classic job compute, which is the compute whose node types and sizes you choose, because right-sizing means changing those choices. Serverless bills on usage rather than on a cluster you have sized, so there is no node family, node size or worker count for the recommendation engine to change. --- ## What are the two parts of a Databricks job’s cost? *Two meters on every job, and right-sizing brings down both.* A Databricks job’s hourly bill has two meters. Databricks charges a [DBU rate](https://altimate.ai/blog/understanding-databricks-compute-costs) for the compute tier, and the cloud provider charges for the infrastructure underneath it, the worker and driver machines and any storage attached to them. Because Auto Tune resizes the node itself, its family and size, its disk and its network, a single change reduces both meters at once: the DBU spend and the infrastructure spend. Infrastructure is usually the larger of the two, so across the jobs tuned so far it accounts for most of the realized savings, while the DBU charge comes down alongside it. --- *Savings are reported as relative shares of each job’s own prior cost. Customer names withheld.* Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [Databricks](https://altimate.ai/blog?tag=databricks) [databricks cost](https://altimate.ai/blog?tag=databricks-cost) [databricks compute](https://altimate.ai/blog?tag=databricks-compute) [Cost Optimization](https://altimate.ai/blog?tag=cost-optimization) [finops](https://altimate.ai/blog?tag=finops) [Data Engineering](https://altimate.ai/blog?tag=data-engineering) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Cut Databricks Cost: Auto Termination and More URL: https://altimate.ai/blog/beyond-the-databricks-cost-calculator-6-levers-that-actually-cut-your-bill Six Databricks configuration levers that cut compute spend up to 35%: compute type, Photon, auto-termination, driver sizing, job compute, spot. [Back to blog](https://altimate.ai/blog) Databricks · Data Engineering · Cost Optimization ## Beyond the Databricks Cost Calculator: 6 Cost Levers, Including Auto Termination Sudhanshu Prajapati Sep 11, 2026 Beyond the Databricks Cost Calculator: 6 Cost Levers, Including Auto Termination The official [Databricks price calculator](https://www.databricks.com/product/pricing/product-pricing/instance-types) gives you the DBU rate times the hours you plan to run. It also shows the instance type for each cloud provider. It prices the VMs, or compute types, for the kind of workloads you want to run. It also lists other basic details such as the compute options available. But the cost calculator does not say which configuration parameters affect your overall costs. It also does not tell you how to set those parameters. For instance, the calculator doesn't tell you if you're paying for compute hours you didn't need. It doesn't show whether your compute type charges more for identical work. It also doesn't flag a cluster still sized for a large but outdated workload that shrank a quarter ago. Six configuration levers decide how much of the calculated number you pay. They are compute type, Photon, Databricks auto termination, driver sizing, job compute, and spot or reserved instances. To learn how Databricks compute costs work, read our [*Understanding Databricks Compute Cost*](https://altimate.ai/blog/understanding-databricks-compute-costs) post. That post explains the total cost of running Databricks, including cloud infrastructure and workload pricing per DBU. Let’s start with a very common yet underused cost lever. The few people who do use this lever often don’t use it in an optimal way. ## Cost Lever #1: Match Compute Type to Workload Before Anything Else Before selecting your Databricks compute type, analyze your workload requirements. Is it a bursty BI dashboard query that runs once a week? Or is it a job that runs daily for 1 hour or more? We discussed this topic in our blog post [*Databricks Serverless vs Classic Compute: Which One Is Actually Cheaper?*](https://altimate.ai/blog/databricks-serverless-vs-classic-compute-which-one-is-actually-cheaper) That article will help you decide which workload type you should select, serverless or classic compute. To summarize: **Databricks Serverless Compute** is the right call for bursty, unpredictable, or interactive work. That work includes ad hoc SQL, BI dashboards, and short jobs. In a short job, a user runs a query, looks at the result, and then the job is complete. You stop paying for startup or idle time. Scheduled ETL tasks are prime candidates for serverless job clusters. Many teams leave these running on more expensive interactive compute by mistake, missing out on immediate cost savings. We will dive deeper into scheduled job optimization later. **Databricks Classic Compute** is the right call for steady workloads that you understand well enough to size deliberately. Classic wins on those workloads because spot pricing moves the bill there. Serverless doesn't expose spot pricing, because you never see the underlying VM. All-purpose Classic Compute makes sense when you don’t know the kind of workloads you will be running but it also costs you more. Whenever possible, it is better to go with a specific type of compute to save on cost. Refer to the table below (taken from the [*Understanding Databricks Compute Cost*](https://altimate.ai/blog/understanding-databricks-compute-costs) blog) for more information on how to plan the workloads. TABLE: HOW TO SELECT THE CORRECT DATABRICKS COMPUTE TYPE BASED ON WORKLOAD: | Workload | Compute | Why | | --- | --- | --- | | Interactive BI dashboards | Serverless SQL warehouse | Bursty access, cold starts hurt users, idle dominates cost | | Ad hoc SQL and exploration | Serverless SQL warehouse | Unpredictable usage, so idle is the main waste | | Short frequent jobs | Serverless jobs | Startup is most of the runtime | | Scheduled ETL and batch | Job compute, classic | High duty cycle, fault tolerant, every classic lever applies | | Streaming | Job compute or serverless pipelines | Check triggered vs continuous before anything else | | ML training | Classic with GPU | Serverless GPUs exist but are narrower; check Databricks' [current offering.](https://docs.databricks.com/aws/en/machine-learning/ai-runtime/) | ## Cost Lever #2: Only Turn on Photon for the Workloads It Actually Accelerates Photon replaces Spark's JVM execution layer with a native C++ engine. Photon supports SQL, DataFrame API calls, ETL pipelines, and stateless streaming. It does not natively execute custom code such as Python UDFs, RDDs, or Dataset APIs. Photon hands those parts of the query back to standard Spark when a workload hits these operations. | Aspect | Photon (native C++ engine) | Standard Spark (JVM) | | --- | --- | --- | | Handles | SQL, DataFrame API calls, ETL pipelines, stateless streaming | Custom code: Python UDFs, RDDs, Dataset APIs | | Execution model | Replaces the JVM execution layer for supported operations | Falls back to execute the parts Photon can't | | Trigger for fallback | N/A | Whenever a query hits an unsupported operation, Photon hands that part back to Spark | ### Where Photon Will Run (is this thing on?) Most folks enable Photon by default during the set-up phase on classic all-purpose compute, job compute, and classic pipelines. Photon is always on for serverless and every SQL warehouse. So before deciding whether to turn on Photon, check whether it already is on by default. The exception is compute provisioned through the Clusters or Jobs API, which needs `runtime_engine` set to `PHOTON` explicitly. Photon does nothing for GPU training or vector search. On GPU instance types you can't even enable it. ### Understanding Databricks Photon Pricing Photon instances cost more DBUs than standard runtime instances, so make sure the extra speed justifies the price. Workloads dominated by Python UDFs, RDDs, Dataset APIs, or other unsupported operations may spend much of their time in standard Spark even with Photon enabled. Queries under two seconds see little benefit, because query planning takes most of that time. Check instead of guessing. In the Spark UI's SQL/DataFrame, Photon operators are orange, whereas Spark operators are blue. The query profile reports what percentage of task time ran in Photon. One caveat before disabling it anywhere: predictive I/O and dynamic file pruning in MERGE, UPDATE, and DELETE all require Photon. Run the analysis both ways to improve speed and cut costs, then act on the results: - **Disable Photon:** Identify UDF-heavy jobs where Photon is enabled but gives zero acceleration. Disable Photon on those jobs. - **Enable Photon:** Identify API-provisioned clusters running SQL workloads that never had Photon enabled. Turn Photon on for those clusters. - **No Action:** Leave MERGE-heavy pipelines alone. ## Cost Lever #3: Set Databricks Auto Termination Just Below the Idle Gap Databricks auto termination helps control idle compute costs. A timeout set too high wastes money. A timeout set too low can cause cold start latency or workload failures. Idle cost matters most on all-purpose compute and SQL warehouses, where you pay for the resource while it sits running and idle. Serverless notebooks and jobs bill on usage, so idle time is not a factor. Teams across time zones can also benefit from tighter timeouts during off hours, when long gaps between handoffs can leave clusters running unused. You want to set auto-termination just below the idle gap you want to catch. Check how busy the cluster is, not only the timeout value. Auto termination only triggers after the configured period of inactivity. For example, take a cluster with a 60 min timeout that runs for 10 minutes at the top of every hour. That cluster never reaches the 60 min gap, so it stays running. You effectively pay for six times the compute you need, even though the setting worked as configured. A healthy cluster spends well over 50% of its running time doing actual work. A cluster busy for under 25% of its running time is a serious savings opportunity. Timeline diagram titled Why the 60-Minute Timeout Never Fires: a 10-minute job runs at the top of each hour across a 180-minute window, leaving three 50-minute idle gaps where the cluster still bills, because the next job always starts before the 60-minute idle clock completes. That does not mean shorter inactivity timeouts are always better. If idle gaps are under 10 minutes (the minimum Databricks allows for classic compute auto termination), lowering the timeout will not help much. Classic clusters also take three to eight minutes to start, as we typically observe. The start time varies by cloud, node type, and whether a pool is in play. So even slightly longer gaps may not justify cycling classic clusters. Weigh the cold start and latency costs before you tighten the timeout on unpredictable workloads, particularly on all-purpose compute and serverless. Aim for the timeout that minimizes idle spend without disrupting workloads. Use these rules of thumb to pick a Databricks auto termination setting for each workload shape: TABLE: RULE OF THUMB GUIDANCE ON PICKING DATABRICKS AUTO-TERMINATE PERIODS | Workload shape | What you'd see in the data | Prescription | | --- | --- | --- | | Ad-hoc notebook exploration, 1-2 analysts | Gaps of 5-20 min between commands, cluster up all day | **15-20 min.** Below 15 mins you pay the 3-8 min restart more often than you save. | | Shared dev cluster, one time zone | Short gaps 9-6, then dead overnight | **20-30 min + a scheduled terminate at end of day.** The timeout handles the daytime tail; only a schedule handles the night. | | Team split across time zones | Cluster up 18-20 hrs, never idle for 30 min straight | **Don't lower the timeout: it will never fire.** Split into per-region clusters, or move the workload to a serverless SQL warehouse. | | Scheduled batch running on all-purpose compute (our 10-min-per-hour example) | Regular short bursts, gaps shorter than the timeout | **Wrong lever.** Move to job compute, which terminates on completion, so zero idle time. Timeout tuning caps saving at ~50 min/hr; job compute captures the whole thing. | | Frequent short jobs (every 15-60 min) on job compute | Paying 3-8 min startup on every run | **Instance pool** with Min Idle = 0. Set idle-instance auto-termination just above the gap. Databricks' own example is 20 min for hourly jobs. | | BI dashboards on a **serverless** SQL warehouse | Queries clustered in business hours | **Auto Stop 10 min** (the default). Serverless restarts in seconds, so there's no reason to run higher. UI floor is 5 min; the API goes to 1 min. This is a good approach for a warehouse serving a single scheduled refresh. | | BI on a **pro/classic** SQL warehouse | Same pattern, 3-8 min cold start | Default is 45 min. **Cut to 20-30.** If you're tempted to go lower, that's the signal to move to serverless, not to keep tightening. | | Structured Streaming using DStreams | Continuous | **Set to 0 (no auto-termination).** Databricks doesn't report DStream activity, so an auto-terminating cluster can be killed mid-run. | | Long single-command ML training | One 4-hour cell, no other activity | **10-15 min is safe.** The clock measures time since the last command *ran*, and a running command counts as activity. | ### Measuring Idle Share **Idle share = running minutes with no work / total running minutes.** Plotted across those workload shapes, the percentages spread wider than most teams expect. Bar chart of typical 30-day idle share by workload shape on a Databricks cluster: ad-hoc notebooks 88% and shared dev in one time zone 71% call for tuning the timeout; batch jobs on all-purpose compute 83% and shared dev across three time zones 57% call for changing the compute type; DStream streaming 41%, short jobs on job compute with a pool 22%, and ML training 12% are left alone. Idle is driver CPU below 5% in system.compute.node\_timeline. *The idle share across the workload shaped in the table above is based on a 30-day* benchmark. *The percentage alone doesn't tell you the fix. The top two bars waste a similar share of their running time. But one is a timeout problem and the other is a wrong-compute-type problem. SQL warehouses don't appear because of the* `system.compute.*` *doesn't cover them.* Streaming shows 41% idle *even though the job is live the whole time*. That happens because the gaps between micro-batches look like idle minutes to a CPU-based measure. And the query below filters on the driver node. So a cluster with a thin driver and busy executors will report a higher idle share than it deserves. Always check the shape of the workload before you act on the number. To find Databricks idle clusters yourself, run this query over the last 30 days. It returns one row per cluster name. ```sql SELECT c.cluster_name, COUNT(*) AS running_minutes, ROUND(100 * SUM(CASE WHEN t.cpu_user_percent + t.cpu_system_percent < 5 THEN 1 ELSE 0 END) / COUNT(*), 1) AS pct_idle FROM system.compute.node_timeline t JOIN (SELECT DISTINCT cluster_id, cluster_name FROM system.compute.clusters) c USING (cluster_id) WHERE t.start_time >= current_date() - INTERVAL 30 DAYS AND t.driver GROUP BY 1 ORDER BY running_minutes DESC; ``` Note: `system.compute.*` excludes SQL warehouses and serverless. Those go through query history and billing tables instead. A histogram of recoverable Databricks idle spend by workload shape *Recoverable idle spend comes from the same* benchmark. *We priced a five-node all-purpose cluster at $3.20/hr all-in (DBU + EC2). Ad-hoc notebooks have the highest idle share but only the third-largest bill. That is because a cluster that is up ten hours a day can only waste ten hours a day. Streaming and long training show $0. They still log idle minutes, but there is nothing to reclaim.* This is why idle percentage is a triage signal rather than a priority list. Sort your own query output by `running_minutes * pct_idle` instead of by `pct_idle`. The clusters worth fixing first then move to the top of the list. Finally, constrain `autotermination_minutes` in a compute policy (range with a max, or fixed + hidden). Then audit for the value `0`, which means "never terminate" and leaves idle clusters running until someone stops them. ## Cost Lever #4: Size the Driver Independently of the Workers Sizing the driver is different from tuning the autoscaling range for your worker pool. This lever is about the *driver* specifically. Almost nobody touches the driver setting because Databricks sets the driver node type to match the worker node type by default. The [Databricks compute configuration docs](https://docs.databricks.com/aws/en/compute/configure) say: "*The default value of the driver node type is the same as the worker node type*". But that default can be wrong in both directions: - Databricks recommends a larger driver when you pull large result sets back using `.collect()`, or when multiple notebooks share the same cluster. - On the other hand, standard ETL jobs that only transform and write data rarely need large drivers. For those jobs, the default setting forces you to pay for an oversized driver that sits mostly idle. Always size the driver to match what your specific workload requires rather than trusting the default. ## Cost Lever #5: Move Scheduled Databricks Workloads Off All-purpose Clusters Running production schedules on all-purpose clusters often looks like a scheduling or efficiency issue. But **the rate difference** is what drives the high bill. Databricks job compute is typically two to three times cheaper than all-purpose compute. That ratio is a rule-of-thumb. Your actual savings depend on how long the original cluster sat idle. Savings also depend on whether DBUs or raw cloud infrastructure made up the bulk of your costs. Migrating scheduled workloads off all-purpose clusters onto job compute changes how resources behave. An all-purpose cluster stays warm and ready between runs, whereas a job cluster spins up fresh and shuts down every time. For daily batch runs, accepting that brief cold start is well worth the cost savings. For high-frequency jobs running every few minutes, you have to weigh the startup overhead against the lower compute rates. ### Why is My Databricks Production Job Still on an All-purpose Cluster? Development teams often start their work on an all-purpose cluster because development is interactive by nature. They run a cell, look at the output, change something, and run it again. > Nobody *decided* production should run on an all-purpose cluster. Production just *inherited a decision* made in development Once the things get approved and signed off, they get promoted to a schedule. And the pipeline code that lands in the main branch still points at the same cluster ID or policy the team wrote it against. That happens because nobody reviewed the compute config when promoting the branch. Nobody *decided* production should run on an all-purpose cluster. Production just *inherited a decision* made in development, the same way the driver inherited the worker's size in the last lever. Auto termination helps here (see lever 3) but doesn't fix it. It cuts the idle-time waste on that cluster. It doesn't move the job to the compute type that's actually cheaper for the work it's now doing on a schedule. This is hard to catch by scanning dashboards, because a job on the wrong compute type looks completely healthy. Finding it manually means joining `system.billing.usage` against `system.lakeflow.job_task_run_timeline` and asking which production jobs still show up against an all-purpose SKU. Even this join can't attribute the cost accurately. Databricks says precise cost accounting for jobs sharing all-purpose compute isn't possible with full accuracy, because several workloads draw on the same cluster at once. That accuracy limit is itself an argument for moving the job rather than pricing it after the fact. ## Cost Lever #6: Use Spot or Reserved Instances Spot instances can significantly cut Databricks compute costs. But the cloud provider can reclaim Spot instances, so use them for worker nodes, not the driver. If the provider reclaims a Spot worker, Spark can retry its tasks elsewhere. If the provider reclaims the driver, the entire job fails because the driver coordinates the cluster. > Keep one rule: use spot instances only for worker nodes, and keep your driver node on standard on-demand compute. On AWS, use fleet instance types so Databricks can choose from multiple instance types based on availability and price. Do not rely on a single instance type, because that type may not be available when you need it. For predictable workloads, Reserved Instances (RIs) have huge CUDs (Committed Use Discounts) and can provide significant savings without Spot's interruption risk. Use Spot when your workload can tolerate worker replacement. Use RIs when you need consistent capacity and predictable savings. Both Spot and RIs reduce the cloud VM cost, not DBUs. So Spot and RIs are more impactful for job compute, where infrastructure is a larger share of the bill. Spot and RIs are less impactful for all-purpose compute, where DBUs typically are a larger share of the total bill. Know which area of the bill you're attacking before you decide to go for spot instances or RIs. ### When Databricks Spot Compute is the wrong call, Spot is good where interruption is tolerable: retry-friendly batch ETL, checkpointed pipelines, and fault-tolerant model training. Conversely, a low-latency dashboard query or an SLA-bound job is the wrong place to try your first spot rollout. ## Databricks Auto Tune: Put Optimization on Autopilot Now that we have seen all 6 levers, the work comes down to a cost reduction sprint within the team. That sprint gets each lever set to its optimum position. However, getting the levers right once doesn’t guarantee value over time. You need to adjust these levers every now and then, as the demands and requirements change. Constantly keeping the levers in check requires time and effort. The [Altimate Platform](https://altimate.ai/platform) provides precisely such a feature called **Auto Tune**. Auto Tune automatically sets all six levers we discussed, with safe configuration backoff, and goes well beyond Databrick’s Predictive Optimization. ### How Altimate Auto Tune Automatically Pulls Levers Auto Tune analyzes how each SQL warehouse, cluster job, and other workloads are *actually* used. Including CPU, memory, and worker utilization across recent runs, and then right-sizing things such as - **Worker instance type**: moving workers to a cheaper, better-fitting family if the current one is over-provisioned. - **Driver instance type**: right-sized as per need from historic data, since drivers are frequently much larger than the workload requires. - **Worker count / autoscaling bounds**: tightens a fixed count or the min-max range so the cluster never runs more workers than the job needs. Auto Tune only changes **the configuration** and never touches the code, schedule, tasks, libraries, and tags. The diagram below gives you a quick overview of how Auto Tune works: Auto Tune flow diagram: snapshot the full cluster config as an audit record and rollback source, apply the new auto-stop or size with all other fields carried forward, verify the config by reading it back, then monitor p95 latency against the week before. If performance holds, savings accrue; if latency hits 1.5x baseline with a seconds-level increase, Auto Tune backs off and restores the snapshot exactly. For more information on how you can save more with the Altimate Platform, see [Databricks Cost Optimization](https://altimate.ai/use-cases/altimate-for-databricks). The use cases and screenshots below show the levers we discussed in action, providing real cost savings. The Auto Tune settings page shows how Auto Tune proposes the correct family and instance type by observing the workload and requirements. Auto Tune then gives you cost-saving options if you choose that setting. A screenshot of the Altimate UI, a platform that provides cost intelligence and governance for modern data teams. Altimate analyzes actual workload behavior to detect when a job is running on the suboptimal compute family. It provides you with visibility into where the mismatch exists so you can trust a system recommendation and decisions. *A screenshot of the Altimate UI, a platform that provides cost intelligence and governance for modern data teams. Altimate analyzes actual workload behavior to detect when a job is running on the suboptimal compute family. Altimate shows you where the mismatch* exists, so you can trust the system's recommendations *and decisions.* We previously discussed how Databricks production jobs sometimes keep on running on an all-purpose cluster. We often miss those jobs because they work just fine. They just cost way more than they should. This issue becomes immediately visible with Altimate’s “discover” tab: The Discover page in the Altimate.ai enterprise platform runs a version of that same join continuously and surfaces exactly this pattern: a job still running on an interactive cluster long after the workload stopped being interactive, flagged with the switch to make and the same accuracy limit attached. You can read more about in Understanding Databricks Compute Cost blog post. *The Discover page in the Altimate.ai enterprise platform continuously runs a version of that join. The join matches system.billing.usage against system.lakeflow.job\_task\_run\_timeline. Discover surfaces exactly this pattern: a job still running on an interactive cluster long after the workload stopped being interactive. Discover flags the job with the switch to make and attaches the same accuracy limit.* Further information on the Altimate Platform can be found in [Databricks Cost Optimization](https://altimate.ai/use-cases/altimate-for-databricks), or if you use Snowflake, read [Adaptive Compute vs. Altimate AI Auto Tune Optimization](https://altimate.ai/blog/adaptive-compute-vs-auto-tune-a-practical-guide-to-optimizing-snowflake-warehouses) If you want to explore Altimate, you can [book some time with the team here](https://calendly.com/d/cncb-mfh-wpq/demo-request-platform-cost-savings). ## Which Half of the Databricks Bill Each Lever Changes The Databricks price calculator shows none of the six levers. Each lever is a configuration decision rather than a billing rate. Compute type, Photon, and job compute change the DBU half of the bill. Spot workers change only the cloud infrastructure half, and driver size mostly changes that half too. Databricks auto termination changes both halves. ***Note on the savings column below:*** *ranges are directional and apply to the workload not to your whole bill. Treat them the way we treated the auto-terminate example: useful for prioritizing, but not a quote. In our experience, teams that work through all six typically land in the 20-35% range on total Databricks compute spend.* | Lever | Setting to change | What you give up | Which half of the bill is improved (DBU or Infra) | Typical savings (directional, on the affected workload) | | --- | --- | --- | --- | --- | | 1) Match compute type to workload | Serverless or classic, per workload shape | Spot pricing, on anything you move to serverless | DBU | 50-75% of a bursty cluster's cost when utilization was under 25% | | 2) Turn on Photon where it applies | Photon toggle on the cluster or warehouse | Nothing if it's out of scope, but you pay the higher Photon rate for no speedup | DBU | Turning Photon off where it adds no value -> up to roughly half that job's DBU line on classic compute. | | 3) Tighten auto-termination | Auto-terminate minutes | Cold-start waits, and the risk people leave clusters up to avoid them | Both | 20-40% of classic compute spend | | 4) Size the driver for its job | Driver node type, unpinned from the worker type | A bigger driver costs more; too small and.collect() fails the job | Infrastructure, mostly | Single-digit % of that cluster's infra bill | | 5) Move scheduled work to job compute | Cluster type on the job definition | The warm interactive session the notebook was developed against | DBU | 40-60% off that job's total cost. | | 6) Spot workers, on-demand driver | Spot for workers, Fleet types on AWS, driver stays on-demand | Interruption tolerance, so not for SLA-bound work | Infrastructure only | 60-90% off the workers' VM price (DBUs unchanged) | For the two-bill split and the system tables behind all six, see our blog post, [Understanding Databricks Compute Costs](https://altimate.ai/blog/understanding-databricks-compute-costs). ## Frequently Asked Questions ### What Databricks Auto Termination Setting Should I Use? The right Databricks auto termination setting depends on the workload shape. Ad hoc notebook clusters work well at 15 to 20 minutes. Shared dev clusters in one time zone need 20 to 30 minutes plus a scheduled terminate when the workday ends. A pro or classic SQL warehouse defaults to 45 minutes, so cut it to 20 to 30. A serverless SQL warehouse can stay at its 10-minute default. Set Structured Streaming clusters that use DStreams to 0, because Databricks does not report DStream activity. A cluster shared across time zones never idles for 30 minutes straight, so split it per region or move the workload to serverless. ### Why Does My Databricks Cluster Keep Running With Auto Termination On? Databricks auto termination only fires after a full period with no activity. Take a cluster with a 60-minute timeout that runs a 10-minute job at the top of every hour. It never sits idle for 60 minutes, so it never terminates. Move that schedule to job compute, which terminates when each run completes. ### How Do I Find Databricks Idle Clusters? To find Databricks idle clusters, query `system.compute.node_timeline` and count the running minutes where driver CPU stays below 5%, as the SQL in lever 3 does. Sort the result by `running_minutes * pct_idle` so the clusters with the most idle minutes come first. The query reads the driver only, so a thin driver with busy executors shows more idle time than it has. The `system.compute.*` tables do not cover SQL warehouses or serverless compute. Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [Databricks](https://altimate.ai/blog?tag=databricks) [Data Engineering](https://altimate.ai/blog?tag=data-engineering) [Cost Optimization](https://altimate.ai/blog?tag=cost-optimization) [#apache-spark](https://altimate.ai/blog?tag=%23apache-spark) [big data](https://altimate.ai/blog?tag=big-data) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Databricks Serverless vs Classic Compute Cost URL: https://altimate.ai/blog/databricks-serverless-vs-classic-compute-which-one-is-actually-cheaper Serverless costs more per DBU but avoids idle time. Benchmarks show a 30-minute break-even point for picking the cheaper Databricks compute option. [Back to blog](https://altimate.ai/blog) databricks cost · databricks serverless · databricks compute ## Databricks Serverless vs Classic Compute: Which One Is Actually Cheaper? Altimate.ai Team & Saurabh Arora & Sudhanshu Prajapati Sep 4, 2026 Databricks Serverless vs Classic Compute: Which One Is Actually Cheaper? Whenever somebody mentions *serverless*, two things come to mind: it should be cheaper than normal resources and no infrastructure management, right? This is certainly what has been claimed both from Databricks and from several (potentially sponsored) case studies. But it is still a very debatable topic. In this blog post, I will be dissecting **Databricks Serverless**, which was initially introduced in 2017 for Apache Spark. Today, almost every offering in Databricks has a serverless option. This hosting option has gained a lot of interest. So, let’s look at: - What Databricks Serverless is, exactly - Compare the cost of Databricks serverless vs classic compute - Under which circumstances should you pick one over the other But first, let’s talk first about serverless in general. ## What is Serverless Compute? And Why Might You Use It? In simple terms, serverless compute means you run your workloads on third-party infrastructure without managing anything yourself. The managed service does the infrastructure work for you. Think of serverless compute as a flexible, on-demand service. You use it for the exact time you need and pay only for that period. You do not maintain hardware or handle backend upkeep costs. Just like any on-demand service, serverless compute has fair-use rules and boundaries. The provider guarantees fast access and manages capacity behind the scenes. The provider still applies operational limits. Databricks provides multiple types of serverless compute. Below are the key serverless compute types: - **Serverless SQL Warehouses**: On-demand elastic compute for running SQL queries from the SQL editor, dashboards, and BI tools. Warehouses start in seconds and scale automatically with demand, so there's no sizing decision and no idle cluster sitting around. Billed per DBU-second on its own serverless SQL SKU. - **Serverless All Purpose Compute:** This serverless compute removes the need to manage the underlying cluster infrastructure. It scales compute resources automatically based on workload demand. It suits interactive development, notebooks, and ad hoc analysis. Pick it for that work when fast startup and less cluster management matter more than cost. - **Serverless Jobs Compute** Compute for scheduled and triggered jobs (what used to be Workflows). Databricks provisions compute for every run and scales it to match the workload, with smart retries and cloud failover built in. It supports notebook, Python script, dbt, Python wheel, and JAR tasks. With serverless Jobs Compute, Photon and autoscaling are always on. - **Serverless Spark Declarative Pipelines:** The compute behind Lakeflow Spark Declarative Pipelines. You define the transformation logic and Databricks runs the pipeline without you configuring or deploying any infrastructure. It’s a good fit for streaming and incremental ETL where cluster sizing is hard to predict. - **Serverless GPU compute:** Still in beta and aimed at deep learning: training and fine-tuning custom models without managing GPU fleets or drivers. It runs on A10s for smaller work and H100s (8 GPUs per node) for large interactive or distributed training. ## Standard vs. Performance-Optimized vs. Serverless Compute Let’s compare three things: - **Standard Databricks Compute** means any instance type with little effort spent to optimize performance. - **Performance-optimized compute** refers to an instance family and configuration suited to the kind of workload you want to run. An example is a scheduled long-running job that runs every Monday morning. - **Serverless compute**, where you don’t need to think about what instance family or thing you’re using. Here's how the three stack up. | | Standard (classic) | Performance-optimized (classic) | Serverless | | --- | --- | --- | --- | | Best fit for | Anything where nobody has looked at the bill yet | Steady, predictable jobs whose shape you already know | Bursty, unpredictable, or interactive work | | What you configure | A generic instance type and cluster size | Instance family, node type, and autoscale range chosen for the workload | Nothing. Databricks chooses and scales it per run | | Startup time | Minutes per cluster start (zero once running, billed while idle) | Minutes, same as standard | Seconds | | Billed for | DBUs plus the underlying cloud VM, for however long the cluster is up | Same as Standard, but sized to cut waste | DBUs only, at a higher per-DBU rate, for the seconds the job actually runs | | Who tunes it | You, once, when the cluster policy is set | You, per workload, based on profiling the job | Databricks, automatically, on every run | ## Should you use Databricks Serverless Compute? There are two main reasons to choose serverless. First, you skip routine maintenance and do not provision capacity up front. So your team focuses on building pipelines instead of managing machines. Second, resources start in seconds instead of minutes, which removes long wait times and stops you from paying for idle compute. Serverless tends to be the right call when: - You want zero cluster management, with no configs to write and no sizing to tune. You want startup in seconds rather than minutes. - The workload is spiky, interactive, or unpredictable. Examples are ad hoc SQL, BI dashboards, notebooks, or short jobs. - You are paying for idle time on classic clusters today and want to get rid of it. - Iteration speed matters more to you than fine-grained cost control. [Cost-wise](https://altimate.ai/blog/understanding-databricks-compute-costs), Databricks serverless carries a higher per-DBU rate than classic compute. In exchange, you pay only for the seconds a job runs. Classic is cheaper per DBU. But you pay for however long the cluster stays up, including idle time. You also carry cloud VM costs separately. The share of billed time spent doing work decides which option costs less, more than the list price does. In our benchmark of Databricks serverless vs classic compute, serverless didn’t require much cluster-management work and delivered the shortest runtime. However, the trade-off is price. The serverless run costs **$8.90** after a 6% effective-rate discount. The least-expensive on-demand configuration costs **$3.33**. The least-expensive spot configuration costs **$2.32**. The serverless premium is **2.7x** over on-demand and **3.8x** over spot. Cost per TPC-DS run ## Benchmark setup Below is benchmark setup we used in our comparison: - **Workload:** TPC-DS at scale factor 1,000 - approximately a 1 TB decision-support dataset under the [TPC scaling model](https://www.tpc.org/tpc_documents_current_versions/pdf/tpc-ds_v1.4.0.pdf). - **Serverless:** default platform settings. - **Classic compute:** `m6id.xlarge`, `c5d.xlarge`, and `r6id.xlarge` workers, each tested with on-demand and spot capacity. - **Cluster policy:** autoscaling from 4 to 12 workers; `m6id.large` on-demand driver; Databricks Runtime 14.3 LTS with Spark 3.5.0. - **Sampling:** three paired runs per day 06:00, 14:00, and 22:00 for 21 days, giving 63 observations per worker type. Serverless and classic jobs were launched concurrently to reduce time-of-day bias. - **Primary measures:** effective cost per successful run and end-to-end elapsed time. Runs should use the same region, table snapshot, file layout, query set, and cache policy. Retries and spot reclamations belong in the cost of the completed run. The two compute models require different accounting. For classic compute, integrate driver and worker uptime across autoscaling events. Then add both infrastructure and DBU charges: ``` classic cost = Σ(worker-seconds × [VM rate + DBU rate]) + driver + storage + retry overhead ``` For serverless, we used metered DBUs at the effective contracted rate. Databricks selects the worker shape and scaling policy for serverless. So this benchmark is a platform-mode comparison rather than an instance-for-instance test. **Default serverless settings may also differ in Photon use and execution-engine behavior. Check query plans and cache state before you attribute the runtime gap solely to elasticity**. ## Benchmark Results for Serverless and Classic Compute Average runtime by compute option Serverless finishes about **5 minutes sooner than on-demand** and **21 minutes sooner than spot**. But the additional spend is **$5.57** and **$6.58** per run, compared with the cheapest option in each class. Put differently, the cost premium is roughly **$1.11 in additional cost per minute saved** vs on-demand and **$0.31 per minute saved** vs spot. Latency-sensitive pipelines may justify the serverless premium. Serverless gives those pipelines tighter completion windows, faster scale-up, and lower operational load. Classic compute remains the stronger cost choice for scheduled workloads with slack in the service-level objective. Classic saves the most when you tune worker families, Photon, shuffle storage, and autoscaling bounds to the query mix. So weigh each decision against the value of a minute saved, rather than cost or runtime in isolation. [Qubika](https://qubika.com/blog/databricks-cost-series-part-2-serverless-vs-classic/) reports the same job swinging from 30% cheaper to 2x more expensive on serverless, depending on the workload. Qubika concludes that you should benchmark your own jobs rather than trust a fixed ratio either way. The cheaper choice between Databricks serverless and classic compute depends on how long and how often your job runs. Serverless removes startup and idle overhead, which dominates short jobs. A 4-minute job that previously consumed 12 minutes of cluster time now consumes 4. The longer the job runs, the smaller the startup and idle overhead becomes as a share of the total. On longer jobs, classic compute's lower per-DBU rate starts to win on its own. The rough break-even is around 30 minutes of runtime. Below that, serverless is usually cheaper before you even account for the operational savings. Above that, classic wins unless serverless is delivering a 30 to 60% speedup that classic cannot match. That provides you three ways to decide what to go for once you know a job's typical run duration and how steady its schedule is: - **Choose serverless** for jobs that run under 30 minutes. Serverless also fits an unpredictable or interactive schedule, such as ad hoc SQL or BI dashboards. Pick serverless when you don’t want infrastructure management or cluster spin-up. With serverless, you also get no separate bill for cloud resource usage. - **Choose Classic** for jobs that run long and steady. You can size the cluster once and push the cost down further with spot instances or committed-use discounts. Classic also gives you finer control over the cost. Classic also needs more of your own monitoring and debugging. - **Run a hybrid**, which is where most shops end up. Use serverless for ad hoc and interactive work. Use classic job clusters for the long, predictable ETL that runs the same way every night. Decision flow: choosing serverless or classic compute for a workload ## When to Keep Classic Databricks Compute You can optimize classic compute to a certain extent with spot instances, committed-use discounts, and right-sizing the cluster once. Classic compute also remains the better choice outright in a few concrete situations: - You need custom configuration. Examples are a specific instance type, GPUs, init scripts, OS-level libraries, or nonstandard Spark configs. - The workload is long-running and steady. A steady workload lets you use spot instances or reserved and committed-use discounts. Spot instances and those discounts push the cost well below what serverless would charge for the same hours. - You have strict networking requirements, such as a workload that has to run inside your own VPC or VNet. - You are running heavy ML training or another specialized workload that serverless does not fully support yet. The Databricks serverless GPU offering is in beta as of now. ## How to Decide Between Serverless and Classic for Each Job Neither Databricks compute option is cheaper in every case. Choosing between Databricks serverless and classic compute comes down to each job's runtime and how steady its schedule is. Pull up your longest-running and shortest-running jobs and check where they fall against the 30-minute break-even. For the other levers that cut the bill, such as spot workers, Photon, and auto termination, read [Beyond the Databricks Cost Calculator](https://altimate.ai/blog/beyond-the-databricks-cost-calculator-6-levers-that-actually-cut-your-bill). ## Frequently Asked Questions ### Is Databricks Serverless Cheaper Than Classic Compute? The answer depends on job runtime. Serverless removes startup and idle time. That overhead dominates short jobs, so serverless usually wins below about 30 minutes of runtime. Above that, the lower per-DBU rate of classic compute usually wins, unless serverless runs the job 30 to 60% faster. ### How Much Did Serverless Cost in the TPC-DS Benchmark? Serverless cost $8.90 per successful TPC-DS run at scale factor 1,000, after a 6% effective-rate discount. The cheapest on-demand classic configuration cost $3.33, and the cheapest spot configuration cost $2.32. Serverless finished about 5 minutes sooner than on-demand and 21 minutes sooner than spot. ### When Should You Keep Classic Databricks Compute? Keep classic compute when you need a specific instance type, GPUs, init scripts, OS-level libraries, or nonstandard Spark configs. Classic compute also costs less for long, steady workloads that can use spot instances or committed-use discounts. Workloads that must run inside your own VPC or VNet also need classic compute. Heavy ML training stays on classic compute while serverless GPU compute is in beta. Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [databricks cost](https://altimate.ai/blog?tag=databricks-cost) [databricks serverless](https://altimate.ai/blog?tag=databricks-serverless) [databricks compute](https://altimate.ai/blog?tag=databricks-compute) [Databricks](https://altimate.ai/blog?tag=databricks) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Understanding Databricks DBU and Compute Costs URL: https://altimate.ai/blog/understanding-databricks-compute-costs How Databricks compute costs actually work: DBUs vs. cloud infrastructure, serverless vs. classic, and the system tables that show where the money goes. [Back to blog](https://altimate.ai/blog) Databricks · Altimate · finops ## Understanding Databricks DBU and Compute Costs Saurabh Arora Aug 19, 2026 Understanding Databricks DBU and Compute Costs Authors and Databricks itself publish many guides on Databricks cost optimization. They cover techniques, strategies, approaches, and what to do and not do. The guides converge on the same checklist: turn on auto-termination, use job clusters, try spot instances, enable Photon... None of that is wrong. The problem is that it assumes you already know where your money is going, and most teams don't. We still need to make sense of how Databricks compute actually works. What decides your Databricks bill when you get billed for the usage in a month? It sounds easier to just say, "Hey, you’ve used 300 DBU. The DBU rate is $0.40 per DBU. That means 300 DBU times $0.40 = $120, right?" But then you might be wondering about that DBU 300; is that the only cost you need to take into account? What did it cost to keep the compute running that consumed those DBUs? And if someone asked you what the marketing team's dashboards cost last month, or how much of that was compute sitting idle, could you answer? ## How Databricks DBU Cost and Cloud Infrastructure Cost Add Up The Lakehouse model stores data once in [open formats](https://opendataformats.org/) and brings compute engines to that data, instead of copying it into a separate warehouse for analysis. That is one of the core economic principles behind the Databricks model. To understand how this affects your Databricks billing, see how Databricks decouples three core layers: 1. Storage: Holds the data indefinitely in open formats at raw cloud storage rates. 2. Governance: Controls access and security across the entire data estate via Unity Catalog. 3. Compute: Executes work against the stored data on demand. ### The Databricks Compute layer Databricks compute is the execution layer. It is where queries run, notebooks execute, jobs transform data, and machine learning workflows consume governed datasets and turn them into outputs. In the Databricks operating model, storage preserves data and governance defines who can use it, but compute is where value is actually created. #### Databricks Compute types Databricks offers three types of compute: - **Serverless Compute**: Fully managed compute that is available instantly. Databricks handles cluster provisioning, scaling, and maintenance. - **Classic Compute**: Compute clusters provisioned directly in your cloud account, giving you full control over node types and configurations. - **SQL Warehouses**: Specialized compute optimized specifically for SQL queries, BI dashboards, and data warehousing workloads. Separating storage and compute does not remove their costs. The costs become visible, more distributed, and more operational. So the total Databricks cost becomes a **two-bill problem**. The first bill is **platform cost,** where you pay for the Databricks platform in DBUs. A DBU, or *Databricks Unit*, is Databricks’ billing unit for compute consumption. A DBU measures compute power consumed over time. Databricks tracks your bill by compute SKU, usage type, and workload category instead of a single flat "cluster cost". For example, in mid-2026 an all-purpose compute cluster typically costs around $0.55 per DBU-hour for a standard runtime. The second bill comes from **your cloud provider**. That bill covers the infrastructure cost: virtual machines, disks, and associated network costs. Serverless compute is the one major exception. Its Databricks DBU charge already includes the virtual machine cost, because Databricks manages that compute itself. Now go back to the $120 calculation from the intro. That $120 is not the total cost. DBUs are only the Databricks half. The EC2 or VM hours, the disks, the NAT gateway, and the cross-zone traffic arrive on a separate invoice. You don’t see that invoice on the Databricks console. This split is why two common cost conversations go sideways: 1. A team cuts its Databricks DBU cost by 20% and wonders why the total bill barely moved. The infrastructure half didn't change. 2. A team compares the serverless DBU rate against the classic DBU rate and concludes serverless costs several times more. But serverless includes the VM whereas Classic doesn't. Diagram titled "Solving the two-bill problem," contrasting the traditional split of a Databricks invoice for DBUs plus a separate cloud provider invoice for VMs, disks, and networking against the serverless exception, where infrastructure costs are bundled into one consolidated DBU invoice. Your Databricks DBU cost covers only half of what you pay. The other half is the infrastructure you kept running to consume those DBUs. ## Start With Databricks System Tables and Know Their Gaps To understand usages, Databricks system tables are the right place to start, and most teams underuse them. Four tables matter most for compute cost, plus one more if you run SQL warehouses: | Table | What it gives you | | --- | --- | | `system.billing.usage` | DBU quantities, SKU names, and a `usage_metadata` struct with `cluster_id`, `warehouse_id`, `job_id`, and `job_run_id`, plus custom tags and the identity that ran the workload | | `system.billing.list_prices` | List prices per SKU over time, so you can turn DBUs into dollars | | `system.compute.clusters` | Cluster configuration history, so you see what a cluster looked like when it ran | | `system.compute.node_timeline` | Per-minute CPU and memory per node. | Add `system.query.history` if you run SQL warehouses, because it records how long queries spent waiting for compute. To see Databricks DBU cost in dollars, join `system.billing.usage` to `system.billing.list_prices` on the SKU name and the period each price was in effect. This query is adapted from the Databricks [billing sample queries](https://docs.databricks.com/aws/en/admin/usage/system-tables) and returns usage and list cost per SKU for the last 30 days: ```sql SELECT usage.sku_name, usage.usage_unit, SUM(usage.usage_quantity) AS quantity, SUM(usage.usage_quantity * list_prices.pricing.effective_list.default) AS list_cost FROM system.billing.usage JOIN system.billing.list_prices ON list_prices.sku_name = usage.sku_name WHERE usage.usage_end_time >= list_prices.price_start_time AND (list_prices.price_end_time IS NULL OR usage.usage_end_time < list_prices.price_end_time) AND usage.usage_date >= current_date() - INTERVAL 30 DAYS GROUP BY usage.sku_name, usage.usage_unit ORDER BY list_cost DESC; ``` The result uses list prices and does not reflect any negotiated discount. Before writing anything custom, you can import Databricks' [Lakeflow observability dashboard](https://docs.databricks.com/aws/en/admin/system-tables/jobs-cost). It ships as JSON you drop into your workspace, and the page publishes the SQL behind every tile. Note - if you use the Lakeflow observability dashboard, be aware that `usage_metadata.job_id` is only populated for jobs on job compute or serverless. So filtering billing usage for an all-purpose SKU where `job_id` is not null returns nothing at all. The empty result only means the filter never had rows to match. To actually find jobs running on all-purpose clusters, you have to go through `job_task_run_timeline` joined against `compute.clusters`, and Databricks [publishes that query](https://docs.databricks.com/aws/en/admin/system-tables/jobs). *None of these Databricks system tables contains a single dollar of cloud infrastructure cost*. To get a total bill, you join Databricks usage against your cloud provider's cost and usage report based on resource tags. Databricks does propagate cluster tags. Altimate AI Current State view of Databricks costs for Jul 6 to Aug 3, 2026, totaling $49.51K, broken into SQL Warehouse $22.27K, Clusters $20.95K, Lakehouse $3.36K, AI/ML $2.12K, and Platform $801, shown as a daily stacked bar chart of spend by SKU type. *Altimate's Clusters and SQL Warehouses views stitch both halves together and attribute the result down to cluster, warehouse, job, and team. You can read one number per workload instead of reconciling two exports by hand.* ## Where your compute money goes Once you start thinking in terms of two bills, the next step is understanding what usually drives the cost. For most teams, Databricks DBU cost and cloud cost go into a familiar set of buckets: | Where the money goes | Which bill it contributes to | What to look at first | | --- | --- | --- | | Idle time on interactive compute | Both | Cluster uptime against query time | | The wrong compute type for the workload | Databricks (DBU) | All-purpose SKUs running scheduled jobs | | Oversizing | Both | Node utilization in system.compute.node\_timeline | | Loose or poorly bounded autoscaling | Both | The minimum you always pay, and how long max is held | | Long-running interactive resources that nobody turns off | Both | Auto-termination settings | | Streaming patterns that run 24\*7 when the business requirement does not actually need 24\*7 freshness | Both | Trigger mode against the business requirement | | Infrastructure side effects such as networking, egress, or duplicated data movement | Cloud only | Your cloud cost and usage report, by tag | Workloads drift, so revisit the table above on a regular schedule, not only after a bill shock prompts a one-time sprint. Despite all the efforts, a pipeline can be on the wrong SKU, oversized, and idle half the time, all at once. That pipeline will look completely healthy on every alert you have unless you have visibility at that granular level. ### When Serverless Helps, and When It Doesn't As we discussed earlier, serverless compute is good for bursty workloads such as BI. BI workloads often come in bursts. A user refreshes a dashboard, runs a query, inspects results, then goes idle. In non-serverless warehouses, startup time is long enough that teams often leave resources running to avoid waiting. Serverless can help change this trade-off because it can start and scales in seconds and can terminate idle compute sooner than all-purpose/classic compute. What you give up is control. No picking instance families, no spot strategy, no pool tuning. That's fine when your problem is startup delay and idle waste. It hurts when your problem needs precise control over the infrastructure underneath the workload. So skip "is serverless cheaper?" and ask what kind of waste we are trying to eliminate. Serverless is often the best choice at its premium price when the waste comes from slow startups, bursty access, and long idle windows. Classic compute can cost less when the workload is steady, predictable, and suited to deliberate sizing and cloud cost engineering. That is especially true when teams know their workload shape well enough to right-size aggressively. Those teams can also use cloud primitives that serverless does not expose in the same way. This is also why visibility comes before optimization. Without data on burstiness, concurrency, and idle windows, the serverless debate turns ideological instead of operational. ### Idle and the Wrong Compute Type These two problems compound each other. Take a big all-purpose cluster. The work finishes fast because the instance is large. But the cluster hangs around afterward because spinning it back up is slow. You're paying double: a costlier instance, and longer idle time on top of it. All-purpose clusters are also much easier to leave running between interactions. Job compute only runs while there's a job to run. "Job compute is typically 2-3x cheaper than all-purpose," says Databricks’ [cost maturity post](https://www.databricks.com/blog/chaos-control-cost-maturity-journey-databricks). Treat that figure as a starting range, not a rule. What you save depends on how much runtime was idle and which half of the bill dominates for that workload. Teams miss this constantly, and the pattern is always the same: 1. A notebook starts on an all-purpose cluster because development is interactive. 2. The workflow proves useful. 3. It becomes production-critical. 4. Nobody revisits the compute type. 5. The company pays a production bill for a development pattern. A quick fix is auto-termination on all interactive compute resources. Add scheduled restart patterns for business-hour usage where startup delay matters. Auto-termination controls idle waste. It does not fix the underlying pattern when the workload lives on the wrong compute type. It is one thing to tell every team to use job compute. It is more useful to know exactly which production pipelines still run on all-purpose clusters. You also want to know how long those clusters sit idle and what the cost delta looks like. The `system.billing.usage` and `compute.node_timeline` join from the previous section answers those questions. Altimate AI Discover page for Jul 4 to Aug 3, 2026, showing $714–$742 in money savings, 3.8 hours of time savings, and 66 opportunities, above a table of opportunities such as continuous jobs running on an interactive cluster, each with resource details, savings estimates, effort rating, and assigned owner. *The Discover page in the* [*Altimate.ai*](https://altimate.ai) *enterprise platform surfaces this kind of opportunity.* In the screenshot, Discover flags a continuous job running on an interactive cluster and suggests switching it to a serverless job or a job cluster. Read more about [Databricks cost optimization with Altimate](https://altimate.ai/use-cases/altimate-for-databricks). ### Spot and Fleet Instances On classic compute, spot is still the most underused lever. Use spot instances for workloads that can tolerate interruptions. Keep the Spark driver, which is the first instance, on demand. Run the workers on spot capacity. Keep the Fleet instance types on AWS, where Databricks can choose the best-matching physical instance types by price and availability. See more in Databricks' [guidance on choosing optimal resources](https://docs.databricks.com/aws/en/lakehouse-architecture/cost-optimization/best-practices). However, Spot is not a universal answer. It works best when the workload is fault-tolerant and when retry, checkpointing, or restart behavior makes interruption acceptable. Batch ETL, retry-friendly pipelines, and some model-training jobs are natural candidates. Low-latency or interruption-sensitive workloads are not. Spot only touches the cloud bill. It does nothing to reduce Databricks DBU consumption. So on Jobs Compute, where the DBU rate is low and infrastructure is a large share, Spot moves more of the bill. On All-Purpose, where DBUs dominate, fixing the SKU matters more. Know which half you are attacking before you pick the lever. Levers like these are why classic can still beat serverless on cost for stable workloads, and why it demands more operational maturity. Someone has to decide what's interruption-tolerant, enforce the driver/worker pattern, and keep it consistent. > The cheapest Databricks configuration is usually the one whose behavior matches the workload, not the one with the lowest nominal unit rate. It also helps to know which levers that configuration actually exposes. ### Autoscaling That Doesn't Behave How You'd Expect Most people picture autoscaling as something that tracks CPU. Databricks autoscaling does not track CPU. Databricks describes [its optimized autoscaling](https://www.databricks.com/blog/2018/05/02/introducing-databricks-optimized-auto-scaling.html) in terms of workload behavior. Worker allocation reacts to job characteristics, mainly the backlog of pending tasks. A cluster can scale up while its existing nodes sit at 30 percent CPU because the stage produced a lot of small tasks. It can also stay flat under heavy CPU load because the work is never split into enough tasks to create a backlog. Which leads to five things people get wrong: - **A wide min/max range is not free insurance.** If you set the range to 2 to 20, a burst of small tasks pulls you toward 20. Autoscaling holds those nodes for the stage and bills you. Autoscaling did its job. The range was the decision, and the range was a guess. - **Your minimum is a floor you always pay.** A minimum of 8 workers means you never pay for fewer than 8, including the long tail where one straggler task is finishing. - **Streaming does not scale back down the way batch does.** A continuous stream always has work in flight. So the underutilization condition rarely holds. Databricks recommends Lakeflow pipelines with enhanced autoscaling for streaming rather than treating standard autoscaling as universal. Databricks also recommends triggered incremental patterns like AvailableNow when the business does not need 24/7 freshness. Whether you need 24/7 freshness is usually the bigger cost question. - **Autoscaling does not fix oversizing.** It is a range. Give it the wrong range, and it arrives at a wrong number faster. - **Cluster autoscaling and SQL warehouse scaling are not the same system.** Conflating the two is expensive, and it is worth its own explanation below. ### Warehouse size and warehouse scaling are different dials A SQL warehouse has two independent settings: - **Size** (2X-Small up to 4X-Large) is how much compute sits behind one cluster. Each step up doubles the [worker count](https://docs.databricks.com/aws/en/compute/sql-warehouse/warehouse-behavior), from 1 worker at 2X-Small to 256 at 4X-Large, and the DBU rate scales with it. A bigger size makes a single heavy query faster. - **Scaling** (min and max clusters) is how many identical clusters sit behind the warehouse. More clusters means more queries run at once. It does nothing for the speed of any single query. To understand, let’s take an example: users say the warehouse is slow. Someone bumps the size, which doubles the rate. But the actual problem was 30 analysts hitting it at 9am and queueing. No query was compute-starved; they were waiting. The size increase does not help, so someone bumps it again. You are now paying four times the original rate for a concurrency problem that more clusters would have fixed at the original size. The diagnostic is queue time in system.query.history. Significant time waiting for compute usually means scaling out, and long execution with negligible queue time means scaling up. Databricks treats a consistently non-zero queue as a sign that you need either a larger size or more clusters. Check which of the two the queue points to before you turn a dial. ### How to Right-Size Clusters and Warehouses "Right-sizing" is the most overused and underexplained phrase in Databricks cost conversations. Databricks' [cost optimization best practices](https://docs.databricks.com/aws/en/lakehouse-architecture/cost-optimization/best-practices) explain it as: > Sizing in terms of workload demands: total executor cores, total executor memory, local storage, data partitioning, computational complexity, and parallelism needs. It also recommends starting SQL warehouses at smaller sizes and scaling up only as concurrency and query complexity justify it. That definition splits right-sizing into several separate questions: - Is this workload interactive SQL, scheduled ETL, ad hoc exploration, streaming, or ML? - Is it on the right compute type at all? - Is the instance family aligned with CPU, memory, shuffle, or caching needs? - Is the driver/worker balance sensible? - Is the warehouse or cluster too large for its actual concurrency pattern? - Is it too small, forcing longer runtime and creating false savings? Many teams make expensive mistakes because they treat right-sizing as downsizing. But downsizing is only one possible outcome. Right-sizing can mean moving smaller or moving to a different instance family. It can mean moving a workload from all-purpose to job compute, or a bursty workload to serverless. It can also mean accepting a slightly larger resource, because the shorter runtime lowers the total cost. This distinction becomes more important in mixed Databricks environments where SQL warehouses, Spark jobs, development notebooks, and streaming pipelines all coexist. A team that applies one universal sizing philosophy across all of them usually ends up mis-sizing most of them. ## How to pick a right family instance for jobs Instance-family choices are a bit of a cumbersome decision but the highest-impact compute decisions in Databricks cost optimization. Databricks gives some simple rules of thumb in its AWS guidance: - memory-optimized for ML, heavy shuffle, and spill-heavy workloads - compute-optimized for structured streaming and maintenance jobs - storage-optimized for workloads that benefit from caching, such as ad hoc or interactive analysis - general-purpose when there is no specific dominant requirement - GPU only where GPU-accelerated libraries actually justify it, per Databricks’ [cost optimization best practices](https://docs.databricks.com/aws/en/lakehouse-architecture/cost-optimization/best-practices) The same rules apply to the other cloud providers. Instance family is a cost decision as much as a performance one. A badly matched family can create waste even when the cluster size looks reasonable. - A memory-heavy job on compute-optimized nodes may spill and run longer than necessary. - A shuffle-heavy workload on the wrong family may look “cheap” per node but cost more overall because of runtime inefficiency. - A cached interactive workload on the wrong shape can waste both time and money. To understand Databricks compute costs, count the nodes. Then ask what kind they are and whether they match your workload pattern. Auto Tune Settings modal for job email-job-bi-tool, showing Auto Tune enabled for 1 of 1 tasks with automatic rollback if execution time exceeds 1.5x baseline, and a lineage graph card comparing the current driver (r5.2xlarge, $0.18/hr) to the applied proposal (r5.xlarge, $0.09/hr) with the worker type unchanged. *Altimate Auto Tune uses workload behavior to determine if the current compute family is optimal for the workload. Auto Tune shows how your current configuration and our suggested one each perform. You understand the issue before you trust Altimate’s recommendation to make a change.* ## Keeping compute right-sized over time You now know what to set: the instance family, the compute type, and the autoscale range. You set all three on the cluster when you create it. Those values can go out of date within a few months, because of - Concurrency changes - Query patterns change. - Teams change. A development environment becomes shared production infrastructure. - A job that used to be small becomes a major pipeline. - A warehouse sized for one dashboard becomes the default for ten. Right-sizing is an ongoing process. Databricks recommends regular cost audits, ongoing monitoring, tagging, budgets, dashboards, and revisiting strategies as environments scale or change. This is also where operator (FinOps) trust becomes part of the conversation. As soon as you move from *we should size better* to *a system should help us keep sizing current,* the obvious questions appear: - How much access does the system need? - What is the blast radius if it gets something wrong? - Is there an approval step? - Is there an audit trail? - Can I dry-run it first? - Can it roll back? - What does rollback restore, and what does it not restore? Rollback restores configuration. It does not restore time. If a change made a pipeline late, rollback fixes the next run. But the data that landed late stays late, and downstream jobs already read it as stale. #### How Altimate Auto Tune adds control Flowchart of the Auto Tune safety loop: snapshot, apply, verify, then monitor performance, rolling back automatically if it regresses. The rollback limit is why Auto Tune leans on scope, verification, and rollback instead of asking you to trust it. Auto Tune is two agents, one for jobs and one for SQL warehouses. Each agent gets its own guardrails. Auto Tune history log for job 'email\_to\_looker,' listing timestamped audit entries where Auto Tune applied, enabled, or backed off driver changes (such as r5.2xlarge to r5.xlarge and i3.2xlarge to r5.2xlarge), each attributed to the system, a service user, or a named engineer. - **Snapshot, apply, verify:** Every change captures the original config and applies the update. Then Auto Tune reads the config back to confirm the change took. The original is always exactly restorable. - **Performance monitoring with automatic backoff:** Auto Tune watches performance against the recent baseline after every change. It tracks latency for warehouses and run health for jobs, and rolls back automatically if something regresses. - **Granular and approval-gated:** Control is per job, per task, and per warehouse. Nothing changes until you enable a recommendation. Jobs only change while idle and after policy checks pass. - **Stays current:** Once enabled, Auto Tune keeps following newer recommendations as the workload changes, under the same monitoring and backoff. That is how sizing stays current instead of going stale by next quarter. - **Fully audited:** Auto Tune timestamps every requested, applied, failed, and backoff event with who, which job or task, before, and after. You can filter and export those events. **Savings you can see:** Altimate reports realized savings, money already banked, separately from potential savings. Potential savings are ones Altimate identified but did not capture yet. The Autonomous Savings summary page shows the savings, with a breakdown per job and per warehouse if you want to drill down. Altimate AI Summary dashboard showing $83.37K total projected money savings over the next year, split into $74.20K autonomous and $9.17K assisted, with pie charts breaking savings down by autonomous vs. assisted and by DBU vs. cloud infrastructure. ## Which Compute Type for Which Workload | Workload | Compute | Key settings | | --- | --- | --- | | Interactive BI | Serverless SQL warehouse | Right-size, short auto-stop, scale *out* on concurrency | | Ad hoc SQL and exploration | Serverless, or classic with tight auto-termination | Start smaller than feels comfortable, size up on evidence | | Scheduled ETL and batch | Job compute plus Auto Tune | On-demand driver, spot or fleet workers, narrow autoscale range | | Short frequent jobs | Serverless jobs, or classic with a pool | Measure startup as a share of total runtime first | | Streaming | Job compute, or Lakeflow with enhanced autoscaling | Evaluate triggered (AvailableNow) before assuming always-on | | ML training | All-purpose or job compute with GPU | Pools for iteration, right-sized driver | The cheapest option depends on the workload. Compute gets expensive when teams pick one out of habit. ## What to Measure Before You Optimize Databricks Compute On classic compute, your Databricks DBU cost and your cloud infrastructure cost arrive on separate invoices. Read both before you change a setting. Use the Databricks system tables to find which workloads run on which compute type and which clusters sit idle. Most waste comes from idle time, the wrong compute type, and defaults nobody revisited. Join that data to your cloud cost and usage report to measure what spot, autoscaling, auto-termination, and instance-family changes save. --- See the official Databricks [cost optimization best practices](https://docs.databricks.com/aws/en/lakehouse-architecture/cost-optimization/best-practices) *and the well-known* [*cost maturity journey post*](https://www.databricks.com/blog/chaos-control-cost-maturity-journey-databricks) *on the Databricks blog.* ## Frequently Asked Questions ### How Is Databricks DBU Cost Calculated? Your Databricks DBU cost is the DBUs a workload consumes multiplied by the rate for its compute SKU. An all-purpose compute cluster on a standard runtime costs around $0.55 per DBU-hour in mid-2026. Classic compute adds a separate cloud invoice for VMs, disks, and networking. Serverless compute includes the VM cost in the DBU rate. ### How Do You Reduce Databricks DBU Consumption? Idle interactive compute, oversized clusters, and loose autoscaling ranges all waste DBUs. To reduce Databricks DBU consumption, tighten auto-termination and narrow the autoscale range. Check per-node CPU and memory in `system.compute.node_timeline` to find oversized clusters. Moving scheduled jobs from all-purpose to job compute lowers the DBU rate. Databricks says job compute is typically 2 to 3x cheaper. Your saving depends on idle time and on which half of the bill dominates. ### Do Databricks System Tables Show the Full Compute Bill? The Databricks system tables hold DBU usage and list prices, but none of them contains cloud infrastructure cost. To get the total bill, join Databricks usage to the cost and usage report from your cloud provider on resource tags. Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [Databricks](https://altimate.ai/blog?tag=databricks) [Altimate](https://altimate.ai/blog?tag=altimate) [finops](https://altimate.ai/blog?tag=finops) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Debug Slow dbt Models: Power User for dbt's Query Profiler URL: https://altimate.ai/blog/power-user-for-dbt-can-now-tell-you-why-your-model-is-slow New in Power User for dbt: one-click query profiling reads Snowflake, BigQuery, Postgres, and Databricks execution plans to explain why your dbt model is slow. [Back to blog](https://altimate.ai/blog) dbt · SQL · sql optimize ## Power User for dbt Can Now Tell You Why Your Model Is Slow Altimate.ai Team & Anand Gupta & Hoshang Mehta Aug 2, 2026 Power User for dbt Can Now Tell You Why Your Model Is Slow The queries that cost you money usually aren't the ones anyone complains about. A dbt model ships to prod looking fine, then scans ten times the data it needs on every scheduled run for the next six months… Nobody notices, because nothing is broken. It's just quietly expensive. The ones people *do* notice are bad enough: a model that used to build in two minutes now takes twenty. If someone asks why, nobody really knows. But the information needed to spot these issues and answer the questions does exist: your warehouse produces it every time it runs a query. The problem is that almost nobody reads that output, because it was never written for humans. That changes with the new [**Profile this query** feature in Power User for dbt](https://help.altimate.ai/dbt-power-user/test/queryResults/). One click sends your query to Altimate Code, which runs the warehouse's own diagnostics, reads the output, and tells you in plain language what the query is actually doing and where it's going wrong. Altimate Code, the agentic data engineering harness, powers many advanced features of Power User for dbt ## Why slow SQL is so hard to debug In a cloud warehouse, SQL is a *request*, not a set of instructions. You describe the result you want. The warehouse decides how to get it. That decision-making is invisible. Two queries that look almost identical can behave completely differently. One might read a few thousand rows. The other might scan an entire ten-billion-row table because a filter was written in a way the warehouse couldn't use. You can't see this by reading the SQL. The SQL describes *what*, not *how*. And in modern cloud warehouses, *how* is exactly where the time and money go. Compute is billed by usage. A query that scans more data than it needs isn't just slow. It's expensive, every single time it runs. Most dbt users respond to slow queries the only way they can: by guessing. Rewrite a CTE. Add a filter earlier. Try an incremental model. Sometimes it helps. Often it doesn't, because the guess didn't match the actual problem. ## What an execution plan actually tells you If you know how to ask, warehouses will tell you exactly how they run a query. Ask with a command like `EXPLAIN`, and you get back an **execution plan**: the warehouse's step-by-step strategy. Which tables it reads. How much data it expects to scan. How it joins tables together. Where it filters. What each step is estimated to cost. Some databases go further. `EXPLAIN ANALYZE` (in Postgres, for example) actually runs the query and reports real numbers: how long each step took, how many rows actually flowed through, whether the work fit in memory. This is the "ground truth" of query performance. If your query is slow, the reason is in the execution plan. So why does nobody read them? Three reasons. 1. **They're dense.** A plan for a moderately complex query can run to hundreds of lines of nested operators, cost units, and internal jargon. 2. **They're inconsistent.** Snowflake, BigQuery, Postgres, and Databricks each format plans differently, use different terminology, and even use different commands to produce them. 3. **They assume expertise.** Reading a plan well means knowing what a *hash join* is, why "estimated rows: 100, actual rows: 4,000,000" is alarming, and which operators are red flags. That's a database performance skill set, not an analytics engineering one. > **Warehouse execution plans** are like an X-ray of your query: Incredibly informative… if you happen to be a radiologist. ## What Altimate Code does differently [Altimate Code](https://github.com/AltimateAI/altimate-code/) is an open-source AI coding agent built for data work, with a live connection to your warehouse. That last part is the key. It doesn't just look at your SQL. It can run the diagnostic commands your warehouse supports, get the real plan back, and interpret it. In other words: it reads the X-ray for you. Because it knows the differences between warehouse dialects, the same button works whether you're on Snowflake, BigQuery, Postgres, or something else. You don't need to remember which explain command your warehouse uses or what its plan format looks like. (The exact diagnostics available do vary by warehouse. Some expose richer runtime detail than others, so the depth of the profile depends on what your database can report.) ## The dbt profile query workflow The whole thing takes one click. Step 1: **Run a query** in Power User for dbt, as you normally would. Screenshot of the Power User for dbt UI showing the "Run a query" feature. Step 2: When results load, click "**Profile Query"** in the query results panel toolbar. Screenshot of the Power User for dbt UI showing where to click "Profile Query" in the query results panel toolbar Step 3: **Altimate Code chat opens** with your SQL and a profiling prompt already in place. Screenshot of the Power User for dbt UI: Altimate Code chat opens with your SQL and a profiling prompt already in place Step 4: The agent runs or interprets the relevant warehouse diagnostics — the execution plan, and runtime statistics where available. Screenshot of the Power User for dbt UI showing Step 4: Altimate Code runs and interprets the warehouse's execution plan and runtime statistics Step 5: You get back **plain-language findings**: what the query is doing, where the time and cost are going, and what to try next. Screenshot of the Power User for dbt UI showing Step 5: Plain-language findings on what the query is doing, where the time and cost are going, and what to try next There's also "**Explain with Altimate Code"** in the SQL tab, which does something related but different. More on that distinction below. ## What dbt Query Profiler can actually find Here are the kinds of problems that hide in execution plans, and what a profile might surface. These examples are deliberately simple; real findings depend on your data and your warehouse. ### The full table scan ```sql select order_id, customer_id, order_total from analytics.fct_orders where date(created_at) = '2026-08-03' ``` This looks fine. It filters to a single day. But wrapping `created_at` in `date()` can prevent the warehouse from using **pruning** — its ability to skip chunks of data that can't possibly match the filter. Instead of reading one day's worth of data, it may read the whole table and check every row. A profile of this query might surface something like: > *the plan shows a scan of the full table rather than a pruned range; the* `date()` *function on the filter column is likely preventing partition pruning; rewriting the filter as a range on the raw column should let the warehouse skip most of the data:* ```sql where created_at >= '2026-07-20' and created_at < '2026-07-21' ``` Same result. Potentially a small fraction of the data scanned. ### The join that explodes ```sql select c.customer_id, count(*) as event_count from dim_customers c join fct_events e on c.customer_id = e.customer_id where e.event_date >= '2026-07-01' group by 1 ``` Suppose `fct_events` is huge, and the plan shows the join producing 200 million intermediate rows before the date filter is applied. That's a **filter applied too late**: the warehouse did an enormous amount of join work, then threw most of it away. A profile might point out that the row count balloons at the join step, and suggest filtering the events table first (for instance, in a CTE) so the join only touches July's events. The output is identical; the work is not. ### The dbt row estimate that's wildly wrong Warehouses plan queries using **row estimates**: guesses about how many rows each step will produce, based on statistics about your data. When those guesses are badly wrong (i.e.: the plan expected just 100 rows and got 4 million) the warehouse may have picked a join strategy or memory allocation that made sense for 100 rows and falls over at 4 million. This is one of the most common causes of "this query is sometimes fine and sometimes terrible," and it's essentially invisible without looking at a plan. A profile can flag the mismatch and point at the step where the plan's assumptions diverged from reality, which is often the first real clue to stale statistics or a skewed join key. ### The expensive aggregation A `count(distinct user_id)` over a billion rows, or a `group by` on a high-cardinality column, can dominate a query's runtime, sometimes spilling work to disk when it doesn't fit in memory. The plan shows this. A profile can identify that the aggregation step is where the time goes and suggest alternatives worth testing, like pre-aggregating in an upstream model or, where approximate results are acceptable, an approximate distinct count. ## Explain vs. Profile: what's the difference? Power User for dbt now gives you two related (but different) tools: **Explain this query** answers: *what does this SQL do?* It reads the query and describes the logic: the joins, the filters, the transformations. It's great for understanding unfamiliar code, reviewing a teammate's model, or onboarding onto a project. It doesn't need to touch your data. Screenshot of the Power User for dbt UI showing "Explain with Altimate Code" describes the query's logic without touching the warehouse *"Explain with Altimate Code" describes the query's logic without touching the warehouse* Screenshot of the Power User for dbt UI showing the "Explain" panel, breaking down joins, filters, and transformations The Explain panel breaking down joins, filters, and transformations **Profile this query** answers: *how does this query perform against my actual warehouse?* It's grounded in diagnostic output from the database: the execution plan, real or estimated row counts, the actual strategy the warehouse chose. It can tell you things no amount of reading the SQL can, because the answer depends on your data: its size, its distribution, how it's clustered or partitioned. "Profile this query" is grounded in the warehouse's actual execution plan "Profile this query" is grounded in the warehouse's actual execution plan Screenshot of the Power User for dbt UI showing the Profile panel surfacing real row counts and cost data from the warehouse The Profile panel surfacing real row counts and cost data from the warehouse This is also what separates Power User profiling from pasting your SQL into a general-purpose chatbot. A chatbot can only reason about the text of your query. It doesn't know that your events table is 3TB, that your filter isn't hitting the partition column, or that the optimizer's row estimate was off by four orders of magnitude. Those facts live in your warehouse, and profiling is what brings them into the conversation. ## What health query profiling means in practice **Cost.** In cloud warehouses, wasted scans are wasted money, compounded by every scheduled run. A model that scans ten times the data it needs isn't a one-time problem — it's a recurring line item. Profiling makes that waste visible while the query is still in front of you. **Speed.** Faster queries mean faster dbt builds, faster CI, faster dashboards. The fixes that profiling points toward — earlier filters, prunable predicates, leaner joins — tend to be small edits with large effects. **Team productivity.** Perhaps the biggest shift is who can do this work. Today, deep query tuning tends to bottleneck on whoever on the team can read plans. Profiling puts a first-pass diagnosis in every analytics engineer's hands. Instead of "this is slow, can someone look at it," the starting point becomes "the profile says the join is exploding before the filter — I'm going to try restructuring it." One note on positioning: profiling doesn't guarantee a faster query, and it doesn't rewrite your models for you. What it does is close the gap between *"this query is slow"* and *"here is what is likely causing it, and what to try next."* It's still a judgment call when it comes to what to change (and whether the tradeoff is right for your model). ## Try dbt query profiler today Query Profiler is live in Power User for dbt today. [You will find the docs here](https://help.altimate.ai/dbt-power-user/test/queryResults/). You can access the feature as follows: - **Profile this query**: in the query results panel toolbar, visible once query results are loaded. Run a query, then click it. - **Explain with Altimate Code**: in the SQL tab, for when you want to understand what a query does rather than how it performs. Both open Altimate Code chat with the context already loaded, so you can keep asking follow-up questions: "*why is this step expensive?", "What would happen if I filtered earlier?", "Show me the rewritten version."* If you're already running Power User for dbt, **update to the latest version** and try it on the slowest model in your project. You may find the answer has been sitting in your warehouse all along, just waiting for someone to read it. --- [*Altimate Code and Power User for dbt (available on VS Code)*](https://altimate.ai/products/dbt-power-user) *is open source. Have feedback on profiling? We're actively improving dialect coverage and would love to hear what you find.* Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [dbt](https://altimate.ai/blog?tag=dbt) [SQL](https://altimate.ai/blog?tag=sql) [sql optimize](https://altimate.ai/blog?tag=sql-optimize) [Altimate](https://altimate.ai/blog?tag=altimate) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Snowflake AI Cost Optimization With Warehouse Auto-Tuning URL: https://altimate.ai/blog/altimate-lite-autonomous-agents-for-snowflake-cost-optimization Altimate Lite automates Snowflake warehouse auto-tuning for AI cost optimization. Free 21-day trial, ~19% average savings, no performance impact. [Back to blog](https://altimate.ai/blog) Snowflake · AI costs · AI ## Autonomous Agents for Snowflake Cost Optimization: Announcing Altimate Lite Fraser Marlow Jul 28, 2026 Autonomous Agents for Snowflake Cost Optimization: Announcing Altimate Lite ## **TL;DR: it saves you money on Snowflake** [Altimate Lite is a Snowflake native app](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) that automatically tunes warehouse configuration (auto-suspend, scaling, clustering) roughly every 5-6 seconds to cut compute costs, with no performance impact. Private preview customers saved an average of 19% on warehouse costs, with some seeing savings as high as 57.5%. As a Snowflake native app, it’s easy to adopt, secure, and can be paid with Snowflake credits. It's free for 21-days, then just $100/month plus 2.5% of the daily cost of any warehouse you activate it on. [Find it in the Snowflake Marketplace](https://app.snowflake.com/marketplace/listing/GZTYZ1VSPRPWK/altimate-ai-altimate-lite-for-ai-and-warehouse-cost-optimization) today. ## Watch the Launch video [YouTube video player](https://www.youtube.com/embed/pPDlkjMCSJY) ## How AI costs get out of hand on Snowflake If you think your Snowflake instance is costing more than it should, you are not alone. There are several factors that drive this: First, a common pattern keeps showing up across Snowflake teams that adopt AI: everything starts to look like an AI problem. Teams transition away from cheap deterministic tooling and start to adopt inference-based solutions across the board. This gets expensive. Then there is the "Maximilian Fable" persona: always reaching for the newest, most expensive reasoning model. Tasks that a lightweight model like Anthropic’s Haiku would handle just fine often get clobbered with one of the most expensive models like Fable. Snowflake offers some tooling for tuning your instance, but the process is manual and the audits too far apart to get good results. Snowflake users overspend as a result. Finally there's cost visibility. Most teams just get a bill at the end of the month, a token count with no breakdown of what drove it. It’s a single total, no itemization, and no way to explain to finOps why this week's number is double last week's. These costs really add up. There's now a well-known case of a company running up a [$500 million AI bill](https://finance.yahoo.com/sectors/technology/articles/company-blew-500m-claude-ai-173519468.html) in a single month because no check was in place and nobody saw it coming. The fix starts with tracking and visibility: who's consuming AI credits, whether usage is concentrated in a handful of users, and which models and services are actually driving spend. ## Continuous Auto-Tuning: The Other Lever for Cutting Snowflake Warehouse Costs Cost isn't just an AI-model problem. It's also a Snowflake warehouse configuration problem. Snowflake exposes a handful of levers: warehouse size, warehouse type (Gen 1, Gen 2, adaptive), auto-suspend timing, multi-cluster scaling, and max concurrency level. Getting these right, and keeping them right as workloads shift, is normally a manual, periodic exercise: someone looks at usage patterns, makes a best guess, and revisits it weekly or monthly. What if software made these decisions *continuously* instead? Not daily reconfiguration but adjusting configuration **every five to six seconds**, a frequency no human team could sustain. An Agent Decisions chart showing the daily number of auto-tune configuration changes made over a one-month period. By comparison, Snowflake's own auto-suspend only probes warehouse activity every 30 seconds; in that same window, Altimate's auto-tune has already run roughly ten checks, looking at idle state, workload shifts, and predicted upcoming load. ## Real World Results: Up to 57.5% in Snowflake Warehouse Savings Over the last couple of months during private preview, real Altimate Lite customers saw meaningful savings simply by turning on auto-tune, with no degradation in query performance or queue times: - Lowest savings observed on any warehouse, any day: **12.4%** - Average savings across all customers, warehouses, and days: **19%** - Highest single-day savings (typically weekends, when workloads are less consistent): **57.5%** ## Inside Altimate Lite: Cost Dashboard, Auto-Tune, and AI Cost Observability Altimate Lite is a distilled version of [Altimate's enterprise platform](https://altimate.ai/platform) that runs natively on Snowflake. Is was trained on billions of config changes from years of production use. Because it runs entirely inside your Snowflake environment, no data leaves your account, which sidesteps the InfoSec review that a SaaS tool would typically require. [In the demo](https://www.youtube.com/watch?v=pPDlkjMCSJY), Pradnesh walked through the core screens: - **Cost dashboard**: trending costs, potential annual savings, and how many warehouses are auto-tune eligible vs. already enabled. Altimate Lite warehouse detail page for a dbt pipeline warehouse showing Auto Tune enabled, realized savings of $351, and projected annual savings of $4,269. - **Per-warehouse view**: estimated 30-day cost, potential (or realized) savings, and a one-click toggle to turn auto-tune on or off. Altimate Lite warehouses dashboard showing 30-day total cost, potential annual savings, auto-tune eligible and enabled warehouse counts, and a daily spend and savings chart. - **Decision history**: a full audit trail of every action taken (cluster suspensions, resize decisions) timestamped and exportable as CSV. On a single warehouse, the system logged 118 tuning decisions in one day. Agent Decisions chart and Auto Tune history log showing a timestamped list of warehouse suspend actions. - **AI cost observability**: a separate view tracking spend, token usage, and adoption across roughly eight AI services (AI SQL functions, Cortex, Code CLI tools, and more), filterable by service, user, and model, with CSV export for further analysis. Daily AI cost chart in Altimate Lite, stacked by Cortex service, showing spend trends over a one-month period. *Daily AI cost chart showing spend trends over a one-month period.* Altimate Lite per-user AI usage detail page showing cost, tokens, queries, and daily spend for a single Snowflake user. *Per-user AI usage showing cost, tokens, queries, and daily spend for a single Snowflake user.* ## Altimate Lite Pricing: Free 21-Day Trial, Then $100/Month Altimate made a deliberate choice to keep pricing transparent and self-serve. No sales call required: - **21-day free trial**, no cost. - **$100/month** flat fee after the trial. - **2.5% of daily warehouse cost (after savings)** for each warehouse you activate auto-tune on. You only pay for the warehouses you turn on, regardless of how many others exist in your account. The recommended path: turn it on for one or two warehouses, watch the savings and audit history for a bit, and expand from there once you're comfortable. Find Altimate Lite on the Snowflake Marketplace. ## Altimate Lite vs. Enterprise: What's the Difference? Altimate Lite covers auto-tune and AI cost visibility. [The Enterprise edition](https://altimate.ai/platform), which has been running in production for customers processing billions of queries, adds: - **Auto-resize**: automatic warehouse right-sizing as workloads shift. - [Altimate](https://altimate.ai/platform#studio) **AI Studio**: a collaborative agent workspace for deeper savings analysis, scheduled reporting, and team-based showback/chargeback with SSO integration. - **Assisted optimization**: guided recommendations for improving query code, table and storage configuration, and dbt models, safely assignable to teams. - **Altimate Code**: [an open-source CLI and VS Code extension](https://github.com/AltimateAI/altimate-code/) that catches costly SQL and dbt mistakes during development. On data handling: Enterprise is a hosted SaaS product that analyzes query history and telemetry (not your underlying business data) to power these features. The Altimate platform is SOC 2 compliant and has been through security review with financial services and healthcare customers, including in Europe. ## Altimate Lite: Key Questions Summary: **Is Altimate Lite free to try?** Yes. Altimate Lite includes a 21-day free trial with no cost. After that, it's $100/month plus 2.5% of the daily cost of any warehouse where you've enabled auto-tune. **Does my data leave my Snowflake account?** No. Altimate Lite runs entirely inside your Snowflake environment as a native app, a walled garden, and none of your data ever leaves your account. **How much can I save?** Private preview customers saved an average of 19% on warehouse costs, with a low of 12.4% and weekend peaks as high as 57.5%, with no measurable impact on query performance. **How is this different from Snowflake's built-in auto-suspend?** Snowflake's auto-suspend checks warehouse activity every 30 seconds. Altimate's auto-tune runs roughly ten checks in that same window and adjusts configuration every five to six seconds based on predicted workload. **How to Get Started with Altimate Lite's Free Trial** Altimate Lite is live now on the Snowflake Marketplace. Start your 21-day free trial today. If you want to talk through auto-resize, Enterprise features, or just how to think about AI cost management more broadly, reach out to the Altimate team, no obligations, no high-pressure sales. Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [Snowflake](https://altimate.ai/blog?tag=snowflake) [AI costs](https://altimate.ai/blog?tag=ai-costs) [AI](https://altimate.ai/blog?tag=ai) [Altimate](https://altimate.ai/blog?tag=altimate) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # AI Agents in Data Engineering: Guardrails & Validation URL: https://altimate.ai/blog/what-ai-agents-are-really-changing-in-data-engineering How Altimate AI's Pradnesh Patil validates AI agents with data diffs, guardrails, and governance for production data pipelines. [Back to blog](https://altimate.ai/blog) AI · Data Engineering ## What AI Agents Are Really Changing in Data Engineering Fraser Marlow Jul 22, 2026 What AI Agents Are Really Changing in Data Engineering Now that job candidates write their resumes and rehearse their interview answers with AI, every application reads the same. "Now every resume is perfect," Pradnesh Patil said on DataCamp's "[The New Data Engineering Team](https://www.datacamp.com/resources/webinars/the-new-data-engineering-team) " panel, part of its Data Science and Engineering Week. Patil, co-founder and CEO of Altimate AI, joined host Richie Cotton alongside [Neha Tharani](https://www.linkedin.com/in/nehatharani/), data foundation lead at the reinsurer SCOR, and [Lisa Mirkovic](https://www.linkedin.com/in/lisa-mirkovic/), a data and AI strategy adviser who previously ran data engineering at Capital Group. Generative AI changed how data engineers work, and it also broke the funnels companies use to hire them. [Catch the full episode here.](https://www.datacamp.com/resources/webinars/the-new-data-engineering-team) ## Hiring when the resume stopped carrying signal Resumes are not the only signal AI degraded. Video interviews are straightforward to game with live prompting tools, which leaves conversation as a weak test of anything. Altimate AI rebuilt its process around work instead: exercises capped at sixty to ninety minutes, followed by a short discussion. Candidates are told the rules upfront, in Patil's words, "you can use as much AI as you want." The exercise measures what someone builds with AI available, not whether they can work without it. Patil noted that startups across the San Francisco Bay Area are changing how they interview for the same reason. Skills assessments built around real tasks are displacing credential and resume screening. ## Validate agent output with a data diff, not a second model Large language models are probabilistic, so the validation layer around their output is the thing you actually build. Patil's example was SQL. When an agent rewrites or optimizes a query, the test is whether it returns the same data as the original: "is it producing the same data with a new query?" You do not need a second model to check that, only a data diff tool that compares the two result sets directly. The pattern generalizes to anywhere an agent touches a pipeline, a transformation, or a query. Each of those places needs a lightweight deterministic check attached to it rather than an assumption of correctness. ## AI documents move the cost to the reviewer AI also created a collaboration cost that is easy to miss. A teammate generates a six-page document in five minutes, then expects a colleague to spend forty-five minutes reading it. Patil's rule is to edit AI output down before passing it on, because the models are built to overproduce. As he put it, "all these LLM models are text hungry. If you ask it to do something instead of writing five lines, it's going to write five pages." ## Governance became a daily job Governance is where the new risks concentrate, and Patil pointed to public incidents rather than hypotheticals: an agent that ran a query costing more than five thousand dollars, and production databases dropped at larger companies. "If you let it scale without guardrail, it's going to do crazy things," he said. A per-tool setting does not contain that, because an agent moves across tools and the guardrail has to move with it. Altimate AI open-sourced a rules-and-permissions framework, Altimate Core, under an MIT license as one starting point. ## Where agents already pay off Fenced in properly, agents produce measurable wins. Patil described a Fortune 500 company that built agents to manage its data infrastructure, resizing instances and adjusting configuration automatically as workloads shifted. Utilization climbed and the bill fell by 30 to 40 percent. The reason is bandwidth: "as humans, we can't change infrastructure configurations, thousand times a day, but a machine can." For platform teams deciding where to deploy agentic automation first, infrastructure tuning is the low-risk, high-return place to start. ## Getting knowledge out of people's heads Mirkovic described where a team's working knowledge actually lives, "hidden in Wikis and Slack channels and in a senior developer who is there twenty seven years, and only that person knows quirks that are not even in a GitHub repo." Patil sees teams democratizing that knowledge so business users self-serve the simple questions and bring back only the hardest 20 percent. That is the direction both agreed on: less work done by hand, more work made repeatable for AI and for everyone else. Teams building 2026 roadmaps can use that split as the test for where agents belong in the stack and where human judgment still has to lead. If the governance layer is the gap, start from Altimate Core instead of writing your own rules engine. Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [AI](https://altimate.ai/blog?tag=ai) [Data Engineering](https://altimate.ai/blog?tag=data-engineering) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Why AI Agents Break in Production: The Missing Harness URL: https://altimate.ai/blog/why-ai-agents-break-in-production-the-missing-harness-in-your-data-stack AI agents silently write wrong SQL joins and query phantom tables. Data engineering teams need a governance harness—not bigger LLMs—to fix it. [Back to blog](https://altimate.ai/blog) Data Science · Data Engineering · AI ## Why AI Agents Break in Production: The Missing Harness in Your Data Stack Pradnesh Patil Jul 15, 2026 Why AI Agents Break in Production: The Missing Harness in Your Data Stack *Listen to the full conversation on DataScienceWithSam, available on* [*Apple Podcasts*](https://podcasts.apple.com/us/podcast/ep-45-why-ai-agents-break-in-production-the-missing/id1587954336?i=1000776852944) *and* [*Spotify*](https://open.spotify.com/episode/0nKtoDZCNpcgD2NOAyFVzO?si=Vzos9X0YTkSeS0bUXoF9Mw&nd=1&dlsi=3d298d39cd99420e). Most data teams have lived some version of this same story: an AI agent runs a query, the query returns a result, the number looks reasonable, and the dashboard ships. Weeks later, someone traces a bad decision back to that query and finds a join that never should have executed the way it did. Nobody caught it in the moment because nothing about the failure looked like a failure. On this episode of DataScienceWithSam, Pradnesh Patil, Co-Founder and CEO of Altimate AI argues that this is not a symptom of weak models. It is the predictable result of asking an agent to act without the infrastructure required to know what is true about the data. The gap is not in what the model can do, it is in what the model has been given to work with. ## AI Agent Harness vs. System Prompt: Why Context Grounding Matters A system prompt tells a model what to do. A harness tells it what is actually true. That distinction, Patil says, is the one most teams get wrong when their first AI data agent rollout stalls out or quietly produces bad numbers. Teams write increasingly elaborate instructions into the prompt, hoping better wording will compensate for the model having no grounded view of the schema, the lineage, or the cost of the action it is about to take. It rarely does. Patil breaks the agentic data engineering harness into five components: context, governance, MCP tools, shared skills, and agent infrastructure. Each one answers a different question an agent needs answered before it acts responsibly. Context tells the agent what the data actually looks like today. Governance tells it what it is allowed to do. Tools give it the concrete means to act. Shared skills capture the institutional knowledge a team has already built up. Agent infrastructure ties all of it together into something that runs reliably at scale. Miss any single one of those five components, Patil says, and the resulting failures are silent. The query runs, it returns a result, maybe the result looks correct to any human checking... but it is not, and nothing in the interface tells anyone that until the damage has already spread downstream into a report, a model, or a decision. Between 27% and 33% of AI-generated queries reference tables that do not exist in the schema, phantom references that a grounded agent would never produce in the first place. A further 78% contain what Patil calls "silent wrong joins": joins that execute cleanly, return a syntactically valid result, and still return the wrong data. Neither statistic describes a model that is bad at SQL. Both describe what happens in the absence of infrastructure that keeps a model grounded in the real, current state of the data it is querying. ## Why a Well-Designed Harness Outperforms Larger LLMs (Sonnet vs. Opus Benchmark) The instinct when an agent underperforms is to reach for a bigger model. Patil pushes back on that instinct directly, and points to [Altimate Code](https://altimate.ai/products/altimate-code) 's performance on ADE-Bench as the clearest evidence against it. Altimate Code topped the leaderboard running on Anthropic's Sonnet, while competing systems relied on Anthropic's Opus, a materially larger and more expensive model, to reach lower scores. The takeaway is not that a smaller model can outperform a larger one under identical conditions, but that the system surrounding the model, the harness that grounds it in context and governs what it is allowed to do, determines outcomes more than raw model capability does. For data teams evaluating where to invest, that reframes the question. The more valuable investment is often not the next model upgrade. It is the infrastructure that makes any model reliable once it is put to work on real data. ## Deterministic vs. Probabilistic Reasoning in AI Data Agents A recurring theme here is the boundary between deterministic and probabilistic reasoning. Validation, cost checks, and query correctness are deterministic problems, Patil argues: they have a right answer that does not depend on interpretation, and they should never be handed to a model to reason about probabilistically. A query either references a table that exists or it does not. A join either matches the keys correctly or it does not. A cost estimate either falls inside budget or it does not. None of that needs a language model's judgment. Hard rules should handle hard rules, and the model's judgment should be reserved for genuine ambiguity, the calls that actually require weighing context and making a reasoned choice. Building a harness that respects this boundary, rather than routing everything through the model, is what keeps an agent from confidently producing a wrong answer with the same tone of voice it uses for a right one. ## Context Compaction for Long-Running Data Pipelines: Preserving Schema and Lineage Standard LLM context compaction was designed for general-purpose chat, where discarding older turns of a conversation carries little cost. Applied to a long-running data engineering task, the same compaction approach tends to discard exactly the schema and lineage context an agent needs to keep working correctly. An agent that loses that context mid-task does not usually stop. It keeps going, with a thinner and thinner grip on what is actually true, until its output drifts from correct to merely plausible. Altimate's response is a compaction approach built specifically for this kind of task, one that treats schema and lineage as context to be preserved rather than as expendable conversational history. The distinction matters most on the tasks that take the longest: multi-step pipeline builds, large-scale migrations, and anything else where an agent is expected to stay grounded over many turns rather than a handful. ## Ungoverned AI Query Costs: A $5,000 Cortex AI Bill Case Study Governance is not an abstract concern here. Patil cites a real $5,000 Cortex AI query bill as the kind of outcome that permission-based governance helps avoid. The bill was not the result of malicious intent or a rare edge case. It was the predictable outcome of an agent operating without limits on what it was allowed to query and at what cost. Without [**controls on scope and spend**](https://altimate.ai/use-cases/altimate-for-snowflake), Patil argues, agents can go rogue on cloud cost just as easily as they can go rogue on correctness, and the two failure modes compound each other. An ungoverned agent that is also ungrounded in the real schema is not just likely to produce a wrong answer. It is likely to produce a wrong and expensive answer, executed against the wrong tables, at a cost nobody approved in advance. ## The Future of Data Engineering: From Hand-Written SQL to Agent Governance Patil's broader claim is that the era of hand-writing SQL is ending. The role is shifting toward building and governing the systems that let agents do that work reliably: defining the context that grounds them, the governance that constrains them, and the shared skills that let institutional knowledge outlive any one person's tenure on a team. His advice reflects that same emphasis on infrastructure over individual features: Build open source, build cross-platform, so that the tools work across Snowflake, Databricks, and every other environment your data estate actually spans. Avoid siloed AI features that solve a narrow problem for one vendor's users without moving the broader field forward. For Patil, the teams that internalize this shift early, and start investing in the harness rather than the next model swap, are the ones that will find AI agents genuinely trustworthy in production, rather than merely fast and occasionally, silently, wrong. *Listen to the full conversation on DataScienceWithSam, available on* [*Apple Podcasts*](https://podcasts.apple.com/us/podcast/ep-45-why-ai-agents-break-in-production-the-missing/id1587954336?i=1000776852944) *and* [*Spotify*](https://open.spotify.com/episode/0nKtoDZCNpcgD2NOAyFVzO?si=Vzos9X0YTkSeS0bUXoF9Mw&nd=1&dlsi=3d298d39cd99420e). Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [Data Science](https://altimate.ai/blog?tag=data-science) [Data Engineering](https://altimate.ai/blog?tag=data-engineering) [AI](https://altimate.ai/blog?tag=ai) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # 4 Blind Spots of Coding Agents in Data Engineering URL: https://altimate.ai/blog/4-blind-spots-of-general-coding-agents-in-data-engineering Discover where general coding agents like Claude fail at data engineering—schema hallucination, cross-warehouse gaps, and more. [Back to blog](https://altimate.ai/blog) Data Engineering · AI · llm ## 4 Blind Spots of General Coding Agents in Data Engineering Syed Haider Jul 6, 2026 4 Blind Spots of General Coding Agents in Data Engineering ## **A coding agent enters the data warehouse** A data engineer drops a one-liner into Claude Code: *"Add* `patient_visit_count` *to* `dim_patient`*."* Twenty seconds later, Claude has read the project, scanned `sources.yml`, and written plausible-looking SQL. `dbt compile` is happy. The PR merges Friday evening. At 2 AM Saturday, the scheduled job fires. The model references `PATIENT_ID`, but the actual warehouse column is `SUBJECT_ID`. The run fails in production. Nothing about the SQL looked obviously wrong. Claude inferred the schema from surrounding code and naming patterns instead of verifying it against the actual warehouse. Claude Code can write a dbt model. What it can't do, by itself, is run that SQL against your warehouse before you ship it. This is one instance of a broader pattern. General coding agents like Claude, Cursor, and Copilot are designed for software where the source code *is* the truth. Data engineering inverts that: the *warehouse* is the truth, dbt YAML is its description, the agent's SQL is a guess. When the warehouse and the code disagree, only the warehouse is right, and the agent has no way to ask. A single trace of a coding agent writing a dbt model. The agent reads the project, infers column names from sources.yml, writes plausible SQL, dbt compile passes, the PR merges. Hours later in production: column does not exist, scheduled job fails, pager fires at 2 AM. This isn't a hypothetical. In our first experiment, bare Claude shipped a dbt model with **two hallucinated column names** that looked right based on the surrounding code but didn't exist in the actual Snowflake table. `dbt compile` passed. The model would have failed on first real run. The altimate-code plugin catches this because Claude calls it before generating, and it actually queries the warehouse for the real columns. > The altimate plugin gives Claude warehouse hands: schema introspection, cross-warehouse execution, dbt-idiomatic defaults. That's what turns a code generator into something that finishes the task. We ran three categorically different experiments to find out where this matters and where it doesn't. The answer is more interesting than "always" or "never.", and reveals four common blind spots when deploying coding agents on data engineering tasks. --- ## **The three experiments** We deliberately picked task shapes that stress different parts of an AI data engineering workflow: Three side-by-side experiment panels. Task 1 Snowflake patient mart: plugin caught hallucinated columns at parity cost. Task 2 MSSQL to Snowflake migration: cross-warehouse the only path that finished. Task 3 Local DuckDB benchmark: passive bias rescued one of five hard failures. Each task answers a different question about where the plugin actually earns its keep. Taken together, they give you a framework to predict where it'll pay off on *your* workloads. --- ## **Blind spot #1: Schema hallucination on real warehouses** **The task:** A real Snowflake mart-layer refactor of a patient-360 model with HIPAA constraints, and dozens of column-level decisions. **What happened bare:** Claude generated a working-looking model. Two column names were inferred from context, close to right but wrong. `dbt compile` was happy. Code review by a human would likely have missed them (they looked plausible). **What happened with the plugin:** Claude delegated schema introspection to [altimate-code](https://altimate.ai/products/altimate-code) 's `altimate-dbt columns` tool, which hit Snowflake directly and returned the actual columns. The hallucinated names were caught before generation. Two side-by-side traces of the same dbt task. Bare Claude infers column names, compile passes, then production breaks. With altimate plugin Claude introspects the warehouse for real columns and ships correct SQL. **The cost was a wash.** Plugin runs landed at near-parity per-token spend. So the correctness win is essentially free for any real-warehouse work. > *If your Claude is touching real warehouse models, the (free) plugin pays for itself by not shipping broken column names.* --- ## Blind spot #2: Cross-warehouse work the agent can't reach **The task.** Migrate a customer's project from MSSQL to Snowflake. Validate parity. This requires connecting to *both* warehouses, reading from one, comparing aggregates to the other. **What happened bare.** Five attempts in a row, Claude crashed on the connectivity layer. No `pyodbc` driver. No `pymssql`. No idea how to find MSSQL credentials. **Five runs produced zero working output.** **What happened with the plugin.** altimate-code's pre-baked warehouse adapters drove the cross-DB diff end-to-end across four internal sub-sessions, with 9+ real `pyodbc.connect()` calls and a working parity report at the end. **The honest cost:** ~6× a bare-Claude run. But bare-Claude's run produced *nothing*. The right comparison isn't dollars-per-attempt, it's **dollars-per-correct-answer**, and bare-Claude's denominator is zero. Two bars showing total spend per configuration on the cross-warehouse migration task. Bare Claude spent about $1.10 across 5 attempts producing 0 correct outputs. Altimate plugin spent $6.94 in 1 attempt producing 1 correct output. The cost-per-correct-answer metric flips the apparent picture. > *For cross-warehouse migrations, parity work, or source-of-truth reconciliation, the plugin isn't a nice-to-have. It's the only path that finishes.* --- ## Blind spot #3: Default SQL patterns aren't dbt-idiomatic This was the surprise. **The benchmark.** ADE-Bench: 24 dbt tasks running against local DuckDB, objectively scored. Crucially, Claude has *everything it needs locally*: no cross-warehouse, no missing drivers. This is the "where the plugin shouldn't matter" scenario. **The headline result.** Of 5 hard tasks bare Claude failed, the plugin's *passive presence* (just being installed; no Skill tool invocation, no subprocess delegation) **rescued 1**. Specifically: an ADE-Bench task called `intercom003` went from FAIL → PASS for an extra ~$0.13 in tokens. Here's what changed in the output: | Operation | Bare Claude (FAIL) | With plugin loaded (PASS) | | --- | --- | --- | | **Removing duplicate rows** | implicit join, missed dupes | `QUALIFY ROW_NUMBER() OVER (...)` | | **Handling missing values** | "empty values expected" | `COALESCE(..., 0)` everywhere | | **Computing time differences** | `date_diff('second',...)` | `EXTRACT(EPOCH FROM...) / 60` | Bare Claude defended its wrong output as expected behavior. Plugin Claude wrote dbt-idiomatic SQL. **The Skill tool never fired.** The plugin's value here was purely the loaded skill descriptions sitting in Claude's context, biasing it toward correct patterns. And the `intercom003` task wasn't the only one we ran. We tested the full 5 hard ADE-Bench tasks across 4 configurations (passive, forced consultation, mandatory delegation) to map which mechanisms actually flipped failures: ADE-Bench rescue matrix. Five hard tasks where bare Claude failed, tested across four configurations. Two rescues total. Intercom003 rescued by passive plugin presence at near-zero extra cost. Analytics engineering 006 rescued by mandatory delegation at eighteen times bare cost. Three of four forced-skill consultations did not translate to delegation, rescued zero. Two rescues, two completely different mechanisms. The **passive presence** rescue (`intercom003`) was nearly free. The **mandatory delegation** rescue (another ADE-Bench task, `analytics_engineering006`) cost 18× a bare run but turned 0/7 tests into 7/7. Forcing the Skill tool to fire without demanding actual delegation rescued nothing. Claude reads the guidance, then proceeds to do the work itself. > *Even if Claude never delegates to altimate-code, the plugin's presence is a free correctness prior on dbt-shaped code.* The cheapest mode of adoption (just enable the plugin and walk away) measurably improves dbt output quality. --- ## **Blind spot #4: Wasted exploration on tasks the agent could solve** *The first three blind spots are about things Claude can't do at all without the plugin. This one is different: it's about tasks Claude can already solve and whether the plugin makes those cheaper or more expensive. The answer is surprising: just having the plugin loaded changes how Claude explores, even when no plugin tools ever fire.* The `intercom003` task rescue hinted at something deeper than tool augmentation: the plugin was changing Claude's *search behavior* even when no plugin tools fired. So we asked whether that same passive context bias also reduced cost on tasks Claude *already* solved correctly. We replayed three ADE-Bench medium tasks where bare Claude already produced correct, test-passing answers (`f1006` semantic data fix, `f1010` analysis gotchas, `airbnb009` missing-days debug), this time with the plugin enabled. Both configurations passed all tests. We compared turns, tool calls, and spend. Token efficiency across three ADE-Bench medium tasks where both bare Claude and Claude with the altimate plugin produced correct answers. Aggregate cost dropped 18.6 percent. Per-task deltas: f1006 down 41 percent, f1010 down 8.7 percent, airbnb009 up 2.5 percent. *(Note:* `airbnb009` *above is a medium-complexity task, and different from* `airbnb005`*\-shape, the simpler task class discussed in the next section of this article.)* **The aggregate is a 18.6% cost reduction with the plugin enabled, on tasks where both produce identical, correct outputs.** Two of three tasks were cheaper with the plugin; one was essentially flat. The biggest win, `f1006`, saw the plugin use **4 fewer turns and 15 fewer tool calls** than bare. Importantly, this isn't from the plugin's deterministic tools firing. In all three runs, `altimate-dbt` **was never invoked**. Claude used the same Read/Bash/Edit tools either way. The savings come from the skill descriptions in Claude's system prompt biasing exploration: plugin Claude reads fewer files, skips more dead-end debugging paths, converges to the answer with less noise. > *Loaded plugin context is a prior, not a tax. It guides exploration toward the answer, even when the bundled tools never fire.* This generalizes the `intercom003` task result: passive context bias improves both correctness *and* efficiency on real dbt work. --- ## **The decision framework** Here's a single chart you can use to decide, task by task, which mode of the plugin matters and what it'll cost: Decision framework chart for when each plugin mechanism pays off. Passive context bias is near-free and helps idiom-sensitive dbt SQL. Schema introspection adds parity cost and catches hallucinated columns on real warehouses. Cross-warehouse delegation has a premium cost but is the only path that finishes multi-DB workflows. Use this to predict where the plugin will pay off on *your* workloads. --- ## **Where the plugin doesn't help** It's not a free lunch everywhere. Two patterns to know about before you adopt: - **On the simplest model-creation tasks (**`airbnb005`**\-shape), plugin presence costs +25%** with no correctness gain. These are tasks bare Claude one-shots in a handful of turns; there's no exploration loop for the context bias to short-circuit. On a richer mix of dbt mediums (Blind spot #4) the aggregate flipped the other way to −18.6%, but the simplest end of the spectrum is a small tax. - **The Skill tool is advisory, not delegating.** Even forcing it to "fire" via system-prompt nudges doesn't translate to Claude actually executing altimate-code. We tested this. Don't over-invest in prompt tricks. Just install the plugin and let the passive context bias do the work, then *prompt invoke* altimate-code for cross-warehouse tasks. Two panels showing where the altimate plugin does not help. Trivial code generation pays a 25 percent token tax with no benefit. Forced skill consultation fires the skill tool but Claude reads guidance and does work directly, three of four times. The pattern is consistent: **the plugin shines on the work that's actually hard**. For trivial tasks, it's a small tax. For warehouse-shaped, multi-DB, or idiom-sensitive work, it's the difference between a working answer and no answer. **Recommendation:** Leave the plugin on by default. The +25% overhead on trivial tasks is small in absolute terms (these are cheap, short runs), and dynamically toggling introduces friction that costs more in engineering time than it saves in tokens. The aggregate across a realistic mixed workload still favors leaving it enabled. If you're running a batch of pure staging model scaffolding with no warehouse lookups, that's the one case worth disabling it — but it's the exception, not the workflow to optimize for. --- ## **How to adopt: — the 5-minute path** Four-step adoption path. Step one enable plugin in settings.json. Step two verify it loaded with a CLI check. Step three for cross-DB tasks explicitly demand altimate-code in the prompt. Step four pre-bake warehouse credentials once. **Step 1.** Add to `~/.claude/settings.json`: ```json { "enabledPlugins": { "data-engineering-skills": true } } ``` **Step 2.** Verify it loaded: ```bash claude --print "list available skills" | grep -i altimate ``` **Step 3.** For cross-warehouse and parity tasks, demand altimate-code explicitly in the prompt: ```plaintext Use altimate-code to drive the cross-DB diff ``` **Step 4.** Pre-bake your warehouse credentials in altimate-code's connection registry once. Saves re-doing it per task and avoids credential leakage into prompts. --- ## **The bottom line** The altimate plugin doesn't make Claude smarter at SQL. It makes Claude **competent at the parts of data engineering that aren't SQL**: schema introspection, cross-warehouse connectivity, dbt-idiomatic defaults. And on the dbt work it already could do, it does it with fewer tokens. For data teams running Claude on real warehouse work, that's the difference between a code generator and an agent that finishes the task. > *Bare Claude is cheap when it works. When it doesn't (when the warehouse can't be reached, or when the column names are wrong), the cost is undefined. And even when it does work, plugin Claude usually gets there cheaper.* --- **Try it.** [Altimate Data Engineering Skills Plugin for Claude Code](https://github.com/AltimateAI/data-engineering-skills) If you run it on your next migration or parity task, we'd love to hear what shape of work it helped, or didn't. --- *All three experiments used Claude Sonnet 4.6 uniformly. Sample size is small per task (1-2 runs per configuration); the task shapes are deliberately divergent, so treat the numbers as directional rather than precise.* Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [Data Engineering](https://altimate.ai/blog?tag=data-engineering) [AI](https://altimate.ai/blog?tag=ai) [llm](https://altimate.ai/blog?tag=llm) [AI Tools](https://altimate.ai/blog?tag=ai-tools) [SQL](https://altimate.ai/blog?tag=sql) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Where AI Agents Belong in Data Engineering URL: https://altimate.ai/blog/where-ai-agents-belong-in-data-engineering-the-correctness-layer Why a deterministic correctness layer is essential for production-grade pipelines: a practical blast-radius analysis example with Altimate Code. [Back to blog](https://altimate.ai/blog) Altimate · harnessengineering · Data Engineering ## Where AI Agents Belong in Data Engineering: The Correctness Layer Simon Späti Jun 29, 2026 Where AI Agents Belong in Data Engineering: The Correctness Layer With ever-changing models, new and better ones coming out every few months, it's great if we don't have to rely on them too heavily. The better your tooling, the less dependent you become on any single model. That's also why the deterministic harness matters: a correctness layer that lets you reproduce outputs and trace lineage regardless of which model you're running underneath. This is especially true during maintenance or extending the project, where verification is the real job. The danger isn't only a crash or an error message, but a wrong number that didn't break. It might be a clean query, but it introduces duplicated rows. In this article, we go through the three levels of AI agents in data engineering, how to structure projects so the AI delivers its best outcomes, and how dedicated agents with a deterministic core help us build higher-quality pipelines — ones we can actually trust. And we look at a practical example of how it works with a blast radius analysis. ## The Three Levels of AI Agents in Data Engineering Why should we use agents for data engineering? And at what levels can agents help us productively? As LLMs will always have some error tolerance, as humans do too, we need a way to be more confident in producing the code. ### Chat-phase, Autonomous and Dedicated Tooling There are different levels of confidence and levels on which the agents can help us. 1. The initial **chat-phase**: the development where we prompt Claude or ChatGPT. The model tries to understand the context based on what it has access to. It takes a decent amount of tokens, as it needs to scan everything from scratch. 2. The **autonomous approach**, where Claude Code or Codex also have access to the tools humans have, mostly the CLI on the terminal, making it possible to query Postgres with psql or read from S3 or Parquet with DuckDB to verify queries and data. A much higher quality outcome. 3. **Dedicated agents** for the task at hand. E.g., for data, the tools know dbt or know how to transpile SQL code deterministically, meaning not from training data only, but with an actual tool that does it much faster and more reliably. Built-in checks and features a "general" agent can't provide. Diagram of three levels of AI agents in data engineering: chat tools, autonomous agents such as Claude Code, and dedicated data agents, with confidence rising and token cost falling at each level *Showcasing the three levels of AI agents in data engineering* Ideally, we'd want to always use the dedicated tools, but there isn't always one. ### Where in the DE Lifecycle Each Level Actually Helps BI Dashboards vs. Plumbing the Data Pipelines, or Creating Source Ingestions, or Maintaining? For data engineering, the question is not only if there is dedicated agent tooling, but also on what part of the [data engineering lifecycle](https://www.oreilly.com/library/view/fundamentals-of-data/9781098108298/?ref=ssp.sh) AI agents can help data engineers and analysts the most, and potentially even domain experts? The lifecycle contains the ingestion part, ETL, or understanding the business in great detail, or is it just to visualize the result? Or should it cover maintenance in case of overnight ETL errors, or the full data lifecycle? In general, before we go into more details later, agents can help us on the full cycle, but it always depends on who you are and what role you play. Building from scratch with **no knowledge** or **seniority** is dangerous. Why? Because they can't verify if the produced code is correct. Okay for a side project or a proof of concept, but not for actual production. ### What's the Engineering Discipline for Working with AI? There's also a part that is less technical, a way of guiding the agents in the right direction. Especially if we want to safely use it in large projects or organizations, we can't just let it run without guidance. For that we need: 1. clear **project structure** in which the agents can flourish. The more is given, the fewer tokens are used for this work, and it will be more aligned across the project. (Another reason a deterministic workflow such as `uv init` is best, because it will always be the same). 2. build with clear **instructions** ([agentic skills](https://altimate.ai/skills), [superpowers](https://github.com/obra/superpowers), etc.) on how the tools are used (basically providing CLIs and API documentation). This is the bulk of the work anyway. That's the data architecture, the brainstorming with fellow humans before you build something, instead of missing a key insight in the beginning and then letting the agent run down the wrong path. Also, be realistic: prompting "be correct" or "use state-of-the-art" won't make it more correct or more state-of-the-art than the model was trained on. So if it's a rather new architecture, it's a must that you provide these links and hints. 3. **set up** the project in a modular fashion, so the agents cannot break the whole project if they make a small change, so you don't end up in a scenario like [dependency hell](https://xkcd.com/2347/) with everything dependent on each other. 4. use a **declarative approach**, with descriptive configuration that says the what and not the how, so that you can **collaborate** on these configs with the agents, version them, and easily revert or change something, as well as decouple the implementation logic from the actual business logic. With these steps, you can get the best out of the agents of today. I'd say the model matters less, but the structure does, and as Mario says, so does the workflow approach. For example, extensively plan (the process before writing a single line) and correct the model before any implementation that could lead down the wrong path is written. Also, don't overthink it. But this is only the workflow and learning the **soft skills and discipline of working with agents**. How does that look in a real-world project? > !Note: > > The key is to get use out of AI, not to get more work. E.g., most developers used to think about the problem. Today, most drown in PRs. When the AI tooling gets better, AI can provide more quality code that is correct, that needs less review or fewer iterations, which means fewer PRs and less work for the developers to go through. ## The Correctness Layer for Data Engineers A key insight is that AI agents should support the "human in the loop" for **correctness**, or a [correctness layer](https://altimate.ai/blog/the-correctness-layer-in-ade). And rather than making more work to verify more code, we should be confident in the process and know that the code it produces is verified and ultimately correct. But how do we get more "correct" work and a layer in which we can verify it? The biggest argument is a deterministic-validation architecture in full. E.g., [Altimate Code](https://github.com/AltimateAI/altimate-code) splits the agent into a probabilistic layer on top and a deterministic Rust/TS layer underneath that does the actual SQL ops such as parsing, validating, and equivalence checks, so that the agent itself never has to be trusted on those questions. Architecture diagram of Altimate Code: a probabilistic LLM agent on top, a deterministic TypeScript harness in the middle, and the altimate-core Rust library for SQL parsing, validation and lineage at the bottom *An example of how Altimate Code is built with its probabilistic agent, deterministic harness, and deterministic core | Image from the article* [*The Correctness Layer: Why Data Agents Need Determinism*](https://altimate.ai/blog/the-correctness-layer-in-ade) Altimate Code, for example, is built on a probabilistic agent, deterministic harness, and deterministic core. The **probabilistic agent** with the LLM does the creative work of reading intent, picking a strategy, drafting SQL, summarizing results, and recovering when something goes wrong. Below the boundary sits the **deterministic harness**, a TypeScript layer that intercepts every tool call: a dispatcher checks `hasNativeHandler` before the call runs, and routes it either to a native, deterministic handler or back to the model. Those handlers don't reimplement logic themselves, they call into the **deterministic core**, a Rust engine (`altimate-core`) that exposes SQL operations as pure functions over ASTs and schemas, wired in via napi-rs bindings. Parsing, validating, transpiling, checking query equivalence, diffing schemas, extracting column lineage, diffing rows across warehouses — all of it runs sub-millisecond, and all of it returns the same answer on the same input, every time. Like a compiler, the agent never *decides* whether two queries are equivalent or a column exists upstream. Instead, it calls a function that proves it against the parsed AST and the schema, the same way a type-checker proves a program compiles rather than guessing. Flow diagram comparing a bare agent, which guesses the column PATIENT\_ID and fails at 2 AM, with an agent behind a correctness layer, which resolves SUBJECT\_ID from the dbt manifest before merge *How the correctness layer adds additional verification* That's the distinction that makes the output easier to review, as factual checks have been run and the output is either correct, or there's a bug that it can fix directly. The rest a human can re-verify. On the dilemma of having stopped to hand-write code and approving it faster than humanly possible to check, you can also read more at [You Are the Trust Layer](https://altimate.ai/blog/you-are-the-trust-layer-managing-data-engineering-ai-agents-at-scale). > !Note: There's another factor: being wrong > > Bare agent use might be cheap, but only until they're wrong, and then the cost is unbounded. ### Improvements for Better Usage of Tokens Altimate, or data engineering agents that have deterministic functions and integrated understanding of how to work, can help you save tokens and be token lean (the opposite of [tokenmaxxing](https://en.wikipedia.org/wiki/Token_maxxing), which is popular on Twitter/X, using as many tokens as possible and having an agent running at all times). Because in large enterprises, token costs are a real budget point. To slow down the tokens, an easy trick is to instruct the model to use fewer tokens and words itself - [caveman](https://github.com/JuliusBrussee/caveman/) is a good example of that, but you can also add a singular prompt to your `CLAUDE.md`, Codex, or model of choice in combination with Altimate Code. Screenshot of Altimate Code tracing the cat column in spend\_categories through its dbt models, beside the Altimate Trace view of the same session showing token counts and tool calls *An example of Altimate Code showing a trace of data lineage and a web UI for it.* There's a second, less obvious cost: the token itself isn't a stable unit. When Anthropic shipped Opus 4.7, the same prompt that cost X tokens on 4.6 [started costing roughly 1.4X](https://altimate.ai/blog/the-great-token-heist-of-26) (same input, same answer, more tokens, same price per token). In [The Great Token Heist of '26](https://altimate.ai/blog/the-great-token-heist-of-26), the Altimate team makes the case that "*cost-per-token is the wrong number to optimize*", since the meter itself can move with a vendor's next model update, and what we should track instead is **cost-per-task**. I fully agree, and this is where deterministic function calls work around that volatility by not using a model/tokens for every task, making it less expensive. ## Typical Use Cases In this chapter we go through typical AI agent use cases for data engineering. There are many of them. You can use them to educate yourself or your team, build production data pipelines, build data apps, and visualize your data in new innovative ways (usually HTML web pages with React and other JavaScript frameworks). But in general, the use cases fit into these approaches: 1. **Start a new project from scratch example**: Building a data landscape with more open source. 2. **Extending an existing project or data warehouse**: Adding new data pipelines. 3. **Maintaining current setup**: Update and verify it still works when changes come in. 4. **Migration**: Migrate from one database or tooling to the next. 5. **Finding the Blind Spots**: Two similar-sounding IDs might be wrongly used for a join, or missing data in a column that got missed in a nightly load, or anything in between. If agents can do these checks, that would be super beneficial. With more access to CLI, Model Context Layer, and deterministic tooling, these things are truly possible. Below we go through extending and changing an existing warehouse with a change of column, and using Altimate Code to give us a Blast-radius assessment. ### Showcases: Blast-Radius Example A [Blast-radius](https://en.wikipedia.org/wiki/Blast_radius) refers to the **potential extent of damage**. For example, before you knock down a wall in your house, you want to know if there's plumbing behind it, electrical wiring within it, or if it's holding up the floor above. The same is true for a data warehouse or a data project with lots of ETL. For example, if a data engineer cleans up the table `fct_orders` by joining `orders` to `order_items` and summing `order_total`. It compiles, the dbt tests pass, nothing errors. But the join changes the grain, so any order with several line items now gets counted once per item, and revenue quietly inflates. It's best to know, before you **rename a column** or add a new join, the downstream (data that comes after the current task) dependencies to the dashboard — that's what the blast-radius report does. With Altimate Code we can achieve this. Before any change goes through, it maps out the full impact automatically and produces a detailed blast-radius report with what will break, what's safe, what needs someone to sign off, and also performs the changes. Here is what this looks like: #### Rename and Change Columns and Logic As an example, in this prepared [ecommerce repo](https://github.com/sspaeti/ecommerce_demos) with different DWH layers such as `staging -> intermediate -> marts`, I prompted this request to change unit from cent to dollars: Screenshot of the Altimate Code terminal with a prompt to rename shipping\_cents to shipping\_dollar in the stg\_order dbt model and list every model, test and metric that would break It recognized the dbt name and invoked `dbt-analyze` automatically: Screenshot of Altimate Code loading the dbt-analyze skill to run a blast radius analysis on the shipping\_cents rename It gave me a full Blast-radius report and the impact my changes would have on the project: Screenshot of the Altimate Code impact audit for the shipping\_cents rename, showing the lineage chain and four breaking changes by file and line Including semantics only, to point out what's safe and what's not: Screenshot of the Altimate Code impact audit listing one semantic-only change in sem\_orders.yml and two files that are safe to leave unchanged With a fixed order to address breaking changes, semantics and docs, and intentionally untouched: Screenshot of the recommended fix order from Altimate Code: six steps, ending with a rebuild of stg\_orders and its downstream models Notice, I hadn't said anything about blast analysis or using dbt-analyze. It did it on its own, ran dbt, and analyzed it deterministically. This shows how **Altimate Code looks behind the walls of data engineering**, just like blast radius analysis. If you want to see another example and a full blog post on Blast Radius, check out [Blast Radius Analysis Using Altimate Code](https://altimate.ai/blog/blast-radius-analysis-using-altimate-code), and what Altimate Code did as in the [video](https://www.youtube.com/watch?v=Npf7fHK43-k). Or Altimate provides many more examples and [Showcase](https://docs.altimate.sh/examples/) on their website such as [Migrate SQL Server to Snowflake with dbt](https://www.youtube.com/watch?v=7MtD0NJjZS4) or showing how to resolve [An Upstream Schema Changed](https://docs.altimate.sh/examples/#an-upstream-schema-changed-what-just-broke). > !Note: Connect a model to Altimate > > Make sure to connect to a model with `/connect` and choose an existing subscription with API credits, or any other subscription. I used [opencode zen](https://opencode.ai/zen) for my example, which includes e.g. Opus 4.8. ## Correctness Over Confidence I hope you got a better understanding of why AI agents can be genuinely useful, especially when provided with the right tools and applied with the right discipline. You've also seen how deterministic tooling, purpose-built for data engineering and analytics problems, gets you both better correctness and better token economics than general-purpose agents alone. Coming back to where we started: not every task needs a level-three agent. A quick chat-phase agent is fine for exploring a dataset or drafting a query you'll review yourself. But the moment that output touches production or serious work, a dashboard, a nightly job, a number someone makes a decision on, you want the deterministic core underneath it, not just a model that sounds confident. That's the gap [Altimate Code](https://github.com/AltimateAI/altimate-code) is built to close. It runs on deterministic functions purpose-built for DE workloads, it's open-source via the OpenCode TUI, and for teams wanting more, there's Altimate Studio — a paid, multi-agent platform with extras like warehouse cost optimization, dbt development acceleration, and migration tooling. --- Check out [Altimate Code](https://github.com/AltimateAI/altimate-code), it's free and open-source. Give them a star if you like them, and find more information on their [docs](https://help.altimate.ai/) and new [website](https://altimate.ai/). Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [Altimate](https://altimate.ai/blog?tag=altimate) [harnessengineering](https://altimate.ai/blog?tag=harnessengineering) [Data Engineering](https://altimate.ai/blog?tag=data-engineering) [AI](https://altimate.ai/blog?tag=ai) [llm](https://altimate.ai/blog?tag=llm) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # You Are the Trust Layer: Managing AI Agents at Scale URL: https://altimate.ai/blog/you-are-the-trust-layer-managing-data-engineering-ai-agents-at-scale AI writes the code, but you approve it. Learn why developers are now trust layers for AI agents—and why that's becoming unsustainable. [Back to blog](https://altimate.ai/blog) AI · Software Engineering · AI Tools ## You Are the Trust Layer: Why AI Code Review Is Breaking Down Anand Gupta Jun 15, 2026 You Are the Trust Layer: Why AI Code Review Is Breaking Down ## Your Job Just Became QA for an AI Pipeline You Didn't Design Sometime this year your job changed, and nobody sent the memo. You used to write code. Now you mostly approve it: you read a summary, it looks right, you hit accept, and you've moved on before you could have checked it even if you wanted to. The machine produces faster than any person can verify, and the distance between what it ships and what you actually confirmed grows every week. So what does that make you? Not a tool's user. You are managing a labor force whose work you never read, and you are the only check between its confident output and what reaches production. You are the trust layer. > The defining problem of this era isn't that AI can't do the work. It's that it does the work far faster than anyone can check it. ## I Instrumented My Own AI Usage Like a Data Pipeline I know because I measured it. I wrote a small proxy, pointed it at my own machine, and logged every request that crossed the wire for forty-eight hours. One developer, two days: **11,302 requests across 109 concurrent sessions and sub-agents**. The last change one of those agents handed me had taken it six minutes and something close to an afternoon of human work. **I looked at it for eleven seconds, then hit accept.** I read what surfaced and trusted the rest. So do you. > I Logged 11,302 AI Requests over two days. Only 37.5% Were Actually Correct. ## Sub-Agents Are Unmonitored Nodes in Your Pipeline The first problem with a trust layer made of one person is that it's uneven. How much gets caught depends entirely on who is holding the line. A senior catches a surprising amount by reflex: the function that's subtly wrong, the assumption that doesn't hold. A junior catches less, because spotting the bad answer is the same skill as writing the good one, and that skill is the thing they're still building. And the ten sub-agents your agent spawned that nobody opened? Those are checked by no one at all. Across a real team, the work is thin on checking, and thinnest exactly where no one is looking. ## Why AI Generation Outpaces QA Capacity The second problem is worse, because it doesn't care how good you are. Generation is parallel and nearly free: one prompt fans out into ten agents. Your attention is serial and fixed: you read one thing at a time, and there are only so many hours. So as the volume climbs, the fraction any human can actually check falls, and not gently. It heads for zero. The senior's reflexes don't beat this. They just start from a higher number on the way down. ## 96.9% Looked Clean. Only 37.5% Passed Validation. You might assume the part you don't check is mostly harmless, or you'd have noticed by now. I assumed that too, until I scored it. Almost everything the agents produced looked like a correct answer: clear, confident, well-formed. Barely a third of it actually was one, once I checked it against the code and the task in front of it. Looking right and being right turn out to be very different numbers, and the gap between them is exactly the part a quick skim waves through. Bar chart comparing AI agent outputs that looked correct (96.9%) against outputs that were actually correct (37.5%) across 47 production traces, with the gap between them labeled "what a glance waves through." ## Bad Lineage Happens One Accepted Diff at a Time And the work you don't check doesn't disappear. It ships, things get built on top of it, and (this is what turns a backlog into debt) the next agent reads it as context and repeats it as fact. A wrong assumption from Tuesday is settled truth by Friday, cited in code none of you wrote by hand. You took the debt on to move fast. It accrues interest quietly, and it comes due later: the incident, the rollback, the week spent tracing where the data went wrong. > The person clicking accept is almost never the person who pays. ## Data Debt Doesn't Show Up on the Dashboard Until It's Expensive That's why you, personally, feel fine. The cost is displaced. It lands in time, weeks after the commit that caused it, and it lands on other people: the junior who trusts the senior's pace without the senior's reflexes, the teammate who inherits the module, next quarter's version of you. It's also why the best engineer on the team is the most sure there's no problem. He's right about his own desk. He just isn't the one holding the bag. One developer can absorb this by hand for a while, which is the whole reason it stays invisible. An organization can't. Multiply it across everyone shipping this way and there is no trust layer left, only thousands of accepts a day, a rising balance of unverified work, and no single place that can tell you which of it was real. Today the trust layer is you: by reflex, in the seconds between accepting one diff and starting the next. That was never going to hold. The only open question left is whether you replace it on purpose, with verification that runs as fast as the work is produced, or keep paying the debt until it picks the moment to collect. --- *Numbers from a local proxy run against one machine over a 48-hour window: 11,302 requests, 109 concurrent sessions and sub-agents, and a 47-trace correctness sample in which 96.9% of output looked right and 37.5% was actually right. Measured, not modeled.* Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [AI](https://altimate.ai/blog?tag=ai) [Software Engineering](https://altimate.ai/blog?tag=software-engineering) [AI Tools](https://altimate.ai/blog?tag=ai-tools) [developer productivity](https://altimate.ai/blog?tag=developer-productivity) [llm](https://altimate.ai/blog?tag=llm) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # DeepSeek vs Claude Sonnet: AI Agent Cost, Speed & Accuracy URL: https://altimate.ai/blog/deepseek-vs-claude-sonnet-ai-agent-benchmark We ran 540 trials comparing DeepSeek v4 Pro vs Claude Sonnet 4.6 in a data agent. Cost was just the start — the trace tells a very different story. [Back to blog](https://altimate.ai/blog) AI · llm · Deepseek ## DeepSeek Cost 62% Less Than Claude. The Surprising Part Wasn't the Savings… Altimate.ai Team & Syed Haider & Surya Iyer Jun 10, 2026 DeepSeek Cost 62% Less Than Claude. The Surprising Part Wasn't the Savings… If you're running an AI agent on data (pipeline diagnostics, schema exploration, ad-hoc SQL, lineage queries), you've looked at the model bill and asked the same question we did: > *"The cheaper model on the leaderboard is only a few points behind the one I'm using. The endpoints are drop-in compatible. The agent code doesn't care which backbone is wired in. Why shouldn't I switch?"* That's a defensible question. It's also a question with a more interesting answer than the leaderboard suggests. We're going to walk through what we found when we ran the swap: Claude Sonnet 4.6 to DeepSeek v4 pro inside the same data agent, run across the same benchmark, with the same workspace, prompts, and tools. The accuracy difference is small and roughly what the leaderboard predicts. The behavioral difference is not. By the end of this post we'll have a concrete framework for thinking about LLM selection in an agent context that's a lot more useful than "*which model is on top of which leaderboard this week.*" Throughout, the only thing we change is the backbone model. \***Note:** We did not compare against Claude Opus series since Opus would intensify the cost gap (roughly 5× Sonnet's per-trial cost) without changing the model family. We plan more inter- and intra-model family comparisons soon. Two abstract traces side by side, Sonnet on left dense and chatty, DeepSeek on right denser in tool calls but quieter in text. *Same task — Beyoncé's "Get Me Bodied" on DAB’s music\_brainz dataset. The dots are tool calls. The gray bars are narration chunks. You can tell the models apart without looking at the model field.* ## **The setup, in two paragraphs** We use [altimate-code](https://github.com/altimateai/altimate-code), the open-source agent runtime behind our data engineering platform. It's the harness currently sitting at #1 on the [DataAgentBench (DAB)](https://ucbepic.github.io/DataAgentBench/)  leaderboard for Pass@1 stratified accuracy, scored at 0.6320 with GPT 5.5 and Claude Sonnet 4.6. DAB is UC Berkeley's data-agent benchmark: 54 queries across 12 real-world datasets (Postgres + MongoDB + SQLite + DuckDB), 5 trials per query, 270 trial runs per submission. Real schemas, real ambiguity, validators that check the exact shape of the answer. For this post we ran two complete Pass@5 passes through altimate-code: one with Claude Sonnet 4.6, one with DeepSeek v4 pro. 270 trials each. 540 trials total. Everything else (agent code, prompts, tools, validators, dataset hints, workspace setup) stayed identical. The result is a clean A/B with a single variable. That's what makes the contrast interesting. ## **The numbers that the leaderboard would show you** Here's how the two runs compare on the dimensions a leaderboard would surface: ```plaintext Sonnet 4.6 DeepSeek v4 proᵃ Stratified Pass@1 60.4% 56.9% Raw Pass@1 63.0% 60.0% Cost per trial $0.76 $0.29 Median duration / trial 4 min 9 minᵇ ᵃThis run has not been sumbitted to DAB, we're working on an improved approach. ᵇDeepSeek latency measured via OpenRouter without upstream-provider pinning; OpenRouter's routing across DeepSeek's underlying providers contributes to wall-clock variance. ``` - Accuracy gap: 3.5 stratified points (the benchmark metric). - Cost gap: 2.7× cheaper for DeepSeek. - Latency gap: 2.2× faster per trial for Sonnet. If you're running an overnight batch workload where wall-clock doesn't matter, those numbers point at DeepSeek. If you're running an interactive agent where users are waiting, they point at Sonnet. That's the surface-level read, and it's not wrong. It's also incomplete in ways that turn out to matter. ## **The trace tells a different story** We picked the model swap as the only variable so we could look at everything else. Inside an agent run, the score is one bit of information: the final ANSWER, did it pass or fail. The trace (every tool call, every text emission, every step boundary, with timestamps) is thousands of bits. The agent loop produces a `events.jsonl` file per trial, and once you start reading them side-by-side, the score gap stops being the interesting number. Here’s a snippet: Annotated events.jsonl trace snippet with callouts pointing to empty SQL result, LIKE retry, and final ANSWER write. The trace signature is what an operator stares at when something goes wrong in production at 3 AM. It's what you log, what you alert on, what you debug from. We found that the signature changes more when you swap the model than the accuracy does. By a lot. Here's what that looks like, concretely. ## **Finding 1: One model talks. The other doesn't.** Per trial: ```plaintext Sonnet 4.6 DeepSeek v4 pro Chat-text events (mean) 13 10 Characters of narration (mean) 5.7 k 1.7 k Tools called — mean 36 36 Tool calls — median 32 36 Tool calls — min 0 0 Tool calls — max 82 75 ``` Left: horizontal narration bars, Sonnet 5700 vs DeepSeek 1700 characters. Right: token-mix annotation showing cache for Sonnet, reasoning tokens for DeepSeek. Both models handled the same workload with the same tool volume. Sonnet writes about 3× more chat text along the way. It narrates as it works ("Let me check the schema for the orders table" / "I'll filter for active customers first" / "Trying a different join condition"). DeepSeek does its planning silently and only surfaces what it has to. This shows up in the cost structure too. Sonnet writes 49 k tokens into Anthropic's prompt cache per trial and reads 1.22 M back from it; reasoning tokens are zero. DeepSeek writes nothing to a prompt cache, but burns 8 k reasoning tokens per trial doing chain-of-thought inline. The downstream consequences are concrete: - If you're tailing logs from an agent in production, Sonnet sounds like a thinking colleague. DeepSeek sounds like a quiet one. - If you're building token-cost alerts, they'll be calibrated for one and miss the other. Sonnet's bill is input tokens + prompt cache. DeepSeek's is reasoning tokens, with no cache line at all. So a Sonnet-tuned alert ("input spend spike", "cache hit-rate drop") sits silent on DeepSeek while reasoning-token cost you're not watching adds up. - If your platform's observability is built on chat-text logging, swapping to DeepSeek will quietly cut your visibility by two-thirds. None of this affects the accuracy column on the leaderboard. All of it affects your team building on top of these models. ## **Finding 2: One model commits early. The other doesn't always commit.** altimate-code reads the agent's final answer from a file called ANSWER. Every `write` tool call against that file is a timestamped event. We plotted the first ANSWER write per trial as a fraction of trial elapsed: Horizontal histogram showing when the first ANSWER write lands as a percentage of trial elapsed. Sonnet peaks in the 60–80 percent bin with 130 trials. DeepSeek peaks in the 80–100 percent bin with 125 trials. The 'never' row is the operationally important one: 32 trials for Sonnet, 79 for DeepSeek. Sonnet's median first commit lands around 70% of the way through a trial. DeepSeek's lands around 90%. And the "never" row is the operationally important one: **79 of 270 DeepSeek trials never wrote anything to ANSWER at all.** For Sonnet, that number is 32. These are two strategies, both legitimate. Sonnet commits early and refines: writes a first draft mid-trial, then rewrites it as confidence grows (mean: 1.47 ANSWER writes per trial). DeepSeek explores until forced to commit, then commits once (mean: 0.88 writes per trial). Why DeepSeek explores-then-commits is partly architectural. Its reasoning models emit reasoning\_content and content on two parallel channels ([DeepSeek API docs](https://api-docs.deepseek.com/guides/reasoning_model) ). The agent loop consumes content; reasoning\_content is inspection-only and must be stripped before resubmission (the API 400s otherwise). DeepSeek plans privately in the reasoning channel and bursts the plan into content as tool calls + ANSWER in one transition. Sonnet has no such split: its chain-of-thought shares the channel the loop consumes, so drafts and retractions stream out as visible turns. Channel architecture: DeepSeek has reasoning\_content private channel and content public channel with a burst arrow; Sonnet has a single content channel carrying drafts and final answer together. If you're building anything where users see partial progress (a dashboard, a streaming UI, an agent that hands off intermediate results), Sonnet gives you a draft to show by minute two. DeepSeek leaves you with an empty file for eight minutes, and almost a third of the time, leaves it empty forever. ## **Finding 3: One model asks the database questions. The other writes scripts.** Look at the tool-call distribution across all 270 trials per model: Grouped bar chart of mean tool calls per trial for five tools, comparing Sonnet and DeepSeek. The agent and tool surface were identical; the tool preferences were not. Sonnet falls back to bash 45% more often than DeepSeek. It runs Python scripts for filtering, aggregating, formatting. When in doubt, it writes code. DeepSeek prefers the native data tools the harness exposes. It inspects schemas 2.7× more often. It runs SQL through `sql_execute` 64% more often. When in doubt, it asks the database. The discovery cadence diverges before the first SQL even runs. Mean schema-related calls before the first SQL execution: 2.8 for Sonnet, 4.9 for DeepSeek. DeepSeek does a column-by-column tour before writing any SQL. Sonnet reads the prompt, takes one schema snapshot, and gets to work. Both rhythms produce correct answers on plenty of queries. They get there with different tool budgets and different turn structures. > If your platform logs the shape of tool calls in production, you can identify which backbone is running without looking at the model field. ## **Finding 4: When they fail, they fail differently** This is the finding we found most operationally consequential. Every failed trial fits one of four shapes: ```plaintext Sonnet 4.6 DeepSeek v4 pro Empty (nothing written) 7 (3%) 27 (10%) Short wrong (single value) 43 (16%) 26 (10%) Medium wrong (small table) 30 (11%) 33 (12%) Long wrong (multi-row) 20 (7%) 22 (8%) Total failures 100 108 ``` Failure shapes grouped bar chart: Sonnet skews toward short-wrong, DeepSeek skews toward empty. Same totals, opposite shapes. Sonnet's failures cluster on *short wrong*: confident commits to a wrong value. DeepSeek's cluster on *empty*: the agent worked, explored, iterated, and never wrote anything. The total failure count is the same; the shapes are opposite. Both shapes have architectural roots. DeepSeek's reasoning API can return HTTP 200 with `reasoning_content` populated but `content` empty, meaning the model reasons through to a conclusion without ever generating output text. The failure mode is tracked in [DeepSeek-R1 issue #314](https://github.com/deepseek-ai/DeepSeek-R1/issues/314): "the status code returned is 200 (indicating a successful request in the HTTP protocol sense), the actual content of the response is empty." The issue was closed stale without a fix. Sonnet has the opposite problem for the opposite reason: with no separate reasoning channel, its planning happens in the same `content` stream the loop consumes. Drafts surface as soon as the model has a candidate, and once a candidate is in ANSWER there's no separate "verify-before-commit" gate to slow it down. The model *satisfices*: it picks the first plausible answer, polishes the prose around it, and moves on. That's how “short wrong” gets produced: a confident commit on partial evidence, with the verification step folded into the same channel that wrote the answer in the first place. Here's a concrete example. DAB’s “GitHub Repos” query 3 asks for the count of Shell-language commits in Apache-2.0 repos under a length filter. The ground truth is `1077`. ```plaintext Sonnet: "963 114" ← committed close-but-wrong on turn 8 DeepSeek: "1077" ← landed correctly after exhaustive schema inspection ``` Two swim lanes for GitHub Repos q3. Top lane is Sonnet with 21 tool calls and an early wrong commit at turn 8. Bottom lane is DeepSeek with 40 tool calls and a correct commit at turn 40. DeepSeek did 40 tool calls, including 4 `schema_inspect` calls and 12 `sql_execute` iterations. It got there because it kept asking. Sonnet did 21 tools, jumped to bash at turn 8 with a count that was nearly right, and never noticed it was wrong. DeepSeek won this query 4 out of 5 trials. Sonnet won 0. For an operator, the two failure modes have different downstream costs: - **Vocal-wrong** (Sonnet's signature) gives you a bad answer that may pass type checks and propagate to consumers. The downstream system sees a number and assumes it's right. - **Silent-stuck** (DeepSeek's signature) gives you nothing: a missing row that is easy to detect with a presence check but hard to debug without the trace. Which is worse depends on what your downstream consumers do. If you have validators downstream that can re-check correctness, silent-stuck is more recoverable. If you don't, vocal-wrong is the silent killer. ## **When the signature disappears** There are times, however, when models do *not* fail differently. When a question is clean (the schema is obvious, the format is pinned down, the dataset hint spells out the join recipe), Sonnet and DeepSeek behave almost identically. We watched this on a question from DAB's `bookreview` corpus (query 3) that asks for a multi-row table from columns that store Python-dict syntax as text. Both models passed it. Both took exactly 27 tool calls. The first eight tools in each trace were nearly identical: read question, read format\_hint, list warehouses, index schema, inspect a table, write the SQL. The same effect shows up when the agent has good orientation upstream of the main loop. We run a pre-orientation pass where a lighter model walks the schema, samples rows, identifies non-obvious column formats, and proves out join candidates with mechanical probes. The output gets dropped into the agent's workspace as a single markdown file. When that file is well-formed, you can't tell the models apart from the trace. The trace signature shows up at the *edges* of the agent's capability, where it has to decide for itself when to commit, how aggressively to explore, what to do when stuck. So before you assume model selection matters for your workload, look at your queries. If most of them are clean (well-documented schemas, unambiguous prompts, validators that accept reasonable variants), the model choice may genuinely not move your needle. Pick on cost. If your real queries push the agent into open-ended decisions (which is what most production data-engineering work actually does), the trace signature is what you should be evaluating. ## **How we choose now** A short field guide, based on what we measured: **Pick Sonnet 4.6 when:** - You need progress visibility mid-run. For dashboards, streaming UIs, and partial-trace debugging, you need incremental output. Sonnet writes a draft answer at the 70% mark and improves it. DeepSeek doesn't write anything for 90% of the trial, and a third of the time, never writes anything. - Latency matters per trial. Sonnet finishes in ~4 minutes median; DeepSeek in ~9. For interactive use, the gap is felt. - Silent failures are expensive downstream. Vocal-wrong is easier to detect than silent-stuck. If your pipelines don't double-check the agent's output, you want it to commit (so you can catch wrong answers) rather than disappear (so you don't even know it failed). **Pick DeepSeek v4 pro when:** - Cost matters more than wall-clock. DeepSeek runs at 38% of Sonnet's per-trial cost for similar accuracy on this benchmark, making the math straightforward for overnight batch, retroactive backfills, and async report generation. - The agent needs to explore. DeepSeek's tendency to inspect the schema thoroughly before writing SQL is the right behavior when the schema is unfamiliar and the question demands precision. - You have downstream validators that catch silent-stuck. If your platform notices missing outputs and can retry, DeepSeek's caution costs you less than Sonnet's overconfidence might. **Run both when:** - You're doing consensus or majority voting. The two backbones disagree on 19% of trials in our run. That's a real ensemble margin, not noise. A judge model can pick the right answer often enough to lift the combined score above either alone. ***Case in point:*** *In this comparison, Sonnet and DeepSeek disagreed on 51 trials. A perfect oracle judge over those would lift ensemble Pass@1 by ~9-10pp; a realistic LLM judge by ~3-5pp. Full consensus math and a real judge run is a follow-up.* We use all of the above in production. [altimate-code](https://altimate.ai/products/altimate-code) is built to make this choice cheap: any backbone, swappable per workload, same trace format regardless. The selection logic is yours. The decision is yours. ## **A footnote on substrates** This post is the operational complement to an argument we made earlier in [*The Correctness Layer in ADE*](https://altimate.ai/blog/the-correctness-layer-in-ade). That one argues that for the parts of data engineering that have deterministic ground truth (two queries are equivalent, a lineage edge exists, a row-level diff is correct), the LLM is the wrong substrate. The substrate should be deterministic code. The argument here is the complement: even when you do leave a task to the LLM, *which* LLM you pick changes the operator experience. The substrate is still the story. The LLM is just the part of the substrate that varies the most when you swap providers. ## **If you want to look yourself** Everything in this post is reproducible. [altimate-code](https://altimate.sh) is [open source](https://github.com/AltimateAI/altimate-code). DataAgentBench is open. You can obtain the per-trial `events.jsonl` files which preserve every tool call, every step boundary, every token count, every chat emission. The two runs we compared above are from our own `benchmark_runs` collection; the analysis script that produced the numbers is in the repo. If your team is making a similar swap, the same kind of diff is one fork and a few hours of compute away. ## Conclusion The leaderboard tells you what a model can do on a benchmark. The trace tells you what your agent will look like in production. We built [altimate-code](https://altimate.sh) to win on both. That's why we sit at [#1 on DataAgentBench](https://web.archive.org/web/20260608100121/https://ucbepic.github.io/DataAgentBench/) with Sonnet, and why the same runtime runs DeepSeek at 38% of the cost just as cleanly. The tools, prompts, and trace format are identical across both. The model becomes a variable per workload, not a quarterly commitment. The runtime carries its share too. Tools emit a compact, structured payload to the model and a richer, human-readable one to the UI, so the agent's context window isn't burned on formatting it'll never read. Long sessions get compacted instead of truncated. Token budgets stretch further on every backbone you point at it. Pick the model that fits the workload. Pick the runtime that doesn't lock you to one. Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [AI](https://altimate.ai/blog?tag=ai) [llm](https://altimate.ai/blog?tag=llm) [Deepseek](https://altimate.ai/blog?tag=deepseek) [Data Engineering](https://altimate.ai/blog?tag=data-engineering) [AI Tools](https://altimate.ai/blog?tag=ai-tools) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # LLM Token Inflation: Why Your AI Bill Keeps Growing URL: https://altimate.ai/blog/the-great-token-heist-of-26 Token counts aren't standard across AI vendors. Learn how tokenizer changes, hidden fees, and currency gaps inflate your real LLM costs. [Back to blog](https://altimate.ai/blog) Altimate · AI · Tokenomics ## The Great Token Heist of ‘26 Altimate.ai Team & Surya Iyer & Syed Haider Jun 1, 2026 The Great Token Heist of ‘26 Your LLM provider charges by the token, but a token is not a portable unit of work, a unit of meaning, or any unit you can compare across vendors. It only has meaning inside one vendor's system and loses it the moment you step outside. The token is a currency: vendors mint it, vendors re-denominate it, and the exchange rate with English (meaning how many tokens it takes to express the same sentence) is theirs to set. So when two vendors quote the same dollars-per-token, they are still quoting in different currencies. On April 17, 2026, the day after Anthropic shipped Claude Opus 4.7, a Hacker News user named *anabranch* posted a public leaderboard they had built at [tokens.billchambers.me](https://tokens.billchambers.me) with crowdsourced before-and-after token counts on the same prompts run through Opus 4.6 and Opus 4.7. The thread title, ["Opus 4.7 to 4.6 Inflation is ~45%"](https://news.ycombinator.com/item?id=47816960), climbed to 117 points before lunch. anabranch's summary: *"It can be more than 2x for small prompts."* The comments filled with developers reporting specific impacts on their bills. *hgoel*, on the Claude Max 5x plan, [hit their five-hour usage limit in under two](https://news.ycombinator.com/item?id=47817737). *tiffanyh* [exceeded the weekly limit after seven prompts on a single-file HTML/CSS/JS project under 300 lines](https://news.ycombinator.com/item?id=47817627). *KellyCriterion*: ["killed my weekly limit with just three prompts."](https://news.ycombinator.com/item?id=47817801) *manmal* put the number on it: ["The same prompt costs roughly 30% more now, for input."](https://news.ycombinator.com/item?id=47817534) None of this was a price change. [Opus 4.7 launched at the same $5 per million input tokens and $25 per million output as 4.6.](https://www.anthropic.com/news/claude-opus-4-7) Independent measurements followed within days. Simon Willison upgraded his Claude Token Counter to compare the two models on the same input and [posted](https://simonwillison.net/2026/apr/20/claude-token-counts/): *"Opus 4.7 does appear to use 1.46x times the tokens for text"*, with up to 3× for images, priced the same as Opus 4.6 on a per-token basis. [OpenRouter ran the numbers across more than a million real customer requests](https://openrouter.ai/announcements/opus-47-tokenizer-analysis) and published a bucketed breakdown: 32–45% more native tokens for equivalent text (highest on prompts under 2,000 tokens), with a 12–27% increase in net cost after prompt caching absorbed some of the inflation. What changed was not the price per token. It was the token itself. The same prompt that cost you X tokens on 4.6 now costs roughly 1.4X on 4.7: same input, same answer, same work. The vendor changed the resolution. If that sounds like a camera going from 12 to 18 megapixels, Anthropic would accept that comparison: more resolution, better model, same price per unit. But a megapixel upgrade is a one-time capability you choose. This tokenizer change bills you per pixel on every shot, cannot be switched off, and shipped with no announcement. And more megapixels never guaranteed a better photo, only a bigger file. More tokens do not guarantee a better answer. They guarantee a bigger bill. ## The currency got re-denominated Buried in Anthropic's [migration guide](https://platform.claude.com/docs/en/about-claude/models/whats-new-claude-4-7) is one easy-to-miss sentence: Opus 4.7 uses a new tokenizer that *"may use roughly 1x to 1.35x as many tokens"* as previous models, varying by content. The price per token did not move; the number of tokens required to express the same English sentence did. The honest defense is that this is an upgrade: a finer tokenizer can mean better resolution and better answers, and you simply pay in more tokens for them. But "better answers" is a measurable claim. If the quality is real, it shows up as a higher success rate and the cost per finished task still holds. If it does not, you are paying more tokens for the same result. The catch is that they shipped the cost with no announcement and no opt-out, while leaving the "better outcomes" half for you to verify on your own bill. If this happened in a fiat currency, we would call it a redenomination: the kind of operation central banks use when they need to recalibrate without admitting they are recalibrating. Your salary stays "the same" in the new currency, but the new currency buys 30% less coffee. Anthropic, of course, does not have the obligations of a central bank, so there is no announcement, no press conference, and no notice period. The tokenizer is part of the model. It changes when the model changes. The Hacker News reactions were not just complaints about bills. They were the predictable response to opacity. *Shailendra\_S*, a bootstrapped founder, [argued that 45% inflation breaks the unit economics for most indie products](https://news.ycombinator.com/item?id=47817588) and pushes builders toward open models, or back to the more token-efficient Opus 4.5. *dakiol* [announced his team had dropped Claude entirely](https://news.ycombinator.com/item?id=47817610). *Boris Cherny from the Claude Code team* [showed up in the thread](https://news.ycombinator.com/item?id=47817526), but to address a separate complaint about over-fixated safety warnings, not the cost question. The cost question is hard to address. The currency really did get smaller, and the vendor can keep calling it "the same price." *"Nobody raised prices. The currency got smaller."* ## There is no standard token Anthropic's redenomination is not unusual. It is the rule, dressed up as an exception. Tokens are made by tokenizers, which are deterministic functions that split text into integer IDs. Each vendor trains its own. OpenAI's [tiktoken](https://github.com/openai/tiktoken) is open-source: you can download the vocabulary files and count locally before you pay. GPT-3.5 and GPT-4 use [`cl100k_base`](https://github.com/openai/tiktoken), with 100,256 entries. GPT-4o, GPT-5, and the o-series use `o200k_base`, with 200,019. The transition from one to the other in 2024 was not priced as a change. It was, of course, a change. The pattern repeats across vendors. [Llama 2 to Llama 3 jumped from a 32K to a 128K vocabulary](https://huggingface.co/blog/llama3), and Meta says that buys up to 15% fewer tokens for the same text. [Mistral's Tekken tokenizer](https://mistral.ai/news/mistral-nemo), introduced for NeMo and Pixtral, compresses code about 30% more efficiently than its SentencePiece predecessor, and Korean and Arabic by 2–3×. [Gemini's tokenizer](https://docs.rs/gemini-tokenizer), currently a 262,144-token SentencePiece model shared with Gemma 3, has rotated through versions across Gemini 1.5, 2.x, and 3.x. DeepSeek's [V3 paper](https://arxiv.org/abs/2412.19437) documents a 100K → 128K vocabulary expansion from V2. The vendor with the most aggressive policy of opacity is Anthropic. The legacy `claude-v1-tokenization.json` was published for Claude 1 and never again. For Claude 3, 3.5, 4, and 4.7, the only authoritative way to count tokens is to call Anthropic's [`count_tokens` endpoint](https://platform.claude.com/docs/en/build-with-claude/token-counting). The repository `javirandor/anthropic-tokenizer` is the community workaround, reverse-engineered by asking Claude to repeat text and observing what comes back. It is approximate, and for many teams it is also the only way to estimate Claude cost without a round-trip to Anthropic's servers. You can quote two vendors at the same dollar-per-million-token. You cannot quote them in the same units. ## The AI exchange rate is not the same for everyone If tokens are a currency, English speakers got a significantly better deal on it. Petrov, La Malfa, Torr, and Bibi published ["Language Model Tokenizers Introduce Unfairness Between Languages"](https://arxiv.org/abs/2305.15425) at NeurIPS 2023. They [evaluated 17 tokenizers across 200+ languages](https://aleksandarpetrov.github.io/tokenization-fairness/) on FLORES-200, the standard multilingual parallel corpus. The single most cited number from that paper: the same translated sentence can produce *"differences up to 15 times"* in token count across languages, and the disparity persists even on tokenizers explicitly designed for multilingual support. [Yennie Jun's parallel study](https://www.artfish.ai/p/all-languages-are-not-created-tokenized) across 52 languages and 2,033 messages quantified the everyday cost. English averaged 7 tokens per message. Spanish, French, and Portuguese came in at similar rates. Hindi and Bengali ran approximately 5× English. Armenian ran 9×. Burmese reached 72 tokens, roughly 10× English for the same sentence. Her summary: *"some languages require up to 10 times more tokens."* Newer tokenizers narrow the gap without closing it. Tamil's நீ ("you") [was four tokens in `cl100k_base` and is a single token in `o200k_base`](https://www.njkumar.com/gpt-o-multilingual-token-compression/). Gemini's larger SentencePiece vocabulary handles CJK (Chinese, Japanese, Korean) noticeably better than the GPT-4 generation did. But the underlying training distribution is still English-dominant, and the structural penalty has not been engineered away. What this means in practice: if your customer base is Hindi-speaking and your model uses an English-trained tokenizer, you are paying roughly four to five times what an equivalent English workload costs you, for content of equivalent meaning. The vendor did not charge you more per token. The exchange rate did the charging. ## The AI token fees you cannot see on the menu Once you accept that the token is the currency and the rate is the vendor's, the rest of the LLM bill becomes simultaneously more legible and stranger. **Reasoning tokens.** Newer models think before they answer, and the thinking counts. OpenAI's o-series, [DeepSeek's V4](https://api-docs.deepseek.com/news/news260424) in its thinking mode, Anthropic's extended-thinking models (Opus 4.7 by default now runs its `xhigh` effort tier in Claude Code), and Gemini's thinking modes all generate internal chain-of-thought that gets billed at the output rate. [OpenAI exposes a count of reasoning tokens](https://developers.openai.com/api/docs/guides/reasoning) in `output_tokens_details.reasoning_tokens`, though the actual chain-of-thought is summarized or hidden. [Anthropic in Opus 4.7 omits thinking content from the response by default](https://platform.claude.com/docs/en/about-claude/models/whats-new-claude-4-7), but the count is billed whether or not you request a summary. A 10,000-token reasoning budget on [GPT-5.5 at $30/M output](https://openai.com/index/introducing-gpt-5-5/) costs thirty cents before the user sees a word. Most teams running translation or classification through reasoning models are paying many times what they should because nobody turned thinking off. **Caching, which sometimes is not a discount.** OpenAI applies cache discounts automatically, up to 90% off on cached input, with no write premium and no code changes required. Anthropic uses explicit [`cache_control` markers](https://platform.claude.com/docs/en/build-with-claude/prompt-caching): reads are 10% of base input (the deepest discount on the market), but writes cost 1.25× base on a five-minute TTL or 2× base on a one-hour TTL. [Anthropic's own pricing page](https://platform.claude.com/docs/en/about-claude/pricing) says the math amortizes after one cache hit on the five-minute tier, or two on the hour-long one. But if your prefix, the stable front of the prompt you mark for caching, gets written and rarely re-read, caching costs you more than not caching. Gemini layers in a [per-hour storage fee](https://ai.google.dev/gemini-api/docs/pricing) ($4.50 per million tokens cached per hour on 3.1 Pro), and the cache-read rate also doubles above 200K context, so the long-context cliff applies to caching too. **Context cliffs.** Most vendors charge a flat per-token rate across their full context window. A few do not. Gemini 3.1 Pro is $2 in / $12 out per MTok up to 200K, then $4 in / $18 out above. [OpenAI's GPT-5.5 jumps to $10 in / $45 out above 272K input tokens](https://openai.com/api/pricing/). Anthropic's Opus 4.6 and 4.7 are flat across their 1M window, with one caveat: there is a 1.1× surcharge if you request US-only inference routing. A RAG pipeline whose prompt-length distribution flirts with these thresholds can change cheapest-vendor depending on the day's typical input length. **Batch.** This is the one place the price card stops misrepresenting your actual cost. [Google's batch API is half the standard rate](https://ai.google.dev/gemini-api/docs/batch-api); OpenAI and Anthropic match. If your workload tolerates async latency, batch is the single most impactful switch you can make, more than swapping vendors, and usually more than switching models within a vendor. The number of production pipelines that are eligible for batch and not using it is alarming. ## What the LLM price card actually tells you Pull all of this together and the dollar-per-million-token figure on a vendor's pricing page tells you exactly one thing: the dollar cost of one million units of that vendor's specific currency at this moment for the specific model variant and inference region and effort tier and TTL choice you picked. It does not tell you the cost of doing your work. The translation from "currency" to "work" sits inside the tokenizer, the language mix of your content, your reasoning-token policy, your cache hit pattern, your batch share, and your prompt-length distribution against context cliffs. The honest unit for cross-vendor comparison is not cost-per-token. It is cost-per-task: what you pay, end-to-end, for a unit of output your users would accept. The formula is: *cost\_per\_task = (price\_per\_token × tokens\_per\_request × (1 + reasoning\_overhead) × (1 − cache\_hit\_rate × cache\_discount) × (1 − batch\_share × 0.5)) ÷ task\_success\_rate* Every term is measurable. Token count comes from each vendor's official counter. Reasoning overhead comes from the usage block on each response. Cache hit rate and batch share come from your own logs. Task success rate comes from your eval. The headline $/MTok is one term on the right side, not the answer to the equation. Cost is not the only thing that moves. More tokens means a longer sequence, which brings more latency, more KV-cache memory, and a higher chance of crossing the 200K/272K context cliffs in ways that can change how an agent loop behaves. Tokenizer inflation lands in your latency and memory budgets, not just the bill. ## The state of the major LLM currencies, May 2026 Here is what each vendor is actually doing under their published rate card, in the dimensions that matter for real bills: | **Vendor** | **Can you count tokens locally?** | **How stable is the tokenizer across versions?** | **Caching** | **Long-context cliff** | **Batch** | | --- | --- | --- | --- | --- | --- | | **OpenAI** | Yes: tiktoken is open | Changed once: GPT-4 → GPT-4o (cl100k → o200k) | Automatic, ≤90% off, no write premium | Flat to 1M on GPT-4.1; 272K cliff on GPT-5.5 (≈2× input, 1.5× output) | 50% off | | **Anthropic** | No: only via count\_tokens API | Changed 4.6 → 4.7: 1.0–1.35× per Anthropic, 1.46× measured by Willison | Explicit cache\_control; 0.10× reads, 1.25–2× write premium | Flat to 1M; 1.1× surcharge on US-only inference | 50% off | | **Google Gemini** | Partial: Gemma 3 SentencePiece (262,144 vocab) is published | Differs across 1.5 / 2.x / 3.x lines | Explicit + per-hour storage fee; cache reads also double above 200K | 2× input, 1.5× output above 200K on 3.1 Pro | 50% off | | **DeepSeek** | Yes: BBPE (byte-level BPE) published | V2 → V3 expanded 100K → 128K; V4 added dual thinking/non-thinking modes | Automatic disk cache, no write premium | Flat | Off-peak discount | | **Meta Llama** | Yes: tokenizer.json ships with the model | Llama 2 → 3: 32K → 128,256 | Host-dependent (Bedrock, Together, Groq differ) | Host-dependent | Host-dependent | | **Mistral** | Yes | SentencePiece → Tekken (NeMo, Pixtral) | Host-dependent | Host-dependent | Host-dependent | The vendors split into two groups. Open-tokenizer providers (OpenAI, DeepSeek, Meta, Mistral) let you compute cost client-side before you send the request, which is useful for budgeting, useful for routing, and useful when you are trying to A/B against another model. Closed-tokenizer providers (Anthropic, Gemini in part) require a round-trip to find out what the request will cost. That asymmetry is itself a feature of the pricing, even though it does not appear on any pricing page. ## Auditing your own AI currency exposure The practical version of this post is a worksheet, not a checklist. Sit with your team and answer: 1. **What does your token actually buy?** Take 50–100 real prompts from production logs. Run each through every candidate vendor's official token counter. Compare the counts. A 20% gap on your real content is normal; a 50% gap on non-English content is normal. Now multiply each count by each vendor's $/MTok and look at the result. This is the real per-call cost, not the price card cost. 2. **How much are you paying to think?** For every reasoning-capable model you use, pull the reasoning-token count from the usage block. Multiply by output price. If a non-trivial fraction of your bill is thinking tokens on workloads that do not need them, such as translation, classification, extraction, or short Q&A, turn thinking off. This is almost always the largest single optimization available. 3. **Does caching cost you or save you?** Pull the actual ratio of cache writes to cache reads from your logs. On Anthropic, if your prefix gets written more often than it gets read, the write premium is making things worse, not better. On Gemini, work out whether the per-hour storage fee on rarely-hit caches is worth the discount on the hits. 4. **Where in your prompt-length distribution do the cliffs sit?** Histogram your input lengths against 200K (Gemini) and 272K (GPT-5.5). If the 90th percentile is anywhere near those numbers, you are paying tier-2 rates more often than you realize. 5. **What fraction of your traffic could be batch?** If 30% of your work is async-tolerant and you are not batching it, you are leaving ~15% of your total bill on the table. 6. **What's your real success rate per dollar?** A cheaper-per-token model that fails your eval 20% more often is more expensive in production. Build the cost-per-accepted-task number. Reorder vendor preferences accordingly. ## Why this state of affairs persists It is tempting to read all of the above as vendors being deliberately opaque. Mostly, they are not. The Opus 4.6 → 4.7 tokenizer change was, on the merits, the right engineering trade for downstream quality. Anthropic's cache write premium reflects real storage and warm-cache costs. Gemini's 200K cliff is honestly disclosed. OpenAI's automatic caching is genuinely user-friendly. Every individual decision makes sense on its own. What is missing is a market norm requiring vendors to publish their tokenizers, announce changes, and standardize the comparison unit. Until that norm exists, and there is no commercial incentive for any vendor to ship it unilaterally, the dollar-per-million-token figure on a pricing page will continue to be the most quoted and least informative number in the industry. The fix lives on the buyer side. Track cost-per-task, not cost-per-token. Tokenize your real prompts before you migrate. Route by workload, not by vendor. Audit reasoning overhead like you audit any other line item. Treat the price card as one input to a model of your true unit economics, not the model itself. And when a thread titled "Opus 4.7 to 4.6 Inflation is ~45%" climbs the front page of Hacker News on the day after a model launch, check the tokenizer before you check the price card. Pixel-art illustration of a masked figure with hands raised being arrested by a police officer beside a spilled bag of tokens, outside a neon-lit arcade called Tokens Arcade, evoking a heist over token pricing. --- *— Written from the Altimate team. We build [*Altimate-Code*](https://altimate.ai/), an agentic data engineering platform for enterprises. If you're working through this kind of vendor economics for your own stack, we'd like to compare notes.* Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [Altimate](https://altimate.ai/blog?tag=altimate) [AI](https://altimate.ai/blog?tag=ai) [Tokenomics](https://altimate.ai/blog?tag=tokenomics) [openai](https://altimate.ai/blog?tag=openai) [Dev Tools](https://altimate.ai/blog?tag=dev-tools) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # The Correctness Layer: Why Data Agents Need Determinism URL: https://altimate.ai/blog/the-correctness-layer-in-ade How Altimate's data agent tops benchmarks by keeping LLMs out of SQL validation, lineage, and diffs with a deterministic Rust correctness layer. [Back to blog](https://altimate.ai/blog) AI · Data Engineering · Altimate ## The Correctness Layer Altimate.ai Team & Syed Haider & Surya Iyer May 20, 2026 The Correctness Layer We started [altimate-code](https://altimate.ai/products/altimate-code) with one goal: build a harness that data engineers could actually trust with their pipelines and platforms. The first thing that became clear is that LLMs are the wrong tool for half of that job. Two queries either are or are not semantically equivalent. A lineage edge either does or does not exist. A row-level diff is either correct or not. None of those are creative questions, and a probability distribution is the wrong answer to any of them. Over the past year, we have built a three-layer architecture that puts the LLM in charge of the work it is good at (strategy, intent-parsing, code generation) and a deterministic Rust and TypeScript stack underneath for everything that has to be reproducible. The split (detailed later) is what gets us to the top of the ADE and the [DAB](https://ucbepic.github.io/DataAgentBench/) benchmarks, and more importantly, what makes the result repeatable on someone else's hardware. This post walks through how the split actually works: where the boundary sits, the three operations we moved out of the model entirely, and what thousands of structured benchmark traces taught us about where agents actually fail. ## LLMs are probability distributions Run the same prompt through the same model twice, and you'll get two different answers. That's not a bug. It's the substrate. Language models sample from a probability distribution every time they generate a token, and the distribution is shaped — sometimes broadly — by sampling temperature\[1\], prompt-cache state, and the internal nondeterminism of large batched inference on shared hardware\[2\]. For creative work, this is exactly what you want. Drafting a SQL query, summarizing a result, recovering from an unexpected error — these are open-ended tasks with many acceptable answers, and probabilistic output is what makes the model useful. For correctness, it's a liability. "Are these two queries semantically equivalent?" has one right answer. "What columns does this output project from this source?" has one right answer. "Are these two tables byte-equal?" has one right answer. When a probabilistic system answers a deterministic question, you can't cache the answer (it might be different next time), you can't debug it (a single sample tells you nothing about the distribution), and you can't benchmark it (a single pass-rate is a coin flip away from a different number). ## What this looks like on a real benchmark We took one task off the [ADE benchmark](https://github.com/dbt-labs/ade-bench), a dbt refactor called `asana004`, and ran it through altimate-code three times with the same model, the same prompt, and the same starting state. The agent passed it once and failed it twice. Across the three runs, dbt's test counts came back as 6/6, then 4/6, then 5/6. If you are building a data-engineering agent, that is the problem you have to design around. You cannot trust a system whose output is intrinsically random. You cannot cache it. You cannot reproduce it. You cannot debug it. And you cannot reliably benchmark it. ## The comfortable go-to (but wrong) answer The instinct is to make the model itself more deterministic. Lower the temperature. Tighten the prompts. Introduce multi-agent voting. Run test-time consensus across N samples. Vendors in the data space are responding to this demand, and their responses make sense on some level. Temperature=0 is easy (provides marginal repeatability at the cost of much of the agent's ability to explore or recover from mistakes), ensemble voting is well-understood, and prompt engineering keeps getting better. This works, partially. Lower temperature reduces variance but does not eliminate it. Multi-agent voting trades dollars for variance reduction but never reaches zero. None of it changes the fundamental shape of the problem. The model is a probability distribution, and you are sampling from it. ## So stop fixing the model The LLM should stay out of the correctness layer. That is the bet behind altimate-code. The LLM stays in charge of strategy, intent-parsing, and code generation, which are the genuinely creative parts of the work. Everything else runs in a deterministic Rust and TypeScript layer underneath: SQL validation, schema diffing, query equivalence, lineage extraction, cross-database join inference, data-parity diffing. The agent does not decide whether two queries are equivalent. It calls a function that proves they are. I keep coming back to this framing because it is the only one I have seen that survives contact with a real benchmark. Once you have watched the same prompt produce three different answers, you stop wanting the model to be more reliable. You start wanting it to be less involved. > *Once you have watched the same prompt produce three different answers, you stop wanting the model to be more reliable. You start wanting it to be less involved.* ## Three layers altimate-code's architecture has three layers. The decision that matters most is where the boundary between them sits. Diagram of altimate-code's three-layer architecture: a probabilistic LLM agent (intent, strategy, codegen, summarize, recovery) sits above a deterministic TypeScript harness (dispatcher, registry, skills, helpers, hasNativeHandler, join inference), which calls via napi-rs bindings into a deterministic Rust core (altimate-core) running 34 SQL operations across 4,600+ tests and 34 dialects, validating 1,000 queries in 30ms. The **deterministic core** is a Rust library (`altimate-core`) that exposes 34 SQL operations as pure functions over abstract syntax trees and schemas. It parses, validates, transpiles, fingerprints, checks equivalence, diffs schemas, classifies PII, extracts column lineage, and diffs rows across warehouses. Every operation runs sub-millisecond. Our benchmark battery validates 1,000 SQL queries in 30 ms and lints 1,000 queries in 250 ms. All of it is reproducible. All of it is zero-cost at the LLM API layer because it never touches the model. The engine carries 5,700+ Rust tests, runs on a single sqlparser-based AST representation, and targets 34 SQL dialects, from Postgres and Snowflake to Trino and TSQL. The **deterministic harness** is the TypeScript code that runs the agent. A dispatcher maintains a registry of native handlers. When the agent calls `altimate_core.transpile`, the dispatcher routes the call to compiled Rust via napi-rs bindings, not to the model. Skills (encoded playbooks) prescribe step orderings. Helpers like dialect-aware identifier quoting prevent whole classes of "the model generated almost-correct SQL" failures. The **probabilistic agent** is the LLM. It reads task descriptions, plans, picks tools, writes drafts, summarizes results, and recovers from failures. It sits in the creative layer, not the correctness layer. The agent does not always know which layer it is calling into. But it always can know. A `hasNativeHandler` check classifies a tool call as deterministic or not before it runs. That asymmetry is most of the value. Consider what happens when the agent calls `altimate_core.track_lineage`. The tool wrapper packages the arguments. The dispatcher looks up the handler registered for that method name. The handler normalizes the agent-supplied schema, accepting either the flat `{table: {col: TYPE}}` shape or the nested `SchemaDefinition` shape, and converts both to a canonical form before passing through. The napi-rs binding calls into compiled Rust. The Rust engine builds the lineage graph and returns it. The tool formats the output for the agent. From the agent's perspective, it called a tool and got back lineage. From the architecture's perspective, every step is deterministic, including the schema normalization, which the harness does, not the model. Sequence diagram of a tool call crossing the agent/harness boundary in seven steps: the LLM agent calls a tool, a TypeScript wrapper packages the arguments, a dispatcher looks up the handler, the handler normalizes the schema, hasNativeHandler() routes it to compiled Rust, a lineage engine builds the graph, and the formatted result returns to the agent, captioned "the agent does not always know which layer it is calling into, but it always can know." ## What this looks like in practice I will walk through three concrete operations, because adjectives do not earn the claim. Take two queries written differently but doing the same thing: ```sql -- Writer A SELECT id FROM users WHERE status = 'active' AND created > NOW() - INTERVAL 30 DAY -- Writer B SELECT id FROM users WHERE created > DATE_SUB(NOW(), INTERVAL 30 DAY) AND status = 'active' ``` `altimate_core.checkEquivalence` proves these are semantically identical without running either of them. The comparison happens on parsed ASTs against the provided schema. It handles predicate reordering, equivalent date-arithmetic functions, and identical column projection. The result is a boolean plus a confidence score, not a sample of an LLM's opinion. Downstream, that means refactor verification, migration safety checks, and regression tests that confirm a query's behavior did not change. None of it requires a live database or a model that "looks at" the queries. Dialect translation is the same story. `transpile(sql, fromDialect, toDialect)` takes a Snowflake query and produces a BigQuery one (or BigQuery to Postgres, or Postgres to Databricks) via `sqlparser-rs` and dialect-specific AST transforms. It runs sub-millisecond. The agent never has to memorize that Snowflake's `IFF(cond, a, b)` becomes `CASE WHEN cond THEN a ELSE b END` in ANSI dialects. We have measured the LLM-direct alternative. The failure mode is not "wrong output." It is almost-right output that compiles in dev and breaks in prod when an edge-case function shows up. Determinism here is not about speed. It is about coverage. The AST transform handles every case the parser handles, not every case the model happens to have seen in training. `schemaFingerprint(schema)` returns a SHA-256 hash of a serialized schema. That is the entire interface. The hash is what makes caching trustable. When the agent is validating many queries against a large schema, we compute the fingerprint once and use it as a cache key. Identical schemas produce identical hashes, and the cache hits. An LLM-derived "is this schema the same?" cannot offer that guarantee. A model might say "yes" twice and "no" the third time, and you would have to debug whether the third answer was the wrong one. The fingerprint also acts as the index for progressive context compression. The core exposes five disclosure levels: fingerprint-only, table list, column names (the default), full detail for relevant tables only, and the complete schema - and the agent picks the level it needs for the question at hand. The selection is deterministic, the cache key is deterministic, and nothing about either step is the LLM's opinion of "is this schema close enough?" Reliability and performance turn out to be the same lever pulled twice. Determinism is what makes the cache safe. The cache is what makes the agent fast. You cannot get the second without the first. ## Case study: cross-database join inference The cleanest illustration of this principle is a tool we shipped in late April 2026. [altimate-code](https://github.com/AltimateAI/altimate-code) is often connected to several warehouses at once (Snowflake, BigQuery, Postgres), and the agent needs to suggest cross-database joins. One warehouse calls an entity `customer_42`. Another calls it `cust_42`, or `c-42`, or `42`. Naming and formatting disagree, so naive column-name matching fails. A pure-LLM approach to this is straightforward to describe and unreliable to ship. You prompt the model with the schemas and ask "which columns join across these DBs?" It works on the cases the model has seen. It hallucinates plausible-but-wrong matches on the cases it has not. Different runs return different rankings. Four-step algorithm for matching join keys across databases without hallucination: sample about 50 string values per column from Snowflake and Postgres, compute the longest common prefix, strip the prefixes and intersect the remaining suffixes, then rank by overlap size, ending in a worked example matching Snowflake's businessid\_42 to Postgres's businessref\_42 as a candidate with score 1.0. The deterministic version is about 400 lines of TypeScript. The algorithm is small enough to summarize: 1. From each connection, sample ~50 string values per column. 2. Compute the longest common prefix per column, trimmed back to the last `_`, `-`, or `:` separator. So `["businessid_42", "businessid_43", …]` produces prefix `"businessid_"`. Sequences without a separator are rejected. 3. For each pair of columns from different databases, require distinct non-empty prefixes (so the prefix actually distinguishes the two columns), strip the prefixes, and compute the set intersection of suffixes. 4. Emit a candidate when the suffix overlap is non-empty. Rank by overlap size, then by `overlap / min(left, right)`. `businessid_42` on one DB and `businessref_42` on another both reduce to suffix `42`. The algorithm pairs them with score 1.0 and reports the rule as "shared suffix after stripping distinct prefixes." The function that computes the common prefix is fewer than 30 lines. It walks character-by-character and trims back to the last separator. Identical inputs produce identical outputs, every time, in any order, on any machine. We can ship the algorithm to a customer and tell them exactly what will happen. ## Case study: column lineage with depth tiering Lineage is the question of where each output column comes from. Given a query with joins, CTEs, and expressions like `SUM(o.amount + o.tax)`, you need to know which upstream columns flow into each output. An LLM can answer this question in plain English. It cannot answer it precisely enough to chain operations on. Agents that act on lineage need exact edges, not summaries. `altimate_core.column_lineage` does this as an AST walk. Each output column gets a list of source columns plus a transformation lens: direct projection, function call, expression, or aggregation. The walk is deterministic by construction. Subqueries and CTEs are inlined. Window functions decompose into their partition and order clauses. CASE expressions report all branches as sources. The piece worth featuring is the *depth tiering*. The same engine exposes three modes: - **Basic**: direct source-target edges only, with minimal context. - **Deep**: adds transformation lenses (expression, function, aggregation), confidence scores, and the internals of every CTE the lineage flows through. - **Full**: transitive closure across multi-query chains, plus indirect dependencies from `WHERE`, `JOIN`, and `GROUP BY` predicates that affect row selection. The agent picks the depth it needs for the question at hand. The classification of what counts as "basic" versus "deep" is encoded. It is not an LLM call disguised as one. The pattern repeats: every level of detail an agent can ask for is a level the engine knows how to produce, and the choice between them is a parameter, not a guess. The same principle controls the output format. Lineage, like every operation in altimate-core, ships with two formatters: a verbose one for humans, and a compact one optimized for an agent's context window. Compact is the default. The agent gets a headline token (`LINEAGE | 5 cols, 3 tables, depth=basic`) followed by structured `column <- source` rows it can parse without spending tokens on prose. Depth chooses how much detail; the format chooses how that detail is encoded; both compound across a trajectory of hundreds of tool calls. ## Case study: data parity A different shape of problem: prod was migrated to a new warehouse, and we need to know whether the rows match. The naive answers are bad. You write a 200-line custom SQL diff query and hope you covered the edge cases, or you ask an LLM to "compare these two tables" and get an unactionable summary. Diagram of the HashDiff algorithm bisecting two 10-million-row tables across Snowflake and BigQuery by checksum, recursively splitting any row range whose SHA-256 checksums disagree through levels L0 to L5 until it pinpoints the single row where the payload hash differs. The engine implements two complementary algorithms. **HashDiff** uses bisection plus checksums for cross-database diffs. It partitions both tables, checksums each segment, recursively bisects any segment whose checksums disagree, and terminates at the row level on exactly the segments that contain mismatches. The bisection lets it pinpoint the exact rows that differ instead of scanning every row, and the module ships with 303 dedicated tests covering every dialect pair we support. **JoinDiff** uses a same-database `FULL OUTER JOIN` for when both tables live in the same warehouse. The cost characteristics differ. The correctness guarantee is the same. The engine ships with a cooperative state-machine API. It emits the SQL it wants executed, the caller runs it, and the engine steps forward based on the result. There is no point in the loop where the engine guesses, or where its behavior depends on which LLM is in front. Given identical inputs, the diffs come out identical, every time. "*Are these two tables byte-equal?*" is the antithesis of a creative task. ## Smaller wins, in the harness So far the story has been about the Rust core. The TypeScript harness gets the same treatment, and the wins are smaller per-instance but they compound. A single function called `quoteIdentForDialect` holds the truth about identifier quoting across our supported warehouses. MySQL gets backticks, T-SQL gets brackets, and everything else gets ANSI double quotes. The LLM never has to remember which is which, because it never builds quoted SQL directly. It passes column names through tools that route through `quoteIdentForDialect`. That is not a clever optimization. It is a category of failure removed from the system. The same pattern applies to `EXPLAIN`\-plan retrieval. Postgres has `EXPLAIN (ANALYZE, BUFFERS)`. Snowflake has `EXPLAIN USING TEXT`. BigQuery has no `EXPLAIN` statement at all, and you read plans from job metadata. A ~50-line decision table maps each warehouse to its plan-retrieval mechanism. The agent does not ask the model "what is the BigQuery EXPLAIN syntax?" because a tool already knows. Skills are altimate-specific procedures encoded in markdown. The `dbt-develop` skill prescribes a four-step workflow: plan (check the layer and naming conventions), discover (use schema-search tools, read existing YAML), write (generate SQL with validated joins), and validate (call `altimate_core_validate`, then `column_lineage`, then `dbt build`). The skill does not replace the LLM. It constrains where the LLM is creative and where the harness is prescriptive. Our trace data shows skill-driven sessions complete in fewer turns and with fewer tool errors than ad-hoc sessions on the same task. None of these are individually dramatic. Cumulatively they are the difference between an agent that almost works and one you can deploy. ## What thousands of session traces taught us We ran the ADE-Bench DuckDB benchmark against altimate-code and captured structured traces for every session. Our benchmark topping score was 74.4%.\[2\] The traces are where the interesting story lives. Failure has three distinct causes, and they require three different responses. The first cause is stochastic LLM noise. Two airbnb tasks (`airbnb005`, `airbnb006`) failed the original baseline but then passed all three reruns. `asana004` (the task from the opening) passed once out of three. The inputs are the same, the outputs differ, and the cause is intrinsic to the model. There is nothing to fix here except the model. The second cause is deterministic capability gaps. `asana003` failed at exactly 16 of 17 dbt tests, every single time across reruns. Same failure mode every time, with the same dbt-test count. The agent's output has a stable bug we can investigate and fix. The third cause is bench-fixture bugs. Three airbnb tasks (`airbnb001`, `airbnb002`, `airbnb008`) failed reproducibly, but the root cause was a pre-existing bug in the bench's own dbt incremental models: a broken `is_incremental()` filter that collapses the date series to a single row, so the `LAG` window function in the MoM/WoW aggregation returns `NULL` on rerun and overwrites the correct first-run values. This is not an altimate-code problem. The maintainers shipped a test-setup workaround in dbt-labs/ade-bench#106, but the model SQL itself is still broken — fix proposed in [dbt-labs/ade-bench#145](https://github.com/dbt-labs/ade-bench/pull/145). From the outside, all three look identical. The test failed. Only the deterministic substrate (structured traces, reproducible reruns, exact dbt-test counts) lets us separate them. Without that, we would be guessing about what is actually wrong, and our improvement work would be aimed at the wrong target. ## Where the market is Most coding agents in the field are general-purpose. Cursor, Cline, Claude Code, Codex, and GitHub Copilot Workspace all sit in that category. They are built for the generic developer-with-a-text-editor case. When a data engineer uses them to write a Snowflake query or refactor a dbt model, the tools treat SQL like any other code. Text goes in, text comes out, and the model decides. There is no deterministic substrate for the data-specific operations because there does not need to be one for general coding. The data-native AI products (dbt Cloud's AI features, in-warehouse LLMs like Snowflake Cortex) are closer to the right surface area, but they are still primarily LLM wrappers with a different UI. They generate SQL. They do not prove SQL. Our bet is that for data engineering specifically, the LLM-as-creative-layer / code-as-correctness-layer split is the right architecture, and that the deterministic infrastructure is where the actual moat lives. Models will get better. APIs will get cheaper. Prompt techniques will improve. None of that changes the calculus on whether two queries are semantically equivalent, or whether a cross-warehouse data diff is correct. Those are facts about code and data, decidable by code that operates on code and data. altimate-code currently sits at number one on the [ADE benchmark](https://altimate.ai/benchmarks), ahead of Claude Code, Cortex Code CLI, and dbt Labs, by a wide margin. It also recently scored #1 on the [Berkeley EPIC Data Lab DAB](https://ucbepic.github.io/DataAgentBench/) (Data Agent Benchmark) ## What is still LLM territory It would be dishonest to claim we replaced the LLM. We did not, and we will not. The LLM still owns intent-parsing ("refactor the `asana__project` model into an intermediate one that…"), code generation, description-writing for dbt model YAML, and strategy under uncertainty. When a build fails for reasons that are not immediately classifiable, the model is what figures out the next move. ## What's next ***Comparative trace analysis across models***. Sonnet, Opus, DeepSeek v4 pro, Kimi K2, and Qwen 2.5 Coder all run through the same altimate-code stack on the same benchmarks. What we don't yet have is a systematic comparison of how they differ on data-engineering tasks — which failure modes are model-specific vs architecture-specific, which models pair best with which skills, where the cost-quality frontier sits per task category. The traces are captured; we’ll share the analysis soon. ***Harness resilience***. The same trace data surfaces a handful of harness-side next moves. We are making the agent loop resilient to truncated model outputs from streaming providers. We are adding a per-generation timeout so a hung request does not tie up a worker for 25 minutes. We are detecting when the agent has fallen into a same-tool-same-argument loop and forcing a replan. Each one is a place where the harness can be more deterministic about how it handles a probabilistic model. ***Cost / quality frontier mapping***. Our DeepSeek v4 pro run on DAB scored 0.5693 stratified Pass@1 for $9 of inference, ahead of the then benchmark leader Pi's Opus 4.6 submission — is one data point in a larger frontier. We’re in touch with DAB maintainers to add this run to the leaderboard. The natural follow-up is systematic: across the model lineup we already run through altimate-code, at what cost does each model fall below a quality threshold, broken out by task category (SQL refactor, dialect translation, lineage extraction, cross-DB joins)? The architecture is model-agnostic; the right backbone for a given customer workload may be category-dependent. ## Conclusion The claim is narrower than "no LLM." It is that the LLM stays out of correctness. The probabilistic layer handles probabilistic work. The deterministic layer handles deterministic work. The boundary is where the payoff lives. Probabilistic models are a remarkable substrate for the things they are good at: language, intent, creativity. They are a poor substrate for proof, equivalence, and reproducibility. The architectural question for any agent in this space is not "can we use the LLM for X?" It is *"should we, and if not, what should we use instead?"* For the data-engineering work altimate-code does, the answer is the correctness layer underneath: a deterministic core in Rust, a deterministic harness in TypeScript, with the LLM kept exactly where it belongs. \[1\]: [Claude API Docs](https://platform.claude.com/docs/en/api/messages/create#create.temperature) \[2\]: [Defeating Nondeterminism in LLM Inference](https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/) *altimate-code is an open source project:* [*github.com/AltimateAI/altimate-code*](https://github.com/AltimateAI/altimate-code) *.* *If you are building in this space and want to compare notes, reach out by email at* info-at-altimate.ai\*.\* Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [AI](https://altimate.ai/blog?tag=ai) [Data Engineering](https://altimate.ai/blog?tag=data-engineering) [Altimate](https://altimate.ai/blog?tag=altimate) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Blast Radius Analysis for dbt with Altimate Code URL: https://altimate.ai/blog/blast-radius-analysis-using-altimate-code Altimate Code maps the full blast radius of a dbt column rename, catching breaking changes across SQL, YAML, and semantic layers before you push. [Back to blog](https://altimate.ai/blog) altimate-code · Data pipelines · dbt ## Blast Radius Analysis Using Altimate Code Muhammad Anas Farooqui May 18, 2026 Blast Radius Analysis Using Altimate Code ## What is Blast Radius Blast radius refers to the potential extent of damage. For example, before you knock down a wall in your house, you want to know if there's plumbing behind it, electrical wiring within it, or if it's holding up the floor above. Same idea with data. Before you rename a column, change a formula, or update a filter, you need to know what depends on it. That could include downstream models, dashboards, reports, or other workflows. --- ## Need for Blast Radius Analysis Often when you rename a column and push it to production, everything seems fine. Two days later, finance pings you on Slack: the weekly revenue dashboard has been broken since Tuesday. You dig in. The column you renamed was referenced in three other models. One of those models feeds a Tableau workbook. That field is the denominator in the revenue metric on the CFO's dashboard. Nobody documented any of this. The dependencies existed in the relationships between systems, invisible until something broke. This is the blast radius problem. [Altimate Code](https://docs.altimate.sh/getting-started/) fixes this. Before any change goes through, it maps out the full impact automatically. It produces a detailed blast radius report (what will break, what's safe, what needs someone to sign off) and also performs the changes. I ran a workflow using Altimate Code to verify this. --- ## The workflow: renaming `total_cents` to `gross_revenue_cents` I ran this on my open-source [CartWave E-commerce dbt project](https://github.com/altimateanas/ecommerce_demos), a real analytics codebase with staging models, intermediate transformations, mart tables, and a semantic layer on Snowflake. `stg_orders` is a simple staging model that reads raw order data and explicitly picks the columns that are needed. The finance team wants us to rename `total_cents` to `gross_revenue_cents` to align with GAAP terminology. ### The prompt **I gave Altimate Code this prompt:** > *"Rename the column total\_cents to gross\_revenue\_cents in stg\_orders as the finance team wants to align with GAAP terminologies. Before performing the changes, show me the blast radius of the same."* Altimate Code is an agentic harness for data engineering. *One column, one rename. It should have been straightforward, but it wasn't.* --- ### What Altimate Code did After receiving the prompt, Altimate Code ran through a series of steps before changing a single line of code. Here's what happened, in order. [YouTube video player](https://www.youtube.com/embed/Npf7fHK43-k) --- **Step 1: It loaded the** `dbt-analyze` **skill and scanned the project.** Altimate Code started by tracing the dependencies. It read the model SQL, searched every file in the project for references to `stg_orders`, and checked YAML config files for column-level references. Within seconds it found five files that depend on `stg_orders`: Terminal output listing six files requiring changes after a column rename, including staging, intermediate, and semantic layer files, with a note that downstream mart models are safe because they reference a derived column name. | # | File | What it is | | --- | --- | --- | | 1 | `int_order_enriched.sql` | Combines order data with customer info | | 2 | `int_customer_lifetime.sql` | Calculates lifetime value per customer | | 3 | `mart_return_analysis.sql` | Report table for analyzing returns | | 4 | `sem_orders.yml` | Defines business metrics like "total revenue" | | 5 | `_staging.yml` | Documents what each column means | Five files depend on this model. But how many of them actually use the column we're renaming? That's the question that matters. --- **Step 2: It checked each file for column-level references to** `total_cents`**.** Altimate Code checked more than whether a file depended on `stg_orders`. It opened each downstream file and looked for direct references to `total_cents`. Here's what it found: Summary table classifying the blast radius of a column rename into four categories: 4 breaking models, 2 cascading-but-safe mart models, 1 safe model, and 1 YAML doc update. `int_order_enriched.sql` – **BREAKING** Directly selects `orders.total_cents` on line 31: ```sql orders.total_cents, -- directly uses the column we're renaming ``` **Explanation:** This file asks for a column called `total_cents`. After the rename, that column won't exist anymore. The query will fail. `int_customer_lifetime.sql` – **BREAKING** Sums `total_cents` across all orders on line 22: ```sql sum(total_cents) as lifetime_revenue_cents, -- adds up all order values ``` **Explanation:** Same problem. The column disappears, the math breaks. `mart_return_analysis.sql` – **SAFE** This one depends on `stg_orders` but only uses `customer_id` and `order_date`: ```sql orders.customer_id, -- doesn't touch total_cents orders.order_date, -- doesn't touch total_cents ``` **Explanation:** It never references the renamed column. No changes needed. `sem_orders.yml` **(semantic layer)** – **BREAKING** Two metric definitions reference `total_cents` by name in their formulas: ```yaml measures: -name: total_revenue_cents agg: sum expr: total_cents # references the column by name -name: avg_order_value_cents agg: average expr: total_cents # references the column by name ``` **Explanation:** This is the one that would have bitten me. The semantic layer is a YAML config file, not SQL. It doesn't show up if you're just grepping through `.sql` files. But if you rename the column and forget to update these metric definitions, every dashboard using "total revenue" or "average order value" breaks silently. --- **Step 3: It traced the impact further downstream.** Altimate Code didn't stop at the first layer. It followed the chain: if a breaking model feeds another model, that downstream model could be affected too. Blast radius report for renaming total\_cents to gross\_revenue\_cents in stg\_orders, showing the changed model's dependency tree with each downstream model marked BREAKING, CASCADING, or SAFE. Two mart tables sit downstream of `int_customer_lifetime`: `mart_customer_360` (full customer profiles, contains PII) and `mart_customer_cohorts` (customer segmentation). But neither one references `total_cents` directly. They use a derived column called `lifetime_revenue_cents`, which `int_customer_lifetime` produces from `total_cents` and then passes downstream under its own name. Think of it like fixing a pipe in the basement. The faucets upstairs keep working as long as the pipe gets repaired properly. These two marts don't need code changes, they just need `int_customer_lifetime` to be fixed first. One thing worth noting: `mart_customer_360` is tagged `compliance: 'CCPA / GDPR'` and `contains_pii: true`. It doesn't need code changes here, but in a governed environment, you still want to know that a compliance-tagged table sits in the blast radius. --- **Step 4: It presented the full blast radius report.** Altimate Code assembled everything into a single view: Detailed breakdown table listing each of the eight affected files, its type, impact classification (rename, breaking, safe, or cascading), and the specific line and code change involved. | # | File | What it is | Impact | Details | | --- | --- | --- | --- | --- | | 1 | `stg_orders.sql` | The model being changed | RENAME | `total_cents` becomes `gross_revenue_cents` | | 2 | `_staging.yml` | Column documentation | RENAME | Update column name and description | | 3 | `int_order_enriched.sql` | Order enrichment model | BREAKING | Directly references `orders.total_cents` | | 4 | `int_customer_lifetime.sql` | Customer lifetime value | BREAKING | Uses `sum(total_cents)` in a calculation | | 5 | `mart_return_analysis.sql` | Return analysis report | SAFE | Uses `stg_orders` but never touches `total_cents` | | 6 | `mart_customer_360.sql` | Customer profile (PII) | CASCADING | Safe once `int_customer_lifetime` is fixed | | 7 | `mart_customer_cohorts.sql` | Customer segmentation | CASCADING | Safe once `int_customer_lifetime` is fixed | | 8 | `sem_orders.yml` | Business metric definitions | BREAKING | Two metrics reference `total_cents` by name | **The bottom line: Five files need changes. Three are safe.** Then the agent asked me: > *"I've identified 6 files that need changes across 3 layers (staging, intermediate, semantic). The 2 mart models downstream are safe because they reference derived column names. Shall I proceed with the rename?"* Interactive prompt showing Altimate Code asking whether to proceed with the rename across all six affected files, with options to proceed, stop at the analysis, or type a custom answer. It didn't just run ahead. It stopped, showed me everything, and waited for the go-ahead. --- **Step 5: I approved. It applied all changes at once.** After I said yes, Altimate Code updated all five files in a single pass. No partial updates, no window where half the models use the old name and half use the new one. Chat transcript showing the user approving Altimate Code's proposed rename, followed by a to-do checklist tracking each file update and the final build-and-verify step. `stg_orders.sql` – aliased the source column: ```sql total_cents as gross_revenue_cents, -- raw data keeps actual source column name, downstream sees new name ``` The raw data still has a column called `total_cents`. Instead of changing the source, the staging model gives it a new name. Everything downstream sees `gross_revenue_cents` from here on. `int_order_enriched.sql` – updated the reference: ```sql orders.gross_revenue_cents, -- was: orders.total_cents ``` `int_customer_lifetime.sql` – updated the aggregation: ```sql sum(gross_revenue_cents) as lifetime_revenue_cents, -- was: sum(total_cents) ``` `sem_orders.yml` – updated both metric definitions: ```yaml -name: total_revenue_cents description:"Sum of gross revenue in cents (GAAP-aligned)" expr: gross_revenue_cents # was: total_cents -name: avg_order_value_cents description:"Average order value in cents (GAAP-aligned)" expr: gross_revenue_cents # was: total_cents ``` `_staging.yml` – updated the documentation: ```yaml -name: gross_revenue_cents description:"Gross revenue in cents, aligned with GAAP terminology" ``` --- **Step 6: It built the entire project to confirm nothing broke.** Altimate Code ran `altimate-dbt build` across all thirty-four models: Terminal log of a full dbt project build after the rename, ending in "Full build: PASS=36, WARN=0, ERROR=0 — all green!" before the agent verifies column lineage. Thirty-six passed. Zero errors. Every staging view, intermediate table, and mart, including the PII-tagged `mart_customer_360`, built cleanly. --- **Step 7: It verified the column lineage end to end.** As a final check, Altimate Code traced the data flow from source to destination to confirm the rename propagated correctly: Summary confirming the rename is complete, column-level lineage is verified end to end, and listing the five files changed with their exact before-and-after edits. The source column `TOTAL_CENTS` flows through the alias in staging, propagates correctly to both intermediate models, and the derived column names downstream (`lifetime_revenue_cents`) stay the same. Exactly what the blast radius report predicted. --- ## Column Changes Are Never Just Column Changes A simple rename turned out to touch five files across three layers plus the semantic layer. Without the blast radius check, I would have updated `stg_orders`, run the build, and moved on. The semantic layer would have broken silently. Finance would have found out before I did. --- ## How to read a blast radius report Every downstream asset gets one of four labels: | Label | What it means | What to do | | --- | --- | --- | | **BREAKING** | Directly uses the thing being changed. Will fail. | Update before or alongside the change | | **CASCADING** | Depends on a breaking file, but doesn't use the changed column directly | Verify it works after the upstream fix. Usually no code changes. | | **SAFE** | Depends on the changed model but never uses the specific column | Nothing | | **UNKNOWN** | Can't determine impact (dynamic SQL, runtime-generated column names) | A human needs to look at it | The report also flags compliance tags. In our case, `mart_customer_360` carries `compliance: 'CCPA / GDPR'` and `contains_pii: true`. It didn't need code changes here, but those flags make sure the right people know a compliance-tagged asset is in the blast radius. --- ## Try it out yourself: **I ran this on a real project. To reproduce it:** 1. Clone the CartWave demo project: [https://github.com/altimateanas/ecommerce\_demos.git](https://github.com/altimateanas/ecommerce_demos.git) 2. Set up your database connection 3. Launch [Altimate Code](https://docs.altimate.sh/getting-started/) 4. Give Altimate Code the prompt "*Rename the column total\_cents to gross\_revenue\_cents...*" from the article above It'll scan the project, map every dependency at the column level and show you a structured report before changing anything. Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [altimate-code](https://altimate.ai/blog?tag=altimate-code) [Data pipelines](https://altimate.ai/blog?tag=data-pipelines) [dbt](https://altimate.ai/blog?tag=dbt) [Data Engineering](https://altimate.ai/blog?tag=data-engineering) [Snowflake](https://altimate.ai/blog?tag=snowflake) [Dev Tools](https://altimate.ai/blog?tag=dev-tools) [#ai-tools](https://altimate.ai/blog?tag=%23ai-tools) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Data Engineering Skills for Claude Code URL: https://altimate.ai/blog/we-created-data-engineering-skills-for-claude-code Extend Claude Code with reusable skills for dbt, SQL review, data parity, schema migrations, PII auditing, and warehouse cost analysis. [Back to blog](https://altimate.ai/blog) ## We Created Data Engineering Skills for Claude Code Steven Johnson May 11, 2026 We Created Data Engineering Skills for Claude Code Data engineering work spreads beyond SQL into lineage, tests, docs, cost, schema changes, and PII. We built skills that bring those workflows into Claude Code, covering dbt development, SQL review and translation, data parity checks, schema migration safety, Snowflake cost analysis, PII auditing, and visualization. Data engineering looks like software development from a distance, but the work has a different center of gravity. You still write code, review changes, and ship to production, but the real risk often lives outside the diff: downstream lineage, metric correctness, warehouse cost, schema drift, PII exposure, and whether the business can still trust the data after a change lands. Most data engineering work also does not begin and end with writing SQL. A small model change can require checking upstream sources, understanding downstream impact, updating dbt docs, adding tests, comparing outputs, reviewing query performance, and making sure sensitive data is handled correctly. We created data engineering skills for Claude Code to bring those workflows into the coding agent experience, giving data teams reusable playbooks for the work they already do every day. You can also access these skills through Altimate Code, our open-source LLM harness for data engineering, instead of installing each one individually in Claude Code. Altimate Code includes the skills out of the box and is ready to use with Claude: ```bash npm install -g altimate-code ``` The skills are open source, so you can use Altimate Code or browse the full set directly on GitHub: [https://github.com/AltimateAI/altimate-code/tree/main/.opencode/skills](https://github.com/AltimateAI/altimate-code/tree/main/.opencode/skills) ## **Skills as Data Engineering Playbooks** A skill is a reusable workflow Claude Code can load when a task calls for a specific kind of expertise. For data engineering, that means more than a prompt template. Each skill gives Claude Code guidance on what context to gather, what checks to run, what tools to use, what risks to watch for, and what kind of answer or artifact to produce. For example, a SQL review skill should not only say whether a query is syntactically valid. It should look for anti-patterns, safety issues, readability problems, performance risks, and whether the query is appropriate for the warehouse or dialect. A dbt testing skill should understand model structure, schema YAML, unit test patterns, and the kinds of edge cases that usually cause production data issues. The goal is to make Claude Code less dependent on one giant perfect prompt from the user. Instead of asking a data engineer to spell out every step, the skill carries the workflow: build the model, document the columns, add the tests, inspect the lineage, compare the outputs, review the SQL, and summarize what changed. ## **The Data Engineering Workflows We Covered** We organized the skills around the workflows data teams repeat every week: building models, reviewing SQL, validating changes, debugging warehouse issues, governing sensitive data, and communicating results. Each skill is focused on a specific job, but the real value comes from how they work together across the lifecycle of a data change. The current skill set covers six areas: - Building and maintaining dbt projects - Reviewing and improving SQL - Validating changes before they ship - Operating the warehouse - Governing data and team knowledge - Visualizing and explaining results --- ## **Building and Maintaining dbt Projects** The dbt skills cover the full lifecycle of a model: creating it, documenting it, testing it, understanding its downstream impact, and troubleshooting it when something breaks. #### **Creating and Changing Models** At it’s core, dbt is a tool that allows users to create SQL models, so we had to have a skill that creates dbt models. It is called `dbt-develop`. This skill teaches Claude how to both write new dbt models and change existing ones. If you are a dbt user, you know that is not as simple as just writing SQL. You have to understand the project setup, dialect of the database, patterns across the staging, intermediate, and mart models, and the lineage of data flowing throughout the project. #### Documenting the Project If you are anything like me, you’ve spent days or even weeks creating an intricate dbt project that takes messy, raw data and forms it into beautiful marts that you deliver to your stakeholders. You neglect to create good documentation for your work because you created it so why would I need to write what it does? A few weeks later you receive a message from a colleague asking what a specific column does, and you have no idea what the answer is. Documentation might have helped out a bit here. When I say documentation, I mean going past the basic notes of *primary key* or *unique ID for customer.* Our skill, `dbt-docs`, writes documentation that will answer questions about models and columns that you might not have thought about. The skill guides Claude to write about aggregations, primary and foreign keys, upstream tables, etc. Our hope is that this skill will allow you to stop answering questions about your models and columns and spend more time building more. [https://youtu.be/PY63\_Eu3Si4](https://youtu.be/PY63_Eu3Si4) ### **Writing Tests for dbt Models** Before working in data, I was a teacher, so I have always loved a good test. Not the kind that exists just to make someone nervous, but the kind that makes expectations clear. A good test tells you what should happen, what matters, and where the gaps are. That is exactly why tests matter in dbt. There is a special kind of confidence that comes from seeing a model run successfully. There is also a special kind of pain that comes from realizing later that the model ran successfully and still produced the wrong answer. The `dbt-test` skill helps Claude Code add schema tests, unit tests, and data quality checks to dbt models. It can guide Claude Code through common checks like `not_null`, `unique`, `relationships`, and `accepted_values`, as well as custom generic or singular tests when a model needs something more specific. For more complex model logic, the `dbt-unit-tests` skill goes deeper. It helps generate dbt unit tests by analyzing the model SQL, identifying upstream refs and sources, creating mock inputs, and assembling YAML that tests things like `CASE` statements, joins, window functions, null handling, aggregations, and incremental behavior. The point is not to test for the sake of testing. It is to protect the assumptions your model depends on, so the next change does not quietly break the meaning of the data. [https://youtu.be/O49qIMh2QPQ](https://youtu.be/O49qIMh2QPQ) #### **Understanding and Fixing Impact** dbt changes have a habit of traveling. A small edit to a source model can ripple into downstream marts, dashboards, tests, and metric definitions. The `dbt-analyze` skill helps Claude Code reason about that blast radius before a change ships by inspecting dependency and lineage context, including column-level lineage where available. The other side of impact is troubleshooting. When a dbt project fails to compile, a model errors at runtime, a test starts failing, or a dashboard suddenly looks wrong, the `dbt-troubleshoot` skill gives Claude Code a diagnostic workflow instead of leaving it to guess from the error message alone. It helps separate compilation issues, warehouse errors, test failures, incorrect data, and performance problems. Together, these skills help Claude Code answer two questions that come up constantly in dbt work: “What could this change affect?” and “What is actually going wrong?” --- ## **Reviewing and Improving SQL** Even in a dbt-heavy world, SQL is still the language data teams use to express business logic. But good SQL is not just SQL that runs. It should be readable, safe, efficient, and appropriate for the warehouse it runs on. The SQL skills help Claude Code review, optimize, and translate queries across the systems data teams actually use. #### **Reviewing SQL Before It Ships** Every data team has seen a query that technically works but makes you nervous. Maybe it has a risky join, a missing filter, unclear business logic, a performance issue hiding in plain sight, or a pattern that will be painful for the next person to maintain. The `sql-review` skill gives Claude Code a pre-merge review workflow for SQL. It guides Claude Code to look for syntax issues, anti-patterns, readability problems, performance risks, and safety concerns before the query lands in production. The goal is not just to ask, “Does this query run?” It is to ask, “Is this query understandable, reliable, and safe enough for the data system it is about to become part of?” #### **Optimizing Slow or Expensive Queries** Slow queries are rarely just an inconvenience in data engineering. They can block development, delay dashboards, frustrate stakeholders, and quietly drive up warehouse costs. A query that was fine on a small table can become a problem as data volume grows or as more teams start depending on it. The `query-optimize` skill helps Claude Code inspect SQL with performance in mind. It can look for inefficient joins, unnecessary scans, filtering issues, aggregation patterns, and opportunities to rewrite the query in a way that is easier for the warehouse to execute. This is where Claude Code becomes useful beyond code generation. It can help reason through why a query is slow, what tradeoffs are available, and how to make the SQL more efficient without losing the business logic that made the query valuable in the first place. #### **Translating Across Dialects** If you’ve done much traveling, you know that speaking the same language does not always mean speaking it the same way. SQL is similar. Snowflake, BigQuery, Databricks, Postgres, Redshift, MySQL, SQL Server, and DuckDB all share the same broad language, but each warehouse has its own syntax, functions, conventions, and sharp edges. A query written for BigQuery will not always translate cleanly into Snowflake. The same is true when moving models between warehouses, supporting multiple customer environments, or trying to standardize logic across a mixed data stack. That is why we built the `sql-translate` skill. It helps Claude Code translate SQL from one dialect to another while preserving the intent of the query. Think of it as Duolingo for Claude Code, except instead of asking it to practice ordering coffee, you are asking it to keep your business logic intact across warehouses. If your team is migrating warehouses or working across multiple SQL engines, this skill gives Claude Code a much better starting point for making those translations safely. --- ## **Validating Changes Before They Ship** In software engineering, tests often tell you whether a change broke expected behavior. In data engineering, validation can be harder to pin down. A model can run, a query can return rows, and a migration can apply successfully while the meaning of the data has still changed. That is why this category matters so much. Data teams do not just need help making changes. They need help understanding whether those changes are safe to ship. The validation skills help Claude Code compare outputs, inspect lineage, and look for migration risks before a change reaches production. ### **Comparing Data Outputs** One of the most common questions in data work is also one of the most important: did this change alter the data? Sometimes the answer should be yes. You fixed a bug, changed a business definition, or added new logic. But often, especially during a refactor or migration, the goal is to preserve behavior while changing the implementation. That is where the `data-parity` skill becomes useful. The `data-parity` skill helps Claude Code compare two tables or query results and diagnose exactly how they differ. It can be used for migration validation, ETL regression checks, and query refactor verification. Instead of stopping at “these counts do not match,” the workflow pushes toward understanding where they differ, why they differ, and whether the difference is expected. Think of it as asking Claude Code to check the receipt before you leave the store. The change may look fine at a glance, but you want to know whether the numbers actually add up. ### **Seeing How Lineage Changed** A data change is not only about the rows that come out the other side. It is also about how values move through the system. When the lineage changes, the meaning of a column can change with it. The `lineage-diff` skill helps Claude Code compare column-level lineage between two versions of a SQL query or model. It can show which data flow edges were added, removed, or changed, giving the reviewer a clearer picture of how the transformation shifted. This is especially useful when reviewing changes that look small in the SQL but may affect important downstream fields. A join changes. A source column is swapped. A derived field starts pulling from a different upstream path. The query may still run, but the story of the data has changed. The goal is to make those invisible changes visible before they surprise someone downstream. ### **Catching Migration Risk** Schema changes are another place where “it ran successfully” is not enough. A migration can apply cleanly and still introduce data loss, break assumptions, or create problems for downstream consumers. The `schema-migration` skill helps Claude Code analyze DDL changes before they are applied. It looks for risks like type narrowing, dropped columns, missing defaults, removed constraints, and other breaking column changes that can quietly turn into production issues. This matters because schema migrations often feel mechanical until they are not. Renaming a column, changing a type, or tightening a constraint might look straightforward in code, but those changes can affect ingestion jobs, BI tools, reverse ETL flows, contracts, and every team that has built something on top of the data. The skill gives Claude Code a migration review mindset: not just “Can this statement execute?” but “What could this break if it does?” --- ## **Operating the Warehouse** Once data work is running in production, a different set of questions starts to matter. What is expensive? What is slow? What changed? Which workloads are driving spend? Which queries or models need attention? The warehouse is not just where data lives. It is also where performance and cost decisions show up. The warehouse operations skills help Claude Code reason about those questions as part of the engineering workflow. Instead of treating cost and performance as separate admin tasks, these skills bring them closer to the code, queries, and models that created them. ### **Understanding Cost** Warehouse spend can be difficult to reason about because the cost is usually spread across queries, users, jobs, warehouses, and workloads. A dashboard may feel slow, a bill may jump, or a team may notice that a routine transformation suddenly got more expensive, but finding the cause is not always obvious. The `cost-report` skill helps Claude Code analyze Snowflake query costs and identify optimization opportunities. It can help look at expensive queries, warehouse usage, credit consumption, unused resources, and patterns that may point to waste or inefficient workloads. This gives Claude Code a FinOps lens. Not in the “please enjoy this spreadsheet of guilt” sense, but in the useful sense: where is the money going, what changed, and what can we do about it? ### **Diagnosing Operational Issues** Cost is only one part of operating a warehouse. Data teams also have to deal with slow queries, failing jobs, runtime errors, and models that behave differently as volume grows. This is where several skills work together. The `query-optimize` skill helps Claude Code reason about slow or inefficient SQL. The `dbt-troubleshoot` skill gives it a workflow for compilation failures, runtime database errors, failing tests, incorrect data, and performance issues in dbt projects. The goal is to make Claude Code useful when something is not healthy in production. It can help move from symptoms to causes: from “this dashboard is slow,” “this model failed,” or “our warehouse spend jumped” toward a clearer explanation of what is happening and what to try next. --- ## **Governance, Privacy, and Team Knowledge** Data engineering also carries responsibilities that do not fit neatly into “write the model” or “make the query faster.” Teams need to know where sensitive data lives, whether a query exposes it, and whether new work follows the standards the team has already agreed on. The governance and training skills help Claude Code operate with more awareness of privacy, compliance, and team-specific conventions. ### **Auditing Sensitive Data** Sensitive data has a way of showing up where you least expect it. An email field gets added to a downstream model. A phone number moves into an analytics table. An IP address appears in a query result that was supposed to be safe to share. Unlike pie, PII is much better when it is not casually passed around. The `pii-audit` skill helps Claude Code classify schema columns for personally identifiable information and sensitive data, including direct identifiers like SSNs, emails, phone numbers, names, addresses, and credit card numbers, as well as quasi-identifiers like dates of birth, zip codes, IP addresses, and device IDs. It can also check whether a query or dbt model exposes PII, distinguish between sensitive fields used internally and sensitive fields returned in the output, and help generate a PII inventory for compliance workflows like GDPR, CCPA, and HIPAA. The point is not to turn Claude Code into a compliance department. It is to give it enough privacy awareness to pause at the right moments, surface risk, and help data teams avoid accidentally spreading sensitive data through models, reports, or ad hoc queries. [https://youtu.be/KtWwjVIhlGI](https://youtu.be/KtWwjVIhlGI) ### **Teaching Claude Code Your Team’s Standards** That same teaching instinct shows up in the `teach`, `train`, and `training-status` skills. A lot of what makes someone effective on a data team is not just knowing SQL or dbt. It is learning the patterns, preferences, definitions, and little bits of context that make work fit the team around it. Every data team has conventions that are obvious to the people who have been there long enough and invisible to everyone else. How staging models should be named. What belongs in marts. Which patterns are encouraged. Which shortcuts should be avoided. Which business definitions have sharp edges. Most of that knowledge lives in scattered docs, review comments, Slack threads, and the brains of the people who have answered the same question ten times. The `teach` skill lets you show Claude Code an example file from your codebase and extract reusable patterns from it. The `train` skill helps Claude Code learn team standards from a document, style guide, or review checklist. The `training-status` skill shows what it has learned so far. These skills help Claude Code move closer to the way your team actually works. The goal is not just technically valid output. It is output that reflects your standards, your naming conventions, your modeling patterns, and the context your team has built over time. --- ### **Visualizing and Explaining Results** Data work does not end when the query returns rows. At some point, someone needs to understand what the data is saying. That might mean a chart for a trend, a dashboard for a team, a KPI view for leadership, or a more interactive way to explore a dataset. The `data-viz` skill helps Claude Code turn data into visual interfaces: charts, dashboards, KPI cards, analytics views, and reporting experiences. It guides Claude Code toward modern component libraries like Recharts, Tremor, Nivo, D3, Victory, visx, and shadcn/ui, depending on the project. This matters because the last mile of data work is often communication. A model can be correct, tested, documented, and optimized, but if people cannot understand the result, the work is not finished. The `data-viz` skill helps Claude Code move from “here is the data” to “here is what the data means.” --- Claude Code is already a powerful place to work with code. These skills make it more useful for the specific work data teams do every day: building models, writing tests, reviewing SQL, validating changes, checking lineage, auditing PII, understanding cost, translating dialects, and explaining results. The goal is not to replace data engineers. It is to give them a better teammate inside the workflows they already know. Data engineering requires code, context, caution, and communication. These skills give Claude Code more of that context, so it can help with the work around the code, not just the code itself. If you want to use the skills without installing them one by one in Claude Code, they are included in Altimate Code, our open-source LLM harness for data engineering. Altimate Code comes with these skills installed and is ready to work with Claude: ```bash npm install -g altimate-code ``` Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Best Data Conferences 2026: The Complete Guide URL: https://altimate.ai/blog/best-data-conferences-2026-the-complete-guide A complete guide to the best data conferences in 2026, from Snowflake Summit to dbt Summit. Find the right event for your role, stack, and goals. [Back to blog](https://altimate.ai/blog) ## Best Data Conferences 2026: The Complete Guide Steven Johnson May 11, 2026 Best Data Conferences 2026: The Complete Guide --- If you're trying to figure out which remaining data conferences in 2026 are worth your time and travel budget, this is the list. We've covered the biggest events on the calendar, from global platform summits drawing 20,000+ attendees to practitioner-focused gatherings where the real technical conversations happen. Listed in chronological order. --- ## [Gartner Data & Analytics Summit](https://www.gartner.com/en/conferences/na/data-analytics-us) **March 9-11 | Gaylord Palms, Orlando, FL | ~5,000 attendees** If your job involves data strategy, AI governance, or making the case to leadership for data investments, this one belongs near the top of your list. The summit runs on research from Gartner analysts: over 60 experts delivering 137 sessions across five main tracks and three spotlight tracks. This year's focus was squarely on moving past AI pilots and into production-grade AI that actually delivers ROI. The crowd skews toward senior roles: CDAOs, heads of AI, data architecture leads, data management executives. The format matches that audience: keynotes and roundtables built for strategic conversation, not product demos. There are 130+ vendors in the Exhibit Showcase if that's useful, but the sessions are where the value is. Gartner also runs this summit in London and Sydney for teams outside North America. --- ## [Tableau Conference](https://www.salesforce.com/tableau-conference/) **May 5-7 | San Diego Convention Center, San Diego, CA | ~8,000 attendees** **TC26** is three days of 300+ expert-led sessions, 150+ hands-on trainings, and the kind of community programming that makes this feel less like a tech conference and more like a reunion. Tableau's community calls itself the DataFam for a reason. The culture is genuinely welcoming in a way that most data conferences aren't. Content this year centers on agentic analytics, interoperability, and new product integrations. If your team uses Tableau day-to-day, the skill-building alone makes this worth attending. Sessions run from beginner dashboard design to advanced AI integration. There's also Iron Viz (a live data viz competition), Tableau Doctor for 1:1 expert help, and Salesforce+ livestreams for anyone who can't make it in person. --- ## [Snowflake Summit](https://www.snowflake.com/en/summit/) **June 1-4 | Moscone Center, San Francisco, CA | ~20,000 attendees** **Snowflake Summit 2026** has 500+ sessions, hands-on labs, training and certification, and an expo floor with 190+ partner booths. The Builders Hub is specifically for developers: bootcamp sessions and live demos. The Industry Zone covers vertical-specific content for finance, healthcare, retail, and more. The theme this year is agentic intelligence: how to move from isolated data strategies to connected, AI-powered enterprise experiences. If your team runs on Snowflake, this is where you get ahead on what's coming from the platform and hear real implementation stories from companies that have already shipped it. Main keynotes are livestreamed free for non-registrants. Catch the Altimate AI team at booth 1311 --- ## [Databricks Data + AI Summit](https://www.databricks.com/dataaisummit) **June 15-18 | Moscone Center, San Francisco, CA | ~22,000 attendees** **Data + AI Summit** brings together data engineers, scientists, architects, ML engineers, and executives from over 160 countries across 800+ sessions covering data engineering, warehousing, governance, analytics, agentic AI, and AI/BI. If you're working with Apache Spark, Delta Lake, Apache Iceberg, MLflow, or dbt alongside Databricks, this is where roadmaps get announced and where you'll find the most advanced technical content on those tools. The conference also offers new certification courses on-site. One caveat: it's a big event. Plan your session schedule before you arrive or you'll spend the week just trying to navigate the venue. Catch the Altimate AI team at booth 577 --- ## [Ai4](https://ai4.io/vegas/) **August 4-6 | Las Vegas, NV | ~12,000 attendees** America's largest independent AI conference, drawing 12,000+ executives and decision-makers from 90+ countries. Ai4 is enterprise-focused. The emphasis is on real-world AI implementation and measurable business outcomes, not research. Tracks are organized by industry (finance, healthcare, retail, manufacturing) and by function (AI strategy, generative AI, AI agents, automation). The audience skews toward business and technical leadership. If you're evaluating AI strategy or want to see how peers in adjacent industries are actually deploying AI in production, this is a useful venue. For teams wondering whether their AI initiatives are in line with what the broader market is doing, the comparison is genuinely worth making. --- ## [dbt Summit](https://coalesce.getdbt.com/) *(formerly Coalesce)* **September 15-18 | The Cosmopolitan, Las Vegas, NV | ~3,000 attendees** Coalesce has a new name. **dbt Summit 2026** is the world's largest gathering of dbt users: analytics engineers, data engineers, data architects, and the data leaders working alongside them. The rebrand reflects how the event has grown from a community conference into a full industry summit, spanning four days with 100+ sessions across keynotes, breakout tracks, and hands-on labs. The conversations here are practitioner-driven. You won't find a lot of polished vendor pitches. You'll find engineers being honest about what's working and what isn't. Key themes include dbt Mesh, data contracts, AI-ready data architectures, incremental models, and how analytics engineering intersects with ML workflows. For data teams building on dbt, this is the one worth planning around. Catch the Altimate AI team on the Expo floor --- ## [Big Data LDN](https://www.bigdataldn.com/en-gb.html) **September 23-24 | Olympia London, London, England | Free to attend** The UK's largest data, analytics, and AI event and the most important data conference on the European calendar. Big Data LDN brings together engineers, architects, data leaders, and practitioners for two days of 400+ seminars across 15 free-to-attend conference theatres. Speakers come from Google, AWS, TikTok, Spotify, Snowflake, Databricks, and a wide range of startups. Free registration, 130+ exhibitors ranging from major platform vendors to early-stage companies, and sessions covering AI governance, data engineering, and data products. In 2026, a co-located event called Data Driven LDN runs on September 22nd, focused specifically on AI agents, AI governance, and data products for senior practitioners who want deeper dives. Catch the Altimate AI team on the Expo floor --- ## How to Pick the Right Data Conference in 2026 Nobody can do all of these, so a few filters are worth applying. If your team runs on Snowflake, Snowflake Summit is the obvious call. The same logic applies for dbt and dbt Summit. Platform-specific conferences tend to have the highest signal-to-noise for teams already using those tools. Data leaders and strategists get the most out of Gartner and Ai4. Practitioners and engineers tend to get more hands-on value from dbt Summit and the Databricks summit. Tableau Conference is its own category: best for teams where Tableau is the primary visualization layer. European teams should have Big Data LDN near the top of the list. It's free, comprehensive, and the audience quality is high. Pick the ones where your stack, your role, and your team's current problems overlap with the program. That overlap is usually obvious once you look. Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Claude Code vs. Altimate Code for Data Engineering URL: https://altimate.ai/blog/claude-code-or-altimate-code-for-data-engineering An analysis of the performance of Claude Code vs. the open source Altimate Code agentic harness for data engineering tasks. [Back to blog](https://altimate.ai/blog) Dev Tools · altimate-code · Altimate ## Claude Code or Altimate Code for Data Engineering? Muhammad Anas Farooqui Apr 23, 2026 Claude Code or Altimate Code for Data Engineering? Claude Code is fast, writes good software code, and can even handle dbt models and other data engineering tasks to certain extent. At first glance [Altimate Code](https://docs.altimate.sh/getting-started/), the data engineering harness, may seem to do similar things. So, do we need both? Is one a replacement for the other? We ran an experiment to find out. We used the same prompt, the same model (Claude Opus 4.6), and the same codebase. One run with Claude Code alone, one run with Altimate Code. The results were not close. --- ## What Claude Code Actually Is Claude Code is a general-purpose agentic coding tool that reads files, runs shell commands, edits code etc. For data engineering, it draws on Claude's extensive training coverage of SQL dialects and dbt conventions. But Claude Code is no data engineering expert. It has no deterministic SQL anti-pattern engine, no static lineage tracer, no PII classifier, no schema diff tool that programmatically flags breaking changes. And there is no evidence the team at Anthropic is focused on making it any better at these tasks. ## What Altimate Code Actually Is [Altimate Code](https://docs.altimate.sh/getting-started/) is an open-source data engineering harness with 100+ specialized tools for building, validating, optimizing, and shipping data products. It uses LLMs (Claude, GPT, Gemini, or any of 17+ providers) as its AI backend, but routes every task through domain-specific tooling that general-purpose agents do not have: - **Live warehouse connection** -- connects directly to various warehouses with auto-discovery from profiles.yml or environment variables. - **dbt-native build and test** -- runs real `dbt build` against your warehouse, materializing tables and executing every data test. - **Column-level lineage** -- traces every column from source through joins, CTEs, and subqueries to final output in real time. - **PII detection** -- scans schemas across 15+ PII categories (SSN, email, phone, DOB, health data) with confidence scores. - **Impact analysis and schema diff** -- calculates blast radius across your full dbt DAG and produces column-level before/after diffs with breaking change classification. - **SQL quality grading** -- scores SQL on syntax, style, safety, and complexity (A-F) for objective, reproducible code review. - **Enforced agent modes** -- Builder (can modify), Analyst (read-only), Plan (design only) -- enforced at the harness level, not by prompt. You cannot DROP TABLE in Analyst mode regardless of what the LLM suggests. - **Project conventions via AGENTS.md** -- team-wide rules loaded into every session's system prompt for consistency across engineers and CI. ## How Claude Code and Altimate Code Work Together Claude Code and Altimate Code are not competitors or alternatives.They occupy different layers of the stack. Claude Code is a general-purpose coding agent. Altimate Code is a domain-specific data engineering harness. When used together, Claude Code handles task orchestration and conversational context while Altimate Code’s tools handle warehouse connectivity, lineage tracing, PII scanning, build execution, and impact analysis. The LLM’s reasoning improves when it has access to specialized tools enabling access to real data, real schemas, and real test results instead of guessing from file contents alone. The comparison in this post demonstrates why domain-specific tooling matters for data engineering work. --- ## The Experiment: Claude Code vs. Altimate Code As an experiment we took a realistic, broken dbt model `mart_patient_360` from a demo dbt healthcare project `medflow-analytics` — a scenario that data engineering teams encounter regularly — and gave the exact same prompt to two setups: 1. Claude Code standalone and 2. Altimate Code + Claude Code, same codebase, and same task. **The goal:** to see whether domain-specific data engineering tooling produces meaningfully different outcomes than a general-purpose coding agent when the task involves schema accuracy, HIPAA compliance, build verification, and downstream impact analysis — the things that actually matter in production data work. **Model:** Claude Opus 4.6 was used in both experiments to ensure a fair comparison. **The prompt we used:** > *The* `mart_patient_360` *model is incomplete. Right now it joins patients, encounters, diagnoses, medications, and lab\_results but the SELECT is mostly empty — it’s missing the* `patient_id` *primary key, has no aggregated metrics, and just exposes raw PII fields like SSN and phone number. I need you to build this out into a proper patient 360 view: add the* `patient_id` *key, total encounter count, unique diagnosis count, active medication count, most recent lab result date, days since last visit, and a patient risk tier (high/medium/low based on encounter frequency and diagnosis count). The model is tagged as PII/HIPAA-restricted. Make sure the final model is safe to materialize, fix any compliance issues you see, and tell me what downstream impacts or governance concerns I should be aware of before merging.* **Claude Code:** Screenshot of Claude Code's terminal welcome screen with a user prompt asking it to build out an incomplete mart\_patient\_360 model, adding a patient ID key, aggregated metrics, and a risk tier, while fixing PII/HIPAA compliance issues before merging. **Altimate Code:** Screenshot of the Altimate Code interface showing the same patient-360 HIPAA-compliance prompt, under a "Claude Code vs Altimate Code: Head-to-Head Results" heading comparing the two agents' outputs. --- ## Head-to-Head Results: Data Engineering Harness Vs. General Purpose Coding Assistant ### Claude Code Output: Enhanced the model but without execution or proper validation, surfaced limited insights: Screenshot of the Claude Code output ### Altimate Code Output: Unlike Claude Code, Altimate Code enhanced the dbt model, executed it, did proper validation, and surfaced detailed insights: Screenshot of the Altimate Code output ## Detailed Findings ### 1\. PII and HIPAA Compliance This is where the gap was most visible and most consequential. | Aspect | Claude Code (Opus 4.6) | Altimate Code (Opus 4.6) | | --- | --- | --- | | **SSN handling** | Hashed with `sha2(ssn, 256)` — SSN still flows through the query pipeline. | Removed entirely — SSN is never selected into any CTE. It never touches the query. | | **full\_name, phone, email, address** | Kept in the final model output. Still materialized to Snowflake disk. | Dropped completely from the model with explicit per-column rationale. | | **PII verification** | None — assumed the code changes were sufficient. | Ran automated `altimate_core_classify_pii` scan on the output schema. Caught that `full_name` was still flowing through a CTE even though it wasn’t in the final SELECT. Removed it in a second pass. | | **Philosophy** | “Mask the PII.” Sensitive data still exists in the table, just obfuscated. | “Eliminate the PII.” The mart never touches it. Consumers who need PII use RBAC on the staging layer. | **Why this matters:** Claude Code’s SHA-256 hash of SSN is a common pattern, but it’s a weaker approach than most teams realize. SSNs are 9 digits — roughly 900 million possible values. Altimate Code’s approach of full elimination is the correct HIPAA-compliant pattern for analytical marts. In our experiment, Altimate Code’s `lineage_check` tool revealed that `full_name` (a PII field) was flowing through a CTE even though it wasn’t in the final SELECT. Claude Code missed this entirely. Altimate Code’s lineage engine claims 100% edge match accuracy across 500 benchmark queries. ### 2\. Schema Accuracy: Did It Actually Build? | Aspect | Claude Code (Opus 4.6) | Altimate Code (Opus 4.6) | | --- | --- | --- | | **Columns referenced** | Used `gender`, `race`, `ethnicity`, `primary_care_provider_id` from `stg_patients` (none of these exist in the actual SQL) plus `blood_type` (which exists in SQL but isn’t in the YAML). Also grouped `stg_diagnoses` and `stg_medications` by `patient_id` directly. | Only used columns confirmed to exist in the actual staging SQL. | | **The problem** | The four phantom columns are documented in `_staging.yml` but **not selected** in `stg_patients.sql`. The actual SQL only selects: `patient_id, full_name, ssn, date_of_birth, phone, email, address, blood_type, insurance_id, created_at`. Additionally, `stg_diagnoses` and `stg_medications` do not contain `patient_id` — Claude Code assumed they did. | Cross-referenced the YAML documentation against the actual SQL files AND the seed CSV headers to identify exact available columns. | | **Diagnosis join** | Grouped `stg_diagnoses` by `patient_id` directly. | Joined `stg_diagnoses` to `stg_encounters` via `encounter_id` to get `patient_id`, then grouped. Same pattern applied for `stg_medications.` | | **Would it build?** | **No:** would fail on at least six missing column references (4 phantom from `stg_patients`, plus `patient_id` in both `stg_diagnoses` and `stg_medications`). | **Yes:** **PASS=40, WARN=0, ERROR=0** across the full project. | **Why this matters:** Claude Code trusted the YAML documentation, which was out of sync with the actual SQL in multiple directions. Some columns were documented but missing from the SQL, while `blood_type` was the reverse case (in the SQL but undocumented). This is extremely common in real dbt projects. Altimate Code verified against multiple sources (SQL, YAML, seed data) and resolved the discrepancies. The `altimate_core_schema_diff` tool produces a column-level before/after comparison with explicit breaking change classification (e.g. `[BREAKING] Column 'ssn' removed`). In our experiment, this confirmed 16 schema changes with 6 breaking giving the team an exact migration checklist. ### 3\. Build Verification and Data Validation | Aspect | Claude Code (Opus 4.6) | Altimate Code (Opus 4.6) | | --- | --- | --- | | **dbt build attempted** | No | Yes: `altimate-dbt build --model mart_patient_360.` | | **Tests run** | Never executed | 8 data tests, all passing (unique, not\_null, accepted\_values). | | **Full project build** | Never attempted | **PASS=40, WARN=0, ERROR=0** — 20 models, 10 seeds, 8 tests, 2 project hooks. | | **Data validation** | None | Queried Snowflake directly: confirmed 1,000 patients, verified risk distribution (719 low, 278 medium, 3 high), spot-checked high-risk and low-risk patients. | | **SQL quality checks** | None | Ran `sql_analyze`, `altimate_core_check`, and `altimate_core_grade.` | **Why this matters:** Claude Code wrote the code and declared it done. Altimate Code wrote the code, built it on Snowflake, ran every test, queried the output data, and verified the results made clinical sense. In production data engineering, “the SQL looks right” is not the same as “it works.” ### 4\. Downstream Impact Analysis | Aspect | Claude Code (Opus 4.6) | Altimate Code (Opus 4.6) | | --- | --- | --- | | **Blast radius assessment** | Manually identified `vw_patient_summary_deidentified` as downstream and updated it | Ran automated `impact_analysis` — confirmed 0/20 downstream dbt models affected. | | **Schema diff** | Described breaking changes in a text table. | Ran `altimate_core_schema_diff` — automated analysis: **16 changes, 6 breaking**, with exact column-level detail. | | **Breaking change detail** | Listed columns removed. | Categorized each: `[BREAKING] Column 'mart_patient_360.ssn' removed`, `[info] Column 'mart_patient_360.patient_risk_tier' added (VARCHAR).` | | **External consumer warnings** | Generic: “verify that Snowflake row-access policies are correctly scoped”. | Specific: BI tools, RBAC enforcement, CI check suggestion for `restricted` tag containment, non-determinism warning for `current_date` usage. | **Why this matters:** In production data environments, the most dangerous changes are the ones that look safe in isolation. Altimate Code's impact\_analysis tool traverses the full DAG programmatically, and its schema\_diff produces a migration checklist that a team lead can review. ### 5\. Governance Recommendations | Topic | Claude Code (Opus 4.6) | Altimate Code (Opus 4.6) | | --- | --- | --- | | **date\_of\_birth** | Mentioned Safe Harbor in passing | Specific recommendation: “consider age-banding for de-identified datasets per HIPAA Safe Harbor” — included in YAML column description. | | **Non-determinism** | Not mentioned | Flagged that `days_since_last_visit` and `active_medication_count` use `current_date`, making the table non-deterministic. Recommended documenting refresh cadence. | | **Tag enforcement** | Not mentioned | Recommended CI check to prevent `restricted`\-tagged models from being referenced by non-restricted downstream models. | | **Risk tier thresholds** | Suggested making thresholds dbt vars | Used different (more conservative) thresholds: high tier requires `>=5 encounters AND >=3 diagnoses` (AND logic), vs. Claude’s `>=10 encounters OR >=5 diagnoses` (OR logic). Medium tier in both used OR logic. | **Why this matters:** Claude Code offered textbook advice: reasonable, but generic. Altimate Code's recommendations were actionable at the PR level: a specific YAML annotation for Safe Harbor, a specific CI check for tag containment, a specific warning about non-deterministic columns that would produce different results depending on when the pipeline runs. These are the details that prevent a compliance review from becoming a compliance finding. --- ## The Extra Steps Altimate Code Took These are capabilities that Claude Code simply does not have access to: | Tool Used | What It Did | Why It Matters | | --- | --- | --- | | `altimate_core_classify_pii` | Automated PII scan on the final schema — flagged `patient_id` (0.75 confidence) and `date_of_birth` (0.9 confidence) as remaining quasi-identifiers | Catches PII that humans miss in code review | | `lineage_check` | Traced column-level lineage from sources through CTEs to output | Caught `full_name` leaking through a CTE even though it wasn’t in final SELECT | | `impact_analysis` | Automated blast radius calculation across the full DAG | Confirms safety with certainty, not guessing | | `altimate_core_schema_diff` | Column-level before/after diff with breaking change classification | Documents exactly what changes for downstream consumers | | `sql_execute` (warehouse) | Queried actual Snowflake tables to verify data distribution | Validates that the model produces clinically sensible results | | `altimate-dbt build` | Full project build + test execution on Snowflake | Proves the code actually works, not just “looks right” | --- ## Summary: The Scorecard | Capability | Claude Code (Opus 4.6) | Altimate Code (Opus 4.6) | | --- | --- | --- | | SQL generation quality | Good structure, but used phantom columns | Verified against actual schema — builds cleanly | | PII handling | Masked (hash) — PII still in pipeline | Eliminated — PII never enters the query | | Build verification | Not attempted | Built + tested on Snowflake (PASS=40) | | Data validation | None | Queried warehouse, verified distribution | | Downstream impact | Manual guess about one view | Automated blast radius + schema diff (16 changes, 6 breaking) | | PII audit | None | Automated scan with confidence scores | | Column-level lineage | Not performed | Traced end-to-end, caught PII leak in CTE | | Governance recommendations | Generic HIPAA mention | Specific: RBAC, Safe Harbor age-banding, non-determinism, CI tag enforcement | | Would the model build? | No — missing column references | Yes — full project green | ## In Conclusion: An AI Coding Assistant Needs a Domain Expert Harness to Master Data Engineering The takeaway of all this is that **general-purpose AI + domain-specific intelligence** produces categorically better results than either alone. For data engineering work where correctness, compliance, and safety matter, the domain layer is not optional. Altimate Code's value is not that it replaces Claude Code Its value is that it surrounds Claude Code with 100+ specialized tools that verify, build, test, scan, and validate before declaring the job done. For data engineering teams shipping to production, that difference is the entire gap between "looks right" and "is right." --- ## Steps To Reproduce This Analysis **We’ve open-sourced the full analysis so you can reproduce it:** **Repository:** [github.com/altimateanas/altimate\_code\_enterprise\_demos](https://github.com/altimateanas/altimate_code_enterprise_demos) 1. Clone the repo: `git clone https://github.com/altimateanas/altimate_code_enterprise_demos` 2. Navigate to `medflow-analytics/`directory It's a healthcare dbt project running on Snowflake with patient data, claims, encounters, diagnoses, medications, and lab results. 3. Setup your snowflake target in dbt profiles.yml. 4. Run the prompt in section "The Experiment: Claude Code vs. Altimate Code" in Claude Code (standalone) then observe the output 5. Connect Altimate Code and run the same prompt and compare **Make sure to use the same underlying LLM in both runs.** Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [Dev Tools](https://altimate.ai/blog?tag=dev-tools) [altimate-code](https://altimate.ai/blog?tag=altimate-code) [Altimate](https://altimate.ai/blog?tag=altimate) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Altimate Code: the agentic harness for data engineering URL: https://altimate.ai/blog/introducing-altimate-code Altimate code is an agentic harness for data engineering. It outperforms other AI solutions on data tasks based on common benchmarks. [Back to blog](https://altimate.ai/blog) AI Tool · Data Engineering · Altimate ## We built AI that actually works for data engineering. We crushed the ADE benchmark. Anand Gupta Mar 19, 2026 We built AI that actually works for data engineering. We crushed the ADE benchmark. tl;dr: Today, we are launching Altimate Code, an open-source agentic data engineering harness that far exceeds generic LLMs (and other leading models) on data engineering tasks. [Check out the project on GitHub](https://github.com/AltimateAI/altimate-code), or catch the [March 25th overview and AMA on YouTube](https://www.youtube.com/watch?v=g-ACWwz9TGg). --- AI agents have transformed how software gets built. But for data engineering, they have fallen short… An AI coding agent on Replit [deleted an entire production database](https://fortune.com/2025/07/23/ai-coding-tool-replit-wiped-database-called-it-a-catastrophic-failure/) during a code freeze, then [created 4,000 fake records](https://www.eweek.com/news/replit-ai-coding-assistant-failure/) to fill the empty tables. An [independent evaluation](https://www.atscale.com/blog/troubleshooting-snowflake-cortext-analyst/) of Snowflake's Cortex Analyst found 38% logical accuracy. Six out of ten queries were wrong, but compiled and ran just fine. One team got a [$5,000 bill from a single Cortex AI query](https://seemoredata.io/blog/snowflake-cortex-ai/) their resource monitors never caught. [78% of AI-generated SQL errors are silent wrong joins](https://medium.com/tr-labs-ml-engineering-blog/is-your-ai-agent-lying-with-perfect-sql-3a6a7d69bccf), queries that return confidently incorrect data. These aren't edge cases. They're what happens when AI agents operate on data infrastructure without safety layers. No schema validation, no lineage, no cost controls, no permission enforcement. The agent doesn't know your schema. It can't trace what breaks downstream. It doesn't know what a query will cost. And the system prompt telling it "don't drop tables" stops working at 100K context tokens. The problem isn't the model. It's everything around it. ## The missing layer General-purpose coding agents treat SQL like application code. It isn't. SQL operates on schemas that change, across dialects that diverge on every function name, through lineage chains no LLM can reliably trace, with cost implications that scale to thousands of dollars per mistake. What data engineering needs is a layer of compiled, deterministic tools that operate *outside* the LLM's reasoning loop. Tools that validate SQL against your actual schema in 2ms, trace column-level lineage through CTEs deterministically, and catch anti-patterns with zero false positives. Not better prompts. Better engines. That's what we built. ## Introducing Altimate Code Today we're open-sourcing **Altimate Code,** a data engineering harness built on [OpenCode](https://github.com/anomalyco/opencode), the open-source coding agent. Built by the team behind [dbt Power User](https://marketplace.visualstudio.com/items?itemName=innoverio.vscode-dbt-power-user), the most widely used dbt VS Code extension. The core idea: **the LLM reasons; compiled engines validate; neither replaces the other.** We ran dbt Labs' [ADE-bench](https://www.getdbt.com/blog/ade-bench-dbt-data-benchmarking), the open standard for measuring AI agents on real data engineering tasks. Pixel-art ALTIMATE CODE logo above a chat prompt asking it to build out an incomplete patient-360 dbt model while fixing PII/HIPAA compliance issues. **A cheaper model with compiled tools outperformed a more expensive model without them.** The difference isn't the model. It's the harness. Further details can be found [here](https://altimate.ai/benchmarks). ## What the harness catches ADE-Bench leaderboard showing Altimate Code (Sonnet 4.6, open source) in first place at 74.4%, ahead of Cortex Code (Opus 4.6, Snowflake-native) at 65.0% and Claude Code (Sonnet 4.6, general-purpose) at about 40%. Your agent writes a query. Before it touches your warehouse: The wrong table name that would have triggered a 30-second Snowflake error? **Caught in 2ms** with a fix suggestion. Five agent fix cycles cost 10ms and not 2.5 minutes of warehouse round-trips. The cartesian join that would have silently inflated your numbers by 100x? **Caught before execution.** 26 compiled anti-pattern rules, zero false positives across 1,077 benchmark queries. The column your downstream dashboard depends on? **Traced to its source** through every JOIN, CTE, and subquery with 100% edge match on 500 benchmark queries. The agent reasons on verified lineage, not guesses. The PII in the staging table you're about to expose? **Flagged and masked.** The SQL injection hiding in a Jinja template? **Blocked.** All compiled. All deterministic. All in milliseconds. ## Making the harness your own Engines are the foundation. But what makes Altimate Code *yours* is what you build on top of them. **Persistent memory** spans across sessions in two scopes: global (your preferences) and project (team knowledge versioned in git). Tell the agent "we never use FLOAT for money columns"… it remembers. Next session, next teammate, the knowledge is there. When one engineer corrects the agent, every teammate inherits the fix on `git pull`. No Slack message. No wiki update. The correction just propagates. **Governed agent modes** enforce permissions at the engine level, not through prompt instructions that models ignore at long context lengths. The Analyst can't INSERT, UPDATE, DELETE, or DROP. Not because of a system prompt, but because the compiled engine won't execute it. The Builder gets full read/write with your SQL rules applied. The Planner maps tasks without executing. **The compactor** summarizes long sessions while preserving data engineering state: warehouse connections, schema context, dbt project state, lineage findings. Multi-hour sessions maintain continuity across compaction boundaries. **The tracer** captures every LLM call, tool invocation, and warehouse metric locally. No external services. No data leaving your machine. Run `/trace` for an interactive viewer. ## Why independent? Snowflake shipped [Cortex Code](https://www.snowflake.com/en/blog/cortex-code-cli-expands-support/). Databricks launched [Genie Code](https://www.databricks.com/blog/introducing-genie-code). Both recognized that general-purpose agents don't work for data. But both shipped solutions that suit their ecosystem. Cortex Code won't help you migrate *off* Snowflake and onto BigQuery. Genie Code is not designed to optimize a Redshift query. Your Airflow DAGs don't run inside Databricks. Your warehouses span multiple providers. Your governance crosses every platform boundary. Your AI agents should too. Altimate Code connects to Snowflake, BigQuery, Databricks, PostgreSQL, Redshift, DuckDB, MySQL, SQL Server, Oracle, and SQLite. It runs with Anthropic, OpenAI, Google, AWS Bedrock, Azure, Ollama, and OpenRouter. With local models and the local-only tracer, it runs fully air-gapped so no data leaves your machine. Platform agents will always have telemetry we can't access. We'll always have independence they can't offer. Neither Snowflake nor Databricks will build first-class support for the other's warehouse. Neither will tell your agent a query is unnecessary when their revenue depends on you running it. Your harness should be yours. ## Try it ```plaintext npm install -g @altimateai/altimate-code altimate /discover ``` Two commands. It auto-detects your warehouse, indexes your schema, and you're building. Open source on [GitHub](https://github.com/AltimateAI/altimate-code). Docs at [altimate-code.ai](https://help.altimate.ai/code/getting-started/). Ten data stores. Any LLM. No platform tax. ## What we're building next, and where you can help The compiled engines ship today. Here's where we're headed and where we need the community: **Pipeline monitoring** for Airflow and Dagster. We want this to be proactive, not just interactive. **Blast radius analysis.** Before the agent acts, show what breaks downstream. **Decision memory:** extract *why things were built* from your Git history so agents stop undoing decisions they don't know were made. **Agent Data sandboxes:** changes prove themselves before touching production and many more Some of these we'll build. Some of them you'll build first. The roadmap, the benchmarks, and every known gap are in the repo. PRs welcome. Issues welcome. Forks welcome. We'd rather build this in the open than wait for it to be built inside a walled garden. If you're a data engineer who stitches tools together for a living, come break it. ## Want to learn more or get involved? Three options: 1. [Check out the project on Github](https://github.com/AltimateAI/altimate-code) - star the repo to support the project 2. [Join us on Slack](https://altimate.studio/join-agentic-data-engineering-slack) - we are launching a new Slack channel for our Agentic Data Engineering efforts. [Join us here](https://altimate.studio/join-agentic-data-engineering-slack). 3. Sign up for [next week's live overview and AMA](https://youtube.com/live/g-ACWwz9TGg) on YouTube. Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [AI Tool](https://altimate.ai/blog?tag=ai-tool-) [Data Engineering](https://altimate.ai/blog?tag=data-engineering) [Altimate](https://altimate.ai/blog?tag=altimate) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Claude Code Skills for Data Engineering URL: https://altimate.ai/blog/teaching-claude-code-the-art-of-data-engineering-introducing-altimate-skills Altimate Skills: Open-source Claude Code skills for dbt and Snowflake to boost analytics engineering with structured workflows and project context. [Back to blog](https://altimate.ai/blog) Altimate · Data Engineering · AI Tools ## Teaching Claude Code the Art of Data Engineering: Introducing Altimate Skills Anand Gupta Jan 22, 2026 Teaching Claude Code the Art of Data Engineering: Introducing Altimate Skills --- Today, we're open-sourcing **Altimate Skills** — a collection of [Claude Code skills](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview) specifically designed for analytics engineers. We are starting with skills for dbt and Snowflake. These encode the workflows and best practices that transform AI from a basic code generator into a capable data engineering assistant. [YouTube video player](https://www.youtube.com/embed/kvIo5PmF0Ns) **Key Results:** - **+25% improvement** on model creation tasks (40% → 65%) - **+22% faster execution** (TPC-H 1TB) with 100% logically equivalent queries generated for SQL optimization. - **53% accuracy** on [ADE-bench](https://github.com/dbt-labs/ade-bench) (43 real-world dbt tasks) - Skills that actually **teach Claude *how* to work**, not just *what* to write ```bash # Get started in 30 seconds /plugin marketplace add AltimateAI/data-engineering-skills /plugin install dbt-skills@data-engineering-skills ``` **GitHub:** [https://github.com/AltimateAI/data-engineering-skills](https://github.com/AltimateAI/data-engineering-skills) and ⭐ the repo. --- ## Solving the C**ontext and Workflow Issue** If you've used Claude Code, Cursor, or any AI coding assistant for dbt development, you've experienced the frustration: **The task:** "Create a staging model for the Stripe payment source." **What you expect:** ```sql -- models/staging/stripe/stg_stripe__payments.sql {{ config( materialized='view', schema='staging' ) }} with source as ( select * from {{ source('stripe', 'payments') }} ), renamed as ( select id as payment_id, amount_cents / 100.0 as amount, currency, status, created_at from source ) select * from renamed ``` **What you get:** ```sql SELECT * FROM {{ source('stripe', 'payments') }} ``` No `stg_` prefix. No `{{ source() }}` reference. No config block. No CTEs. No understanding of your project's conventions. ### Why This Happens The core issue isn't that LLMs lack knowledge — Claude knows dbt syntax perfectly well. The problem is **context and workflow**: 1. **No project awareness** — Claude doesn't know your naming conventions, folder structure, or existing patterns 2. **No verification loop** — Claude declares "done" after writing code, without running `dbt build` 3. **No convention discovery** — Claude guesses at patterns instead of reading existing models first 4. **Compile ≠ Success** — `dbt compile` passes, but the model produces the wrong output This leads to a frustrating cycle: AI writes code → You review and fix → AI loses context → Repeat. --- ## What Are Claude Code Skills? Anthorpic introduced [Claude Skills](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview) in October 2025. Skills are markdown files that teach Claude **how to approach tasks**, not just what syntax to use. Think of them as encoding the workflow an experienced analytics engineer follows. Skills matter because they make Claude more reliable and specialized: you can standardize repeatable work (like formatting outputs, following internal conventions, or running a known process) and reuse it across projects. Claude can also load Skills progressively, starting with lightweight metadata and pulling in deeper instructions only when needed, giving you targeted behavior without bloating context. Practically, a Skill is typically packaged as a small folder of structured instructions (and optionally templates, scripts, or reference files) that define *how* Claude should approach a workflow. See our [data-engineering-skills](https://github.com/AltimateAI/data-engineering-skills) repo folders as an example. A skill has two parts: **1\. Trigger conditions** — When should this skill activate? ```yaml --- name: creating-dbt-models description: | Guide for creating dbt models. ALWAYS use this skill when: (1) Creating ANY new model (staging, intermediate, mart) (2) Task mentions "create", "build", "add" with model/table (3) Modifying model logic or columns --- ``` **2\. Workflow instructions** — What steps should Claude follow? ```markdown # dbt Model Development **Read before you write. Build after you write. Verify your output.** ## Critical Rules 1. ALWAYS run `dbt build` after creating models - compile is NOT enough 2. ALWAYS verify output after build using `dbt show` 3. If build fails 3+ times, stop and reassess your approach ## Workflow ### 1. Understand Requirements - What columns are needed? - What is the grain (one row per what)? - What calculations are required? ### 2. Discover Project Conventions cat dbt_project.yml find models/ -name "*.sql" | head -20 Read 2-3 existing models to learn patterns... ``` When Claude encounters a task that matches the trigger conditions, it automatically applies the skill's workflow. --- ## The Skills We Built ### [dbt Skills](https://github.com/AltimateAI/data-engineering-skills/tree/main/skills/dbt) | Skill | Purpose | Key Behaviors | | --- | --- | --- | | **creating-dbt-models** | Model creation | Convention discovery → Write → Build → Verify output | | **debugging-dbt-errors** | Error troubleshooting | Read full error → Check upstream → Apply fix → Rebuild | | **testing-dbt-models** | Schema tests | Study existing test patterns → Match project style | | **documenting-dbt-models** | Documentation | Analyze model → Generate descriptions | | **migrating-sql-to-dbt** | Legacy SQL conversion | Parse SQL → Create proper dbt model | | **refactoring-dbt-models** | Safe restructuring | Track dependencies → Apply changes → Verify downstream | ### [Snowflake Skills](https://github.com/AltimateAI/data-engineering-skills/tree/main/skills/snowflake) | Skill | Purpose | Key Behaviors | | --- | --- | --- | | **optimizing-query-text** | Performance tuning (from SQL) | Profile query → Identify bottlenecks → Apply patterns | | **optimizing-query-by-id** | Performance tuning (from query ID) | Fetch query from history → Analyze → Optimize | | **finding-expensive-queries** | Cost analysis | Query ACCOUNT\_USAGE → Identify top spenders | --- ## Deep Dive: How creating-dbt-models Changes Claude's Behavior Let's trace through a real task from our benchmark to see the difference. > ### Task: "Create a model called dim\_customer that renames id to customer\_id and makes it a unique primary key." ### Without Skills (Baseline Claude) **Claude's approach:** 1. Write the SQL immediately: ```sql SELECT id AS customer_id, * FROM customers ``` 1. Run `dbt compile` → passes 2. Declare done **Outcome:** Model compiles, but: - Wrong table reference (raw table vs staging model) - No config block matching project patterns - No verification that the output is correct - No `unique` test added for primary key ### With creating-dbt-models Skill ([See the specific SKILL.md file here](https://github.com/AltimateAI/data-engineering-skills/tree/main/skills/dbt/creating-dbt-models) ) **Claude's approach:** **Step 1: Understand Requirements** - Columns needed: `customer_id` (renamed from `id`), plus all other columns - Grain: One row per customer - Constraint: Must be unique on `customer_id` **Step 2: Discover Project Conventions** ```bash $ cat dbt_project.yml # Shows model-paths, naming conventions $ find models/ -name "*.sql" | head -10 models/staging/stg_customers.sql models/staging/stg_products.sql models/marts/dim_products.sql $ cat models/marts/dim_products.sql # Learns: CTEs named 'source' and 'final', config blocks, column naming ``` **Step 3: Find Upstream Models** ```bash $ cat models/staging/stg_customers.sql # Discovers the correct source to reference ``` **Step 4: Write Model (following discovered patterns)** ```sql -- models/marts/dim_customer.sql {{ config( materialized='table' ) }} with source as ( select * from {{ ref('stg_customers') }} ), final as ( select id as customer_id, name, email, created_at from source ) select * from final ``` **Step 5: Update schema.yml with test** ```yaml models: - name: dim_customer columns: - name: customer_id tests: - unique - not_null ``` **Step 6: Build and Verify** ```bash $ dbt build --select dim_customer # Actually runs the model and tests $ dbt show --select dim_customer --limit 5 # Verifies output looks correct ``` **Outcome:** Model matches project conventions, has proper tests, output verified. --- ## Benchmarking: ADE-bench Results We evaluated our skills using [ADE-bench](https://github.com/dbt-labs/ade-bench), a framework for evaluating AI agents on analytics engineering tasks created by dbt Labs. ### Test Setup - **43 tasks** across 5 projects (Airbnb reviews, F1 racing, Asana projects, Analytics engineering, Intercom conversations) - **Task types:** Model creation, bug fixing, debugging, refactoring, data analysis - **Model:** Claude Sonnet 4.5 - **Database:** Snowflake - **Evaluation:** Automated tests comparing model output to expected results ### Task Difficulty Distribution | Difficulty | Example Task | | --- | --- | | **Easy** | "Fix the surrogate\_key deprecation warning." | | **Medium** | "Create a dim\_customer model with unique primary key." | | **Hard** | "Identify which top-N tables have inconsistent results due to tie.s" | ### Overall Results | Configuration | Accuracy | Tasks Resolved | Avg Runtime | Avg Cost | | --- | --- | --- | --- | --- | | Baseline Claude (no skills, no MCP) | 46.5% | 20/43 | 152s | $0.33/task | | Claude + Skills | 53.5% | 23/43 | 182s | $0.40/task | ### Results by Task Category | Category | Baseline | With Skills | Improvement | | --- | --- | --- | --- | | **Model Creation** | 40% | 65% | **+25 pts** | | **Bug Fixing** | 60% | 70% | +10 pts | | **Debugging** | 35% | 50% | +15 pts | | **Refactoring** | 30% | 35% | +5 pts | | **Analysis** | 25% | 30% | +5 pts | ### What Worked **Model creation** saw the biggest improvement. The creating-dbt-models skill's workflow of "discover conventions → write → build → verify" catches errors that baseline Claude misses: 1. **Convention discovery** prevents wrong naming/structure 2. **Mandatory** `dbt build` catches runtime errors that `compile` misses 3. **Output verification** ensures the model produces correct data **Example success — Task** `analytics_engineering003`: > "Create a model called 'dim\_customer' that renames id to customer\_id, and makes that row a unique primary key." - Baseline: Created model but wrong column reference, no test - With skills: Discovered existing staging model, matched project patterns, and added proper unique test ### What Didn't Work **Complex analysis tasks** remain challenging. Tasks requiring deep reasoning about data behavior (like identifying which queries have non-deterministic results due to ties) still need human insight. **Example failure — Task** `f1003`: > "Identify which top-N tables have inconsistent results due to ties in the data" This task requires: 1. Understanding the semantic meaning of "ties" 2. Analyzing actual data values across 8 models 3. Reasoning about SQL ordering behavior Skills can't encode this kind of domain reasoning — they work best for **workflow guidance**, not **analytical judgment**. --- ## The Overhead Trade-off Skills add overhead. The "discover conventions" step takes 15-30 seconds of additional LLM calls. Is it worth it? | Metric | Without Skills | With Skills | | --- | --- | --- | | Avg task time | 152 seconds | 182 seconds | | Success rate | 46.5% | 53.5% | | Time to first success | ~5-6 min | ~3-4 min | | Human intervention needed | High | Low | **Our conclusion:** The 30-second overhead is worth it because: 1. Successful tasks need no human review 2. Failed tasks fail faster (3-failure rule) 3. Time saved on human review >> time spent on convention discovery ## Benchmarking: SQL Query Optimization We evaluated the `optimizing-query-text` skill on TPC-H SF1000 (1TB dataset). ### Test Setup - **10 queries** from TPC-H benchmark - **Model:** Claude Sonnet 4.5 - **Evaluation:** Automated comparison of query results + execution time ### Overall Results | Configuration | Pass Rate | Avg Time Improvement | | --- | --- | --- | | Baseline Claude (no skills) | 80% (8/10) | +25% (on passing queries) | | Claude + Skills | **100% (10/10)** | +22% | ### What Failed Without Skills Baseline failed 2 queries by making "optimizations" that changed what the query returned: - Changed deduplication behavior, returning extra rows - Renamed columns, breaking downstream compatibility --- ## Installation & Usage ### Add the Marketplace ```bash /plugin marketplace add AltimateAI/data-engineering-skills ``` ### Install Skills You can browse and install via the CLI, or directly install plugins: ```bash # Install dbt skills /plugin install dbt-skills@data-engineering-skills # Install Snowflake skills /plugin install snowflake-skills@data-engineering-skills ``` After installing, skills activate automatically when you mention relevant tasks. ### Available Skills **dbt Skills:** - `creating-dbt-models` — Model creation with convention discovery - `debugging-dbt-errors` — Systematic error troubleshooting - `testing-dbt-models` — Schema tests and data quality - `documenting-dbt-models` — Generate descriptions in schema.yml - `migrating-sql-to-dbt` — Convert legacy SQL to dbt models - `refactoring-dbt-models` — Safe restructuring with impact analysis **Snowflake Skills:** - `optimizing-query-text` — Optimize SQL you provide - `optimizing-query-by-id` — Optimize using query ID from history - `finding-expensive-queries` — Find top cost/time queries ### Usage Skills activate automatically based on your request: | Your Request | Skill Activated | | --- | --- | | "Create a new orders model." | `creating-dbt-models` | | "Fix this compilation error." | `debugging-dbt-errors` | | "Add tests to the customers model." | `testing-dbt-models` | | "Document the revenue metrics". | `documenting-dbt-models` | | "This query is slow; optimize it." | `optimizing-query-text` | | "Why is query X expensive?" | `optimizing-query-by-id` | | "What are our most expensive queries?" | `finding-expensive-queries` | --- ## Combining Skills with Altimate MCP Tools Skills become even more powerful when combined with [Altimate's MCP server](https://docs.myaltimate.com/). The MCP server provides real-time access to your dbt project and data warehouse: | MCP Tool | What It Provides | | --- | --- | | `dbt_project_info` | Project structure, model list, sources | | `dbt_model_details` | Column types, dependencies, compiled SQL | | `dbt_compile` | Compile models without CLI | | `snowflake_query_history` | Recent query executions and stats | | `snowflake_table_stats` | Row counts, clustering info | **Example: Skills + MCP workflow** User: "The daily\_revenue model is producing wrong numbers." Claude (with skills + MCP): 1. **debugging-dbt-errors skill activates** 2. Uses `dbt_model_details` to get model SQL and dependencies 3. Uses `dbt_compile` to check for errors 4. Queries upstream models to verify input data 5. Identifies the issue (e.g., missing WHERE clause) 6. Fixes and rebuilds 7. Uses `dbt show` to verify the correct output --- ## What We Learned Building This ### 1\. Workflow > Knowledge The biggest wins came from encoding **workflows**, not facts. Claude already knows dbt syntax — what it lacks is the discipline to: - Check existing patterns before writing - Run `dbt build` instead of `dbt compile` - Verify output after build ### 2\. The 3-Failure Rule We added this to every skill: > "If build fails 3+ times, STOP. Step back and reassess your entire approach." This prevents Claude from making tiny tweaks hoping they work. Instead, it forces a fundamental rethink. ### 3\. Skills Can't Replace Domain Expertise Skills work best for **procedural tasks** with clear success criteria. They struggle with: - Tasks requiring business context - Ambiguous requirements ("make this better") - Deep analytical reasoning about data behavior ### 4\. Convention Discovery Is Essential The #1 source of Claude errors was mismatched conventions. Simply adding "read 2-3 existing models first" eliminated most of these. --- ## What's Next We're actively developing: - **Airflow skills** — DAG development, debugging, testing - **Cross-platform migration** — dbt ↔ SQL Server, Oracle - **Snowflake cost optimization** — Warehouse sizing, query patterns - **Data quality workflows** — Anomaly detection, freshness checks ### Contributing Altimate Skills is open source (MIT License). We welcome: - **New skills** for workflows we haven't covered - **Improvements** to existing skills based on your team's patterns - **Benchmark results** on different datasets **GitHub:** [https://github.com/AltimateAI/data-engineering-skills](https://github.com/AltimateAI/data-engineering-skills) --- ## Try It Now ```bash # Add the marketplace /plugin marketplace add AltimateAI/altimate-skills # Install the plugins you need /plugin install dbt-skills@altimate-skills /plugin install snowflake-skills@altimate-skills ``` **Resources:** - [GitHub Repository](https://github.com/AltimateAI/data-engineering-skills) and ⭐ the repo. - [Altimate MCP Server Docs](https://docs.myaltimate.com/) - [ADE-bench Framework](https://github.com/dbt-labs/ade-bench) - [dbt Slack #tools-dbt-power-user](https://app.slack.com/client/T0VLPD22H/C05KPDGRMDW) --- *Built by the team at* [*Altimate AI*](https://altimate.ai/) *— Making data engineering delightful.* Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [Altimate](https://altimate.ai/blog?tag=altimate) [Data Engineering](https://altimate.ai/blog?tag=data-engineering) [AI Tools](https://altimate.ai/blog?tag=ai-tools) [Snowflake](https://altimate.ai/blog?tag=snowflake) [dbt](https://altimate.ai/blog?tag=dbt) [AI](https://altimate.ai/blog?tag=ai) [claude.ai](https://altimate.ai/blog?tag=claude.ai) [Dev Tools](https://altimate.ai/blog?tag=dev-tools) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Adaptive Compute vs. Altimate AI Auto Tune Optimization URL: https://altimate.ai/blog/adaptive-compute-vs-auto-tune-a-practical-guide-to-optimizing-snowflake-warehouses Optimize Snowflake warehouses with Adaptive Compute and Altimate Auto Tune solutions. Learn their functionalities, tradeoffs, and decide based on your needs [Back to blog](https://altimate.ai/blog) Altimate · Data Engineering · Snowflake ## Adaptive Compute vs. Auto Tune: A Practical Guide to Optimizing Snowflake Warehouses Dana Van Aken Jul 30, 2025 Adaptive Compute vs. Auto Tune: A Practical Guide to Optimizing Snowflake Warehouses > **Updated 16 Sep 2026.** Snowflake made Adaptive Compute generally available on AWS on 16 June 2026. The GA release renamed the settings this post describes. For the current parameters, billing and conversion limits, read [Adaptive Compute vs Auto-Tune: Optimizing Snowflake Warehouses](https://altimate.ai/blog/adaptive-compute-vs-auto-tune-optimizing-snowflake-warehouses). Managing Snowflake warehouses efficiently is a constant balancing act. Set them too large, and you're burning money on idle compute. Too small, and your queries slow to a crawl, frustrating users and missing SLAs. For many organizations, this manual tuning process is time-consuming while still leaving significant cost savings on the table. Two solutions promise to solve this challenge in very different ways: Snowflake's new **Adaptive Compute** eliminates warehouse management complexity entirely, while **Altimate AI's Auto Tune** brings intelligent automation to standard warehouses with granular cost controls. This guide breaks down how each works, their tradeoffs, and which to choose based on your specific needs. ## What is Adaptive Compute? Today, managing Snowflake warehouses requires juggling multiple decisions: choosing the right size (X-Small through 6X-Large), configuring cluster policies, setting auto-suspend timers, and more. Get these wrong, and you're likely to be overpaying for idle compute or suffering from poor query performance. **Adaptive Compute** changes this equation entirely. Instead of fixed-size warehouses, **Adaptive Warehouses** act as pointers to a shared, elastic compute pool that Snowflake manages behind the scenes. When you submit a query to an Adaptive Warehouse, Snowflake automatically: - Analyzes the query plan and resource requirements - Allocates the right amount of compute from the shared pool - Routes the query to available resources - Scales compute up or down based on actual needs Think of it like switching from buying dedicated servers to using serverless functions – you stop worrying about infrastructure and focus on your workloads. With Adaptive Warehouses, customers still interact with named warehouses, but under the hood, each warehouse is simply a pointer to a shared compute pool within the account. Queries are automatically routed to appropriately sized compute clusters based on their resource needs and availability. ***Figure 1:*** *With Adaptive Warehouses, customers still interact with named warehouses, but under the hood, each warehouse is simply a pointer to a shared compute pool within the account. Queries are automatically routed to appropriately sized compute clusters based on their resource needs and availability.* ### From Many Settings to Just Two Snowflake’s standard warehouses require configuring multiple parameters: - Warehouse size - Minimum and maximum cluster count - Scaling policy (Standard or Economy) - Auto-suspend timing - Auto-resume behavior - Query Acceleration Adaptive Compute replaces all of these with just two settings: 1. **Warehouse Credit Limit:** The maximum credits the warehouse can consume per hour. This acts as your primary control lever, determining how many concurrent queries can run based on their compute requirements. When the warehouse reaches its credit limit, additional queries are queued until resources become available. 2. **Target Statement Size (temporary):** An optional hint about your expected query sizes. Currently, Adaptive Compute only supports scaling down for smaller queries, and this setting defines the maximum target size. Note that Snowflake intends to deprecate this setting once Adaptive Compute’s autoscaling capabilities mature\[1 \], so don’t build processes around it. Side-by-side Snowflake warehouse creation dialogs: a Standard warehouse with Auto-resume, Auto-suspend and multi-cluster settings on the left, next to an Adaptive warehouse with only a credit limit and target statement size field on the right. ***Figure 2:*** *Adaptive Compute reduces configuration complexity by replacing multiple tuning settings with just two controls: a Warehouse Credit Limit and an optional Target Statement Size.* If you convert a standard warehouse to Adaptive, Snowflake sets the initial Warehouse Credit Limit based on the formula: `Warehouse Credit Limit = Current Warehouse Size Credits × Max Cluster Count`. It also sets the Target Statement Size equal to the original warehouse size. For example, if you convert a Large warehouse (8 credits/hour) with `max_clusters=4` to Adaptive, you get: - **Warehouse Credit Limit:** 32 credits/hour (8 credits × 4 clusters) - **Target Statement Size:** Large Your Adaptive warehouse can then concurrently run queries totaling up to 32 credits per hour; for example, four Large queries, sixteen Small queries, or any combination fitting within the credit limit. ## Where Adaptive Compute Delivers ### **Simplified Infrastructure Management** Adaptive Compute removes much of the manual setup required to manage Snowflake warehouses, freeing up data teams to concentrate on driving business outcomes and delivering innovation. This hands-off model is especially beneficial for platform teams managing large fleets of warehouses, and less technical users who don’t want to think about warehouse tuning. ### **Smarter Resource Allocation** Because Snowflake's optimizer has access to detailed query plan metadata (e.g., estimated row counts, join strategies, data volumes), it can predict resource needs more accurately than external tools. Assigning just enough CPU and memory per query also reduces the likelihood of over-provisioning. **Example scenario:** Two very different queries are submitted to the same Adaptive Warehouse simultaneously: | **Query** | **Description** | **Predicted Compute Needs** | | --- | --- | --- | | A | Complex 5-way join with 100M rows | Gets Large-sized compute | | B | Simple filter on 1000 rows | Gets X-Small-sized compute | As shown in the table, Adaptive Compute predicts greater compute needs for the Query A (the heavier query) than Query B (the simpler query). ### **Seamless Conversion from Standard Warehouses** Converting to Adaptive Compute is simple and requires no downtime: `ALTER WAREHOUSE my_warehouse SET TYPE = 'ADAPTIVE';` Running queries complete on the old compute while new queries immediately use adaptive resources. There’s no need to update connection strings, modify scripts, or coordinate downtime windows. ### **Solving Warehouse Sprawl** A common issue is warehouse sprawl, where companies create separate warehouses for different teams, workload types, or usage patterns. For example, a marketing team might operate five different warehouses: - `MARKETING_REPORTS_L` (idle 22 hours/day) - `MARKETING_ETL_XL` (runs 2 hours nightly) - `MARKETING_DASHBOARD_L` (idle 20 hours/day) - `MARKETING_ADHOC_M` (sporadic usage) - `MARKETING_TEST_XS` (barely used) **The Problem:** Managing many separate warehouses leads to unnecessary costs due to idle compute time and minimum billing increments. Even quick 5-second queries incur charges for a full minute of compute. Across hundreds of warehouses, this adds up to significant wasted spend each month. **The Adaptive Solution:** With Adaptive Compute, the marketing team can convert each of their five warehouses to Adaptive, and still maintain the same warehouse names for accounting purposes. The shared compute pool ensures resources are available when needed without incurring idle time charges across five warehouses. ## Critical Unknowns: What's Still Unclear While Adaptive Compute offers clear operational benefits, it’s still in private preview and several important details remain unclear. And these details matter, especially for organizations under tight cost or performance SLAs. ### **Limited Cost Control Mechanisms** Adaptive Compute optimizes for performance and simplicity, not cost reduction. If Target Statement Size is deprecated as planned\[1 \], you'll have only one lever: the Warehouse Credit Limit. Without any settings to guide compute size per query, cost-saving strategies like pushing low-priority workloads to smaller compute are no longer possible. **Consider this scenario:** Your team runs a daily report that takes 2 minutes on Small compute (2 credits/hour = 0.067 credits) or 1.5 minutes on Medium compute (4 credits/hour = 0.10 credits). With standard warehouses, your team can choose Small to save 33% on credits, but Adaptive Compute might decide on Medium due to the speed improvement, increasing your spend regardless of your priorities. **What you lose:** - Ability to force non-critical workloads onto smaller compute to save money (e.g., running non-critical reports on X-Small warehouses) - Option to set aggressive auto-suspend for development warehouses (e.g., setting 1-minute auto-suspend for ad-hoc warehouses) - Control over running specific workloads on larger warehouses when deadlines require it ### **Cache Behavior Uncertainty** Snowflake’s local disk cache can significantly boost query performance, with warm-cache queries often running 5–10x faster than cold ones. Standard warehouses with longer auto-suspend settings are more likely to retain this cache between queries. But with Adaptive's shared compute model, cache behavior becomes unpredictable, potentially impacting both performance and costs. **Why this matters:** - Dashboard queries that typically benefit from warm cache might see inconsistent performance - ETL pipelines that read the same base tables repeatedly could lose optimization opportunities - Cost implications if queries that previously hit cache now require full table scans ### Monitoring and Debugging Challenges With standard warehouses, you can directly observe and control which warehouse a query runs on. With Adaptive Compute, the routing decisions happen behind the scenes, and it’s unclear what additional metadata Snowflake will surface around how those routing decisions are made, such as: - What size compute ran each query and how routing logic behaved - Whether the router over‑ or under‑sized compute for specific queries - Whether queries are being queued due to credit limits or resource availability ### **Impact on Spend** Snowflake's launch materials for Adaptive Compute emphasize ease of use and performance, with no mention of cost savings\[2, 3, 4 \]. This positioning suggests Adaptive Compute isn't designed to reduce spend, which is a critical consideration for organizations with FinOps accountability or tight budget constraints. ## How Altimate AI's Auto Tune Delivers Control and Automation For organizations that want automation without sacrificing cost control, Altimate AI's Warehouse Auto Tune offers a powerful alternative. It brings intelligent optimization to your existing standard warehouses to optimize size, idle time, and cluster configuration while providing even finer-grained control over speed versus savings. Altimate AI Auto Tune dashboard, Performance tab, daily bar chart of Snowflake spend and savings from Jun 26 to Jul 27, 2025, with a tooltip on Jul 23 showing $263.40 total cost and $34.85–$45.38 in Auto Tune savings. ***Figure 3:*** *Daily cost breakdown showing Auto Tune savings over time. Each bar displays actual spend (yellow) and realized savings (green) from warehouse suspension, scaling, and sizing optimizations. On July 23, 2025, Auto Tune saved between $34.85 and $45.38, with most savings from Warehouse Suspension and Scaling. Total realized savings during the selected period exceeded $1.1K.* ### **Proactive Suspension and Cluster Scaling** Auto Tune minimizes idle time by scaling back clusters and suspending warehouses before Snowflake's native auto-suspend triggers. It continuously monitors activity and proactively reduces compute allocation during quiet periods. ### **Predictive Workload-Based Resizing** Auto Tune doesn't just react to current load – it predicts upcoming demand based on your historical workload patterns to determine when warehouses can be safely downsized without impacting performance. **How it works in practice:** Let's say you have a Large warehouse that processes heavy ETL jobs from 2-6 AM but runs mostly small queries the rest of the day. Auto Tune identifies this workload pattern and automatically: - Maintains Large size during the 2-6 AM ETL window - Downsizes to Medium during low-demand periods - Returns to Large before the next ETL window ### **Granular Performance Safeguards** Auto Tune provides multiple mechanisms to protect critical workloads while optimizing costs: **1\. Custom Schedule Blocks** Custom schedule blocks define specific time windows where warehouses must maintain a certain size. Examples: - Weekdays 9-10 AM: Keep `EXECUTIVE_DASHBOARD_WH_XL` at default size X-Large for C-suite daily reviews to ensure performance - Sunday 2-5 AM: Keep `ARCHIVE_ETL_WH` downsized to Small while reprocessing past months of data to reduce costs **2\. Real-Time Backoff Configuration** Set performance thresholds that automatically reverse downsizing decisions if query latency or queue times exceed acceptable levels. When triggered, the system reverts to default size and pauses auto-resizing for one hour. **Example Backoff Configurations for Different Workload Types:** | **Workload Type** | **Latency Threshold** | **Queue Threshold** | **Rationale** | | --- | --- | --- | --- | | Customer-facing dashboards | 120% | 125% | Minimal tolerance for delays | | Internal analytics | 150% | 150% | Balanced cost/performance | | Batch ETL jobs | 180% | 170% | Cost savings prioritized | ### **Transparency and Control Over Decisions** Auto Tune provides full visibility into every optimization decision: - Daily/weekly/monthly spend trends with savings clearly separated by agent - Detailed logs explaining why each resizing decision was made - Calendar view of all scheduled and completed optimization windows Auto tune history timeline listing warehouse resize and suspend events by DataPilot, with one entry expanded showing the reasoning for downsizing from Large to Medium based on light query workload. ***Figure 4:*** *Auto Tune history log showing recent warehouse actions taken. The agents resized warehouses dynamically based on query patterns, including downsizing from Large to Medium size to reduce cost without impacting performance.* ### **Warehouse Eligibility and Smart Targeting** Auto Tune automatically evaluates each warehouse to determine if it's a good candidate for resizing optimization. Eligible warehouses typically have: - Periods of low utilization with mainly small queries - Predictable daily or weekly patterns - Meaningful compute spend to optimize This ensures resizing agents only focus on warehouses where they can deliver real value and help cut costs. ## Making the Right Choice: Adaptive vs. Auto Tune Here are a few pointers for choosing the right solution. ### **Choose Adaptive Compute When:** - **Simplicity is paramount:** You want to eliminate warehouse management entirely - **Workloads are unpredictable:** Highly variable query patterns make sizing difficult - **Performance matters most:** You're willing to pay more for consistent speed - **You're consolidating many underutilized warehouses:** The reduction in idle time offsets other costs ### **Choose Auto Tune When:** - **Cost control is critical:** You have specific spending targets to hit - **Workloads follow patterns:** Your queries have predictable daily/weekly cycles - **You need granular control:** Different workloads require different cost/performance tradeoffs - **You want transparency:** Detailed visibility into optimization decisions and savings ### **Hybrid Approach** **Many organizations will benefit from using both solutions strategically.** For example: - Convert your `DATA_SCIENCE_SANDBOX` warehouse to Adaptive Compute since data scientists run unpredictable queries at random times - Keep your `PRODUCTION_ETL` warehouse on standard with Auto Tune, reducing spend through predictable downsizing during known quiet periods - Use Adaptive for all development/test warehouses to eliminate idle charges - Apply Auto Tune to customer-facing warehouses where you need cost control with performance guarantees ## Final Thoughts Snowflake's Adaptive Compute and Altimate AI's Auto Tune represent two philosophies for warehouse optimization: total simplicity versus intelligent control. Adaptive Compute eliminates management overhead but sacrifices cost optimization levers. Auto Tune maintains the flexibility of standard warehouses while adding automation that typically reduces costs by 10-20%. Start by analyzing your warehouse utilization patterns and cost targets. If predictable savings matter more than operational simplicity, Auto Tune is likely your best choice. If you're drowning in warehouse management complexity and can accept higher costs for simplicity, Adaptive Compute offers compelling benefits. For most organizations, a hybrid approach will deliver the best of both worlds. If your team already has tools in place for analyzing warehouse usage patterns, monitoring cost targets, and automating warehouse optimization, you're all set. If not, we can help you with the analysis, optimization, and choosing between Adaptive and standard warehouses based on your needs. [Reach out to us](https://app.myaltimate.com/contactus) for a no-cost POC. ## References 1. [What's New: Faster Insights with Snowflake Standard Warehouse - Gen2 and Adaptive Compute](https://reg.snowflake.com/flow/snowflake/summit25/sessions/page/catalog/session/1741712659023001GszH). Snowflake Summit 2025 Session Catalog. June 3, 2025. 2. [Snowflake Unveils Next Wave of Compute Innovations For Faster, More Efficient Warehouses and AI-Driven Data Governance](https://www.snowflake.com/en/news/press-releases/snowflake-unveils-next-wave-of-compute-innovations-for-faster-more-efficient-warehouses-and-ai-driven-data-governance/). Snowflake press release, June 3, 2025. 3. [Introducing Even Easier-to-Use Snowflake Adaptive Compute with Better Price/Performance](https://www.snowflake.com/en/blog/adaptive-compute-smarter-warehouses/). Snowflake Blog, June 3, 2025. 4. [How Snowflake’s Adaptive Warehouse Is Revolutionizing Data Operations at Pfizer](https://medium.com/snowflake/how-snowflakes-adaptive-warehouse-is-revolutionizing-data-operations-at-pfizer-f90f1730ef0c). Snowflake Builders Blog on Medium; Matt Massey, June 3, 2025. --- Deliver more performance and cost savings with Altimate AI's cutting-edge AI teammates. These intelligent teammates come pre-loaded with the insights discussed in this article, enabling you to implement our recommendations across warehouses, tables, and millions of queries effortlessly. Ready to see it in action? Request a recorded demo by [sending us a chat message](https://app.myaltimate.com/contactus) ) and discover how AI teammates can transform your data teams. [Promotional banner reading ‘AI Data Teammates that work for you 24/7’ beside a screenshot of the Auto Tune savings dashboard, with a Book a Demo button.](https://app.myaltimate.com/contactus) Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [Altimate](https://altimate.ai/blog?tag=altimate) [Data Engineering](https://altimate.ai/blog?tag=data-engineering) [Snowflake](https://altimate.ai/blog?tag=snowflake) [Analytics](https://altimate.ai/blog?tag=analytics) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Power User for dbt's Embedded MCP Server for Cursor URL: https://altimate.ai/blog/supercharging-cursor-ide-how-the-dbt-power-user-extensions-embedded-mcp-server-unlocks-ai-driven-dbt-development Power User for dbt runs an embedded MCP server in VS Code, so the AI assistant in Cursor can query your dbt schema, compile models, and run tests. [Back to blog](https://altimate.ai/blog) Dev Tools · dbt · Data Engineering ## Supercharging Cursor IDE: How the dbt Power User Extension’s Embedded MCP Server Unlocks AI-Driven dbt Development Anand Gupta Mar 21, 2025 Supercharging Cursor IDE: How the dbt Power User Extension’s Embedded MCP Server Unlocks AI-Driven dbt Development ## Introduction We’re excited to announce a major new capability in the **dbt Power User** VSCode extension: an embedded **Model Context Protocol (MCP)** server. This MCP server is now built directly into the extension, acting as a bridge between AI-powered development tools and your dbt project. The motivation behind implementing an embedded MCP server is to enable seamless communication between AI assistants (like those in Cursor IDE) and dbt, eliminating the friction of context switching. In practical terms, this means your AI coding assistant can query your dbt project’s schema, compile models, run tests, and more – all through a standardized protocol – leading to faster development workflows and greater efficiency for dbt developers. By leveraging the MCP server, dbt Power User can expose rich project context and operations to AI agents in real-time. This empowers data engineers to automate repetitive tasks (like looking up column definitions or running model builds) and accelerates development. The embedded MCP server unlocks new AI-driven workflows for dbt, from intelligent autocompletion to on-demand documentation generation, all while keeping the interaction localized and secure within your development environment. ## Technical Architecture and Cursor IDE Integration At a high level, the MCP server runs as a lightweight web service within the dbt Power User extension environment. The server spins up automatically when you open a dbt project (if you have enabled the MCP feature) and listens on a local port. It selects an available port dynamically at launch to avoid conflicts, and registers itself with Cursor by updating Cursor’s MCP configuration. Once running, the Cursor IDE detects the MCP server and establishes a connection so its AI assistant can invoke dbt-related “tools” exposed by the server. Architecture diagram of the dbt Power User MCP server: Cursor IDE’s user interface and MCP Client connect over SSE to the MCP server’s Service Core and Tool Registry, which drives a dbt Project Container that parses Catalog.json, Manifest.json and Project Files, calls dbt Core and dbt Cloud, and watches the file system for changes. **Server Components and Design:** The MCP server architecture is composed of a few key components working in tandem: - **Server Core:** the core HTTP server (built with the MCP SDK) that handles incoming requests and responses. It defines the MCP endpoints and protocol handling (e.g. registering available tools, managing sessions, etc.). The server core orchestrates requests from Cursor’s AI and ensures each is routed to the appropriate handler in the extension. - **Tool Registry:** a registry of **tools** – individual operations or actions related to dbt. Each tool has a name and a function implementing its behavior. When the server starts, it registers a suite of tools (detailed in the next section) that allow the AI to perform various dbt tasks. The Tool Registry essentially maps MCP requests to the extension’s internal methods or dbt CLI calls. It also defines metadata for each tool (like input parameters and descriptions) that Cursor’s AI can utilize to formulate requests. - **SSE Transport:** the server uses **Server-Sent Events (SSE)** for real-time communication back to the client. After Cursor connects to the MCP server, it maintains an open SSE channel. When a tool is executed, results (and any interim output or logs) are sent as a stream of events over this channel to Cursor. This means the AI can receive data or status updates from long-running tasks in real time, without polling. For example, if you execute a model run via the AI, the MCP server can stream back logs or a success/failure message as soon as it’s available. Using SSE for output ensures a responsive, event-driven interaction between the extension and the AI assistant. Simplified diagram of Cursor IDE’s AI assistant exchanging MCP tool requests with the MCP server, which queries the dbt Artifact Manager and runs models through the dbt Integration to reach the data warehouse. *Architecture of the MCP Server integration with Cursor.* ***The diagram shows the Cursor IDE’s AI assistant interacting with the MCP server via tool requests.*** *The MCP server (within the dbt Power User extension) processes these requests using its registered tools. It interacts with in-memory dbt project artifacts for quick context and, when needed, invokes the dbt Integration module (which in turn may query the data warehouse) to fulfill the request. Results and logs are returned back to the AI client in real-time via SSE.* **Communication Flow:** The end-to-end flow of a typical interaction involves several steps that connect the AI to your dbt project: 1. **AI Initiates a Tool Request:** Within Cursor, the AI assistant decides to call a tool (for example, “get the SQL definition of model X” or “run tests for model Y”) and sends a request through the MCP protocol. 2. **MCP Server Receives the Request:** The server core matches the incoming request to one of the registered tools in its registry and hands off execution to that tool’s handler. 3. **Tool Execution:** The tool’s handler runs inside the extension process. It may fetch information from the in-memory dbt artifacts (e.g. to get a model’s definition or lineage) or execute a dbt operation. The handler leverages the extension’s existing capabilities – for example, calling a function in the extension that wraps dbt operations to perform the task. 4. **In-Memory Artifact Access:** If the tool needs to read project structure (models, sources, tests, etc.), it queries the up-to-date in-memory representation of the dbt project (more on this in the next section). This avoids expensive disk or CLI operations for retrieving metadata. If the tool needs to run SQL or dbt operations, it might invoke the dbt integration module and use the database connections configured in the dbt profile. In essence, the MCP server turns the dbt extension into a real-time API for your dbt project. Cursor’s AI no longer operates in a vacuum – it can actively query your project’s state and perform operations, treating the dbt project as an interactive context it can reason about. All of this happens locally within your development environment, leveraging the work the extension is already doing to maintain dbt context. ## In-Memory dbt Artifacts Lifecycle and Performance Optimization One of the standout benefits of the MCP server is that it can serve data to the AI assistant **directly from memory**, without constantly hitting your filesystem or database. The dbt Power User extension maintains an in-memory representation of your project’s artifacts as you work. This includes the parsed project manifest (models, sources, tests, exposures, etc. along with their relationships), and metadata like the active target and project name. By tapping into these in-memory artifacts, the MCP server can answer many questions instantly and run operations more efficiently. **Creation and Initialization:** When you open a dbt project in VS Code, the extension immediately initializes and parses the project. It loads manifest information (by invoking dbt’s parse operation under the hood and translating it to its own in-memory representation through its own parser) to build a graph of models and references in memory. This means the extension knows about all your model definitions, sources, tests, and their lineage as soon as the project is ready. At the moment the MCP server starts (which typically coincides with project load or when you enable the feature), it attaches to this ready-to-use project context. The server core may wait until the dbt project is fully initialized and then mark its tools as available for the AI. Essentially, **the server boots up after the dbt context is prepared**, ensuring the AI always sees a consistent project state. **Live Updates:** As you modify files or the project state changes, the in-memory artifacts are updated incrementally. The extension listens to file system events and dbt execution outcomes – for example, if you edit a model file, it can re-parse that model and update the manifest graph in memory. Likewise, running `dbt deps` will refresh the list of installed packages in memory. The MCP server subscribes to these internal events. If the manifest is updated or a new run is completed, the server can invalidate or refresh relevant cached data. This way, tools like “get children models of X” will always use the latest information. The lifecycle is managed such that **any time the dbt project changes, the in-memory state and the MCP server’s view of it remain in sync**. This is much faster than re-running `dbt` commands for each query and ensures the AI doesn’t get stale data. **Efficient Querying:** Because the data is in memory, tools that retrieve info (e.g. listing models, showing a model’s SQL, computing lineage) execute extremely quickly. There’s no need to spawn a subprocess or read from disk for these – the extension can simply look up the Python/TypeScript object representing that model. Even complex lineage traversals or dependency graphs can be resolved by the extension’s code using pre-computed relationships. This optimization is critical for performance: it allows the AI to ask a series of detailed questions about your project without incurring heavy overhead. For example, an AI reasoning about why a model’s test failed can rapidly pull the model’s SQL, its parent models, and the test definition via multiple MCP tool calls in seconds, whereas doing this via CLI repeatedly would be far slower. **Cleanup and Lifecycle End:** The MCP server’s lifetime is tied to your VS Code session and project. If you close the project or disable the extension, the server is stopped and the in-memory artifacts are freed. The implementation ensures that sockets are closed and event listeners are disposed to avoid resource leaks. Notably, the server core exposes a clean **dispose()** method that the extension calls on shutdown, which shuts down the SSE stream and prevents further tool calls. Overall, the approach of leveraging in-memory artifacts means that the MCP server can provide a **fast, rich context** to the AI. It capitalizes on the work already being done by the dbt Power User extension’s internal DBT parser and project tracker. This not only speeds up AI interactions but also avoids unnecessary load on your data warehouse – the AI can get a lot of information without executing a single query, unless absolutely necessary for the task. ## Capabilities Unlocked: Tools Provided by the MCP Server The MCP server exposes a robust set of tools that the AI assistant can use. These tools cover most day-to-day dbt tasks and are grouped into a few categories for clarity. Below is a breakdown of major capabilities and examples of tools in each category: ### Project & Environment Management Tools - **Project Information:** Usually the AI will fetch all the dbt projects and its roots first, to know on which project it should work. The AI will use the root of the project as a parameter to the other tools. For instance, a tool `GET_PROJECT_NAME` returns the dbt project name for the given root, and `GET_SELECTED_TARGET` returns which target/profile is active. This is useful for the AI to contextually understand which environment it’s working with (production or development). ### Model and Source Exploration Tools - **Model SQL and Schema:** Tools in this category allow the AI to fetch the content or properties of dbt models and sources. For example, a `COMPILE_QUERY` tool can return the raw SQL of a model file, and a `GET_COLUMNS_OF_SOURCE` could return details about a source (like database/schema and columns as defined in YAML). With these, the AI can, say, read the logic of a model to help debug or suggest changes. - **Lineage and Dependency Queries:** We provide tools like `GET_CHILDREN_MODELS` and `GET_PARENT_MODELS`, which return the immediate downstream and upstream models of a given model. The AI can use these to understand the dependency graph – for instance, finding what will be impacted if a model is changed or which upstream data sources feed into it. Internally, these tools leverage the manifest graph in memory to quickly compute the relationships. Additionally, there are count-based queries (like a tool to count the number of models depending on a source) to aid impact analysis. - **Column Lineage and Documentation:** (Upcoming) The extension already has features for column-level lineage and documentation. Through MCP, we intend to expose tools so the AI can answer questions like “which models use column X from source Y” or even retrieve documentation strings for a model or column. This will bridge the gap between documentation and code by letting the AI pull in documented context as needed. ### SQL Compilation & Execution Tools - **Compile SQL:** The AI might want to see how dbt renders a model’s SQL with Jinja templating. The `COMPILE_QUERY` tool takes a query (and optionally the model it is derived from) and returns the compiled SQL, exactly as dbt would generate it. This is extremely useful for checking logic or debugging macros – the AI can get the SQL that would run on the warehouse, without actually running it. It uses dbt’s compilation engine (via the extension’s interface to dbt Core) under the hood. - **Execute Arbitrary SQL:** Sometimes the AI will need to run a query to answer a question (for example, “preview the first 10 rows of model X”). The `EXECUTE_SQL` tool (with a configurable row limit) allows running a SQL query against the project’s data warehouse connection and returns results, ensuring we don’t pull massive datasets by accident. The AI could use this to validate assumptions or provide sample data to the user. Users must explicitly permit this feature, as executing SQL will share data with the AI. - **Run dbt Models:** For full pipeline execution, we offer tools to run or build models. The `RUN_MODEL` tool triggers `dbt run` for a specific model (and optionally its dependencies), while `BUILD_MODEL` can run the model and its tests (equivalent to `dbt build` for that model). There’s also a `BUILD_PROJECT` for running the entire project (`dbt build`). These commands execute in the background through the extension’s job runner. The AI can thus initiate a model run or a full project build from within the conversation – for example, “Re-run the revenue model to see if the issue is fixed.” ### Testing and Validation Tools - **Execute Tests:** The server provides a `RUN_TEST` tool to execute a specific dbt test by name, as well as `RUN_MODEL_TEST` to run all tests associated with a particular model. This allows the AI to verify data quality or detect if recent changes broke any tests. For instance, after modifying a model, the AI might call `RUN_MODEL_TEST` to ensure all tests still pass and then report the results. Test results or failures will be returned via the event stream. - **SQL Validation:** (Planned) Even without running on the warehouse, the extension can do a lightweight SQL parse/validation. A tool is available to validate a SQL query (checking for syntax errors, missing refs, etc.) without executing it. This is similar to the extension’s existing **SQL Lint** feature, now accessible to the AI. It can prevent the AI from running a bad query by first validating it and catching errors. - **Performance Analysis:** (Planned) We aim to add tools that use dbt’s compilation stats or query planner insights to help the AI assist in performance tuning. For example, a tool to show the **compiled query explain plan** (if the warehouse supports `EXPLAIN`) or to estimate the cost of a query. Such tools would let the AI caution the user if a query might be expensive or if a model lacks proper pruning filters, etc. ### Package Management and Repository Tools - **Install Packages:** Managing dbt packages is made easier with tools like `INSTALL_DEPS` which runs `dbt deps` to install the packages listed in `packages.yml`. There’s also an `ADD_DBT_PACKAGES` tool that can add new packages on the fly. For example, if the AI suggests using a package (like dbt-utils), it could call `ADD_DBT_PACKAGES` with the package name and version – the extension will add it to `packages.yml` and run `dbt deps` internally to install it, returning the result. This showcases agentic behavior: the AI can not only suggest but also take action to modify your project structure (with your permission). - **Upgrade dbt Version:** Although not explicitly a tool in this release, the infrastructure could allow the AI to assist in upgrading the dbt version or dependencies. For example, by reading `requirements.txt` or checking the installed dbt version, and then running pip installs. This would likely be added as a safe tool in future once we ensure such actions are gated behind user approval. - **Git Operations:** While primarily focused on dbt tasks, the larger vision includes enabling some project-level Git interactions via MCP (for example, checking out a new branch for a fix, or opening a pull request). The current server does not directly include Git tools, but it lays the groundwork for integrating such capabilities alongside dbt tasks, so the AI could orchestrate end-to-end workflows (code change -> run model -> test -> commit). All these tools are defined in the extension and registered with the MCP server at startup. When Cursor’s AI is formulating a plan to help you, it queries the MCP server for an index of available tools (the MCP protocol allows the client to discover tools and their input schema). The AI sees something like “I have a tool called `get_children_models` that needs a `table` name” and it can then decide to use it if the user asks a question about model dependencies. ## Under the Hood: Port Management, Error Handling, and Security Building an embedded server inside an IDE extension comes with its own engineering challenges. We put significant effort into making sure the MCP server runs smoothly without disrupting your workflow. **Port Management:** Since the MCP server is local, it needs to choose a port to listen on. We default to letting the OS assign an open port (by binding to port 0, which yields an ephemeral port). Once the server is listening, the extension logs the chosen port. If for some reason the port is not reachable or Cursor cannot connect, the extension will surface an error message. We also handle the case where the server might need to restart (for example, if you re-initialize the project) by cleaning up the old server and starting a new one potentially on a new port. The extension ensures there is only one MCP server instance per workspace to avoid port conflicts. In the future, we may allow a fixed port configuration if users want to expose the server to other tools, but in this beta it’s fully managed and isolated. **Error Handling and Resilience:** The MCP server includes robust error handling so that failures in tools do not crash the server or hang the AI. Each tool execution is wrapped in try/catch logic. If a tool throws an exception (say, dbt fails to compile a query due to a syntax error), the error is caught and sent back as an error event to the AI, and the server remains up for subsequent requests. We’ve instrumented comprehensive logging via the extension’s output channel (the “dbt Power User” output panel in VS Code) to record any exceptions or problematic requests. This makes it easier to debug if something goes wrong – both for us as developers and for users who can check the logs. Overall, the server is designed to be **long-running and robust**, recovering gracefully from errors. In our tests, issues like missing profiles or syntax errors are caught and reported back to the AI without terminating the server. **Security Measures:** Since the MCP server allows automated actions on your project, we have restricted its accessibility to ensure safety. The server **binds only to localhost** and is not accessible from external machines. This means only processes on your computer (like the Cursor IDE) can communicate with it, mitigating risk of any outside interference. Additionally, the server enforces that requests conform to the expected MCP schema – it will reject any malformed requests or unknown tool calls rather than executing them arbitrarily. We also incorporated a simple authentication check in this beta: the MCP server feature will only start if certain preconditions are met (for instance, verifying that you have a valid Altimate API key configured, which is used to enable AI features in the extension). This is mainly to gate the feature during beta and ensure usage can be monitored and supported. In the future, we plan to add more sophisticated security, such as an authentication token for the MCP connection or user consent prompts in VS Code before executing potentially destructive actions (like modifying files or running DDL statements). It’s important to note that the AI agent in Cursor will only use tools in ways you allow – you’re always in control and can disable or limit certain tools if desired. We recommend keeping the extension and Cursor up-to-date to get the latest security improvements. As the community and usage of MCP servers grow, we’ll continue to harden the server (for example, adding rate limiting or sandboxing if the scenario requires it). **Resource Usage:** Running an MCP server is lightweight. It piggybacks on the extension’s process and uses minimal additional memory (mostly just for the HTTP server overhead). The in-memory artifact store is already in place for the extension’s normal operation. We’ve observed negligible impact on VS Code’s performance when the server is idle. During heavy use (like a lot of model runs triggered by AI), there is some CPU and memory use, but it’s equivalent to you running those dbt commands manually. When idle, the server simply holds open a socket for Cursor. If needed, you can always stop the server by disabling the feature in settings, which immediately frees the port and associated resources. **Future Enhancements:** In upcoming releases, we plan to enhance both the capabilities and the internals of the MCP server. On the capability side, we’re looking at adding more tools (as mentioned, possibly Git operations, documentation queries, etc.) and integrating deeper with other systems (imagine the AI triggering an Airflow DAG after a successful dbt run, via a future Airflow MCP server). We’re also considering a plugin mechanism where users could write custom MCP tools for their own macros or operations and have the server load them. Finally, as the MCP protocol evolves (it’s an open standard in active development), we will adapt our server to remain compliant and take advantage of new features. For example, Anthropic’s Claude models recently embraced MCP natively, and we expect more LLMs to follow – our goal is to have the dbt MCP server work with any AI agent that speaks MCP, not just Cursor’s. ## Conclusion: Our Vision Toward an AI-Integrated Data Platform The introduction of the MCP server in the dbt Power User extension is a significant step toward an **AI-integrated data development platform**. We’ve turned the VS Code + dbt environment into an AI-accessible service, enabling smarter assistance and automation. This capability is not an isolated feature – it’s part of our broader vision to integrate AI with all facets of the modern data stack. In the near future, you can expect similar integrations with **Snowflake and Postgres** (for example, an AI agent that can profile or manipulate your warehouse data), with **Airflow** (an AI that can inspect DAGs or trigger workflows), and more. Our **DataPilot platform** is evolving into an ecosystem of AI “teammates” that collaborate across tools. The dbt MCP server is one of these teammates, focused on analytics engineering, and it will collaborate with others (imagine an AI assistant that understands your data warehouse, your pipelines, and your dbt models collectively). We invite you to try out the MCP server. Since it’s a beta release, we greatly value your feedback. Join our community Slack channel ([`#tools-dbt-power-user`](https://getdbt.slack.com/archives/C05KPDGRMDW) on dbt Slack) to share your experiences or ask any questions.. If you need help setting it up or want to explore how this could fit into your team’s workflow, please [**reach out to us**](https://app.myaltimate.com/contactus). We’re also eager to collaborate with teams interested in pushing the boundaries of AI in data engineering. Whether you have ideas for new MCP tools or want to integrate our platform with your own, let’s start a conversation. Together, let’s redefine what a “power user” can do by combining the strengths of dbt with the intelligence of AI. This launch is just the beginning, and we’re excited to build a more connected, intelligent data platform with your input. Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [Dev Tools](https://altimate.ai/blog?tag=dev-tools) [dbt](https://altimate.ai/blog?tag=dbt) [Data Engineering](https://altimate.ai/blog?tag=data-engineering) [Altimate](https://altimate.ai/blog?tag=altimate) [#ai-tools](https://altimate.ai/blog?tag=%23ai-tools) [AI](https://altimate.ai/blog?tag=ai) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # AI Teammates' Role in Snowflake Cost Efficiency URL: https://altimate.ai/blog/could-ai-teammates-be-the-secret-to-efficient-snowflake-cost-management Discover how AI teammates can revolutionize Snowflake cost management and enhance performance by working alongside human administrators [Back to blog](https://altimate.ai/blog) Snowflake · AI · AI Tools ## Could AI Teammates Be the Secret to Efficient Snowflake Cost Management? John Ryan Nov 13, 2024 Could AI Teammates Be the Secret to Efficient Snowflake Cost Management? In the past five years, Snowflake has disrupted the database, data storage, and analytics industry by massively simplifying the entire data and analytics landscape. However, a new wave of innovation is horizon - Artificial Intelligence and AI Teammates. This article won't discuss [Generative AI](https://research.ibm.com/blog/what-is-generative-AI), which is (shockingly) already old news, but how AI Teammates working with AI Agents can collaborate alongside your (human) Snowflake System Administrator to help maximize performance and optimize spending on your Snowflake system. After reading this article, you’ll realize just how powerful this technology is and what a potentially massive game-changer it presents. ## What is an Agentic AI? Unless you've been living under a rock for the past two years, you'll be well acquainted with [Chat-GPT](https://chatgpt.com/) (other [AI Chatbots](https://bestarion.com/generative-ai-chatbots/#:~:text=Jasper%20AI,of%20text%20quickly%20and%20efficiently.) are available), but these are simply passive assistants that quietly wait for your questions and generate text. That's not denigrate the impact of [Large Language Models](https://www.snowflake.com/resource/generative-ai-and-llms-for-dummies/?utm_source=google&utm_medium=paidsearch&utm_campaign=em-gb-en-nb-genaillm-exact&utm_content=go-rsa-evg-eb-generative-ai-and-llms-for-dummies&utm_term=c-g-large%20language%20model-e-690007389282&gad_source=1&gbraid=0AAAAADCzRJUdbdjyUGTGfqHV7XQ5NYv_t&gclid=EAIaIQobChMI4LfK1tGSiQMVEYuDBx0LvAnxEAAYAyAAEgKlXvD_BwE) (LLMs), as a recent [McKinsey Report](https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/the-economic-potential-of-generative-ai-the-next-productivity-frontier#key-insights) indicates that generative AI will add up to $4 trillion to the global economy with up to 70% less employee work required. Indeed, Artificial Intelligence is well on the way to being integrated into almost [every aspect](https://hogonext.com/how-to-apply-ai-effectively-for-data-warehousing/) of Data Warehousing. However, unlike Chat GPT that passively waits for user requests, an AI Teammate works alongside existing staff to monitor query patterns, identify potential issues and offer actionable tuning suggestions. They act as an artificially intelligent advisor, perhaps using machine learning to refine recommendations based on administrator feedback. AI teammates work autonomously to recommend real-time system configuration and processing adjustments, reacting to events and recommending remedial actions to improve Snowflake customer experience or optimize cost. Whereas an AI Teammate provides a consulting role, communicating with System Admins in natural language, [AI Agents](https://www.ibm.com/think/topics/ai-agents) work autonomously to handle the low-level monitoring, analyzing query patterns and resource usage. In many ways, AI Agents are the back-room workers, providing data to be analyzed and interpreted by the AI Teammates which in turn support the Snowflake Admins. Taken together, these are often referred to as [Agentic AI](https://aisera.com/blog/agentic-ai/#:~:text=Agentic%20AI%20uses%20reinforcement%20learning,reactive%20approach%20of%20Gen%20AI.). ## Key Features of “AI Teammates” and “Agentic AI” It's all well and good using jargon like “AI Agents” - but what makes a solution “[Agentic](https://www.ssonetwork.com/intelligent-automation/articles/what-is-agentic-ai#:~:text=Agentic%20AI%20refers%20to%20a,without%20requiring%20direct%20human%20intervention.) ” as opposed to, (for example), a Python script that uses machine learning to identify and alert users of potential problems. Here's a list of the key features an “[Agentic AI](https://www.ssonetwork.com/intelligent-automation/articles/what-is-agentic-ai#:~:text=Agentic%20AI%20refers%20to%20a,without%20requiring%20direct%20human%20intervention.) ” system. - **Dynamic Problem Solving**: Instead of relying on fixed scripts, in exactly the same way Chat-GPT can currently generate and execute python code, AI agents will generate source code and scripts to analyze data and solve problems. AI Teammates work just like a real person — assessing the situation, deciding what needs to be done, and then executing the plan. - **Reduced Technical Skills**: IT automation typically needs technical experts (DBAs and System Admins) to configure and maintain it through scripting. In contrast, Agentic AI allows anyone, including non-coders, to instruct the system in natural language. For example, a user could give the AI an objective like “reduce warehouse costs by reducing idle time” and the AI would figure out the best way to do that, without needing predefined scripting. - **Autonomous Operation**: Agentic AI doesn’t need constant supervision. Once given a goal, it will continuously monitor for opportunities to act and take the next steps without waiting for further instructions. - **Real-World Interaction**: These AI systems will connect with external tools and databases, pulling in real-time information, and even executing actions like sending emails or making configuration changes. For example, an AI Teammate tasked with reducing Snowflake Virtual Warehouse costs could automatically resize warehouses in real-time to maintain a balance between cost and performance. - **Efficiency & Adaptability**: These systems can handle complex workflows by breaking down tasks into smaller steps, calling different tools, and even calling upon AI Agents to handle specific tasks. As AI Teammates are largely autonomous, they are highly efficient needing little human interaction and constantly adapt to changing circumstances. ## How can AI Teammates work on Snowflake? AI Teammates can play a vital role in optimizing a complex Snowflake Data Warehouse deployment by automating performance tuning tasks. Here are some practical examples of how Agentic AI could be deployed: ### **Tuning Query Performance** - **Identifying Slow Queries**: An AI Teammate could continuously monitor query elapsed times against an objective, prioritizing performance over cost. It would then suggest optimizations, such as [improving partition pruning](https://articles.analytics.today/snowflake-cluster-keys-and-micro-partition-elimination-best-practices) or moving a given workload to a different [warehouse size](https://blog.altimate.ai/does-size-matter-what-else-is-important-in-choosing-a-snowflake-warehouse). - **Query Rewriting**: If a query can be rewritten for better performance (e.g., using specific functions like LIMIT to [reduce data scanning](https://articles.analytics.today/boost-your-snowflake-query-performance-with-these-10-tips) ), the AI Teammate could automatically suggest or implement these changes after analyzing the query execution plan. Of course, this may need a “human in the loop” to verify, test, and deploy the changed code. ### **Virtual Warehouse Management** The screenshot below illustrates how AI Agents deployed by Altimate AI automatically monitor and tune system configuration to identify potential cost savings and implement configuration changes to eliminate waste. [Line chart of AI Agent Decisions per day from Oct 14–21, 2024, with tabs to switch to Query Execution Time, Total Bytes Scanned, Query Queuing Time, and Query Count.](https://altimate.ai/) - **Warehouse Configuration**: AI Teammates can continuously monitor [virtual warehouse](https://articles.analytics.today/snowflake-virtual-warehouses-what-you-need-to-know) usage patterns and (for example), automatically adjust the AUTO\_SUSPEND, WAREHOUSE SIZE or CLUSTER COUNT settings to optimize efficiency. For instance, if a warehouse remains idle longer than expected, the AI Teammate could reduce the auto-suspend timeout or automatically resize warehouses during off-peak hours to reduce costs. - **New Feature deployment:** Agentic AI could automatically detect and recommend potential improvements using new Snowflake features. For example, an AI Teammate tasked with maintaining strict performance targets could recommend deploying Snowflake Query Acceleration or Search Optimization Service to improve query performance. This reduces the need for System Administrators to constantly research, understand, and test new Snowflake features, reducing the dependency upon highly skilled (and expensive) resources. ### Managing Snowflake Cluster Keys - **Identifying Clustering Opportunities:** [Snowflake Cluster Keys](https://articles.analytics.today/snowflake-cluster-keys-and-micro-partition-elimination-best-practices) can significantly impact Snowflake query performance, but deployment is more an art than a science. An AI Teammate could automatically analyze query patterns and data distribution to recommend adding, modifying, or removing cluster keys. For instance, if certain columns are frequently used in filter conditions but are not well clustered, the system could recommend creating cluster keys on those columns to speed up query performance. - **Monitoring Clustering Efficiency:** As query workloads change over time, once appropriate cluster keys may become less efficient over time. For example, if a data pipeline changes, adds frequent `update` operations, or changes the data cardinality, this could make clustering less cost-effective. An AI Teammate could monitor the situation and recommend or even automatically deploy changes to improve query performance and optimize cost. ### Dynamic Workload Allocation [Triangle diagram showing three competing warehouse goals — Control Cost, Maximize Throughput, and Maximize Query Performance — connected by bidirectional arrows illustrating the trade-offs between them.](https://hashnode.com/edit/cm0a6anfv000509l4h9qk86ui) The diagram above illustrates the overall challenge faced by System Administrators. Despite constantly increasing system workloads, we need to balance the conflicting needs to maximize system throughput (for example loading and transforming data), with maximizing end-user query performance (for example, delivering query results to analytics), and optimizing spend. Agentic AI can help with this challenge by automating system monitoring and adjusting workloads as needed. The diagram below illustrates a typical mixed workload frequently found on Snowflake MEDIUM size warehouses. [Diagram of a mixed workload over a 24-hour period: one long low-frequency job (yellow), an hourly job (red), a several-times-an-hour job (blue), and a many-times-an-hour job (green), overlapping across the day.](https://hashnode.com/edit/cm0a6anfv000509l4h9qk86ui) While it's not obvious, research into [Snowflake Workload Segmentation](https://blog.altimate.ai/simple-question-is-snowflake-expensive-the-answer-is-it-depends) demonstrates that the above workload would be more efficiently deployed over two or four warehouses, separating workloads by size and frequency. This has the potential to improve system performance and cost efficiency significantly. Most customers deploy [workload separation](https://blog.altimate.ai/does-size-matter-what-else-is-important-in-choosing-a-snowflake-warehouse) by team, which leads to poor utilization and high spending. AI Teammates constantly monitor jobs, recommending or even moving them to an appropriate warehouse size to achieve the best balance of throughput and cost efficiency. For example, high-priority analytical workloads could be processed on larger warehouses while smaller, less important workloads could be shifted to more cost-effective, smaller warehouses configured for batch processing and prioritizing throughput and spend over individual query performance. ### **Proactive Snowflake Monitoring and Alerts** The screenshot below illustrates automated Snowflake cost monitoring across all sectors, including Warehouse computing, Storage, and Snowflake Serverless processing. AI agents can build upon basic monitoring to alert administrators proactively or take action. Daily total-cost bar chart from Oct 14–20, 2024, broken into compute (blue), storage (yellow), and serverless (purple) segments, with compute consistently the largest share of spend. **Anomaly Detection**: AI teammates can track the system’s performance and detect anomalies such as sudden spikes in query times, sudden increases in storage usage, or unexpected credit consumption. Once detected, the system could either notify System Admins or attempt to automatically resolve the issue (e.g., by suggesting or implementing query optimizations). In one case, a major Snowflake customer in the UK accidentally spent over $120,000 due to a simple manual configuration change that took a week to identify and resolve, which could have been automatically identified by an AI Teammate within seconds, potentially avoiding a significant bill. ### **End-to-End ELT Optimization** According to Snowflake, automated data transformation tasks account for around 80% of customer spending. However, tuning increasingly complex pipelines can be remarkably resource-intensive, often requiring an army of highly skilled data engineers. The diagram below illustrates the challenge with a simple transformation pipeline with potentially conflicting delivery timelines. The challenge comes from optimizing data delivery within the required SLA, but avoiding unnecessary processing. [Diagram showing hourly Sales data and daily Stores data each transformed separately, then merged into one daily-refreshed table delivered to an analyst, illustrating mismatched pipeline refresh frequencies.](https://articles.analytics.today/how-snowflake-dynamic-tables-simplify-etl-pipelines) In one example, a Snowflake customer in Munich, Germany was able to reduce overall system cost by 50% simply by suspending real-time analytics during out of hours. In short, if nobody is consuming the data, there's no need to produce it. AI Teammates can help in the following ways: - **Transformations and Load Efficiency**: AI Teammates can analyze transformation pipelines, looking for inefficiencies or bottlenecks. They can recommend breaking down complex SQL statements into smaller tasks or [parallelizing operations](https://articles.analytics.today/boost-your-snowflake-query-performance-with-these-10-tips) to improve load and transformation speeds. - **Dynamic Table Optimization**: Snowflake's [Dynamic Tables](https://articles.analytics.today/how-snowflake-dynamic-tables-simplify-etl-pipelines) massively simplify and streamline transformation pipelines with opportunities to dynamically change processing frequency and warehouse deployment. AI Teammates can automatically monitor both workload size and data consumption and adjust the pipeline configuration to balance and optimize performance and cost. - The diagram below illustrates one potential tuning step in which the ETL pipeline can be automatically adjusted to match changing the frequency at which data must be refreshed. In this case, a critical data feed is refreshed every five minutes. Once deployed, an AI Teammate can [automatically adjust the refresh frequency](https://blog.altimate.ai/introducing-snowflake-dynamic-tables-what-you-need-to-know) for example reducing the frequency during out-of-hours to optimize cost. [Diagram showing Sales and Stores data each transformed on a downstream schedule, then merged and delivered to an analyst every 5 minutes, illustrating a real-time refresh requirement.](https://articles.analytics.today/how-snowflake-dynamic-tables-simplify-etl-pipelines) ## **Summary & Conclusion:** AI Teammates in a Snowflake environment can act as intelligent assistants that continuously monitor and optimize various aspects of a Snowflake deployment. They can provide actionable recommendations and potentially even implement changes themselves—whether it’s tuning queries, managing warehouse resources, or optimizing storage usage. This can lead to improved performance, reduced costs, and more efficient use of Snowflake’s powerful features, all with minimal manual intervention from human operators. In conclusion, having spent nearly 10 years as a Senior Solution Architect much of that time working for Snowflake UK, I’ve a fair understanding of the huge challenges that Snowflake System Administrators face. I firmly believe that AI Teammates are indeed the secret to efficient Snowflake cost and performance management. However, unlike the [pessimistic view](https://en.wikipedia.org/wiki/AI_takeover) of Artificial Intelligence which warns about the machines taking over, the fact that AI Teammates work collaboratively with DBAs and Admins mean they’ll simply make an almost impossible task way more effective. --- Deliver more performance and cost savings with Altimate AI's cutting-edge AI teammates. These intelligent teammates come pre-loaded with the insights discussed in this article, enabling you to implement our recommendations across millions of queries and tables effortlessly. Ready to see it in action? Request a recorded demo by just sending a chat message ([here](https://app.myaltimate.com/contactus) ) and discover how AI teammates can transform your data teams. [Promotional banner reading ‘AI Data Teammates that work for you 24/7’ beside a screenshot of the Auto Tune savings dashboard, with a Book a Demo button.](https://www.notion.so/altimate/Blog-Post-Demo-Hook-12041b89292680e09a26f8c718dbaad4?pvs=4) Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [Snowflake](https://altimate.ai/blog?tag=snowflake) [AI](https://altimate.ai/blog?tag=ai) [AI Tools](https://altimate.ai/blog?tag=ai-tools) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Streamlining Data Documentation: Generative AI Best Practices URL: https://altimate.ai/blog/write-better-data-docs-faster Discover best practices for writing effective data documentation and learn how Gen AI tools can streamline the process, reducing errors and saving time [Back to blog](https://altimate.ai/blog) dbt · Altimate · Analytics ## Write Better Data Docs Faster Thomas Schmidt Sep 19, 2024 Write Better Data Docs Faster ## Once upon a time, an analyst in the trouble Imagine Leila, an analyst, asked to build a monitoring dashboard for finance - given a tight deadline. While searching for a useful dataset, she comes across a table called “sh\_monthly\_revenue” that seems perfect based on its name and the columns it contains. However, there’s little documentation available, and the person who created it has already left the company. With time ticking, she makes a few assumptions based on the column names and dives into the analysis. Weeks later, after presenting the dashboard findings, one of the business stakeholders points out that the data does not match the numbers in the financial reports at all. It turned out that the dataset she used wasn’t appropriate since it only covered revenue from a subset of customers, leading to incorrect assumptions and, ultimately, flawed conclusions. This scenario didn’t just waste time and resources - it also damaged Leila’s credibility and may have led to misguided business decisions. If this sounds familiar, you’re not alone. Many data professionals - whether analysts, data scientists, or engineers - have found themselves in a similar situation. Poor documentation doesn’t just slow down work; it can lead to significant errors with far-reaching consequences. On the other hand, writing and maintaining comprehensive documentation can be a tedious and time-consuming task. In this post, we’ll explore best practices for writing better documentation for your data models and how GenAI can help make the process less tedious and time-consuming. ## Good documentation matters for many reasons It’s easy to overlook documentation when you're in the final stages of building a data model, eager to push your work into production. We've all been there: you’ve just completed a model you’re proud of, and you’re ready to ship it out the door. But then comes the task of writing documentation (and possibly adding a few data tests). This critical step often gets rushed or skipped altogether, yet there are compelling reasons to take the time to do it right: - Reduces time spent on data discovery - Reduces dependency on tribal knowledge - Reduces Ad-hoc questions you get from data consumers - Your org makes you follow governance and best practices guidelines ^\_^ Understanding why documentation matters is just the first step. The real challenge lies in consistently applying best practices to ensure your documentation is both comprehensive and usable. Let’s explore how you can achieve this with a few best practice tips. ## Best practices for writing excellent data documentation Creating effective data documentation isn’t just about listing columns and definitions - it’s about ensuring that anyone who interacts with your data can understand and use it correctly, even if they have no prior context. Here are some best practices to help you design documentation that’s both comprehensive and user-friendly: ### “Always Consider the Reader’s Perspective” When documenting a data model, it’s easy to fall into the trap of writing from your perspective, assuming that others will inherently understand the nuances you do. Instead, try to **put yourself in the shoes of someone without prior knowledge of the dataset**. When it comes to writing the docs for your model, you can follow these guidelines: - **Clearly outline the Purpose and Utility:** Define the key questions this dataset is designed to answer and the specific use cases it supports. This context helps users understand the data’s intent and how best to leverage it. - **Identify Caveats:** Highlight any limitations or potential pitfalls associated with the data, such as missing information, known inaccuracies, or situations where the data might be misleading. These warnings are crucial for preventing misuse. - **Specify Granularity:** Detail the level of detail provided by the data, whether it’s aggregated daily, weekly, or monthly, or at the transaction level or summarized. Understanding the granularity helps users determine the dataset’s appropriateness for their needs. In the case of columns, include information about primary keys, etc. - **Explain Contextual Background:** Provide the reasoning behind the table’s creation, including any business processes or decisions that shaped its structure. This background helps users appreciate the dataset’s origins and relevance within the broader business context. - **Highlight Key Dimensions and Metrics:** Point out the most important dimensions (such as date, location, and product) and metrics (like sales, revenue, and click-through rate) within the dataset. Explain how these elements interact and any specific considerations for their use. In the case of categorical columns, you can possibly include details about accepted values. - **Optimize for AI and Semantic Search:** As AI tools become more integrated into data management, ensure your documentation includes relevant business terms, acronyms, and synonyms associated with the data. This optimization enhances discovery and usability, particularly for AI-driven tools like semantic search engines or analytics agents. Here is an example of how this could look like for our model from above: Screenshot of a dbt schema.yml file showing an AI-generated model description for daily\_product\_sales, including example use cases, caveats, key dimensions and metrics, additional context, and keyword tags. ## Make documentation less painful by using AI Despite your best efforts, documentation can still be time-consuming and tedious, especially when balancing tight deadlines and complex data models. This is where generative AI (GenAI) comes into play, turning what once felt like a chore into a largely automated and streamlined process. Tools like the [Power User for dbt (DataPilot)](https://docs.myaltimate.com/document/generatedoc/) are designed to handle the heavy lifting of documentation, automatically generating detailed table and column descriptions from your existing data models. These tools use the metadata context available in your environment automatically like schema information, column lineage etc. and write so good documentation that has surprised me many times. Some of the key features include: - **Effortlessly Document Hundreds of Columns:** Have 900 columns to document? No problem. With **bulk generation**, you can quickly create reliable descriptions for every column, allowing you to focus your efforts on refining and curating the details that matter most. - **Tailored Documentation for Different Audiences:** The extension allows you to **generate documentation for different user personas**. Use technical language for your intermediate layer tables and a more business-friendly tone for end-user-facing tables. This ensures that each audience gets the information they need in the most effective format. - **Seamless Integration with Your Workflow:** The tool **integrates directly with your dbt YAML schemas**, injecting newly generated documentation into existing schemas or updating current documentation with just a few clicks. All this happens directly within the model SQL file, so you can avoid those days when your IDE tabs blow up like confetti due to the number of open file tabs. More details can be found [here](https://docs.myaltimate.com/document/generatedoc/) Overall, documentation is an important aspect of analytics engineering and it’s essential that we follow best practices, otherwise, we run the risk of writing documentation for namesake or not writing it at all! GenAI based automation is going to be the major boon in this area, it has its kinks but early signs are very promising making me quite excited for the future! Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [dbt](https://altimate.ai/blog?tag=dbt) [Altimate](https://altimate.ai/blog?tag=altimate) [Analytics](https://altimate.ai/blog?tag=analytics) [Dev Tools](https://altimate.ai/blog?tag=dev-tools) [Data Engineering](https://altimate.ai/blog?tag=data-engineering) [#ai-tools](https://altimate.ai/blog?tag=%23ai-tools) [AI](https://altimate.ai/blog?tag=ai) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # How AI Agents Can Optimize Dynamic Table Spend URL: https://altimate.ai/blog/can-ai-agents-reduce-the-cost-of-snowflake-dynamic-tables Understand the surprising cost implications of Snowflake Dynamic Tables and how AI Agents can help to massively optimize Snowflake spend [Back to blog](https://altimate.ai/blog) Snowflake · AI Tools · AI ## Can AI Agents reduce Snowflake Cost using Dynamic Tables? John Ryan Sep 7, 2024 Can AI Agents reduce Snowflake Cost using Dynamic Tables? In my previous article on [Dynamic Tables](https://blog.altimate.ai/introducing-snowflake-dynamic-tables-what-you-need-to-know), I explained their purpose, to help simplify building transformation pipelines for either batch or real time data feeds in Snowflake. In this article, I'll explain what they cost to run and how [Artificial Intelligent Agents](https://blog.altimate.ai/will-ai-agents-replace-snowflake-system-administrators) can be used to automatically monitor, alert and even deploy remedial action to control cost. Skeptical? Read on to see how to reduce the annual cost of running three Dynamic Tables from $6.3m to just $26,280. ## Dynamic Tables Cost Components It may not be obvious but Dynamic Tables include multiple cost elements including: - **Cloud Compute:** Dynamic Tables are automatically refreshed on a repeating sequence depending upon the TARGET\_LAG. For example, once per minute or once per hour. - **Data Storage:** Like any table, Dynamic Tables store data, and this will be billed at £23 per terabyte per month. - **Warehouse Cost:** Every time the Dynamic Table is refreshed it (currently) needs a [Virtual Warehouse](https://articles.analytics.today/snowflake-virtual-warehouses-what-you-need-to-know) to execute the SQL, and this is billed on a per-second basis with a minimum of 60 seconds. Let's consider each of these elements and it's potential impact upon the overall cost of ownership. ## Cloud Compute Cost This is by far the least expensive of the three cost elements and results from the automatic scheduling executed in the [Cloud Services](https://docs.snowflake.com/en/user-guide/intro-key-concepts#cloud-services) layer. Determined by the TARGET\_LAG, the DT "wakes up" and checks if there any changes in the upstream table(s). This is likely to be thousands of a dollar each time, and for 99% of customers is free, unless it becomes more than 10% of your daily warehouse cost. In short, don't worry about this. ## Dynamic Table Storage Cost While you need to store data in Snowflake (and therefore you'll be billed a passthru cost of £23 per terabyte), it's not clear that Dynamic Tables add to the [storage cost](https://articles.analytics.today/how-to-cut-snowflake-data-storage-costs-with-zero-copy-clones). Consider the diagram below. Diagram showing two source tables T1 and T2 feeding two dynamic tables T3 and T4, which both feed a third dynamic table T5 delivered to an end user. In the above diagram tables T1 and T2 are default tables, wheres T3, T4 and T5 are dynamic tables. If we assume T1 and T2 hold 1TB of data storage, and the transformation to T3 and T4 is simply to perform a basic data quality correction (for example, setting in incorrect SALES\_TYPE column to NULL when the values are incorrect), then tables T3 and T4 will equally hold around 1TB of storage. If the results of T3 and T4 are then combined using a JOIN operation into a single table T5 - ready for querying, this adds another 2TB to storage. In total we store 6TBs of storage. However, if we were not using Dynamic Tables, then tables T1 and T2 could be deleted down each time the new entries where processed. Likewise, once the process to join the tables and produce T5 was complete, tables T3 and T4 could be deleted. This means, a solution without Dynamic Tables could deploy the same functionality in 2TBs compared to 6 TBs. Data storage is a relatively minor cost on most Snowflake systems, but it's a cost worth understanding. ## Dynamic Tables Warehouse Cost This is almost certainly the greatest cost associated with Dynamic Tables, although unlike the data storage example above, there's no additional cost when compared to a transformation pipeline without DTs. Effectively, each time a Dynamic Table triggers a refresh (provided there's changed data in the upstream tables), Snowflake executes the associated SQL on a Virtual Warehouse. However, even this has a potential gotcha. Consider again the diagram below. Diagram of a Sales and Stores pipeline where each source feeds a downstream dynamic table, merging into a final dynamic table that refreshes every 1 minute for the end user. The diagram above illustrates the same transformation pipeline, with two source tables and three dynamic tables. Notice that two of the tables are marked as Downstream? This means their refresh rate is determined by the downstream table - in this case a table refreshed every minute. Let's assume new data is automatically inserted into the Sales table using [Snowpipe](https://docs.snowflake.com/en/user-guide/data-load-snowpipe-intro) which runs on a 24x7 basis. The Stores data is however, refreshed once per day. Can you see what might happen to the cost profile of this simple transformation pipeline? As new sales are automatically loaded into the Sales table (T1), Dynamic Tables T3 and T5 are automatically refreshed based upon the one minute TARGET\_LAG. Let's assume all Dynamic Tables were initially deployed on an XSMALL warehouse, this incredibly simple transformation pipeline would cost up to $1,576,800 per year. Worse still, if the warehouse was subsequently [increased](https://articles.analytics.today/snowflake-virtual-warehouses-is-bigger-faster-but-more-expensive) in size to a MEDIUM, this would increase to a shocking $6.3m a year for a single warehouse. Around four years ago, I did a consultancy visit to a customer in Munich Germany who had deployed a similar pipeline, also running 24x7 and were shocked by the high cost. The customer justified the 24x7 operation because the system tracked global internet based sales, and could therefore receive sales 24x7. After some time on the whiteboard to map out the equally simple pipeline, I understood the problem and asked the customer - "***So, where are your Data Analysts based***". I'm sure they thought it was a stupid question, as their reply was like talking to a five year-old. "***Here in Munich, John***". In response, I asked whether the Data Analysts were looking at the data at 4am, and if not, why were we processing data at 4am and paying for a virtual warehouse to wake up every minute leading to a huge bill? Finally, I also questioned, why they were continuously processing data every minute. What action would the analysts take that needed the data to be as little as one minute stale? In conclusion, the same transformation pipeline cost, even on a MEDIUM size warehouse could be cut by 50%, simply by suspending the Dynamic Table refresh from 8pm to 8am using a [Snowflake Task](https://docs.snowflake.com/en/sql-reference/sql/create-task). We could further reduce the cost by a factor of four by executing the following SQL statements. ```sql alter dynamic table T3 set warehouse = DT_XSMALL_WAREHOUSE; alter dynamic table T4 set warehouse = DT_XSMALL_WAREHOUSE; ``` The two simple SQL statements move the refresh operation to an XSMALL warehouse costing 25% of the cost. Of course, we'd need to check the refresh performance was acceptable, but assuming an INCREMENTAL refresh method, this should be pretty fast to complete. Finally, we should question whether the Data Analytics **really need** the data refreshed every minute. Executing the following SQL would reduce costs by a factor of 60. ```sql alter dynamic table T5 set target_lag = '1 Hour'; ``` > "Never give a user what they **ask for**, give them what they **need**" - Dr. David Pearson. ## Can Artificial Intelligence Help? As I've described before, a new breed of [Artificial Intelligent Agents](https://blog.altimate.ai/will-ai-agents-replace-snowflake-system-administrators) can be used to tirelessly monitor, alert or even adjust system configuration. It's like having an Artificially Intelligent Database Administrator proactively tuning your system. However, the challenge of tuning a multi-petabyte analytic platform which charges using a pay-for-use basis is more challenging that you'd expect. Triangle diagram showing the three competing priorities in managing a Snowflake platform — control cost, maximize throughput, and maximize query performance — each pulling against the other two. The diagram above illustrates the primary challenge, balancing the need to maximize throughput of large (mainly batch) transformation jobs, and maximizing performance of end-user queries while also optimizing spend. There are also some critical facts about Snowflake which need to be understood: 1. Around 80% of the cost of running a Data Warehouse comes from repeating automated jobs. The majority of these tend to be batch oriented and many prioritize throughput over performance - the need to regularly deliver data, perhaps once per hour or once per day. 2. Years of research with over 50 Snowflake customers and thousands of queries against real customer data have demonstrated that most customers have a poor [virtual warehouse deployment](https://blog.altimate.ai/does-size-matter-what-else-is-important-in-choosing-a-snowflake-warehouse) strategy which potentially leads to significant cost overspend. 3. [Snowflake Dynamic Tables](https://articles.analytics.today/how-snowflake-dynamic-tables-simplify-etl-pipelines) have the fastest growth in uptake of any Snowflake feature and are likely to become the gold standard for data transformation on Snowflake. However, unlike hand-coded ELT jobs, the warehouse size and execution frequency can be easily adjusted as these are entirely configurable in the Dynamic Table definition. Taking both the problem (balancing throughput, performance and cost), with the configurable nature of Dynamic Tables means it's possible to automatically monitor and even change system configuration in real-time using [Agenic AI](https://blog.altimate.ai/will-ai-agents-replace-snowflake-system-administrators). Step chart showing the number of automated AI agent configuration decisions made on a Snowflake Dynamic Table pipeline per day over about a month. The screenshot above shows the frequency of AI Agent decisions used to monitor and adjust Snowflake to maximize performance and optimize costs. For example, if an ELT pipeline normally needs a MEDIUM sized warehouse but regular end-of-month processing would justify an X4LARGE, an [AI Agent](https://blog.altimate.ai/will-ai-agents-replace-snowflake-system-administrators) could automatically identify the opportunity and even change the configuration subject to System Administrator approval. This would not only deliver results over 100 times faster, but it would avoid a common mistake made by almost every Snowflake customer, using an i[nappropriate virtual warehouse size](https://blog.altimate.ai/does-size-matter-what-else-is-important-in-choosing-a-snowflake-warehouse), and effectively the largest possible machine for the highest expected workload. ## Conclusion While the cost of Cloud Services compute and even storage for Dynamic Tables is likely to negligible, we need to consider the cost of running a virtual warehouse. With just three SQL statements, we reduced the maximum cost of this simple transformation pipeline from $6.3m per year down to $26,280. To summarize what we learned: - **Cloud Compute and Storage** costs can (in most cases) be ignored. They shouldn't add significant cost to a pipeline deployed using a Dynamic Table. - **Warehouse Costs** however, can be remarkably high and lead to an eye-watering annual bill if deployed without thought. - **Snowflake is not expensive!** Like any tool, if used incorrectly it can lead to a poor outcome. Applying some common sense goes a long way. - **TARGET\_LAG:** Is critical in determining the frequency with which a Dynamic Table query is executed. If the data changes frequently, it can lead to significant cost. Carefully choose a TARGET\_LAG based on how fresh you **need** the data. Don't just set it to once per minute because you can. - **Dynamically Change the lag:** You can set up a simple schedule to suspend Dynamic Table refreshes or adjust the TARGET\_LAG during off-peak or holiday periods. Suspending the refresh overnight will immediately reduce the costs by up to 50%. Finally, in addition to manually adjusting Dynamic Table configuration, we could deploy [AI Agents](https://blog.altimate.ai/will-ai-agents-replace-snowflake-system-administrators) to autonomously monitor, alert or even tune Dynamic Tables to automatically achieve the almost impossibly complex goal of balancing the requirements of throughput, performance and cost. --- Deliver more performance and cost savings with Altimate AI's cutting-edge AI teammates. These intelligent teammates come pre-loaded with the insights discussed in this article, enabling you to implement our recommendations across millions of queries and tables effortlessly. Ready to see it in action? Request a recorded demo by just sending a chat message ([here](https://app.myaltimate.com/contactus) ) and discover how AI teammates can transform your data teams. [Promotional banner reading "AI Data Teammates that work for you 24/7" with a Book a Demo button, next to a screenshot of the Auto Tune dashboard showing realized and estimated savings on a Snowflake warehouse.](https://app.myaltimate.com/contactus) Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [Snowflake](https://altimate.ai/blog?tag=snowflake) [AI Tools](https://altimate.ai/blog?tag=ai-tools) [AI](https://altimate.ai/blog?tag=ai) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Cut dbt Development Time by 20% with Power User for dbt URL: https://altimate.ai/blog/how-i-cut-dbt-development-time-by-20-with-the-power-user-vscode-extension Power User for dbt extension in VSCode cuts dbt development time by 20% through expert skills and AI-driven documentation automation [Back to blog](https://altimate.ai/blog) dbt · Altimate · analytics tools ## How I Cut dbt Development Time by 20% with the Power User Vscode Extension Anand Kumar Apr 14, 2024 How I Cut dbt Development Time by 20% with the Power User Vscode Extension Working with data in any capacity—be it as a data/analytics engineer, data scientist, or data analyst—inevitably dealing with the need for documentation. Whether it's clarifying unclear fields, understanding dataset structures, or seeking explanations for data discrepancies, clear and comprehensive documentation is important. In the constantly evolving data world, where accuracy and clarity are non-negotiable, tools like dbt (data build tool) emerge as lifesavers. Recognizing the importance of clear documentation in the data world, dbt™ makes documentation significantly easier through its dbt™ docs feature. dbt provides a way to generate documentation for your dbt™ project. The documentation for your project includes: - Project Information: including model code, a DAG of your project, any tests you've added to a column, and more. - Data warehouse Information: including column data types, and table sizes. This information is generated by running queries against the information schema. A standout feature of dbt™ lies in its capability to enhance documentation comprehensively. Users can add detailed descriptions to models, columns, sources, and other project elements, enriching the documentation with context and clarity. This not only aids in understanding the data structures but also streamlines collaboration and knowledge sharing within teams. Here's an example of how dbt™ models can be documented: ```yaml version: 2 models: - name: events description: This table contains clickstream events from the marketing website columns: - name: event_id description: This is a unique identifier for the event tests: - unique - not_null - name: user-id quote: true description: The user who performed the event tests: - not_null ``` **Overcoming Documentation Challenges** Despite the robust dbt™ docs feature, documentation often takes a backseat due to the inherent gap between project work and the documentation process. After completing a model, users frequently find themselves navigating to various directories to locate and manually edit schema YAML files for documentation purposes. This manual approach is time-consuming and prone to errors, relying heavily on individual skills and consistency. **Introducing Power User for dbt Extension: A Game-Changer in Documentation** To address these challenges and streamline the documentation workflow, Altimate AI introduces the [Power User for dbt](https://marketplace.visualstudio.com/items?itemName=innoverio.vscode-dbt-power-user) vscode extension. This extension leverages AI capabilities to revolutionize documentation generation, making it a seamless and efficient process. Specifically, Altimate AI is designed to emulate the behavior of human engineers by leveraging contextual insights drawn from schema, sources, column-level lineage, metadata, and existing documents to create tailored documentation, rather than just generating generic documentation. Within the documentation editor, users can access a range of functionalities: 1. **Model Documentation**: Users can edit individual model documentation directly within the editor, eliminating the need for manual YAML editing. Additionally, AI-driven generation allows for instant and accurate descriptions, tailored to specific use cases such as professional or more casual documentation styles Screenshot of the dbt Power User documentation editor for the orders model, with the Documentation Editor tab circled, showing an AI-generated model description and a Sync with the Database button above the column list. 2. **Column Documentation**: The extension enables users to generate detailed documentation for individual columns, automatically saving them in the 'schema.yml' file with the appropriate syntax. This automation not only accelerates development but also ensures consistency and accuracy across documentation. Side-by-side view of the dbt Power User documentation editor generating column descriptions for a sales model next to the resulting schema.yml, with AI-written descriptions and data types for each column. 3. **Multiple Languages Documentation**: Users have the flexibility to create documentation in several languages, including French, English, Dutch, and German. This feature enables the generation of content that caters to diverse linguistic needs. 4. **Persona-Specific Documentation:** The extension offers a variety of presets designed for different audiences. Users can select the persona that best matches their needs, whether it be a technical user, business user, or general user. For general users, the documentation is crafted without a specific persona focus, ensuring it is accessible and understandable to a broad audience. **Impact of Effective Documentation in dbt™** Quality documentation significantly improves how models are used and reduces unnecessary communications between teams, boosting productivity. Using the dbt Power User extension, you can save at least two minutes per column in documentation tasks. Depending on the size and complexity of the models, this efficiency can lead to a reduction of 10%-20% in development expenses. This not only translates to substantial cost savings but also frees up engineers to focus on more critical tasks. **Harnessing the Power of AI for Documentation Excellence** Maintaining documentation in a dynamic data environment presents a substantial challenge. As business requirements expand and projects evolve, the task of updating documentation to reflect changes in models, schemas, or sources becomes increasingly complex and resource-intensive. This constant need for updates can significantly drain time and productivity. Hence, the integration of AI-driven documentation generation within dbt Power User extension represents a significant advancement in data documentation practices. By automating tedious tasks and offering customizable options, dbt™ enables engineers to concentrate on development work while ensuring that the documentation stays comprehensive and up-to-date. **Final Thoughts** In the rapidly changing world of data management, understanding the essence of quality documentation has never been more critical, dbt Power User extension documentation capabilities shine as a beacon of efficiency and excellence. By leveraging advanced technologies like AI, dbt Power User extension not only simplifies the documentation process but also promotes data transparency, collaboration, and innovation. Embracing these powerful tools paves the way for enhanced productivity, informed decision-making, and data-driven success. Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [dbt](https://altimate.ai/blog?tag=dbt) [Altimate](https://altimate.ai/blog?tag=altimate) [analytics tools](https://altimate.ai/blog?tag=-analytics-tools) [Dev Tools](https://altimate.ai/blog?tag=dev-tools) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register) --- # Optimize Snowflake Queries: Strategies for Data Engineers URL: https://altimate.ai/blog/optimizing-snowflake-queries-expert-strategies-for-data-engineers-and-analysts Unlock the secrets to turbocharging your Snowflake queries with our expert guide. Discover tips and best practices tailored for data engineers & analysts [Back to blog](https://altimate.ai/blog) Snowflake · Data Engineering · AI ## Optimizing Snowflake Queries: Expert Strategies for Data Engineers and Analysts John Ryan Jan 28, 2024 Optimizing Snowflake Queries: Expert Strategies for Data Engineers and Analysts For data teams, the clock is always ticking. In a business landscape defined by rapidly expanding datasets, slow queries can grind insights to a halt. That's why performance tuning is especially critical when working with a platform like Snowflake. This platform, renowned for its lack of traditional indexes, presents unique challenges and opportunities for optimizing query performance. To navigate these uncharted waters, data engineers and analysts need a mastery of specialized optimization tactics. Join me as we dive into the art of writing fast code that unlocks the full capabilities Snowflake offers. We’ll review examples ranging from basic query tuning to more advanced optimizations that can mean the difference between swift analytics and stalled projects. ## Snowflake Performance Tuning Strategy Before we start, let’s put the overall performance-tuning strategy in context. The diagram below illustrates the typical data processing pipeline you’ll see on any Snowflake platform, which consists of the following: - **Data Loading:** This involves copying data from Cloud Storage into a Table Landing zone. This normally uses the Snowflake COPY or SNOWPIPE commands. - **Data Transformation:** Data from multiple sources and loading into Data Integration and Data Delivery zones. This involves the execution of Insert, Update, or Merge statements to clean, manipulate, and transform the data ready for consumption. - **Data Querying & Consumption:** The data by end user dashboards, reports, or analysis. This typically involves SELECT statements against the Data Delivery zone. Diagram of a Snowflake data pipeline: a source system loads files through cloud storage staging into a Table Landing zone, through two ETL steps into Data Integration and Data Delivery zones, before reaching the end user. It’s worth considering the above because the approach and solutions tend to differ. Let’s consider each in turn. ## Data Delivery As indicated above, data delivery is all about quickly using SELECT statements to deliver results to end-users. This means it's vital SQL is tuned for maximum performance. When tuning query performance, the first place to look is the query profile. Query profile overview panel showing total execution time split by processing, local disk I/O, remote disk I/O, synchronization, and initialization, with remote disk I/O highlighted at 88.6% of the 5.1-second runtime. The screenshot above illustrates the first place to check for performance issues. Many assume this indicates the time spent, but it's the time waiting for resources. The above case shows the query spent just 10% of its time waiting on CPU resources, almost zero (0.2%) of the time waiting for Local Disk I/O to complete, but nearly 90% of the time waiting on Remote Disk I/O. Reviewing the overall Snowflake hardware architecture illustrated in the diagram below is helpful to understand what this means. Diagram of Snowflake's architecture layers — cloud services (access control), compute services (virtual warehouse), and cloud storage — showing how different teams' queries flow through access control to a shared virtual warehouse and storage. The above diagram shows how queries submitted to Snowflake are handled by the Cloud Services Layer, determining the best way to execute the query. The SQL is then passed to a Virtual Warehouse, a physical computer with CPU cores, memory, and SSD (Local) storage. When queries are executed, the data is read from Remote Storage, cached in SSD, and results computed and returned to the user. The key insight here is that (SSD) Local Storage is hundreds of times faster than Remote (disk) storage. One of the most effective ways to maximize query performance is limiting Remote Disk accesses. The most effective way to reduce remote disk I/O is using the query WHERE clause to filter out rows. Complex SQL queries often include a series of Common Table Expressions (CTEs or inline views), and the best practice is to filter out rows as early as possible. The fewer rows fetched from the Remote Disk, the faster the query executes. The next step in tuning Snowflake query performance is to review the query statistics illustrated in the screenshot below. Query statistics panel highlighting partitions scanned (710) out of a total of 3,911 partitions, alongside scan progress, bytes scanned, and percentage scanned from cache. Snowflake stores data in 16MB blocks called micro partitions and automatically prunes these to maximize query performance, but it's entirely dependent upon the WHERE clause. If (for example), a query includes no WHERE clause, Snowflake will need to fetch the entire table from Local or Remote storage with a significant impact upon performance. ## Tuning the query WHERE clause You can make simple changes to the WHERE clause to maximize query performance. Take, for example, the following SQL statement: ```sql select * from SNOWFLAKE_SAMPLE_DATA.TPCH_SF1000.ORDERS where to_char(o_orderdate,'DD-MON-YYYY') = '01-APR-1992'; ``` The above query returns ORDERS placed on 1st April 1992. However, we limited Snowflake's ability to prune query performance automatically because we formatted the O\_ORDERDATE as a character in the WHERE clause. ```sql select * from SNOWFLAKE_SAMPLE_DATA.TPCH_SF1000.ORDERS where o_orderdate = to_date('01-APR-1992','DD-MON-YYYY'); ``` The above query produces the same result ten times faster, completing in under four seconds instead of nearly 40 seconds. The reason is we left the O\_ORDERDATE column unmodified. Surprisingly, despite converting a DATE to a CHARACTER field, Snowflake can still perform some partition pruning. However, modifying WHERE clause columns using a user-defined function will almost certainly lead to a full table scan. ```sql select * from orders where format_date_udf(o_orderdate,'DD-MON-YYYY') = '01-APR-1992'; ``` The above query will perform even worse as Snowflake cannot evaluate the output of a User-Defined Function and will always lead to a full table scan. ## Tuning Sort Operations Designing an analytics solution without including an ORDER BY or GROUP BY clause in the query is almost impossible. However, sorts are the most computationally expensive operations and are often the most significant cause of query performance issues. Often, data analysts need to determine whether a given column is unique and will execute the following SQL: ```sql select count(distinct l_orderkey) from snowflake_sample_data.tpch_sf1000.lineitem; ``` However, the following query will execute around 600% faster with an accuracy of around 99%: ```sql select count_approx_distinct(l_orderkey) from snowflake_sample_data.tpch_sf1000.lineitem; ``` The COUNT\_APPROX\_DISTINCT is one of a series of approximation functions that accurately estimate large data volumes orders of magnitude faster. ## Dealing with high-cardinality data While dealing with large data volumes is often necessary, you should be especially careful when sorting high cardinality data, for example, by a unique key. Take, for example, the following SQL statements, which appear similar but have wildly different query performance. ### Query 1 - Unique Key ```sql select o_orderkey, sum(o_totalprice) from orders group by 1; ``` ### Query 2 - Low Cardinality ```sql select o_orderpriority, sum(o_totalprice) from orders group by 1; ``` You may be surprised to find that query 2 (Low Cardinality) runs around 22 times faster than query 1 (Unique Key), as illustrated in the screenshot below: Side-by-side query profile comparison of a GROUP BY on a unique high-cardinality key versus a low-cardinality key: the high-cardinality query takes 6 minutes 4 seconds and spills 27.52GB to local storage, while the low-cardinality query finishes in 11 seconds with no spilling. The reason for the huge query performance difference is that Query 1 - Unique Key (which takes over six minutes to complete), includes a GROUP BY on a column with 1.5 billion unique values whereas Query 2 - Low Cardinality (executing in 11 seconds) has just five distinct values. In this case, the best practice is to be aware of the high cost of sort operations and avoid GROUP BY or ORDER BY operations on highly unique keys. ## Using the Snowflake LIMIT or TOP option There are sometimes use cases where we need to ORDER very large data volumes by unique keys, but it’s still possible to improve query performance using the simple LIMIT or TOP options on SELECT statements. Consider the following SQL statements: ### Query 1 - Order by Every Row ```sql select * from lineitem order by l_quantity; ``` ### Query 2 - Order by With Limit ```sql select * from lineitem order by l_quantity limit 10; ``` Using the LIMIT 10 option on Query 2 (only first ten rows), leads to an 11 times improvement in query performance from over 17 minutes to just 1m 35s. The reason for the huge difference in query performance is revealed in the Query Plan. Query 1 (Order by every row) on the left includes a **Sort** operation that returns 600 million rows, whereas Query 2 (order by with limit) on the right uses a **SortWithLimit** operation to reduce the entries from 600m to just 10. Side-by-side query plan comparison of ORDER BY versus ORDER BY with LIMIT on 600 million rows: without a limit, the Sort step processes all 600 million rows at 34.8% of runtime, while SortWithLimit with a LIMIT 10 only carries 10 rows forward and drops the sort cost to 0.1%. Another significant contributor to the query performance is that the Order by with limit scans half the number of micro-partitions and eliminates spilling to storage, significantly reducing query execution time. ## Spilling to Storage The diagram below illustrates the internal structure of an XSMALL virtual warehouse. It (currently) consists of eight CPU cores, gigabytes of memory, and hundreds of gigabytes of fast SSD storage attached to slower Remote Storage. Diagram of an X-Small Snowflake virtual warehouse's internal structure, showing its eight CPU cores, RAM chips, and SSD local storage connected to remote cloud storage. hen Snowflake executes a query with an ORDER BY or GROUP BY clause, it must sort the results it attempts to complete in super-fast main memory. However, sorting large data volumes in a smaller warehouse runs out of memory and needs to spill working data to SSD, which is thousands of times slower. When sorting even bigger data volumes, it may need to spill to Remote Storage, which is many hundreds of times slower again. It’s good practice to avoid spilling to storage where possible, but how to do this? There are several Snowflake best practice tuning strategies to avoid spilling to storage, including: 1. **Reduce sort volume:** Similar to the best practices above, this involves using the WHERE clause to filter out results as early as possible to reduce the number of sorted rows. 2. **Use a LIMIT:** As we can see above, a LIMIT clause is pushed down into the query, eliminating spilling to storage with a huge improvement in Snowflake query performance. 3. **Scale Up:** The diagram below illustrates the internal structure of a MEDIUM-size warehouse, which includes four servers\*\*.\*\* By scaling up to a larger warehouse, we can spread the workload across multiple servers, which gives (in this case) four times the memory and SSD, which is the simplest way to resolve the issue. 4. **Cluster the Data:** A final (although sometimes challenging) approach involves clustering the data by the sort key. This often eliminates the SORT operation and can have a massive impact on query performance. ## Clustering the Data The quickest and most simple way of clustering data is to sort it by one or more columns. The SQL snippet below will produce a sorted version of a table: ```sql insert overwrite into orders select * from orders order by o_orderdate; ``` The above query replaces the contents of the ORDERS table with data sorted by ORDER DATE, which means any GROUP BY or ORDER BY won’t need to sort the data. This approach is, however, not without its drawbacks, and it’s worth reading the article [Snowflake Clustering Keys Best Practice](https://www.analytics.today/blog/snowflake-clustering-best-practice) for further guidance. ## Maximizing Snowflake Data Transformation Performance Data Transformation involves fetching, cleaning, merging, and transforming data into a form suitable for end-user consumption, and this often accounts for 60-80% of the overall system cost. It, therefore, must be the primary focus for Snowflake query tuning. In addition to producing results faster, tuning transformation jobs can significantly impact Snowflake's cost. ## Avoid row-by-row-processing Data Engineers and Data Scientists have the option to execute procedural code against Snowflake using Python, Java, Javascript and Snowflake Scripting in addition to Scala using Snowpark. There's a risk however, that coding in a language which supports LOOPS, leads to repeated single-row query execution - also know as "Row-by-Row" processing. > Row by Row Processing - is "Slow-by-Slow" - [Tom Kyte](https://asktom.oracle.com/Misc/slow-by-slow.html) One of the most important query-tuning best practices on Snowflake is to avoid row-by-row processing. Unlike on-premises databases like Oracle, Snowflake is designed primarily for set-at-a-time processing of large data volumes rather than single-row inserts, updates, or deletes. Take the following row-by-row solution, which takes around ten seconds to copy just ten rows into the CUSTOMER table: ```sql declare v_counter number; begin for v_counter in 1 to 10, do insert into customer select * from snowflake_sample_data.tpcds_sf100tcl.customer limit 1; end for; end; ``` Then, consider the following set-at-a-time query that copies data from the same table in 9.4 seconds on an XSMALL warehouse. ```sql insert into customer select * from snowflake_sample_data.tpcds_sf100tcl.customer limit 15000000; ``` In conclusion, using set-at-a-time queries, Snowflake can process fifteen million rows simultaneously as row-by-row processing copies ten rows. That’s a performance improvement of 1.5 million times. The only exception to the above scenario is when dealing with Snowflake Hybrid Tables, designed for OLTP-type applications processing single-row DML operations in milliseconds. ## Scale up for larger Data Volume. This best practice is true for almost any end-user or transformation queries and involves executing long-running queries on a larger virtual warehouse. In principle, provided the query workload can be distributed across all the available servers, any query will run twice as fast on the next warehouse size. Consider the following table showing the benchmark results of copying a 1.3 TB table on different warehouse sizes. Table showing query elapsed time and percentage improvement across Snowflake warehouse sizes, from 4 hours on XSMALL down to 60 seconds on X6LARGE, with roughly 80 to 100 percent improvement at each size increase. The results above demonstrate very large data volumes can be scaled from an XSMALL warehouse running for four hours to an X6LARGE in just 60 seconds. Notice the improvement is around 76% to 100% each time. This is the most important key performance indicator (KPI) when warehouse sizing. Provided the workload completes around twice as fast each time, it’s worth trying the workload on the next size up. The challenge of identifying the optimum warehouse size is illustrated in the diagram below. Chart plotting query execution time against warehouse size from XSMALL to X2LARGE, showing execution time dropping 67 to 115 percent faster at each larger size while query cost stays roughly flat. The above diagram shows the performance of a benchmark query executed on an XSMALL to X2LARGE warehouse. In this case, the query processed a relatively small data volume (several gigabytes), and although each time we scaled up to a larger warehouse, (for example from an XSMALL to a SMALL), the query was executed twice as fast. Since the charge rate per second doubles with warehouse size, we can conclude that each step where the performance is at least double, that the cost will remain the same. > Snowflake has an amazing super-power. You can often get the same work done twice as fast for the same cost. - [Kent Graziano](https://kentgraziano.com/snowflake-resources/) (The Data Warrior) However, each time the performance gain is less than 100%, it costs slightly more, although of course you get the extra performance improvement. For example, running the query on an XLARGE warehouse would cost 12% more than a LARGE warehouse for an 88% performance improvement. The real insight here, however, is that the query time is reduced from 80 minutes on an XSMALL warehouse to just 8 minutes on a LARGE for the same cost. Be aware though, there's almost no performance benefit in scaling up relatively simple, short running queries. The best approach is to execute your workload (all the queries for a given job or set of dashboard users), on a warehouse size, and then scale up until the performance improvement is less than 100%. ## Maximizing Snowflake Data Loading Performance One of the single biggest mistakes made by data engineers is assuming a COPY operation will execute faster on a bigger virtual warehouse. The correct answer is, yes, it will, but it depends. If you execute the following SQL to load data files from the @SALES stage, Snowflake will use the currently assigned virtual warehouse to load data into the SALES table. Let’s assume you use a MEDIUM-size warehouse to maximize load performance. ```sql copy into sales from @sales_stage; ``` If you’ve only a single file to load, the diagram below illustrates how Snowflake executes the COPY operation using a single CPU. Diagram of a single CSV file loading into a multi-cluster Snowflake warehouse: only one of four clusters' CPU, RAM, and SSD is activated, while the other three sit idle, illustrating why a single-file COPY can't use additional clusters. In summary, this won’t improve COPY performance and will take the same time to load on a MEDIUM-size warehouse as an XSMALL. If you deliver hundreds of files, each file will be loaded in parallel on a separate CPU core with massive load performance. As a MEDIUM size warehouse has four servers, each with eight CPU cores, you can load up to 32 files in parallel, effectively improving load performance by 3,200%. Diagram of many files loading in parallel into the same multi-cluster warehouse, with all four clusters' CPU, RAM, and SSD active simultaneously, illustrating how splitting data into more files lets a COPY operation use every available cluster. Be aware, however, it’s best practice to ensure data files are sized between 100-250MB, which means you’ll need a huge volume of data to justify splitting files. It’s, therefore, good practice to use just two warehouses to load all data into Snowflake. One XSMALL warehouse for single files (data loads of up to 250MB compressed), and another MEDIUM or larger warehouse for huge data loads split into file chunks of up to 250MB. Remember, an XSMALL warehouse can load just eight files in parallel; during very busy times, you may be loading more than eight files in parallel. It’s, therefore, also a best practice to configure data-loading warehouses to use the Snowflake scale-out feature. The SQL statement below illustrates how: ```sql create warehouse load_vwh with Warehouse_size = XSMALL min_cluster_count = 1 max_cluster_count = 8 auto_suspend = 30; ``` The diagram below illustrates how the solution works. It shows how multiple parallel COPY operations are executed on a single XSMALL warehouse. As the number of operations reaches eight parallel loads, Snowflake automatically adds additional servers to cope with the additional workload. Diagram of three CSV files each loading in parallel through their own compute instance within a single X-Small multi-cluster warehouse into separate destination tables, illustrating how Snowflake scales out clusters as concurrent COPY operations increase. ## Conclusion In conclusion, Snowflake deploys a revolutionary hardware and software architecture to maximize query performance, supporting scaling up (to a larger warehouse) and out (for more concurrent queries). Using the simple query performance strategies outlined in this article can greatly impact overall query performance, often for very little effort. --- Deliver more performance and cost savings with Altimate AI's cutting-edge AI teammates. These intelligent teammates come pre-loaded with the insights discussed in this article, enabling you to implement our recommendations across millions of queries and tables effortlessly. Ready to see it in action? Request a recorded demo by just sending a chat message ([here](https://app.myaltimate.com/contactus) ) and discover how AI teammates can transform your data teams. [Promotional banner reading "AI Data Teammates that work for you 24/7" with a Book a Demo button, next to a screenshot of the Auto Tune dashboard showing realized and estimated savings on a Snowflake warehouse.](https://app.myaltimate.com/contactus) Want the next one first? New benchmarks, cost-optimization breakdowns, and data-engineering deep dives — sent when we publish something worth reading, nothing else. [Snowflake](https://altimate.ai/blog?tag=snowflake) [Data Engineering](https://altimate.ai/blog?tag=data-engineering) [AI](https://altimate.ai/blog?tag=ai) Share Get started ## Ready to get started? You are only a few clicks away from experiencing your own autopilot for data. [Get Started for Free →](https://app.myaltimate.com/register)