Unravel launches free Snowflake native app Read press release

Snowflake

Snowflake Warehouse Sizing Isn’t a Substitute for Query Optimization

Enterprise data teams frequently respond to slow query performance by increasing warehouse size. A Medium warehouse scaled to Large, then X-Large, then 2X-Large over several months represents an 8x increase in compute costs. This pattern typically […]

  • 7 min read

Enterprise data teams frequently respond to slow query performance by increasing warehouse size. A Medium warehouse scaled to Large, then X-Large, then 2X-Large over several months represents an 8x increase in compute costs. This pattern typically occurs without analyzing whether the performance bottleneck stems from insufficient compute resources or inefficient query design.

Most slow queries require query optimization, not larger warehouses. A query scanning entire tables due to functions in WHERE clauses preventing partition pruning will run faster on a larger warehouse but still processes unnecessary data. The larger warehouse makes the inefficient query more expensive to execute while masking the underlying optimization opportunity.

Effective Snowflake warehouse sizing requires understanding when scaling genuinely helps versus when query optimization is required.

When Snowflake Warehouse Sizing Actually Improves Performance

Snowflake warehouse sizing options range from X-Small to 6X-Large. Each tier doubles cores, memory, and storage. X-Small uses 1 credit per hour, Medium uses 4, Large uses 8. Strategic warehouse sizing delivers 2x performance for the same cost when queries are genuinely compute-constrained, but only when the query actually benefits from additional compute.

CPU-Bound Operations

Complex aggregations calculating percentiles and correlations utilize every available core. These workloads genuinely benefit from larger warehouses, showing 90%+ CPU utilization across all nodes in the query profile.

Doubling warehouse size from Medium to Large doubles available cores, cutting execution time roughly in half. This only works when queries perform heavy computation across large datasets and CPU is actually the bottleneck.

Memory-Intensive Workloads and Spillage

When Snowflake runs out of memory during execution, performance degrades drastically.

Data spills first to local SSD storage, then to remote cloud storage if local storage fills. Spillage kills performance because disk I/O is orders of magnitude slower than memory, and remote storage I/O is even slower than local disk.

Spillage to local storage measured in gigabytes signals memory pressure. Any spillage to remote storage clearly indicates that current Snowflake warehouse sizing is undersized for the query’s memory requirements. A query spilling 100GB to remote storage on Medium might complete 3-4x faster on Large with double the available memory, turning a 45-minute query into a 12-minute query for the same total credit consumption.

Workloads that consume substantial memory:

  • Large hash joins processing billions of rows
  • Window functions over massive partitions
  • Complex CTEs with large intermediate results
  • Aggregations on high-cardinality dimensions

Query profiles reveal spillage clearly. Remote spillage makes queries run 10-100x slower. Increasing Snowflake warehouse sizing to eliminate this spillage often saves money despite higher per-hour costs because the query completes in a fraction of the time.

I/O-Bound Queries on Massive Datasets

Full table scans on very large tables measured in terabytes are I/O-bound. When a query must scan 5TB distributed across 50,000 micro-partitions, more compute nodes parallelize the scan through Snowflake’s micro-partition architecture that allows each core to retrieve partitions independently.

TableScan operations consuming 80%+ of query execution time indicate I/O-bound workloads. Appropriate Snowflake warehouse sizing accelerates these scans through increased parallelization.

Recognizing When Larger Warehouses Help

Query profiles reveal whether Snowflake warehouse sizing changes will improve performance.

Performance indicators that benefit from larger warehouses:

  • High CPU utilization across all nodes (90%+ throughout execution)
  • Spillage to local storage exceeding several gigabytes
  • Any spillage to remote storage (critical indicator)
  • Heavy TableScan operations with thousands of micro-partitions

Performance indicators unrelated to warehouse size:

  • Low CPU utilization (20-30% across nodes)
  • No spillage anywhere
  • Queries dominated by network transfer time
  • Single-threaded operations that can’t parallelize

Scaling up provides no benefit when queries show these patterns.

When Snowflake Warehouse Sizing Wastes Money

Most slow queries stem from poor query design, not insufficient compute resources. Adjusting Snowflake warehouse sizing masks these problems while making them more expensive to run.

Poor Query Design

Cartesian products from missing JOIN conditions process exponentially more data regardless of warehouse size. A query joining three tables without proper WHERE clauses might generate billions of intermediate rows that must all be processed, sorted, and filtered before returning results.

SELECT *
FROM customers c, orders o, products p
WHERE c.region = 'US';

Cartesian product: every order matched with every product. With a million orders and 10,000 products, that’s 10 billion intermediate rows that must all be processed, sorted, and filtered.

Doubling warehouse size processes this massive intermediate result 2x faster, but the query still performs unnecessary work. The fix is adding proper JOIN conditions, not Snowflake warehouse sizing adjustments that simply make the wasteful query more expensive.

Queries using SELECT * from wide tables retrieve far more data than needed. Snowflake’s columnar storage only reads columns referenced in the query, but SELECT * forces retrieval of every column. A query needing 5 columns from a 200-column table that uses SELECT * transfers 40x more data than necessary.

Scaling doesn’t address this waste. It just moves wasted data faster.

Missing Optimization Opportunities

Many slow queries result from missing or improper use of Snowflake’s optimization features.

Increasing Snowflake warehouse sizing masks these problems.

Clustering key issues appear when queries filter on high-cardinality columns that aren’t clustered. A table storing two years of transaction data might have excellent natural clustering on ingestion timestamp but terrible clustering on customer_id. Queries filtering by customer_id scan most micro-partitions because data scatters across thousands of partitions.

Adding a clustering key on customer_id organizes data so queries can prune partitions effectively. A properly clustered table might reduce scanned data by 95%, making the query fast on Small where it previously struggled on X-Large.

Functions in filter predicates prevent partition pruning. A query with WHERE YEAR(transaction_date) = 2024 must scan all micro-partitions because Snowflake can’t use partition-level statistics when functions wrap the filtered column.

Rewriting as WHERE transaction_date >= '2024-01-01' AND transaction_date < '2025-01-01' enables partition pruning, potentially reducing scanned partitions by 95%+. The query becomes fast on any warehouse size because it scans less data.

Materialized views eliminate repeated computation. If a dashboard query aggregates 10TB of transaction data every hour, that’s expensive computation regardless of Snowflake warehouse sizing. Creating a materialized view that maintains daily aggregates incrementally eliminates 99% of the work. Dashboard queries complete in seconds on X-Small instead of minutes on X-Large.

Network Bottlenecks

Queries returning millions of rows spend execution time transferring data over the network.

A query returning 5 million rows generates 400MB-1GB of result data. Transferring this volume takes time that scales with bandwidth, not warehouse size. The query computation might complete in seconds, but result transfer takes minutes. Increasing Snowflake warehouse sizing from Large to X-Large makes query execution 2x faster but has no impact on result transfer that dominates total time.

Result sets larger than a few hundred thousand rows generally indicate design problems.

Stop wasting Snowflake spend—act now with a free health check.

Request Your Health Check Report

Enterprise Patterns That Waste Warehouse Spend

At enterprise scale, improper Snowflake warehouse sizing compounds across hundreds of warehouses and millions of queries.

The Reflexive Scaling Pattern

Teams develop a reflex: query runs slowly, increase warehouse size. This creates a ratchet effect where warehouses only scale up, never down, because teams fear reintroducing performance problems once they’ve “solved” them with larger warehouses.

A data engineering team launches a new ETL pipeline on Medium. As data volumes grow, some jobs slow down. Scale to Large. Performance improves. Three months later, a complex transformation times out. Scale to X-Large. Another quarter passes, user-facing reports timeout, scale to 2X-Large.

The warehouse has grown from 4 credits per hour to 32 credits per hour. An 8x cost increase. But nobody analyzes whether those decisions addressed actual compute constraints or masked query optimization opportunities. Perhaps half the pipeline could run efficiently on Medium with optimized queries. Maybe 30% is genuinely memory-bound and needs X-Large, but 70% is overpaying for resources it doesn’t use.

This reflexive pattern appears across teams:

  • Marketing analytics scales their warehouse for year-end reporting load, then leaves it there year-round
  • Finance provisions Large “just to be safe” for monthly close processes that run fine on Medium
  • Data science teams request X-Large for exploratory work that mostly involves small sample queries
  • Engineering defaults to “next size up” without profiling actual resource utilization

Mixed Workload Inefficiency

A warehouse handling both 2-second dashboard queries and 2-hour batch ETL jobs can’t be sized well for either workload.

Simple queries don’t benefit from Large or X-Large warehouses. Distributing trivial work across many nodes adds overhead that can actually slow execution. But these fast queries still pay the full Large credit rate despite using a fraction of available resources.

The solution isn’t finding one “right” Snowflake warehouse sizing for everything. It’s separating workloads: Small for dashboards, Large for analytics, X-Large for batch processing. Match warehouse size to actual workload characteristics instead of forcing everything through the same oversized warehouse.

Intelligent Snowflake Warehouse Sizing with Unravel

Manually analyzing query patterns across thousands of queries becomes impractical at enterprise scale.

Teams need automated intelligence that doesn’t just identify problems but implements solutions.

Unravel’s FinOps Agent analyzes query execution patterns and resource utilization continuously across all Snowflake workloads. Built natively on Snowflake System Tables, the FinOps Agent identifies genuine compute constraints like remote spillage, CPU-bound operations, and I/O bottlenecks, then distinguishes them from queries requiring optimization rather than Snowflake warehouse sizing changes.

The FinOps Agent provides specific recommendations and implements changes based on your governance settings. Query X shows Cartesian product? Rewrite JOIN logic (recommendation requiring approval). Warehouse Y shows remote spillage? Automatically resize to Large during ETL window (auto-implemented based on proven pattern). This intelligence separates queries needing better SQL from queries genuinely benefiting from Snowflake warehouse sizing adjustments.

Teams control the automation level:

  • Start with recommendations requiring manual approval for all changes
  • Enable auto-implementation for specific optimization types as confidence builds
  • Implement full automation with governance controls for proven patterns

This moves from insight to action. The FinOps Agent implements fixes based on your governance preferences rather than just flagging issues.

Results include 25-35% sustained cost reduction by eliminating reflexive warehouse scaling that masks optimization opportunities. Teams automatically right-size workloads based on actual compute constraints, scaling up for genuine spillage while flagging queries that need better SQL, not bigger warehouses.

Optimize First, Scale Second

Snowflake warehouse sizing is a legitimate performance optimization technique for queries genuinely constrained by CPU, memory, or I/O capacity. Remote spillage, high CPU utilization, and massive parallel scans benefit from larger warehouses. But most slow queries need better SQL, not bigger warehouses.

Cartesian products, missing clustering keys, functions in predicates, and inefficient JOINs waste resources regardless of Snowflake warehouse sizing. Scaling up just makes these bad queries expensive instead of slow.

The optimization-first approach:

  • Check query profiles for actual bottlenecks
  • Verify partition pruning effectiveness
  • Optimize JOIN strategies
  • Confirm proper clustering
  • Review result sizes

Fix the query design first. Then adjust Snowflake warehouse sizing if genuinely needed after optimization. Teams that default to scaling for every slow query build expensive technical debt, while teams that optimize queries first and scale strategically control costs while delivering performance.

 
 

Other Useful Links