Databricks Optimization Webinar
Join Us Aug 6
Close navigation menu icon in white

Know before you move it, not after you're billed.

Before a workload moves to serverless, you need real answers on its duration, data volume, concurrency, and custom requirements, plus what the move will actually cost. Databricks native tooling answers some of that on its own. The rest is on you, unless you know exactly where to look.

The same five questions apply to every workload you run, not just one. Native tooling answers some of this manually, one workload at a time, if you go looking. A few of these questions, it can't answer at all. Here's what that actually looks like for one real workload, then the breakdown for each question below.

Real example
+275% daily cost, same workload
A nightly ETL job with a steady two-hour runtime looks like an easy serverless candidate because it runs unattended. It isn't. This is exactly the profile a flat-rate classic cluster is priced for, and per-second serverless billing punishes long, steady runtimes instead of rewarding them: $16 a day on classic, $60 a day on serverless, same workload, a 275% jump.
Critical Questions to Answer

What to ask, why, and how to answers it.

Each question below carries the threshold it's checking, how far Databricks native tooling gets you, and what only Unravel can tell you.

Duration

Does it run under or over 30 minutes?

Serverless bills per second and starts almost instantly, which rewards short workloads that spin up and finish. Classic clusters have real startup and idle overhead, but once a cluster's running, flat hourly pricing doesn't care how long the job takes. 30 minutes is roughly where those two economics cross: below it, serverless's speed and per-second billing win; above it, classic's flat rate starts winning instead.

Yes
Under 30 min
Serverless fits. Instant startup and pay-per-use work in your favor.
NO
Over 30 min
Classic fits better. Startup cost becomes negligible against predictable hourly pricing.
Answerable by
Native Databricks
✓ Yes
The Jobs UI shows run duration directly in your job's run history.
Unravel
✓ Yes
Tracks run duration across every workload automatically, flagging jobs that are creeping past the 30-minute line before it becomes a problem.
Data volume

How much data does it move, and how variable is that?

Data volume drives both the compute a workload needs and what that compute costs under each pricing model. Steady volumes are easy to size a classic cluster for once and leave alone. Variable volumes are harder to size correctly on classic: you're either overprovisioning for the peak or getting squeezed during it. Serverless sidesteps that sizing problem entirely by scaling to whatever the workload needs, run by run.

Yes
Smaller, variable batches
Serverless fits. It scales automatically without needing to size for peak volume.
NO
10GB+ regularly
Classic fits better. A properly sized cluster handles predictable, large volume efficiently.
Answerable by
Native Databricks
◐ Partial
Spark UI and run details show data read and written, but one run at a time. No trend view across history.
Unravel
✓ Yes
Aggregates volume across every historical run automatically and surfaces the variability, no manual digging.
Concurrency

Is its concurrency steady or spiky?

Concurrency determines how much parallel capacity a workload needs at any given moment. Steady concurrency is easy to provision for on classic: size the cluster once and it stays right-sized. Spiky concurrency is where classic clusters either sit idle most of the time or get overwhelmed during bursts, because a fixed cluster can't flex fast enough. Serverless absorbs those spikes automatically, since it scales per request instead of per pre-provisioned capacity.

Yes
Spiky
Serverless fits. It scales to absorb concurrency spikes automatically.
NO
Steady
Classic fits better. A fixed-size cluster efficiently handles predictable concurrency.
Answerable by
Native Databricks
✕ No
Not surfaced as a pattern. You'd have to reconstruct it yourself from cluster metrics, run by run.
Unravel
✓ Yes
Correlates concurrent execution over time and flags whether the pattern is steady or spiky.
Custom needs

Does it need GPU, custom libraries, or fine-tuning?

Serverless compute runs in a shared, managed environment that Databricks controls end to end, which is what makes the instant start and automatic scaling possible. That same managed environment is also why it doesn't support every configuration: GPU access, custom libraries, and fine-tuned runtime settings all require more control over the underlying compute than serverless is built to give you.

Yes
Standard SQL or Python, no tuning
Serverless fits. No custom configuration required.
NO
GPU, custom libraries, or fine-tuning
Classic fits better. You need direct control over the compute environment.
Answerable by
Native Databricks
✓ Yes
This is your own job configuration, visible directly in the job and cluster definition you already built.
Unravel
✓ Yes
Surfaces GPU, custom library, and fine-tuning requirements across every workload automatically, so you're not opening each job's configuration one at a time to check.
Cost comparison

What would it actually cost on serverless vs. classic?

This is the question the other four feed into. Duration, data volume, concurrency, and custom requirements all shape the number, but none of them tell you the number itself. Only a real cost simulation, run against this specific workload's actual telemetry, does that.

Yes
Serverless comes in lower
Serverless fits. The cost projection favors serverless for this workload.
NO
Classic comes in lower
Classic fits better. The cost projection favors classic for this workload.
Answerable by
Native Databricks
✕ No
Databricks has no side-by-side cost simulation across compute types for a given workload.
Unravel
✓ Yes
Simulates cost under both pricing models using the workload's actual telemetry: DBUs, runtime, data volume.