Databricks Serverless Webinar
Register Now
Close navigation menu icon in white

Use Databricks serverless compute by default for jobs, notebooks and pipelines. Databricks now recommends it for almost all pipelines, and it bills only while work runs. Stay on classic compute when a workload needs something serverless doesn't support: JAR libraries, GPUs or the ML runtime, Scala or R, init scripts, instance pools, or runs longer than 7 days.

Cost is the other reason to stay. In our benchmarks, long-running batch jobs cost more on serverless and ran slower. Below: how the two differ, how each is billed, a classic-only checklist, and how to check that a switch paid off.

How serverless and classic compute differ

Classic compute runs clusters in your own cloud account. You pick the instance types, the Databricks Runtime version, autoscaling limits and Spark settings. Serverless runs in a compute plane Databricks manages. You don't size, patch or tune anything.

Classic computeServerless compute
InfrastructureClusters in your cloud account. You size and maintain themManaged by Databricks. No cluster to configure
BillingDBUs from Databricks, plus VMs and storage from your cloud providerDBUs only. The serverless rate includes the infrastructure
Billing unitPer second while the cluster is up, busy or idlePer second while work runs
StartupMinutes, unless you keep warm instance poolsFast in performance-optimized mode. 4 to 6 minutes in standard mode
Spark configurationFull controlA short list of supported properties
Runtime versionYou pin a specific Databricks RuntimeVersionless. You choose an environment version for dependencies
LanguagesPython, SQL, Scala, RPython and SQL
LibrariesAnything, including JARs and init scriptsNotebook-scoped and environment dependencies. No JARs or init scripts
GPUs and ML runtimeSupportedNot supported
Cost controlsCluster policiesServerless budget policies. Cluster policies don't apply
Max run lengthNo serverless-style cap7 days

Sources: Databricks serverless compute limitations, Serverless vs. classic compute for pipelines.

How each one is billed

Both bill in Databricks Units (DBUs), metered per second. The difference is what the DBU covers.

  • Classic: you pay Databricks for DBUs, and you pay your cloud provider separately for the VMs and storage under the cluster. An idle cluster still runs up both bills until it terminates.
  • Serverless: one higher DBU rate that already includes the infrastructure. Billing stops when the work stops.

The serverless DBU rate is higher, but it covers the infrastructure too. Compare total cost per run.

Approximate list rates on AWS (Premium tier, as of Oct 2026):

ComputeAbout per DBUCloud VMs billed separately?
Classic Jobs compute$0.15Yes
Classic All-Purpose compute$0.55Yes
Jobs Serverless$0.35 to $0.40No
SQL Serverless$0.70No
Interactive Serverless (notebooks)$0.75No

Rates vary by cloud, region, tier and contract. Check the Databricks pricing page for yours.

Cost per run, for each option:

  • Classic: (DBUs per hour x DBU rate + VM cost per hour) x hours the cluster is up, including startup and idle time
  • Serverless: DBUs consumed x serverless DBU rate

[cta-banner]

What we see when workloads move to serverless

Results swing by workload type. When we benchmarked serverless against classic across customer environments (full write-up), the same kinds of jobs landed in very different places:

WorkloadCost on serverlessSpeed on serverless
Short-duration jobs47% lower15% faster
Optimized queries33% lowerAbout the same
Complex analytical workloads234% higher78% faster
Long-running batch jobs7% higher103% slower

Short and well-written work wins on serverless. Heavy analytics can finish faster but cost more than three times as much. Long batch jobs can lose on both cost and time.

Inefficient code costs more on serverless, because you pay for every second of work. We see about 40% of enterprise Databricks spend going to inefficiency, and more than half of that comes from poorly written code. Fix the code before you move it.

When to stay on classic compute

If a workload needs anything in this list, it stays on classic for now:

Classic-only needWhat to know
JAR libraries or Maven coordinatesNot supported on serverless
GPUs, Databricks Runtime ML or Spark MLlibNot supported. Keep model training on classic
Scala or RServerless supports Python and SQL only
Spark RDD APIsServerless supports Spark Connect APIs only
Init scripts or compute-scoped librariesNot supported. Use environment dependencies where you can
Instance poolsNot supported
DataFrame or SQL cache APIs (CACHE TABLE and similar)Not supported. Calls raise an exception
Databricks Container ServicesNot supported
Runs longer than 7 daysBreak them into smaller runs, or stay on classic
Some private networking setupsCheck the limitations page for your cloud

Cost can also be a reason to stay. A long, steady, well-tuned batch job on a right-sized job cluster may cost less on classic. Test it before you move it.

When serverless costs more than classic

Watch for these patterns after a move:

  • Long batch jobs at steady load. A tuned job cluster with spot instances can beat the all-in serverless rate.
  • Heavy analytics. Faster runs don't help if each run costs 3x.
  • Inefficient code. Bad joins, wide shuffles and repeated scans cost more when you pay per second of work.
  • Performance-optimized mode left on. It's the default. It keeps warm compute ready so jobs start fast. Standard mode starts in 4 to 6 minutes and can cut cost by up to 70% for jobs and pipelines, per Databricks. Both modes use the same SKU. Standard just uses fewer DBUs. It isn't available for notebooks. Use performance-optimized only where an SLA needs fast starts.

[cta-banner]

How to switch a workload from classic to serverless

Start with short, bursty jobs and notebooks that use standard Python or SQL. Move long, heavily tuned jobs last, or not at all. For each move, answer four questions.

What needs to change in the code or config?

Custom Spark configs, init scripts, instance types and compute-scoped libraries don't carry over. Move dependencies into the serverless environment, and check that every Spark property you rely on is on the supported list. A quick test from Databricks: run the job on classic compute with standard access mode and Databricks Runtime 14.3 or above. If it runs there, it's ready for serverless. Then run the real job on serverless before you switch the schedule.

Is this a low-risk candidate to move now?

Good first candidates run quickly, have unpredictable volume and don't depend on a custom environment. Hold back anything on the classic-only list above.

Will anything downstream break?

Dashboards, downstream jobs and triggers that assume a certain runtime or finish time can break quietly when compute changes. Map dependencies first. For streaming, use Lakeflow pipelines in continuous mode or Trigger.AvailableNow(). Serverless doesn't support default or time-based triggers.

Did the move add new waste?

A job can pass its first test and drift weeks later as data grows. Recheck runtime and cost after the first few production runs, then again after a month. Cluster policies don't apply to serverless, so set serverless budget policies to tag and track spend by team.

How to check the switch paid off

Compare cost per run before and after in Databricks System Tables. This query totals list cost by SKU:

SELECT u.sku_name,
       SUM(u.usage_quantity * p.pricing.default) AS list_cost
FROM system.billing.usage u
JOIN system.billing.list_prices p
  ON u.cloud = p.cloud
 AND u.sku_name = p.sku_name
 AND u.usage_start_time >= p.price_start_time
 AND (p.price_end_time IS NULL OR u.usage_start_time < p.price_end_time)
GROUP BY u.sku_name;

Filter by job ID and date range to compare the same job on classic and on serverless. Remember the classic number is missing the cloud VM bill.

Doing this for one job is easy. Doing it for every job, every week, is where teams fall behind. Arvix, Unravel's autonomous AI operator, traces cost to every workload, resizes compute based on actual demand, and tracks the savings from each change. You decide how autonomous it is, per workload or per team. For more on serverless planning, see the Databricks Serverless Compute guide.

Find out which of your workloads belong on serverless

Get a free Databricks Health Check. We'll show which jobs are good serverless candidates, which should stay on classic, and where your current spend is going. Learn more about Databricks cost optimization with Unravel.

Databricks Serverless Compute: The Platform Owner's Guide to Making Better Serverless Decisions
Read the Guide
Unlock your data environment health with a free health check.
Request Your Health Check Report
Table of Contents