Databricks Serverless: Critical Questions During Your Migration
Catch what breaks now, not after it's live.
Once you've decided to move a workload, the code and config changes aren't optional, and some of them quietly introduce new inefficiency instead of fixing anything. Native tooling flags some compatibility issues. The rest surfaces only after something's already broken.
The same four checks apply to every workload you migrate, not just one. Native tooling catches some of what needs to change; it won't tell you whether a completed migration introduced a regression, or what else depends on the workload you just moved. Here's what that gap looked like for one real pipeline, then the breakdown for each check below.
+121% runtime, same pipeline logic
What breaks, why, and how to catch it.
Each question below is one of the checks a workload needs during and right after it moves, how far Databricks native tooling gets you toward catching it, and what only Unravel can tell you.
What code or config actually needs to change for this workload?
Every workload carries some combination of environment variables, init scripts, unsupported languages, DBFS mounts, or cache calls that classic clusters tolerate and serverless doesn't. None of it is a reason to avoid serverless, but each one is a specific line of code or config that has to change before the job runs correctly, and missing one is exactly what makes a migration look done in QA and then misbehave once it's live.
Answerable by
Is this a good low-risk candidate to migrate now, or should it wait?
Not every workload is equally ready today, even if all of them would benefit from serverless eventually. A workload with heavy custom configuration, sensitive concurrency patterns, or dependencies on still-maturing serverless support is a worse candidate to migrate right now than one that's simple and stable. Sequencing which workloads move first, and which wait, matters as much as the code changes themselves.
Answerable by
Did the last change introduce new inefficiency?
A migration can pass every functional test and still get slower or more expensive, because serverless's compute allocation and scaling behavior aren't identical to classic's. Partition skew, shuffle behavior, and scaling responsiveness can all play out differently under serverless even when the underlying job logic hasn't changed at all, and a passing QA suite catches correctness, not a performance regression.
Answerable by
Is anything this workload depends on going to break?
Workloads rarely run in isolation. Upstream tables, shared libraries, downstream jobs, and scheduled dependencies can all be affected by a migration even when the workload being migrated looks self-contained. A change that's safe for the job itself can still break something else that reads its output or shares its environment.
