Visit us in SF Data + AI Summit

** DO NOT REMOVE Hidden Margin Required **

Azure Databricks Integration Overview

Lakehouse on Azure meets data observability.

** DO NOT REMOVE Hidden Margin Required **
** DO NOT REMOVE Hidden Margin Required **
AI-ENABLED PERFORMANCE, COST, AND QUALITY MANAGEMENT FOR AZURE DATABRICKS

Azure Databricks + Unravel = Intelligent Lakehouse

Whether you are building a lakehouse with Azure Databricks for cloud data pipelines, BI, streaming analytics, or data science, Unravel’s AI-powered data observability for Azure Databricks simplifies DataOps challenges, increases efficiency, and boosts productivity, while reducing total cost of ownership. Unravel provides AI insights to proactively pinpoint and resolve data pipeline performance issues, ensure data quality, and define automated guardrails for predictable spend. Try Unravel for free.

** DO NOT REMOVE Hidden Margin Required **
** DO NOT REMOVE Hidden Margin Required **
** DO NOT REMOVE Hidden Margin Required **
** DO NOT REMOVE Hidden Margin Required **
** DO NOT REMOVE Hidden Margin Required **

Unravel tackles your biggest headaches

COST GOVERNANCE

Understand, optimize and actively govern your costs

Get Cloud Spending Under Control

AI-enabled cost governance identifies where you’re spending more than you have to (and how to fix it), with guardrails to proactively manage costs and prevent budget overruns.

  • Allocate costs with pinpoint precision
  • Preemptively prevent budget overruns
  • Nip cost overruns in the bud
Visit Use Case

OPTIMIZATION

Optimize for performance and cost before you deploy

Enable Self-Service Tuning

Automated AI recommendations eliminate trial-and-error tuning. Unravel cuts to the chase to tell you exactly how to change code or configurations for better performance and cost.

  • See all performance details in one place
  • Optimize 100s (or 1000s) of jobs automatically
  • Optimize cloud costs at the cluster level
Visit Use Case

TROUBLESHOOTING

Less effort, more problem-solving—faster, easier

Reduce Firefighting & Accelerate MTTR

No more spending hours (or days) doing time-consuming manual detective work. Unravel’s automated root cause analysis pinpoints why jobs fail or where pipelines are bottlenecked.

  • Diagnose why jobs failed in minutes, not hours
  • Pinpoint pipeline problems automatically
  • Drill into details down to individual lines of code
Visit Use Case

DATA QUALITY

Automatically correlate external data quality checks with AI-driven insights

Move fast to triage quality issues

Unravel integrates data quality check results from other tools, correlates all data details into a workload-aware context, and applies AI analysis for automated insights.

  • Drill down into all data details from single unified view
  • Understand the upstream/downstream impact in seconds
  • Apply automated circuit-breaker guardrails
Visit Use Case

CLOUD MIGRATION

Avoid landmines and setbacks before, during, and after migration

Migrate on schedule & under budget

Avoid migration setbacks and cost overruns. Unravel’s deep intelligence and automation enables confident, data-driven decisions before, during and after your move to the cloud.

  • Map workloads and calculate costs in seconds
  • Discover & visualize clusters automatically
  • Avoid tuning/replatforming delays
Visit Use Case

EXPLORE KEY FEATURES

See how Unravel’s top features and capabilities work

What Can Unravel Do for You?

2-minute demo videos and self-paced guided tours walk you through the “best of” Unravel.

  • Get precise recommendations for right-sizing resource
  • Pinpoint root causes of performance problems in seconds
  • Automatically see where and how to optimize configurations
Explore Key Features

Understand, optimize and actively govern your costs

Get Cloud Spending Under Control

AI-enabled cost governance identifies where you’re spending more than you have to (and how to fix it), with guardrails to proactively manage costs and prevent budget overruns.

  • Allocate costs with pinpoint precision
  • Preemptively prevent budget overruns
  • Nip cost overruns in the bud
Visit Use Case

Optimize for performance and cost before you deploy

Enable Self-Service Tuning

Automated AI recommendations eliminate trial-and-error tuning. Unravel cuts to the chase to tell you exactly how to change code or configurations for better performance and cost.

  • See all performance details in one place
  • Optimize 100s (or 1000s) of jobs automatically
  • Optimize cloud costs at the cluster level
Visit Use Case

Less effort, more problem-solving—faster, easier

Reduce Firefighting & Accelerate MTTR

No more spending hours (or days) doing time-consuming manual detective work. Unravel’s automated root cause analysis pinpoints why jobs fail or where pipelines are bottlenecked.

  • Diagnose why jobs failed in minutes, not hours
  • Pinpoint pipeline problems automatically
  • Drill into details down to individual lines of code
Visit Use Case

Automatically correlate external data quality checks with AI-driven insights

Move fast to triage quality issues

Unravel integrates data quality check results from other tools, correlates all data details into a workload-aware context, and applies AI analysis for automated insights.

  • Drill down into all data details from single unified view
  • Understand the upstream/downstream impact in seconds
  • Apply automated circuit-breaker guardrails
Visit Use Case

Avoid landmines and setbacks before, during, and after migration

Migrate on schedule & under budget

Avoid migration setbacks and cost overruns. Unravel’s deep intelligence and automation enables confident, data-driven decisions before, during and after your move to the cloud.

  • Map workloads and calculate costs in seconds
  • Discover & visualize clusters automatically
  • Avoid tuning/replatforming delays
Visit Use Case

See how Unravel’s top features and capabilities work

What Can Unravel Do for You?

2-minute demo videos and self-paced guided tours walk you through the “best of” Unravel.

  • Get precise recommendations for right-sizing resource
  • Pinpoint root causes of performance problems in seconds
  • Automatically see where and how to optimize configurations
Explore Key Features
** DO NOT REMOVE Hidden Margin Required **
** DO NOT REMOVE Hidden Margin Required **

Commonly asked questions

Are cloud compute and storage costs included in Azure Databricks Units (DBUs)?

No. Azure Databricks Units (DBUs) are reference units of Databricks Lakehouse Platform capacity used to price and compare data workloads. DBU consumption depends on the underlying compute resources and the data volume processed. Cloud resources such as virtual machines (VMs) and cloud storage are priced separately. Azure Databricks is billed by the second. You can estimate costs online for Azure Databricks, then add estimated cloud compute and storage costs with the Azure pricing calculator.

Learn more about FinOps for data teams

Where can I see cloud compute and storage costs associated with my Azure Databricks clusters?

Cost 360 for Databricks provides trends and chargeback by app, user, department, project, business unit, queue, cluster, or instance. You can see a cost breakdown for Azure Databricks clusters in real time, including related services such as DBUs and VMs for each configured Azure Databricks account on the Databricks Cost Chargeback details tab. In addition, you get a holistic view of your cluster, including resource utilization, chargeback, and instance health, with automated AI-based cluster cost-saving recommendations and suggestions.

Learn more about cost governance

How can I improve my Azure Databricks performance?

Azure Databricks offers optimization suggestions and troubleshooting tips for certain scenarios, for example, auto optimize to compact small files. Unravel provides recommendations, efficiency insights, and tuning suggestions on the Applications page and the Jobs tab. With a single Unravel instance, you can monitor all your clusters, across all instances, and workspaces in Azure Databricks to ​​speed up your applications, improve your resource utilization, and identify and resolve application problems.

Learn more about AI-enabled optimization

Does Overwatch provide real-time reporting on my Databricks resource usage?

Real time monitoring and alerting with Databricks Overwatch requires a time-series database. Azure Databricks refreshes your cost and usage data about every 24 hours and anomalies are identified daily. Since Microsoft Cost Management and Billing includes usage and costs of other services, you should tag your Azure Databricks resources and you may consider creating custom tags to get granular reporting on your Databricks cluster resource usage. Unravel simplifies this process with Cost 360 for Databricks to provide full cost observability, budgeting, forecasting and optimization in near real time. Cost 360 includes granular details about the user, team, data workload, usage type, data job, data application, compute, and resources consumed to execute each data application. In addition, Cost 360 provides insights and recommendations to optimize clusters and jobs as well as estimated cost improvements to prioritize workload optimization.

Learn more about cost governance

How can I ensure high-quality data in my data lake?

Data teams spend most of their time preparing data—data aggregation, cleansing, deduplication, synchronizing and standardizing data, ensuring data quality, timeliness, and accuracy, etc.—rather than actually delivering insights from analytics. Everybody needs to be working off a “single source of truth” to break down silos, enable collaboration, eliminate finger-pointing, and empower more self-service. Although the goal is to prevent data quality issues, assessing and improving data quality typically begins with monitoring and observability, detecting anomalies, and analyzing root causes of those anomalies.

Learn more about flexible data quality

Does Azure Databricks provide insights to help me tune my clusters?

Azure Databricks collects monitoring and operational data in the form of logs, metrics, and monitoring for your Azure Databricks job flows. Azure Databricks metrics can be used to detect basic conditions such as idle clusters and nodes or clusters that run out of storage. Troubleshooting slow clusters and failed jobs involves a number of steps such as gathering data and digging into log files. Data application performance tuning, root cause analysis, usage forecasting, and data quality checks require additional tools and data sources. Unravel accelerates the troubleshooting process by creating a data model using metadata from your applications, clusters, resources, users, and configuration settings, then applying predictive analytics and machine learning to provide recommendations and automatically tune your Azure Databricks clusters.

Learn more about automated troubleshooting

Do I need to set up VNET peering for Azure Databricks?

Virtual network (VNET) peering enables you to create a network connection between Azure Databricks clusters and your Azure resources, even across regions, enabling you to route traffic between them using private IP addresses. For example, if you are running both an Unravel VM and Azure Databricks cluster in the East US region but configured with different VNET and subnet, there is no network access between the Unravel VM and Databricks cluster by default. To enable network access, you can set up VNET peering between your Azure Databricks master node and your Unravel VM.

Learn more about cloud migration

How does Unravel help when migrating to Azure Databricks?

Unravel provides granular Insights, recommendations, and automation for before, during and after your Spark, Hadoop and data migration to Azure Databricks.

Get granular chargeback and cost optimization for your Azure Databricks workloads. Unravel for Azure Databricks is a complete data observability platform to help you tune, troubleshoot, cost-optimize, and ensure data quality on Azure Databricks. Unravel provides AI-powered recommendations and automated actions to enable intelligent optimization of data pipelines and applications.

Learn more about cloud migration

Show More
** DO NOT REMOVE Hidden Margin Required **
*** Do NOT REMOVE Hidden Margin Required ***