Announcing Unravel 4.8.1: Maximize business value with Google Cloud BigQuery Editions pricing

Google recently introduced significant changes to its existing BigQuery pricing models, affecting both compute and storage. They announced the end of sale for flat-rate and flex slots for all BigQuery customers not currently in a contract. Google announced an increase to the price of on-demand analysis by 25% across all regions, starting on July 5, 2023.

Main Components of BigQuery Pricing

Understanding the pricing structure of BigQuery is crucial to effectively manage expenses. There are two big components to BigQuery pricing:

• Compute (analysis) pricing is the cost to process queries, including SQL queries, user-defined functions, scripts, and certain data manipulation language (DML) and data definition language (DDL) statements
• Storage pricing is the cost to store data that you load into BigQuery. Storage options are logical (the default) or physical. If data storage is converted from logical to physical, customers cannot go back to logical storage.

Selecting the appropriate edition and accurately forecasting data processing needs is essential to cloud data budget planning and maximizing the value derived from Google Cloud BigQuery.

Introducing Unravel 4.8.1 for BigQuery

Unravel 4.8.1 for BigQuery includes AI-driven FinOps and performance optimization features and enhancements, empowering Google Cloud BigQuery customers to see and better manage their cloud data costs. Unravel helps users understand specific cost drivers, allocation insights, and performance and cost optimization of SQL queries. New Unravel features are align with the FinOps phases:

Inform

• Compute and storage costs
• Unit costs and trends for projects, users, and jobs

Optimize

• Reservation insights
• SQL insights
• Data and storage insights
• Scheduling insights

Operate

• OpenSearch-based alerts on job duration and slot-ms
• Alert customization: ability to create custom alerts

Improving visibility, optimizing data performance, and automating spending guardrails can help organizations overcome resource limitations to get more out of their existing data environments.

Visibility into BigQuery compute and storage spend

Getting insights into your cloud data spending starts with understanding your cloud bill. With Unravel, BigQuery users can see their overall spend as well as spending trends for their selected time window, such as the past 30 days.

Unravel provides cost analysis, including the average cost of both compute and storage per project, job, and user over time. Compute spending can be further split between on-demand and reserved capacity pricing.

Armed with this detail, BigQuery customers can better understand both infrastructure and pricing tier usage as well as efficiencies by query, user, department, and project. This granular visibility enables accurate, precise cost allocation, trend visualization, and forecasting.

Unravel’s AI-driven cloud cost optimization for BigQuery delivers insights based on Unravel’s deep observability of the job, user, and code level to supply AI-driven cost optimization recommendations for slots and SQL queries, including slot provisioning, query duration, autoscaling efficiencies, and more.

With Unravel, BigQuery users can speed cloud transformation initiatives by having real-time cost visibility, predictive spend forecasting, and performance insights for their workloads.

AI-driven cloud cost optimization for BigQuery

At the core of Unravel Data’s data observability and FinOps platform is the AI-powered Insights Engine. It is purpose-built for data platforms—including BigQuery—to understand all the unique aspects and capabilities of each modern data stack and the underlying infrastructure to optimize efficiency and performance.

Unravel’s AI-powered Insights Engine continuously ingests and interprets millions of metadata inputs to provide real-time insights into application and system performance, along with recommendations to improve performance and efficiency for faster results and greater positive business impact for your existing cloud data spend.

Using Unravel’s cost and performance optimization intelligence based on its deep observability at the job, user, and code level, users get recommendations such as:

• Reservation sizing that achieves optimal cost efficiency and performance
• SQL insights and anti-patterns to avoid
• Scheduling insights for recurring jobs
• Quota insights with respect to workload patterns
• and more

With Unravel, BigQuery customers can speed cloud transformation initiatives by having predictive cost and performance insights of existing workloads prior to moving them to the cloud.

Visualization dashboards and unit costs

Visualizing unit costs not only simplifies cost management but also enhances decision-making processes within your organization. With clear insights into spending patterns and resource utilization, you can make informed choices regarding optimization strategies or budget allocation.

With Unravel, BigQuery customers can customize dashboards and alerts with easy-to-use widgets to enable at-a-glance and drill down dashboards on:

• Spend
• Performance
• Unit economics

From a unit economics perspective, BigQuery customers can build dashboards to show unit costs in terms of average cost per user, per project, and per job.

Take advantage of visualization dashboards in Unravel for BigQuery to effortlessly gain valuable insights into unit costs.

Additional features included in this release

Unravel 4.8.1 includes additional features, such as showback/chargeback reports, SQL insights and anti-patterns. You can compare two jobs side-by-side, enabling you to point out any metrics that are different between two runs, even if the queries are different.

With this release, you also get:

• Top-K projects, users, and jobs
• Showback by compute and storage types, services, pricing plans, etc.
• Chargeback by projects and users
• Out-of-the-box and custom alerts and dashboards
• Project/Job views of insights and details
• Side-by-side job comparisons
• Data KPIs, metrics, and insights such as size and number of tables and partitions, access by jobs, hot/warm/cold tables

Use case scenarios

Unravel for BigQuery provides a single source of truth to improve collaboration across functional teams and accelerates workflows for common use cases. Below are just a few examples of how Unravel helps BigQuery users for specific situations:

Role	Scenario	Unravel benefits
FinOps Practitioner	Understand what we pay for BigQuery down to the user/app level in real time, accurately forecast future spend with confidence	Granular visibility at the project, job, and user level enables FinOps practitioners to perform cost allocation, estimate annual cloud data application costs, cost drivers, break-even, and ROI analysis
FinOps Practitioner / Engineering / Operations	Identify the most impactful recommendations to optimize overall cost and performance	AI-powered performance and cost optimization recommendations enable FinOps and data teams to rapidly upskill team members, implement cost efficiency SLAs, and optimize BigQuery pricing tier usage to maximize the company’s cloud data ROI
Engineering Lead / Product Owner	Identify the most impactful recommendations to optimize the cost and performance of a project	AI-driven insights and recommendations enable product and data teams to improve slot utilization, boost SQL query performance, leverage table partitioning and column clustering to achieve cost efficiency SLAs and launch more data jobs within the same project budget
Engineering / Operations	Live monitoring with alerts	Live monitoring with alerts speed MTTR and prevent outages before they happen
Data Engineer	Debugging a job and comparing jobs	Automatic troubleshooting guides data teams directly to the pinpoint the source of job failures down to the line of code or SQL query along with AI recommendations to fix it and prevent future issues
Data Engineer	Identify expensive, inefficient, or failed jobs	Proactively improve cost efficiency, performance, and reliability before deploying jobs into production. Compare two jobs side-by-side to find any metrics that are different between the two runs, even if the queries are different.

Get Started with Unravel for BigQuery

Learn more about Unravel for BigQuery by joining our upcoming webinar on August 24, reviewing the docs and creating your own free account.