Monday, 9 a.m.: the overnight job finished at 7 instead of 3, and finance wants to know why the Databricks bill is climbing. If you have ever hunted for answers to both questions inside the workspace UI, you know how slow that is. Wiring up Databricks Dynatrace solves much of it, provided you pick the right extension, because there are two and they answer different questions.
This piece starts from that operations scenario: decide which extension to deploy, collect what you need from the right teams, then know what to expect from the data. The facts come from the official Dynatrace and Databricks pages, read on 8 November 2026. We have not deployed the extension ourselves, so treat this as a critical reading of the documentation, not a production report.
Dynatrace also published an overview post, turn Databricks telemetry into actionable decisions. Here there is no showcase, just a practitioner’s view of what to prepare and where it bites.
Databricks Dynatrace: which extension fits the question you are asked
Rather than compare feature lists, start from the question on your desk.
- « Is the node saturated? What is Spark doing? » You need measurements taken on the machines. That is the OneAgent-based extension, installed on each cluster through a global init script, with the Extension Execution Controller enabled on the host. It reads the Spark and Databricks APIs plus Ganglia, which is gone from Databricks Runtime 13 onwards.
- « Which job failed, what did it cost, is that serving endpoint coping? » You need a per-resource view. That is the Databricks Workspace extension, which queries the workspace from outside through its APIs and system tables.
- « We run serverless. » Nothing agent-like can go in there, and the Workspace extension page positions itself exactly as the complement for places where OneAgent cannot be installed.
You can run both: the documentation calls them complementary. The rest of this article follows the Workspace route, which is lighter to install and closer to what finance asks.
SaaS or Managed? The two pages we read belong to the SaaS documentation. They are silent about Dynatrace Managed, and we have not verified these extensions there: assume nothing and read the documentation for your version. For the ActiveGate’s role, see Environment ActiveGate versus Cluster ActiveGate.
What to ask of each team
Rollouts seldom stall on Dynatrace itself; they stall on getting agreements. Here is the shopping list, split by who holds the key.
From the network team: an Environment ActiveGate that reaches the workspace URL. A detail that catches people out: it also fetches SQL query results from pre-signed cloud storage URLs, so outbound access to those domains must be open. The page gives *.s3.*amazonaws.com and *.blob.core.windows.net as examples. Without it, the workspace connection succeeds but the system tables stay silent. No route without an ActiveGate is described.
From the Databricks admin:
- Unity Catalog enabled, otherwise no system tables.
- A running SQL warehouse for billing and model serving; the ID of an existing one will do.
- A dedicated identity, ideally a service principal rather than a personal token. The page requires it to be a workspace admin, or to hold Can view on the jobs and endpoints to follow; for Spark metrics, add Can attach to on each cluster.
- Rights on the system tables.
On that last point, Databricks’ own documentation says these tables are read-only and free to browse: only the query compute is billed. Access goes through USE CATALOG, then USE SCHEMA and SELECT. The Dynatrace page supplies its own GRANT statements, schema by schema. Our example below, with a made-up principal, covers only billing and audit: copy the complete list from the source.
From you, on the Dynatrace side: an access token with the single scope openTelemetryTrace.ingest, kept in a vault and never in a repository.
To see exactly which APIs and tables the service principal must be able to read, the next screenshot reproduces the API scopes (jobs, clusters, model-serving, sql, instance-pools, pipelines) and the nine system tables the documentation lists.
Switching it on, in this order
With the agreements in hand, configuration takes few steps. In Dynatrace you create a monitoring configuration holding the workspace URL, the Dynatrace token, then either the Databricks token or the service principal’s client ID and secret. We name no fields, as the page lists none, the warehouse ID included.
That leaves the feature sets: Job Metrics, Resource Utilization, Model Serving Endpoint, Job Cost, Spark Executor and others. My practitioner’s advice: do not tick everything on day one. Begin with Job Metrics, which answers Monday’s 9 a.m. question, then add costs once the first data looks trustworthy. Each feature set raises the ingested volume and the load on the warehouse. At least one metric has to be collected after activation, otherwise the configuration tells you nothing useful.
Checking that data arrives
The documentation offers no DQL query. The two lines below are therefore our own, built from the timeseries syntax and the metric keys on the page. We have not run them, and we use no dimension such as the job name, since none is documented.
timeseries avg(databricks.job.duration.run), from:-24h
timeseries avg(databricks.job.success_rate), from:-24h
The first charts average run duration, the second the success rate. The page does not state the unit, so confirm it before deriving a threshold. To go beyond copy and paste, the DQL commands filter, summarize and timeseries show how to slice these series.
In the UI the documentation points to three landmarks: the Job Runs dashboard (with a link to traces), the Databricks Cost Management dashboard, and services named Databricks - <workspace name> in which each job is an endpoint and each run a distributed trace.
Delays and costs: why your charts have gaps
Before calling it an outage, reread these behaviours.
- A job only sends its metrics and traces once it has finished: a run in progress is invisible.
- Billing data trails by roughly 8 hours; model serving usage, cluster utilisation and audit logs by roughly 1 hour. Widen the window before concluding data was lost.
-
databricks.job.costworks on Azure Databricks only. - Every system table read burns warehouse compute, billed by Databricks, not by Dynatrace.
- On the Dynatrace side the page gives volume formulas, for instance
8 * number of jobsdatapoints for job metrics andnumber of jobs * (1 + tasks per job)spans for traces. Run the numbers for your estate before enabling widely.
Best practices
- A dedicated service principal with strictly listed rights.
- A warehouse sized for regular, modest queries, not heavy analytics.
- One feature set first, then add them one at a time while watching the bill on both sides.
- A job success rate tied to an objective, as in DQL metrics and alerts.
Frequently asked questions
Do you need OneAgent to monitor Databricks with Dynatrace?
Not for the Workspace extension, which installs nothing on the clusters. The cluster extension, by contrast, rolls out OneAgent through a global init script.
Does the Workspace extension work without an ActiveGate?
The documentation requires an Environment ActiveGate and describes no other path.
What data reaches Dynatrace?
Job metrics and traces, costs, model serving, billing, Spark metrics, cluster rightsizing recommendations, plus audit and cluster activity logs.
Conclusion
To answer Monday’s 9 a.m. questions without installing an agent, the Workspace extension is enough: jobs, costs and endpoints, at the price of an ActiveGate, a warehouse and tightly scoped rights. Keep the cluster extension for node-level questions. Start with one feature set, watch the delays, then widen.
span.observer is an independent blog, not affiliated with Dynatrace LLC (nor Databricks). Brands mentioned belong to their owners. Tools change fast: check options in the official documentation, notably the Databricks Workspace extension page.
Originally published on span.observer. French version: https://span.observer/fr/databricks-dynatrace-extension-workspace/.
Top comments (0)