A data engineer notices that queries against Unity Catalog tables fail with an authorization error on a cluster running Databricks Runtime 10.4 LTS. The same queries succeed on a SQL warehouse and on a cluster running Databricks Runtime 13.3 LTS. What is the most likely root cause?

Question

Accepted Answer

B. Unity Catalog is supported on clusters running Databricks Runtime 11.3 LTS or above; Runtime 10.4 LTS does not support all Unity Catalog GA features. Databricks documentation explicitly states that Unity Catalog is supported on clusters running Databricks Runtime 11.3 LTS or above, and that clusters running on earlier versions do not provide support for all Unity Catalog GA features and functionality. Runtime 10.4 LTS predates this minimum version, explaining why the same queries that succeed on a SQL warehouse and a DBR 13.3 cluster fail on this older cluster. Instance profile issues and metastore differences would affect all runtimes equally.

A data engineer notices that queries against Unity Catalog tables fail with an authorization error on a cluster running Databricks Runtime 10.4 LTS. The same queries succeed on a SQL warehouse and on a cluster running Databricks Runtime 13.3 LTS. What is the most likely root cause?

Related Questions

Discussion