ADLS Overview

Prizm connects to Azure Data Lake Storage (ADLS) Gen2 to catalog, monitor, and run quality checks against structured, semi-structured, and raw files — without duplicating the underlying data. A lightweight Prizm agent runs inside your environment and connects outbound only; there are no inbound connections, no VPN, and no firewall changes beyond outbound port 443.

​

Two ways to process your files

If you’re happy using Prizm Spark and don’t need Unity Catalog Iceberg materialization, no additional setup is required beyond configuring the source itself.

​

Unity Catalog Iceberg materialization (optional)

When you connect your own Databricks cluster, Prizm can also materialize selected ADLS assets as Unity Catalog–managed Iceberg-compatible tables (via Delta UniForm), so they become persistent, governed, queryable objects — with a catalog.schema.table identity and Unity Catalog–enforced permissions — rather than existing only transiently inside a job. Because materialized tables expose Delta’s UniForm Iceberg metadata, engines like Trino can query them directly through the metastore’s Iceberg REST endpoint — no export step, no second copy of the data.

​

Architecture

When using your own Databricks cluster, the Prizm Agent’s Core, Connector, and Trino components run inside your Azure environment (not in the DQLabs-managed platform):

• Two independent Azure connections. The Agent authenticates to Azure resources via Microsoft Entra, separate from its outbound-only channel to Prizm.
• Connector drives the Databricks job — it invokes the notebook running on your Databricks cluster.
• Databricks reads and writes ADLS — the cluster reads source data from your ADLS source storage account and writes materialized Iceberg files to a separate ADLS storage account/container you designate for Prizm.
• Trino surfaces metrics and metadata back into Prizm for quality jobs and materialization tracking.

​

Supported Environments

​

Next Steps