> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dqlabs.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Azure Data Factory Setup

> Azure permissions, authentication options, and step-by-step instructions for connecting Azure Data Factory to Prizm.

## Prerequisites

Prizm is read-only on Azure Data Factory. It reads metadata (pipelines, activities, datasets, data flows, linked services, and triggers) and run history through the Azure Resource Manager REST API, and never creates, changes, triggers, or deletes anything in ADF. Choose one authentication type below.

Create an identity in Microsoft Entra ID for Prizm to use, then give it read-only access to the data factories you want to observe. Which identity you create depends on the authentication type you'll choose:

* **Service principal** — an app registration with a client secret.
* **Certificate** — an app registration with a certificate uploaded, and the matching PEM-encoded private key.
* **Managed identity** — a system-assigned or user-assigned managed identity that Prizm can use.

### Before You Start

You need to sign in to the Azure portal as a user who can create app registrations in Microsoft Entra ID and assign roles (**Owner** or **User Access Administrator** on the target scope). Collect these values:

| Item | Where to find it |
| :- | :- |
| Tenant ID | Microsoft Entra ID → Overview |
| Subscription ID | Subscriptions → your subscription |
| Resource group | The resource group that contains the factory |
| Data factory name | Data factories → your factory |

Managed identity applies only when the Prizm agent itself runs on Azure (a virtual machine, AKS, App Service, or Container Apps). To keep secrets out of Prizm, store the client secret or the PEM in Azure Key Vault and grant the identity Prizm uses **Key Vault Secrets User** on that secret.

| Authentication type | Best for | Stored secret |
| :- | :- | :- |
| **Service Principal (Client Secret)** | The quickest setup, or evaluation | Client secret |
| **Service Principal (Certificate)** | Production where secrets are discouraged | Private key and certificate (PEM) |
| **Managed Identity** | A Prizm agent that runs on Azure | None |
| **Vault (Azure Key Vault)** | Customers who must not paste secrets into Prizm | Held in Key Vault and referenced by Prizm |

**To create a service principal:** in the Azure portal, go to **Microsoft Entra ID → App registrations → New registration**, name it (for example `prizm-adf-reader`), choose single tenant, and leave the redirect URI empty. Note the **Application (client) ID** and **Directory (tenant) ID**. Under **Certificates & secrets**, create a client secret and copy its value immediately, because it is shown only once. For a certificate, upload the public certificate (`.crt`) instead and keep the private key and certificate together as a PEM file.

**To use a managed identity:** enable a system-assigned identity on the resource that hosts the Prizm agent, or create a user-assigned identity and attach it. Note the identity's object (principal) ID, and for a user-assigned identity its client ID.

### Required Azure Roles

| Purpose | Azure role | Scope |
| :- | :- | :- |
| List the subscriptions and data factories the identity can see, and read pipelines, datasets, data flows, linked services, and triggers | **Reader** (built-in) | The data factory, or its resource group or subscription |
| Query pipeline, trigger, and activity runs | A custom role, for example **Prizm ADF Reader**, with the actions below | The same scope as Reader |

The built-in Reader role can't query run history, because the run-query actions are POST actions. Create a custom role with these read-only actions and assign it alongside Reader:

```text theme={null}
Microsoft.DataFactory/factories/read
Microsoft.DataFactory/factories/pipelines/read
Microsoft.DataFactory/factories/datasets/read
Microsoft.DataFactory/factories/dataflows/read
Microsoft.DataFactory/factories/linkedservices/read
Microsoft.DataFactory/factories/triggers/read
Microsoft.DataFactory/factories/pipelineruns/read
Microsoft.DataFactory/factories/triggerruns/read
Microsoft.DataFactory/factories/querypipelineruns/action
Microsoft.DataFactory/factories/querytriggerruns/action
Microsoft.DataFactory/factories/pipelineruns/queryactivityruns/action
```

If you'd rather not create a custom role, Data Factory Contributor also works, but it grants write access that Prizm doesn't use.

Assign both roles to the identity at the data factory scope:

```bash theme={null}
az role assignment create --assignee <client-id-or-principal-id> --role "Reader" \
  --scope /subscriptions/<sub-id>/resourceGroups/<rg>/providers/Microsoft.DataFactory/factories/<factory>

az role assignment create --assignee <client-id-or-principal-id> --role "Prizm ADF Reader" \
  --scope /subscriptions/<sub-id>/resourceGroups/<rg>/providers/Microsoft.DataFactory/factories/<factory>
```

> [!NOTE]
>
> These roles are read-only. If Prizm gets a `403` only when it queries pipeline runs, the run-query actions are missing from the identity's roles. Prizm doesn't need write access to ADF, access to the underlying data stores, or access to linked-service credentials.

### Information Prizm Needs

| Field | Service Principal (Secret) | Service Principal (Certificate) | Managed Identity |
| :- | :- | :- | :- |
| Tenant ID | Required | Required | Not needed |
| Client ID | Required | Required | User-assigned only |
| Client Secret | Required | Not needed | Not needed |
| Certificate (PEM) and passphrase | Not needed | Required | Not needed |
| Subscription, resource group, and factory | Selected in Prizm | Selected in Prizm | Selected in Prizm |

Prizm lists the subscriptions, resource groups, and factories the identity can see, so the scope of the role assignment controls what appears.

### Optional Access

Assign **Log Analytics Reader** on the workspace that receives your ADF diagnostics (the `ADFPipelineRun` and `ADFActivityRun` tables) if you want richer failure detail. It isn't required.

### Check Access Before You Connect

Test with the same identity Prizm will use. Get a token for `https://management.azure.com/.default`, then call the Azure Resource Manager API:

| Call | Expected result |
| :- | :- |
| List factories: `GET /subscriptions/<sub-id>/providers/Microsoft.DataFactory/factories?api-version=2018-06-01` | `200` |
| List pipelines for your factory | `200` |
| `POST queryPipelineRuns` for your factory | `200`. An empty list is fine |

| Result | Meaning |
| :- | :- |
| `401` | Wrong tenant, client ID, or secret, or the secret has expired |
| `403` on list or pipelines | Reader role missing or not yet propagated at this scope. Role assignments can take a few minutes |
| `403` only on `queryPipelineRuns` | The run-query actions are missing from the custom role |
| Empty list of factories | The role is assigned at the wrong scope or on the wrong subscription |

### Network Access

Prizm must reach `https://login.microsoftonline.com` and `https://management.azure.com` on port 443. No firewall change is needed on the data factory itself.

## Connector Setup

**1. Open the connector wizard**

Navigate to **Source** and click the **+** icon next to the source list. Select **Azure Data Factory** from the connector grid.

**2. Enter source details**

| Field | Description |
| :- | :- |
| Connection Name | A label for this connection (for example, `adf-prod`) |
| Description | Optional notes about what this connection covers |
| Authentication Type | **Service Principal**, **Certificate**, or **Managed Identity** |

![The Azure Data Factory Source Details form with Service Principal selected](https://mintcdn.com/dqlabs/Yuz3_M2E4CwHob88/images/sources/adf/form.png?fit=max\&auto=format\&n=Yuz3_M2E4CwHob88\&q=85\&s=2b0e164a534b179f989b2a925ad46e88)

Azure Data Factory source details. Service Principal is the default authentication type

**3. Provide credentials**

The fields depend on the authentication type. For service principal and certificate connections you can toggle **Use Vault** to retrieve the credential from a vault instead.

> [tab] Service Principal
>
> | Field | Description |
> | :- | :- |
> | Tenant ID | Azure AD tenant (directory) ID |
> | Client ID | Application (client) ID of the app registration |
> | Client Secret | The app registration's client secret |

> [tab] Certificate
>
> | Field | Description |
> | :- | :- |
> | Tenant ID | Azure AD tenant (directory) ID |
> | Client ID | Application (client) ID of the app registration |
> | Certificate (PEM) | Upload the PEM file containing the certificate and private key |
> | Certificate Password | Password of the private key. Leave blank if there is none |
> | Send Certificate Chain | Select to send the full certificate chain when authenticating |

> [tab] Managed Identity
>
> | Field | Description |
> | :- | :- |
> | Managed Identity Client ID | Client ID of a user-assigned managed identity. Leave blank to use the system-assigned identity |

> [tab] Use Vault
>
> | Field | Description |
> | :- | :- |
> | Select Vault | Choose from the vault instances already configured for your organization |
> | Key | The key in that vault holding the secret. For a service principal it holds the tenant ID, client ID, and client secret. For a certificate it holds the tenant ID, client ID, and PEM private key, plus the certificate password if there is one |

**4. Continue to scope**

Click **Next** to continue to the source configuration step, where you choose the data factories and pipelines Prizm should observe.

**5. Save**

Complete the configuration. The source appears in the **Source** list, and pipelines appear in the catalog once the first scan completes.

## Default Behavior

| Setting | Default |
| :- | :- |
| Initial run history load | Previous 30 days |
| Later loads | Only runs updated since the last successful sync |
| Run history retention | 30 days |

The metadata and run-collection schedules are set on the source's job schedule, so they follow whatever your Prizm administrator configured.

## Next Steps

> [card] [What We Collect](https://docs.dqlabs.ai/sources/adf/what-we-collect.md)
>
> See the pipeline metadata, run history, and metrics Prizm extracts.

> [card] [FAQ](https://docs.dqlabs.ai/sources/adf/faq.md)
>
> Common questions about credentials, scoping, and permissions.

> [card] [Troubleshooting](https://docs.dqlabs.ai/help/troubleshooting.md)
>
> Diagnose connection failures, permission errors, and timeouts.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.