Plugin Setup

The Prizm Airflow Plugin hooks into Airflow’s run lifecycle. When a DAG run completes — success or failure — the plugin sends that run’s metadata to Prizm immediately, with no polling interval and no Airflow credentials stored in Prizm.

The plugin never reads or transmits Airflow connection secrets, variable values outside the ones listed in this guide, or data from your warehouses. It reports only DAG, task, and run metadata — plus, optionally, source code and failed-task logs, both of which you can disable.

Before you start

Admin access to your Airflow deployment — you need to add a package to your Airflow requirements and restart the scheduler
Admin or Editor access to your Prizm workspace
Airflow 2.5 or later, with the requests package available in the scheduler’s Python environment (present in most Airflow installs by default)
Network access from the Airflow scheduler host to your Prizm workspace URL

Installation

Create the Airflow source in Prizm

In Prizm, go to Sources → Add Source → Airflow and set the Integration Type to Plugin.Give the source a Name and write it down exactly as typed — you will reuse it verbatim as prizm_connection_name in Step 4. Prizm uses this name to match incoming plugin data to the right source.

The name must match exactly, including case and spacing. The plugin sends its configured connection name with every request. If it doesn’t match the source name character for character, Prizm rejects the data.

Generate a Prizm API access token

Go to Settings → Organization → Access Tokens and generate a new token.

Copy the token immediately — it is shown only once. Store it as a secret: an Airflow Variable marked secret, or a secrets backend. Never commit it to source control.

Add the plugin package to your Airflow requirements

Add the prizm-airflow-plugin package to the requirements your Airflow environment installs from:

requirements.txt

prizm-airflow-plugin

Installing the package automatically registers the plugin in your Airflow plugins — there is no file to download and nothing to copy into a plugins/ folder. If you manage packages directly instead of through a requirements file, install it into the scheduler’s Python environment:

pip install prizm-airflow-plugin

Docker, Astronomer, Composer, or MWAA? Add the package wherever your platform manages Python dependencies — for example, requirements.txt for Docker, Astronomer, and MWAA, or the PyPI packages setting for Cloud Composer — then rebuild or update the environment so the scheduler and workers pick it up.

Upgrading from the manual file install? If you previously copied prizm_airflow_plugin.py into your Airflow plugins/ folder, remove that file when you switch to the package so the plugin is not loaded twice. Your existing Airflow Variables and Prizm source stay as they are.

Configure the plugin

Set the following as Airflow Variables via Admin → Variables in the Airflow UI, or via the CLI:

airflow variables set prizm_url "https://your-workspace.prizmdata.ai"
airflow variables set prizm_api_token "<token from Step 2>"
airflow variables set prizm_connection_name "<source name from Step 1>"

The plugin also reads PRIZM_URL, PRIZM_API_TOKEN, and PRIZM_CONNECTION_NAME from the process environment first, falling back to Airflow Variables. Use environment injection (Docker, Helm) if your scheduler is already configured that way.You can use an existing Airflow secrets backend—such as AWS Secrets Manager, Google Cloud Secret Manager, Azure Key Vault, or HashiCorp Vault—instead of storing values directly in Airflow Variables. Store prizm_url, prizm_api_token, and prizm_connection_nameusing the backend’s Airflow Variable naming convention; the plugin resolves them automatically. Alternatively, inject prizm_url, prizm_api_token, and prizm_connection_name as environment variables.Optional variables for filtering and log capture are listed in the Variable reference section below.

Restart the scheduler

The plugin loads only on scheduler startup. Restart the scheduler — and any workers running the same image — after installing the package and setting the variables.systemd:

sudo systemctl restart airflow-scheduler

Docker Compose:

docker compose restart airflow-scheduler

The exact command depends on your deployment. The requirement is that the scheduler process restarts in the same environment where you installed the package. To confirm the package is present, run pip show prizm-airflow-plugin in that environment.

Verify

Trigger or wait for any DAG run to complete. In Prizm, open Sources and select your source — the run appears within a few minutes, along with the DAG’s tasks and lineage.If nothing shows up, see Troubleshooting below.

How the plugin works

The plugin registers two listeners on scheduler startup:

on_dag_run_success — fires when a DAG run completes successfully
on_dag_run_failed — fires when a DAG run ends in a failure state

On each trigger, the plugin builds a payload for that run and POSTs it to {prizm_url}/api/v1/schedule/airflow/plugin using the configured Bearer token. Limitations to be aware of:

Failed pushes are dropped. If Prizm is briefly unreachable when a run completes, that run’s data is lost. Check the scheduler logs for entries from prizm_airflow_plugin to detect dropped pushes.
Only success and failure fire. Runs that are killed, marked as zombie, or otherwise end without triggering either listener produce no push — leaving a gap in the catalog.
Plugin pushes never prune. Each push contains data for the run that just finished, not a complete DAG inventory. Deactivating deleted or renamed DAGs requires a full sync from an API Pull or CLI source.

Variable reference

Variable	Required	Default	Description
`prizm_url`	Required	—	Your Prizm workspace base URL. The plugin appends the ingest path itself; a trailing slash is fine.
`prizm_api_token`	Required	—	Access token from Step 2, sent as a Bearer token with every push.
`prizm_connection_name`	Required	—	Must exactly match the Airflow source’s Name in Prizm.
`prizm_allowed_dags`	Optional	All DAGs	Comma-separated glob patterns (e.g., `prod_, etl_`) — only matching DAGs are reported.
`prizm_excluded_dags`	Optional	None	Comma-separated glob patterns to exclude. Takes priority over `prizm_allowed_dags`.
`prizm_extract_source_code`	Optional	`true`	Sends each DAG’s Python source so Prizm can display it and derive cross-DAG lineage. Set to `false` to skip.
`prizm_include_full_log`	Optional	`false`	Sends the full task log for failed tasks only. Successful tasks never send log content. Useful for troubleshooting failures directly in Prizm.

Troubleshooting

Message or symptom	Cause and fix
`Source not found, inactive, or not Airflow type`	No active Airflow source in Prizm matches `prizm_connection_name`. Re-check the name against Step 1 — it is case-sensitive — and confirm the source Status is active.
`Source is not configured for Airflow plugin integration`	The source exists but its Integration Type isn’t Plugin. Create a new source with Plugin selected.
`connection_name does not match Prizm source name`	`prizm_connection_name` resolved to a source, but the value doesn’t exactly match that source’s name string. Fix the Airflow Variable and restart the scheduler.
`connection_name is required`	`prizm_connection_name` is unset or empty. Confirm it was set in Step 4 and that the scheduler was restarted afterward.
No data arrives, no error visible	Check scheduler logs for entries from `prizm_airflow_plugin`. Confirm `prizm_url` is reachable from the scheduler’s network, and that `prizm_api_token` is set and not revoked.
Plugin installed but never fires	Scheduler was not restarted after installing the package, or the package was installed in a different Python environment than the one the scheduler runs in. Confirm with `pip show prizm-airflow-plugin` in the scheduler’s environment, then restart the scheduler process.
Plugin fires for some runs but not others	Failed pushes are dropped on transient network issues, or some runs ended without triggering a listener (killed/zombie). Check scheduler logs.

Get Started

Core Concepts

Platform

Architecture

Data Sources

Integrations

Security & Access Control

Deployment

Help

Settings

MCP

Release Notes

Before you start

Installation

How the plugin works

Variable reference

Troubleshooting

Next Steps

Setup

What We Collect

​Before you start

​Installation

​How the plugin works

​Variable reference

​Troubleshooting

​Next Steps

Setup

What We Collect

Before you start

Installation

How the plugin works

Variable reference

Troubleshooting

Next Steps