> ## Documentation Index > Fetch the complete documentation index at: https://docs.dqlabs.ai/llms.txt > Use this file to discover all available pages before exploring further. # Airflow Overview > How Prizm connects to Apache Airflow to catalog DAGs and tasks, monitor pipeline health, and stitch lineage across Airflow and the warehouses it orchestrates. Prizm connects to Apache Airflow to catalog DAGs and tasks, monitor pipeline health through operational metrics, and stitch lineage across Airflow and the data warehouses it orchestrates. Once connected, every DAG run is tracked — duration, success rate, freshness — and failures surface as Prizm alerts alongside your data quality alerts. ## Connection Modes Prizm supports three ways to connect to Airflow. The mode determines who initiates the connection, where credentials live, and how quickly metadata arrives. **The mode is set at source creation and cannot be changed afterward** — switching requires creating a new source. | | **API Pull** | **CLI** | **Plugin** | | :----------------------- | :----------------------------------------------- | :--------------------------------------------- | :--------------------------------------------------------- | | **Who initiates** | Prizm polls your Airflow REST API | You (or CI) run a CLI command | Airflow itself pushes on every DAG run | | **Network direction** | Prizm → your Airflow | Your machine → Airflow, then → Prizm | Airflow scheduler → Prizm only | | **Credentials in Prizm** | Yes — Airflow username/password, encrypted | No | No — only a Prizm Bearer token held in an Airflow Variable | | **Freshness** | Scheduled (daily by default) | Whenever you run it | Real-time, immediately after each run completes | | **Retry on failure** | Automatic — up to 3 retries with backoff | None — re-run manually | None — a failed push is logged and dropped | | **Airflow version** | 2.x or 3.x | 2.x or 3.x | 2.5 or later | | **Best for** | Simplest setup when Prizm can reach your Airflow | Airflow environments Prizm can't reach inbound | Lowest-latency metadata, no stored Airflow credentials | API Pull is the only mode with automatic retries. CLI and Plugin both fail silently on a transient network issue. If you choose either and freshness matters, monitor for gaps via scheduler logs. ## Metrics Prizm evaluates four pipeline health metrics for each connected Airflow source: | Metric | Level | Default | How it works | | :------------------- | :---- | :------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Freshness** | DAG | On | Time elapsed between consecutive DAG run watermarks. Auto-thresholded — the normal interval is learned from run history, so a daily DAG and an hourly DAG have appropriately different expectations. | | **Execution Status** | DAG | Off | `1.0` if the run succeeded; `0.0` on failure. Fixed threshold — any value below `1.0` is a breach. | | **Job Duration** | DAG | Off | Total DAG run duration in milliseconds. Auto-thresholded against historical run durations for this specific DAG. | | **Execution Time** | Task | On | Per-task instance duration. Auto-thresholded — isolates which task within a DAG is running slower than normal. | Auto-thresholded metrics derive a normal band from few runs up to 100 prior historical values based on settings. They need sufficient run history before thresholds activate; new DAGs with few runs will show the metric value without an alert threshold until enough history accumulates. Separately, two alert options fire directly on run state regardless of which metrics are enabled: | Alert | What triggers it | | :--------------------- | :----------------------------------------------- | | **DAG Alert Failure** | A DAG run ending in a failure state | | **Task Alert Failure** | An individual task instance failure within a run | ## Lineage Support Prizm supports three distinct types of lineage: 1. **Task Lineage (no additional connector needed):** Shows task-to-task dependencies within a single DAG — this comes straight from Airflow's own DAG structure, so it works out of the box no additional connection or connector required. 2. **Cross-DAG Lineage:** Shows dependencies between DAGs based on control flow, not data — detected by reading DAG source code for TriggerDagRunOperator (one DAG kicks off another) or ExternalTaskSensor/ExternalTaskMarker (one DAG waits on a task in another). 3. **Table/Data Lineage (needs a warehouse connector):** Shows which DAG feeds which table — e.g., "Task X wrote to table B." There are two methods: 1. Query Tag: Prizm reads the query tag attached to the warehouse query and uses it to identify and build the lineage between the DAG/task and the table. 2. OpenLineage: Prizm captures OpenLineage job/dataset events directly and builds the lineage from them, ## What Gets Cataloged | Object | Prizm asset | Notes | | :---------------- | :-------------------------------------- | :--------------------------------------------------------------------------- | | **DAG** | One asset per DAG | Includes schedule, tags, owners, paused/active state, source code (optional) | | **Task** | One asset per task, parented to its DAG | Includes operator type, retry config, upstream/downstream task IDs | | **DAG Run** | Run history record | State, type, execution timestamps, duration — feeds metrics | | **Task Instance** | Per-task run record | State, try number, error message, log reference | ## Next Steps Create an Airflow source and configure API Pull or CLI mode. Install and configure the Prizm Airflow Plugin for real-time push-based metadata. Full breakdown of every field Prizm extracts from DAGs, tasks, and run history.