> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dqlabs.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Setup

> Create an Airflow source in Prizm and configure API Pull or CLI mode. For Plugin mode, see the Plugin Setup guide.

<script type="application/ld+json">
  {`{
            "@context": "https://schema.org",
            "@type": "TechArticle",
            "headline": "Airflow Setup",
            "description": "Create an Airflow source in Prizm and configure API Pull or CLI mode. For Plugin mode, see the Plugin Setup guide.",
            "url": "https://docs.dqlabs.ai/sources/airflow/setup",
            "publisher": {
              "@type": "Organization",
              "name": "DQLabs Inc",
              "logo": "https://media.brand.dev/332adc35-5bc4-4d2b-bf78-256aa4a5e414.svg"
            }
            }`}
</script>

## Step 1 — Create an Airflow source

In Prizm, go to **Sources → Add Source → Airflow**.

### Identity fields

| Field                | Description                                                                                                                                                                                                                                                         |
| :------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Name**             | The source's identifier inside Prizm. For CLI and Plugin modes, the `connection_name` in every push must match this value exactly — character for character, including case. Choose a stable name; renaming requires updating your CLI command or Airflow Variable. |
| **Description**      | Optional free text describing this connection.                                                                                                                                                                                                                      |
| **Integration Type** | `API Pull`, `CLI`, or `Plugin`. **Locked once the source is saved** — switching requires creating a new source.                                                                                                                                                     |

### Lineage & profiling fields

| Field                         | Description                                                                                                                                          |
| :---------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Lineage Method**            | `Query Tag` (default) or `Open Lineage`. Fixed to Query Tag for CLI sources. See the [Lineage](/connectors/airflow/lineage) page for details.        |
| **Open Lineage Endpoint URL** | Appears when Open Lineage is selected. Read-only, computed by Prizm per source — copy this URL into your Airflow OpenLineage provider configuration. |
| **Event Automatic Profiling** | Whether ingest events automatically trigger profiling of newly discovered assets.                                                                    |

### Alert fields

| Field                  | Description                                                    |
| :--------------------- | :------------------------------------------------------------- |
| **DAG Alert Failure**  | Raise a Prizm alert when any DAG run ends in a failure state.  |
| **Task Alert Failure** | Raise a Prizm alert when any task instance fails within a run. |

These alert on run state directly — they are independent of the metric thresholds in [What We Collect](/connectors/airflow/what-we-collect).

***

## Step 2 — Configure your connection mode

<Tabs>
  <Tab title="API Pull">
    Prizm authenticates against your Airflow REST API and polls on a schedule. This requires a network path from Prizm to your Airflow webserver.

    Prizm auto-detects whether your Airflow is running REST API v1 (`/api/v1`) or v2 (`/api/v2`) based on the reported version, so upgrading Airflow between major API versions does not require changing the source config.

    ### Additional fields (API Pull only)

    | Field                              | Description                                                                                                                                                     |
    | :--------------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------- |
    | **Server**                         | Your Airflow webserver's base URL (e.g., `https://airflow.internal`)                                                                                            |
    | **Authentication Type**            | `Username/Password` or `Base Auth Key` (pre-built Basic auth header value)                                                                                      |
    | **Username / Password**            | Airflow account credentials — encrypted at rest in Prizm                                                                                                        |
    | **Base Auth Key**                  | Alternative to username/password — encrypted at rest                                                                                                            |
    | **Use Vault**                      | Source credentials from a configured vault at connection time instead of storing them in Prizm                                                                  |
    | **No. of Runs / Run History Unit** | How much run history each sync window pulls (e.g., last 30 runs, or last 7 days). Limits backfill so Prizm doesn't pull Airflow's entire history on every sync. |

    ### Schedule

    Prizm polls **daily at 02:30 UTC by default**. On transient failures, Prizm retries automatically up to 3 times before skipping that sync window. You can adjust the schedule from the connector detail page after saving.

    ### Required Airflow permissions

    The account used for API Pull needs read access to DAG and Task resources in Airflow. In Airflow's built-in RBAC, the **Viewer** role covers this.

    <Tip>
      After saving, trigger a manual sync immediately from the source page to confirm the connection before waiting for the scheduled window.
    </Tip>
  </Tab>

  <Tab title="CLI">
    The CLI runs on your machine or in CI — it reads from Airflow and uploads to Prizm in a single command. Prizm never needs inbound network access to your Airflow, and never stores an Airflow credential.

    ### Install

    ```bash theme={null}
    pip install prizm-airflow-cli
    ```

    ### Step 1 — Dry run (recommended first)

    A dry run confirms the CLI can reach and authenticate against Airflow and lets you inspect the extracted bundle — without uploading anything to Prizm or needing your Prizm API token. It is the fastest way to isolate an Airflow-side problem from a Prizm-side one.

    ```bash theme={null}
    export AIRFLOW_PASSWORD='your-airflow-password'

    prizm-airflow extract \
      --prizm-url "https://your-workspace.prizmdata.ai" \
      --connection-name "<source name from Step 1>" \
      --hostname "https://your-airflow-webserver" \
      --username "<airflow-username>" \
      --dry-run
    ```

    Inspect `airflow_bundle.json` — you should see your DAGs, their tasks, and recent runs.

    ### Step 2 — Upload

    Drop `--dry-run` and supply the Prizm API token via environment variable:

    ```bash theme={null}
    export AIRFLOW_PASSWORD='your-airflow-password'
    export PRIZM_API_TOKEN='your-prizm-token'

    prizm-airflow extract \
      --prizm-url "https://your-workspace.prizmdata.ai" \
      --connection-name "<source name from Step 1>" \
      --hostname "https://your-airflow-webserver" \
      --username "<airflow-username>" \
      --runs 30
    ```

    <Warning>
      Do not pass credentials as command-line flags (`--password`, `--api-token`). Anything on the command line lands in shell history and is visible in CI build logs. Always use environment variables for secrets.
    </Warning>

    <Note>
      **Known issue — upload endpoint:** In the current release, the CLI's derived upload path may not match Prizm's ingest route, causing a 404. If you see a 404 on upload, add the endpoint explicitly:

      ```bash theme={null}
      --endpoint-url "https://your-workspace.prizmdata.ai/api/v1/schedule/airflow/ingest"
      ```
    </Note>

    ### CLI flag reference

    | Flag                    | Required       | Default                    | Notes                                                                                    |
    | :---------------------- | :------------- | :------------------------- | :--------------------------------------------------------------------------------------- |
    | `--prizm-url`           | Yes            | —                          | Your Prizm workspace base URL                                                            |
    | `--endpoint-url`        | See note above | Derived from `--prizm-url` | Full ingest URL; overrides the derived default                                           |
    | `--api-token`           | Yes (upload)   | —                          | Prizm API access token. Prefer `PRIZM_API_TOKEN` env var.                                |
    | `--client-secret`       | Yes (upload)   | —                          | Alias for `--api-token`                                                                  |
    | `--connection-name`     | Yes            | —                          | Must match the source Name in Prizm exactly                                              |
    | `--hostname`            | No             | `http://localhost:8080`    | Airflow webserver URL                                                                    |
    | `--username`            | Yes            | —                          | Airflow account username                                                                 |
    | `--password`            | Yes            | —                          | Airflow password. Prefer `AIRFLOW_PASSWORD` env var.                                     |
    | `--runs`                | No             | `30`                       | Number of recent DAG runs to extract                                                     |
    | `--tasks`               | No             | `true`                     | Include task-level metadata                                                              |
    | `--extract-source-code` | No             | `true`                     | Include each DAG's Python source. Required for cross-DAG lineage and PR impact analysis. |
    | `--dry-run`             | No             | off                        | Write the bundle to `airflow_bundle.json` instead of uploading                           |
    | `--out`                 | No             | `airflow_bundle.json`      | Output path when using `--dry-run`                                                       |

    ### Keeping it fresh

    Nothing runs on a schedule for CLI mode — Prizm receives data only when you run the command. Wire it into a cron job or CI pipeline step to automate it, and monitor the exit code. A silently failing cron job looks identical to an Airflow environment that stopped changing.
  </Tab>
</Tabs>

***

## Step 3 — Verify your connection

After setup, confirm the following. If any step fails, see the [troubleshooting](#troubleshooting) section below.

| Check               | Where to look                  | Expected                                                                                |
| :------------------ | :----------------------------- | :-------------------------------------------------------------------------------------- |
| Source is connected | Sources → your Airflow source  | No error state on the source record                                                     |
| DAGs cataloged      | Assets, filtered to the source | One asset per active DAG                                                                |
| Tasks cataloged     | Any DAG asset → children       | One asset per task, parented to its DAG                                                 |
| Run history present | DAG asset → Metric tab         | Recent runs with states and durations                                                   |
| Metrics evaluating  | DAG asset → Metric tab         | Freshness (DAG) and Execution Time (task) populated                                     |
| Lineage edges       | Lineage graph                  | Edges to warehouse tables (Snowflake only — see [Lineage](/connectors/airflow/lineage)) |

***

## Troubleshooting

| Symptom                             | Likely cause                                  | Fix                                                                                     |
| :---------------------------------- | :-------------------------------------------- | :-------------------------------------------------------------------------------------- |
| CLI upload fails with 404           | Ingest endpoint mismatch                      | Pass `--endpoint-url` explicitly — see the note in Step 2.                              |
| CLI or Plugin push lands nowhere    | `connection_name` doesn't match source Name   | Compare character for character, including case.                                        |
| 401 / 403 from Airflow              | Insufficient credentials or role              | Verify the account has read access to DAG and Task resources.                           |
| Source connects but shows zero DAGs | Sync hasn't run yet                           | Trigger a manual sync, or run the CLI with `--dry-run` and inspect the bundle.          |
| No lineage edges appear             | Warehouse not supported, or query tags absent | Check the [warehouse support matrix](/connectors/airflow/lineage#cross-source-lineage). |

***

## Next Steps

<CardGroup cols={2}>
  <Card title="Plugin Setup" icon="puzzle-piece" href="/sources/airflow/plugin-setup">
    Set up real-time push-based metadata with the Prizm Airflow Plugin.
  </Card>

  <Card title="What We Collect" icon="database" href="/sources/airflow/what-we-collect">
    Full field-level breakdown of every metadata object Prizm extracts from Airflow.
  </Card>
</CardGroup>