> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dqlabs.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Lineage

> Comprehensive data lineage tracking — from source to consumption — with automated impact analysis and root cause identification.

<script type="application/ld+json">
  {`{
            "@context": "https://schema.org",
            "@type": "TechArticle",
            "headline": "Lineage",
            "description": "Comprehensive data lineage tracking - from source to consumption - with automated impact analysis and root cause identification.",
            "url": "https://docs.dqlabs.ai/architecture/lineage",
            "publisher": {
              "@type": "Organization",
              "name": "DQLabs Inc",
              "logo": "https://media.brand.dev/332adc35-5bc4-4d2b-bf78-256aa4a5e414.svg"
            }
            }`}
</script>

## Overview

Lineage in Prizm provides automated, end-to-end visibility into how data moves through your entire data ecosystem — from raw source tables through transformations and models to dashboards and reports. Every relationship is captured and made queryable, so you can trace where data comes from, understand what it feeds, and assess the blast radius of any change before making it.

<CardGroup cols={3}>
  <Card title="Upstream analysis" icon="arrow-up-to-line">
    Trace any asset or column back to its origin across multiple hops, source systems, and transformation layers.
  </Card>

  <Card title="Downstream impact" icon="arrow-down-from-line">
    Map every consumer and dependent asset affected by a change to a dataset or column — before you make it.
  </Card>

  <Card title="Column-level lineage" icon="columns-3">
    See exactly which source column feeds which target column, including the transformation logic applied at each hop.
  </Card>
</CardGroup>

***

## Features

<CardGroup cols={2}>
  <Card title="Deep Lineage" icon="snowflake" href="/sources/snowflake/overview">
    Deep, native lineage integration for data sources, queries and pipelines.
  </Card>

  <Card title="Impact Analysis" icon="explosion" href="/lineage/impact-analysis">
    Compute blast radius before making changes to any upstream asset.
  </Card>

  <Card title="Criticality Scoring" icon="star" href="/architecture/criticality">
    Understand which assets are most critical to downstream business processes.
  </Card>
</CardGroup>

***

## Lineage View

The Lineage tab on any asset page renders a **force-directed graph** of all upstream and downstream relationships. The screenshot above shows `SILVER_DIM_DATE` — a dimension table in the PRIZM\_RETAIL source — with its full dependency graph expanded to depth 4.

### Canvas controls

| Control                      | What it does                                                                |
| ---------------------------- | --------------------------------------------------------------------------- |
| **+  / −**                   | Zoom in and out on the graph                                                |
| **Focus** (crosshair)        | Re-centre the graph on the currently selected asset                         |
| **Rotate**                   | Rotate the layout orientation                                               |
| **Lock**                     | Pin the layout and prevent nodes from being repositioned                    |
| **VIEW toggle**              | Switch between upstream view, downstream view, and end-to-end view          |
| **Expand / Collapse arrows** | Expand all nodes or collapse to the focused asset                           |
| **DEPTH selector**           | Control how many hops the graph renders — 1 to n levels deep. Default is 4. |
| **ALL toggle**               | Show or hide all asset types in the graph                                   |
| **ATTR toggle**              | Show or hide attribute (column) level lineage edges                         |
| **LINK toggle**              | Show or hide link-type relationships (e.g. dashboard → dataset)             |

### Reading the graph

Each **node** in the graph represents an asset — a table, view, model, pipeline, or dashboard. Each **edge** (line between nodes) represents a lineage relationship between two assets.

* Assets to the **left** of the focal asset are upstream — they feed data into it.
* Assets to the **right** are downstream — they consume data from it.
* The **depth** setting controls how many hops in each direction are rendered. At depth 4, Prizm traces up to 4 transformation steps away from the focal asset in both directions.
* When the **ATTR** toggle is enabled, the graph expands to show individual column-level edges within each asset node, revealing exactly which column flows into which.

***

## Lineage levels

Prizm captures lineage at two levels, each serving a different purpose.

### Asset lineage

Asset lineage tracks relationships between tables, views, models, pipelines, files, dashboards, and reports. It answers: *which assets depend on this asset, and which assets does this asset depend on?*

Asset lineage is extracted automatically during source onboarding and refreshed whenever schema changes are detected. It is always enabled and cannot be turned off — it is a core platform capability.

| Relationship type | Description                                                                                             |
| ----------------- | ------------------------------------------------------------------------------------------------------- |
| `depends_on`      | One asset requires another to function — e.g. a view that queries a table                               |
| `derived_from`    | An asset was created based on another — e.g. a gold-layer table derived from a silver-layer model       |
| `transforms`      | Data is transformed when moving from source to destination — e.g. an ETL pipeline enriching a raw table |
| `copies`          | Data is copied without transformation — e.g. a replication job moving data between regions              |
| `aggregates`      | The destination contains aggregated data from the source — e.g. a summary table rolling up daily totals |

### Column (attribute) lineage

Column lineage traces individual columns through transformation logic, capturing exactly how each source column maps to each target column and what transformation was applied. It answers: *where does this specific value come from, and how was it calculated?*

Column lineage is extracted independently from asset lineage, executes after criticality processing, and completes before structural metrics run. It can be scoped by criticality level — by default all criticality levels are included.

```text theme={null}
raw.customers.email
      │
      │  LOWER(TRIM(email))
      ▼
stg.customers_cleaned.email_cleaned
      │
      │  direct mapping
      ▼
fct_customers.email_cleaned
      │
      │  direct mapping
      ▼
customer_360_dashboard.email
```

***

## Onboarding execution order

When a new source is onboarded, Prizm runs lineage jobs in a strict sequence to ensure each stage has the data it needs:

```text theme={null}
1. Technical Catalog       ← discover assets, columns, and metadata
2. Table / View Lineage    ← extract asset-level relationships
3. Criticality             ← score each asset's importance and fan-out
4. Attribute Lineage       ← extract column-level relationships (filtered by criticality)
5. Structural Metrics      ← run profiling on columns
6. Remaining Platform Jobs ← observability, scoring, alerting
```

<Info>
  Attribute lineage must complete **before** structural metrics because profiling uses lineage to understand column relationships and compute criticality-aware quality scores. Skipping or delaying lineage delays the entire profiling pipeline.
</Info>

***

## Schema change and lineage refresh

When Prizm detects a schema change on an asset (column added, removed, renamed, or type changed), it automatically refreshes both table lineage and attribute lineage for the impacted assets only. A full source-wide lineage refresh is not required.

<Warning>
  Schema-driven lineage refresh processes only the impacted assets. If a schema change affects a column that feeds many downstream assets, all downstream attribute lineage edges referencing that column are re-evaluated and updated. Downstream assets that had lineage through the changed column will show updated edge metadata on their next Lineage tab load.
</Warning>

***

## Criticality and attribute lineage

Attribute lineage extraction can be scoped by **criticality level** to balance coverage against compute cost. By default, all criticality levels are enabled — Prizm extracts column-level lineage for every asset regardless of its criticality score.

For large environments where attribute lineage is expensive to extract, you can limit extraction to higher criticality levels (Critical and High only) to focus coverage on the assets that matter most.

| Setting              | Behaviour                                                                         |
| -------------------- | --------------------------------------------------------------------------------- |
| All levels (default) | Column lineage extracted for every asset                                          |
| Critical + High      | Column lineage extracted only for assets with Critical or High criticality scores |
| Critical only        | Column lineage extracted only for business-critical assets                        |

<Note>
  Table and view lineage is **never** affected by criticality settings. It always runs for all assets. Criticality filtering applies only to attribute (column) lineage extraction.
</Note>

***

## Integration with the rest of Prizm

Lineage is not a standalone feature — it enriches every other part of the platform:

| Integration                  | How lineage contributes                                                                                                                                            |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Anomaly detection**        | When an alert fires on an asset, Prizm checks lineage to surface which downstream assets may also be affected and highlights them in amber on the lineage graph    |
| **Schema change monitoring** | When a column is removed or renamed, lineage shows the exact downstream assets and columns that reference it — the blast radius panel on the schema diff view      |
| **Trust scoring**            | Criticality scores (used in trust scoring) are calculated from fan-out and downstream depth — both derived from the lineage graph                                  |
| **Data catalog**             | Lineage edges appear as related assets on every catalog page, and search results include lineage-connected assets                                                  |
| **Quality recommendations**  | Prizm uses lineage to suggest where to add quality checks — high fan-out upstream assets with no monitoring are flagged as unprotected critical paths              |
| **Governance**               | Lineage provides the dependency map required for regulatory compliance — demonstrating data provenance, GDPR data flow documentation, and change impact assessment |

***

## Lineage Capabilities

### Upstream & Downstream Tracking

Prizm automatically captures and visualizes the complete journey of data:

* **Upstream Analysis** — Traces data back to its origins across multiple hops, identifying source systems, transformation logic, and intermediary datasets
* **Downstream Impact** — Maps all consumers and dependent assets affected by changes to a specific dataset or column
* **End-to-End Visibility** — Provides a unified view of data movement across diverse technologies, platforms, and organizational boundaries

### Root Cause & Impact Analysis

Lineage serves as a powerful troubleshooting tool:

* **Issue Propagation** — Traces how data quality problems cascade through pipelines and affect downstream consumers
* **Change Management** — Evaluates potential impacts before implementing schema changes or pipeline modifications
* **Incident Response** — Accelerates time-to-resolution by pinpointing failure points and affected systems during outages

### Criticality & Dependency Calculation

Prizm's lineage engine computes metrics that quantify data relationships:

| Metric                | Description                                                                              |
| --------------------- | ---------------------------------------------------------------------------------------- |
| **Criticality Score** | Measures asset importance based on downstream usage, business impact, and consumer count |
| **Dependency Depth**  | Calculates the number of transformation hops between source and target                   |
| **Fan-Out Ratio**     | Identifies high-impact datasets with numerous dependent consumers                        |
| **Usage Weighting**   | Applies weights based on consumption patterns (analytical vs. operational)               |

***