> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dqlabs.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# What We Collect

> Field-level breakdown of every metadata object, observability signal, and lineage edge Prizm extracts from Sigma.

<script type="application/ld+json">
  {`{
            "@context": "https://schema.org",
            "@type": "TechArticle",
            "headline": "Sigma: What We Collect",
            "description": "Field-level breakdown of every metadata object, observability signal, and lineage edge Prizm extracts from Sigma.",
            "url": "https://docs.dqlabs.ai/sources/sigma/what-we-collect",
            "publisher": {
              "@type": "Organization",
              "name": "DQLabs Inc",
              "logo": "https://media.brand.dev/332adc35-5bc4-4d2b-bf78-256aa4a5e414.svg"
            }
            }`}
</script>

Prizm extracts Sigma metadata via the Sigma REST API (`/v2/...`) and, for lineage and usage, the Sigma Logs API.

<Tabs>
  <Tab title="Catalog">
    **What this collects:** Workspaces, Workbooks, Pages, Data Elements, and Datasets into the Prizm Asset Catalog.

    | Sigma object         | Primary source                                  | Key properties extracted                                          |
    | :------------------- | :---------------------------------------------- | :---------------------------------------------------------------- |
    | Workspace            | `GET /v2/workspaces`                            | id, name, description, createdAt, updatedAt, url                  |
    | Workbook             | `GET /v2/workbooks`                             | id, name, description, createdAt, updatedAt, url, path, owner     |
    | Page                 | `GET /v2/workbooks/{id}/pages`                  | id, title, workbook\_id                                           |
    | Data Element (Table) | `GET /v2/workbooks/{id}/elements`               | displayName, type, columns, dataSourceId                          |
    | Data Element (Pivot) | `GET /v2/workbooks/{id}/elements`               | displayName, type, rowKeys, columnKeys, values                    |
    | Data Element (Viz)   | `GET /v2/workbooks/{id}/elements`               | displayName, chartType, xAxis, yAxis, colorDimension              |
    | Data Element Field   | `GET /v2/workbooks/{id}/elements/{eid}/columns` | displayName, formula, type (calculated or base)                   |
    | Dataset              | `GET /v2/datasets`                              | id, name, description, createdAt, updatedAt, url, warehouseSource |
    | Tag                  | `GET /v2/workbooks/{id}/tags`                   | tag\_name, tag\_value, attached\_asset\_type, attached\_asset\_id |

    **Metadata captured per Workbook:** id, name, description, path, workspace\_id, workspace\_name, owner (display name and email), created\_at, updated\_at, deep-link URL back to Sigma, tag names/values, page count, and data element count.

    **Metadata captured per Dataset:** id, name, description, url, created\_at, updated\_at, and `warehouse_source` (connection name, database, schema, table) — used for lineage.

    <Note>
      Pagination is handled transparently for Workbooks and Datasets with more than 100 items. Assets in workspaces excluded by the connector's workspace filter are not catalogued. Endorsement level (certified / promoted) is not currently exposed by the Sigma API and is not captured.
    </Note>
  </Tab>

  <Tab title="Data Elements & Calculated Fields">
    Sigma's semantic layer sits between raw warehouse tables and the end visualisation. Prizm distinguishes two field types:

    | Field type       | Description and storage in Prizm                                                                                                                                           |
    | :--------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
    | Base field       | Direct reference to a warehouse column; stored with column name and warehouse path for lineage resolution                                                                  |
    | Calculated field | Field defined by a Sigma formula expression; stored with `displayName` and `sigmaDataElementFieldFormula`; marked as calculated in the catalog with a **Calculated** badge |

    Formula changes in calculated fields are detected as field-level schema drift — a formula diff between consecutive catalog runs generates a drift event in the Prizm observability feed.
  </Tab>

  <Tab title="Lineage">
    **What this collects:** the full lineage chain from a warehouse table through a Sigma Dataset, Workbook, Page, and Data Element to end consumers.

    | Source (in priority order)               | What it provides                                                                                          |
    | :--------------------------------------- | :-------------------------------------------------------------------------------------------------------- |
    | Sigma Logs API (`GET /v2/usage/queries`) | Query logs recording which warehouse tables each Sigma element executed against — primary lineage source  |
    | Dataset metadata (`warehouseSource`)     | Direct link from a Sigma Dataset to a warehouse table/schema                                              |
    | Data Element column references           | Fallback resolution via base warehouse columns, using the warehouse connector already configured in Prizm |

    | Asset            | Upstream link                                                        | Downstream link                                               |
    | :--------------- | :------------------------------------------------------------------- | :------------------------------------------------------------ |
    | Dataset          | Warehouse connection + database + schema + table                     | Workbooks and Data Elements referencing this Dataset          |
    | Workbook         | Datasets used by its Data Elements                                   | Pages; end consumers (usage log users)                        |
    | Data Element     | Parent Page + Workbook; upstream Dataset or direct warehouse columns | Visualisations or downstream consumers embedding this element |
    | Calculated Field | Base fields or other calculated fields referenced in its formula     | Other calculated fields that reference this field             |

    <Note>
      Workbooks with no Dataset (querying the warehouse directly) still get lineage resolved via query log analysis. Lineage from an upstream dbt or ADF pipeline into a Sigma Dataset resolves automatically when both connectors are active in Prizm.
    </Note>
  </Tab>

  <Tab title="Criticality & Monitoring">
    **Criticality** for Workbooks and Datasets uses the same BI-asset framework as Power BI and Tableau: 60% Dependency (lineage to underlying sources) + 40% Usage (unique users). Scores are re-evaluated whenever upstream table criticality changes or workbook usage changes materially.

    **Monitoring signals:**

    | Signal                          | Detection logic                                                             | Default severity |
    | :------------------------------ | :-------------------------------------------------------------------------- | :--------------- |
    | Zero views in 30 days           | No usage log entries in a 30-day window                                     | Low              |
    | Dataset not refreshed           | Dataset `last_updated` age exceeds the refresh SLA                          | High             |
    | Upstream schema change          | A warehouse column referenced by a Sigma Data Element is dropped or renamed | Critical         |
    | Calculated field formula change | Formula diff between consecutive Catalog Job runs                           | Medium           |
    | Usage spike / drop              | 30-day rolling view count deviates >50% from the 90-day baseline            | Medium           |

    Stale workbooks appear in the observability feed within one hour of an SLA breach (Operational job cycle). Upstream schema-change alerts propagate from the warehouse connector to affected Sigma Workbooks via lineage within one evaluation cycle. Zero-view workbooks are listed in the unused-assets report with last view date and owner.
  </Tab>

  <Tab title="Metrics">
    | Metric                             | Workbook                        | Dataset                         | Source                                                          |
    | :--------------------------------- | :------------------------------ | :------------------------------ | :-------------------------------------------------------------- |
    | Freshness                          | Default enabled                 | Default enabled                 | `last_updated` from the Sigma API; age = now − last\_updated    |
    | Reliability (refresh success rate) | Default enabled                 | Default enabled                 | Failed refresh events from usage log / Dataset refresh status   |
    | Usage (view count)                 | Default disabled                | Default disabled                | `view_count_30d` from the Sigma usage log                       |
    | Unique users (30d)                 | Default disabled                | Default disabled                | Distinct users from the Sigma usage log                         |
    | DQ Score                           | Propagated from upstream tables | Propagated from upstream tables | Inherited via lineage from the upstream warehouse table/dataset |

    <Warning>
      Reliability and Usage metrics depend on Sigma's audit/query log access, which requires a **Premier-tier** Sigma account with logging enabled. On accounts without that access, these metrics are unavailable rather than approximated.
    </Warning>

    DQ Score propagation is controlled by the **Propagate Scores Based On** toggle in the connector configuration and can be disabled per connection without affecting the upstream warehouse alerts themselves.
  </Tab>

  <Tab title="Alerts & Issues">
    Sigma extends the same Prizm Alert Module framework used across all connectors:

    | Alert                           | Trigger                                                                    | Propagation path                                          | Severity         |
    | :------------------------------ | :------------------------------------------------------------------------- | :-------------------------------------------------------- | :--------------- |
    | Upstream DQ rule violation      | A quality rule fires on a warehouse table feeding a Sigma Dataset          | Warehouse Table → Sigma Dataset → Workbook → Data Element | Inherited        |
    | Upstream schema change          | Column referenced by a Sigma Data Element base field is dropped or renamed | Warehouse Table → Sigma Dataset → affected Data Elements  | Critical         |
    | Calculated field formula change | Formula diff between catalog runs                                          | Sigma Data Element Field                                  | Medium           |
    | Workbook freshness breach       | `last_updated` exceeds SLA threshold                                       | Sigma Workbook                                            | High             |
    | Unused workbook                 | Zero views in a 30-day window                                              | Sigma Workbook                                            | Low (governance) |
    | Dataset refresh failure         | `last_updated` stale beyond the configured window                          | Sigma Dataset                                             | High             |

    Propagated alerts carry the upstream source asset name and lineage path in the notification payload, and the DQ score shown on a Sigma asset indicates whether it's native to Sigma or propagated from upstream.

    **Event & workflow management:** freshness breaches, refresh failures, upstream schema changes, and unused-workbook signals automatically create Prizm issues for Critical and High severity events. Each issue links to the affected Sigma asset with a deep-link back to the Workbook in Sigma, and Jira/ADO integration cascades fire from Sigma-sourced issues the same way they do for any other connector.
  </Tab>
</Tabs>

## Next Steps

<CardGroup cols={2}>
  <Card title="Setup" icon="plug" href="/sources/sigma/setup">
    Connect Sigma to Prizm.
  </Card>

  <Card title="FAQ" icon="circle-question" href="/sources/sigma/faq">
    Common questions about the Sigma connector.
  </Card>
</CardGroup>
