> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dqlabs.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# SQL Server Overview

> What Microsoft SQL Server is, why connecting it to Prizm unlocks data intelligence, and what capabilities Prizm supports.

<script type="application/ld+json">
  {`{
            "@context": "https://schema.org",
            "@type": "TechArticle",
            "headline": "SQL Server Overview",
            "description": "What Microsoft SQL Server is, why connecting it to Prizm unlocks data intelligence, and what capabilities Prizm supports.",
            "url": "https://docs.dqlabs.ai/sources/sql/overview",
            "publisher": {
              "@type": "Organization",
              "name": "DQLabs Inc",
              "logo": "https://media.brand.dev/332adc35-5bc4-4d2b-bf78-256aa4a5e414.svg"
            }
            }`}
</script>

Microsoft SQL Server is a widely used relational database system deployed on-premises, in private data centers, and on Azure. Once connected, Prizm continuously monitors your SQL Server tables for quality anomalies, tracks schema and volume changes, builds lineage graphs from foreign key relationships and stored procedure dependencies, and surfaces performance insights — all without writing pipelines or custom queries.

## Why Connect SQL Server to Prizm?

Connecting SQL Server gives Prizm access to three layers of intelligence:

**Catalog & Context** Prizm discovers every database, schema, table, view, stored procedure, and column in your SQL Server instance and indexes them in the Prizm catalog. Descriptions sourced from `MS_Description` extended properties are imported automatically. Ownership, classifications, and custom metadata can be managed in Prizm.

**Data Quality & Profiling** Prizm runs profile scans on your tables to compute null rates, cardinality, min/max, distribution, and completeness scores at the column level. Quality scores are tracked over time so you can see trends and catch degradation before it reaches consumers.

**Observability** Prizm monitors every in-scope table for freshness (time since last user update via `sys.dm_db_index_usage_stats`), volume (row count changes from `sys.partitions`), and schema drift (added, removed, or renamed columns). Machine-learning anomaly detection sets adaptive thresholds so alerts fire on real deviations — not noise.

## Metric Support

Every metric in Prizm belongs to a **context** — the stakeholder lens that defines who the metric serves and why it matters. The following table provides the list of all supported metrics.

<Tabs>
  <Tab title="Operational">
    Operational metrics monitor the **day-to-day health of data assets** — whether data is arriving on time, in the expected volume, and with the correct structure. They run at the **Asset** level and are the primary driver of alerting and anomaly detection in Prizm.

    <Note>
      All Operational metrics feed directly into the Alerts dashboard. They are the most actively monitored metrics in a typical Prizm deployment, with Freshness and Volume generating the highest alert volumes in production.
    </Note>

    | Metric       | Subcategory | Dimension    | Level | Monitor | Description                                                                                                                                                                      |
    | :----------- | :---------- | :----------- | :---- | :------ | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
    | Row          | Volume      | Completeness | Asset | Yes     | Total number of rows at the time of last scan. Tracks volume over time and triggers alerts on drops or spikes.                                                                   |
    | Volume       | Volume      | Completeness | Asset | Yes     | Higher-level measure combining row count and data size signals. Used to detect significant changes in the amount of data arriving.                                               |
    | Data size    | Volume      | Completeness | Asset | Yes     | Physical size of the asset in bytes (from `sys.allocation_units`). Complements row count — a large size increase with no row increase may indicate wide or bloated records.      |
    | Freshness    | Freshness   | Availability | Asset | Yes     | Time elapsed since the asset was last updated (from `sys.dm_db_index_usage_stats.last_user_update`). Fires an alert when data has not been refreshed within the expected window. |
    | Last updated | Freshness   | Timeliness   | Asset | Yes     | Timestamp of the most recent data write. Verifies pipelines are running on schedule and consumers are not working with stale records.                                            |
    | Column       | Schema      | Validity     | Asset | Yes     | Current number of columns. Detects column additions or removals indicating an unannounced schema change at the source.                                                           |
    | Schema       | Schema      | Validity     | Asset | Yes     | Snapshot of the full column schema — names, types, and order. Alerts when any column name or type changes, which can silently break downstream transformations.                  |
    | Schema name  | Schema      | Validity     | Asset | Yes     | Name of the schema the asset belongs to. Tracks unexpected schema renames that would break hard-coded references.                                                                |
    | Duplicates   | Duplicate   | Uniqueness   | —     | Yes     | Count of duplicate rows based on a defined key or all columns. Unexpected duplicates often indicate a pipeline re-run without deduplication or broken merge logic.               |

    <Info>
      Operational metrics do **not** contribute to trust scoring (`Score: No`). They are observability signals — designed to trigger alerts, not to roll up into a quality score.
    </Info>
  </Tab>

  <Tab title="Performance">
    Performance metrics measure **how efficiently the data platform itself is running** — query execution time, session counts, and query volume. They operate at the **Source and Database level** rather than the individual asset level.

    | Metric                | Level    | Profile Type | Description                                                                                                          |
    | :-------------------- | :------- | :----------- | :------------------------------------------------------------------------------------------------------------------- |
    | Total queries per day | Database | Performance  | Total queries executed per day (from `sys.dm_exec_query_stats`). Tracks usage trends and surfaces unexpected spikes. |
    | Query success rate    | Database | Performance  | Percentage of queries completing without error. A falling rate signals pipeline instability or permission issues.    |
    | Execution time        | Database | Performance  | Average query execution time. Rising times indicate missing indexes or poorly optimized SQL.                         |
    | Session count         | Database | Performance  | Number of active sessions (from `sys.dm_exec_sessions`). Spikes indicate unusual concurrency or runaway processes.   |
    | Database storage      | Database | Performance  | Total storage consumed by the database including tables, indexes, and transaction logs. Tracks growth trends.        |

    <Info>
      Performance metrics do **not** contribute to trust scoring (`Score: No`). They are platform observability signals for infrastructure and database administrators.
    </Info>
  </Tab>

  <Tab title="Structural">
    Structural metrics profile the **internal shape of data at the column level** — how values are distributed, how often patterns repeat, and what the numbers say. All structural metrics are **auto-generated** when an asset is profiled — no configuration required.

    **Distribution** — How values are composed and spread:

    | Metric            | Subcategory  | Dimension    | Profile Type | Description                                                                                       |
    | :---------------- | :----------- | :----------- | :----------- | :------------------------------------------------------------------------------------------------ |
    | Null count        | Completeness | Completeness | Essential    | Total null values. High null count signals missing data that may break downstream logic.          |
    | Empty             | Completeness | Completeness | Essential    | Count of empty strings (`""`). Distinct from null — present but containing no meaningful content. |
    | Non-empty         | Completeness | Completeness | Advanced     | Count of values that are neither null nor empty.                                                  |
    | Space             | Completeness | Completeness | Essential    | Values containing only whitespace.                                                                |
    | Zero value        | Numeric      | Completeness | Essential    | Numeric values equal to zero. Flags columns where zero may be a placeholder for missing data.     |
    | Distinct          | Uniqueness   | Uniqueness   | Essential    | Count of unique values.                                                                           |
    | Repeating         | Uniqueness   | Uniqueness   | Advanced     | Count of values appearing more than once.                                                         |
    | Leading space     | Space        | Validity     | Advanced     | Values with spaces at the start. Can cause join failures and mismatches in downstream queries.    |
    | Trailing space    | Space        | Validity     | Advanced     | Values with spaces at the end.                                                                    |
    | Special character | Character    | Validity     | Advanced     | Values containing at least one special character.                                                 |

    **Frequency** — How often values, lengths, and ranges repeat:

    | Metric     | Subcategory | Dimension | Profile Type | Description                                                                                                             |
    | :--------- | :---------- | :-------- | :----------- | :---------------------------------------------------------------------------------------------------------------------- |
    | Enum       | Value       | Validity  | Advanced     | Distinct values and their frequencies. Reveals category distribution and flags values outside an expected allowed list. |
    | Min value  | Value       | Validity  | Essential    | Smallest value in the column.                                                                                           |
    | Max value  | Value       | Validity  | Essential    | Largest value in the column.                                                                                            |
    | Min length | Length      | Validity  | Essential    | Shortest string length found. Catches truncated values.                                                                 |
    | Max length | Length      | Validity  | Essential    | Longest string length found.                                                                                            |

    **Pattern** — Structural format templates:

    | Metric              | Subcategory | Dimension | Profile Type | Description                                                                             |
    | :------------------ | :---------- | :-------- | :----------- | :-------------------------------------------------------------------------------------- |
    | Short pattern       | Pattern     | Validity  | Advanced     | Condensed format — collapses consecutive identical character types into a single token. |
    | Long pattern        | Pattern     | Validity  | Advanced     | Character-by-character format — preserves full length and composition.                  |
    | Regular expressions | Pattern     | Validity  | Essential    | User-defined regex patterns to validate that column values conform to a known format.   |

    **Statistics** — Numeric summaries:

    | Metric             | Subcategory | Dimension | Profile Type | Description                                   |
    | :----------------- | :---------- | :-------- | :----------- | :-------------------------------------------- |
    | Mean               | Descriptive | Accuracy  | Advanced     | Arithmetic average of all non-null values.    |
    | Median             | Descriptive | Accuracy  | Advanced     | Middle value when sorted.                     |
    | Standard deviation | Descriptive | Accuracy  | Advanced     | Spread of values around the mean.             |
    | Sum                | Descriptive | Accuracy  | Advanced     | Total of all non-null numeric values.         |
    | Variance           | Shape       | Accuracy  | Advanced     | Average of squared differences from the mean. |
  </Tab>

  <Tab title="Business">
    Business metrics are **user-defined measurements aligned to domain rules, KPIs, and data contracts**. Unlike structural and operational metrics which are auto-generated, business metrics are created by analysts, engineers, and domain SMEs to capture logic the platform cannot infer automatically.

    | Type        | Description                                                                                              |
    | :---------- | :------------------------------------------------------------------------------------------------------- |
    | Custom      | User-defined metric applied to a specific asset using filters, expressions, or threshold rules.          |
    | Conditional | Metric applying different logic or thresholds depending on another column's value.                       |
    | Query       | Metric defined by a SQL query returning a scalar value. Enables joins, aggregations, and CTEs.           |
    | Standalone  | Metric not bound to a specific asset. Used for org-wide KPIs or cross-domain aggregations.               |
    | Behavioral  | Metric evaluating patterns over time rather than a single point. Detects trend changes or gradual drift. |
  </Tab>

  <Tab title="Reconciliation">
    Reconciliation metrics **compare data across sources, snapshots, or reference tables** to verify consistency and correctness. Critical for cross-system data contracts, ETL validation, and regulatory compliance.

    | Type       | Description                                                                                                                                    |
    | :--------- | :--------------------------------------------------------------------------------------------------------------------------------------------- |
    | Comparison | Compares a column or aggregate value between two registered assets — e.g. row count in a source table versus the same table after an ETL load. |
    | Lookup     | Validates that every value in a column exists in a reference dataset or allowed set. Used for referential integrity checks.                    |

    <Warning>
      Comparison metrics require both source and target assets to be registered and accessible in Prizm. If a source asset is unreachable, the metric shows `Score: NA` and raises an availability alert.
    </Warning>
  </Tab>

  <Tab title="Semantics">
    Semantic metrics are derived at a **logical or business grouping level** — domain, product, application, tag, or connection — rather than at the individual asset level. They aggregate quality signals across ownership boundaries to enable domain- and product-level KPIs and SLOs.

    | Grouping       | Example Metric              | Description                                                                  |
    | :------------- | :-------------------------- | :--------------------------------------------------------------------------- |
    | **Domain**     | DQ score for Finance domain | Weighted quality score rolled up across all assets assigned to a domain      |
    | **Product**    | Data product health score   | Aggregate quality and freshness across all assets in a defined data product  |
    | **Connection** | Source reliability score    | Aggregate quality score across all assets from a given SQL Server connection |
  </Tab>
</Tabs>

## Catalog & Lineage Support

| Category    | Capability                                                      | Support |
| :---------- | :-------------------------------------------------------------- | :------ |
| **Catalog** | Metadata discovery (databases, schemas, tables, views, columns) | ✅       |
| **Catalog** | Stored procedure discovery                                      | ✅       |
| **Catalog** | MS\_Description import (existing column/table descriptions)     | ✅       |
| **Lineage** | Table-level lineage (via FK constraints)                        | ✅       |
| **Lineage** | View-level lineage (via `sys.dm_sql_referenced_entities`)       | ✅       |
| **Lineage** | Stored procedure lineage                                        | ✅       |
| **Lineage** | Column-level lineage                                            | ✅       |

## Supported SQL Server Objects

| Object               | Catalog | Quality | Observability | Lineage |
| :------------------- | :------ | :------ | :------------ | :------ |
| **Table**            | ✅       | ✅       | ✅             | ✅       |
| **View**             | ✅       | —       | Schema only   | ✅       |
| **Stored Procedure** | ✅       | —       | —             | ✅       |

## Supported Deployment Modes

| Deployment                 | Supported |
| :------------------------- | :-------- |
| On-premises SQL Server     | ✅         |
| Self-hosted VM             | ✅         |
| Azure SQL Database         | ✅         |
| Azure SQL Managed Instance | ✅         |

<Note>
  For air-gapped or private network deployments, use the Prizm Outbound Agent with the SQL Server connector. The agent runs inside your network and connects outbound to Prizm over WSS port 443 — no inbound firewall rules required.
</Note>

## Next Steps

<CardGroup cols={2}>
  <Card title="Setup" icon="plug" href="/sources/sql/setup">
    Connect SQL Server to Prizm — prerequisites, authentication, and configuration.
  </Card>

  <Card title="What We Collect" icon="database" href="/sources/sql/what-we-collect">
    Full field-level breakdown of every metadata object Prizm extracts from SQL Server.
  </Card>

  <Card title="FAQ" icon="circle-question" href="/sources/sql/faq">
    Common questions about the SQL Server connector.
  </Card>

  <Card title="Snowflake Connector" icon="snowflake" href="/sources/snowflake/overview">
    Connect your Snowflake data warehouse.
  </Card>
</CardGroup>
