> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dqlabs.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# BigQuery Setup

> APIs to enable, roles to grant, and authentication options for connecting BigQuery to Prizm.

This page lists what you need in Google Cloud before connecting BigQuery to Prizm: the APIs to enable, the authentication method, and the roles to grant for each kind of metadata you want Prizm to extract.

## Authentication

Choose one of the following. Both need the APIs and roles described below.

| Method | Description |
| :- | :- |
| Service account key | A key file for a service account that holds the required roles |
| Workload Identity | Keyless authentication through Google's identity federation. This is why the IAM Service Account Credentials API and Security Token Service API are on the list below. It's offered on the connection form but hasn't been verified end to end, so use a service account key unless your Prizm team has confirmed otherwise |

## Enable the Required APIs

Enable these APIs in the host project, and in each monitored project where noted.

```bash theme={null}
gcloud services enable \
  bigquery.googleapis.com \
  datalineage.googleapis.com \
  dataplex.googleapis.com \
  datacatalog.googleapis.com \
  cloudresourcemanager.googleapis.com \
  iamcredentials.googleapis.com \
  sts.googleapis.com \
  --project=<PROJECT_ID>
```

| API | Used for |
| :- | :- |
| `bigquery.googleapis.com` | Datasets, tables, columns, job history |
| `datalineage.googleapis.com` | Lineage from the Data Lineage API |
| `dataplex.googleapis.com` | Dataplex Catalog tags and aspects |
| `datacatalog.googleapis.com` | Data Catalog tags and aspects |
| `cloudresourcemanager.googleapis.com` | Listing projects and resource tags |
| `iamcredentials.googleapis.com` and `sts.googleapis.com` | Workload Identity authentication |

> [!WARNING]
>
> **Enable the Data Lineage API early, in every project where jobs run.** Lineage is recorded only from the time the API is enabled. There is no backfill, and Google retains lineage records for a limited period. Check Google's Data Lineage documentation for the current retention window.

## Grant the Required Roles

Grant only the roles for the features you want. Roles marked per project go on each project Prizm monitors.

| What you want | Where Prizm reads it | Role or permission |
| :- | :- | :- |
| List projects | Resource Manager API | `roles/browser`, or the `resourcemanager.projects.list` permission, at organization or folder level. Alternatively, `roles/cloudasset.viewer` for Cloud Asset Inventory |
| Datasets, tables, and columns | `INFORMATION_SCHEMA` views (`SCHEMATA`, `TABLES`, `COLUMNS`, `COLUMN_FIELD_PATHS` for nested fields) or the REST API | `roles/bigquery.metadataViewer` on each project, plus `roles/bigquery.jobUser` to run queries |
| Labels | `SCHEMATA_OPTIONS`, `TABLE_OPTIONS`, or the datasets and tables APIs | `roles/bigquery.metadataViewer` |
| Policy tags (column mapping) | `COLUMN_FIELD_PATHS` policy tags, or the table schema from the tables API | `roles/bigquery.metadataViewer` |
| Resource tags (keys and values) | Resource Manager tag bindings | `roles/resourcemanager.tagViewer` (or `tagUser`) |
| Data Catalog tags and aspects | Dataplex Catalog and Data Catalog entries | `roles/dataplex.catalogViewer` and `roles/datacatalog.viewer`. Grant the one that matches what your organization uses |
| Lineage from the Data Lineage API | Data Lineage API | `roles/datalineage.viewer` |
| Lineage from job logs | `INFORMATION_SCHEMA.JOBS` | `roles/bigquery.resourceViewer` |
| Usage and performance | `JOBS_BY_PROJECT`, `JOBS_BY_ORGANIZATION`, and `JOBS_TIMELINE` | `roles/bigquery.resourceViewer`, which includes `bigquery.jobs.listAll`. `JOBS_BY_ORGANIZATION` needs the role at organization level |

### Example: grant roles to a service account

```bash theme={null}
SA="prizm-reader@<PROJECT_ID>.iam.gserviceaccount.com"

for ROLE in \
  roles/bigquery.metadataViewer \
  roles/bigquery.jobUser \
  roles/bigquery.resourceViewer \
  roles/datalineage.viewer; do
  gcloud projects add-iam-policy-binding <PROJECT_ID> \
    --member="serviceAccount:${SA}" --role="${ROLE}"
done
```

Repeat for each monitored project, and add the organization- or folder-level roles (project listing, tags, `JOBS_BY_ORGANIZATION`) where you need them.

> [!NOTE]
>
> Prizm doesn't create, change, or delete datasets or tables. Beyond metadata and job history, it needs to run queries (`bigquery.jobs.create`, covered by `roles/bigquery.jobUser`) and, for quality and profiling checks, to read table data (`bigquery.tables.getData`). For external tables, it also needs to read the storage bucket (`storage.buckets.get`, `storage.buckets.list`, `storage.objects.get`, `storage.objects.list`). Tag import needs `resourcemanager.tagKeys.list`, `resourcemanager.tagValues.list`, `datacatalog.taxonomies.list`, and `datacatalog.taxonomies.get`.

## Connect BigQuery in Prizm

**1. Open the connector wizard**

Navigate to **Source** and click the **+** icon next to the source list. Search for **BigQuery** and select it from the connector grid.

**2. Enter source details**

| Field | Description |
| :- | :- |
| Connection Name | A label for this connection |
| Description | Optional notes about what this connection covers |
| GCP Project ID | The Google Cloud project that contains your BigQuery datasets |
| GCP Region / Location | The location of your datasets, for example `US`, `EU`, or `us-central1`. Prizm needs it to query `INFORMATION_SCHEMA` |
| Extract Lineage | On by default. Turn it off to skip lineage |
| Lineage Mode | **Via DataPlex API** (default) or **Job Logs**. See [Overview](/sources/bigquery/overview#lineage-sources) |

![The BigQuery Source Details form](https://mintcdn.com/dqlabs/-llXdpktU4fg82jU/images/sources/bigquery/form-top.png?fit=max\&auto=format\&n=-llXdpktU4fg82jU\&q=85\&s=9b60c63433a330c31bcfaee44785d194)

**3. Choose an authentication method**

Select **Service Account Key** (the default) or **Workload Identity**. For a service account key, upload the JSON key file, or turn on **Use Vault** to retrieve it from a configured vault.

![The BigQuery lineage and authentication fields with the Service Account Key upload area](https://mintcdn.com/dqlabs/-llXdpktU4fg82jU/images/sources/bigquery/form-bottom.png?fit=max\&auto=format\&n=-llXdpktU4fg82jU\&q=85\&s=93c072418e116e6dcbe42c017f676732)

**4. Validate and choose scope**

Click **Next** to validate the connection, then choose which datasets Prizm should catalog.

**5. Save**

Complete the configuration. The source then appears in the **Source** list, where the **Valid** column shows a green check for a working connection.

## Next Steps

> **[What We Collect](/sources/bigquery/what-we-collect)**
>
> See what Prizm extracts from BigQuery.

> **[FAQ](/sources/bigquery/faq)**
>
> Common questions about authentication, roles, and lineage.

> **[Troubleshooting](/help/troubleshooting)**
>
> Diagnose connection failures, permission errors, and timeouts.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.