Connect BigQuery to Rupert
Adi Cohen avatar
Written by Adi Cohen
Updated over a week ago

To build flexible alerts based on your BigQuery data you can connect it to Rupert following the steps listed below


Locate your dataset id

  • Head over to the BigQuery workspace explorer, choose the relevant dataset and copy its dataset ID


Locate your project id

This is the Google Cloud project where your BigQuery instance is hosted

  • Click the project name

  • Copy the ID next to the project name


Service account key - 2 ways to generate the JSON file

Service account key is a JSON file containing the info needed to connect to Rupert

1. Automatically

Generate your service account key JSON file by copying the data below, replacing the default values with your warehouse info and uploading it in the Rupert connect screen

RUPERT_ACCOUNT_NAME=rupert-bigquery
RUPERT_GCP_PROJECT=rupert-internal
RUPERT_DATASET_ID=rupert-internal.test

# Create the service account
gcloud --project=$RUPERT_GCP_PROJECT iam service-accounts create $RUPERT_ACCOUNT_NAME \
--description="Rupert service account for BigQuery" \
--display-name="rupert-bigquery"

# Create the role to be assigned to the service account
gcloud --project=$RUPERT_GCP_PROJECT iam roles create bigquery.jobsViewer \
--title="BigQuery - Jobs Viewer" \
--permissions="bigquery.jobs.listAll,bigquery.savedqueries.get,bigquery.savedqueries.list" \
--stage=GA

# Assign the custom role to the service account
gcloud projects add-iam-policy-binding $RUPERT_GCP_PROJECT \
--member="serviceAccount:$RUPERT_ACCOUNT_NAME@$RUPERT_GCP_PROJECT.iam.gserviceaccount.com" \
--role="projects/$RUPERT_GCP_PROJECT/roles/bigquery.jobsViewer"

# Assign the dataset reader role to the service account
gcloud projects add-iam-policy-binding $RUPERT_GCP_PROJECT \
--member="serviceAccount:$RUPERT_ACCOUNT_NAME@$RUPERT_GCP_PROJECT.iam.gserviceaccount.com" \
--role="roles/bigquery.dataViewer"

# Create and save the service account credentials to ./rupert-sa-credentials.json
gcloud --project=$RUPERT_GCP_PROJECT iam service-accounts keys create "rupert-sa-credentials.json" \
--iam-account="$RUPERT_ACCOUNT_NAME@$RUPERT_GCP_PROJECT.iam.gserviceaccount.com"

2. Generate your service account key JSON by yourself

1. Create a new service account for Rupert

2. Assign the service account the following role: roles/bigquery.dataViewer
(BigQuery Data Viewer)

3. Create a custom role and assign it the following permissions:

  • bigquery.jobs.listAll

  • Bigquery.savedqueries.get

  • bigquery.savedqueries.list

4. Assign the custom role to Rupert's service account


Did this answer your question?