To build flexible alerts based on your BigQuery data you can connect it to Rupert following the steps listed below.
Locate your dataset id
Head over to the BigQuery workspace explorer, choose the relevant dataset and copy its dataset ID
Locate your project id
This is the Google Cloud project where your BigQuery instance is hosted
Click the project name
Copy the ID next to the project name
Service account key - 2 ways to generate the JSON file
Service account key is a JSON file containing the info needed to connect to Rupert
1. Automatically
Generate your service account key JSON file by copying the data below, replacing the default values with your warehouse info and uploading it in the Rupert connect screen
RUPERT_ACCOUNT_NAME=rupert-bigquery
RUPERT_GCP_PROJECT=rupert-internal
RUPERT_DATASET_ID=rupert-internal.test
# Create the service account
gcloud --project=$RUPERT_GCP_PROJECT iam service-accounts create $RUPERT_ACCOUNT_NAME \
--description="Rupert service account for BigQuery" \
--display-name="rupert-bigquery"
# Create the role to be assigned to the service account
gcloud --project=$RUPERT_GCP_PROJECT iam roles create bigquery.jobsViewer \
--title="BigQuery - Jobs Viewer" \
--permissions="bigquery.jobs.listAll,bigquery.savedqueries.get,bigquery.savedqueries.list" \
--stage=GA
# Assign the custom role to the service account
gcloud projects add-iam-policy-binding $RUPERT_GCP_PROJECT \
--member="serviceAccount:$RUPERT_ACCOUNT_NAME@$RUPERT_GCP_PROJECT.iam.gserviceaccount.com" \
--role="projects/$RUPERT_GCP_PROJECT/roles/bigquery.jobsViewer"
# Assign the dataset reader role to the service account
gcloud projects add-iam-policy-binding $RUPERT_GCP_PROJECT \
--member="serviceAccount:$RUPERT_ACCOUNT_NAME@$RUPERT_GCP_PROJECT.iam.gserviceaccount.com" \
--role="roles/bigquery.dataViewer"
# Create and save the service account credentials to ./rupert-sa-credentials.json
gcloud --project=$RUPERT_GCP_PROJECT iam service-accounts keys create "rupert-sa-credentials.json" \
--iam-account="$RUPERT_ACCOUNT_NAME@$RUPERT_GCP_PROJECT.iam.gserviceaccount.com"
2. Generate your service account key JSON by yourself
1. Create a new service account for Rupert
2. Assign the service account the following roles:
roles/bigquery.dataViewer
(BigQuery Data Viewer)BigQuery Job User
3. Create a custom role and assign it the following permissions:
bigquery.jobs.listAll
Bigquery.savedqueries.get
bigquery.savedqueries.list
4. Assign the custom role to Rupert's service account