TechPrufer
Get started
GitLab
GitLab logoGitLab · Feature trail

Secure BigQuery Data Publishing with GitLab CI/CD

1 post · Mar 2025

What this trail covers

GitLab CI/CD is enhanced to automate and secure data exports from BigQuery to Google Cloud Storage. This involves version-controlling SQL queries, authenticating with Google Cloud via service accounts or Workload Identity Federation, executing queries, exporting results as CSV, and setting public read access on the exported files. This provides an auditable, repeatable, and secure data publishing workflow.

All GitLab trails
  1. Tutorial: Secure BigQuery data publishing with GitLabThis post details a tutorial for securely publishing BigQuery data to Google Cloud Storage using GitLab CI/CD. It covers setting up Google Cloud credentials (service account key or Workload Identity Federation), creating a SQL query file, and configuring a `.gitlab-ci.yml` pipeline to authenticate with Google Cloud, execute the query, export results to a CSV, upload to GCS, and set public read access. It highlights benefits like security, auditability, repeatability, version control, and automation.