Cloud Run Jobs + Cloud Scheduler: Production Batch Workloads with Terraform | 2026

What Are Cloud Run Jobs with Cloud Scheduler?

A Cloud Run job is a Cloud Run workload that runs code to completion and exits — unlike a service, it listens on no port and serves no traffic. Each execution runs one or more tasks (copies of your container) that can run in parallel and retry independently. Cloud Scheduler is GCP's managed cron: on a schedule you define, it POSTs to the Run Admin API endpoint `projects/.../jobs/JOB:run` authenticated with an OAuth access token from a service account you designate, which starts a new execution. Together they replace cron VMs, App Engine cron, and lightweight Composer DAGs for scheduled batch work — ETL exports, report generation, cache warm-ups, cleanup sweeps.

Think of a Cloud Run job like a courier company you call only when there's a delivery — you don't keep vans idling in a depot. Cloud Scheduler is the dispatcher who phones in the order every night at 02:00, tasks are the individual couriers who each take one neighborhood (a shard of the work), and task retries are the standing instruction to try the delivery again if the gate was locked — but only a set number of times before flagging it as failed.

Why Cloud Run Jobs + Scheduler Instead of Functions, Composer, or GKE CronJobs?

Scheduled batch work on GCP tends to drift into one of three traps: Cloud Functions squeezed past their comfort zone (60-minute max timeout, one invocation = one unit of work, no native sharding), a full Cloud Composer environment (Airflow) burning $300+/month to run a single nightly script, or a GKE CronJob that forces you to own a cluster 24/7 for a container that runs 20 minutes a day. All three also hide the same operational gaps: no built-in task-level retries, no parallel fan-out primitive, and cron config living in UIs or gcloud one-liners that nobody can review.

Cloud Run jobs give you a container that runs to completion with up to 10,000 sharded tasks, per-task retries and timeouts, and per-second compute billing that stops when the task exits — while `google_cloud_scheduler_job` keeps the cron definition, timezone, and auth in version-controlled Terraform. Secrets come from Secret Manager via env injection (see [GCP Secret Manager Terraform Module with Automatic Rotation](/tutorial/gcp-secret-manager-terraform-module-automatic-rotation)), and if you already use Scheduler with Pub/Sub for event-driven starts (see [Event-Driven VM Auto-Scheduling on GCP](/tutorial/event-driven-vm-auto-scheduling)), the HTTP-target pattern here is its direct sibling.

Prerequisites

• GCP project with billing enabled and owner/editor-level IAM to grant roles
• gcloud CLI v480.0+ installed and authenticated (gcloud auth login)
• Terraform >= 1.5 with the hashicorp/google provider >= 5.0 (v2 Cloud Run resources)
• A container image for the batch workload pushed to Artifact Registry

Step-by-Step Guide

Step 1: Enable APIs and Create Two Dedicated Service Accounts

Enable the Run, Scheduler, Secret Manager, and Artifact Registry APIs, then create two identities — one the job runs as, one Cloud Scheduler authenticates with. Two accounts, not one, is the whole point of the least-privilege model. The default Compute Engine service account is an Editor on the project; a batch job running as it can read every bucket and secret in the project. A compromised or buggy job should be able to touch exactly its own inputs and outputs, and the scheduler identity should be able to do nothing but start the job.

gcloud services enable \
  run.googleapis.com \
  cloudscheduler.googleapis.com \
  secretmanager.googleapis.com \
  artifactregistry.googleapis.com \
  --project=$PROJECT_ID

# Identity the job's tasks run AS
gcloud iam service-accounts create nightly-etl-runner \
  --display-name="Cloud Run job runtime SA (nightly-etl)" \
  --project=$PROJECT_ID

# Identity Cloud Scheduler authenticates WITH
gcloud iam service-accounts create nightly-etl-scheduler \
  --display-name="Cloud Scheduler invoker SA (nightly-etl)" \
  --project=$PROJECT_ID

Step 2: Define the Cloud Run Job with Tasks, Parallelism, and Retries

Declare the job in Terraform with `google_cloud_run_v2_job` — `task_count` shards the work, `parallelism` caps concurrency, and the inner template sets `max_retries`, the per-task `timeout`, resources, and the runtime service account. These four numbers are the entire reliability and blast-radius contract of the job. parallelism: 5 means your Cloud SQL instance sees at most 5 concurrent task connections even with 10,000 tasks defined; max_retries: 3 absorbs transient 503s without paging anyone; timeout bounds a hung task instead of letting it bill forever.

resource "google_cloud_run_v2_job" "nightly_etl" {
  name     = "nightly-etl"
  location = var.region
  project  = var.project_id

  template {
    task_count  = 20   # 20 shards of work per execution
    parallelism = 5    # never more than 5 running at once

    template {
      timeout         = "3600s" # per-task cap; max is 604800s (7 days)
      max_retries     = 3       # per-task retries; max is 10
      service_account = google_service_account.job_runner.email

      containers {
        image = "${var.region}-docker.pkg.dev/${var.project_id}/batch/nightly-etl:1.3.0"

        resources {
          limits = {
            cpu    = "2"
            memory = "4Gi"
          }
        }

        env {
          name  = "TARGET_BUCKET"
          value = "acme-etl-output"
        }
      }
    }
  }
}

Step 3: Make Tasks Idempotent and Shard by Task Index

The container contract: read CLOUD_RUN_TASK_INDEX / CLOUD_RUN_TASK_COUNT to pick its shard, key every write on an idempotency key derived from the execution name and task index, and exit 0 on success / non-zero only for retryable failures. A retried task re-runs from the start — if your writes aren't idempotent, retries and duplicate scheduler triggers silently double-charge customers or double-insert rows. The execution name is unique per run, so execution-task_index-record_id is a stable dedupe key across retries of the same task but distinct across executions.

import os
import sys
import hashlib

def idem_key(record_id: str) -> str:
    execution = os.environ["CLOUD_RUN_EXECUTION"]   # unique per run
    task_index = os.environ["CLOUD_RUN_TASK_INDEX"] # 0..task_count-1
    raw = f"{execution}-{task_index}-{record_id}"
    return hashlib.sha256(raw.encode()).hexdigest()[:32]

def main() -> int:
    task_index = int(os.environ["CLOUD_RUN_TASK_INDEX"])
    task_count = int(os.environ["CLOUD_RUN_TASK_COUNT"])

    # Each task processes only records where hash % task_count == task_index
    for record in load_records():
        if shard_of(record.id) % task_count != task_index:
            continue
        upsert(record, dedupe_key=idem_key(record.id))  # safe to replay
    return 0  # 0 = success; any non-zero exit triggers a task retry

if __name__ == "__main__":
    try:
        sys.exit(main())
    except TransientUpstreamError as e:
        print(f"retryable: {e}", file=sys.stderr)
        sys.exit(1)   # -> task retry, up to max_retries
    except Exception as e:
        print(f"fatal: {e}", file=sys.stderr)
        sys.exit(1)

Step 4: Inject Secrets from Secret Manager at Runtime

Store the job's API key in Secret Manager, grant the job's runtime SA secretAccessor on that one secret, and mount it as an env var via value_source.secret_key_ref inside the job template. Plaintext env vars in Terraform state and the job spec are readable by anyone with run.jobs.get. Secret Manager gives you versioned, access-logged secrets where the value never appears in state beyond the secret_version resource — and rotation is a new version, not a redeploy (with version = "latest").

resource "google_secret_manager_secret" "etl_api_key" {
  secret_id = "nightly-etl-api-key"
  project   = var.project_id

  replication {
    auto {}
  }
}

resource "google_secret_manager_secret_version" "etl_api_key" {
  secret      = google_secret_manager_secret.etl_api_key.id
  secret_data = var.etl_api_key # passed via TF_VAR_etl_api_key
}

resource "google_secret_manager_secret_iam_member" "job_accessor" {
  project   = var.project_id
  secret_id = google_secret_manager_secret.etl_api_key.secret_id
  role      = "roles/secretmanager.secretAccessor"
  member    = "serviceAccount:${google_service_account.job_runner.email}"
}

# Inside google_cloud_run_v2_job.nightly_etl -> containers block:
# env {
#   name = "EXTERNAL_API_KEY"
#   value_source {
#     secret_key_ref {
#       secret  = google_secret_manager_secret.etl_api_key.secret_id
#       version = "latest"
#     }
#   }
# }

Step 5: Grant Least-Privilege IAM — jobs.run Is Not routes.invoke

Bind the scheduler service account to roles/run.invoker on the job resource itself, and grant the runtime SA only the data-plane roles its workload needs (bucket access, secretAccessor from step 4). The role name is identical for jobs and services but the permission differs: on a job, roles/run.invoker grants run.jobs.run; on a service it grants run.routes.invoke. Granting at project level gives the scheduler identity the ability to invoke every service and run every job in the project — resource-level bindings keep one leaked OAuth token scoped to one job.

# Scheduler identity: can ONLY start this one job
resource "google_cloud_run_v2_job_iam_member" "scheduler_invoker" {
  project  = var.project_id
  location = var.region
  name     = google_cloud_run_v2_job.nightly_etl.name
  role     = "roles/run.invoker"
  member   = "serviceAccount:${google_service_account.scheduler_invoker.email}"
}

# Runtime identity: only the data access the workload needs
resource "google_storage_bucket_iam_member" "etl_output_writer" {
  bucket = "acme-etl-output"
  role   = "roles/storage.objectUser"
  member = "serviceAccount:${google_service_account.job_runner.email}"
}

Step 6: Create the Cloud Scheduler Cron Trigger with an OAuth Token

Define `google_cloud_scheduler_job` with an HTTP target pointing at the Run Admin API `:run` endpoint, an explicit timezone, and an `oauth_token` block — not `oidc_token` — because the target is a Google API, not a Cloud Run service URL. OIDC tokens carry an audience claim that Cloud Run services validate; the Run Admin API (run.googleapis.com) validates OAuth2 access tokens with the cloud-platform scope. Sending OIDC to the Admin API is the single most common cause of 401s in this architecture, and the two blocks look almost identical in Terraform.

resource "google_cloud_scheduler_job" "nightly_etl_trigger" {
  name             = "nightly-etl-trigger"
  project          = var.project_id
  region           = var.region
  schedule         = "0 2 * * *"       # every night at 02:00
  time_zone        = "Europe/Warsaw"   # explicit — gcloud default is Etc/UTC
  attempt_deadline = "180s"            # :run call itself is fast
  paused           = false

  http_target {
    http_method = "POST"
    uri = "https://run.googleapis.com/v2/projects/${var.project_id}/locations/${var.region}/jobs/${google_cloud_run_v2_job.nightly_etl.name}:run"

    # OAuth, not OIDC: the Run Admin API is a Google API
    oauth_token {
      service_account_email = google_service_account.scheduler_invoker.email
      scope                 = "https://www.googleapis.com/auth/cloud-platform"
    }

    # Empty body = run with the job's configured template
    body = base64encode("{}")
  }
}

Step 7: Execute On Demand and Inspect Execution Status

Before trusting the cron, run the job manually with `--wait` and inspect the resulting execution's task-level status — succeeded, failed, retried counts per execution are the ground truth for whether your exit-code contract works. A job that "starts" tells you nothing; a task that fails 3 times and exhausts retries still shows an execution resource that exists. Checking succeededCount vs taskCount is the difference between "triggered" and "done", and it's the same check your alerting will automate in step 8.

# Run now, block until completion
gcloud run jobs execute nightly-etl \
  --region=$REGION --project=$PROJECT_ID --wait

# Recent executions
gcloud run jobs executions list \
  --job=nightly-etl --region=$REGION --limit=5

# Task-level detail on the latest execution
gcloud run jobs executions describe \
  $(gcloud run jobs executions list --job=nightly-etl \
    --region=$REGION --limit=1 --format="value(metadata.name)") \
  --region=$REGION \
  --format="yaml(status.succeededCount,status.failedCount,status.retriedCount,status.completionTime)"

Step 8: Alert on Failed Executions with a Log-Based Metric

Create a logs-based metric that counts failed job executions, then a Cloud Monitoring alert policy that fires when the count is non-zero — this pages on batch outcomes, not on individual ERROR log lines that a successful retry would make irrelevant. Batch workloads are supposed to retry. Alerting on severity>=ERROR raw logs pages you for the transient 503 that retry #2 absorbed; alerting on the execution's terminal failure state pages you only when retries were exhausted and the night's batch actually did not happen.

resource "google_logging_metric" "job_failed_executions" {
  name    = "cloud_run_job_failed_executions"
  project = var.project_id

  filter = <<-EOT
    resource.type="cloud_run_job"
    resource.labels.job_name="nightly-etl"
    resource.labels.location="${var.region}"
    severity=ERROR
    jsonPayload.message=~"Execution .* has failed"
  EOT

  metric_descriptor {
    metric_kind = "DELTA"
    value_type  = "INT64"
    unit        = "1"
  }
}

resource "google_monitoring_alert_policy" "nightly_etl_failure" {
  display_name = "nightly-etl: execution failed after retries"
  project      = var.project_id
  combiner     = "OR"

  conditions {
    display_name = "Failed executions > 0 in 10m"
    condition_threshold {
      filter          = "resource.type=\"cloud_run_job\" AND metric.type=\"logging.googleapis.com/user/cloud_run_job_failed_executions\""
      comparison      = "COMPARISON_GT"
      threshold_value = 0
      duration        = "0s"

      aggregations {
        alignment_period   = "600s"
        per_series_aligner = "ALIGN_SUM"
      }
    }
  }

  notification_channels = [google_monitoring_notification_channel.oncall.id]
}

Step 9: Force-Run the Schedule and Verify End to End

Trigger the Cloud Scheduler job manually — bypassing the clock — and trace the full chain: scheduler delivery status, new execution created, tasks succeeded, output rows deduplicated on a second force-run. terraform apply succeeding proves resources exist, not that auth, URI, and IAM line up. The force-run exercises exactly the path the 02:00 cron will take — same OAuth token, same SA, same endpoint — so a green force-run is a real production readiness signal, not a proxy for one.

# Trigger the cron now, as Cloud Scheduler would
gcloud scheduler jobs run nightly-etl-trigger \
  --location=$REGION --project=$PROJECT_ID

# Scheduler's own view of the attempt
gcloud scheduler jobs describe nightly-etl-trigger \
  --location=$REGION \
  --format="yaml(status.lastAttemptTime,status.status.code)"

# Confirm a new execution appeared and finished clean
gcloud run jobs executions list \
  --job=nightly-etl --region=$REGION --limit=1 \
  --format="table(metadata.name,status.completionTime,status.succeededCount,status.failedCount)"

# Force-run again — row counts in the output must NOT double
gcloud scheduler jobs run nightly-etl-trigger --location=$REGION

Verification & Health Check

Best Practices

• Idempotency Keys on Every Side Effect
• Reserve Non-Zero Exits for Retryable Failures
• Cap Parallelism Against Downstream Limits
• One Service Account per Identity, Never the Default

Common Mistakes

• {"errorCode":"HTTP_401_UNAUTHENTICATED (Scheduler -> jobs:run)","symptoms":"Cloud Scheduler job shows status.code: 16 / UNAUTHENTICATED after every attempt; no executions ever appear on the Cloud Run job.","rootCause":"The http_target uses an oidc_token block (or gcloud --oidc-service-account-email) against the Run Admin API. The Admin API expects an OAuth2 access token, not an audience-bound OIDC ID token — OIDC is for invoking Cloud Run service URLs.","fixCommand":"gcloud scheduler jobs update http nightly-etl-trigger \\\n --location=$REGION \\\n --oauth-service-account-email=nightly-etl-scheduler@$PROJECT_ID.iam.gserviceaccount.com\n","code":"http_target {\n http_method = \"POST\"\n uri = \"https://run.googleapis.com/v2/projects/${var.project_id}/locations/${var.region}/jobs/nightly-etl:run\"\n\n oauth_token { # NOT oidc_token\n service_account_email = google_service_account.scheduler_invoker.email\n scope = \"https://www.googleapis.com/auth/cloud-platform\"\n }\n}\n","language":"hcl","filename":"fix-scheduler-auth.tf","prevention":"Template the scheduler job as a reusable module that only exposes oauth_token, and unit-test the module — the two auth blocks differ by one word."}
• {"errorCode":"HTTP_403_PERMISSION_DENIED (run.jobs.run)","symptoms":"Scheduler attempts return 403; Cloud Logging shows Permission 'run.jobs.run' denied on resource 'projects/.../jobs/nightly-etl'.","rootCause":"The scheduler service account lacks roles/run.invoker on the job — either the binding was never created, was placed on a same-named service, or was granted on a different project/region than the job lives in.","fixCommand":"gcloud run jobs add-iam-policy-binding nightly-etl \\\n --region=$REGION --project=$PROJECT_ID \\\n --member=\"serviceAccount:nightly-etl-scheduler@$PROJECT_ID.iam.gserviceaccount.com\" \\\n --role=\"roles/run.invoker\"\n","code":"resource \"google_cloud_run_v2_job_iam_member\" \"scheduler_invoker\" {\n project = var.project_id\n location = var.region\n name = google_cloud_run_v2_job.nightly_etl.name\n role = \"roles/run.invoker\"\n member = \"serviceAccount:${google_service_account.scheduler_invoker.email}\"\n}\n","language":"hcl","filename":"fix-iam.tf","prevention":"Keep the IAM binding in the same Terraform file as the scheduler job so terraform plan shows both resources together, and run gcloud run jobs get-iam-policy in CI after apply."}
• {"errorCode":"TASK_TIMEOUT / DEADLINE_EXCEEDED","symptoms":"Executions end with tasks in state Failed and log lines Task was killed: reached its timeout; longer shards always die at exactly the same wall-clock mark.","rootCause":"Work per shard grew with data volume until it exceeded the job's timeout (default 600s in many templates). The timeout is per task attempt, not per execution, so only the slowest shards fail — making it look intermittent.","fixCommand":"gcloud run jobs update nightly-etl \\\n --region=$REGION --task-timeout=3600 \\\n --tasks=40 # more tasks -> smaller shards -> shorter per-task runtime\n","code":"template {\n task_count = 40 # was 20 — halves shard size\n parallelism = 5\n template {\n timeout = \"3600s\" # per-task; platform max is 604800s (7 days)\n max_retries = 3\n }\n}\n","language":"hcl","filename":"fix-timeout.tf","prevention":"Track p95 task duration as a custom metric and alert when it crosses 50% of the configured timeout — data growth makes today's headroom next quarter's outage."}
• {"errorCode":"SECRET_PERMISSION_DENIED at Task Startup","symptoms":"Tasks fail within seconds of starting; logs show Permission 'secretmanager.versions.access' denied for secret projects/.../secrets/nightly-etl-api-key.","rootCause":"The secretAccessor binding was granted to the scheduler invoker SA (or the default compute SA) instead of the job's runtime service_account — secret resolution happens as the runtime identity when the task starts.","fixCommand":"gcloud secrets add-iam-policy-binding nightly-etl-api-key \\\n --project=$PROJECT_ID \\\n --member=\"serviceAccount:nightly-etl-runner@$PROJECT_ID.iam.gserviceaccount.com\" \\\n --role=\"roles/secretmanager.secretAccessor\"\n","code":"resource \"google_secret_manager_secret_iam_member\" \"job_accessor\" {\n project = var.project_id\n secret_id = google_secret_manager_secret.etl_api_key.secret_id\n role = \"roles/secretmanager.secretAccessor\"\n member = \"serviceAccount:${google_service_account.job_runner.email}\" # runtime SA, not scheduler SA\n}\n","language":"hcl","filename":"fix-secret-iam.tf","prevention":"Reference the SA resource (not a hardcoded email) in every IAM binding so Terraform's graph enforces creation order and renames propagate everywhere."}

Cost Analysis

References

• Create and execute Cloud Run jobs
• Execute Cloud Run jobs on a schedule with Cloud Scheduler
• Cloud Run quotas and limits (tasks, retries, timeouts)
• Cloud Run pricing (jobs compute, free tier)
• Terraform google_cloud_run_v2_job resource
• Terraform google_cloud_scheduler_job resource