gke-backup-dr · diff

v1.0.0 to v1.0.0

32 added, 36 removed. Audit A to A.

---

name: gke-backup-dr

description: >-

Configures Backup for GKE: the BackupRestore cluster addon, BackupPlan and

RestorePlan resources, restore workflows, and CMEK-encrypted backups. Use

for backup policies, disaster recovery, or GKE cluster restores. Don't use

for database backups.

metadata:

version: "1.0.0"

category: Storage

---

# GKE Backup & Disaster Recovery

Protects stateful GKE workloads using Backup for GKE. Backup for GKE can capture

both Kubernetes resource metadata (manifests, configurations, and secrets) and

the underlying persistent volume (PV) data — but volume data and secrets are

**only** captured when the backup plan explicitly enables them (see the flags

below).

## CLI Reference

- ```bash

- # Enable the BackupRestore addon (Slow cluster-level update)

- gcloud container clusters update {cluster_name} \

- --update-addons=BackupRestore=ENABLED --location={location} --quiet

+ > A change goes out as a pull request to the GitOps repository, never as a write from here: Kubernetes objects as YAML through `gke-manifest-generation`, cluster settings and other Google Cloud resources as Config Connector YAML through `gcp-config-connector`, opened with `submit-suggestion`. Each profile's config offers only the `gke` MCP server's read tools, and the broker refuses `kubectl` and `gcloud` writes.

- # Create Backup Plan

- gcloud beta container backup-restore backup-plans create {plan_name} \

- --project={project_id} --location={location} \

- --cluster=projects/{project_id}/locations/{location}/clusters/{cluster_name} \

- --all-namespaces \

- --include-volume-data --include-secrets \

- --backup-retain-days={days} --cron-schedule="{cron}" --quiet

+ Each change below is proposed as Config Connector YAML through `gcp-config-connector`:

- # Trigger Manual Backup

- gcloud beta container backup-restore backups create {backup_name} \

- --backup-plan={plan_name} --location={location} --quiet

+ - **BackupRestore addon** (slow cluster-level update):

+ `spec.addonsConfig.gkeBackupAgentConfig.enabled: true` on the cluster's

+ `ContainerCluster`.

+ - **Backup plan** `{plan_name}` for

+ `projects/{project_id}/locations/{location}/clusters/{cluster_name}`: all

+ namespaces, volume data and secrets included, retained `{days}` days, on

+ the cron schedule `{cron}`.

+ - **Restore plan** `{restore_plan_name}` targeting

+ `projects/{project_id}/locations/{location}/clusters/{target_cluster_name}`

+ from the backup plan `{source_backup_plan_name}`: all namespaces, volume data

+ restored from the backup, the cluster-resource conflict policy that keeps

+ existing versions, and the namespaced-resource restore mode that fails on

+ conflict (the safe defaults in Restore Safety below).

+ - **A manual backup** `{backup_name}` and **a restore** `{restore_name}` of it

+ also go through `gcp-config-connector`; a restore needs the confirmation in

+ Restore Safety below.

- # Create Restore Plan

- gcloud beta container backup-restore restore-plans create {restore_plan_name} \

- --location={location} \

- --cluster=projects/{project_id}/locations/{location}/clusters/{target_cluster_name} \

- --backup-plan=projects/{project_id}/locations/{location}/backupPlans/{source_backup_plan_name} \

- --all-namespaces \

- --volume-data-restore-policy=restore-volume-data-from-backup \

- --cluster-resource-conflict-policy=use-existing-version \

- --namespaced-resource-restore-mode=fail-on-conflict --quiet

+ The backup plan, restore plan, backup and restore are Backup for GKE resources,

+ not `ContainerCluster` fields. If `gcp-config-connector` has no resource for one

+ of them, it has no declarative path in a Config Connector repository, and the

+ agent blocks rather than inventing a field.

- # Execute Restore

- gcloud beta container backup-restore restores create {restore_name} \

- --restore-plan={restore_plan_name} --location={location} \

- --backup=projects/{project_id}/locations/{location}/backupPlans/{source_backup_plan_name}/backups/{backup_name} \

- --quiet

+ Verify the restore status:

- # Verify Restore Status

+ ```bash

gcloud beta container backup-restore restores describe {restore_name} \

--restore-plan={restore_plan_name} --location={location}

```

- > [!WARNING] **`--include-volume-data` and `--include-secrets` BOTH DEFAULT TO

- > FALSE.** If you omit them, the backup plan silently produces **config-only

- > backups** with no persistent volume snapshots and no Secrets. Always pass both

- > flags explicitly when the goal is full workload protection.

+ > [!WARNING] **Volume data inclusion (`--include-volume-data`) and secret

+ > inclusion (`--include-secrets`) BOTH DEFAULT TO FALSE.** If you omit them, the

+ > backup plan silently produces **config-only backups** with no persistent volume

+ > snapshots and no Secrets. Always enable both explicitly in the backup plan when

+ > the goal is full workload protection.

Notes:

- The `backup-restore` command group is on the `gcloud beta` track

(`gcloud beta container backup-restore ...`), which this image's gcloud

already includes. Do not run `gcloud components install`: the credential

proxy refuses it.

- `--cluster` requires the full resource path

`projects/{project_id}/locations/{location}/clusters/{cluster_name}` (or

`projects/{project_id}/zones/{zone}/clusters/{cluster_name}` for zonal

clusters), not a bare cluster name.

- Restore plans require exactly one namespaced-resource scope flag:

`--all-namespaces`, `--selected-namespaces={ns1},{ns2}`,

`--excluded-namespaces=...`, `--selected-applications=...`, or

`--no-namespaces`.

## Restore Safety (CRITICAL)

A restore writes into a **live cluster** and, depending on the conflict policy,

can overwrite or delete existing resources:

- `--cluster-resource-conflict-policy=use-existing-version` keeps existing

cluster-scoped resources (safe default); `use-backup-version` **deletes**

the existing version first — deleting a CRD deletes all of its CRs.

- `--namespaced-resource-restore-mode=fail-on-conflict` aborts on any conflict

(safe default); `merge-skip-on-conflict` skips conflicting resources;

`merge-replace-on-conflict` and `merge-replace-volume-on-conflict`

**overwrite** existing resources or volumes; `delete-and-restore` **deletes

entire conflicting namespaces** (and all resources in them) before

restoring.

**Rules:**

1. Validate the restore in a non-production target cluster first.

2. Prefer the safe defaults (`use-existing-version` + `fail-on-conflict`)

unless the user explicitly needs to revert live resources.

3. **Always obtain explicit user confirmation before executing a restore into a

production cluster**, and state which conflict policy is in effect and what

it may overwrite or delete.

## Best Practices

1. **CMEK Encryption**: Encrypt backup plans using Customer-Managed Encryption

Keys:

`--encryption-key=projects/{project_id}/locations/{location}/keyRings/{ring}/cryptoKeys/{key}`.

2. **Scope**: Prefer backing up specific namespaces rather than the entire

cluster: `--selected-namespaces={ns1},{ns2}` (instead of

`--all-namespaces`).

3. **Application Consistency**: Recommend quiescing the database or pausing

application writes (e.g. using pre-backup hooks or database-specific tools)

prior to backups to ensure data integrity.

4. **CSI Volume Snapshots**: Ensure that stateful backups utilize GKE's CSI

(Container Storage Interface) driver for volume snapshots to capture

persistent volume data.

5. **Service Terminology**: Always explicitly refer to the service as **Backup

for GKE** in your response. This distinguishes it from the broader (but

complementary) Google Cloud **Backup and Disaster Recovery (DR)

Service**, as **Backup for GKE** is built specifically for GKE.

## Golden Path Backup Defaults

The recommended production golden path configuration for Backup for GKE:

- **Addon**: BackupRestore addon enabled

(`--update-addons=BackupRestore=ENABLED`).

- **Volume Inclusion**: `--include-volume-data` explicitly passed (enabled,

since the service default is false).

- **Secret Inclusion**: `--include-secrets` explicitly passed (enabled, since

the service default is false).

- **Retention**: Defined retention period (e.g. 30 days via

`--backup-retain-days=30`).

- **Encryption**: CMEK enabled (`--encryption-key=...`).

## Recent Changes

- **Cross-project backup and restore (GA)**: Backup plans can store backups in

a different project than the source cluster, and restore plans can target

clusters in a third project. Enables centralized backup projects (with

immutability/retention managed by a platform team) and cross-project

environment seeding without granting access to the source project.

- **Pricing change (effective 2026-03-02)**: The backup management fee moved

from **pod-based** to **NAMESPACE-based** pricing — charged per non-system

namespace in the most recent successful backup of each plan (system

namespaces like `kube-system` are excluded). Existing committed use discount

(CUD) holders keep pod-based management pricing until their commitment ends;

everyone else moves to the new model. See

https://cloud.google.com/products/backup-for-gke/pricing-changes.

- **Smart Scheduling**: RPO-driven backup scheduling as an alternative to

fixed cron schedules — pass `--target-rpo-minutes={minutes}` instead of

`--cron-schedule` when creating the backup plan (optionally with RPO

exclusion windows via `--exclusion-windows-file`).

- **Hyperdisk support**: Backup and restore of **Hyperdisk ML** and

**Hyperdisk Balanced High Availability** volumes is supported on GKE

clusters running **1.33.1-gke.1959000 and later** (Hyperdisk throughput,

extreme, and balanced types are also supported).

## Troubleshooting & Common Pitfalls (CRITICAL)

> [!IMPORTANT] **Slow Operations**: Enabling the BackupRestore addon

> (`--update-addons=BackupRestore=ENABLED`) triggers a slow Google Cloud control

> plane cluster update that takes several minutes. * **Rule**: **Do not run a

> terminal loop waiting for the GKE Backup addon to become active.** *

- > **Action**: Provide the command to enable the addon, explain that the

- > operation will proceed in the background, and immediately proceed to write the

- > backup plan configs. Do not block.

+ > **Action**: Propose the addon change as a pull request, explain that the

+ > operation will proceed in the background once it applies, and immediately

+ > proceed to write the backup plan configs. Do not block.