gke-workload-scaling git:20261019.895f23dA

Immutable. This exact content never changes and is served at /api/v1/blob/c2ae9aa8fa8822d3.

---
name: gke-workload-scaling
description: >-
  Manages scaling for GKE workloads using HPA and VPA. Use when configuring
  Horizontal Pod Autoscaler (HPA), configuring Vertical Pod Autoscaler (VPA),
  or applying best practices for GKE workload autoscaling. Do not use for
  cluster-level autoscaling (Cluster Autoscaler), static cluster sizing,
  or configuring node-level machine styles directly.
metadata:
  category: Containers
---

# GKE Workload Scaling

This skill provides workflows and best practices for scaling applications on
Google Kubernetes Engine (GKE). It covers manual scaling, Horizontal Pod
Autoscaling (HPA), and Vertical Pod Autoscaling (VPA).

> A change goes out as a pull request to the GitOps repository, never as a write from here: Kubernetes objects as YAML through `gke-manifest-generation`, cluster settings and other Google Cloud resources as Config Connector YAML through `gcp-config-connector`, opened with `submit-suggestion`. Each profile's config offers only the `gke` MCP server's read tools, and the broker refuses `kubectl` and `gcloud` writes.

## Workflows

### 1. Manual Scaling

Scale a deployment to a fixed number of replicas. Useful for immediate manual
intervention or testing. Set `spec.replicas` in the Deployment's manifest and
propose it as a pull request through `submit-suggestion`:

```yaml
spec:
  replicas: {number}
```

Once it applies, verify the scale event:

```bash
kubectl get deployment {deployment_name} -n {namespace}
```

### 2. Horizontal Pod Autoscaling (HPA)

Automatically scale the number of pods based on observed CPU utilization, memory
utilization, or custom metrics.

**Prerequisites:**

-   Metrics Server must be running (enabled by default on GKE).
-   Containers clearly define resource requests/limits.

**Manifest:** Use a YAML manifest for version-controlled configuration,
proposed as a pull request through `submit-suggestion`. See
[assets/hpa-example.yaml](assets/hpa-example.yaml) for a template (for example
target CPU utilization 50%, `minReplicas: 1`, `maxReplicas: 10`). Once it
applies, verify the HPA is created and fetching metrics:

```bash
kubectl get hpa
```

**Custom Metrics & External Metrics:** For GKE, the modern and recommended
approach for scaling based on Cloud Monitoring metrics (e.g., Pub/Sub queue
length) is to use the **External** metric type, which is natively supported by
the GKE control plane without requiring the Custom Metrics Adapter. For
application-specific metrics exposed via Prometheus, you can use **Google Cloud
Managed Service for Prometheus** or the Prometheus Adapter.

### 3. Vertical Pod Autoscaling (VPA)

Automatically adjust the CPU and memory reservations for your pods to match
actual usage. This is critical for right-sizing workloads.

**Prerequisites:**

-   VPA must be enabled on the cluster.
    -   **Autopilot:** Enabled by default.
    -   **Standard:** Must be enabled manually.

**Enable VPA on Standard Cluster:** `spec.verticalPodAutoscaling.enabled: true` on
the cluster's `ContainerCluster`, changed as Config Connector YAML through
`gcp-config-connector`.

**Update Modes:**

-   `Off`: Calculates recommendations but does not apply them. Good for "dry
    run" analysis.
-   `Initial`: Assigns resources only at pod creation time.
-   `Auto`: Updates running pods by restarting them if recommendations differ
    significantly from requests.
-   `InPlaceOrRecreate`: Attempts to update Pod resources without recreating the
    Pod. If in-place update is not possible, it reverts to `Auto` mode (requires
    GKE 1.34+).

**Example:** See [assets/vpa-example.yaml](assets/vpa-example.yaml) for a
configuration template, proposed as a pull request through `submit-suggestion`.

## Best Practices

1.  **Define Resource Requests:** HPA and VPA rely on accurate resource
    requests. Always define them in your container specs.
2.  **Avoid Metric Conflicts:** Do not configure HPA and VPA to use the same
    metric (e.g., both CPU). This causes thrashing.
    -   *Typical Pattern:* HPA on CPU, VPA on Memory.
3.  **Pod Disruption Budgets (PDBs):** Define PDBs to ensure application
    availability during scaling events or node upgrades.
4.  **HPA Lag:** HPA has a stabilization window (default 5 mins) to prevent
    rapid fluctuation.
5.  **VPA "Auto" Mode Risks:** In "Auto" mode, VPA restarts pods to change
    resources. Ensure your application handles restarts gracefully (e.g.,
    handles SIGTERM).
    -   *Note:* By default, VPA requires at least 2 replicas to perform
        evictions (to prevent a situation where the only running replica is
        evicted, causing downtime). In GKE 1.22+, you can override this by
        setting `minReplicas` in `PodUpdatePolicy`.

## Rightsizing Workflow

1.  Propose VPA in `Off` mode as a pull request and let it run for 24+ hours
2.  Read recommendations: `kubectl describe vpa {deployment_name}-vpa -n
    {namespace}`
3.  Compare `target` values against current `requests`
4.  Apply with 20% buffer: `new_request = target * 1.2`
5.  Update the deployment manifest with the new resource requests and propose
    it as a pull request

Condition                     | Recommendation                       | Risk
----------------------------- | ------------------------------------ | ------
CPU request >5x P95 actual    | Reduce to `P95 * 1.2`                | Medium
Memory request >3x P95 actual | Reduce to `P95 * 1.2`                | Medium
CPU request >2x P95 actual    | Rightsizing with 20% buffer          | Low
No resource limits set        | Add limits to prevent noisy-neighbor | Low

<!-- kube-agents: local addition (auto-injected by sync-upstream-skills.py) -->

## Before recommending GPU/TPU or large-shape capacity for a scale-up

Before recommending capacity for a GPU/TPU or large-shape scale-up, load the
[capacity-obtainability](../capacity-obtainability/SKILL.md) skill and run its diagnostics: the
regional quota for the exact accelerator metric, then live obtainability advice for the requested
shape across zones and provisioning models — and, for a deadline-bound batch scale-up, its
**Future windows** section (`gcloud beta compute advice calendar-mode`). That skill owns what to
probe and how to report it; follow it rather than restating it here.