gke-workload-security ยท diff
git:20260908.e4edc3b to git:20261014.72456d6
68 added, 68 removed. Audit A to A.
---
name: gke-workload-security
description: Workflows for auditing and hardening the security of GKE workloads.
---
# GKE Workload Security
This skill provides workflows and best practices for securing GKE workloads. It
covers security auditing, Identity and Access Management (Workload Identity),
Network Security (Network Policies), and Node Security.
+ > You are read-only: each change below is a proposal you report with your findings, never a command you run.
+
## Workflows
### 1. Security Audit
Assess the current security posture of your cluster using the provided audit
script.
**Capabilities:**
- Checks for Workload Identity.
- Verifies Network Policy is enabled.
- Checks if Shielded Nodes are enabled.
- Checks if Binary Authorization is enabled.
- Checks for Private Cluster configuration.
**Command:**
```bash
./scripts/audit_cluster.sh <cluster-name> <region> <project-id>
```
### 2. Configure Workload Identity
Workload Identity allows Kubernetes Service Accounts (KSAs) to impersonate
Google Service Accounts (GSAs). This is the recommended method for workloads to
access Google Cloud APIs.
**Steps:**
- 1. **Create Namespace and KSA:**
-
- ```bash
- kubectl create namespace workload-identity-test-ns
- kubectl create serviceaccount <ksa-name> \
- --namespace workload-identity-test-ns
- ```
+ Each step is a change you propose.
- 2. **Bind KSA to GSA:**
+ 1. **Namespace and KSA:** propose the Namespace and the KSA:
- ```bash
- gcloud iam service-accounts add-iam-policy-binding <gsa-name>@<project-id>.iam.gserviceaccount.com \
- --role roles/iam.workloadIdentityUser \
- --member "serviceAccount:<project-id>.svc.id.goog[workload-identity-test-ns/<ksa-name>]"
+ ```yaml
+ apiVersion: v1
+ kind: Namespace
+ metadata:
+ name: workload-identity-test-ns
+ ---
+ apiVersion: v1
+ kind: ServiceAccount
+ metadata:
+ name: <ksa-name>
+ namespace: workload-identity-test-ns
+ annotations:
+ iam.gke.io/gcp-service-account: <gsa-name>@<project-id>.iam.gserviceaccount.com
```
- 3. **Annotate KSA:**
-
- ```bash
- kubectl annotate serviceaccount <ksa-name> \
- --namespace workload-identity-test-ns \
- iam.gke.io/gcp-service-account=<gsa-name>@<project-id>.iam.gserviceaccount.com
- ```
+ 2. **Bind KSA to GSA:** propose granting `roles/iam.workloadIdentityUser` on
+ the GSA `<gsa-name>@<project-id>.iam.gserviceaccount.com` to the member
+ `serviceAccount:<project-id>.svc.id.goog[workload-identity-test-ns/<ksa-name>]`.
- 4. **Verify Example Pod:**
- Use existing asset `assets/workload-identity-pod.yaml` to test the
- configuration. Update the `<ksa-name>` in the file first.
+ 3. **Annotate KSA:** the `iam.gke.io/gcp-service-account` annotation in the
+ manifest above links the KSA to the GSA.
- ```bash
- kubectl apply -f ./assets/workload-identity-pod.yaml -n workload-identity-test-ns
- ```
+ 4. **Example Pod:** propose a test Pod in `workload-identity-test-ns` with
+ `serviceAccountName: <ksa-name>` and the node selector
+ `iam.gke.io/gke-metadata-server-enabled: "true"`. A complete example is in
+ `assets/workload-identity-pod.yaml`.
### 3. Implement Network Policies
Control traffic flow between Pods using Network Policies. By default, all
traffic is allowed.
**Check Network Policy Enforcement & Dataplane:**
Before modifying cluster networking, inspect whether NetworkPolicy enforcement
is already active or provided natively by Dataplane V2:
```bash
gcloud container clusters describe <cluster-name> \
--location <location> \
--format='value(networkConfig.datapathProvider,networkPolicy.enabled)'
```
- If `datapathProvider` is `ADVANCED_DATAPATH` (Dataplane V2), NetworkPolicy
enforcement is built-in natively via eBPF/Cilium from cluster creation. Calico
addons cannot be enabled and are not needed.
- If `networkPolicy.enabled` is `True`, Calico enforcement is already enabled on nodes.
- - If neither is active, enable Calico network policy enforcement using the two-step
- sequence below.
-
- **Enable Network Policy Enforcement (non-DPv2 clusters):**
-
- Enabling network policy enforcement on clusters without Dataplane V2 requires
- two sequential commands in this order: first enable the Calico addon on the
- control plane, then enable network policy enforcement on the nodes. GKE rejects
- `--enable-network-policy` with HTTP 400 until the addon is enabled, and `gcloud`
- rejects both flags in a single invocation.
+ - If neither is active, propose enabling Calico network policy enforcement in
+ the two steps below.
- ```bash
- # Step 1: Enable the NetworkPolicy addon on the control plane
- gcloud container clusters update <cluster-name> \
- --update-addons=NetworkPolicy=ENABLED \
- --region <region>
+ **Network Policy Enforcement (non-DPv2 clusters):**
- # Step 2: Enable NetworkPolicy enforcement on the nodes (node pools may be recreated; this can take several minutes)
- gcloud container clusters update <cluster-name> \
- --enable-network-policy \
- --region <region>
- ```
+ On clusters without Dataplane V2 this is two sequential changes, in this order:
+ first enable the NetworkPolicy addon on the control plane (Config Connector
+ `ContainerCluster` field `spec.addonsConfig.networkPolicyConfig.disabled: false`,
+ with `spec.networkPolicy.enabled: false` alongside it, since the addon setting
+ does nothing without a `networkPolicy` block), then enable network policy
+ enforcement on the nodes (`spec.networkPolicy.enabled: true`, `provider: CALICO`).
+ GKE rejects node enforcement with HTTP 400 until the addon is enabled, and
+ node pools may be recreated, which can take several minutes. Propose both, with
+ that order stated.
- **Apply Default Deny Policy:**
+ **Default Deny Policy:**
Isolate namespaces by denying all ingress and egress traffic by default.
- **Replace <target-namespace> with the namespace you want to isolate.**
+ Propose this manifest, with `<target-namespace>` set to the namespace to
+ isolate. [assets/default-deny-netpol.yaml](assets/default-deny-netpol.yaml)
+ holds the same policy without the `metadata.namespace` line.
- ```bash
- kubectl apply -f ./assets/default-deny-netpol.yaml -n <target-namespace>
+ ```yaml
+ apiVersion: networking.k8s.io/v1
+ kind: NetworkPolicy
+ metadata:
+ name: default-deny-all
+ namespace: <target-namespace>
+ spec:
+ podSelector: {}
+ policyTypes:
+ - Ingress
+ - Egress
```
### 4. Enable Shielded Nodes
Ensure nodes are running with verifiable integrity.
- **Command:**
-
- ```bash
- gcloud container clusters update <cluster-name> \
- --enable-shielded-nodes \
- --region <region>
- ```
+ Propose enabling Shielded Nodes (Config Connector `ContainerCluster` field
+ `spec.enableShieldedNodes: true`).
### 5. GKE Sandbox (gVisor)
Run untrusted workloads in a sandbox for extra isolation.
- **Enable GKE Sandbox:**
-
- ```bash
- gcloud container clusters update <cluster-name> \
- --enable-gke-sandbox \
- --region <region>
- ```
+ **GKE Sandbox:**
+ Propose a node pool with GKE Sandbox enabled (Config Connector
+ `ContainerNodePool` field `spec.nodeConfig.sandboxConfig.sandboxType: gvisor`).
**Run a Sandboxed Pod:**
Add `runtimeClassName: gvisor` to your Pod spec.
### 6. Pod Security Standards
Enforce security policies on namespaces using labels.
**Enforce Restricted Profile:**
+ Propose these labels on the namespace:
- ```bash
- kubectl label --overwrite ns <namespace> \
- pod-security.kubernetes.io/enforce=restricted \
- pod-security.kubernetes.io/enforce-version=latest
+ ```yaml
+ apiVersion: v1
+ kind: Namespace
+ metadata:
+ name: <namespace>
+ labels:
+ pod-security.kubernetes.io/enforce: restricted
+ pod-security.kubernetes.io/enforce-version: latest
```
> [!NOTE]
> Using `latest` ensures you use the policies corresponding to the cluster's current version. You can pin it to a specific version (e.g., `v1.30`) to lock down the namespace to policies of a specific release.
### 7. Secret Manager Integration (CSI Driver)
Mount secrets from Google Cloud Secret Manager directly as volumes in your pods.
**Prerequisites**: Secret Manager CSI driver must be enabled on the cluster.
**Example SecretProviderClass:**
```yaml
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: my-secret-provider
spec:
provider: gcp
parameters:
secrets: |
- resourceName: "projects/<project-id>/secrets/my-secret/versions/latest"
fileName: "my-secret-file"
```
**Example Pod Spec excerpt:**
```yaml
spec:
containers:
- name: my-app
volumeMounts:
- name: secrets-store-inline
mountPath: "/mnt/secrets"
readOnly: true
volumes:
- name: secrets-store-inline
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: "my-secret-provider"
```
### 8. Enable Network Policy Logging
If using GKE Dataplane V2, you can log allowed and denied connections.
**Steps:**
- 1. Configure the `NetworkLogging` custom resource.
+ 1. Propose the `NetworkLogging` custom resource.
**Example NetworkLogging Manifest:**
```yaml
apiVersion: networking.gke.io/v1alpha1
kind: NetworkLogging
metadata:
name: default
spec:
cluster:
allow:
log: true
delegate: true
deny:
log: true
delegate: true
```
This will log connection details to Cloud Logging.
## Best Practices
1. **Least Privilege:** Always use Workload Identity with minimal IAM roles. Avoid using Node default service accounts.
2. **Network Isolation:** Use Network Policies to restrict Pod-to-Pod communication. Enable Network Policy Logging for visibility.
3. **Image Security:** Use Binary Authorization to ensure only trusted images are deployed.
4. **Secret Management**: Use Secret Manager CSI driver instead of default Kubernetes secrets for sensitive data.
5. **Pod Security**: Enforce `baseline` or `restricted` Pod Security Standards on all non-system namespaces.
6. **Policy Enforcement**: Consider using **Policy Controller** (Gatekeeper) to enforce custom security and compliance policies across the cluster.