kubectl-coco

A kubectl plugin to deploy Confidential Containers (CoCo) applications.

kubectl-coco is designed primarily for developers to CoCo-fy their applications and test them with Trustee, the Remote Attestation Solution for CoCo. It's not meant for production deployment of CoCo applications. Read more about CoCo at confidentialcontainers.org.

Overview

kubectl-coco simplifies the process of transforming regular Kubernetes manifests into CoCo-enabled manifests. It automatically handles:

• RuntimeClass Configuration: Sets the appropriate CoCo runtime
• Secrets Management: Converts K8s secrets to sealed secrets for upload to Trustee KBS via kbs populate
• ImagePullSecrets: Handles private registry credentials with automatic Trustee KBS integration
• InitData Generation: Creates aa.toml, cdh.toml, and policy.rego configurations

Features

• ✅ KBS Management: Deploy in-cluster Trustee KBS or register an external instance; upload resources via kbs populate
• ✅ Automatic Secret Conversion: Detects and converts K8s secrets to sealed format; generates a trustee-secrets.yaml for upload via kbs populate
• ✅ ImagePullSecrets Support: Handles private registry credentials with Trustee KBS integration
• ✅ Secure Access Sidecar: Optional mTLS-secured sidecar for status reporting and secure port forwarding (see sidecar/README.md)
• ✅ Multi-Resource Support: Works with Pod, Deployment, StatefulSet, ReplicaSet, Job, DaemonSet
• ✅ InitData Generation: Creates properly formatted and encoded configurations
• ✅ Backup Management: Saves transformed manifests with -coco suffix

Quick Start

1. Install

# Download latest release
OS=$(uname -s | tr '[:upper:]' '[:lower:]')
ARCH=$(uname -m)
if [ "$ARCH" = "x86_64" ]; then ARCH="amd64"; fi
curl -LO "https://github.com/confidential-devhub/cococtl/releases/latest/download/kubectl-coco-${OS}-${ARCH}"

# Install
sudo install -m 0755 kubectl-coco-${OS}-${ARCH} /usr/local/bin/kubectl-coco
sudo ln -sf /usr/local/bin/kubectl-coco /usr/local/bin/kubectl_complete-coco

# Verify
kubectl coco --version

See Installation for detailed options.

2. Initialize

Deploy Trustee and create configuration:

kubectl coco init

This creates ~/.kube/coco-config.toml and deploys Trustee to your cluster.

3. Transform

Use --skip-apply to generate the transformed manifest and secrets file without deploying yet:

kubectl coco apply -f your-app.yaml --skip-apply

4. Upload Secrets to KBS

Secrets must be in KBS before the pods start:

kubectl coco kbs populate -f <app>-trustee-secrets.yaml

5. Deploy

kubectl apply -f your-app-coco.yaml

Note: There are some sample manifests under examples folder which you can try.

kubectl coco apply runs kubectl apply automatically unless --skip-apply is set. Use --skip-apply when you need to upload secrets to KBS before the workload starts (recommended for first deployments).

What Gets Transformed

kubectl-coco performs these transformations:

1. Sets RuntimeClass to kata-cc (configurable)
2. Converts Secrets:
   • Detects all secret references (env, envFrom, volumes)
   • Creates sealed secrets with -sealed suffix
   • Writes KBS resource references to <app>-trustee-secrets.yaml (upload with kbs populate)
   • Updates manifest to use sealed secret names
3. Handles ImagePullSecrets:
   • Keeps imagePullSecrets in manifest (for CRI-O)
   • Writes credentials to <app>-trustee-secrets.yaml (upload with kbs populate)
   • Adds KBS URI to initdata CDH configuration
   • Falls back to default service account if not specified
4. Generates InitData: Creates aa.toml, cdh.toml, policy.rego
5. Places Annotations: Correctly adds initdata on pod templates
6. Adds Custom Annotations: From your config file

For detailed information, see TRANSFORMATIONS.md.

Prerequisites

• Go 1.24+ (for building from source)
• kubectl (for applying manifests)
• Kubernetes cluster with CoCo runtime installed

Installation

From Release Binary

1. Download the latest release:

   OS=$(uname -s | tr '[:upper:]' '[:lower:]')
   ARCH=$(uname -m)
   if [ "$ARCH" = "x86_64" ]; then ARCH="amd64"; fi
   curl -LO "https://github.com/confidential-devhub/cococtl/releases/latest/download/kubectl-coco-${OS}-${ARCH}"

   For a specific version:

   VERSION=v0.1.0
   curl -LO "https://github.com/confidential-devhub/cococtl/releases/download/${VERSION}/kubectl-coco-${OS}-${ARCH}"

2. Validate (optional):

   curl -LO "https://github.com/confidential-devhub/cococtl/releases/latest/download/kubectl-coco-${OS}-${ARCH}.sha256"
   echo "$(cat kubectl-coco-${OS}-${ARCH}.sha256)" | sha256sum --check

3. Install:

   System-wide (requires sudo):

   sudo install -m 0755 kubectl-coco-${OS}-${ARCH} /usr/local/bin/kubectl-coco
   sudo ln -sf /usr/local/bin/kubectl-coco /usr/local/bin/kubectl_complete-coco

   Or user directory:

   mkdir -p ~/.local/bin
   install -m 0755 kubectl-coco-${OS}-${ARCH} ~/.local/bin/kubectl-coco
   ln -sf ~/.local/bin/kubectl-coco ~/.local/bin/kubectl_complete-coco
   export PATH=$PATH:~/.local/bin  # Add to ~/.bashrc or ~/.zshrc

4. Verify:

   kubectl coco --version

From Source

git clone https://github.com/confidential-devhub/cococtl
cd cococtl
make build
sudo make install

Shell Completion

kubectl-coco supports command autocompletion for bash and zsh shells.

Bash

Prerequisites:

Install bash-completion.

For MacOS:

brew install bash-completion@2

# Add to your ~/.bash_profile:
[[ -r "/opt/homebrew/etc/profile.d/bash_completion.sh" ]] && . "/opt/homebrew/etc/profile.d/bash_completion.sh"

# Reload profile:
source ~/.bash_profile

For Linux:

# Ubuntu/Debian:
apt-get install bash-completion

# CentOS/RHEL:
yum install bash-completion

Installation:

For current session:

source <(kubectl-coco completion bash)

For all sessions (permanent):

# MacOS:
kubectl-coco completion bash > $(brew --prefix)/etc/bash_completion.d/kubectl-coco

# Linux:
kubectl-coco completion bash > /etc/bash_completion.d/kubectl-coco

# Then restart your shell

For kubectl plugin (kubectl coco):

Install kubectl completion first:

# MacOS:
kubectl completion bash > $(brew --prefix)/etc/bash_completion.d/kubectl

# Linux:
kubectl completion bash > /etc/bash_completion.d/kubectl

Zsh

Enable completion if not already enabled:

echo "autoload -U compinit; compinit" >> ~/.zshrc

Install kubectl-coco completion:

kubectl-coco completion zsh > "${fpath[1]}/_kubectl-coco"

Start a new shell for completion to take effect.

Note: The kubectl_complete-coco symlink (created during installation) enables kubectl coco plugin completion. If kubectl coco <TAB> doesn't work, verify:

ls -la /usr/local/bin/kubectl_complete-coco
# Should point to kubectl-coco

Usage

Initialize Configuration

Deploy Trustee and create configuration (non-interactive by default):

kubectl coco init

This deploys Trustee to your current namespace and creates ~/.kube/coco-config.toml.

Interactive mode:

kubectl coco init --interactive  # or -i

With custom Trustee:

kubectl coco init --trustee-url https://trustee.example.com:8080

Manage KBS (Key Broker Service)

The kbs subcommand manages the Trustee Key Broker Service that stores your secrets.

Deploy KBS in Kubernetes

kubectl coco kbs start --mode k8s

Deploys Trustee to the current namespace and saves the admin private key to ~/.kube/coco-kbs-auth. The KBS URL is written to ~/.kube/coco-config.toml for use by subsequent commands.

# With custom namespace
kubectl coco kbs start --mode k8s --namespace coco-system

Register an External KBS

kubectl coco kbs start --mode external --url http://kbs.example.com:8080

Records the KBS URL in config without deploying anything. Optionally specify --auth-dir to point at an existing admin key directory.

Upload Resources to KBS

After kubectl coco apply generates a *-trustee-secrets.yaml, upload the secrets:

kubectl coco kbs populate -f app-trustee-secrets.yaml

Other input modes:

# From a Kubernetes Secret
kubectl coco kbs populate --from-k8s-secret my-registry-secret -n my-namespace

# Single file to a specific KBS path
kubectl coco kbs populate --path default/myapp/password --resource-file /path/to/password.txt

# Direct URL (skips in-cluster port-forward)
kubectl coco kbs populate --kbs-url http://kbs.example.com:8080 --auth-key /path/to/private.key -f secrets.yaml

Transform and Apply Manifests

Basic usage:

kubectl coco apply -f app.yaml

Common options:

# Only transform, don't apply
kubectl coco apply -f app.yaml --skip-apply

# Use specific runtime class
kubectl coco apply -f app.yaml --runtime-class kata-remote

# Add attestation initContainer
kubectl coco apply -f app.yaml --init-container

# Enable secure access sidecar
kubectl coco apply -f app.yaml --sidecar

# Disable automatic secret conversion
kubectl coco apply -f app.yaml --convert-secrets=false

# Use custom config file
kubectl coco apply -f app.yaml --config /path/to/config.toml

See TRANSFORMATIONS.md for detailed description on the transformations.

Learn CoCo Transformations

The explain command helps you understand what transformations are applied to your manifests:

# Analyze your manifest
kubectl coco explain -f your-app.yaml

# View built-in examples
kubectl coco explain --list-examples

# Learn with interactive examples
kubectl coco explain --example simple-pod
kubectl coco explain --example deployment-secrets
kubectl coco explain --example sidecar-service

Output formats:

# Human-readable (default)
kubectl coco explain -f app.yaml

# Side-by-side diff view
kubectl coco explain -f app.yaml --format diff

# Markdown for documentation
kubectl coco explain -f app.yaml --format markdown -o transformations.md

The explain command provides:

• Educational analysis of each transformation
• Before/after comparisons for secrets, runtime, and initdata
• Learning points explaining why each change is needed
• Interactive examples to explore CoCo concepts

Perfect for learning how CoCo works without making any changes to your cluster.

Secure Access Sidecar

The secure access sidecar provides mTLS-secured HTTPS access to your CoCo pods.

One-time setup:

kubectl coco init --enable-sidecar

Deploy with sidecar:

# Basic usage
kubectl coco apply -f app.yaml --sidecar

# Enable port forwarding from primary container
kubectl coco apply -f app.yaml --sidecar --sidecar-port-forward 8888

# Custom SANs for LoadBalancer or Ingress
kubectl coco apply -f app.yaml --sidecar \
  --sidecar-san-ips=203.0.113.10 \
  --sidecar-san-dns=myapp.example.com

Note: When --sidecar is enabled, a Kubernetes Service (ClusterIP type) is automatically created with the name <app-name>-sidecar to expose the sidecar's HTTPS port. You can convert it to NodePort or use it with an Ingress for external access.

See sidecar/README.md for detailed configuration and usage.

Configuration File

The configuration file (~/.kube/coco-config.toml) supports:

# Mandatory
trustee_server = 'https://trustee-kbs.default.svc.cluster.local:8080'
runtime_class = 'kata-cc'

# Optional
trustee_ca_cert = '/path/to/ca.crt'
kata_agent_policy = '/path/to/policy.rego'
init_container_image = 'quay.io/fedora/fedora:44'
init_container_cmd = 'curl http://localhost:8006/cdh/resource/default/attestation-status/status'

# Image-related (optional, for CDH [image] section)
container_policy_uri = 'kbs:///default/security-policy/test'
registry_cred_uri = 'kbs:///default/credential/test'
registry_config_uri = 'kbs:///default/registry-configuration/test'

# Custom annotations (optional, only non-empty values applied)
[annotations]
"io.katacontainers.config.runtime.create_container_timeout" = "120"
"io.katacontainers.config.hypervisor.machine_type" = "q35"

# Secure access sidecar (optional)
[sidecar]
enabled = true
image = "ghcr.io/confidential-devhub/coco-sidecar:latest"  # Optional: custom sidecar image
https_port = 8443                                          # Optional: HTTPS port (default: 8443)
forward_port = 8888                                        # Optional: application port to forward
cpu_limit = "100m"                                         # Optional: CPU limit
memory_limit = "128Mi"                                     # Optional: memory limit
cpu_request = "50m"                                        # Optional: CPU request
memory_request = "64Mi"                                    # Optional: memory request

Note: TLS certificates are auto-generated per-app during kubectl coco apply --sidecar.

Development

Building

make build

Testing

make test

Cleaning

make clean

References

• Confidential Containers Documentation
• InitData Feature
• Sealed Secrets
• Trustee Remote Attestation

License

Apache License 2.0

Contributing

Contributions are welcome! Please submit issues and pull requests to the repository.