Handling Credentials with Secrets

Applications running in Kubernetes frequently require access to sensitive information like API keys, database credentials, or TLS certificates. While managing non-sensitive configuration data has dedicated tools, placing sensitive data directly in a Pod manifest or a container image poses a significant security risk. To address this, Kubernetes provides a dedicated object: the Secret.

A Secret is structurally similar to a ConfigMap. It is an object that stores a small amount of sensitive data as key-value pairs. The primary difference is intent and how Kubernetes treats the data. Secrets are designed to be used for confidential information, and the cluster can provide additional protections for them, such as avoiding writes to logs and enabling encryption at rest in etcd.

The Anatomy of a Secret

Secrets decouple sensitive data from your Pod definitions. This allows you to manage credentials and application code independently, following security best practices. By default, the data within a Secret is base64-encoded. It is important to understand that base64 is an encoding format, not an encryption mechanism. Anyone with access to the encoded string can easily decode it. The true security benefit of Secrets comes from integration with Kubernetes' Role-Based Access Control (RBAC), which restricts which users or service accounts can read the Secret object itself.

Kubernetes supports several types of Secrets, specified by the type field in the manifest. The default and most common type is Opaque, which is used for arbitrary user-defined data. Other types exist for specific use cases, such as kubernetes.io/dockerconfigjson for storing private container registry credentials. For most application credentials, you will use the Opaque type.

Creating Secrets

You can create Secrets using imperative kubectl commands or declaratively with a YAML manifest.

Imperative Creation with kubectl

For quick tasks or simple credentials, kubectl create secret is very efficient. The most common variation is generic, which creates a Secret of type Opaque.

To create a secret named db-credentials with a username and password, you can use the --from-literal flag.

kubectl create secret generic db-credentials \
  --from-literal=username=webapp_user \
  --from-literal=password='P@s5w0rd!_123'

This command creates a new Secret in your current namespace. If you run kubectl get secret db-credentials -o yaml, you will see that Kubernetes has automatically base64-encoded the literal values you provided.

Declarative Creation with a Manifest

For production environments and version-controlled infrastructure (GitOps), defining Secrets declaratively in a manifest is the standard approach.

When defining a Secret in YAML, the values in the data field must be base64-encoded. You can generate the encoded string from the command line.

For example, to encode a password:

# The -n flag is important to prevent echo from adding a trailing newline
echo -n 'P@s5w0rd!_123' | base64

The output will be UEBzNXcwcmQhXzEyMw==. You would then use this encoded value in your manifest.

Here is a manifest for the same db-credentials Secret:

apiVersion: v1
kind: Secret
metadata:
  name: db-credentials
type: Opaque
data:
  username: d2ViYXBwX3VzZXI= # "webapp_user" encoded
  password: UEBzNXcwcmQhXzEyMw== # "P@s5w0rd!_123" encoded

Manually encoding values can be inconvenient. Kubernetes provides a stringData field as an alternative. When you use stringData, you can provide the raw, unencoded string, and Kubernetes will handle the encoding for you when the Secret is created. The stringData field is write-only; it will not be visible when you inspect the Secret later.

apiVersion: v1
kind: Secret
metadata:
  name: api-key-secret
type: Opaque
stringData:
  # Kubernetes will encode this value automatically
  apiKey: "abC123XyZ-Your-Super-Secret-Api-Key"

Using Secrets in Pods

Once a Secret is created, your Pods can consume its data in two primary ways: as environment variables or as files mounted into a volume.

Diagram illustrating the two primary methods for a Pod to consume data from a Secret.

Injecting Secrets as Environment Variables

Injecting secrets as environment variables is a common pattern for applications designed to read configuration from their environment. You can use the env or envFrom field in a container's definition.

To inject a specific key from a Secret, use valueFrom:

apiVersion: v1
kind: Pod
metadata:
  name: my-app-pod
spec:
  containers:
  - name: my-app
    image: nginx
    env:
      - name: DATABASE_USER
        valueFrom:
          secretKeyRef:
            name: db-credentials  # The name of the Secret
            key: username        # The key within the Secret
      - name: DATABASE_PASSWORD
        valueFrom:
          secretKeyRef:
            name: db-credentials
            key: password

In this example, the container will have two environment variables, DATABASE_USER and DATABASE_PASSWORD, populated with the decoded values from the db-credentials Secret.

Mounting Secrets as Volumes

For secrets that contain multiple values or represent entire configuration files (like a TLS private key or a JSON service account key), mounting them as a volume is more appropriate. When a Secret is mounted as a volume, each key-value pair in the Secret becomes a file in the mount directory. The filename is the key, and the file's content is the decoded value.

apiVersion: v1
kind: Pod
metadata:
  name: my-app-pod-volume
spec:
  containers:
  - name: my-app
    image: nginx
    volumeMounts:
    - name: secret-volume
      mountPath: "/etc/app-secrets"
      readOnly: true
  volumes:
  - name: secret-volume
    secret:
      secretName: db-credentials

After this Pod starts, the container will have a directory at /etc/app-secrets. Inside this directory, there will be two files: username and password, containing the respective credential values. The readOnly: true flag is a good practice to prevent the application from accidentally modifying the secret files.

Security Measures

While Secrets are the designated way to handle credentials in Kubernetes, it is important to follow security best practices:

1. Limit Access with RBAC: The primary defense for Secrets is strong RBAC policy. Only grant get, list, and watch permissions on Secrets to the users, groups, or service accounts that absolutely need them.
2. Enable Encryption at Rest: By default, Secret data in etcd is not encrypted. In a production cluster, you should configure encryption at rest to protect your secret data if an attacker gains read access to the etcd backup.
3. Use External Secret Stores: For higher security requirements, consider integrating an external secrets manager like HashiCorp Vault or AWS Secrets Manager. These systems provide centralized management, auditing, and dynamic secret generation. Kubernetes operators are available to sync secrets from these external stores into your cluster.
4. Avoid Checking Manifests into Git: Do not check raw Secret manifests containing base64-encoded data into version control. Use tools like Bitnami Sealed Secrets or SOPS (Secrets OPerationS) to encrypt the Secret manifest before committing it.