Unlocking Kubernetes Secrets: CSI Drivers vs. External Secret Operator

Container orchestration has been transformed by Kubernetes, but securely managing storage and secrets is still a difficult task. In this post, we'll examine how Kubernetes expands its functionality via the Container Storage Interface (CSI) and contrast two well-liked methods for incorporating external secret managers: the External Secret Operator and the Secrets Store CSI Driver.

CSI (Container Storage Interface) Drivers: What Are They?

Kubernetes and other container orchestrators can communicate with a range of storage devices via a standardized method called the Container Storage Interface (CSI). Consider the CSI driver as a plugin that connects Kubernetes to external storage sources, such as secret managers, cloud storage platforms, and more.

Important Points:

1. Container Storage Interface: A standard interface for container orchestrators to communicate with storage systems is defined by the Container Storage Interface (CSI) specification.

2. Driver Role: Acts as a bridge between Kubernetes and external systems. It dynamically provisions, attaches, and mounts storage resources to pods.

3. Usage Example: The secrets-store.csi.k8s.io CSI driver allows Kubernetes to securely inject secrets (passwords, API keys, etc.) from external providers (e.g., Azure Key Vault, AWS Secrets Manager) directly into pods.

SecretProviderClass: Securely Injecting Secrets

When using the Secrets Store CSI Driver, you configure a SecretProviderClass custom resource. This resource tells Kubernetes exactly which secrets to fetch and how to mount them into your pods.

How It Works (Using Azure Key Vault as an Example):

SecretProviderClass → Pod mounts volume via CSI Driver → Secrets injected directly into the pod

1. CSI Driver Role:

   • The CSI driver (e.g., secrets-store.csi.k8s.io) communicates with Azure Key Vault to retrieve secrets.

   • It then mounts these secrets as files in a volume inside the pod.

2. SecretProviderClass: The SecretProviderClass is a custom resource that defines:

   • Defines what secrets to fetch: For example, specific secrets from Azure Key Vault.

   • Specifies authentication details: Using methods such as Azure Managed Identity or Service Principal.

   • Provides key vault information: Including vault name, region, and tenant ID.

Sample SecretProviderClass Configuration:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: azure-kvname-user-msi
spec:
  provider: azure
  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "true"          # Set to true for using managed identity
    userAssignedIdentityID: <client-id>   # Set the clientID of the user-assigned managed identity to use
    keyvaultName: <key-vault-name>        # Set to the name of your key vault
    cloudName: ""                         # [OPTIONAL for Azure] if not provided, the Azure environment defaults to AzurePublicCloud
    objects:  |
      array:
        - |
          objectName: secret1
          objectType: secret              # object types: secret, key, or cert
          objectVersion: ""               # [OPTIONAL] object versions, default to latest if empty
        - |
          objectName: key1
          objectType: key
          objectVersion: ""
    tenantId: <tenant-id>                 # The tenant ID of the key vault

Pod Volume Mount Example:

In your pod spec, the CSI volume references the SecretProviderClass to determine which secrets to mount:

kind: Pod
apiVersion: v1
metadata:
  name: busybox-secrets-store-inline-user-msi
spec:
  containers:
    - name: busybox
      image: registry.k8s.io/e2e-test-images/busybox:1.29-4
      command:
        - "/bin/sleep"
        - "10000"
      volumeMounts:
      - name: secrets-store01-inline
        mountPath: "/mnt/secrets-store"
        readOnly: true
  volumes:
    - name: secrets-store01-inline
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: "azure-kvname-user-msi"

Key Benefits:

• Enhanced Security: Secrets never reside in Kubernetes etcd.

• Dynamic Updates: With options like --enable-secret-rotation and --rotation-poll-interval, secrets can be refreshed without needing to restart pods.

---

External Secret Operator: Connecting Kubernetes Secrets with External Systems

While the Secrets Store CSI Driver injects secrets directly into pods, the External Secret Operator synchronizes secrets from external managers (like AWS Secrets Manager or HashiCorp Vault) into native Kubernetes Secret objects.

Workflow Overview:

SecretStore → ExternalSecret → Kubernetes Secret → Pod consumes the Secret

• ExternalSecret Custom Resource:

  • Define which external secrets to fetch.

  • Specify how to map these secrets to a Kubernetes Secret.

• External Secret Operator Actions:

  • The operator continuously watches for changes in ExternalSecret CRs.

  • It then retrieves the secrets from the external provider and creates/updates the corresponding Kubernetes Secret.

• Pod Consumption:

  • Applications consume these Kubernetes Secret objects as environment variables or mounted volumes.

Example ClusterSecretStore Configuration:

apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
  name: azure-secret-store
  namespace: external-secrets
spec:
  provider:
    azurekv:
      tenantId: "XXXXX-d7f9-XXXXXX"
      vaultUrl: https://XXXX..vault.azure.net/
      authSecretRef:
        clientId:
          name: azure-secret-sp
          key: ClientID
          namespace: external-secrets
        clientSecret:
          name: azure-secret-sp
          key: ClientSecret
          namespace: external-secrets

Example ExternalSecret Configuration:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: external-secret
  namespace: app-namespace
spec:
  refreshInterval: "1h"
  secretStoreRef:
    kind: ClusterSecretStore
    name: azure-secret-store
  target:
    name: k8s-secret-name # Name of the resulting Kubernetes Secret
    creationPolicy: Owner # optional
  data:
    secretKey: cloud-credentials  # Key in the Kubernetes Secret
    remoteRef:
      key: key-vault-secret-name  # Key in the external secret manager

Use Cases:

• External Secret Operator: Best suited for legacy applications that expect secrets in the form of Kubernetes Secret objects.

• Secrets Store CSI Driver: Ideal for security-sensitive workloads where bypassing etcd storage is a priority.

Key Differences:

Key Differences in CRDs:

Conclusion:

There are two primary methods for incorporating external secret managers into your Kubernetes environment. While the Secrets Store CSI Driver with its SecretProviderClass offers a more secure approach by injecting secrets directly into pods without exposing them in etcd, the External Secret Operator is ideal for situations where you need to transform external secrets into Kubernetes-native objects.

Both approaches have advantages and disadvantages, and the optimal option for you will depend on your unique application security needs and operational procedures. For your Kubernetes clusters, you may create a strong, safe, and flexible secret management plan by being aware of the subtleties of these tools.

Thanks for reading this far! I appreciate your comments and feedback. 😬

Happy coding and secure deployments!🤞

#Azure Key Vault #External Secrets Operator #Integration #Vault #Kubernetes

About The Author: Prod1gy 😴: “kubern8is made me feel dumb !!”🫶