Unit 8C — Key Lifecycle Management: KMS and KMIP

Organizations generate and store vast quantities of sensitive data. Encryption serves as a critical line of defense, safeguarding information by rendering it unreadable without a decryption key. However, the security of encrypted data hinges entirely on the proper management of those keys.

Key lifecycle management encompasses the entire lifecycle of cryptographic keys, from their secure generation and distribution to their controlled usage, storage, archiving, and eventual destruction. Its primary objective is to ensure the confidentiality and integrity of encrypted data by safeguarding keys from cybercriminals, unauthorized individuals, or accidental exposure.

BYOK is a key management strategy that empowers organizations to retain control over their encryption keys across various platforms and services. The benefits:

• Independent key generation — verify the integrity and security of encryption keys by independently generating them using trusted entropy sources. This ensures compliance with stringent security and regulatory requirements.
• Exclusive key ownership — maintain exclusive ownership, ensuring complete control over data access and usage. This eliminates reliance on a third party for key management.
• Disaster recovery — by retaining the original copy of the key material outside of the cloud environment, organizations can implement robust disaster recovery plans to protect against data loss due to cloud service disruptions or failures.
• Seamless integration — enables organizations to use the same encryption keys across different platforms.

Vault offers two key lifecycle management solutions:

1. Key Management secrets engine — enables centralized lifecycle management of cryptographic keys while allowing copies of the key material to be securely distributed to cloud key management service providers for cryptographic operations.
2. KMIP secrets engine — Vault acts as a KMIP server, allowing services and applications to delegate cryptographic key management to Vault using a standardized protocol.

As organizations adopt a cloud operating model, key management becomes a significant security and operational challenge. In particular, it is challenging for organizations to maintain centralized control over the generation and lifecycle of key material while still being able to take advantage of native cloud integrations for securing data. Operating in multiple clouds can further amplify these challenges.

That is the pitch in one sentence: keep central control of key material without giving up native cloud integrations.

Supported providers: Azure Key Vault, AWS KMS, GCP Cloud KMS.

Requirements:

• Vault Enterprise license with the Advanced Data Protection (ADP) module
• Any of the cloud KMS providers
• Review the compatible key types for each KMS provider

Cloud KMS providers generally support: importing of key material using secure key wrapping methods; rotation of keys/versions; enabling and disabling of keys/versions; deletion and archival of keys/versions.

The cloud KMS providers allow you to create a KMS key without key material and then import your own key material into that KMS key. The ImportKey implementation will carry out the secure key import procedure for the cloud KMS provider as specified in their BYOK specification. Key material will always be securely transferred in accordance with the key import specification of the supported KMS providers.

Setup and distribution

vault secrets enable keymgmt

Create a named key:

vault write -f keymgmt/key/example-key type="rsa-2048"

Configure the KMS provider:

vault write keymgmt/kms/example-kms \
    provider="azurekeyvault" \
    key_collection="keyvault-name" \
    credentials=client_id="${CLIENT_ID}" \
    credentials=client_secret="${CLIENT_SECRET}" \
    credentials=tenant_id="${TENANT_ID}"

The values supplied to the generic set of parameters will differ depending on the specified provider.

Distribute the key:

vault write keymgmt/kms/example-kms/key/example-key \
    purpose="encrypt,decrypt" \
    protection="hsm"

This operation distributes a copy of the named key to the KMS provider with a specific purpose and protection. The purpose defines the set of cryptographic capabilities that the key will have in the KMS provider. The protection defines where cryptographic operations are performed with the key in the KMS provider.

Rotation and version control

vault write -f keymgmt/key/example-key/rotate

Rotating a key creates a new key version that contains new key material. The key will be rotated in both Vault and the KMS provider that the key has been distributed to. The new key version will be enabled and set as the current version for cryptographic operations in the KMS provider.

vault write keymgmt/key/example-key min_enabled_version=2

All versions of the key less than the min_enabled_version will be disabled for cryptographic operations in the KMS provider that the key has been distributed to. Setting a min_enabled_version of 0 means that all key versions will be enabled.

Same caution as Transform key versions in Unit 8B.6 — setting the minimum too aggressively disables versions that existing ciphertext still depends on.

Removal

vault delete keymgmt/kms/example-kms/key/example-key

This operation results in the key being deleted from the KMS provider. The key will still exist in the secrets engine and can be redistributed to a KMS provider at a later time. Permanent removal requires the delete key API on the engine itself.

Best practices

Define a process. Establish a key lifecycle management strategy encompassing key generation, distribution, utilization, rotation, and decommissioning to guarantee consistent and secure key handling throughout their existence.

Key generation. Key has to be selected based on the purpose that defines a set of cryptographic capabilities that the key will have in the KMS provider. Supported values include “encrypt”, “decrypt”, “sign”, “verify”, “wrap”, and “unwrap”.

Key rotation. Establish a defined key rotation schedule based on risk assessment and regulatory requirements. Keep at least two versions of encryption keys to facilitate smooth transitions during rotation, considering that the time required to distribute a key to a KMS provider is proportional to the number of key versions.

If a timeout occurs when distributing a key to a KMS provider, you may need to increase the value of VAULT_CLIENT_TIMEOUT. That is the failure signature to recognize when key distribution fails on a key with many versions.

Key revocation. Define a revocation process to promptly deactivate and replace compromised keys. Effective key revocation is critical to preventing data exposure and limiting the damage caused by security incidents.

Monitoring and auditing. Continuous monitoring and auditing capabilities to track key usage patterns and identify anomalous behavior will have to be established at the respective KMS provider. Note where that responsibility sits — at the KMS provider, not in Vault’s audit log. Vault sees distribution and rotation; it does not see cryptographic operations performed in the cloud KMS.

The OASIS Key Management Interoperability Protocol standard is a widely adopted protocol for handling cryptographic workloads and secrets management for enterprise infrastructure such as databases, network storage, and virtual/physical servers.

While Vault’s Transit secrets engine provides transparent data encryption through its native APIs, many organizations have existing systems and applications that require cryptographic operations which rely on external key management services. To address this, Vault provides a KMIP secrets engine, enabling it to function as a KMIP server.

Vault’s KMIP secrets engine serves as a secure and efficient bridge that integrates organizations’ legacy systems with modern key management practices, offering a standardized interface while leveraging Vault’s inherent security and flexibility.

That framing matters: KMIP is how Vault reaches systems that will never call a REST API — storage arrays, databases with native encryption, appliances.

Supported profiles — Vault implements version 1.4 of: Baseline Server; Symmetric Key Lifecycle Server; Basic Cryptographic Server; Asymmetric Key Lifecycle Server; Advanced Cryptographic Server.

Architecture

The KMIP secrets engine is composed of two components:

• The KMIP server component, which exposes a listener to accept KMIP client requests. Client-server communication will occur over TCP with TLS using the KMIP protocol.
• Regular Vault HTTP API paths for configuring the listener, managing roles, and generating client certificates.

Vault’s KMIP secrets engine listens on a separate port from the standard Vault listener. Each Vault server in a Vault cluster configured with a KMIP secrets engine uses the same listener configuration, defaulting to port 5696. The KMIP listener port is configurable, which is necessary if there are multiple KMIP secrets engine mounts configured on a cluster. KMIP clients connect and authenticate to this listener port using Vault-generated TLS certificates.

The load balancer constraints — two of them, and they conflict with normal practice

The KMIP listener will only be enabled for performance standby servers. This means that if performance standby is not used, the load balancer must ensure that the KMIP route is directed only to the leader of the cluster. This can be achieved through health checks via the cluster port (8200) to determine the current leader node.

To ensure successful KMIP client authentication, Vault clusters operating behind a load balancer require the load balancer to be configured in pass-through mode. This configuration is essential for proper client certificate validation by Vault, as the KMIP protocol mandates direct client-server communication for authentication. The load balancer should support long-lived connections and it may use a round robin routing algorithm since Vault servers will forward to the primary Vault server if necessary.

Two things to carry into a design review: KMIP needs pass-through TLS (consistent with Unit 2.4’s general guidance) and long-lived connection support, which is not the default on many load balancers.

KMIP client requests to performance standbys and secondary clusters will be forwarded to the leader node on the primary cluster. So KMIP does not scale out across replicas — put it beside tokenization (8B.5) and Secrets Sync (6C.3) in the primary-bound category.

Certificate authority

When the KMIP secrets engine is initially configured, Vault generates a KMIP Certificate Authority to authenticate KMIP client certificates. Vault uses the internal KMIP CA to generate unique certificates for each client authenticating to Vault over the KMIP protocol. You cannot import an externally generated KMIP certificate authority: all KMIP authentication must use the internal KMIP CA.

That is a hard constraint worth surfacing early. A customer expecting to issue KMIP client certificates from their existing enterprise PKI cannot — regardless of how their PKI hierarchy in Unit 8.3 is structured.

Authorization — Vault policies do not apply

Vault policies do not come into play during these KMIP requests. The KMIP secrets engine determines the set of KMIP operations the clients are allowed to perform based on the roles that are applied to the TLS client certificate.

Scopes are created to partition KMIP managed object storage into multiple named buckets. Within a scope, roles can be created which dictate the set of allowed operations that a particular role can perform. TLS client certificates can be generated for a role, which services and applications can then use when sending KMIP requests.

This is a genuinely separate authorization system from everything in Unit 5. Scope → role → client certificate, with operations granted per role. Don’t assume ACL policies constrain KMIP behavior; they don’t.

Supported operations, granted using the convention operation_<KMIP_operation> such as operation_activate or operation_revoke:

Two pseudo-operations allow or deny all operation capabilities for a role: operation_all and operation_none.

Debugging

To facilitate debugging and troubleshooting for KMIP-specific operations, you must enable additional environment variables before starting Vault:

export KMIP_CONNECTION_DEBUGGING=1
export KMIP_REQUEST_DEBUGGING=1
export KMIP_RESPONSE_DEBUGGING=1
export KMIP_DECODING_DEBUGGING=1
export KMIP_INDEX_DEBUGGING=1

Note these require a restart — plan for it in a maintenance window rather than discovering it mid-incident.

Integration reality

Vault KMIP can be integrated with any platform that functions as a KMIP client. Since the integration heavily depends on how the KMIP client is constructed and designed, the implementation of Vault KMIP may vary across different platforms.

Documented examples include MongoDB field-level encryption, IBM Storage Scale as an RKM key server, and AWS KMS External Key Store (XKS) through the PKCS#11 provider. HashiCorp maintains a list of compatible platforms in the interoperability matrix under “Vault as an external key management system” — which includes all the platforms that have been tested to work.

Check that matrix before committing to a KMIP integration in a customer design. “Any KMIP client” is true in principle and uneven in practice.

General considerations

• Vault does not require or recommend the use of a separate KMIP cluster, but a separate KMIP cluster for high-traffic environments or demanding KMIP operations should be considered.
• KMIP credentials are used for authentication to Vault. However, these keys are typically stored according to the implementation approach of various KMIP clients. It is important to ensure that appropriate security measures are in place to protect these KMIP credentials.
• Vault audits all KMIP requests and responses. Establish monitoring and auditing to track key usage and identify suspicious behavior.