Unit 8 — PKI

The PKI secrets engine dynamically issues X.509 certificates, allowing clients to obtain certificates without manually generating key pairs, submitting certificate signing requests, or waiting for approval workflows, which enhances agility. Vault’s authentication and authorization mechanisms act as an identity-driven PKI registration authority, enforcing role-based access control and policy-based request validation to ensure tightly controlled certificate issuance.

That phrase — identity-driven registration authority — is the one to carry into customer meetings. Vault isn’t replacing their CA; it’s replacing the RA function and the manual approval workflow around it.

Each application instance receives a cryptographically unique certificate (and optionally, private key), eliminating shared credentials and simplifying both certificate and key lifecycle management. By adopting short-lived certificates, organizations not only bolster their security posture but can also minimize the need for revocation, reducing, or eliminating reliance on certificate revocation lists and improving scalability.

Short TTLs are a scalability strategy, not just a security one. Revocation infrastructure is the expensive, fragile part of traditional PKI.

While the engine provides the foundation for secure, automated issuance (the producer), a complete solution also requires a client-side mechanism (the consumer) to handle retrieval, renewal, and integration.

• Platform operator: maintains and scales the Vault platform from an infrastructure perspective. This role may reside within security-focused teams or broader infrastructure platform groups.
• Secrets producer: configures and manages PKI secrets and operational patterns in Vault. Responsibilities include configuration of certificate authorities, setting up certificate issuance policies, defining access controls, and ensuring compliance. This role may include Vault administrators, PKI specialists, or security engineers.
• Secrets consumer: integrates Vault-issued certificates with various applications. This category includes DevOps engineers, application developers, and service owners.

People and process: establish shared workflows between platform and application teams and define ownership boundaries clearly, for example, who is responsible for debugging failed certificate renewals or policy misconfigurations. Regularly review access controls, PKI roles, and machine identity configurations. Integrate automated testing, such as through CI/CD pipelines, to validate consumption behaviors. Use a version-controlled repository to manage PKI configurations and policies. Provide onboarding guides and training. Maintain a centralized knowledge base. Monitor certificate issuance frequency and details via audit logs; set up alerts for failures in the renewal process, and on certificate expiration especially those used in critical systems; use dashboards to visualize usage trends and inform capacity planning.

Ask the ownership-boundary question in discovery. It has no obvious answer and surfaces the gap immediately.

Namespaces provide administrative isolation, making them well-suited for multi-tenancy scenarios. While namespaces are often used to separate resources and configurations at the organizational level, namespaces can also serve as strong boundaries for CA isolation. Each namespace can host one or more PKI secrets engines.

Proposed structure: an admin namespace hosting a centralized, Vault-managed subordinate CA (signed by an external root CA), used to sign issuing CA certificates; tenant namespaces hosting intermediate CAs used to issue leaf certificates for application workloads.

Root CA. While Vault is perfectly capable of acting as a root, keep the root CA and corresponding key material outside of Vault while managing any intermediate and issuing CAs within Vault. This approach aligns with industry best practices and reduces the risk of compromise of the root CA, which should be strongly protected by a secure key management system or hardware security module. HSM systems may optionally be air-gapped, depending on your specific environment and compliance requirements.

Regardless of the specific approach to root CA management, the root CA should never directly issue leaf certificates; instead, it should delegate this responsibility to intermediate CAs managed by Vault.

Vault-managed signing CA. Within Vault, a central signing CA (an intermediate CA signed by the root) resides in the administrative namespace and is responsible for signing subordinate issuing CAs. Since its primary role is to sign intermediate CAs rather than leaf certificates directly, you configure it with a relatively long lifespan. This design enhances flexibility and simplifies the maintenance of the certificate hierarchy, avoiding the complexity and process that often accompany root CA interactions.

Factors:

• Single issuer: the signing CA should only sign Intermediate CA requests from tenant-specific PKI secrets engines.
• Key type: Elliptic Curve private keys (for example, P-384 for CAs and P-256 for leaf certificates) provide a good balance of security, performance, and compatibility across use cases.
• Long TTL, staggered: its TTL should not be so long that its certificate expires simultaneously with the root CA. For example, if your Root CA has a TTL of 10 years, set the maximum TTL for the Vault-managed signing CA to 5-7 years to ensure staggered certificate renewals.
• Security restrictions: keep the managed central signing CA under strict security controls; similar to the root CA, you should never use it to issue leaf certificates directly, and any interactions should be closely monitored.

Once the Vault-managed central signing CA generates a CSR, your external root CA must sign it. Use your organization’s established methods, which may involve collaboration with other teams or individuals who manage PKI infrastructure, approvals, and processes.

The staggering point is subtle and customers miss it. If root and intermediate expire in the same window, you get a simultaneous root ceremony and a full reissuance event.

Issuing CA per tenant namespace. You can delegate the administration of the issuing CAs to namespace administrators while maintaining the centralized management and integrity of the central signing CA.

When generating the tenant issuing CAs, the signing CA can impose strong restrictions on the resulting capabilities of the issuers. For example, various URI and DNS domain constraints, exclusions, and path length controls you can and should enforce according to the desired governance model.

Tenant administrators should have the ability to configure and manage their PKI engines according to their needs, within the constraints set by the signing CA. Clients of a given namespace should only be able to interact with the PKI engine in their namespace.

This is the answer to the hardest PKI question customers ask: “if we give team A their own CA, what stops them issuing a certificate for team B’s domain?” The constraint is baked into the issuing CA certificate at signing time, so it holds even if the tenant admin is compromised.

You can use Sentinel policies to enforce universal governance of PKI engine configurations at the platform level — setting a global maximum TTL for certificates; enforcing or restricting the storage of certificates within Vault; restricting hostnames, subdomains, and wildcards to specific values.

Outside global controls, each tenant namespace should host its own issuing CA to prevent certificate issuance across trust boundaries. Best practices for issuing CAs:

• Certificate lifetimes: keep application leaf certificate TTLs short (≤30 days) to minimize reliance on CRLs.
• Automatic tidying: if storing certificates, enable automatic cleanup of expired or revoked certificates.
• Automatic CRL rebuilding: if using CRLs, automatically manage CRL rotation on a schedule to avoid expiration.
• Storage settings: use store mode where required, such as for ACME challenge validation; use no-store mode when you do not need to persist certificates.

Managed keys — use Vault’s managed key feature to store the CA key material in an HSM (using Vault’s PKCS#11 integration) or cloud-based KMS solution. This ensures key material never leaves tamper-resistant hardware.

The tradeoff: while offloading key material storage and signing operations to a trusted external system may enhance security and help you achieve certain policy requirements, doing so introduces a performance tradeoff in the form of increased latency. This latency is outside of Vault’s control and is a byproduct of both the required network communication and the chosen HSM or KMS solution’s performance capabilities. Selecting elliptic curve key types, which are considerably more efficient, and avoiding certificate storage can help offset this latency, but you should test in your environment.

Hybrid: in some cases, customers may decide to optimize performance by using managed keys only for the centralized signing CA, while allowing Vault to store and manage the individual issuing CA key material.

Seal wrap: for cases where managed keys are not used, you can configure the PKI secrets engine with seal wrapping. In addition to standard keyring-based encryption, seal wrapping also encrypts private keys using the seal device before storage, adding an extra layer of data security. With seal wrap enabled, Vault caches the private key in memory while in use to optimize the performance of signing operations.

The hybrid pattern is the practical recommendation for most financial services customers: HSM protection where auditors look, Vault-native performance where volume lives.

While Vault never retains private keys even when configured to generate them for clients, it can optionally store issued leaf certificates in its storage backend on a per-role basis.

Several factors should inform the decision: the certificate’s validity period, revocation workflow requirements, and any applicable internal policies or governance requirements.

The economics: in general, the shorter the certificate’s TTL, the less critical revocation becomes. For example, if a certificate is valid for only 10 minutes, the process of revoking it, updating the CRL, and distributing that new CRL before the certificate naturally expires is often impractical and resource-intensive. In such cases, the cost of revocation may outweigh the benefits.

The standard method for revoking certificates in Vault is to use the certificate’s serial number, which requires the certificate to be present in Vault’s storage. When certificates are not stored (no_store=true), this administrative revocation path is unavailable.

Vault supports revocation with CRLs or OCSP, even when certificates are not stored. Vault’s Bring Your Own Certificate feature enables this, which allows you to supply the full certificate material at revocation time. You can optionally configure your audit device to write issued certificate PEM data in plaintext to better enable this approach.

Performance: disabling certificate storage also improves PKI issuance performance, as Vault can issue certificates without writing to storage. This allows performance standby nodes to service issuance requests directly without proxying to the cluster leader, resulting in significant performance gains in high-throughput environments. Internal testing demonstrates a 3-5x improvement in leaf issuance rates when certificates are not stored.

By default, only the active node handles PKI certificate issuance requests. To distribute the workload, performance standby nodes can handle requests if certificates are not stored in Vault. This setup requires a Vault Enterprise license and an appropriate load balancer configuration that distributes traffic across all cluster nodes.

The load balancer requirement is what customers miss. They set no_store=true, see no improvement, and conclude the feature doesn’t work — because the LB still sends everything to the active node.

Leases: if you enable certificate storage, you can configure PKI roles to associate Vault leases with issued certificates. However, turn off lease generation for certificates and rely instead on PKI-native revocation semantics. Leases present non-trivial resource utilization and management overhead that you should avoid for PKI use cases.

The question nobody asks: it is important to consider that verification of certificate validity and revoked status is a client responsibility that is not inherent or automatic. You should investigate whether certificate status is actively and consistently verified by clients that operate in your environment, and if so, what protocol they use. Often, organizations believe that they must support revocation and certificate storage for compliance purposes, only to find out that no clients respect the CRL or OCSP information contained in leaf certificates. If clients do not check for revocation, there is arguably no need to maintain a CRL or run OCSP responders. HVD still recommends that you enable and configure CRL and OCSP so that revocation remains an option.

Bring this up in every PKI engagement. It reframes an expensive requirement as an assumption worth testing.

Cross-cluster: in performance replication topologies, each cluster maintains its own certificate and lease storage, as well as CRL. Even though they share CA configurations and key material, clusters execute signing and issuance operations independently. Configure Authority Information Access and CRL Distribution Point URLs for each issuer to ensure consistent revocation data across all clusters.

• AIA templates: configure AIA URLs to support unified CRLs across replicated topologies and ACME services. This ensures that clients can accurately validate certificates regardless of which cluster issued the certificate.
• Cross-cluster revocation: enable it to synchronize revocation data across all clusters when using performance replication.
• OCSP: configure endpoints to provide real-time certificate status information.
• If the CRL becomes excessively large, there are potential performance penalties to consider. In a high-volume environment where revocation support is necessary, forgoing CRLs in favor of a unified (cross-cluster) OCSP view may prove to be the best option.

Automatic CRL rebuilding: enable auto_rebuild=true to ensure consistent, timely rotation and to prevent CRLs from expiring unexpectedly. The latter can occur if no revocations occur for a span of time that extends beyond the CRL expiration date. However, enabling this setting disables the default behavior of immediately rebuilding the CRL upon each certificate revocation. In this case, if you revoke a certificate and must reflect it in the CRL right away, you must trigger a manual CRL rebuild.

That tradeoff is a real incident-response gap. A customer revoking a compromised certificate on an auto_rebuild schedule, without knowing to force a rebuild, has a compromised certificate that still validates.

Tidying cleans up expired certificates, issuers, revocation data, and associated metadata. Practices: set appropriate tidying intervals based on deployment size and certificate issuance rate (for example, 24h), using daily tidying as a starting point; regularly monitor the performance impact and adjust; enable tidying for all tenant PKI mounts to maintain consistency.

A PKI role granularly defines the rules and constraints for certificate issuance, similar to a certificate template or policy in alternative PKI solutions. The template analogy is the right one for customers coming from Microsoft ADCS.

Create multiple roles, each scoped to a specific application or use case, rather than create overly permissive roles that allow unnecessary latitude in request criteria.

vault write pki/roles/team-a \
  allowed_domains="tenant-1.example.com" \
  use_csr_common_name=true \
  require_cn=true \
  use_csr_sans=true \
  allow_subdomains=true \
  allow_bare_domains=false \
  allow_glob_domains=false \
  allow_localhost=false \
  allow_wildcard_certificates=false \
  allow_ip_sans=false \
  enforce_hostnames=true \
  server_flag=true \
  client_flag=false \
  code_signing_flag=false \
  organization="HashiCorp" \
  policy_identifiers="1.3.6.1.4.1.99999.999.1" \
  key_usage="DigitalSignature,KeyAgreement,KeyEncipherment" \
  key_type="ec" \
  key_bits="256" \
  max_ttl="720h" \
  no_store=true \
  generate_lease=false \
  no_store_metadata=false

resource "vault_pki_secret_backend_role" "team-a" {
  backend                     = vault_mount.tenant_issuing_ca.path
  name                        = "team-a"
  allowed_domains             = ["tenant-1.example.com"]
  allow_subdomains            = true
  allow_bare_domains          = false
  allow_glob_domains          = false
  allow_localhost             = false
  allow_wildcard_certificates = false
  allow_ip_sans               = false
  enforce_hostnames           = true
  server_flag                 = true
  client_flag                 = false
  code_signing_flag           = false
  use_csr_common_name         = true
  require_cn                  = true
  use_csr_sans                = true
  organization                = ["HashiCorp"]
  policy_identifiers          = ["1.3.6.1.4.1.99999.999.1"]
  key_usage                   = ["DigitalSignature", "KeyAgreement", "KeyEncipherment"]
  key_type                    = "ec"
  key_bits                    = 256
  max_ttl                     = 2592000 # 720h in seconds
  no_store                    = true
  generate_lease              = false
  no_store_metadata           = false
}

What it encodes:

• Domain and SAN restrictions: limits certificate issuance to subdomains of tenant-1.example.com, and restricts the use of IP SANs and wildcard certificates. Valid: app1.tenant-1.example.com, app2.tenant-1.example.com. Invalid: tenant-1xyz.example.com, *.tenant-1.example.com.
• Key type and size: EC P-256, balancing security and performance.
• TTL: maximum lifetime of 30 days to reduce revocation needs. Requests for certificates with longer TTLs are overridden and issued with the maximum allowed value.
• Storage optimization: no_store=true, generate_lease=false, and no_store_metadata=false enables metadata storage for auditing and tracking purposes, even if not storing certificates.

Note tenant-1xyz.example.com — the classic prefix-matching attack, blocked by allow_bare_domains and allow_glob_domains both false. And note the TTL behavior: silent capping, not rejection. Applications requesting a year and receiving 30 days fail later, not now.

Metadata carries a penalty: if a certificate request contains metadata and you set no_store_metadata to false, the system forwards the request to the active leader node. This carries a performance penalty. That partially undoes the no_store=true optimization — decide deliberately.

Certificate metadata was introduced in Vault 1.17. Examples include application, certificate owner, contact information, business unit, risk profile, host. The system stores metadata separately from the certificates themselves, enabling you to utilize this feature even if you choose not to store issued certificates. Add it via cert_metadata, base64-encoded before sending to the API.

vault read -field cert_metadata pki/cert-metadata/<serial> | base64 -d | jq

This answers “who owns this certificate and who do we call when it expires” — pitch it as an operational feature, not a security one.

1. Machine authentication per tenant — each tenant namespace hosts one or more machine authentication mounts that are specific to the designated consuming application ecosystem.
2. Policy-based access to PKI roles — Vault issues clients a time-bound token with an associated policy that grants access to specific API paths and methods. Policies are dynamically mapped based on the business segment/team, and every business segment has a designated role.
3. Path templating — reduces manual policy configuration overhead while maintaining strict role-based access controls.

path "pki/issue/team-a" {
   capabilities = ["update"]
}

path "pki/issue/{{identity.entity.metadata.TeamName}}" {
  capabilities = ["update"]
}

Sentinel EGP to prevent unauthorized manipulation of certificate templates — ensuring applications and developers can only request certificates for subdomains present in the requestor’s entity metadata, and only a trusted orchestrator or an entity with predefined metadata can request certificates:

## This policy is to restrict the common name while issuing a pki cert
import "strings"

# Only care about write and update operations against pki/issue
precond = rule {
  request.operation in ["write", "update"] and
  strings.has_prefix(request.path, "pki/issue")
}

# Check if the trusted orchestrator makes the request
trusted_orchestrator_check = func() {
  print ("trace:identity.entity.name", identity.entity.name)
  if identity.entity.name matches "terraform" {
    return true
  }
  return false
}

# Check common_name matches the entity metadata
common_name_check = func() {
  print ("trace:Request.data:", request.data)
  print ("trace:TLSDomain", identity.entity.metadata.TLSDomain)

  if length(request.data else 0) is 0 {
    return false
  }
  if length(request.data.common_name else 0) is 0 {
    return false
  }
  if request.data.common_name matches identity.entity.metadata.TLSDomain {
    return true
  }
  return false
}

main = rule when precond {
  trusted_orchestrator_check() or common_name_check()
}

The mental model: ACLs control which endpoint you may call. Roles control what the resulting credential can do. Sentinel controls what the request payload may contain, given context. Customers who want delegation with central control need all three.

Both endpoints are available to any PKI role, provided the client has appropriate access via ACL policy.

issue — for fully automated workflows. On request, Vault generates a private key and issues a certificate, returning both to the client. The private key is ephemeral, is not retained by Vault, and must be securely handled by the requesting application.

sign — supports a more traditional CA workflow, where the client generates its own key pair and submits a CSR to Vault. Vault signs the CSR and returns only the certificate. The private key remains under the exclusive control of the client and is never seen by Vault.

While both methods are valid, Vault Agent templating requires the issue endpoint.

This is a fork in the design. Customers with a hard requirement that private keys never traverse the network cannot use the Agent templating pattern. Establish this early.

Exposing both endpoints to clients allows developers to choose the workflow that works best for their application.

Why Agent rather than custom code: teams might build and maintain custom automation for authentication, certificate retrieval, renewal timing, and persistence. However, this approach may not scale well in large enterprises with diverse platforms and runtimes, or may be wholly inappropriate for the target application. Vault Agent simplifies client-side integration by abstracting authentication, token management, and secrets consumption. For PKI workloads, it automates the certificate lifecycle and provides flexible templating features. In addition, Vault Agent can trigger scripted actions after it issues a new certificate, such as reloading a service or calling a webhook.

Auth: use a platform identity source (AWS IAM, Kubernetes service accounts, Azure MSI, and so on) instead of a static credential. For workloads without a built-in source of identity, we recommend a trusted orchestrator pattern to provide credentials, such as an AppRole secret ID.

auto_auth {
   method "aws" {
       mount_path = "auth/aws"
       config = {
           type = "iam"
           role = "my-app"
       }
   }
}

Auto-auth can also specify token sinks that store the Vault token in a local file. Sinks are optional and you should not include them unless your use case requires direct access to a Vault token.

template {
    source = "/vault-agent/pkiCerts.tmpl"
    destination = "/vault-agent/template-output/pki.data"
}

template_config {
   lease_renewal_threshold = 0.75
}

lease_renewal_threshold determines how long the template engine waits to attempt a renewal of the underlying certificate. It is defined as a fraction of the certificate’s total lifetime and defaults to 90 percent. Therefore, if your certificate has a TTL of 10 hours, the agent begins its renewal attempts approximately 1 hour before the expiration date.

For short-TTL certificates the default 90% leaves a thin margin. Lowering it buys retry room at the cost of more issuance volume.

The destination file acts as a persistent cache for the pkiCert function output and consuming applications do not use it directly. It is important to configure a default template output containing unique secret data so that the template engine establishes a source of comparison when deciding whether to request a new certificate.

secret vs pkiCert: when rendering a certificate using secret, the Vault Agent will always fetch a new certificate at startup or during re-authentication, even if the current certificate is valid. The pkiCert function manages rendering and renewals by checking the file system for an existing certificate. If no certificate exists at the destination path, the agent retrieves and renders a new certificate. If a certificate is already present, the agent examines its expiration date. If the certificate has expired or is past the renewal threshold, a new one replaces it. However, if the certificate is still valid, the existing certificate remains in place. We recommend the pkiCert function for all certificate management use cases.

Using secret for certificates means every Agent restart burns a new certificate — an issuance-rate problem masquerading as normal operation.

Helper keys: .Cert (the certificate body), .Key (the private key), .CAChain (the CA chain defined in the PKI engine configuration). The .CAChain field is only available in versions 1.18 or above of Vault Agent.

{{- with pkiCert "pki/issue/team-a" "common_name=app.tenant-1.example.com" "ttl=14d" "remove_roots_from_chain=true" -}}
{{- .Key -}}
{{- .Cert -}}
{{- .CAChain -}}
{{- .Key | writeToFile "/etc/nginx/certs/private.key" "" "" "0600" -}}
{{- .Cert | writeToFile "/etc/nginx/certs/server.crt" "" "" "0644" -}}
{{- range .CAChain -}}
{{- . -}}
{{- . | writeToFile "/etc/nginx/certs/server.crt" "" "" "0644" "append" -}}
{{- end -}}
{{- end -}}

writeToFile "[output-path]" "owner" "group" "permission-bits".

The root certificate is deliberately omitted, as we expect to distribute anchor certificates out-of-band and install them in device trust stores outside this workflow.

Metadata via templating:

{{- scratch.MapSet "certMetadata" "teamName" "team-a" -}}
{{- scratch.MapSet "certMetadata" "application" "my-app" -}}
{{- scratch.MapSet "certMetadata" "contact/email" "team-a@example.com" -}}
{{- scratch.MapSet "certMetadata" "contact/slack" "#team-a" -}}
{{- $certMetadata := scratch.Get "certMetadata" | explodeMap | toJSON | base64Encode -}}
{{- $certMetadataArg := printf "cert_metadata=%s" $certMetadata -}}

{{- with pkiCert "pki/issue/team-a" "common_name=app.tenant-1.example.com" "ttl=14d" "remove_roots_from_chain=true" $certMetadataArg -}}

The / separator can be used when defining the keys to create nested objects.

Post-render commands. NGINX does not automatically reload the TLS certificate when it changes on disk. Instead, it continues to use the previously loaded certificate and key until the configuration is reloaded.

template {
    source = "/vault-agent/pkiCerts.tmpl"
    destination = "/vault-agent/template-output/pki.pem"
    exec {
       command = ["nginx", "-s", "reload"]
    }
}

Ask “how does your application learn about a new certificate?” during design. Every certificate automation project eventually discovers this the hard way.

Deployment posture: if your application is long-running, run the agent in daemon mode (with exit_after_auth set to false) and manage its execution with a service manager, such as systemd or Windows Service Control Manager. If the application runs on a host with other applications that also need secrets or certificates via Vault Agent, deploy a single instance of Vault Agent for each application. This enables each application to authenticate individually, helping you maintain least-privilege access principles. This separation further reduces the blast radius of any potential misconfiguration. Vault Agent supports telemetry and logging, both of which should be used. Consider ingesting your metrics and logs into your enterprise monitoring solutions to proactively identify issues with agents and prevent certificates from not being rotated due to problems like authentication failures.

Applications following this file-based pattern: Apache HTTPD, HAProxy, PostgreSQL, MySQL, MongoDB, Kafka, RabbitMQ, Jetty, Tomcat, and Elasticsearch.