Resolve deduplication impact on ACL policy templates

Fix templated ACL policy behavior for entities and groups renamed during identity deduplication.

Assumptions

You are running Vault 1.19 or later.
You have deduplication renaming targets in your system logs.
You have admin permission on the relevant Vault server or cluster.

How renaming affect ACL policies

Operators use policy templating to specify permissions that dynamically resolve based on specific properties of an authenticated entity. During deduplication, Vault renames entities or groups with duplicate identities by appending a universally unique identifier (UUID) to all but one of the entities to avoid future duplication.

If a templated ACL policy relies on an entity or group name, renaming duplicate identities could inadvertently change the meaning of the policy. In practice, identity deduplication is unlikely to grant unintended access to an existing resource since the renamed identity contains a UUID that should not collide with other resource paths. But the rename might remove access to existing resources after deduplication if the associated resource policy references the old entity or group name.

For example, assume you have the following templated ACL policy that grants users access to a kv plugin:

path "kv/users/{{identity.entity.name}}/*" {
    capabilities = ["read", "create", "update"]
}

The policy grants a user entity named janine permission to read and write key-value pairs to the kv plugin on the path kv/users/janine/. If there are duplicate entities named janine, Vault renames all but one of entities to janine-<UUID> during deduplication.

The ACL policy now gives the renamed entity permission to read and write While the renamed entity can still access the kv plugin, the change in no longer grants access to path kv/users/janine/. As a result, the user can no longer access their data stored under kv/users/janine/.

Find affected ACL policies

Deduplication has the potential to break templated ACL policies that rely on entity or group name keys:

Template key	Description
`identity.entity.name`	Entity name
`identity.entity.aliases.<mount accessor>.name`	Entity alias name for the given mount
`identity.groups.ids.<group id>.name`	Group name for the given group ID
`identity.groups.names.<group name>.id`	Group ID for the given group name
`identity.groups.names.<group name>.metadata.<metadata key>`	Metadata for the given group name and key

You can use the following bash script to check for templated policies using entity or group names:

policy_list=$(vault policy list)

for policy in ${policy_list} ; do
  
  # Get the policy details
  policy_details=$(vault policy read ${policy})

  # Check for a name based template key
  if echo "${policy_details}" | grep -q "\.name" ; then
    echo ""
    echo "Policy name: ${policy}"
  fi
  echo "${policy_details}"  |
    tr -s ' ' '\n'          |
    grep "\.name" --label="  template key" -H
done

policy_list=$(
  curl                                              \
    --silent                                        \
    --request GET                                   \
    --header "X-Vault-Token: ${VAULT_TOKEN}"        \
    ${VAULT_ADDR}/v1/${VAULT_NAMESPACE}/sys/policy  \
  | jq .data.policies | jq -c '.[]' | tr -d '"'
)

for policy in ${policy_list} ; do
  
  # Get the policy details
  policy_details=$(
    curl                                              \
      --silent                                        \
      --request GET                                   \
      --header "X-Vault-Token: ${VAULT_TOKEN}"        \
      ${VAULT_ADDR}/v1/${VAULT_NAMESPACE}/sys/policy/${policy} \
    | jq '.data.rules' | sed 's/\\"/"/g'  | sed 's/\\n/ /g'
  )

  # Check for a name based template key
  if echo "${policy_details}" | grep -q "\.name" ; then
    echo ""
    echo "Policy name: ${policy}"
  fi
  echo "${policy_details}"  |
    tr -s ' ' '\n'          |
    grep "\.name" --label="  template key" -H
done

Compare the returned list of policies and template keys against the list of deduplication targets identified in your system logs to determine if the policy will break after duplication.

If you identify policies that may break during deduplication, consult with the relevant teams to determine if the policies are still used and if you should address the renaming problem before or after deduplication.

If you find policies that need remediation, there are two ways to fix user access:

Update relevant templated ACL policies.
Move the affected resource.

Solution 1: Update relevant ACL policies

The easiest way to deal with broken ACL policy templates due to deduplication is to define new policies or add rules to existing policies that explicitly grants access to previous resources for the affected entities.

If you have a large number of duplicate identities, it may be easier to pre-populate a custom metadata field (e.g., prev_name) for the renamed entities. Then create a single policy that uses the custom metadata field instead of the entity name and attach that policy to the renamed entities.

Assume you have a file called rename-targets.txt with a list of entity names currently attached to a templated ACL policy, which will no longer work after deduplication.

Use vault write and the {namespace}/identity/entity/name/{name} endpoint to update your target entries with a custom metadata field called old_name set to the current entity name:

while read entity_name; do

  if [[ "" = "${entity_name}" ]] ; then continue ; fi

  # Create a payload file with the new metadata field
  echo -n '{"metadata": { "old_name": "'${entity_name}'"}}' > ./metadata.json
      
  # Save the metadata to the entity
  vault write /identity/entity/name/${entity_name} @metadata.json

done < rename-targets.txt

Create a new templatized ACL policy file that replaces the entity name reference with a reference to the metadata field. For example:
```
path "kv/users/{{identity.entity.metadata.old_name}}/*" {
  capabilities = ["read", "create", "update"]
}
```

Use vault policy write to create a new ACL policy with the policy definition file:

$ vault policy write <policy_name> <path_to_policy_file>

For example:

$ vault policy write "kv-access-preservation" ./dedupe-policy.hcl

Use vault read with the {namespace}/identity/entity/name/{name} path to read in the existing policy assignments and add the new policy to the target entities. For example:

policy_name="<policy_name>"
while read entity_name; do

  if [[ "" = "${entity_name}" ]] ; then continue ; fi
  
  # Create a payload file with new policy added to any existing policy
  # assignments
  vault read                              \
    -field policies                       \
    -format json                          \
    /identity/entity/name/${entity_name}  \
    | jq  ". + [\"${policy_name}\"] | {policies: .}" > policy_update.json

  # Update the policy assignment for the entity
  vault write /identity/entity/name/${entity_name} @policy_update.json

done < rename-targets.txt

Assume you have a file called rename-targets.txt with a list of entity names currently attached to a templated ACL policy, which will no longer work after deduplication.

Use the {namespace}/identity/entity/name/{name} endpoint to update your target entries with a custom metadata field called old_name set to the current entity name:

while read entity_name; do

  if [[ "" = "${entity_name}" ]] ; then continue ; fi
  
  # Create a payload file with the new metadata
  echo -n '{"metadata": { "old_name": "'${entity_name}'"}}' > ./metadata.json
  
  # Save the metadata to the entity
  curl                                        \
    --request POST                            \
    --header "X-Vault-Token: ${VAULT_TOKEN}"  \
    --data @./metadata.json                   \
    ${VAULT_ADDR}/v1/${VAULT_NAMESPACE}/identity/entity/name/${entity_name}
  
done < rename-targets.txt

Create a new templatized ACL policy file that replaces the entity name reference with a reference to the metadata field. For example:
```
path "kv/users/{{identity.entity.metadata.old_name}}/*" {
  capabilities = ["read", "create", "update"]
}
```

Escape your policy file and make a POST call to the {namespace}/sys/policy/{policy_name} with your policy details:

$ jq -Rs '{ "policy": . | gsub("[\\r\\n\\t]"; "") }' <path_to_policy_file> |
  curl                                        \
    --request POST                            \
    --header "X-Vault-Token: ${VAULT_TOKEN}"  \
    "$(</dev/stdin)"                          \
    ${VAULT_ADDR}/v1/${VAULT_NAMESPACE}/sys/policy/<policy_name>

For example:

$ jq -Rs '{ "policy": . | gsub("[\\r\\n\\t]"; "") }' ./dedupe-policy.hcl |
  curl                                        \
    --request POST                            \
    --header "X-Vault-Token: ${VAULT_TOKEN}"  \
    --data "$(</dev/stdin)"                   \
    ${VAULT_ADDR}/v1/${VAULT_NAMESPACE}/sys/policy/kv-access-preservation

/sys/mounts/{plugin_mount_path} does not return data on success.

Make a GET call to the {namespace}/identity/entity/name/{name} endpoint to read in the existing policy assignments and add the new policy to the target entities. For example:

policy_name="<policy_name>"
while read entity_name; do

  if [[ "" = "${entity_name}" ]] ; then continue ; fi

  # Create a payload file with new policy added to any existing policy assignments
  curl \
    --request GET                            \
    --header "X-Vault-Token: ${VAULT_TOKEN}"  \
    ${VAULT_ADDR}/v1/${VAULT_NAMESPACE}/identity/entity/name/${entity_name} \
    | jq  ".data.policies + [\"${policy_name}\"] | {policies: .}" > policy_update.json

  # Update the policy assignment for the entity
  curl                                        \
    --request POST                            \
    --header "X-Vault-Token: ${VAULT_TOKEN}"  \
    --data @policy_update.json                \
    ${VAULT_ADDR}/v1/${VAULT_NAMESPACE}/identity/entity/name/${entity_name}
    
done < rename-targets.txt

Solution 2: Move the affected resource

Some resources are immovable and others are difficult to move. If you decide you can, and want, to move resources affected by the renaming:

Inventory the relevant resources and the associated mount paths.
Determine which resources you will migrate before activating the deduplication flag.
- Resources migrating before deduplication will be unavailable on the old and new paths until deduplication completes.
- Resources migrating after deduplication will be unavailable until migration completes.
Move any resources flagged for migration before deduplication and alert the relevant users that the paths will remain unavailable until deduplication completes.
Enable the deduplication option for Vault
Move any resources flagged for migration after deduplication and alert the relevant users that the paths will remain unavailable until migration completes and the changes fully propagate.