seal key process ignores region value

[stanych23]

Hi,

I use a single AWS KMS key stored in us-east-1 region for all clusters that I run in different us regions.
This way everything works good, until I recently migrated from vault 1.14.0 on 1.15.1. Now I'm getting below error:

error parsing Seal configuration: error fetching AWS KMS wrapping key information: 
NotFoundException: Alias arn:aws:kms:us-west-2:<account_id>:alias/vault-us is not found.

My seal section under vault configmap file (for both healthy and affected clusters):

seal "awskms" {
      region = "us-east-1"
      kms_key_id = "alias/vault-us"
    }

I tried to test with a totally incorrect seal configuration (using EU regions) for the healthy cluster and it still works:

seal "awskms" {
      region = "eu-central-1"
      kms_key_id = "alias/vault-us"
    }

Vault statefulset has a AWS_REGION set to us-west-2.

Expected behavior:

• Seal process should search an alias with a specified region set in seal section of the configmap file. It should look for a kms key like:
  arn:aws:kms:us-east-1:<account_id>:alias/vault-us

Current behavior:

• despite the seal configuration vault is looking for a key in the default region for this cluster:
  despite

error parsing Seal configuration: error fetching AWS KMS wrapping key information: 
NotFoundException: Alias arn:aws:kms:us-west-2:<account_id>:alias/vault-us is not found.

I suspect that the default region overwrite region value set in configmap.

Could you please advise if this an issue or are there any other setting that might help me?
Thank you

[stevendpclark]

Hello @stanych23,

So I looked into this a bit, I can verify that in 1.15.x we seem to prefer the environment variable AWS_REGION over the configured values which is a change from 1.14.x, that's probably a mistake but I'll confirm with the team.

Can you see if you have the environment variable set or is that what you meant by?

Vault statefulset has a AWS_REGION set to us-west-2.

In either case could you validate, if you remove the environment variable does Vault then use the configured value for region?

[hagaimaya]

@stevendpclark
I have encountered the same issue.
I cannot remove the AWS_REGION because we are using EKS and it is injected by default.
will be a solution for this?

[Slevy35]

we encountered this issue too, we can't upgrade.
any solution or workaround?

[Slevy35]

who can review my pr?