Not able to prune the old upstream CA cert from the trust bundle.

[dzhou3]

I’m testing the forceful rotation of the upstream CA and have encountered an edge case where I’m unable to remove the old upstream CA certificate from the trust bundle. Here are the steps to reproduce the issue (with the spire-server version v1.12.0):

1. set up a simple standalone spire-server with the UpstreamAuthority disk:

root@host:/# spire-server localauthority x509 show
Active X.509 authority:
  Authority ID: 359090c204f106e4874748dcc3526dc45e9f2f2e
  Expires at: 2026-01-13 16:15:06 +0000 UTC
  Upstream authority Subject Key ID: c26a906bede5ee6f191d1ebefd44ef4d6cc07328

Prepared X.509 authority:
  No prepared X.509 authority found

Old X.509 authority:
  No old X.509 authority found

2. update the upstream CA cert and key, and prepare and activate the next local authority:

root@host:/# spire-server localauthority x509 prepare
Prepared X.509 authority:
  Authority ID: 4879c3e772f9c13a3daab5d5aebcd1e507719fb0
  Expires at: 2026-01-13 16:16:25 +0000 UTC
  Upstream authority Subject Key ID: 0d0a593db15165ed12b682a20ee7264863a189eb
root@host:/# spire-server localauthority x509 activate -authorityID 4879c3e772f9c13a3daab5d5aebcd1e507719fb0
Activated X.509 authority:
  Authority ID: 4879c3e772f9c13a3daab5d5aebcd1e507719fb0
  Expires at: 2026-01-13 16:16:25 +0000 UTC
  Upstream authority Subject Key ID: 0d0a593db15165ed12b682a20ee7264863a189eb
root@host:/# spire-server localauthority x509 show
Active X.509 authority:
  Authority ID: 4879c3e772f9c13a3daab5d5aebcd1e507719fb0
  Expires at: 2026-01-13 16:16:25 +0000 UTC
  Upstream authority Subject Key ID: 0d0a593db15165ed12b682a20ee7264863a189eb

Prepared X.509 authority:
  No prepared X.509 authority found

Old X.509 authority:
  Authority ID: 359090c204f106e4874748dcc3526dc45e9f2f2e
  Expires at: 2026-01-13 16:15:06 +0000 UTC
  Upstream authority Subject Key ID: c26a906bede5ee6f191d1ebefd44ef4d6cc07328

3. without tainting and revoking the old upstream authority, prepare the next local authority with the new upstream authority again:

root@host:/# spire-server localauthority x509 prepare
Prepared X.509 authority:
  Authority ID: 0e301d402321c4e601848810a26f4f7e5b82738b
  Expires at: 2026-01-13 16:17:55 +0000 UTC
  Upstream authority Subject Key ID: 0d0a593db15165ed12b682a20ee7264863a189eb
root@host:/# spire-server localauthority x509 show
Active X.509 authority:
  Authority ID: 4879c3e772f9c13a3daab5d5aebcd1e507719fb0
  Expires at: 2026-01-13 16:16:25 +0000 UTC
  Upstream authority Subject Key ID: 0d0a593db15165ed12b682a20ee7264863a189eb

Prepared X.509 authority:
  Authority ID: 0e301d402321c4e601848810a26f4f7e5b82738b
  Expires at: 2026-01-13 16:17:55 +0000 UTC
  Upstream authority Subject Key ID: 0d0a593db15165ed12b682a20ee7264863a189eb

Old X.509 authority:
  No old X.509 authority found

4. now, the trust bundle still has both old and new upstream CA certs, and I can’t taint and revoke the old one:

root@host:/# spire-server bundle show | grep "BEGIN CERTIFICATE"
-----BEGIN CERTIFICATE-----
-----BEGIN CERTIFICATE-----
root@host:/# spire-server upstreamauthority taint -subjectKeyID c26a906bede5ee6f191d1ebefd44ef4d6cc07328
Error: could not taint X.509 upstream authority: rpc error: code = InvalidArgument desc = provided subject key id is not valid: upstream authority didn't sign the old local authority
root@host:/# spire-server upstreamauthority revoke  -subjectKeyID c26a906bede5ee6f191d1ebefd44ef4d6cc07328
Error: could not revoke X.509 upstream authority: rpc error: code = InvalidArgument desc = invalid subject key ID: upstream authority didn't sign the old local authority

Am I misunderstanding how to use the localauthority and upstreamauthority APIs? What’s the correct approach for removing the old upstream CA certificate from the trust bundle if my prod setup has already been in this state?