Skip to content

Commit d2d6039

Browse files
jqmichaeljqmichael
authored
Merge pull request #7 from micahhausler/self-hosted-guide
Added self-hosted cluster setup guide
2 parents 2fd4bec + 2becc07 commit d2d6039

File tree

3 files changed

+221
-0
lines changed

3 files changed

+221
-0
lines changed

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,8 @@ This will:
129129
* Create the deployment, service, and mutating webhook in the cluster
130130
* Approve the CSR that the deployment created for its TLS serving certificate
131131
132+
For self-hosted API server configuration, see see [SELF_HOSTED_SETUP.md](/SELF_HOSTED_SETUP.md)
133+
132134
### On API server
133135
TODO
134136

‎SELF_HOSTED_SETUP.md‎

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
# Self-hosted Kubernetes setup
2+
3+
If you are running your own Kubernetes cluster, there are several steps required for this feature to work.
4+
5+
This feature requires Kubernetes 1.12 or greater.
6+
7+
## Projected Token Signing Keypair
8+
9+
The first thing required is a new key pair for signing and verifying projected
10+
service account tokens. This can be done using the following `ssh-keygen`
11+
commands.
12+
13+
```bash
14+
# Generate the keypair
15+
PRIV_KEY="sa-signer.key"
16+
PUB_KEY="sa-signer.key.pub"
17+
PKCS_KEY="sa-signer-pkcs8.pub"
18+
# Generate a key pair
19+
ssh-keygen -t rsa -b 2048 -f $PRIV_KEY -m pem
20+
# convert the SSH pubkey to PKCS8
21+
ssh-keygen -e -m PKCS8 -f $PUB_KEY > $PKCS_KEY
22+
```
23+
24+
## Public Issuer
25+
26+
As of 1.16, Kubernetes does not include an OIDC discovery endpoint itself (see
27+
[kubernetes/community#1190](https://github.com/kubernetes/enhancements/pull/1190)),
28+
so you will need to put your public signing key somewhere that AWS STS can
29+
discover it. This example, we will create one in a public S3 bucket, but you
30+
could host the following documents any way you'd like on a different domain.
31+
32+
### Create an S3 bucket
33+
34+
```bash
35+
# Create S3 bucket with a random name. Feel free to set your own name here
36+
export S3_BUCKET=${S3_BUCKET:-oidc-test-$(cat /dev/random | LC_ALL=C tr -dc "[:alpha:]" | tr '[:upper:]' '[:lower:]' | head -c 32)}
37+
# Create the bucket if it doesn't exist
38+
_bucket_name=$(aws s3api list-buckets --query "Buckets[?Name=='$S3_BUCKET'].Name | [0]" --out text)
39+
if [ $_bucket_name == "None" ]; then
40+
if [ "$AWS_REGION" == "us-east-1" ]; then
41+
aws s3api create-bucket --bucket $S3_BUCKET
42+
else
43+
aws s3api create-bucket --bucket $S3_BUCKET --create-bucket-configuration LocationConstraint=$AWS_REGION
44+
fi
45+
fi
46+
echo "export S3_BUCKET=$S3_BUCKET"
47+
export HOSTNAME=s3-$AWS_REGION.amazonaws.com
48+
export ISSUER_HOSTPATH=$HOSTNAME/$S3_BUCKET
49+
```
50+
51+
### Create the OIDC discovery and keys documents
52+
53+
Part of the OIDC spec is to host an OIDC discovery and a keys JSON document.
54+
Lets create these:
55+
56+
```bash
57+
# Get the sha of the public key and use it as the key id
58+
KID=$(sha1sum $PUB_KEY | awk '{print $1}')
59+
cat <<EOF > discovery.json
60+
{
61+
"issuer": "https://$ISSUER_HOSTPATH/",
62+
"jwks_uri": "https://$ISSUER_HOSTPATH/keys.json",
63+
"authorization_endpoint": "urn:kubernetes:programmatic_authorization",
64+
"response_types_supported": [
65+
"id_token"
66+
],
67+
"subject_types_supported": [
68+
"public"
69+
],
70+
"id_token_signing_alg_values_supported": [
71+
"RS256"
72+
],
73+
"claims_supported": [
74+
"sub",
75+
"iss"
76+
]
77+
}
78+
EOF
79+
```
80+
81+
Included in this repo is a small go file to help create the keys json document.
82+
83+
```bash
84+
go run ./hack/self-hosted/main.go -key $PKCS_KEY -kid $KID > keys.json
85+
```
86+
87+
After you have the `keys.json` and `discovery.json` files, you'll need to place
88+
them in your bucket. It is critical these objects are public so STS can access
89+
them.
90+
91+
```bash
92+
aws s3 cp --acl public-read ./discovery.json s3://$S3_BUCKET/.well-known/openid-configuration
93+
aws s3 cp --acl public-read ./keys.json s3://$S3_BUCKET/keys.json
94+
```
95+
96+
## Kubernetes API Server configuration
97+
98+
As of Kubernetes 1.12, Kubernetes can issue and mount projected service account
99+
tokens in pods.
100+
101+
In order to use this feature, you'll need to set the following
102+
[API server flags](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/).
103+
104+
```
105+
# This flag is likely already specified for legacy service accounts, you can
106+
# specify this flag multiple times, and you'll need to add this with the path
107+
# to the $PUB_KEY file from the beginning
108+
--service-account-key-file
109+
110+
# Path to the signing (private) key ($PRIV_KEY)
111+
--service-account-signing-key-file
112+
113+
# Identifiers of the API. The service account token authenticator will validate
114+
# that tokens used against the API are bound to at least one of these audiences.
115+
# If the --service-account-issuer flag is configured and this flag is not, this
116+
# field defaults to a single element list containing the issuer URL.
117+
#
118+
# `--api-audiences` is for v1.13+, `--service-account-api-audiences` in v1.12
119+
--api-audiences
120+
121+
# The issuer URL, or "https://$ISSUER_HOSTPATH" from above.
122+
--service-account-issuer
123+
```
124+
125+
## Audiences
126+
127+
The above `--api-audiences` flag sets an `aud` value for tokens that do not
128+
request an audience, and the API server requires that any projected tokens used
129+
for pod to API server authentication must have this audience set. This can
130+
usually be set to `kubernetes.svc.default`, or optionally the DNS name of your
131+
API server.
132+
133+
When using a Kubernetes-issued token for an external system, you should use a
134+
different audience (or in OAuth-2 parlance, `client-id`). The external system
135+
(such as AWS IAM) will usually require an audience, or client-id, at setup. For
136+
AWS IAM, a token's `aud` must match the OIDC Identity Provider's client ID. EKS
137+
uses the string `sts.amazonaws.com` as the default, but when using the webhook
138+
yourself, you can use any audience you'd like as long as the webhook's flag
139+
`--token-audience` is set to the same value as your IDP in IAM.
140+
141+
## Provider creation
142+
143+
From here, you can mostly follow the process in the [EKS
144+
documentation](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html)
145+
and substitue the cluster issuer with `https://$ISSUER_HOSTPATH`.

‎hack/self-hosted/main.go‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
package main
2+
3+
import (
4+
"crypto/rsa"
5+
"crypto/x509"
6+
"encoding/json"
7+
"encoding/pem"
8+
"flag"
9+
"fmt"
10+
"io/ioutil"
11+
"os"
12+
13+
"github.com/pkg/errors"
14+
jose "gopkg.in/square/go-jose.v2"
15+
)
16+
17+
type KeyResponse struct {
18+
Keys []jose.JSONWebKey `json:"keys"`
19+
}
20+
21+
func readKey(keyID, filename string) ([]byte, error) {
22+
var response []byte
23+
content, err := ioutil.ReadFile(filename)
24+
if err != nil {
25+
return response, errors.WithMessage(err, "error reading file")
26+
}
27+
28+
block, _ := pem.Decode(content)
29+
if block == nil {
30+
return response, errors.Errorf("Error decoding PEM file %s", filename)
31+
}
32+
33+
pubKey, err := x509.ParsePKIXPublicKey(block.Bytes)
34+
if err != nil {
35+
return response, errors.Wrapf(err, "Error parsing key content of %s", filename)
36+
}
37+
switch pubKey.(type) {
38+
case *rsa.PublicKey:
39+
default:
40+
return response, errors.New("Public key was not RSA")
41+
}
42+
43+
var alg jose.SignatureAlgorithm
44+
switch pubKey.(type) {
45+
case *rsa.PublicKey:
46+
alg = jose.RS256
47+
default:
48+
return response, fmt.Errorf("invalid public key type %T, must be *rsa.PrivateKey", pubKey)
49+
}
50+
51+
var keys []jose.JSONWebKey
52+
keys = append(keys, jose.JSONWebKey{
53+
Key: pubKey,
54+
KeyID: keyID,
55+
Algorithm: string(alg),
56+
Use: "sig",
57+
})
58+
59+
keyResponse := KeyResponse{Keys: keys}
60+
return json.MarshalIndent(keyResponse, "", " ")
61+
}
62+
63+
func main() {
64+
kid := flag.String("kid", "", "The Key ID")
65+
keyFile := flag.String("key", "", "The public key input file in PKCS8 format")
66+
flag.Parse()
67+
68+
output, err := readKey(*kid, *keyFile)
69+
if err != nil {
70+
fmt.Println(err.Error())
71+
os.Exit(1)
72+
}
73+
fmt.Println(string(output))
74+
}

0 commit comments

Comments
 (0)