Dynamic create docker registry pull image credentials in eks cluster for particular imagePullSecret
When use imagePullSecret to pull image from ecr in eks cluster for particular task etc. Istio WSAM plugin from ecr registry, need specific imagePullSecret
Basic workflow
- create cronjob in eks cluster schedule login to ecr and obtain credentials
- create a secrets in some Namespaces named
aws-registry
save the docker pull credentals - use this secrets in Namespace pull image specific
imagePullSecret
parameter - for example:
1 spec:
2 imagePullPolicy: Always
3 imagePullSecret: aws-registry
The Dockerfile
to construct cronjob docker image
1FROM python:alpine
2MAINTAINER Mike Petersen <[email protected]>
3
4RUN apk --no-cache add curl
5ADD run.sh /run.sh
6
7# Install kubectl
8RUN curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl \
9 && mv kubectl /usr/local/bin \
10 && chmod +x /usr/local/bin/kubectl
11
12RUN adduser -S user
13USER user
14WORKDIR /home/user
15ENV PATH /usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/user/.local/bin
16
17# Install awscli
18RUN pip install awscli --upgrade --user
The Run.sh
to run shell script
1#!/usr/bin/env sh
2set -e
3
4echo "Retrieving Docker Credentials for the AWS ECR Registry ${AWS_ACCOUNT}"
5DOCKER_REGISTRY_SERVER=https://${AWS_ACCOUNT}.dkr.ecr.${AWS_REGION}.amazonaws.com
6DOCKER_USER=AWS
7DOCKER_PASSWORD=`aws ecr get-login --region ${AWS_REGION} --registry-ids ${AWS_ACCOUNT} | cut -d' ' -f6`
8
9for namespace in ${NAMESPACES}
10do
11 echo
12 echo "Working in Namespace ${namespace}"
13 echo
14 echo "Removing previous secret in namespace ${namespace}"
15 kubectl --namespace=${namespace} delete secret aws-registry || true
16
17 echo "Creating new secret in namespace ${namespace}"
18 kubectl create secret docker-registry aws-registry \
19 --docker-server=$DOCKER_REGISTRY_SERVER \
20 --docker-username=$DOCKER_USER \
21 --docker-password=$DOCKER_PASSWORD \
22 --docker-email=[email protected] \
23 --namespace=${namespace}
24 echo
25 echo
26done
27
28echo "Patching default serviceaccount"
29echo kubectl patch serviceaccount default -p '{"imagePullSecrets":[{"name":"aws-registry"}]}'
Create the Kubernetes relative resources
1apiVersion: v1
2kind: Secret
3metadata:
4 name: ecr-registry-helper-secrets
5 namespace: kube-system
6stringData:
7 AWS_ACCOUNT: "xxxxxxx"
8---
9apiVersion: v1
10kind: ConfigMap
11metadata:
12 name: ecr-registry-helper-cm
13 namespace: kube-system
14data:
15 AWS_REGION: "us-east-1"
16 DOCKER_SECRET_NAME: aws-registry
17 NAMESPACES: "default kube-system example"
18---
19apiVersion: batch/v1
20kind: CronJob
21metadata:
22 name: ecr-registry-helper
23 namespace: kube-system
24spec:
25 schedule: "0 */10 * * *"
26 successfulJobsHistoryLimit: 3
27 suspend: false
28 jobTemplate:
29 spec:
30 template:
31 spec:
32 serviceAccountName: sa-aws-ecr
33 containers:
34 - name: ecr-registry-helper
35 image: xxxxx.dkr.ecr.us-east-1.amazonaws.com/aws-kubectl:v1
36 imagePullPolicy: IfNotPresent
37 envFrom:
38 - secretRef:
39 name: ecr-registry-helper-secrets
40 - configMapRef:
41 name: ecr-registry-helper-cm
42 command:
43 - /run.sh
44 restartPolicy: Never
45---
46apiVersion: v1
47kind: ServiceAccount
48metadata:
49 name: sa-aws-ecr
50 namespace: kube-system
51---
52apiVersion: rbac.authorization.k8s.io/v1
53kind: ClusterRole
54metadata:
55 name: role-full-access-to-secrets
56rules:
57- apiGroups: ["*"]
58 resources: ["secrets"]
59 resourceNames: ["aws-registry"]
60 verbs: ["delete"]
61- apiGroups: ["*"]
62 resources: ["secrets"]
63 verbs: ["create"]
64---
65kind: ClusterRoleBinding
66apiVersion: rbac.authorization.k8s.io/v1
67metadata:
68 name: health-check-role-binding
69subjects:
70- kind: ServiceAccount
71 name: sa-aws-ecr
72 namespace: kube-system
73roleRef:
74 kind: ClusterRole
75 name: role-full-access-to-secrets
76 apiGroup: rbac.authorization.k8s.io
77---
Apply these resources to create cronjob to schedule update specific imagePullSecret
1kubectl apply -f ./ecr-credentials-helper.yaml