Use Copa to patch the container images

calendar May 23, 2024 ·  · copa lima  ·
: linkedin copy

The steps for image patch

  • pull the image use: nerdctl pull xxxxx
  • scan the image vulnerable: TRIVY_DEBUG=true trivy image --timeout 10m --scanners vuln --vuln-type os --ignore-unfixed -f json -o ${JSON_FILE_NAME}.json ${IMAGE}
  • patch the image: copa patch -r ${JSON_FILE_NAME}.json -i $IMAGE -t ${2} --addr unix:///run/user/501/buildkit-default/buildkitd.sock --timeout 20m

Use lima to management the buildkit virtual machine

  • installation the lima
1brew install lima
2# launch virtual machine use template, the template can: template://docker, template://k8s
3limactl create --name=default template://buildkit
4# stop the virtual machine
5limactl stop buildkit
6
7# in buildkit virtual machine run the containerd
8nerdctl run -d -p 0.0.0.0:80:80 --name nginx nginx:latest
  • setting up the network can access from host
 1brew install socket_vmnet
 2# edit the lima network config to add the network and config the vmnet
 3# set the path: socketVMNet: "/opt/homebrew/Cellar/socket_vmnet/1.1.4/bin/socket_vmnet" to socket_vmnet binary path
 4vi $LIMA_HOME/_config/networks.yaml
 5
 6# edit the virtual machine yaml config add network section
 7limactl edit buildkit
 8# networks:
 9#  - lima: shared
10
11limactl sudoers >etc_sudoers.d_lima && sudo install -o root etc_sudoers.d_lima "/private/etc/sudoers.d/lima"
12sudo /usr/libexec/ApplicationFirewall/socketfilterfw --unblock /usr/libexec/bootpd

How to upgrade the package for security patch in Debian os

1# find some packages
2dpkg -l | grep glib2.0
3# find upgrade versions
4apt-cache madison libglib2.0-0:amd64
5# libglib2.0-0 | 2.66.8-1+deb11u3 | http://security.debian.org/debian-security bullseye-security/main amd64 Packages
6# libglib2.0-0 | 2.66.8-1+deb11u1 | http://deb.debian.org/debian bullseye/main amd64 Packages
7# installation the new versions for patch
8apt-get install libglib2.0-0=2.66.8-1+deb11u3

The lima virtual machine config file example

 1# Review and modify the following configuration for Lima instance "buildkit".
 2arch: "x86_64"
 3# - To cancel starting Lima, just save this file as an empty file.
 4
 5# A template to use BuildKit
 6# $ limactl start ./buildkit.yaml
 7
 8# To run `buildkit` on the host (assumes buildctl is installed):
 9# $ export BUILDKIT_HOST=$(limactl list buildkit --format 'unix://{{.Dir}}/sock/buildkitd.sock')
10# $ buildctl debug workers
11message: |
12  To run `buildkit` on the host (assumes buildctl is installed), run the following commands:
13  -------
14  export BUILDKIT_HOST="unix://{{.Dir}}/sock/buildkitd.sock"
15  buildctl debug workers
16  -------  
17images:
18  # Try to use release-yyyyMMdd image if available. Note that release-yyyyMMdd will be removed after several months.
19  - location: "https://cloud-images.ubuntu.com/releases/23.10/release-20240307/ubuntu-23.10-server-cloudimg-amd64.img"
20    arch: "x86_64"
21    digest: "sha256:415123eb3b3ba1841e39a25d0dd82da43f968c7625b9cdf6312235b9b8ec17e9"
22  - location: "https://cloud-images.ubuntu.com/releases/23.10/release-20240307/ubuntu-23.10-server-cloudimg-arm64.img"
23    arch: "aarch64"
24    digest: "sha256:373e8866d33909b283b14c86c18f8a48844c8f9fe6aed3ca280288846fc4fb74"
25  # Fallback to the latest release image.
26  # Hint: run `limactl prune` to invalidate the cache
27  - location: "https://cloud-images.ubuntu.com/releases/23.10/release/ubuntu-23.10-server-cloudimg-amd64.img"
28    arch: "x86_64"
29  - location: "https://cloud-images.ubuntu.com/releases/23.10/release/ubuntu-23.10-server-cloudimg-arm64.img"
30    arch: "aarch64"
31containerd:
32  system: false
33  user: true
34portForwards:
35  - guestSocket: "/run/user/{{.UID}}/buildkit-default/buildkitd.sock"
36    hostSocket: "{{.Dir}}/sock/buildkitd.sock"
37mountType: "reverse-sshfs"
38mounts:
39  - location: "~"
40    sshfs:
41      cache: null
42      followSymlinks: null
43      sftpDriver: null
44    writable: true
45networks:
46  - lima: shared

Use Jenkins to run image scanning

  • Prepare the Scanning Dockerfile to include: Copa, Trivy, AWS Cli
 1FROM docker:27-dind
 2
 3ENV DOCKER_CLI_EXPERIMENTAL=enabled
 4
 5RUN apk update \
 6    && cd /tmp \
 7    &&  apk add aws-cli \
 8    && wget http://dl-cdn.alpinelinux.org/alpine/v3.20/community/x86_64/docker-credential-ecr-login-0.7.1-r19.apk \
 9    && apk add --allow-untrusted docker-credential-ecr-login-0.7.1-r19.apk \ 
10    && wget https://github.com/aquasecurity/trivy/releases/download/v0.47.0/trivy_0.47.0_Linux-64bit.tar.gz  && tar -xzvf trivy_0.47.0_Linux-64bit.tar.gz && mv ./trivy /usr/local/bin/ \
11    && wget https://github.com/project-copacetic/copacetic/releases/download/v0.6.2/copa_0.6.2_linux_amd64.tar.gz && tar -xzvf copa_0.6.2_linux_amd64.tar.gz && mv ./copa /usr/local/bin/ \  
12    && rm -rf /tmp/* \  
13    && mkdir /etc/docker && echo '{ "features": {    "containerd-snapshotter": true  } }' > /etc/docker/daemon.json
14
15RUN mkdir ~/.docker \
16    && echo -e "{\n\"credHelpers\": { \n \"public.ecr.aws\": \"ecr-login\",\n \"xxxxxxxx.dkr.ecr.us-east-1.amazonaws.com\": \"ecr-login\"\n}}" > ~/.docker/config.json

  • Configure the Jenkins clouds and create the Pod tempalte

    • Include the two Containers: docker-dind, jnlp
    • The docker-dind container config:
      • Docker image: from dockerfile build out the dind image in previous
      • Command to run: dockerd-entrypoint.sh
      • Working directory: /home/jenkins/agent
      • Advanced: Run in privileged mode
    • The jnlp container config:
      • Docker image: jenkins/inbound-agent:latest-jdk11
      • Working directory: /home/jenkins/agent
      • Command to run: /usr/local/bin/jenkins-agent

  • Create the Jenkins task and to patch the ECR docker images

 1@Library('common-lib')
 2
 3def scanImage(imageUrl) {
 4    sh """
 5        TRIVY_DEBUG=true trivy image --cache-dir /tmp/trivy/  --timeout 10m --scanners vuln --vuln-type os --ignore-unfixed ${imageUrl}
 6    """
 7}
 8
 9node("patch") {
10    
11    container('docker-dind') {
12       stage "pull the image: ${params.IMAGE_URL}"
13       sh "docker pull ${params.IMAGE_URL}"
14       
15       stage "Scan the vulnerability"
16       def jsonFileName = params.IMAGE_URL.split("/")[-1].replace(":", "-")
17       def oldImageTag = params.IMAGE_URL.split("/")[-1].split(":")[-1]
18       def imageName = params.IMAGE_URL.split(":")[0]
19       def newImageURL = "${imageName}:${oldImageTag}-${PATCH_TAG}"
20       sh """
21            TRIVY_DEBUG=true trivy image --cache-dir /tmp/trivy/  --timeout 10m --scanners vuln --vuln-type os --ignore-unfixed -f json -o ${jsonFileName}.json ${params.IMAGE_URL}
22       """
23       
24       stage "Patch the image: ${params.IMAGE_URL}"
25       sh """
26         copa patch -r ${jsonFileName}.json -i ${params.IMAGE_URL} -t ${oldImageTag}-${PATCH_TAG} --timeout 20m
27       """
28       stage "Rescaning the new image: ${newImageURL}"
29       scanImage(newImageURL)
30       
31       stage "Push the new image: ${newImageURL}"
32       sh """
33        docker push ${newImageURL}
34       """
35        
36    }
37}

Reference