More Related Content
Similar to ContainerConf 2022: Hijack Kubernetes
Similar to ContainerConf 2022: Hijack Kubernetes (20)
More from Nico Meisenzahl
More from Nico Meisenzahl (17)
ContainerConf 2022: Hijack Kubernetes
- 2. Nico Meisenzahl
• Head of DevOps Consulting & Operations
at white duck
• Microsoft MVP, GitLab Hero
• Cloud Native, Kubernetes & Azure
© white duck GmbH 2022
Email: nico.meisenzahl@whiteduck.de
Twitter: @nmeisenzahl
LinkedIn: https://www.linkedin.com/in/nicomeisenzahl
Blog: https://meisenzahl.org
- 3. About this talk
• this is not an in-depth security talk
• it should make you aware of common attack vectors and
how to prevent them
• you will see demos on how to hijack a cluster
• you will learn how to prevent those with common best practices
• three more slide, then we will start hijacking
• https://github.com/nmeisenzahl/hijack-kubernetes
© white duck GmbH 2022
- 4. Why do we need to care about security?
https://www.redhat.com/en/resources/state-kubernetes-security-report
- 7. Think about
• ensure secure application / deployment code
• build secure container images
• implement Kubernetes policies
• introduce Kubernetes Network policies
• rely on Container Runtime Security
• many more…
© white duck GmbH 2022
- 9. Ensure secure application code
• automate and enforce code checks
• schedule dependency scanning
• Software Bill of Materials (SBOM)
• Dependabot / Renovate
• enforce Static Application Security Testing (SAST) in PRs
• scans your code to identify potential security vulnerabilities
• more details: https://owasp.org/www-
community/Source_Code_Analysis_Tools
© white duck GmbH 2022
- 10. Build secure container images
• build secure/small container images – less is more
• do only include required dependencies (no debugging tools!)
• use self-contained binaries, “distroless” or “(Un)distro” if
possible
• https://github.com/GoogleContainerTools/distroless
• https://github.com/wolfi-dev/os
• otherwise, use a small and secure Linux distro
• use and enforce SAST for validating your Dockerfiles
• scan your container images (on build and regularly)
© white duck GmbH 2022
- 11. Build secure container images
• build secure/small container images – less is more
• do only include required dependencies (no debugging tools!)
• use self-contained binaries, “distroless” or “(Un)distro” if
possible
• https://github.com/GoogleContainerTools/distroless
• https://github.com/wolfi-dev/os
• otherwise, use a small and secure Linux distro
• use and enforce SAST for validating your Dockerfiles
• scan your container images (on build and regularly)
© white duck GmbH 2022
Would have made it
much harder to
hijack the container
and further expend
Would have shown
the possibility of code
injection
- 12. Ensure secure deployment code
• as important as secure application code and Dockerfiles
• validate your deployment manifests using SAST
• and enforce them via PRs
• can help you to implement best practices like denying
• containers running as root
• mounting hostPath
• …
© white duck GmbH 2022
- 13. Ensure secure deployment code
• as important as secure application code and Dockerfiles
• validate your deployment manifests using SAST
• and enforce them via PRs
• can help you to implement best practices like denying
• containers running as root
• mounting hostPath
• …
© white duck GmbH 2022
Would have made it
much harder to hijack
the node
- 14. Tooling
• Source code
• https://codeql.github.com
• https://security-code-scan.github.io
• https://securego.io
• SBOM
• https://github.com/anchore/syft
• https://github.com/anchore/grype
• Dockerfiles
• https://github.com/aquasecurity/trivy
• https://github.com/bridgecrewio/checkov
• Kubernetes manifests
• https://kubesec.io
• https://github.com/aquasecurity/trivy
• https://github.com/bridgecrewio/checkov
• https://github.com/Checkmarx/kics
• Terraform
• https://github.com/tfsec/tfsec
• https://github.com/aquasecurity/trivy
• https://github.com/bridgecrewio/checkov
© white duck GmbH 2022
- 15. Kubernetes policies
• enforce compliance and governance within clusters
• verifying manifests is not enough!
• examples include enforcement of
• read-only filesystems
• denying hostPath mounts
• denying containers running as root
• …
© white duck GmbH 2022
- 16. Kubernetes policies
• enforce compliance and governance within clusters
• verifying manifests is not enough!
• examples include enforcement of
• read-only filesystems
• denying hostPath mounts
• denying containers running as root
• …
© white duck GmbH 2022
Would have made it
much harder to
further hijack the
nodes and cloud
resources
- 17. Kubernetes policy tooling
• Pod Security Admission
• stable since 1.25
• https://kubernetes.io/docs/concepts/security/pod-security-
admission
• Open Policy Agent Gatekeeper
• https://github.com/open-policy-agent/gatekeeper
• Kyverno
• https://kyverno.io
© white duck GmbH 2022
- 18. Network Policies
• granular deny or explicitly allow between containers and
ingress/egress of the cluster
• limit egress access to the internet
• limit access between applications/namespaces
• deny access to the Cloud provider metadata service
• https://kubernetes.io/docs/concepts/services-
networking/network-policies
© white duck GmbH 2022
- 19. Network Policies
• granular deny or explicitly allow between containers and
ingress/egress of the cluster
• limit egress access to the internet
• limit access between applications/namespaces
• deny access to the Cloud provider metadata service
• https://kubernetes.io/docs/concepts/services-
networking/network-policies
© white duck GmbH 2022
Would have denied
network connections
(reverse shell, Redis,
Internet, metadata
service)
- 20. Container Runtime Security
• helps to detect malicious threads and workloads
• untrusted process within container
• a shell is running inside a container
• container process mounting a sensitive path
• a process making outbound network connections
• container runtime security tools like Falco of Tetragon can
help
• https://github.com/falcosecurity
• https://github.com/cilium/tetragon
© white duck GmbH 2022
- 21. Container Runtime Security
• helps to detect malicious threads and workloads
• untrusted process within container
• a shell is running inside a container
• container process mounting a sensitive path
• a process making outbound network connections
• container runtime security tools like Falco of Tetragon can
help
• https://github.com/falcosecurity
• https://github.com/cilium/tetragon
© white duck GmbH 2022
Would have detect all
our “work” within the
containers
- 22. Further best practises
• do not
• share service accounts between applications
• enable higher access levels for the default service account if not
required
• mount service account token if not required
• https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-
account/#use-the-default-service-account-to-access-the-api-server
• changed with 1.24
• review all third-party snippets before applying them
• implement a Web Application Firewall (WAF) to further secure
your application
© white duck GmbH 2022
- 23. Further best practises
• do not
• share service accounts between applications
• enable higher access levels for the default service account if not
required
• mount service account token if not required
• https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-
account/#use-the-default-service-account-to-access-the-api-server
• changed with 1.24
• review all third-party snippets before applying them
• implement a Web Application Firewall (WAF) to further secure
your application
© white duck GmbH 2022
Wouldn’t have
allowed us to talk to
the API server
Would have denied
our code injection