In this blog we will go through the security features of Kubernetes version 1.30. The previous version 1.29, was one of the most robust in terms of security features, with a focus on improving uptime and reliability for SREs, as well as security-specific improvements for policy enforcement, access, secrets, and security-related plugins. In version 1.30, there are multiple follow-ups on security features from the previous versions, as well as many features now in GA.
Make Kubernetes aware of the LoadBalancer behavior - now in Beta
Cloud Dual-Stack --node-ip Handling now in GA
Reduction of secret-based service account tokens in GA
Bound service account token improvements in beta
Common Expression Language (CEL) for Admission Control & CRD Validation Expression Language in GA
Support User Namespaces in Beta
Split Image Filesystem into readable and writeable Still in Alpha
Structured Authorization Configuration now in Beta
Custom profiling support in kubectl debug command in alpha
Interactive(-i) flag to kubectl delete for user confirmation now in GA
For SREs
Make Kubernetes aware of the LoadBalancer behavior - now in beta
Released as alpha in Kubernetes version 1.29, this capability will be enabled by default. When interfacing with a cloud provider, today, the kube-proxy is either binding the External IPs of the LoadBalancer Service to each node or redirecting packets to go around the LoadBalancer, which doesn’t honor the intended behavior of the load balancer. This could result in issues like inaccurate health checks, or bypassing cloud features that act at the level of the load balancer.
The only current solution is to employ a workaround that enables the cloud provider to disable the kube-proxy’s actions. With this enhancement, the need for hacks and workarounds would be removed, lowering overall risk of performance implications. In this enhancement, the cloud provider would be able to more accurately interface with the kube-proxy so the LoadBalancer wouldn’t have to be bypassed or bound to each node via External IP.
Cloud Dual-Stack --node-ip Handling now in GA
This enhancement is all about making dual-stack a possibility when Kubernetes is deployed with a cloud provider. Dual-stack support in Kubernetes means simultaneous support for IPv4 and IPv6 in networking. Today, the kubelet supports dual-stack behavior in a non cloud environment, but not when it comes to cloud providers.
Node Log Query
This feature has been in development in alpha, since version 1.27, and is now in beta in 1.30 with the release of a new kubectl plugin. Previously, for cluster administrators to view the logs of the API server or the Kubelet they had to log directly into the nodes (e.g. via SSH or a client side reader). This complicates the process of troubleshooting and log gathering. This enhancement proposes an API or plugin to be able to capture these logs, similar to how an administrator interacts with the rest of the cluster, which will improve security, as well as efficiency and ease of use.
For Security
Reduction of secret-based service account tokens in GA
This enhancement has been in progress since version 1.24, and is now being released in GA. The previous state of generating secret-based service account tokens was less secure than the bound token option introduced in version 1.22. This enhancement aimed to reduce the exposure created by the generation of secret-based service account tokens. For example, it aims to remove unused, auto-generated secret-based service account tokens, and the auto-creation of secrets for service accounts.
Bound service account token improvements in beta
This enhancement is now in beta, with docs available around the service account JWT schema and decisions on the specific approach. https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/4193-bound-service-account-token-improvements
Service accounts are the machine or API users in RBAC - aka the most likely ones to be overlooked or forgotten. This enhancement allows for better tracking and quicker transparency into the token usage of service accounts. For example, this enhancement allows somebody to understand who made an original access request from a service account through the token; by binding info about the node for which the token is being used, to the JWT token itself.
This means it’s easier to check if the host identity is indeed the one tied to the token, and determine where the original request for a serviceaccount token came from, without having to perform complicated cross-referencing.
Common Expression Language (CEL) for Admission Control & CRD Validation Expression Language in GA
This enhancement is now in GA, it started as alpha in version 1.26, as an alternative way to support validation rules for CRDs. Validation rules for CRDs mean that CRD creation can be approved or denied based on validation rules instead of complex webhooks. In version 1.28, there were more improvements made to the capability, and in version 1.29 the goal was to generally to provide an alternative to webhooks with in-tree extensions, and to make this core functionality available in an easily consumable way in GitOps and the CI/CD process.
For the CRD piece, the goal is to allow for validating policies to be handled in-tree for the CRD, versus using webhooks to apply validation rules for a CRD. Our blog for Kubernetes version 1.28 goes into detail around the benefits and drawbacks of webhooks versus admission policy.
The work done toward getting to GA included a scalability analysis, defining the policy for the CEL library update, and fixing bugs.
Support User Namespaces in beta
This feature was introduced in alpha in version 1.29 and is now in beta. Today, multiple over permissioning issues arise in Kubernetes because there are no different user/group definitions or distinctions from inside the container versus on the node. For example, to have privileged capabilities with cluster admin, you are also granted extra privileges on the host or other pods. Or there isn’t a way to take an action on the host just within a certain namespace. Or to run a container as root you also get privileges on the host.
In this enhancement, the goal is to use user and group IDs that can map from the container to the host, improving isolation. For example, mapping root privileges in the container to unprivileged user and group IDs in the node so you don’t automatically get root privileges in the container and privileged access on the host. A side benefit of this is then that pods could have capabilities that would only be valid in the pod, instead of always having some 1:1 mapping of capabilities in the pod across to the container.
There have been multiple vulnerabilities caused by the broad permission granted across containers and pods, that could be remedied by enabling or disabling user namespaces in the pod.spec, as well as supporting pods inside user namespaces.
An additional Beta feature includes configuring user and group ID ranges applicable to pods as well as tests for enabling/disabling the feature.
Split Image Filesystem into readable and writeable Still in alpha
This capability was released as alpha in 1.29, and is still in alpha in 1.30. In Kubernetes, there is a layer that doesn’t change very much (the image layer), which is referred to as the readable layer, and the container and node layers, which can be referred to as the writable layer, because it contains the temporary files required for the kubelet to run.
The goal is to enhance the performance of the kubelet by separating readable and writable file systems, while ensuring the writable files required for running the kubelet are available on the kubelet itself.
This enhancement could prevent an unwanted decline in performance of the Kubelet and make garbage collecting more transparent, because today when the image disk gets too full the garbage collector doesn’t have any context on which files it is taking from, as they are both stored in the same place.
Structured Authorization Configuration now in beta
This enhancement adds more capabilities for authorization webhooks to the API server, for example:
- Cluster administrators can add more than one webhook to the authorization chain
- You could configure the authorization requirements within a config file
- It is possible to create clear guidelines and power for denying a request (or skipping a part in the chain)
The proposal includes defining a config file format that would contain the relevant information about the overall authorization chain, while allowing for implementations that differ based on users and resources.
The main functionality was added in version 1.29, and in this version new features include observability and metrics and a default feature flag.
Structured Authentication Configuration in beta
In addition to the changes to authorization above, there are proposed changes to authentication into the Kubernetes API server using OIDC. Today the configuration is flag-based, which doesn’t allow for the growing number of features being asked for, such as adding extra attributes to a user based on their ID token and then forwarding them to an authorization profile based on that.There is also the goal to be able to use more than one OIDC provider.
Custom profiling support in kubectl debug command in alpha
The kubectl debug command deploys a pod to a node that you want to troubleshoot.The purpose of this enhancement is to allow more flexibility for the profiles of the pods that are deployed in this process, beyond a few defined debug profiles. In this debug profile, for example, the pod can access the node’s root file system. Despite these profiles, users still want a way to specify environment variables, security contexts, etc. that should be implemented with these new pods. Currently, to do this, the pod that is deployed needs to be patched manually but at scale this is hard.
The new feature proposes the ability to create custom profiles for pods deployed into nodes for the kubectl debug command.
SE Linux Relabeling in alpha
This enhancement proposes to improve the way files are relabeled with Kubernetes when SELinux is enabled on the system. SELinux is a Linux security kernel module. This feature improves the compatibility of file relabeling with the Kubernetes Container Runtime Interface (CRI) when done on a system running SELinux, allowing users to take advantage of the SELinux security functions.
There are two elements to this:
- The SELinuxMount feature, which is the capability in alpha, and allows for the usage of SELinux context instead of recursive relabeling as a performance optimization
- The SELinuxMountReadWriteOncePod, which is still in beta since version1.27
Admission Webhook Match Conditions in GA
For those webhooks that cannot be migrated, this feature, in GA in version 1.30, allows administrators to more narrowly scope and target webhooks with additional CEL expressions, called “match conditions,” that control whether a webhook is initiated. This improves performance, reliability and also ease of support with managed platforms.
Prevent unauthorized volume mode conversion during volume restore now in GA
PersistentVolumes are a cluster resource and an important part of Kubernetes storage and the CSI driver. There have been many storage-related Kubernetes vulnerabilities in 2023, including the CSI driver Kubernetes vulnerability and a Kubernetes vulnerability in the in-tree plugins for Windows nodes.
To-date, to access persistent volumes, the options are limited for use cases of multiple users needing access on the same node.
This feature was introduced as an alpha feature in version 1.24 to fix an issue created by the VolumeSnapshot feature from version 1.20, where a PersistentVolumeClaim (PVC) could be made from a previously taken snapshot without validating whether the two match. This could easily be taken advantage of by an attacker, to add in malicious script and pose as a previous (non-malicious) PVC.
In GA, the feature adds an access gate that can determine whether a previously generated snapshot can be altered, using a volume mode and a new annotation that are now applied to it.
ReadWriteOncePod PersistentVolume AccessMode is also a feature for PersistentVolumes that went GA in version 1.30.
Recursive Read-Only (RRO) Mounts in alpha
A bind mount is when a directory tree is mounted, or saved into another place. Previously, when mounting a volume to a container with a bind mount, some pieces of the directory of the file are actually not read-only afterward. This creates an obvious security issue. In this feature, the issue is fixed so that the bind mount is indeed read-only.
Runc is the default container runtime underpinning other higher-level runtimes like Docker and CRI-O. Previously, this issue was fixed at the more higher levels, creating inconsistency across other more higher level runtime implementations. The proposal aims to fix the issue now at the more lower-level of runtime, in runC.
Add AppArmor Support to K8s in GA
AppArmor is a way to give access control capabilities to Linux, allowing for the creation of certain profiles that govern who can and can’t access what. Adding support for Kubernetes means that AppArmor can help you get better auditing through system logs and restrict what containers are allowed to do. It is installed as a kernel module on the node.
Interactive(-i) flag to kubectl delete for user confirmation now in GA
In Kubernetes, the command ‘kubectl delete’ can delete a Kubernetes Deployment. The results of the Kubectl delete command are irreversible, so the idea is to mitigate accidental deletes by showing a preview to the user of all the resources planning to be deleted beforehand. This enhancement intends to create a flag, interactive(-i), that asks for confirmation from a user before proceeding to enforce the command kubectl delete.
Conclusion
Many of the security capabilities in this release are now GA, marking the community’s continued dedication to improving security in Kubernetes.The more notable features that demonstrate the continued adoption of Kubernetes across the board include the additional authentication considerations for users logging in, as well as the extension of security concepts to features for other purposes, such as debugging.
