Kubernetes CVE for Windows 2023 | CVE-2023-5528

Intro

Less than one month after three high severity Kubernetes vulnerabilities were published for ingress controllers for nginx, the second high severity CVE relevant for Windows endpoints in 2023 has now been published. Though the severity rating is high, it is relatively unlikely that this vulnerability is relevant in your environment given the double requirement of Windows nodes (much less common than Linux) and in-tree storage plugins (deprecated in favor of CSI since Kubernetes v1.14 in 2019, 4 years ago).

Windows nodes in Kubernetes, in-tree storage plugins, the CSI driver and persistent volumes 

There are three main pieces of Kubernetes involved in this CVE: Windows nodes, in-tree storage plugins and persistent volumes. Let’s examine each to understand their interplay and the chance of any of them being present in your environment. 

Windows nodes

In general, Windows has lagged behind Linux in support for Kubernetes. It was only in 2019, after three years of work, in v1.14, when production level support for Windows nodes was added to Kubernetes. Running Kubernetes on Windows nodes is still not exactly the same as running Kubernetes on Linux nodes. For example, for object permissions, Windows uses SIDs, ACLs and usernames, compared to Linux’s userIDs and groupIDs.  

For the reasons above, along with others, Windows nodes are, in general, not as commonly used as Linux nodes. Though there are large services today using Windows nodes, Windows is rarely used on its own. For example, one of the biggest users of Windows in Kubernetes is the group of Microsoft services powering Office 365 and Microsoft 365, which run on Azure Kubernetes Service (AKS). But AKS itself, for other services, runs on a mix of Linux and Windows nodes.  

In-tree storage plugins and the Kubernetes CSI driver

The Kubernetes Container Storage Interface (CSI) was introduced as an alpha capability to Kubernetes in v1.14. The main idea of the CSI, at its inception, was to create an alternative to plugins that were made to live inside the Kubernetes codebase, specific to each kind of storage vendor. Creating the CSI eliminated the need for these plugins, to enable better central maintainability and control while lowering the attack surface. 

Storage vendors create CSI drivers that integrate with the Kubernetes CSI, keeping the custom code for each vendor outside of Kubernetes, while still allowing Kubernetes to work with the storage vendors. Virtually any Kubernetes environment that has an integration with a storage driver will use some kind of a CSI driver, including managed cloud platforms. 

Today, new in-tree volume plugins are not allowed in Kubernetes, for many reasons, including:

- They create plugin code dependencies and impossible maintenance requirements tied to Kubernetes releases

- In-tree plugin bugs can crash critical Kubernetes components

- In-tree plugins allow volume plugins privileges equal to other kubernetes components (like kubelet and kube-controller-manager)

- In-tree plugin code would need to be public

Persistent Volumes

A PersistentVolume is a resource for a cluster, and can be pointed to via an in-tree storage plugin, as is the case with these in-tree PersistentVolume types:

awsElasticBlockStore - AWS Elastic Block Store (EBS) (not available in v1.27)
azureDisk - Azure Disk (not available in v1.27)
cinder - Cinder (OpenStack block storage) (not available in v1.26)
photonPersistentDisk - Photon controller persistent disk. (not available starting v1.15)
scaleIO - ScaleIO volume. (not available starting v1.21)
flocker - Flocker storage. (not available starting v1.25)
quobyte - Quobyte volume. (not available starting v1.25)
storageos - StorageOS volume. (not available starting v1.25)

How does CVE-2023-5528 work?

In CVE-2023-5528, the data suggests that there is a scenario with in-tree storage plugins for Windows nodes where insufficient input sanitization can allow a user admin privileges on the cluster nodes if that user can already create pods and PersistentVolumes. 

It is not clear exactly how this works, but it is reasonable to assume a scenario where the privileges granted to somebody who can create pods and PersistentVolumes might accidentally become excessive. The Kubernetes SIG-storage team stated one of the reasons in-tree plugins were deprecated was the fact that, “In-tree volume plugins grant volume plugins the same privileges as kubernetes components (like kubelet and kube-controller-manager).”

How likely are you to be affected, and what you can do

Even though this CVE is rated as high severity, it is not as likely that your environment will be affected, compared to the last Kubernetes CVE on Windows nodes, which also pertained to storage.

• This CVE is only relevant to Windows nodes, most Kubernetes containers use Linux. One of the biggest users of Windows in Kubernetes is the group of Microsoft services powering Office 365 and Microsoft 365, which run on Azure Kubernetes Service (AKS). But AKS itself, for other services, runs on a mix of Linux and Windows nodes.
• For this to be an issue,  an in-tree storage plugin must be present - K8s has been migrating from in-tree plugins to CSI (Container Storage Interface) since v1.14 , and surveys show that hardly anybody is running anything as low as 1.14 (most are on v1.24). Furthermore, the Kubernetes community has stopped accepting any more in-tree plugins, so the likelihood is even lower of this being in use in your environment.

That being said, here are examples of in-tree plugins that would be relevant in this CVE: 

You will need to definitively find out if you are using Windows nodes, using this command:  

`kubectl get nodes -l kubernetes.io/os=windows`

In the case that you are running Windows nodes and have in-tree plugins, since the in-tree plugins run on the kubelet, the recommended guidance is to update the Kubelet to:

- kubelet v1.28.4

- kubelet v1.27.8

- kubelet v1.26.11

- kubelet v1.25.16

How KSOC can help

It is hard enough to keep on top of all the new CVEs in Kubernetes and the third party Kubernetes ecosystem, let alone getting central visibility into how you may be affected or where the vulnerability might be exploited.

To prevent exposure, you need to:

1. Know what you’re running in your clusters, generally with a comprehensive asset inventory
   • KSOC provides a Kubernetes Bill of Materials (KBOM) as a free, open source option to create a comprehensive asset inventory
2. Ensure you stay up to date with the latest Kubernetes versions - to be safe 100%, you’ve got to be on version 1.28
   • KSOC’s KBOM will also show which versions you are running
3. Know who has the permissions to create pods and create a PersistentVolume in your clusters 
   • With KSOC you can view your actual audit log activity to check where PersistentVolumes have been created with local path field using special characters.
   • KSOC’s RBAC Insights can review who has permission to create pods and PersistentVolumes.
   • We will recommend ways for you to right size those permissions based on what was seen in the audit logs.
   • We can prevent or create ‘would block’ admission control events, blocking pods with PersistentVolumes that mount paths using special characters 

Conclusion

This vulnerability, while rated high in severity, is less likely to be present in your environment because of the dual requirements of in-tree plugins for storage as well as Windows nodes, which are much less common than Linux. Even so, this is the second high severity vulnerability in Kubernetes within the last month, so it’s important to sort out any potential impact of these CVEs on your or your business partners’ environments.