Kubernetes Security Blog | RAD Security

Verified runtime fingerprints to eliminate zero day software supply chain attacks

Written by Jimmy Mesta | Jan 30, 2024 11:50:10 AM

Introducing the RAD security standard

Today, KSOC announced the availability of the RAD security standard, to help the cloud native security ecosystem combat the wave of software supply chain attacks that is increasing year on year. Powered by eBPF, the standard is the first to baseline ‘good’ behavior of cloud native workloads, with fingerprints that are freely available in an online catalog.

Quick links:

 

Why the RAD security standard? 

Since the introduction of the Biden administration's executive order in 2021, alongside software supply chain security guidelines such as NIST 800-161, the landscape of software supply chain security tools has seen exponential growth, and for good reason. Software supply chain attacks have grown to become the most prevalent attack type in 2023, surpassing ransomware and other destructive attacks. 

The tools that are commonly touted for the software supply chain include image attestation, the creation of software bills of materials (SBOM), Kubernetes Bill of Materials (KBOM), shifting left, and even a few tools that run on the CI/CD operating system to identify attacks that target the CI/CD pipeline itself. 

Despite the plethora of options, the ecosystem today still faces a large vulnerability when it comes to software supply chain security, because: 

  1. Tools like vulnerability scanners or SBOMs are not meant to detect novel software supply chain attacks
  2. The CI/CD agents and runtime tools that are supposed to detect novel threats either operate after the fact, or their black box or rule-based models cannot catch zero day attacks
  3. Image signing cannot validate whether the build itself has been tampered with

In short, until today, there has been no way for the ecosystem to create and share verifiable proof of untampered cloud native software, and there has certainly been no way to detect the kinds of sophisticated, advanced attacks like Codecov, SolarWinds, and 3CX that permeate the software supply chain threat landscape.

With the RAD security standard, as an industry, we can flip the balance of power in the software supply chain between attackers and defenders.

What is the RAD security standard? 

Jimmy Mesta, CTO and Co-Founder of KSOC, explains,

"The goal of the RAD security standard is to reverse the balance of power between software supply chain attackers and software producers. If development teams can compare a verified, clean runtime fingerprint against the same image running in their environment, they have a real chance in defending against the next zero day attack." 

The RAD security standard uses eBPF to codify the baseline behavior of a cloud workload into a behavioral fingerprint, so DevSecOps teams have a transparent, verifiable defense against the next wave of zero day software supply chain attacks.

 

 

With a behavioral fingerprint of runtime processes that always occur, the baseline behavior in the fingerprint provides a simple ‘allowable’ list upon which any other activity can be compared. This takes software supply chain security and zero day detection to an entirely new level of simplicity, transparency and effectiveness.  

Fingerprints can be verified, signed, or tracked, and they include built-in context from connections across the environment. 

 

Creating a fingerprint

Containerized environments are filled with a multitude of image types. Some image types are open source, some are proprietary. For example, data has shown that the most popular open source image types are NGINX, Redis, and Postgres. 

The processes that occur when a container runs can be codified into a fingerprint. These processes can be easily picked up by an eBPF agent and mapped without requiring the highest privileges for deployment. At its most basic level, a fingerprint is composed of a container, it’s processes and child processes, and the programs and files that are a part of those processes.

 

Example of the basic hierarchy for a cloud native workload fingerprint

 

Cert-manager-controller image fingerprint example

Below is an example of the fingerprint generated for a cert-manager-controller image, which adds certificates and certificate issuers as resource types in Kubernetes clusters. Here we can see:

  • In blue, the container that spawned the app
  • The arguments that were used to start the cert-controller program itself 
  • The baseline child processes, programs, files and network activity of this container image 

 

Screenshot of RAD security fingerprint for cert-manager-controller image


Simplification of cert-manager-controller runtime fingerprint



Difference between the RAD approach and other methods of detection

By snapshotting a clean representation of normal behavior, RAD can compare new runtime activity against this fingerprint to detect abnormal behavior:

  • Is this process, program, file, or network activity expected based on the behavior that's been represented in the fingerprint?
  • Does the node appear at the expected location in the hierarchy? 
  • Do the node's properties match the expected properties?  Is this process executed by the expected user?  Is the expected file activity opening the expected file?

Inverting the model--from writing rules to identify suspected bad behavior, to instead flagging activity that deviates from the norm--greatly simplifies and improves detection capabilities. This exception-based security model means that novel attacks become immediately apparent, as they diverge from the expected behavior of the app.  No new signatures need to be written, and there's no lag in detection. Immediately, we can also see that the fingerprint-based analysis can take into account not just the properties of the activity (which is the status quo for runtime activity today), but also context, relationships with other activity, and a stateful knowledge of what activity should be expected in relation to other activity. 

RAD security method vs other container runtime security methods

 

The black box

With the black box, thousands of inputs go into a black box and - poof! - out come anomalies that represent attacks. The limitations of this model include: 

  1. Little transparency into the ‘why’ of its classification of threats 
  2. The results are very noisy, and with no understanding of the underlying methods, it’s very hard to know how to filter out the noise in a practical way. 
  3. Results are stateless many times - meaning that they lack the context needed to truly prioritize the risk. And adding that context in is not easy.
  4. With the black box, a lingering doubt around efficacy remains, and the common perception today is that any nation-state actor could breach any system with enough resources and time. 

 

The opposite of the black box; signature-based detection

In cloud native security, runtime protection has clearly been taken over by a renaissance of signature-based detection tools. Take Falco for instance, which is arguably the most popular runtime protection tool in the cloud native security industry today, with 6.6 thousand stars and an enviable set of rules. When the alerts from the runtime agent match those rules, boom, you know an attacker is there. Or do you? Signature-based detection methods have fundamental flaws that limit their usefulness:

  • Too many false positives from legitimate workloads: One of the major problems with the signature-based approach is that any alert could be a false positive, warranting further investigation. The alerts could signify an attacker . . . or it could simply be insecure behavior occurring as part of a legitimate workload (like an agent that is running a container as root, as it needs to perform its job). 
  • Rule-writing is never-ending: For this approach to be effective, you have to write hundreds, maybe thousands of rules. But you still won’t be able to predict every attack technique. Yet another option is rule-based, behavioral analysis. A research team will observe attacks and then codify those attacks in rules of behavior. In this case, you are limited to what the research team can observe.
  • Unable to catch sophisticated attacks: Signature-based methods are not suitable for catching more sophisticated attacks where legitimate commands were run by the wrong user, or when legitimate processes are utilized for malicious purposes. 

Compared to the signature-based approach, the RAD standard is less noisy, more accurate and stateful, including context by definition.

The cloud native security silver bullet; immutability (or is it?)

And then there is drift detection; based on the concept that whatever is coded up front is what should end up in runtime. Theoretically, you should be able to detect zero days in runtime if what is happening there is different from what was coded up front. It makes sense, but images and containers have a lot of bloat - they can get huge. So how do you do the drift detection analysis in an efficient way, with accuracy? And how do you make an accurate comparison from before deployment to after deployment?

 

Applying workload fingerprints to software supply chain attacks

 

SolarWinds 

In the SolarWinds attack, advanced malware (Sunspot) was implanted into a compromised build server that was used in the CI/CD process for the Orion software. When customers pulled a new version of Orion, they pulled in the attached Sunburst malware, which lay dormant for 2 weeks until it began to act suspiciously. 

In this attack, with RAD, SolarWinds could have been identified and prevented:

  • SolarWinds could have generated contextual fingerprints based on runtime attributes collected using eBPF that are unique to a release and cryptographically verified
  • This verification of the fingerprint would have occurred on the build server, requiring the hash value to match the verified fingerprint 
  • The consumer build system would run a verification to ensure a match, and then continuous drift detection would be enabled for the lifecycle of the workload 

SolarWinds was not a CVE, so a CVE Scanner wouldn’t have worked. Cosign / Sigstore signatures may have added a layer of protection but ultimately may have been bypassed or tampered with in the build compromise. Software Composition Analysis (SCA) tools mostly look for known vulnerabilities and perform source/sink analysis, none of which would have helped with SolarWinds. An Extended Detection and Response (XDR) tool may have flagged blatantly abnormal behavior at runtime but it would have been too late in most cases.

‘Poisoned’ online registry

Two users, vibersastra/ubuntu and vibersastra/golang, were responsible for uploading malicious images on Docker Hub,disguised as popular public images, that were downloaded 10k+ times by unwitting users into Kubernetes or containerized environments. 

With RAD’s verified runtime fingerprinting, the fingerprint of legitimate software can be verified and shared, providing a lifeline against the accidental usage of malicious software masquerading as legitimate software on public repos.

 

It’s time to eliminate zero day software supply chain attacks 

The RAD security standard provides a transparent baseline for expected behavior of cloud native workloads, verifying the integrity of the software supply chain and detecting zero day software supply chain attacks with a level of certainty not possible with current runtime tools that use anomaly detection and signature-based methods.

Here are some options for you and your teams to get started eliminating software supply chain attacks in your environment:

Sign up for Early Access Today

We want to hear your feedback! Leave us a comment below.
View full post