Kubernetes Admission Controller Guide for Security Engineers

Admission controllers are an essential Kubernetes-native option for enforcing policy on your containers and overall Kubernetes environment. But there are tons of flavors and ways to do it.  

In this Kubernetes Admission Controller Guide, we’ll explore:

• Definition of admission controllers
• Types of admission controllers
• Types of admission webhooks
• Kubernetes API server and admission control process
• Admission control policy examples
• Pros and cons of using admission controllers
• Admission controller policy enforcement tools
• Managed Kubernetes admission controllers
• Evaluating Kubernetes admission controllers
• Configuration with RAD Security

By mastering these concepts, security engineers can enforce policies, audit configurations, and prevent unauthorized or harmful deployments, thus playing a crucial role in the organization's security posture. 

What is an admission controller in Kubernetes?

Simply put though, admission controllers allow you to set security policies for your Kubernetes clusters. At a glance, admission controllers provide access control to users and applications within a cluster. They allow administrators to set various policies including which users can add or delete objects in a cluster, as well as which applications can be deployed to the clusters. 

Admission controllers are typically used to ensure that only safe code and configurations are allowed into the production environment. By intercepting and validating requests to the Kubernetes API before they are allowed to persist in runtime in the cluster, admission controllers mitigate risks associated with unauthorized actions, malicious entities, or erroneous configurations. This preemptive policy enforcement ensures that only safe and compliant workloads are deployed.

In practice, admission controllers wield great versatility, facilitating an array of custom policies tailored to specific organizational needs.

They can be leveraged to validate resource requests, evaluate labels and annotations, assess resource quotas, and even trigger webhooks for more advanced validation scenarios.

By offering this flexibility, admission controllers significantly enhance the platform's resilience and overall security posture. Kubernetes even provides Kubernetes Pod Security Admission, with three levels of restrictive policies.

Types of Admission Controllers 

Admission controllers are categorized into two main types: Mutating and Validating. Each plays an important role in securing your Kubernetes environment. 

Mutating Admission Controllers

Mutating controllers are the first to interact with requests. They have the authority to modify or mutate requests before they materialize in the cluster. Their primary role is to implement changes to the incoming requests, ensuring that they align with standards and defaults. This could involve altering the request's content, such as adding labels, annotations, or even changing the specifications of the pods so that they meet specific standards or defaults before the request is validated. 

Validating Admission Controllers

Validating controllers are second to interact with requests. Their role is the realm of validation, where each request is compared against a set of established policies and standards. Unlike their mutating counterparts, validating controllers do not alter the requests; they merely ensure compliance with the cluster’s rules.

The Difference Between Webhooks and Admission Controllers

Webhooks serve as connectors that link external services to the Kubernetes admission control process. They are essentially HTTP callbacks triggered by specific events within the Kubernetes API server, allowing for real-time interaction with external systems for request modification or validation.

Webhooks differ from admission controllers in that they are only triggered when the cluster receives a request. A webhook enables developers to tailor specific instructions when creating new resources on a Kubernetes cluster. It acts as an interface between your application and the Kubernetes API server, allowing for custom validation rules such as resource quotas and limits, user and client authentication requirements, or admission control policies.

Admission controllers, on the other hand, run every time someone attempts to create or modify a resource on the cluster. And to make it even more confusing, there are Kubernetes webhooks that work with admission controllers; namely mutating and validating types. The behavior of these webhooks is determined with the admission controller configuration files.

Types of Admission Webhooks 

MutatingAdmissionWebhook

A MutatingAdmissionWebhook intercepts and processes incoming requests to the Kubernetes API server before they are persisted to the cluster. The primary function of a typical mutating admission controller example in Kubernetes is to modify (mutate) the content of the resource being created or updated. 

This is generally an intrusive option that does not allow for close management of what gets deployed to production, as the mutation is automatic and the enforcement of the policy in the first place has the potential to bring down production.

ValidatingAdmissionWebhook

A ValidatingAdmissionWebhook, on the other hand, focuses on validating and potentially rejecting incoming requests before they are persisted to the cluster. Its primary role is to assess whether the request complies with the defined policies and standards of the cluster. This is much less intrusive than a mutating admission controller because it’s results are generally binary; deploy the workload or not.

The Kubernetes API Server and Admission Control Process

Understanding admission controllers requires understanding the Kubernetes API server, which is how one would interact with Kubernetes objects.

What is the Kubernetes API server? 

The Kubernetes API Server is the medium through which all organization and processing within the cluster takes place. 

The diagram below shows the three steps in the process of making a request to the Kubernetes API to change the state of an object; for example, deploying a pod.

The steps are Authentication, Authorization, and Admission Control. 

In the Authentication step, which has more to do with cloud identity and access management, the server determines whether the presented user is valid. 

Example: Is Bob from my cloud IAM account a valid user?

In the Authorization step, which concerns Kubernetes role-based access control, the server determines what actions the user is allowed to take.

If yes, can Bob create a new pod?

In the Admission Control step, the server determines if there are any specific limitations on the action the user or machine wants to take. 

Etcd is where the data for the cluster is stored, so the state determined by this three-step process will be executed and represented in Etcd.

If yes, can Bob create a new pod with privileges within X namespace?

This would work similarly when substituting Bob, a real user, for a service account, or a machine user. This would work similarly in the case that, instead of the default Kubernetes API, you are trying to perform actions on custom resources, which are basically non-default extensions of the Kubernetes API.

Read more about the container lifecycle, which is necessarily involved with Kubernetes API processes but running on its own schedule.

Admission Control Policy Examples

Admission control policies can be used to define a wide range of security and operational policies for a Kubernetes environment. The following are example policies that can help further secure clusters, ensure that best-practice guidance is followed, and maintain a secure and compliant state:

Enforce Memory and CPU Resource Limits: Prevent any single pod’s ability to monopolize system resources by requiring all pods to specify memory and CPU limits through admission controllers. For example, a policy could reject any pod with a specification that does not include a requests and limits section, thus ensuring a fair allocation of resources and avoiding resource starvation for other pods.

Restrict Image Registries: Allow pods to be deployed using only container images from approved secure registries. This can be done by specifying a list of allowed registries while blocking all other sources; for example, it is only permissible to deploy pods with images from the organization’s private registry or specific trusted public registries.

Block Privileged Containers: Prohibit any pods from being executed with privileges, as this represents a severe threat to the system’s security. For instance, a policy could reject any pod specification including privileged: true in its container security context, thus ensuring that no container will operate with more permissions than it requires.

Require Read-Only Root Filesystems: Lockdown all containers by enforcing that every container should operate with a read-only root filesystem. Such a policy would reject any pod that hasn’t set readOnlyRootFilesystem: true in its security contexts. This prevents malicious intent from making modifications to the container’s filesystem.

Enforce Network Policies: Verify that all new pods or namespaces adhere to predefined network policies before they are permitted to be executed. This can require a network policy to be adequately labeled for enforcement or inspecting new deployments for annotations that correspond to the present network policies.

Validate ConfigMap and Secret Usage: Not allow open attachments of volumes containing ConfigMaps or Secrets. I.e., a policy could mandate the rejection of pods that mount sensitive secrets as clear text, rather than using Kubernetes’s native security context configurations, guaranteeing sensitive data is handled securely.

Prevent NodePort Services: Block services to be upgraded and expose them to the whole public internet using type: NodePort. I.e., a policy could not allow services to be accessible from the NodePort to other systems, ensuring only the clusters and configured ingress points are the edge for the network.

Restrict where pods can pull images from: Using an image from an unauthorized or unverified source could provide an in-road for malware in a Kubernetes cluster. Make sure your pods only pull images from the registries you know you can trust.

Enforce network policies for pod communication: Ensuring that every deployment of a pod is accompanied by an appropriate network policy before the pod is admitted. For example, every pod should have a network policy that has a least privilege restriction on ingress and egress traffic. This would enforce the least privilege principle and reduce the attack surface.

Pros and Cons of Using Kubernetes Admission Controllers

Deciding whether to use a Kubernetes admission controller requires weighing the security and operational benefits against the challenges of complexity and resource overhead. 

Pros of Kubernetes Admission Controllers

Admission controllers offer various benefits when providing access control to your Kubernetes clusters, including real-time validation, authentication enforcement, and policy enforcement. After all, if a resource is created without the proper authorization or validation required by an admission controller, it could create security vulnerabilities within your system.

• Enhanced security and compliance: Admission controllers enable real-time validations and enforcement of security policies, safeguarding against unauthorized access and deployments. 
• Automated policy enforcement: Admission controllers have the potential to eliminate human error and ensure consistent rule application through automated enforcement of policies across all deployments. 
• Customizable: Admission controllers allow custom policies tailored to an organization’s specific security needs and operational practices. 
• Operational efficiency: Admission controllers save resources and time by preemptively blocking non-compliant resources before they can be deployed. 

Cons of Kubernetes Admission Controllers

Most teams simply don’t turn admission controllers on, even if they can do so. Many well-meaning teams define the policies and go to enforce them, only to find that when they do, their production environments go down, and their development teams are not prepared to understand the consequences of deploying containers that violate admission control policy.

Using admission controllers can add complexity to the overall system architecture and may require the allocation of additional resources.

• Configuration and management complexity: With the flexibility and power of admission controllers comes complexity. Setting up and managing policies requires knowledge of Kubernetes (although managed admission controllers are often pre-configured). 
• Risk of disruptions: Admission controllers can block critical updates or deployments if they’re not properly configured or the policies are too restrictive. 
• Resource overhead: Admission controllers can introduce additional load on the system, especially if dynamic. The resources required to operate them can impact a cluster’s overall performance if the environment is already resource-constrained.  
• Team involvement: Effectively implementing admission controllers requires a collaborative effort between your DevOps and security teams to define and test enforcement policies. This can sometimes be challenging to coordinate.

The teams best equipped to work with admission controllers in enforcement mode are those that have worked hard, between DevOps and Security, to define the policies, and who have done lots of practice and testing to understand the consequences of enforcement in any environment.

Kubernetes Admission Controller Policy Enforcement Tools

Multiple platforms are available for building Kubernetes clusters. The platform you are using influences how you should approach admission control. Any of these can be installed via AdmissionControl plugins.

OPA Kubernetes admission controller: OPA (Open Policy Agent) is based on Rego and is often used by teams that want to amass a large, ever-changing rule set that can be queried, QA’d, and managed separately from the Kubernetes resources themselves. Policies are not restricted to allow/deny or binary answers, and checks on a policy can result in structured output with lots of context.  

Kyverno admission controller: Kyverno does not use Rego; it relies on YAML, which makes it easier to use than OPA from that perspective. However, managing Kyverno policies at scale is difficult because you cannot decouple the policies from the Kubernetes resources to which they apply. 

JSPolicy admission controller: JSPolicy is a less popular choice than Kyverno and OPA, using JavaScript-based validation for resources created on a Kubernetes cluster.

Deprecation of the Pod Security Policy Admission Controller

The Kubernetes pod security policy, and the usage of the associated admission controller, was deprecated in Kubernetes version 1.21. The PodSecurityPolicy guidelines set a baseline to strive toward, and the admission controller is available to enforce these baseline guardrails. Many are still on applicable Kubernetes versions, but the recommended replacement for anybody using a version more recent than 1.21 is usage of the Kubernetes pod security standards, with options for privileged, baseline and restrictive levels. 

Managed Kubernetes Admission Controllers

Let’s look at another type of Kubernetes admission controller example.

The configuration of admissions controllers differs slightly when used with managed Kubernetes offerings such as EKS and GKE. In these environments, many admission controllers are already pre-configured within the platform, making it easier for developers to get their applications up and running on a cluster quickly. 

That said, EKS admission controllers offer support for OPA, Kyverno, Kubewarden, JSPolicy, and similar solutions. GKE admission controllers integrate with OPA and Kyverno to provide access control policies out of the box.

A checklist for evaluating a Kubernetes Admission Controller

Given this context, what do you need in an admission controller in order to be successful? Here is a checklist to help you rate vendors’ admission control options and understand whether you are getting something useful, or something you will pay for but never operationalize.

• Are your admission control policies transparent?

Does the vendor keep the Rego policy in a black box? Is it labeled as ‘proprietary’? This can be an issue when you need to customize policies for your own environment. If you are able to transparently view - and even change - the Rego policy as you see fit, your engineers will be more comfortable with the policies you are enforcing and your most advanced teammates will feel they are in control of any downtime in production caused by admission control.

• Can you set them in the CI/CD process across multiple clusters?

One of the issues with using the open source version of OPA (or any other admission controller) is the management and implementation across dozens (or even hundreds) of clusters. Quickly, the duplication and QA process becomes overwhelming to the point that admission control is no longer a practical option. However, if admission control policy creation can be folded directly into the regular GitOps, or CI/CD process, quickly the team will be able to scale across multiple clusters and environments, and the engineering team will have access to the policies themselves, which will reduce back-and-forth between engineering and security.

• Can you do a dry run before you enforce anything?

Admission control will block a workload before deployment which, in some cases, can break production. It is critical to have an understand of what is going to happen before it happens. Can your vendor get a view to share with engineering of what workloads would be blocked, based on the policies in place? If not, it will be a rocky road getting to actual enforcement.

• How broad is the set of criteria available for enforcement?

Some admission controllers are based only on one risk factor, such as critical or high-rated image CVEs. The breadth of criteria available for enforcement is something to check on ahead of time; do they take into account Kubernetes misconfigurations, at a minimum?

• How does the policy validation occur?

Is the vendor requiring you to send data somewhere else in order to make a policy decision? This is worth checking, especially if you have a broad set of criteria you want to enforce. Do they use OPA, do they use Kyverno? This is important to understand, at a minimum in light of understanding the vulnerabilities your admission control will be subject to. For example, there was a recent Kyverno CVE that required a breaking update. You will want to know what third-party risks you are exposed to.

Admission Controller Configuration with RAD Security

At RAD Security, we understand the dynamic between the engineers managing Kubernetes and the security teams trying to secure it; in most cases, both teams are focused on security to some extent. RAD Security’s admission controller capability is created to help security teams make Kubernetes secure by default in partnership with their engineering colleagues, with:

Transparent Policies

RAD Security makes all Rego policies visible in the cluster. Our visibility measures comply with Open Policy Agent (OPA) standards, which perform essential and accessible actions for teams by:

• Improving Understanding: Teams may quickly see and understand the policies in place for their safety, allow for a more profound understanding of secure practices.
• Fostering Collaboration: Making policies visible helps combine the security and engineering teams to make appropriate changes to security. Teams can implement the new policy measures around how they operate securely.
• Allowing customization: Our security teams are free to adjust policies as they see necessary for them. 

Setting Policies Across Multiple Clusters

We integrate with GitHub Actions to set policies for both detection and admission control within the CI/CD pipeline, offering:

• Uniform policies: Every cluster will comply with the same rules, regardless of the differences of each current set rules.
• Agile updates: Communities can rapidly adjust the rules for sectioning clusters to patch up security weaknesses rapidly.
• Simplified management: Users will manage this data as part of their CI/CD process to streamline operations.

Dry-Run Mode

Instead of going straight to enforcement, you can get an easy list to share with engineering of policies that you would have blocked:

• Policy impact preview: Assess new policies without the disruption of existing deployments, leading to better and more informed decision-making. 
• Facilitate communication: RAD Security helps in easily communicating and tracking reports on which policies would have blocked deployments creating a seamless relationship between the security and engineering teams.
• Minimize disruptions: This dry run function is vital in minimizing delays and disruption of operations by ensuring the resolution of issues before they impact production.

Broad Criteria Range

We use a broad range of criteria across Kubernetes configurations and other Kubernetes components, which ensures: 

• Comprehensive Security: Security engineers can address a wide array of security concerns, from container security to network policies.
• Enforcement Flexibility: RAD Security enables users to set up policies that reflect the organization’s specific risks and objectives.
• Proactive risk management: Through its comprehensive range of criteria, RAD Security ensures that users mitigate risks to prevent incidences from happening.

Our threat vector capability also allows you to immediately triage risk across your clusters, which will help tremendously in deciding which admission control policies to set in the first place.

Conclusion

To sum up, this Kubernetes admission controller guide for security engineers champions the role of admission controllers as a central component in the Kubernetes ecosystem. By using mutating and validating admission controllers and dynamic webhooks, organizations can set up a flexible policy enforcement system that is robust and responsive to their own specific security and operational demands.

With a comparison of policy enforcement tools like OPA, Kyverno, and JSPolicy, and our evaluation checklist in hand, security engineers can now decide which policy enforcement tool best fits their needs.

Lastly, RAD Security’s admission controller capabilities show how security engineers can make Kubernetes secure by default. In the future, this strategic adoption of admission controllers will undoubtedly become the bedrock of Kubernetes security, enabling organizations to orchestrate containers confidently.