Comparing four popular Kubernetes policy engines

Intro

Kubernetes is a complex system that requires careful management and control to ensure security and compliance. As clusters grow and evolve, it becomes increasingly challenging to maintain consistency and enforce policies across all deployments. This is where Kubernetes policy engines come into play, providing a way to define, enforce, and audit policies across the entire cluster.

In this post, we’ll compare four popular Kubernetes policy engines: OPA Gatekeeper, Kyverno, Kubewarden, and jsPolicy. We’ll look at their features, strengths, and weaknesses, and provide examples of policies for enforcing runAsNonRoot for Pods.

Tools comparison

Technical aspects comparison table

Example of policies
OPA Gatekeeper
OPA Gatekeeper is a powerful and flexible policy engine based on the Rego language. It uses Kubernetes Admission Control to enforce policies at the pre-admission stage, ensuring that any new objects created in the cluster meet the defined policy requirements. OPA Gatekeeper supports custom policies written in Rego, making it highly extensible and customizable.

However, OPA Gatekeeper has some limitations. Its customization options are somewhat limited, and it can be difficult to use for non-developers or those unfamiliar with the Rego language. Additionally, it can be challenging to debug policies due to the complex nature of the Rego language.

Here’s an example of an OPA Gatekeeper policy for enforcing runAsNonRoot for Pods:

package k8srequiredlabels

violation[{"msg": msg}] {
    pod := input.review.object
    pod.spec.securityContext.runAsNonRoot == false
    msg := "Pod must not run as root."
}

Kyverno
Kyverno is a Kubernetes policy engine that uses YAML/JSON-based policies to enforce policies on resources in the cluster. Kyverno supports both pre-admission and post-admission enforcement, making it a flexible and powerful tool for managing policies. It also provides support for custom resources, allowing users to define their policies on custom resources.

Kyverno’s main strength lies in its simplicity and ease of use. It has an intuitive policy language, making it accessible to non-developers, and provides a rich set of validation rules and transformations. However, Kyverno’s customizability is limited, and its performance can be impacted when handling large volumes of resources.

Here’s an example of a Kyverno policy for enforcing runAsNonRoot for Pods:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-non-root-pods
spec:
  validationFailureAction: audit
  background: false
  rules:
  - name: runasnonroot
    match:
      resources:
        kinds:
        - Pod
    validate:
      message: "Pod must not run as root"
      pattern:
        spec:
          securityContext:
            runAsNonRoot: true

Kubewarden
Kubewarden is a policy engine that uses WebAssembly modules to enforce policies on Kubernetes resources. It supports pre-admission and post-admission enforcement, making it a flexible and powerful tool for managing policies. Kubewarden’s use of WebAssembly allows it to support policies written in any language that can be compiled to WebAssembly.

Kubewarden’s main strength is its flexibility and extensibility. It provides a rich set of APIs for building and deploying custom policies, making it highly customizable. However, its performance can be impacted when handling large volumes of resources, and its use of WebAssembly can be challenging for non-developers or those unfamiliar with the technology. Here’s an example of a Kubewarden policy for enforcing runAsNonRoot for Pods using Rust:

use kubewarden_policy_sdk::admission::AdmissionRequest;
use kubewarden_policy_sdk::policy::Policy;
use kubewarden_policy_sdk::request::ValidationResponse;

fn validate(request: &AdmissionRequest) -> ValidationResponse {
    let pod = &request.object;
    match pod.get("spec") {
        Some(spec) => match spec.get("securityContext") {
            Some(security_context) => match security_context.get("runAsNonRoot") {
                Some(run_as_non_root) => {
                    if let Some(false) = run_as_non_root.as_bool() {
                        return ValidationResponse::invalid("Pod must not run as root");
                    }
                }
                _ => {}
            },
            _ => {}
        },
        _ => {}
    }
    ValidationResponse::valid()
}

kubewarden::policy! {
    "name": "run-as-non-root",
    "resource": "pods",
    "description": "Enforce runAsNonRoot for Pods",
    "validate": validate
}

jsPolicy
jsPolicy is a policy engine that uses JavaScript to enforce policies on Kubernetes resources. It supports pre-admission and post-admission enforcement, making it a flexible and powerful tool for managing policies. jsPolicy’s use of JavaScript allows it to support policies written in any language that can be compiled to JavaScript.

jsPolicy’s main strength is its simplicity and ease of use. It has an intuitive policy language, making it accessible to non-developers, and provides a rich set of validation rules and transformations. However, its customizability is limited, and its performance can be impacted when handling large volumes of resources.

Here’s an example of a jsPolicy policy for enforcing runAsNonRoot for Pods:

function mutate(request) {
  const pod = request.object;
  if (pod.spec.securityContext && pod.spec.securityContext.runAsNonRoot === false) {
    throw new Error("Pod must not run as root");
  }
}

module.exports = {
  mutate
};

The final part:

Here’s an example pod manifest that violates the policies we’ve defined, and one that satisfies the policies:

• Pod manifest violating the policies

apiVersion: v1
kind: Pod
metadata:
  name: root-pod
spec:
  containers:
  - name: nginx
    image: nginx
    securityContext:
      runAsNonRoot: false

In this manifest, the runAsNonRoot field in the pod’s securityContext is set to false, which violates the policy that requires the field to be set to true.

• Pod manifest satisfying the policies

apiVersion: v1
kind: Pod
metadata:
  name: non-root-pod
spec:
  containers:
  - name: nginx
    image: nginx
    securityContext:
      runAsNonRoot: true

In this manifest, the runAsNonRoot field in the pod’s securityContext is set to true, which satisfies the policy that requires the field to be set to true.

In conclusion, Kubernetes policy engines are essential for ensuring security and compliance in Kubernetes clusters. Each of the policy engines we’ve covered has its strengths and weaknesses, and the choice of which one to use ultimately depends on your specific needs and preferences. We hope this post has helped you understand the differences between these popular policy engines and provided useful examples of how to enforce policies using each one.

References:

• OWASP Kubernetes Top Ten - K04: Policy Enforcement
• OPA Gatekeeper’s doc
• kyverno’s doc
• Kubewarden’s doc
• jsPolicy’s doc