Kubernetes is known for its power and flexibility in managing containerized applications, but did you know it can also be extended to orchestrate custom workflows unique to your organization’s needs? Enter Custom Resource Definitions (CRDs). CRDs allow you to define your own resource types, extending Kubernetes beyond its out-of-the-box functionality. This means that Kubernetes can manage not just Pods and Services, but also any other kind of resource that your team requires.

By defining a CRD, we essentially added a new table to Kubernetes’ internal document store via the API server (etcd). This resource could now be interacted with just like any built-in Kubernetes object. Using tools like kubectl, we could create, update, and delete instances of this new resource, seamlessly integrating it with existing workflows.

By using CRDs and controllers, we can offer reusable services to our teams while avoiding the overhead of developing new APIs from scratch. The built-in Kubernetes ecosystem provides authentication, authorization, and even CLI tooling for free, significantly reducing development time and complexity.

Why Use CRDs?

• Kubernetes controllers reconcile the desired state of resources with their actual state, and we can create custom controllers using tools as simple as Bash.
• Custom Resource Definitions (CRDs) allow you to extend Kubernetes to meet your organization’s specific needs, making it a powerful platform not just for containers, but for custom orchestration tasks.
• Instead of building new microservices, leveraging Kubernetes CRDs and controllers lets you create powerful, integrated solutions that are already compatible with Kubernetes’ tooling and infrastructure.
• Consistency: Using CRDs means everything—even your own internal processes—can be treated like a Kubernetes resource, managed consistently with the rest of your cluster.
• Tooling Integration: CRDs get the same benefits as built-in Kubernetes objects. You can use kubectl to interact with them, and they integrate seamlessly with Kubernetes’ authentication, authorization, and RBAC policies.
• Modular Infrastructure: By using custom resources, you can create more modular infrastructure, which makes scaling easier. Your LoadTest or CronTask controller can run independently of your other services.

How CRDs Work

CRDs work by allowing you to register new types of resources with the Kubernetes API server. The Kubernetes API server is the brain behind Kubernetes. It stores information about the cluster’s state, using etcd as an internal document store, and ensures that whatever you’ve defined—a number of Pods running, a particular service being exposed—matches reality. etcd functions as a distributed key-value store where all the cluster’s state information, including your custom resources, is stored. With CRDs, you get to extend the brain to understand new resources that you define.

The process is straightforward:

1. Define Your Custom Resource: You describe what your new resource looks like. This could include fields like method and endpoint for a LoadTest resource.
2. Register It with the API Server: Using a YAML or JSON file, you apply your CRD to the Kubernetes cluster. This lets the API server know about your new resource type.
3. Write Controllers: Controllers are programs that watch your resources and ensure the state matches what you’ve defined. For your LoadTest, this could mean watching for new requests and kicking off an actual load test when one appears.

Once you’ve done this, your Kubernetes cluster now understands this new type of resource—and you can manage these resources using the same tools and commands as any other part of Kubernetes.

A Simple Example: Load Testing as a CRD

Imagine you want a way for your developers to request a load test easily—no extra tooling, no separate scripts, just a simple definition. You could define a LoadTest custom resource, which has fields for the test duration, endpoint to test, and type of HTTP request. Checkout this [Github repo](https://github.com/hash167/kubernetes_controller) for a controller written in bash which watches for events about the CRD.

Here’s a simple LoadTest definition:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: loadtests.platform.mycompany.com
spec:
  group: platform.mycompany.com
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                endpoint:
                  type: string
                duration:
                  type: string
                method:
                  type: string
  scope: Namespaced
  names:
    plural: loadtests
    singular: loadtest
    kind: LoadTest
    shortNames:
    - lt

The updated CRD definition can also be found in the GitHub repository here.

After defining this CRD and applying it, your developers can create LoadTest resources just like they create Pods or Deployments. A controller you build (or use off the shelf) would then watch for LoadTest resources and initiate the load test accordingly.

Why Use CRDs?

• Consistency: Using CRDs means everything—even your own internal processes—can be treated like a Kubernetes resource, managed consistently with the rest of your cluster.
• Tooling Integration: CRDs get the same benefits as built-in Kubernetes objects. You can use kubectl to interact with them, and they integrate seamlessly with Kubernetes’ authentication, authorization, and RBAC policies.
• Modular Infrastructure: By using custom resources, you can create more modular infrastructure, which makes scaling easier. Your LoadTest or CronTask controller can run independently of your other services.

Wrap-Up

CRDs let you harness the full power of Kubernetes beyond managing containers. By integrating custom resources into the Kubernetes API, you bring consistency, power, and scalability to your workflows.

The best way to understand CRDs is to experiment—try extending Kubernetes with something that matches your team’s needs. And remember, Kubernetes is a platform that can grow with you—so don’t be afraid to push its boundaries.