Kubernetes CustomResourceDefinitions (CRDs) | Tutorial

Kubernetes has been the default choice for container orchestration since its debut in 2014. Its core features, like its ability to auto-deploy, scale, and manage containers, solve a wide range of deployment issues for cloud-native applications.

When you’re working with Kubernetes resources, like deployments, services, and pods, sometimes you need to customize them in order to fulfill your use cases, which otherwise would be difficult to achieve with the options available. This is where Kubernetes CustomResourceDefinitions (CRDs) can help you extend the Kubernetes API and achieve the desired result.

In this article, you’ll learn what Kubernetes custom resource objects are and how they can be used and created. You’ll also see a few resource customizations in action.

Use Cases for Custom Resources

A resource in the Kubernetes API is an endpoint that persists a collection of a particular type of object. For example, several built-in objects, like pods and deployments, are exposed via an endpoint, and the API server manages their lifecycle. Kubernetes provides you with an option of extending your object using CRD so that you can introduce your API to the Kubernetes cluster per your requirement. Using CRD on Kubernetes, you are free to define, create, and persist any custom object.

It’s worth mentioning that CRDs themselves don’t contain any logic. The primary purpose of a CRD is to provide a way to create, store, and expose the Kubernetes API for custom objects.

Custom resources can be used in so many different scenarios and help extend the functionality of Kubernetes, object automation, and object creation using kubectl.

Extending Kubernetes Functionality

In some cases, you can’t complete your projects with built-in Kubernetes resources, so you need a way to extend it and customize it. Suppose you are running a database or cache inside the Kubernetes cluster; in this instance, you have specific operational tasks, like creating a backup, restoring from an existing backup, and creating a cluster with some predefined set of server nodes. These tasks are often defined using CRDs and must be combined with a controller for it to take proper action after creating specific resources.

Not only that, in case of failure scenarios (i.e., MySQL instance is down), you can define the number of replicas you want using the CRDs. Then you can apply the Kubernetes operator pattern (i.e., utilizing a controller) on top of it to bring the system back online.

CRDs without controllers are just declarative objects.

Using kubectl for Object Lifecycle Management

You can easily handle the entire lifecycle of the custom object using <terminal inline>kubectl<terminal inline> CLI just like you do for the Kubernetes native objects, like <terminal inline>Deployment<terminal inline>, <terminal inline>Pod<terminal inline>, and <terminal inline>ReplicaSet<terminal inline>.

Enabling Object Automation

CRDs are often used by DevOps or cluster administrators to automate specific complex tasks. For example, if you’ve installed monitoring tools, like Prometheus, on Kubernetes, you know how hard it is to install all the components (i.e., Prometheus and Alertmanager) that are required for a proper setup. Instead, you can use Prometheus Operator, which utilizes CRDs to automate your cluster’s installation and quickly set up monitoring.

K8s Metrics, Logging, and Tracing

Monitor the health of your cluster and troubleshoot issues faster with pre-built dashboards that just work.

Start Free Trial Book a Demo

Understanding CRDs | Tutorial

To better understand CRD, let’s walk through an example that shows you how it can be used to improve your Kubernetes deployments.

To start, you’ll need to have a local Kubernetes cluster set up and kubectl CLI installed. In this example, you’ll be using minikube to demonstrate the creation of CRDs. You’ll spin up the local Kubernetes cluster by running the <terminal inline>minikube start<terminal inline> command.

Create a CRD

Let’s begin by creating a simple CRD. Following is the YAML manifest CRD you’ll use, which was pulled from the Kubernetes documentation, where you can find additional information.

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: crontabs.stable.example.com
spec:
  group: stable.example.com
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                cronSpec:
                  type: string
                image:
                  type: string
                replicas:
                  type: integer
  scope: Namespaced
  names:
    plural: crontabs
    singular: crontab
    kind: CronTab
    shortNames:
      - ct

Let’s try to break down the definitions of each field so that you can better understand the CRD since most of them look different when you compare them with built-in Kubernetes objects.

• <terminal inline bold>apiVersion<terminal inline bold>: specifies that you’ll use the <terminal inline>apiextensions.k8s.io/v1<terminal inline> API.
• <terminal inline bold>kind<terminal inline bold>: specifies that you want to create a <terminal inline>CustomResourceDefinition<terminal inline>.
• <terminal inline bold>name<terminal inline bold>: specifies the name of the resource, which should be in <terminal inline><plural>.<group><terminal inline> form.
• <terminal inline bold>group<terminal inline bold>: mentions the group name for the API.
• <terminal inline bold>versions<terminal inline bold>: mentions the version to be used in the API URL. It can have values like <terminal inline>v2<terminal inline> or <terminal inline>v1aplha<terminal inline>.
• <terminal inline bold>served<terminal inline bold>: controls whether this <terminal inline>version<terminal inline> should be enabled or disabled. You can only mark one version as <terminal inline>storage<terminal inline>.
• <terminal inline bold>schema<terminal inline bold>: specifies a structural schema that you want to validate the CRD of using the openAPIV3Schema validation before you send it to the API server. Then you are also specifying that custom object fields <terminal inline>spec.cronSpec<terminal inline> and <terminal inline>spec.image<terminal inline> must be a string. The field <terminal inline>spec.replicas<terminal inline> must be an integer.
• <terminal inline bold>scope<terminal inline bold>: specifies whether the custom object is namespaced or available cluster-wide. You’ve used the <terminal inline>Namespaced<terminal inline> scope, so it’s only available to the namespace that you’ll use during the creation of the CRD. The default is cluster-scoped.
• <terminal inline bold>plural<terminal inline bold> and <terminal inline bold>singular<terminal inline bold>: specify the plural and singular name of the CRD.
• <terminal inline bold>kind<terminal inline bold>: specifies the type of the custom object.
• <terminal inline bold>shortNames<terminal inline bold>: specifies the short string that you can use in the CLI.

The next step is to create the CRD using the <terminal inline>kubectl apply -f crd.yaml<terminal inline> kubectl command. You’ll get the following response after the CRD creation:

customresourcedefinition.apiextensions.k8s.io/crontabs.stable.example.com created

Now you can verify it using the <terminal inline>kubectl api-resources | grep crontab<terminal inline> kubectl command. This command prints the supported API resources on the server. After using it, you should see the following output:

Create a Custom Object

So far, you’ve created a blueprint of your custom object, but this CRD itself is not helpful unless you create a custom object using the <terminal inline>kind<terminal inline> <terminal inline>cronTab<terminal inline> that you’ve defined in the CRD.

To create the custom object, you’ll use the following YAML manifest:

apiVersion: 'stable.example.com/v1'
kind: CronTab
metadata:
  name: my-cron-object
spec:
  cronSpec: '* * * * */5'
  image: my-cron-image

You’ll create the custom object using the <terminal inline>kubectl apply -f my-crontab.yaml<terminal inline> command. After the creation, you can run the <terminal inline>kubectl get crontab<terminal inline> command for verification and you should see the following output:

Further, you can view the raw YAML using the shortname <terminal inline>ct<terminal inline> you initially defined in the CRD:

It’s a good practice to add some validation before you create your custom resource object. Kubernetes itself won’t do any validation, and you would have to add this validation when you manually define your CRD. You can describe validation constraints using OpenAPI Specification.

You’ve used OpenAPI Specification in the preceding example, but let’s use the following code snippet to understand this functionality:

      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                cronSpec:
                  type: string
                image:
                  type: string
                replicas:
                  type: integer
                environmentType:
                 type: string
                 enum:
                 - development
                 - staging
                 - production

In the above example, you add a new <terminal inline>environmentType<terminal inline> field which accepts only the three predefined values in the <terminal inline>enum<terminal inline>. If you try to create a custom resource for this CRD with values other than what you defined, it will throw an error.

apiVersion: 'stable.example.com/v1'
kind: CronTab
metadata:
  name: my-cron-object
spec:
  cronSpec: '* * * * */5'
  image: my-cron-image
  environmentType: dev

kubectl apply -f my-crontab.yaml
The CronTab "my-cron-object" is invalid: spec.environmentType: Unsupported value: "dev": supported values: "development", "staging", "production"

After providing the correct value, the validated custom resource of kind <terminal inline>CronTab<terminal inline> can be stored to <terminal inline>etcd<terminal inline>, the Kubernetes cluster persistent storage you use for other core Kubernetes objects.

Even though you’ve used <terminal inline>kubectl<terminal inline> to manage the lifecycle of the custom resource, you have the option to use the REST clients, like <terminal inline>curl<terminal inline> or <terminal inline>wget<terminal inline>, to access the REST API of the custom resource:

Now you’re done and can do your cleanup using the following command:

kubectl delete -f crd.yaml 
customresourcedefinition.apiextensions.k8s.io "crontabs.stable.example.com" deleted

As you can see, you can delete or remove the custom object the same way you do with other built-in Kubernetes objects. If you try to get details about the custom object, you’ll be greeted with an error.

kubectl get crontabs      
error: the server doesn’t have a resource type "crontabs"

Final Thoughts

In this article, you learned how to extend the Kubernetes API using the CRD. You can easily manage custom objects created using the CRD the same way you handle built-in objects, but without as much effort.

CRDs aren’t useful unless you combine them with a controller to use them as a declarative API so that the current state and the desired state are always in sync.