This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Generic

1: Audit policy
2: etcd
3: Extra API Server Certificate SANs
4: Global Image Registry Mirror
5: HTTP proxy
6: Image registries
7: Kubernetes Image Repository
8: Users

The customizations in this section are applicable to all providers.

1 - Audit policy

Kubernetes auditing provides a security-relevant, chronological set of records documenting the sequence of actions in a cluster. The cluster audits the activities generated by users, by applications that use the Kubernetes API, and by the control plane itself.

There are currently no configuration options for the Audit Policy customization and this customization will be automatically applied when the provider-specific cluster configuration patch is included in the ClusterClass.

2 - etcd

This customization will be available when the provider-specific cluster configuration patch is included in the ClusterClass.

The etcd configuration can then be manipulated via the cluster variables. If the etcd property is not specified, then the customization will be skipped.

Example

To change the repository and tag for the container image for the etcd pod, specify the following configuration:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          etcd:
            image:
              repository: my-registry.io/my-org/my-repo
              tag: "v3.5.99_custom.0"

Applying this configuration will result in the following value being set:

KubeadmControlPlaneTemplate:

spec:
  kubeadmConfigSpec:
    clusterConfiguration:
      etcd:
        local:
          imageRepository: "my-registry.io/my-org/my-repo"
          imageTag: "v3.5.99_custom.0"

3 - Extra API Server Certificate SANs

If the API server can be accessed by alternative DNS addresses then setting additional SANs on the API server certificate is necessary in order for clients to successfully validate the API server certificate.

This customization will be available when the provider-specific cluster configuration patch is included in the ClusterClass.

Example

To add extra SANs to the API server certificate, specify the following configuration:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          extraAPIServerCertSANs:
            - a.b.c.example.com
            - d.e.f.example.com

Applying this configuration will result in the following value being set:

KubeadmControlPlaneTemplate:

spec:
  kubeadmConfigSpec:
    clusterConfiguration:
      apiServer:
        certSANs:
          - a.b.c.example.com
          - d.e.f.example.com

4 - Global Image Registry Mirror

Add containerd image registry mirror configuration to all Nodes in the cluster.

When the globalImageRegistryMirror variable is set, files with configurations for Containerd default mirror.

This customization will be available when the provider-specific cluster configuration patch is included in the ClusterClass.

Example

To provide an image registry mirror with a CA certificate, specify the following configuration:

If the registry mirror requires a private or self-signed CA certificate, create a Kubernetes Secret with the ca.crt key populated with the CA certificate in PEM format:

kubectl create secret generic my-mirror-ca-cert \
  --from-file=ca.crt=registry-ca.crt

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          globalImageRegistryMirror:
            url: https://example.com
            credentials:
              secretRef:
                name: my-mirror-ca-cert

Applying this configuration will result in following new files on the KubeadmControlPlaneTemplate and KubeadmConfigTemplate resources:

/etc/containerd/certs.d/_default/hosts.toml
/etc/certs/mirror.pem

To use a public hosted image registry (e.g. ECR) as a registry mirror, specify the following configuration:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          globalImageRegistryMirror:
            url: https://123456789.dkr.ecr.us-east-1.amazonaws.com

Applying this configuration will result in following new files on the KubeadmControlPlaneTemplate and KubeadmConfigTemplate resources:

/etc/containerd/certs.d/_default/hosts.toml

5 - HTTP proxy

In some network environments it is necessary to use HTTP proxy to successfuly execute HTTP requests. This customization will configure Kubernetes components (containerd, kubelet) with appropriate configuration for control plane and worker nodes, utilising systemd drop-ins to configure the necessary environment variables.

This customization will be available when the provider-specific cluster configuration patch is included in the ClusterClass.

Example

To configure HTTP proxy values, specify the following configuration:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          proxy:
            http: http://example.com
            https: http://example.com
            additionalNo:
              - no-proxy-1.example.com
              - no-proxy-2.example.com

The additionalNo list will be added to default pre-calculated values that apply on k8s networking localhost,127.0.0.1,<POD CIDRS>,<SERVICE CIDRS>,kubernetes,kubernetes.default,.svc,.svc.cluster.local, plus provider-specific addresses as required.

Applying this configuration will result in the following value being set:

KubeadmControlPlaneTemplate:

spec:
  kubeadmConfigSpec:
    clusterConfiguration:
      files:
        - path: "/etc/systemd/system/containerd.service.d/http-proxy.conf"
          content: <generated>
        - path: "/etc/systemd/system/kubelet.service.d/http-proxy.conf"
          content: <generated>

KubeadmConfigTemplate:

spec:
  files:
    - path: "/etc/systemd/system/containerd.service.d/http-proxy.conf"
      content: <generated>
    - path: "/etc/systemd/system/kubelet.service.d/http-proxy.conf"
      content: <generated>

Applying this configuration will result in new bootstrap files on the KubeadmControlPlaneTemplate and KubeadmConfigTemplate.

6 - Image registries

Add image registry configuration to all Nodes in the cluster.

When the credentials variable is set, files and preKubeadmnCommands with configurations for Kubelet image credential provider and dynamic credential provider will be added.

This customization will be available when the provider-specific cluster configuration patch is included in the ClusterClass.

Example

If your registry requires static credentials, create a Kubernetes Secret with keys for username and password:

kubectl create secret generic my-registry-credentials \
  --from-literal username=${REGISTRY_USERNAME} --from-literal password=${REGISTRY_PASSWORD}

To add image registry credentials, specify the following configuration:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          imageRegistries:
            - url: https://my-registry.io
              credentials:
                secretRef:
                  name: my-registry-credentials

Applying this configuration will result in new files and preKubeadmCommands on the KubeadmControlPlaneTemplate and KubeadmConfigTemplate.

7 - Kubernetes Image Repository

Override the container image repository used when pulling Kubernetes images.

This customization will be available when the provider-specific cluster configuration patch is included in the ClusterClass.

Example

To configure HTTP proxy values, specify the following configuration:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          kubernetesImageRepository: "my-registry.io/my-org/my-repo"

Applying this configuration will result in the following value being set:

KubeadmControlPlaneTemplate:
- /spec/template/spec/kubeadmConfigSpec/clusterConfiguration/imageRepository: my-registry.io/my-org/my-repo

8 - Users

Configure users for all machines in the cluster, the user's superuser capabilities using sudo user specifications, and the login authentication mechanism.

SSH authorized keys are just public SSH keys that are used to authenticate a login. See the SSH man page for more information.
For information on sudo user specifications, see the sudo documentation.
Local password authentication is disabled for the user by default. It is enabled only when a hashed password is provided.

Examples

Creates a user with the name admin, grants the user the ability to run any command as the superuser, and allows you to login via SSH using the username and private key corresponding to the authorized public key.

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          users:
            - name: username
              sshAuthorizedKeys:
                - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAua0lo8BiGWgvIiDCKnQDKL5uERHfnehm0ns5CEJpJw optionalcomment"
              sudo: "ALL=(ALL) NOPASSWD:ALL"

Creates a user with the name admin, grants the user the ability to run any command as the superuser, and allows you to login via serial console using the username and password.

Note that this does not allow you to login via SSH using the username and password; in most cases, you must also configure the SSH server to allow password authentication.

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          users:
            - name: admin
              hashedPassword: "$y$j9T$UraH8eN4XvapXBmmSaUrP0$Nyxdf1cJDGZcp0WDKu.CFHprrkPG4ubirqSqiD43Ix3"
              sudo: "ALL=(ALL) NOPASSWD:ALL"

Generic

1 - Audit policy

2 - etcd

Example

3 - Extra API Server Certificate SANs

Example

4 - Global Image Registry Mirror

Example

5 - HTTP proxy

Example

6 - Image registries

Example

7 - Kubernetes Image Repository

Example

8 - Users

Examples

Admin user with SSH public key login

Admin user with serial console password login