Most big tech companies provide great documentation on their software. Especially, have you looked at Kubernetes documentation? They are quite awesome.
Particularly, I love the examples that are given pretty much for any topic. You can just blindly follow them to understand Kubernetes concepts.
Today, I chose to learn about how to configure multiple schedulers in a Kubernetes cluster, of course from their documentation. I also wanted to use my Minikube cluster. Learn how to install Minikube here.
Start Minikube cluster.
%minikube start --driver=hyperkit
😄 minikube v1.25.2 on Darwin 12.4
✨ Using the hyperkit driver based on user configuration
👍 Starting control plane node minikube in cluster minikube
🔥 Creating hyperkit VM (CPUs=2, Memory=8192MB, Disk=20000MB) ...
🐳 Preparing Kubernetes v1.23.3 on Docker 20.10.12 ...
▪ kubelet.housekeeping-interval=5m
▪ Generating certificates and keys ...
▪ Booting up control plane ...
▪ Configuring RBAC rules ...
🔎 Verifying Kubernetes components...
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟 Enabled addons: default-storageclass, storage-provisioner
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by defaultFind the image used by kube-scheduler pod running in your Minikube cluster.
%kubectl get pod kube-scheduler-minikube -o yaml -n kube-system | grep image
image: k8s.gcr.io/kube-scheduler:v1.23.3
imagePullPolicy: IfNotPresent
image: k8s.gcr.io/kube-scheduler:v1.23.3
imageID: docker-pullable://k8s.gcr.io/kube-scheduler@sha256:32308abe86f7415611ca86ee79dd0a73e74ebecb2f9e3eb85fc3a8e62f03d0e7Create the YAML file for all the required resources
Here’s the YAML file provided in Kubernetes documentation that creates all the resources required to create another scheduler. Since the second scheduler is also going to use the same image as the above, let’s make sure to use this. What that means is to replace the image mentioned in the documentation with the above image.
apiVersion: v1
kind: ServiceAccount
metadata:
name: my-scheduler
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: my-scheduler-as-kube-scheduler
subjects:
- kind: ServiceAccount
name: my-scheduler
namespace: kube-system
roleRef:
kind: ClusterRole
name: system:kube-scheduler
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: my-scheduler-as-volume-scheduler
subjects:
- kind: ServiceAccount
name: my-scheduler
namespace: kube-system
roleRef:
kind: ClusterRole
name: system:volume-scheduler
apiGroup: rbac.authorization.k8s.io
---
apiVersion: v1
kind: ConfigMap
metadata:
name: my-scheduler-config
namespace: kube-system
data:
my-scheduler-config.yaml: |
apiVersion: kubescheduler.config.k8s.io/v1beta2
kind: KubeSchedulerConfiguration
profiles:
- schedulerName: my-scheduler
leaderElection:
leaderElect: false
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
component: scheduler
tier: control-plane
name: my-scheduler
namespace: kube-system
spec:
selector:
matchLabels:
component: scheduler
tier: control-plane
replicas: 1
template:
metadata:
labels:
component: scheduler
tier: control-plane
version: second
spec:
serviceAccountName: my-scheduler
containers:
- command:
- /usr/local/bin/kube-scheduler
- --config=/etc/kubernetes/my-scheduler/my-scheduler-config.yaml
image: k8s.gcr.io/kube-scheduler:v1.23.3
livenessProbe:
httpGet:
path: /healthz
port: 10259
scheme: HTTPS
initialDelaySeconds: 15
name: kube-second-scheduler
readinessProbe:
httpGet:
path: /healthz
port: 10259
scheme: HTTPS
resources:
requests:
cpu: '0.1'
securityContext:
privileged: false
volumeMounts:
- name: config-volume
mountPath: /etc/kubernetes/my-scheduler
hostNetwork: false
hostPID: false
volumes:
- name: config-volume
configMap:
name: my-scheduler-config
Create all the required objects
%kubectl apply -f my-scheduler.yaml
serviceaccount/my-scheduler created
clusterrolebinding.rbac.authorization.k8s.io/my-scheduler-as-kube-scheduler created
clusterrolebinding.rbac.authorization.k8s.io/my-scheduler-as-volume-scheduler created
configmap/my-scheduler-config created
deployment.apps/my-scheduler createdVerify whether the new scheduler is created
%kubectl get deployment -n kube-system
NAME READY UP-TO-DATE AVAILABLE AGE
coredns 1/1 1 1 23h
my-scheduler 1/1 1 1 8s
%kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-64897985d-4vstq 1/1 Running 0 23h
etcd-minikube 1/1 Running 0 23h
kube-apiserver-minikube 1/1 Running 0 23h
kube-controller-manager-minikube 1/1 Running 0 23h
kube-proxy-w4srd 1/1 Running 0 23h
kube-scheduler-minikube 1/1 Running 0 23h
my-scheduler-5546b885fb-mdgcn 1/1 Running 0 18s
storage-provisioner 1/1 Running 0 23hCreate a pod but this time make it use the new scheduler
%cat pod-using-second-scheduler.yaml
apiVersion: v1
kind: Pod
metadata:
name: annotation-second-scheduler
labels:
name: multischeduler-example
spec:
schedulerName: my-scheduler
containers:
- name: pod-with-second-annotation-container
image: k8s.gcr.io/pause:2.0
%kubectl apply -f pod-using-second-scheduler.yaml
pod/annotation-second-scheduler created
%kubectl get event
LAST SEEN TYPE REASON OBJECT MESSAGE
19s Normal Scheduled pod/annotation-second-scheduler Successfully assigned default/annotation-second-scheduler to minikube
18s Normal Pulling pod/annotation-second-scheduler Pulling image "k8s.gcr.io/pause:2.0"
17s Normal Pulled pod/annotation-second-scheduler Successfully pulled image "k8s.gcr.io/pause:2.0" in 1.25784015s
17s Normal Created pod/annotation-second-scheduler Created container pod-with-second-annotation-container
17s Normal Started pod/annotation-second-scheduler Started container pod-with-second-annotation-containerThat’s it. Ta-daa !
