定制资源配置文件复制代码[rootmaster1 scheduler]# cat scheduler-pod-nodename.yamlapiVersion: apps/v1kind: Deploymentmetadata:name: controller-deployment-demo-2spec:replicas: 1selector:matchLabels:app: myapp2template:metadata:labels:app: myapp2 #注意此处labels和selector中label必须相同spec:nodeName: node1.org #指定调度到指定的节点名称可以支持master1.org忽略污点containers:- name: deployment-2image: registry.cn-beijing.aliyuncs.com/wangxiaochun/pod-test:v0.1复制代码#配置解析nodeName 将应用调度到node1.org节点主机上#应用资源配置文件kubectl apply -f scheduler-pod-nodename.yaml#查看效果rootmaster1 argocd]# kubectl get pod -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATEScontroller-deployment-demo-2-67fdd67478-4x7r9 1/1 Running 0 8m14s 10.244.1.229 node1.org如果节点不可用k8s会自动对节点添加污点导致Pod处于pending状态node1节点不可用复制代码[rootmaster1 scheduler]# cat scheduler-pod-nodename.yamlapiVersion: apps/v1kind: Deploymentmetadata:name: controller-deployment-demo-2spec:replicas: 1selector:matchLabels:app: myapp2template:metadata:labels:app: myapp2 #注意此处labels和selector中label必须相同spec:nodeName: master1.org #指定调度到指定的节点名称可以支持master1.org忽略污点containers:- name: deployment-2image: registry.cn-beijing.aliyuncs.com/wangxiaochun/pod-test:v0.1复制代码kubectl apply -f scheduler-pod-nodename.yaml[rootmaster1 scheduler]#[rootmaster1 scheduler]# kubectl get pod -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATEScontroller-deployment-demo-2-5984594788-xr4pw 1/1 Running 0 16s 10.244.0.40 master1.orgkubectl delete -f scheduler-pod-nodename.yaml范例: 使用nodeSelector调度应用到特定的节点上复制代码[rootmaster1 scheduler]# cat scheduler-pod-nodeselector.yamlapiVersion: apps/v1kind: Deploymentmetadata:name: controller-deployment-demo-2spec:replicas: 1selector:matchLabels:app: myapp2template:metadata:labels:app: myapp2 #注意此处labels和selector中label必须相同spec:nodeSelector:node: gpucontainers:- name: deployment-2image: registry.cn-beijing.aliyuncs.com/wangxiaochun/pod-test:v0.1复制代码#配置解析将应用调度到包含node标签并且值为gpu的节点主机上#应用资源配置文件kubectl apply -f scheduler-pod-nodeselector.yaml#查看效果kubectl get node --show-labels | grep gpu#结果显示由于没有适配标签的节点当前的pod状态是 pending[rootmaster1 scheduler]# kubectl get podNAME READY STATUS RESTARTS AGEcontroller-deployment-demo-2-794898fdf4-j8q7b 0/1 Pending 0 37s#查看原因kubectl describe pod controller-deployment-demo-2-794898fdf4-j8q7bWarning FailedScheduling 64s default-scheduler 0/4 nodes are available: 1 node(s) had untolerated taint(s), 3 node(s) didn’t match Pod’s node affinity/selector. no new claims to deallocate, preemption: 0/4 nodes are available: 4 Preemption is not helpful for scheduling.#为node1节点添加标签kubectl label node node1.org nodegpu[rootmaster1 scheduler]# kubectl get podNAME READY STATUS RESTARTS AGEcontroller-deployment-demo-2-794898fdf4-j8q7b 1/1 Running 0 2m24s#结果显示:一旦有节点满足需求pod就被调度到指定的node节点上#为node1节点清除标签kubectl label node node1.org node-#Pod仍然还在节点上说明nodeSelector仅在调度前有效rootmaster1 scheduler]# kubectl get podNAME READY STATUS RESTARTS AGEcontroller-deployment-demo-2-794898fdf4-j8q7b 1/1 Running 0 3m38s[rootmaster1 scheduler]##清理环境节点亲和性 node affinityhttps://kubernetes.io/zh-cn/docs/tasks/configure-pod-container/assign-pods-nodes-using-node-affinity/image节点调度支持亲和性调度表示Pod和节点的亲和性,即将Pod调度至具有某些标签的节点节点亲和性nodeAffinity 和 nodeSelector 功能相似但支持更丰富条件相当于增强版的nodeSelector节点亲和性只针对调度期间Pod有效,调度后条件发生变化,则无效节点亲和性nodeAffinity不能忽略节点的污点注意: Kubernetes中并没有反亲和的概念,但使用污点技术可以实现类似功能节点亲和分为软亲和和硬亲和kubectl explain pod.spec.affinity.nodeAffinity节点软亲和#倾向性,可以违反,打分逻辑进行排序属于优先机制preferredDuringSchedulingIgnoredDuringExecution节点硬亲和#强制,不能违反,定义明确条件必须满足或必须不能满足属于预选机制requiredDuringSchedulingIgnoredDuringExecution节点硬亲和硬亲和是强制条件,如果不满足,将无法启动pod节点硬亲和属性解析属性解析kubectl explain pod.spec.affinity.nodeAffinitykubectl explain deployment.spec.template.spec.affinity.nodeAffinitykubectl explain daemonset.spec.template.spec.affinity.nodeAffinity复制代码requiredDuringSchedulingIgnoredDuringExecution #硬亲和性,必须满足亲和性才能部署或不部署。nodeSelectorTerms #节点选择术语- matchExpressions #匹配表达式,可用如下多种运算符,如果有多个matchExpressions,只要有一个满足即可调度,即or关系keyvalueoperatoroperator有以下几种 In #label 的值在某个列表中存在即可,即只要匹配其中一个值即可,即or关系 NotIn #label 的值不在某个列表中可以实现反亲和功能 Gt #label 的值大于某个值,只支持数据字比较 Lt #label 的值小于某个值,只支持数据字比较 Exists #某个 label 存在 DoesNotExist #某个 label 不存在可以实现反亲和 matchFields #匹配字段,可以不定义标签值requiredDuringSchedulingIgnoredDuringExecution中的IgnoredDuringExecution #表示在Pod运行期间,节点的标签发生变化,导致亲和策略不能满足时仍会继续运行Pod#如果一个nodeSelectorTerms中只定义了一个matchExpressions中指定多个key来匹配多个条件则所有的key对应的条件都必须满足才会调度到对应的节点即and的关系#如果一个nodeSelectorTerms中定义了多个matchExpressions列表指定了多个条件则只要满足其中一个matchExpressions条件就会被调度到相应的节点上即or的关系复制代码节点硬亲和案例范例: 只有节点包含env标签值是dev 或者 test 才允许部署pod复制代码[rootmaster1 scheduler]# cat scheduler-pod-node-required-affinity.yamlapiVersion: apps/v1kind: Deploymentmetadata:name: controller-deployment-demo-2spec:replicas: 1selector:matchLabels:app: myapp2template:metadata:labels:app: myapp2 #注意此处labels和selector中label必须相同spec:affinity:nodeAffinity:requiredDuringSchedulingIgnoredDuringExecution:nodeSelectorTerms:- matchExpressions:- key: envoperator: Invalues:- dev #多个值中只要有一个值匹配即可即或者关系- testcontainers:- name: deployment-2image: registry.cn-beijing.aliyuncs.com/wangxiaochun/pod-test:v0.1[rootmaster1 scheduler]#复制代码#应用资源定义文件kubectl apply -f scheduler-pod-node-required-affinity.yaml#查看效果如果没有满足硬亲和条件的节点Pod处于Pending状态rootmaster1 scheduler]# kubectl get podNAME READY STATUS RESTARTS AGEcontroller-deployment-demo-2-549df5db5d-m2nbd 0/1 Pending 0 4s#任一节点添加标签kubectl label node node1.org envtest#验证结果rootmaster1 scheduler]# kubectl get pod -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATEScontroller-deployment-demo-2-549df5db5d-m2nbd 1/1 Running 0 49s 10.244.1.35 node1.org[rootmaster1 scheduler]##结果显示只有节点上存在符合标签后该资源才会被创建到指定节点#清理环境kubectl label node node1.org env-kubectl delete -f scheduler-pod-node-required-affinity.yaml节点软亲和软亲和不是必要的强制条件,只是一种偏好而已, 尽量部署在满足条件的节点上如果实在所有节点都不满足软亲和的条件,也可以支持调度运行pod节点软亲和属性解析属性解析kubectl explain pod.spec.affinity.nodeAffinitykubectl explain deploy.spec.template.spec.affinity.nodeAffinity复制代码preferredDuringSchedulingIgnoredDuringExecution #软亲和性能满足最好不满足也没关系。weight #权重值1-100,值越高优先级越高preference #偏好设置matchExpressionsmatchFields#对于满足所有调度要求的每个节点调度程序将通过迭代此字段的元素计算总和#并在节点与对应的节点匹配时将“权重”添加到总和。#注意与硬亲和属性不同这里需要注意的是软亲和属性是一个列表对象而preference不是一个列表项。复制代码节点软亲和案例范例: 只要节点包含env标签值是dev 或者 test就优先部署pod#资源文件内容复制代码[rootmaster1 scheduler]# cat scheduler-pod-node-preferred-affinity.yamlapiVersion: apps/v1kind: Deploymentmetadata:name: controller-deployment-demo-2spec:replicas: 1selector:matchLabels:app: myapp2template:metadata:labels:app: myapp2 #注意此处labels和selector中label必须相同spec:affinity:nodeAffinity:preferredDuringSchedulingIgnoredDuringExecution:- weight: 50preference:matchExpressions:- key: envoperator: Invalues:- test- weight: 20preference:matchExpressions:- key: envoperator: Invalues:- devcontainers:- name: deployment-2image: registry.cn-beijing.aliyuncs.com/wangxiaochun/pod-test:v0.1复制代码#注意软亲和的权重配置是与硬亲和的区别所在。#准备节点标签kubectl label node node1.org envdevkubectl label node node2.org envtestkubectl get nodes -l envrootmaster1 scheduler]# kubectl get nodes -l envNAME STATUS ROLES AGE VERSIONnode1.org Ready 17d v1.35.6node2.org NotReady 17d v1.35.6kubectl apply -f scheduler-pod-node-preferred-affinity.yaml#观察结果[rootmaster1 scheduler]# kubectl get pod -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATEScontroller-deployment-demo-2-9fd995f9-tvkqc 1/1 Running 0 15s 10.244.1.36 node1.org[rootmaster1 scheduler]##结果显示由于 node2上的标签权重较高所以就交给了node2#清除node2节点标签后重启资源对象kubectl label node node2.org env-#清理环境kubectl delete -f scheduler-pod-node-preferred-affinity.yaml#再次创建podkubectl apply -f scheduler-pod-node-preferred-affinity.yaml#观察结果[rootmaster1 scheduler]# kubectl get pod -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATEScontroller-deployment-demo-2-9fd995f9-r6mh6 1/1 Running 0 17s 10.244.1.37 node1.org#结果显示这个时候资源调度到了node1上