1. Kubernetes集群部署实践概述在完成Kubernetes集群的基础搭建后上篇已详细介绍我们进入更具挑战性的部署实践环节。作为容器编排的事实标准Kubernetes在生产环境中的部署需要考虑网络策略、存储方案、安全配置等关键因素。本实践基于三节点集群1 master 2 worker架构使用kubeadm工具初始化Calico作为CNI插件。提示生产环境建议至少3个master节点实现高可用本文为演示采用单master架构2. 集群网络配置与优化2.1 Calico网络插件部署Calico是Kubernetes最常用的网络方案之一提供高性能的网络连接和灵活的网络策略kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml部署后需要验证所有节点状态变为Ready跨节点Pod能够互通DNS解析正常常见问题处理如果节点一直处于NotReady状态检查kubelet日志journalctl -u kubelet -f网络不通时排查Calico的felix组件日志kubectl logs -n kube-system -l k8s-appcalico-node2.2 网络策略配置示例生产环境必须配置网络策略实现最小权限访问apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny spec: podSelector: {} policyTypes: - Ingress - Egress3. 存储方案设计与实现3.1 动态存储供应配置使用NFS作为后端存储的StorageClass配置apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: nfs-storage provisioner: example.com/nfs parameters: server: nfs-server.example.com path: /share3.2 StatefulSet有状态应用部署部署Redis集群的StatefulSet示例apiVersion: apps/v1 kind: StatefulSet metadata: name: redis spec: serviceName: redis replicas: 3 selector: matchLabels: app: redis template: metadata: labels: app: redis spec: containers: - name: redis image: redis:6.2 ports: - containerPort: 6379 volumeMounts: - name: data mountPath: /data volumeClaimTemplates: - metadata: name: data spec: accessModes: [ ReadWriteOnce ] storageClassName: nfs-storage resources: requests: storage: 1Gi4. CI/CD流水线集成4.1 Jenkins在K8s中的部署使用Helm部署Jenkinshelm repo add jenkins https://charts.jenkins.io helm install my-jenkins jenkins/jenkins \ --set controller.serviceTypeNodePort \ --set controller.serviceNodePort300004.2 GitLab Runner配置示例Kubernetes executor配置片段[[runners]] name k8s-runner url https://gitlab.example.com token xxxxxx executor kubernetes [runners.kubernetes] namespace gitlab image alpine:latest5. 监控与日志方案5.1 Prometheus-Operator部署使用Helm部署监控栈helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm install kube-prometheus prometheus-community/kube-prometheus-stack5.2 EFK日志收集方案Fluentd的DaemonSet配置要点apiVersion: apps/v1 kind: DaemonSet metadata: name: fluentd spec: template: spec: containers: - name: fluentd image: fluent/fluentd-kubernetes-daemonset:v1.14-debian-elasticsearch7-1 env: - name: FLUENT_ELASTICSEARCH_HOST value: elasticsearch-logging - name: FLUENT_ELASTICSEARCH_PORT value: 92006. 安全加固实践6.1 RBAC权限控制开发人员只读权限的Role示例apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: namespace: dev name: developer-readonly rules: - apiGroups: [] resources: [pods, services] verbs: [get, list, watch]6.2 Pod安全策略限制特权容器的PSP示例apiVersion: policy/v1beta1 kind: PodSecurityPolicy metadata: name: restricted spec: privileged: false runAsUser: rule: MustRunAsNonRoot seLinux: rule: RunAsAny volumes: - configMap - emptyDir7. 应用部署实战案例7.1 微服务应用部署Spring Cloud应用的Deployment配置要点apiVersion: apps/v1 kind: Deployment metadata: name: user-service spec: strategy: rollingUpdate: maxSurge: 1 maxUnavailable: 0 template: spec: containers: - name: user-service image: registry.example.com/user-service:1.2.0 readinessProbe: httpGet: path: /actuator/health port: 8080 initialDelaySeconds: 30 periodSeconds: 107.2 数据库高可用方案MySQL集群的StatefulSet关键配置apiVersion: apps/v1 kind: StatefulSet metadata: name: mysql spec: serviceName: mysql replicas: 3 template: spec: initContainers: - name: init-mysql command: - bash - -c - | # 初始化脚本内容 containers: - name: mysql image: mysql:5.7 env: - name: MYSQL_ROOT_PASSWORD valueFrom: secretKeyRef: name: mysql-secret key: password8. 集群运维与问题排查8.1 常用诊断命令节点资源检查kubectl top nodesPod事件查看kubectl describe pod pod-name8.2 典型问题处理方案Pod一直处于Pending状态的可能原因资源不足CPU/内存没有满足条件的节点节点选择器/污点PV绑定失败排查步骤kubectl get events --sort-by.metadata.creationTimestamp kubectl describe pod pod-name9. 集群升级与维护9.1 滚动升级策略Deployment的升级配置示例spec: strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 25% maxSurge: 19.2 节点维护操作安全排空节点kubectl drain node-name --ignore-daemonsets维护完成后重新调度kubectl uncordon node-name10. 最佳实践总结在实际生产环境中部署Kubernetes集群时有几个关键经验值得注意网络策略应该尽早规划采用零信任模型存储方案需要根据应用特点选择性能 vs 可用性CI/CD流水线要与Kubernetes的部署特性深度集成监控指标需要包含应用指标和集群指标两个维度安全配置应该遵循最小权限原则对于资源限制的配置建议所有Pod必须设置requests和limits使用LimitRange设置默认值使用ResourceQuota限制命名空间资源总量一个经过验证的配置示例resources: requests: cpu: 500m memory: 512Mi limits: cpu: 1000m memory: 1Gi Kubernetes集群部署与网络配置实战指南
1. Kubernetes集群部署实践概述在完成Kubernetes集群的基础搭建后上篇已详细介绍我们进入更具挑战性的部署实践环节。作为容器编排的事实标准Kubernetes在生产环境中的部署需要考虑网络策略、存储方案、安全配置等关键因素。本实践基于三节点集群1 master 2 worker架构使用kubeadm工具初始化Calico作为CNI插件。提示生产环境建议至少3个master节点实现高可用本文为演示采用单master架构2. 集群网络配置与优化2.1 Calico网络插件部署Calico是Kubernetes最常用的网络方案之一提供高性能的网络连接和灵活的网络策略kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml部署后需要验证所有节点状态变为Ready跨节点Pod能够互通DNS解析正常常见问题处理如果节点一直处于NotReady状态检查kubelet日志journalctl -u kubelet -f网络不通时排查Calico的felix组件日志kubectl logs -n kube-system -l k8s-appcalico-node2.2 网络策略配置示例生产环境必须配置网络策略实现最小权限访问apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny spec: podSelector: {} policyTypes: - Ingress - Egress3. 存储方案设计与实现3.1 动态存储供应配置使用NFS作为后端存储的StorageClass配置apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: nfs-storage provisioner: example.com/nfs parameters: server: nfs-server.example.com path: /share3.2 StatefulSet有状态应用部署部署Redis集群的StatefulSet示例apiVersion: apps/v1 kind: StatefulSet metadata: name: redis spec: serviceName: redis replicas: 3 selector: matchLabels: app: redis template: metadata: labels: app: redis spec: containers: - name: redis image: redis:6.2 ports: - containerPort: 6379 volumeMounts: - name: data mountPath: /data volumeClaimTemplates: - metadata: name: data spec: accessModes: [ ReadWriteOnce ] storageClassName: nfs-storage resources: requests: storage: 1Gi4. CI/CD流水线集成4.1 Jenkins在K8s中的部署使用Helm部署Jenkinshelm repo add jenkins https://charts.jenkins.io helm install my-jenkins jenkins/jenkins \ --set controller.serviceTypeNodePort \ --set controller.serviceNodePort300004.2 GitLab Runner配置示例Kubernetes executor配置片段[[runners]] name k8s-runner url https://gitlab.example.com token xxxxxx executor kubernetes [runners.kubernetes] namespace gitlab image alpine:latest5. 监控与日志方案5.1 Prometheus-Operator部署使用Helm部署监控栈helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm install kube-prometheus prometheus-community/kube-prometheus-stack5.2 EFK日志收集方案Fluentd的DaemonSet配置要点apiVersion: apps/v1 kind: DaemonSet metadata: name: fluentd spec: template: spec: containers: - name: fluentd image: fluent/fluentd-kubernetes-daemonset:v1.14-debian-elasticsearch7-1 env: - name: FLUENT_ELASTICSEARCH_HOST value: elasticsearch-logging - name: FLUENT_ELASTICSEARCH_PORT value: 92006. 安全加固实践6.1 RBAC权限控制开发人员只读权限的Role示例apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: namespace: dev name: developer-readonly rules: - apiGroups: [] resources: [pods, services] verbs: [get, list, watch]6.2 Pod安全策略限制特权容器的PSP示例apiVersion: policy/v1beta1 kind: PodSecurityPolicy metadata: name: restricted spec: privileged: false runAsUser: rule: MustRunAsNonRoot seLinux: rule: RunAsAny volumes: - configMap - emptyDir7. 应用部署实战案例7.1 微服务应用部署Spring Cloud应用的Deployment配置要点apiVersion: apps/v1 kind: Deployment metadata: name: user-service spec: strategy: rollingUpdate: maxSurge: 1 maxUnavailable: 0 template: spec: containers: - name: user-service image: registry.example.com/user-service:1.2.0 readinessProbe: httpGet: path: /actuator/health port: 8080 initialDelaySeconds: 30 periodSeconds: 107.2 数据库高可用方案MySQL集群的StatefulSet关键配置apiVersion: apps/v1 kind: StatefulSet metadata: name: mysql spec: serviceName: mysql replicas: 3 template: spec: initContainers: - name: init-mysql command: - bash - -c - | # 初始化脚本内容 containers: - name: mysql image: mysql:5.7 env: - name: MYSQL_ROOT_PASSWORD valueFrom: secretKeyRef: name: mysql-secret key: password8. 集群运维与问题排查8.1 常用诊断命令节点资源检查kubectl top nodesPod事件查看kubectl describe pod pod-name8.2 典型问题处理方案Pod一直处于Pending状态的可能原因资源不足CPU/内存没有满足条件的节点节点选择器/污点PV绑定失败排查步骤kubectl get events --sort-by.metadata.creationTimestamp kubectl describe pod pod-name9. 集群升级与维护9.1 滚动升级策略Deployment的升级配置示例spec: strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 25% maxSurge: 19.2 节点维护操作安全排空节点kubectl drain node-name --ignore-daemonsets维护完成后重新调度kubectl uncordon node-name10. 最佳实践总结在实际生产环境中部署Kubernetes集群时有几个关键经验值得注意网络策略应该尽早规划采用零信任模型存储方案需要根据应用特点选择性能 vs 可用性CI/CD流水线要与Kubernetes的部署特性深度集成监控指标需要包含应用指标和集群指标两个维度安全配置应该遵循最小权限原则对于资源限制的配置建议所有Pod必须设置requests和limits使用LimitRange设置默认值使用ResourceQuota限制命名空间资源总量一个经过验证的配置示例resources: requests: cpu: 500m memory: 512Mi limits: cpu: 1000m memory: 1Gi