1. Kubernetes集群搭建全景指南在云原生技术席卷全球的当下Kubernetes已成为容器编排领域的事实标准。最近我在生产环境中完成了一个多节点Kubernetes集群的部署整个过程既是对理论知识的实践验证也是对各类坑点的集中排查。本文将完整呈现从零开始搭建高可用Kubernetes集群的全过程特别分享那些官方文档未曾提及的实战细节。2. 集群规划与基础环境准备2.1 硬件资源配置方案生产级Kubernetes集群的硬件需求取决于工作负载特性。我的配置方案如下控制平面节点3台实现高可用4核CPU/8GB内存/100GB磁盘SSD专用于运行kube-apiserver、kube-scheduler等核心组件工作节点根据业务需求动态扩展基础配置8核CPU/16GB内存/200GB磁盘SSDGPU节点额外配备NVIDIA T4显卡关键经验控制平面节点必须使用奇数台3/5/7这是etcd集群实现共识算法的基础条件。2.2 操作系统调优所有节点统一使用Ubuntu 20.04 LTS并进行以下内核参数调整# 禁用swapKubernetes强制要求 sudo swapoff -a sed -i / swap / s/^/#/ /etc/fstab # 加载br_netfilter模块 sudo modprobe br_netfilter echo br_netfilter | sudo tee /etc/modules-load.d/k8s.conf # 设置网络参数 cat EOF | sudo tee /etc/sysctl.d/k8s.conf net.bridge.bridge-nf-call-ip6tables 1 net.bridge.bridge-nf-call-iptables 1 net.ipv4.ip_forward 1 EOF sudo sysctl --system2.3 容器运行时选型经过对比测试最终选择containerd作为容器运行时# 安装containerd sudo apt-get update sudo apt-get install -y containerd sudo mkdir -p /etc/containerd containerd config default | sudo tee /etc/containerd/config.toml sudo systemctl restart containerd # 配置cgroup驱动为systemd sudo sed -i s/SystemdCgroup false/SystemdCgroup true/ /etc/containerd/config.toml3. 使用kubeadm部署集群3.1 控制平面初始化在主节点执行注意替换API Server地址sudo kubeadm init \ --control-plane-endpoint192.168.1.100:6443 \ --upload-certs \ --pod-network-cidr10.244.0.0/16 \ --service-cidr10.96.0.0/12 \ --image-repository registry.aliyuncs.com/google_containers初始化成功后按照提示配置kubectlmkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config3.2 高可用控制平面添加其余控制平面节点kubeadm join 192.168.1.100:6443 \ --token token \ --discovery-token-ca-cert-hash hash \ --control-plane \ --certificate-key key3.3 工作节点加入集群在工作节点执行sudo kubeadm join 192.168.1.100:6443 \ --token token \ --discovery-token-ca-cert-hash hash4. 网络插件与核心组件部署4.1 Flannel网络配置选择flannel作为CNI插件kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml验证网络状态kubectl get pods -n kube-system -l appflannel4.2 核心插件部署Metrics Server集群监控kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yamlDashboard可视化界面kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml5. 集群验证与测试5.1 基础状态检查# 查看节点状态 kubectl get nodes -o wide # 检查核心组件健康状态 kubectl get pods -n kube-system # 测试DNS解析 kubectl run -it --rm --restartNever busybox --imagebusybox -- nslookup kubernetes.default5.2 工作负载测试部署测试应用apiVersion: apps/v1 kind: Deployment metadata: name: nginx-test spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:alpine ports: - containerPort: 80验证服务访问kubectl expose deployment nginx-test --port80 --typeNodePort kubectl get svc nginx-test curl http://node-ip:node-port6. 生产环境关键配置6.1 证书自动续期修改kubeadm配置实现证书自动续期sudo sed -i s/rotateCertificates: false/rotateCertificates: true/ /etc/kubernetes/kubelet.conf sudo systemctl restart kubelet6.2 资源配额管理为命名空间设置资源限制apiVersion: v1 kind: ResourceQuota metadata: name: mem-cpu-quota spec: hard: requests.cpu: 2 requests.memory: 4Gi limits.cpu: 4 limits.memory: 8Gi6.3 节点亲和性配置确保关键Pod分布在不同节点affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app operator: In values: - nginx topologyKey: kubernetes.io/hostname7. 故障排查手册7.1 常见问题速查表故障现象排查命令解决方案Pod一直Pendingkubectl describe pod name检查资源配额、节点选择器服务无法访问kubectl get endpoints svc验证标签选择器匹配DNS解析失败kubectl logs -n kube-system coredns-pod检查CoreDNS配置7.2 日志收集技巧集中查看控制平面日志journalctl -u kubelet -f tail -f /var/log/kube-apiserver.log kubectl logs -n kube-system pod-name --previous7.3 网络问题诊断使用nsenter进入容器网络命名空间PID$(docker inspect -f {{.State.Pid}} container-id) sudo nsenter -t $PID -n ip a8. 性能优化实践8.1 etcd调优参数# /etc/kubernetes/manifests/etcd.yaml spec: containers: - command: - etcd - --heartbeat-interval100 - --election-timeout500 - --snapshot-count100008.2 kubelet资源预留防止系统进程资源不足# /var/lib/kubelet/config.yaml systemReserved: cpu: 500m memory: 1Gi kubeReserved: cpu: 500m memory: 1Gi8.3 API Server限流配置# /etc/kubernetes/manifests/kube-apiserver.yaml spec: containers: - command: - kube-apiserver - --max-requests-inflight800 - --max-mutating-requests-inflight400