k8s部署
·
k8s部署——Rocky Linux 9.4 搭建k8s + docker一主多从集群测试环境
一、kubernetes集群规划
集群各节点资源规划分配
| 主机名 | IP地址 | 角色 | 操作系统 | 硬件配置 |
|---|---|---|---|---|
| master | 192.168.100.100 | 管理节点 | Rocky Linux 9.4 | 4core/4G内存/50G |
| node1 | 192.168.100.110 | 工作节点 | Rocky Linux 9.4 | 4core/4G内存/50G |
| node2 | 192.168.100.120 | 工作节点 | Rocky Linux 9.4 | 4core/4G内存/50G |
二、环境准备工作
1、主机名称
#管理节点，192.168.100.100
[root@master ~]# hostname
master
#工作节点，192.168.100.110
[root@master ~]# hostnamectl hostname node1
[root@master ~]# bash
[root@node1 ~]# hostname
node1
#工作节点，192.168.100.120
[root@master ~]# hostnamectl hostname node2
[root@master ~]# bash
[root@node2 ~]# hostname
node2
2、关闭防火墙selinux（略）
3、可以做一下免密钥（全部节点）
[root@master ~]# ssh-keygen
//都回车
[root@master ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub root@192.168.100.110
yes
输入node1主机密码
[root@master ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub root@192.168.100.120
yes
输入node2主机密码
4、配置系统hosts文件（全部节点）
[root@master ~]# vim /etc/hosts
[root@master ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.100.100 master
192.168.100.110 node1
192.168.100.120 node2
[root@master ~]# scp /etc/hosts root@node2:/etc/hosts
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'node2' (ED25519) to the list of known hosts.
hosts 100% 225 40.9KB/s 00:00
[root@master ~]# scp /etc/hosts root@node1:/etc/hosts
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'node1' (ED25519) to the list of known hosts.
hosts 100% 225 49.5KB/s 00:00
5、关闭swap分区（全部节点）
[root@master ~]# free -m
total used free shared buff/cache available
Mem: 3883 392 3559 8 107 3491
Swap: 4095 0 4095
[root@master ~]# vim /etc/fstab
#注释swap的行
#UUID=7525cd5c-280d-47e7-9d7e-42136bbf8b98 none swap defaults 0 0
[root@master ~]# swapoff -a
[root@master ~]# free -m
total used free shared buff/cache available
Mem: 3883 387 3564 8 107 3496
Swap: 0 0 0
6、时钟同步（全部节点）
[root@node1 ~]# yum -y install chrony
[root@node1 ~]# systemctl restart chronyd
[root@node1 ~]# systemctl enable chronyd
[root@node1 ~]# hwclock -w
[root@node1 ~]# timedatectl
Local time: Tue 2025-10-28 15:09:04 CST
Universal time: Tue 2025-10-28 07:09:04 UTC
RTC time: Tue 2025-10-28 07:09:04
Time zone: Asia/Shanghai (CST, +0800)
System clock synchronized: yes
NTP service: active
RTC in local TZ: no
7、配置国内系统镜像源和安装epel源
（1）配置阿里云系统源（全部节点）。阿里云网上的文件名是Rocky.repo，系统上的是rocky-*.repo
[root@master ~]# cd /etc/yum.repos.d/
[root@master yum.repos.d]# ls
rocky-addons.repo rocky-devel.repo rocky-extras.repo rocky.repo
[root@master yum.repos.d]# sed -e 's|^mirrorlist=|#mirrorlist=|g' \
> -e 's|^#baseurl=http://dl.rockylinux.org/$contentdir|baseurl=https://mirrors.aliyun.com/rockylinux|g' \
> -i.bak \
> /etc/yum.repos.d/rocky-*.repo
[root@master yum.repos.d]# ls
rocky-addons.repo rocky-devel.repo rocky-extras.repo rocky.repo
rocky-addons.repo.bak rocky-devel.repo.bak rocky-extras.repo.bak
[root@master yum.repos.d]# yum makecache
Rocky Linux 9 - BaseOS 108 B/s | 4.1 kB 00:38
Rocky Linux 9 - AppStream 137 B/s | 4.5 kB 00:33
Rocky Linux 9 - Extras 964 B/s | 17 kB 00:18
Metadata cache created.
[root@master yum.repos.d]# yum -y install epel-release
[root@master yum.repos.d]# ls
epel-cisco-openh264.repo rocky-addons.repo rocky-devel.repo.bak rocky.repo
epel.repo rocky-addons.repo.bak rocky-extras.repo
epel-testing.repo rocky-devel.repo rocky-extras.repo.bak
（2）配置epel源（全部节点）
文件内容取自我的华为云主机如下
[root@master yum.repos.d]# vim /etc/yum.repos.d/epel.repo
[root@master yum.repos.d]# cat /etc/yum.repos.d/epel.repo
[epel]
name=Extra Packages for Enterprise Linux $releasever - $basearch
# It is much more secure to use the metalink, but if you wish to use a local mirror
# place its address here.
#baseurl=https://download.example/pub/epel/$releasever/Everything/$basearch/
#metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-$releasever&arch=$basearch&infra=$infra&content=$contentdir
baseurl=https://repo.huaweicloud.com/epel/$releasever/Everything/$basearch/
enabled=1
gpgcheck=1
countme=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-$releasever
[epel-debuginfo]
name=Extra Packages for Enterprise Linux $releasever - $basearch - Debug
# It is much more secure to use the metalink, but if you wish to use a local mirror
# place its address here.
#baseurl=https://download.example/pub/epel/$releasever/Everything/$basearch/debug/
#metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-debug-$releasever&arch=$basearch&infra=$infra&content=$contentdir
baseurl=https://repo.huaweicloud.com/epel/$releasever/Everything/$basearch/debug/
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-$releasever
gpgcheck=1
[epel-source]
name=Extra Packages for Enterprise Linux $releasever - $basearch - Source
# It is much more secure to use the metalink, but if you wish to use a local mirror
# place its address here.
#baseurl=https://download.example/pub/epel/$releasever/Everything/source/tree/
#metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-source-$releasever&arch=$basearch&infra=$infra&content=$contentdir
baseurl=https://repo.huaweicloud.com/epel/$releasever/Everything/source/tree/
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-$releasever
gpgcheck=1
[root@master yum.repos.d]# scp /etc/yum.repos.d/epel.repo root@node1:/etc/yum.repos.d/
epel.repo 100% 1692 1.0MB/s 00:00
[root@master yum.repos.d]# scp /etc/yum.repos.d/epel.repo root@node2:/etc/yum.repos.d/
epel.repo 100% 1692 253.6KB/s 00:00
（3）然后全部yum makecache
生成并缓存新的软件源元数据（仓库中软件包的索引信息），后续使用 yum/dnf 安装软件时会更快（无需每次从远程仓库重新拉取元数据）
[root@master yum.repos.d]# yum makecache
Extra Packages for Enterprise Linux 9 - x86_64 10 MB/s | 20 MB 00:01
Extra Packages for Enterprise Linux 9 openh264 (From Ci 276 B/s | 2.5 kB 00:09
Rocky Linux 9 - BaseOS 2.5 kB/s | 4.1 kB 00:01
Rocky Linux 9 - AppStream 4.4 kB/s | 4.5 kB 00:01
Rocky Linux 9 - Extras 15 kB/s | 2.9 kB 00:00
Metadata cache created.
8、修改系统最大打开文件数（全部节点）
#编辑文件，在文件后面添加一下两行数据
[root@master ~]# vim /etc/security/limits.conf
* soft nofile 65535
* hard nofile 65535
9、修改内核参数（全部节点），运行一下命令使得配置的sysctl.conf文件生效
#打开文件，在文件后面填入以下几行
[root@master ~]# vim /etc/sysctl.conf
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_tw_buckets = 20480
net.ipv4.tcp_max_syn_backlog = 20480
net.core.netdev_max_backlog = 262144
net.ipv4.tcp_fin_timeout = 20
[root@master ~]# sysctl -p
10、安装系统性能分析工具和其他（全部节点）
[root@master ~]# yum -y install gcc autoconf sysstat
11、开启bridge网桥过滤（全部节点）
编辑文件写入以下几行，加载br_netfilter模块，并查看，加载配置文件，使其生效
[root@master ~]# vim /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
[root@master ~]# modprobe br_netfilter
[root@master ~]# lsmod | grep br_netfilter
br_netfilter 36864 0
bridge 409600 1 br_netfilter
[root@master ~]# sysctl -p /etc/sysctl.d/k8s.conf
三、 Docker环境准备
1、配置阿里云docker源（全部节点）
[root@master yum.repos.d]# yum -y install yum-utils
[root@master ~]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
Adding repo from: http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
[root@master ~]# cd /etc/yum.repos.d/
[root@master yum.repos.d]# ls
docker-ce.repo rocky-addons.repo rocky-extras.repo
epel-cisco-openh264.repo rocky-addons.repo.bak rocky-extras.repo.bak
epel.repo rocky-devel.repo rocky.repo
epel-testing.repo rocky-devel.repo.bak
2、查看可用的docker版本（全部节点）
[root@master ~]# yum list docker-ce.x86_64 --showduplicates | sort -r
3、安装docker，指定安装25.0.5-1.el9版本（全部节点）
[root@master ~]# yum -y install docker-ce-25.0.5-1.el9
4、配置Docker Cgroup控制组（全部节点）
[root@master ~]# vim /etc/docker/daemon.json
[root@master ~]# cat /etc/docker/daemon.json
{
"registry-mirrors": [
"https://docker.m.daocloud.io",
"https://dockerproxy.com",
"https://docker.mirrors.ustc.edu.cn",
"https://docker.nju.edu.cn"
],
"exec-opts": ["native.cgroupdriver=systemd"]
}
5、启动Docker服务并设置随机自启（全部节点）
[root@master ~]# systemctl daemon-reload
[root@master ~]# systemctl restart docker
[root@master ~]# systemctl enable docker
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.
四、配置cri-docker
1、直接导入包（这里已经下载好了，直接拖进来即可）
[root@master ~]# yum -y install lrzsz
[root@master ~]# rz -E
rz waiting to receive.
[root@master ~]# ls
anaconda-ks.cfg cri-dockerd-0.3.9.amd64.tgz
2、解压cri-docker
[root@master ~]# tar -xvf cri-dockerd-0.3.9.amd64.tgz --strip-components=1 -C /usr/local/bin/
cri-dockerd/cri-dockerd
3、下载cri-docker service文件
[root@master ~]# yum -y install wget
[root@master ~]# wget -O /etc/systemd/system/cri-docker.service https://raw.githubusercontent.com/Mirantis/cri-dockerd/master/packaging/systemd/cri-docker.service
[root@master ~]# wget -O /etc/systemd/system/cri-docker.socket https://raw.githubusercontent.com/Mirantis/cri-dockerd/master/packaging/systemd/cri-docker.socket
4、编辑cri-docker.server
修改ExecStart行内容为
[root@master ~]# vim /etc/systemd/system/cri-docker.service
ExecStart=/usr/local/bin/cri-dockerd --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.9 --container-runtime-endpoint=unix:///var/run/cri-dockerd.sock --cri-dockerd-root-directory=/var/lib/dockershim --cri-dockerd-root-directory=/var/lib/docker
5、编辑cri-docker.socket
修改ListenStream行内容为
[root@master ~]# vim /etc/systemd/system/cri-docker.socket
ListenStream=/var/run/cri-dockerd.sock
6、复制cri-dockerd-0.3.9.amd64.tgz到其他节点
[root@master ~]# ls
anaconda-ks.cfg cri-dockerd-0.3.9.amd64.tgz
[root@master ~]# scp cri-dockerd-0.3.9.amd64.tgz root@node1:/root/
cri-dockerd-0.3.9.amd64.tgz 100% 14MB 68.1MB/s 00:00
[root@master ~]# scp cri-dockerd-0.3.9.amd64.tgz root@node2:/root/
cri-dockerd-0.3.9.amd64.tgz 100% 14MB 66.7MB/s 00:00
7、其他节点解压cri-docker
[root@node1 ~]# tar -xvf cri-dockerd-0.3.9.amd64.tgz --strip-components=1 -C /usr/local/bin/
cri-dockerd/cri-dockerd
[root@node2 ~]# tar -xvf cri-dockerd-0.3.9.amd64.tgz --strip-components=1 -C /usr/local/bin/
cri-dockerd/cri-dockerd
8、复制修改好的文件到其他节点（两个）
[root@master ~]# scp /etc/systemd/system/cri-docker.s* root@node1:/etc/systemd/system/
cri-docker.service 100% 1591 813.0KB/s 00:00
cri-docker.socket 100% 244 194.0KB/s 00:00
[root@master ~]# scp /etc/systemd/system/cri-docker.s* root@node2:/etc/systemd/system/
cri-docker.service 100% 1591 992.5KB/s 00:00
cri-docker.socket 100% 244 188.7KB/s 00:00
9、启动并设置自启动（全部节点）
[root@master ~]# systemctl daemon-reload
[root@master ~]# systemctl restart cri-docker
[root@master ~]# systemctl enable cri-docker
Created symlink /etc/systemd/system/multi-user.target.wants/cri-docker.service → /etc/systemd/system/cri-docker.service.
五、kubeadm部署kubernetes集群
1、配置阿里云k8s源（全部节点）
[root@master ~]# vim /etc/yum.repos.d/k8s.repo
[root@master ~]# cat /etc/yum.repos.d/k8s.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.28/rpm/
enabled=1
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.28/rpm/repodata/repomd.xml.key
[root@master ~]# scp /etc/yum.repos.d/k8s.repo root@node1:/etc/yum.repos.d/
k8s.repo 100% 218 87.1KB/s 00:00
[root@master ~]# scp /etc/yum.repos.d/k8s.repo root@node2:/etc/yum.repos.d/
k8s.repo 100% 218 178.0KB/s 00:00
2、安装集群所需软件包kubelet kubeadm kubectl（全部节点）
[root@master ~]# yum install -y kubelet kubeadm kubectl
3、配置k8s Cgoup控制组（全部节点）
[root@master ~]# vim /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS="--cgroup-driver=systemd"
4、配置kubelet自启动（全部节点）
[root@master ~]# systemctl enable kubelet.service
Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /usr/lib/systemd/system/kubelet.service.
5、 初始化集群（master节点运行）
（1）打印master节点所需的镜像文件（master节点运行）
[root@master ~]# kubeadm config images list
I1028 17:03:55.436696 6562 version.go:256] remote version is much newer: v1.34.1; falling back to: stable-1.28
registry.k8s.io/kube-apiserver:v1.28.15
registry.k8s.io/kube-controller-manager:v1.28.15
registry.k8s.io/kube-scheduler:v1.28.15
registry.k8s.io/kube-proxy:v1.28.15
registry.k8s.io/pause:3.9
registry.k8s.io/etcd:3.5.15-0
registry.k8s.io/coredns/coredns:v1.10.1
（2）打印集群初始化配置文件（master节点运行）
[root@master ~]# kubeadm config print init-defaults > kubeadm-config.yaml
（3）修改参数（master节点）
[root@master ~]# vim kubeadm-config.yaml
…
localAPIEndpoint:
advertiseAddress: 192.168.100.100 #修改，集群初始化的主节点IP
bindPort: 6443
nodeRegistration:
criSocket: unix:///var/run/cri-dockerd.sock #修改使用docker
imagePullPolicy: IfNotPresent
name: master01 #修改节点名称
taints: null
…
…
…
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers #修改使用阿里云镜像仓库
…
（4）使用配置文件初始化（master节点运行）
–upload-certs参数是将集群密钥添加到etcd数据库，可以将输出的命令先存放在一个自定义文件里
[root@master ~]# kubeadm init --config kubeadm-config.yaml --upload-certs
（5）配置环境变量（master节点运行）
根据初始化完成提示运行下面行
[root@master ~]# mkdir -p $HOME/.kube
[root@master ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@master ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config
[root@master ~]# export KUBECONFIG=/etc/kubernetes/admin.conf
（6）工作节点加入集群（工作节点运行）
根据初始化完成提示将工作节点添加入集群
注意：–cri-socket=unix:///var/run/cri-dockerd.sock参数是指定使用docker作为容器管理引擎
node1：
[root@node1 ~]# kubeadm join 192.168.100.100:6443 --token abcdef.0123456789abcdef \
--discovery-token-ca-cert-hash sha256:3046e6b51b8fc9f0693ef86bc0443ded82bbdb30091117a3ea103f2e539a3302 --cri-socket=unix:///var/run/cri-dockerd.sock
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
node2：
[root@node2 ~]# kubeadm join 192.168.100.100:6443 --token abcdef.0123456789abcdef \
--discovery-token-ca-cert-hash sha256:3046e6b51b8fc9f0693ef86bc0443ded82bbdb30091117a3ea103f2e539a3302 --cri-socket=unix:///var/run/cri-dockerd.sock
（7）下载calico文件（master节点运行）
[root@master ~]# rz -E
rz waiting to receive.
[root@master ~]# ls
aa anaconda-ks.cfg calico.yaml cri-dockerd-0.3.9.amd64.tgz kubeadm-config.yaml
（8）查看集群各节点状态（等待一段时间）
[root@master ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master Ready control-plane 21m v1.28.15
node1 Ready <none> 12m v1.28.15
node2 Ready <none> 11m v1.28.15
（9）查看k8s集群的各组件
[root@master ~]# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-9d57d8f49-5zfvc 1/1 Running 0 3m57s
calico-node-nr2ml 1/1 Running 0 3m57s
calico-node-rbpwd 1/1 Running 0 3m57s
calico-node-wfrtx 1/1 Running 0 3m57s
coredns-6554b8b87f-6q2k6 1/1 Running 0 22m
coredns-6554b8b87f-8sd7d 1/1 Running 0 22m
etcd-master 1/1 Running 0 22m
kube-apiserver-master 1/1 Running 0 22m
kube-controller-manager-master 1/1 Running 0 22m
kube-proxy-7j4jz 1/1 Running 0 22m
kube-proxy-95t2g 1/1 Running 0 12m
kube-proxy-pcfqn 1/1 Running 0 12m
kube-scheduler-master 1/1 Running 0 22m
六、在node1和node2中也可以使用kubectl get nodes命令
node1和node2：
[root@node1 ~]# mkdir -p $HOME/.kube
[root@node1 ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
cp: cannot stat '/etc/kubernetes/admin.conf': No such file or directory
[root@node1 ~]# cd /etc/kubernetes/
[root@node1 kubernetes]# ls
kubelet.conf manifests pki
[root@node1 kubernetes]# scp root@master:/etc/kubernetes/admin.conf .
Warning: Permanently added 'master' (ED25519) to the list of known hosts.
admin.conf 100% 5647 1.4MB/s 00:00
[root@node1 kubernetes]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@node1 kubernetes]# sudo chown $(id -u):$(id -g) $HOME/.kube/config
[root@node1 kubernetes]# export KUBECONFIG=/etc/kubernetes/admin.conf
[root@node1 kubernetes]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master Ready control-plane 29m v1.28.15
node1 Ready <none> 19m v1.28.15
node2 Ready <none> 19m v1.28.15
开源鸿蒙跨平台开发社区汇聚开发者与厂商，共建“一次开发，多端部署”的开源生态，致力于降低跨端开发门槛，推动万物智联创新。
更多推荐
25
0- 0
已为社区贡献2条内容
所有评论(0)