导航：首页 > 服务器 >

Kubernetes集群部署的方式有哪些？

发表于：2025-01-23 作者：千家信息网编辑

千家信息网最后更新 2025年01月23日，Kubernetes集群部署的方式有哪些？针对这个问题，今天小编总结这篇有关Kubernetes集群部署的文章，文章重点讲解了Kubernetes二进制的部署等部署方式。1.官方提供三种部署方式min

千家信息网最后更新 2025年01月23日Kubernetes集群部署的方式有哪些？

Kubernetes集群部署的方式有哪些？针对这个问题，今天小编总结这篇有关Kubernetes集群部署的文章，文章重点讲解了Kubernetes二进制的部署等部署方式。

1.官方提供三种部署方式

minikube

Minikube是一个工具，可以在本地快速运行一个单点的Kubernetes，仅用于尝试Kubernetes或日常开发的用户使用。
部署地址: https://kubernetes.io/docs/setup/minikube/

kubeadm

Kubeadm也是一个工具，提供kubeadm init和kubeadm join,用于快速部署Kubernetes集群。

部署地址:https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/

二进制包

推荐，从官方下载发行版的二进制包，手动部署每个组件，组成Kubernetes集群。

下载地址:https://github.com/kubernetes/kubernetes/releases

2. Kubernetes 架构图

3.自签SSL证书

组件	使用的证书
etcd	ca.pem, server.pem, server-key.pem
flannel	ca.pem，server.pem, server-key.pem
kube- apiserver	ca.pem, server.pem, server-key.pem
kubelet	ca.pem, ca-key.pem
kube-proxy	ca.pem, kube-proxy.pem, kube-proxy-key.pem
kubectl	ca.pem, admin.pem, admin-key.pem

4. Etcd数据库集群部署

二进制包下载地址
https://github.com/etcd-io/etcd/releases

k8s单节点部署思路：

第一部分

1、自签ETCD证书
2、ETCD部署
3、Node安装docker
4、Flannel部署（先写入子网到etcd）

第二部分（master）

1、自签APIServer证书
2、部署APIServer组件（token，csv）
3、部署controller-manager（指定apiserver证书）和scheduler组件

第三部分（node）

1、生成kubeconfig（bootstrap，kubeconfig和kube-proxy.kubeconfig）
2、部署kubelet组件
3、部署kube-proxy组件

第四部分（加入群集）

1、kubectl get csr && kubectl certificate approve 允许颁发证书，加入群集
2、添加一个node节点
3、查看kubectl get node 节点

k8s部署规划：

负载均衡

Nginx1:192.168.35.104/24

Nginx2:192.168.35.105/24

Master节点

master1:192.168.35.100/24

master2:192.168.35.103/24

Node节点

node1：192.168.35.101/24

node2：192.168.35.102/24

第一部分

1.1、自签ETCD证书

1.1.1、master操作：

[root@localhost ~]# mkdir k8s[root@localhost ~]# cd k8s/[root@localhost k8s]# ls    ##从宿主机拖进来etcd-cert.sh  etcd.sh[root@localhost k8s]# mkdir etcd-cert[root@localhost k8s]# mv etcd-cert.sh etcd-cert

下面是etcd.sh脚本内容

vim etcd.sh#!/bin/bash# example: ./etcd.sh etcd01 192.168.1.10 etcd02=https://192.168.1.11:2380,etcd03=https://192.168.1.12:2380ETCD_NAME=$1ETCD_IP=$2ETCD_CLUSTER=$3WORK_DIR=/opt/etcdcat <$WORK_DIR/cfg/etcd#[Member]ETCD_NAME="${ETCD_NAME}"ETCD_DATA_DIR="/var/lib/etcd/default.etcd"ETCD_LISTEN_PEER_URLS="https://${ETCD_IP}:2380"ETCD_LISTEN_CLIENT_URLS="https://${ETCD_IP}:2379"#[Clustering]ETCD_INITIAL_ADVERTISE_PEER_URLS="https://${ETCD_IP}:2380"ETCD_ADVERTISE_CLIENT_URLS="https://${ETCD_IP}:2379"ETCD_INITIAL_CLUSTER="etcd01=https://${ETCD_IP}:2380,${ETCD_CLUSTER}"ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"ETCD_INITIAL_CLUSTER_STATE="new"EOFcat </usr/lib/systemd/system/etcd.service[Unit]Description=Etcd ServerAfter=network.targetAfter=network-online.targetWants=network-online.target[Service]Type=notifyEnvironmentFile=${WORK_DIR}/cfg/etcdExecStart=${WORK_DIR}/bin/etcd \--name=\${ETCD_NAME} \--data-dir=\${ETCD_DATA_DIR} \--listen-peer-urls=\${ETCD_LISTEN_PEER_URLS} \--listen-client-urls=\${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 \--advertise-client-urls=\${ETCD_ADVERTISE_CLIENT_URLS} \--initial-advertise-peer-urls=\${ETCD_INITIAL_ADVERTISE_PEER_URLS} \--initial-cluster=\${ETCD_INITIAL_CLUSTER} \--initial-cluster-token=\${ETCD_INITIAL_CLUSTER_TOKEN} \--initial-cluster-state=new \--cert-file=${WORK_DIR}/ssl/server.pem \--key-file=${WORK_DIR}/ssl/server-key.pem \--peer-cert-file=${WORK_DIR}/ssl/server.pem \--peer-key-file=${WORK_DIR}/ssl/server-key.pem \--trusted-ca-file=${WORK_DIR}/ssl/ca.pem \--peer-trusted-ca-file=${WORK_DIR}/ssl/ca.pemRestart=on-failureLimitNOFILE=65536[Install]WantedBy=multi-user.targetEOFsystemctl daemon-reloadsystemctl enable etcdsystemctl restart etcd

下面是 etcd-cert.sh脚本内容

vim etcd-cert.shcat > ca-config.json < ca-csr.json < server-csr.json <

`1.1.2、下载官方包`

[root@localhost k8s]# vim cfssl.shcurl -L https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -o /usr/local/bin/cfsslcurl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /usr/local/bin/cfssljsoncurl -L https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 -o /usr/local/bin/cfssl-certinfochmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson /usr/local/bin/cfssl-certinfo[root@localhost k8s]# bash cfssl.sh  //下载cfssl官方包  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed100  9.8M  100  9.8M    0     0  77052      0  0:02:14  0:02:14 --:--:-- 94447  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed100 2224k  100 2224k    0     0  66701      0  0:00:34  0:00:34 --:--:-- 71949  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed100 6440k  100 6440k    0     0  74368      0  0:01:28  0:01:28 --:--:-- 93942[root@localhost k8s]# ls /usr/local/bin/cfssl  cfssl-certinfo  cfssljson##cfssl 生成证书工具、cfssljson通过传入json文件生成证书、cfssl-certinfo查看证书信息

`1.2、ETCD部署`

`1.2.1、定义证书`

[root@localhost k8s]# cd etcd-cert/cat > ca-config.json <

`1.2.2、实现证书签名`

cat > ca-csr.json <

`1.2.3、生产证书，生成ca-key.pem ca.pem`

cfssl gencert -initca ca-csr.json | cfssljson -bare ca -2020/01/15 18:15:15 [INFO] generating a new CA key and certificate from CSR2020/01/15 18:15:15 [INFO] generate received request2020/01/15 18:15:15 [INFO] received CSR2020/01/15 18:15:15 [INFO] generating key: rsa-20482020/01/15 18:15:15 [INFO] encoded CSR2020/01/15 18:15:15 [INFO] signed certificate with serial number 661808851940283859099066838380794010566731982441

`1.2.4、指定etcd三个节点之间的通信验证`

cat > server-csr.json <

`1.2.5、生成ETCD证书 server-key.pem server.pem`

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare serverwww server-csr.json | cfssljson -bare server2020/01/15 18:24:09 [INFO] generate received request2020/01/15 18:24:09 [INFO] received CSR2020/01/15 18:24:09 [INFO] generating key: rsa-20482020/01/15 18:24:09 [INFO] encoded CSR2020/01/15 18:24:09 [INFO] signed certificate with serial number 6132525683701980356436306356020343230431895064632020/01/15 18:24:09 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable forwebsites. For more information see the Baseline Requirements for the Issuance and Managementof Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);specifically, section 10.2.3 ("Information Requirements").

`1.2.6、复制软件包到centos7中`

[root@localhost etcd-cert]# cd /root/k8s/[root@localhost k8s]# ls              ##直接拉取到目录下cfssl.sh   etcd.sh                          flannel-v0.10.0-linux-amd64.tar.gzetcd-cert  etcd-v3.3.10-linux-amd64.tar.gz  kubernetes-server-linux-amd64.tar.gz

`1.2.7、解压`

[root@localhost k8s]# tar xf etcd-v3.3.10-linux-amd64.tar.gz[root@localhost k8s]# ls etcd-v3.3.10-linux-amd64Documentation  etcd  etcdctl  README-etcdctl.md  README.md  READMEv2-etcdctl.md[root@localhost k8s]# mkdir /opt/etcd/{cfg,bin,ssl} -p    ##配置文件，命令文件，证书[root@localhost k8s]# mv etcd-v3.3.10-linux-amd64/etcd etcd-v3.3.10-linux-amd64/etcdctl /opt/etcd/bin/

`1.2.8、证书拷贝`

##证书拷贝[root@localhost k8s]# cp etcd-cert/*.pem /opt/etcd/ssl/##进入卡住状态等待其他节点加入[root@localhost k8s]# bash etcd.sh etcd01 192.168.35.100 etcd02=https://192.168.35.101:2380,etcd03=https://192.168.35.102:2380

`1.2.9、此时打开另外一个会话，会发现etcd进程已经开启`

[root@localhost etcd-cert]# ps -ef | grep etcdroot      81594      1  0 12:25 ?        00:00:18 /opt/etcd/bin/etcd --name=etcd01 --data-dir=/var/lib/etcd/default.etcd --listen-peer-urls=https://192.168.35.100:2380 --listen-client-urls=https://192.168.35.100:2379,http://127.0.0.1:2379 --advertise-client-urls=https://192.168.35.100:2379 --initial-advertise-peer-urls=https://192.168.35.100:2380 --initial-cluster=etcd01=https://192.168.35.100:2380,etcd02=https://192.168.35.101:2380,etcd03=https://192.168.35.102:2380 --initial-cluster-token=etcd-cluster --initial-cluster-state=new --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --peer-cert-file=/opt/etcd/ssl/server.pem --peer-key-file=/opt/etcd/ssl/server-key.pem --trusted-ca-file=/opt/etcd/ssl/ca.pem --peer-trusted-ca-file=/opt/etcd/ssl/ca.pemroot      82044  79438  0 13:12 pts/0    00:00:00 grep --color=auto etcd

`1.2.10、在master1、node1、node2上都关闭防火墙`

systemctl stop firewalld.service setenforce 0

`1.2.11、拷贝证书去其他节点`

[root@localhost k8s]# scp -r /opt/etcd/ root@192.168.35.101:/opt/[root@localhost k8s]# scp -r /opt/etcd/ root@192.168.35.102:/opt

`1.2.12、启动脚本拷贝其他节点`

[root@localhost k8s]# scp /usr/lib/systemd/system/etcd.service root@192.168.35.101:/usr/lib/systemd/system/[root@localhost k8s]# scp /usr/lib/systemd/system/etcd.service root@192.168.35.102:/usr/lib/systemd/system/

`1.2.13、在node01节点修改`

[root@localhost ~]# vim /opt/etcd/cfg/etcd#[Member]ETCD_NAME="etcd02"ETCD_DATA_DIR="/var/lib/etcd/default.etcd"ETCD_LISTEN_PEER_URLS="https://192.168.35.101:2380"ETCD_LISTEN_CLIENT_URLS="https://192.168.35.101:2379"#[Clustering]ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.35.101:2380"ETCD_ADVERTISE_CLIENT_URLS="https://192.168.35.101:2379"ETCD_INITIAL_CLUSTER="etcd01=https://192.168.35.128:2380,etcd02=https://192.168.35.195:2380,etcd03=https://192.168.35.138:2380"ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"ETCD_INITIAL_CLUSTER_STATE="new"

`1.2.14、在node02节点修改`

[root@localhost ~]# vim /opt/etcd/cfg/etcd#[Member]ETCD_NAME="etcd03"ETCD_DATA_DIR="/var/lib/etcd/default.etcd"ETCD_LISTEN_PEER_URLS="https://192.168.35.102:2380"ETCD_LISTEN_CLIENT_URLS="https://192.168.35.102:2379"#[Clustering]ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.35.102:2380"ETCD_ADVERTISE_CLIENT_URLS="https://192.168.35.102:2379"ETCD_INITIAL_CLUSTER="etcd01=https://192.168.35.128:2380,etcd02=https://192.168.35.195:2380,etcd03=https://192.168.35.138:2380"ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"ETCD_INITIAL_CLUSTER_STATE="new"

`1.2.15、先在master上开启服务`

[root@localhost system]# cd /root/k8s/[root@localhost k8s]# bash etcd.sh etcd01 192.168.35.100 etcd02=https://192.168.35.101:2380,etcd03=https://192.168.35.102:2380

`1.2.16、在node1和node2上开启服务`

[root@localhost ~]# systemctl start etcd[root@localhost ~]# systemctl status etcd

`1.2.17、再去master查看，会发现同步成功`

[root@localhost k8s]# bash etcd.sh etcd01 192.168.35.100 etcd02=https://192.168.35.101:2380,etcd03=https://192.168.35.102:2380

`1.2.18、检查群集状态`

[root@localhost k8s]# cd etcd-cert/[root@localhost etcd-cert]# /opt/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://192.168.35.100:2379,https://192.168.35.101:2379,https://192.168.35.102:2379" cluster-healthmember 12a96220ac829a49 is healthy: got healthy result from https://192.168.35.195:2379member 76797989afd0ecba is healthy: got healthy result from https://192.168.35.128:2379member ff469df2baaba1da is healthy: got healthy result from https://192.168.35.138:2379cluster is healthy

`1.3、Node安装docker`

`部署19版docker`

`1.3.1、安装依赖包`

yum install -y yum-utils device-mapper-persistent-data lvm2

`1.3.2、设置阿里云镜像源`

yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

`1.3.3、安装Docker-CE`

yum install -y docker-ce

`1.3.4、关闭防火墙`

systemctl stop firewalld.service setenforce 0

`1.3.5、开启服务并设为开机自启`

systemctl start docker.service systemctl enable docker.service

`1.3.6、镜像加速`

tee /etc/docker/daemon.json <<-'EOF'{  "registry-mirrors": ["https://cz1numin.mirror.aliyuncs.com"]}EOF##重载并重启服务systemctl daemon-reloadsystemctl restart docker

`1.3.7、网络优化`

vim /etc/sysctl.confnet.ipv4.ip_forward=1##重载配置、重启网卡、重启服务sysctl -pservice network restartsystemctl restart docker

`1.4、Flannel部署`

`1.4.1、写入分配的子网段到ETCD中，供flannel使用`

[root@localhost etcd-cert]# /opt/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://192.168.35.100:2379,https://192.168.35.101:2379,https://192.168.35.102:2379" set /coreos.com/network/config '{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}'{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}

`1.4.2、查看写入的信息`

[root@localhost etcd-cert]# /opt/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://192.168.35.100:2379,https://192.168.35.101:2379,https://192.168.35.102:2379" get /coreos.com/network/config{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}

`1.4.3、拷贝到所有node节点（只需要部署在node节点即可）`

[root@localhost k8s]# scp flannel-v0.10.0-linux-amd64.tar.gz root@192.168.35.101:/rootroot@192.168.35.101's password: flannel-v0.10.0-linux-amd64.tar.gz                 100% 9479KB  66.6MB/s   00:00    [root@localhost k8s]# scp flannel-v0.10.0-linux-amd64.tar.gz root@192.168.35.102:/rootroot@192.168.35.102's password: flannel-v0.10.0-linux-amd64.tar.gz                 100% 9479KB  75.7MB/s   00:00

`1.4.4、所有node节点操作解压`

[root@localhost ~]# tar zxvf flannel-v0.10.0-linux-amd64.tar.gz flanneldmk-docker-opts.shREADME.md

`1.4.5、node1操作`

`（1）k8s工作目录`

[root@localhost ~]# ls /opt/containerd  etcd  rh[root@localhost ~]# mkdir /opt/kubernetes/{cfg,bin,ssl} -p[root@localhost ~]# ls /opt/containerd  etcd  kubernetes  rh[root@localhost ~]# mv mk-docker-opts.sh flanneld /opt/kubernetes/bin/[root@localhost ~]# ls /opt/kubernetes/bin/flanneld  mk-docker-opts.sh[root@localhost ~]# vim flannel.sh#!/bin/bash ETCD_ENDPOINTS=${1:-"http://127.0.0.1:2379"} cat </opt/kubernetes/cfg/flanneld FLANNEL_OPTIONS="--etcd-endpoints=${ETCD_ENDPOINTS} \-etcd-cafile=/opt/etcd/ssl/ca.pem \-etcd-certfile=/opt/etcd/ssl/server.pem \-etcd-keyfile=/opt/etcd/ssl/server-key.pem" EOF cat </usr/lib/systemd/system/flanneld.service[Unit]Description=Flanneld overlay address etcd agentAfter=network-online.target network.targetBefore=docker.service [Service]Type=notifyEnvironmentFile=/opt/kubernetes/cfg/flanneldExecStart=/opt/kubernetes/bin/flanneld --ip-masq \$FLANNEL_OPTIONSExecStartPost=/opt/kubernetes/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.envRestart=on-failure [Install]WantedBy=multi-user.target EOF systemctl daemon-reloadsystemctl enable flanneldsystemctl restart flanneld

`（2）开启flannel网络功能`

[root@localhost ~]# bash flannel.sh https://192.168.35.100:2379,https://192.168.35.101:2379,https://192.168.35.102:2379Created symlink from /etc/systemd/system/multi-user.target.wants/flanneld.service to /usr/lib/systemd/system/flanneld.service.

`（3）配置docker连接flannel`

[root@localhost ~]# vim /usr/lib/systemd/system/docker.service[Service]Type=notify# the default is not to use systemd for cgroups because the delegate issues still# exists and systemd currently does not support the cgroup feature set required# for containers run by dockerEnvironmentFile=/run/flannel/subnet.envExecStart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS -H fd:// --containerd=/run/containerd/containerd.sockExecReload=/bin/kill -s HUP $MAINPIDTimeoutSec=0RestartSec=2Restart=always

`（4）说明：bip指定启动时的子网`

[root@localhost ~]# cat /run/flannel/subnet.envDOCKER_OPT_BIP="--bip=172.17.23.1/24"DOCKER_OPT_IPMASQ="--ip-masq=false"DOCKER_OPT_MTU="--mtu=1450"DOCKER_NETWORK_OPTIONS=" --bip=172.17.23.1/24 --ip-masq=false --mtu=1450"

`（5）重启docker服务`

[root@localhost ~]# systemctl daemon-reload[root@localhost ~]# systemctl restart docker

`（6）查看flannel网络`

[root@localhost ~]# ifconfigdocker0: flags=4099  mtu 1500        inet 172.17.23.1  netmask 255.255.255.0  broadcast 172.17.23.255        ether 02:42:7d:9a:31:89  txqueuelen 0  (Ethernet)        RX packets 0  bytes 0 (0.0 B)        RX errors 0  dropped 0  overruns 0  frame 0        TX packets 0  bytes 0 (0.0 B)        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0ens33: flags=4163  mtu 1500        inet 192.168.35.101  netmask 255.255.255.0  broadcast 192.168.35.255        inet6 fe80::d4e2:ef9e:6820:145a  prefixlen 64  scopeid 0x20        inet6 fe80::2a3:b621:ca01:463e  prefixlen 64  scopeid 0x20        ether 00:0c:29:01:82:fd  txqueuelen 1000  (Ethernet)        RX packets 1148980  bytes 1135571843 (1.0 GiB)        RX errors 0  dropped 0  overruns 0  frame 0        TX packets 594986  bytes 64240523 (61.2 MiB)        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0flannel.1: flags=4163  mtu 1450        inet 172.17.23.0  netmask 255.255.255.255  broadcast 0.0.0.0        inet6 fe80::58e3:97ff:fe97:28f2  prefixlen 64  scopeid 0x20        ether 5a:e3:97:97:28:f2  txqueuelen 0  (Ethernet)        RX packets 0  bytes 0 (0.0 B)        RX errors 0  dropped 0  overruns 0  frame 0        TX packets 0  bytes 0 (0.0 B)        TX errors 0  dropped 141 overruns 0  carrier 0  collisions 0

`（7）创建容器`

[root@localhost ~]# docker run -it centos:7 /bin/bashUnable to find image 'centos:7' locally7: Pulling from library/centosab5ef0e58194: Pull complete Digest: sha256:4a701376d03f6b39b8c2a8f4a8e499441b0d567f9ab9d58e4991de4472fb813cStatus: Downloaded newer image for centos:7[root@809ba7f7fdbe /]#  yum install net-tools -y               #安装工具后才能使用ifconfig命令[root@809ba7f7fdbe /]# ifconfig eth0: flags=4163  mtu 1450        inet 172.17.23.2  netmask 255.255.255.0  broadcast 172.17.23.255        ether 02:42:ac:11:17:02  txqueuelen 0  (Ethernet)        RX packets 17384  bytes 13962588 (13.3 MiB)        RX errors 0  dropped 0  overruns 0  frame 0        TX packets 5915  bytes 323222 (315.6 KiB)        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0lo: flags=73  mtu 65536        inet 127.0.0.1  netmask 255.0.0.0        loop  txqueuelen 1  (Local Loopback)        RX packets 0  bytes 0 (0.0 B)        RX errors 0  dropped 0  overruns 0  frame 0        TX packets 0  bytes 0 (0.0 B)        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

`1.4.6、node2操作（和上述node1操作一样）`

注释：这里过程就省略了，只看下重要的结果

`（1）bip指定启动时的子网`

[root@localhost ~]# vim /usr/lib/systemd/system/docker.service[root@localhost ~]# cat /run/flannel/subnet.envDOCKER_OPT_BIP="--bip=172.17.56.1/24"DOCKER_OPT_IPMASQ="--ip-masq=false"DOCKER_OPT_MTU="--mtu=1450"DOCKER_NETWORK_OPTIONS=" --bip=172.17.56.1/24 --ip-masq=false --mtu=1450"

`（2）查看flannel网络`

[root@localhost ~]# ifconfig docker0: flags=4099  mtu 1500        inet 172.17.56.1  netmask 255.255.255.0  broadcast 172.17.56.255        ether 02:42:4b:66:50:5b  txqueuelen 0  (Ethernet)        RX packets 0  bytes 0 (0.0 B)        RX errors 0  dropped 0  overruns 0  frame 0        TX packets 0  bytes 0 (0.0 B)        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0flannel.1: flags=4163  mtu 1450        inet 172.17.56.0  netmask 255.255.255.255  broadcast 0.0.0.0        inet6 fe80::50f6:daff:fe4e:af02  prefixlen 64  scopeid 0x20        ether 52:f6:da:4e:af:02  txqueuelen 0  (Ethernet)        RX packets 0  bytes 0 (0.0 B)        RX errors 0  dropped 0  overruns 0  frame 0        TX packets 0  bytes 0 (0.0 B)        TX errors 0  dropped 106 overruns 0  carrier 0  collisions 0

`（3）创建容器`

[root@localhost ~]# docker run -it centos:7 /bin/bashUnable to find image 'centos:7' locally7: Pulling from library/centosab5ef0e58194: Pull complete Digest: sha256:4a701376d03f6b39b8c2a8f4a8e499441b0d567f9ab9d58e4991de4472fb813cStatus: Downloaded newer image for centos:7[root@5d34e9d6766e /]# yum install net-tools -y[root@5d34e9d6766e /]# ifconfig eth0: flags=4163  mtu 1450        inet 172.17.56.2  netmask 255.255.255.0  broadcast 172.17.56.255        ether 02:42:ac:11:38:02  txqueuelen 0  (Ethernet)        RX packets 17199  bytes 13954199 (13.3 MiB)        RX errors 0  dropped 0  overruns 0  frame 0        TX packets 6723  bytes 367133 (358.5 KiB)        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0lo: flags=73  mtu 65536        inet 127.0.0.1  netmask 255.0.0.0        loop  txqueuelen 1  (Local Loopback)        RX packets 0  bytes 0 (0.0 B)        RX errors 0  dropped 0  overruns 0  frame 0        TX packets 0  bytes 0 (0.0 B)        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

`1.4.7、测试ping通对方docker0网卡证明flannel起到路由作用`

[root@localhost ~]# ping 172.17.56.1PING 172.17.56.1 (172.17.56.1) 56(84) bytes of data.64 bytes from 172.17.56.1: icmp_seq=1 ttl=64 time=0.307 ms64 bytes from 172.17.56.1: icmp_seq=2 ttl=64 time=0.385 ms64 bytes from 172.17.56.1: icmp_seq=3 ttl=64 time=0.302 ms

`1.4.8、再次测试ping通两个node中的centos:7容器`

[root@809ba7f7fdbe /]# ping 172.17.56.2PING 172.17.56.2 (172.17.56.2) 56(84) bytes of data.64 bytes from 172.17.56.2: icmp_seq=1 ttl=62 time=0.487 ms64 bytes from 172.17.56.2: icmp_seq=2 ttl=62 time=0.273 ms64 bytes from 172.17.56.2: icmp_seq=3 ttl=62 time=0.234 ms

`第二部分（master）`

`2.1、自签APIServer证书`

`2.1.1、在master上操作，api-server生成证书`

[root@localhost k8s]# lsetcd-cert                 etcd-v3.3.10-linux-amd64.tar.gz       master.zipetcd.sh                   flannel-v0.10.0-linux-amd64.tar.gzetcd-v3.3.10-linux-amd64  kubernetes-server-linux-amd64.tar.gz[root@localhost k8s]# unzip master.zipArchive:  master.zip  inflating: apiserver.sh              inflating: controller-manager.sh     inflating: scheduler.sh            [root@localhost k8s]# mkdir /opt/kubernetes/{cfg,bin,ssl} -p[root@localhost k8s]# mkdir k8s-cert                     #apiserver自签证书目录[root@localhost k8s]# cd k8s-cert/[root@localhost k8s-cert]# lsk8s-cert.sh//k8s-cert.sh脚本cat > ca-config.json < ca-csr.json < server-csr.json < admin-csr.json < kube-proxy-csr.json <

`2.1.2、生成k8s证书`

[root@localhost k8s-cert]#  bash k8s-cert.sh2020/02/05 15:04:48 [INFO] generating a new CA key and certificate from CSR2020/02/05 15:04:48 [INFO] generate received request2020/02/05 15:04:48 [INFO] received CSR2020/02/05 15:04:48 [INFO] generating key: rsa-20482020/02/05 15:04:49 [INFO] encoded CSR2020/02/05 15:04:49 [INFO] signed certificate with serial number 2130379016242708957811784067334874586349136774572020/02/05 15:04:49 [INFO] generate received request2020/02/05 15:04:49 [INFO] received CSR2020/02/05 15:04:49 [INFO] generating key: rsa-20482020/02/05 15:04:49 [INFO] encoded CSR2020/02/05 15:04:49 [INFO] signed certificate with serial number 4599001666352018122578616375172914058088251019802020/02/05 15:04:49 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable forwebsites. For more information see the Baseline Requirements for the Issuance and Managementof Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);specifically, section 10.2.3 ("Information Requirements").2020/02/05 15:04:49 [INFO] generate received request2020/02/05 15:04:49 [INFO] received CSR2020/02/05 15:04:49 [INFO] generating key: rsa-20482020/02/05 15:04:49 [INFO] encoded CSR2020/02/05 15:04:49 [INFO] signed certificate with serial number 2295567237546104849186667180081799823807210101922020/02/05 15:04:49 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable forwebsites. For more information see the Baseline Requirements for the Issuance and Managementof Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);specifically, section 10.2.3 ("Information Requirements").2020/02/05 15:04:49 [INFO] generate received request2020/02/05 15:04:49 [INFO] received CSR2020/02/05 15:04:49 [INFO] generating key: rsa-20482020/02/05 15:04:49 [INFO] encoded CSR2020/02/05 15:04:49 [INFO] signed certificate with serial number 4350307101528389300418153297503013234680658767722020/02/05 15:04:49 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable forwebsites. For more information see the Baseline Requirements for the Issuance and Managementof Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);specifically, section 10.2.3 ("Information Requirements").[root@localhost k8s-cert]# ls *pemadmin-key.pem  ca-key.pem  kube-proxy-key.pem  server-key.pemadmin.pem      ca.pem      kube-proxy.pem      server.pem[root@localhost k8s-cert]# cp ca*pem server*pem /opt/kubernetes/ssl/

`2.1.3、解压kubernetes压缩包`

[root@localhost k8s-cert]# cd ..[root@localhost k8s]# lsapiserver.sh                     flannel-v0.10.0-linux-amd64.tar.gzcontroller-manager.sh            k8s-certetcd-cert                        kubernetes-server-linux-amd64.tar.gzetcd.sh                          master.zipetcd-v3.3.10-linux-amd64         scheduler.shetcd-v3.3.10-linux-amd64.tar.gz[root@localhost k8s]# tar zxvf kubernetes-server-linux-amd64.tar.gz

`2.1.4、复制关键命令文件`

[root@localhost k8s]# cd /root/k8s/kubernetes/server/bin/[root@localhost bin]# lsapiextensions-apiserver              kube-controller-manager.tarcloud-controller-manager             kubectlcloud-controller-manager.docker_tag  kubeletcloud-controller-manager.tar         kube-proxyhyperkube                            kube-proxy.docker_tagkubeadm                              kube-proxy.tarkube-apiserver                       kube-schedulerkube-apiserver.docker_tag            kube-scheduler.docker_tagkube-apiserver.tar                   kube-scheduler.tarkube-controller-manager              mounterkube-controller-manager.docker_tag[root@localhost bin]# cp kube-apiserver kubectl kube-controller-manager kube-scheduler /opt/kubernetes/bin/

`2.2、部署APIServer组件（token，csv）`

`2.2.1、随机生成序列号`

[root@localhost k8s]# head -c 16 /dev/urandom | od -An -t x | tr -d ' 'b639a2717c79839fd7fab7bac97dca32               #随机生成的序列号[root@localhost k8s]# vim /opt/kubernetes/cfg/token.csvb639a2717c79839fd7fab7bac97dca32,kubelet-bootstrap,10001,"system:kubelet-bootstrap"                     ##序列号                                      用户名                id                         角色

`2.2.2、二进制文件，token，证书都准备好，开启apiserver`

[root@localhost k8s]#  bash apiserver.sh 192.168.35.100 https://192.168.35.100:2379,https://192.168.35.101:2379,https://192.168.35.102:2379Created symlink from /etc/systemd/system/multi-user.target.wants/kube-apiserver.service to /usr/lib/systemd/system/kube-apiserver.service.

`2.2.3、检查进程是否启动成功`

[root@localhost k8s]# ps aux | grep kuberoot      83786 92.7 14.4 358828 269812 ?       Ssl  15:56   0:06 /opt/kubernetes/bin/kube-apiserver --logtostderr=true --v=4 --etcd-servers=https://192.168.35.100:2379,https://192.168.35.101:2379,https://192.168.35.102:2379 --bind-address=192.168.35.100 --secure-port=6443 --advertise-address=192.168.35.100 --allow-privileged=true --service-cluster-ip-range=10.0.0.0/24 --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction --authorization-mode=RBAC,Node --kubelet-https=true --enable-bootstrap-token-auth --token-auth-file=/opt/kubernetes/cfg/token.csv --service-node-port-range=30000-50000 --tls-cert-file=/opt/kubernetes/ssl/server.pem --tls-private-key-file=/opt/kubernetes/ssl/server-key.pem --client-ca-file=/opt/kubernetes/ssl/ca.pem --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem --etcd-cafile=/opt/etcd/ssl/ca.pem --etcd-certfile=/opt/etcd/ssl/server.pem --etcd-keyfile=/opt/etcd/ssl/server-key.pemroot      83794  0.0  0.0 112676   980 pts/0    S+   15:56   0:00 grep --color=auto kube

`2.2.4、查看配置文件`

[root@localhost k8s]# cat /opt/kubernetes/cfg/kube-apiserverKUBE_APISERVER_OPTS="--logtostderr=true \--v=4 \--etcd-servers=https://192.168.35.100:2379,https://192.168.35.101:2379,https://192.168.35.102:2379 \--bind-address=192.168.35.100 \--secure-port=6443 \--advertise-address=192.168.35.100 \--allow-privileged=true \--service-cluster-ip-range=10.0.0.0/24 \--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \--authorization-mode=RBAC,Node \--kubelet-https=true \--enable-bootstrap-token-auth \--token-auth-file=/opt/kubernetes/cfg/token.csv \--service-node-port-range=30000-50000 \--tls-cert-file=/opt/kubernetes/ssl/server.pem  \--tls-private-key-file=/opt/kubernetes/ssl/server-key.pem \--client-ca-file=/opt/kubernetes/ssl/ca.pem \--service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \--etcd-cafile=/opt/etcd/ssl/ca.pem \--etcd-certfile=/opt/etcd/ssl/server.pem \--etcd-keyfile=/opt/etcd/ssl/server-key.pem"

`2.2.5、监听的https端口`

[root@localhost k8s]# netstat -ntap | grep 6443tcp        0      0 192.168.35.100:6443     0.0.0.0:*               LISTEN      83786/kube-apiserve tcp        0      0 192.168.35.100:44312    192.168.35.100:6443     ESTABLISHED 83786/kube-apiserve tcp        0      0 192.168.35.100:6443     192.168.35.100:44312    ESTABLISHED 83786/kube-apiserve [root@localhost k8s]# netstat -ntap | grep 8080tcp        0      0 127.0.0.1:8080          0.0.0.0:*               LISTEN      83786/kube-apiserve

`2.3、部署controller-manager（指定apiserver证书）和scheduler组件`

`2.3.1、启动scheduler服务`

[root@localhost k8s]# ./scheduler.sh 127.0.0.1Created symlink from /etc/systemd/system/multi-user.target.wants/kube-scheduler.service to /usr/lib/systemd/system/kube-scheduler.service.[root@localhost k8s]# ps aux | grep kupostfix   83202  0.0  0.2  91732  4008 ?        S    15:10   0:00 pickup -l -t unix -uroot      83786  4.3 18.3 427420 342812 ?       Ssl  15:56   0:12 /opt/kubernetes/bin/kube-apiserver --logtostderr=true --v=4 --etcd-servers=https://192.168.35.100:2379,https://192.168.35.101:2379,https://192.168.35.102:2379 --bind-address=192.168.35.100 --secure-port=6443 --advertise-address=192.168.35.100 --allow-privileged=true --service-cluster-ip-range=10.0.0.0/24 --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction --authorization-mode=RBAC,Node --kbelet-https=true --enable-bootstrap-token-auth --token-auth-file=/opt/kubernetes/cfg/token.csv --service-node-port-range=30000-50000 --tls-cert-file=/opt/kubernetes/ssl/server.pem --tls-private-key-file=/opt/kubernetes/ssl/server-key.pem --client-ca-file=/opt/kubernetes/ssl/ca.pem --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem --etcd-cafile=/opt/etcd/ssl/ca.pem --etcd-certfile=/opt/etcd/ssl/server.pem --etcd-keyfile=/opt/etcd/ssl/server-key.pemroot      83904  0.8  1.0  45360 20356 ?        Ssl  16:01   0:00 /opt/kubernetes/bin/kube-scheduler --logtostderr=true --v=4 --master=127.0.0.1:8080 --leader-electroot      83912  0.0  0.0 112676   984 pts/0    R+   16:01   0:00 grep --color=auto ku

`2.3.2、启动controller-manager`

[root@localhost k8s]#  chmod +x controller-manager.sh[root@localhost k8s]# ./controller-manager.sh 127.0.0.1Created symlink from /etc/systemd/system/multi-user.target.wants/kube-controller-manager.service to /usr/lib/systemd/system/kube-controller-manager.service.

`2.3.3、查看master 节点状态`

[root@localhost k8s]#  /opt/kubernetes/bin/kubectl get csNAME                 STATUS    MESSAGE             ERRORscheduler            Healthy   ok                  controller-manager   Healthy   ok                  etcd-2               Healthy   {"health":"true"}   etcd-1               Healthy   {"health":"true"}   etcd-0               Healthy   {"health":"true"}

`第三部分（node）`

`3.1、生成kubeconfig（bootstrap，kubeconfig和kube-proxy.kubeconfig）`

`master上操作`

`3.1.1、把 kubelet、kube-proxy拷贝到node节点上去`

[root@localhost k8s]# cd kubernetes/server/bin/[root@localhost bin]# lsapiextensions-apiserver              kube-controller-manager.tarcloud-controller-manager             kubectlcloud-controller-manager.docker_tag  kubeletcloud-controller-manager.tar         kube-proxyhyperkube                            kube-proxy.docker_tagkubeadm                              kube-proxy.tarkube-apiserver                       kube-schedulerkube-apiserver.docker_tag            kube-scheduler.docker_tagkube-apiserver.tar                   kube-scheduler.tarkube-controller-manager              mounterkube-controller-manager.docker_tag[root@localhost bin]# scp kubelet kube-proxy root@192.168.35.101:/opt/kubernetes/bin/ root@192.168.35.101's password: kubelet                                            100%  168MB  87.4MB/s   00:01    kube-proxy                                         100%   48MB  90.9MB/s   00:00    [root@localhost bin]# scp kubelet kube-proxy root@192.168.35.102:/opt/kubernetes/bin/ root@192.168.35.102's password: kubelet                                            100%  168MB  93.4MB/s   00:01    kube-proxy                                         100%   48MB 100.5MB/s   00:00

`nod01节点操作`

`3.1.2、复制node.zip到/root目录下再解压`

[root@localhost ~]# lsanaconda-ks.cfg                     initial-setup-ks.cfg  公共  图片  音乐flannel.sh                          node.zip              模板  文档  桌面flannel-v0.10.0-linux-amd64.tar.gz  README.md             视频  下载[root@localhost ~]# unzip node.zip Archive:  node.zip  inflating: proxy.sh                  inflating: kubelet.sh

`master操作`

`3.1.3、创建工作目录`

[root@localhost k8s]#  mkdir kubeconfig[root@localhost k8s]# cd kubeconfig/[root@localhost kubeconfig]# lskubeconfig.sh               kubeconfig.sh脚本如下 ：   # 创建 TLS Bootstrapping Token#BOOTSTRAP_TOKEN=$(head -c 16 /dev/urandom | od -An -t x | tr -d ' ')BOOTSTRAP_TOKEN=0fb61c46f8991b718eb38d27b605b008cat > token.csv <

`3.1.2、获取token信息`

[root@localhost kubeconfig]# cat /opt/kubernetes/cfg/token.csvb639a2717c79839fd7fab7bac97dca32,kubelet-bootstrap,10001,"system:kubelet-bootstr

`3.1.3、配置文件修改为tokenID`

[root@localhost kubeconfig]# vim kubeconfig.sh# 设置客户端认证参数kubectl config set-credentials kubelet-bootstrap \  --token=b639a2717c79839fd7fab7bac97dca32 \  --kubeconfig=bootstrap.kubeconfig

`3.1.4、设置环境变量（可以写入到/etc/profile中）`

[root@localhost kubeconfig]# vim /etc/profile//在末行插入export PATH=$PATH:/opt/kubernetes/bin/[root@localhost kubeconfig]# source /etc/profile                #声明[root@localhost kubeconfig]# kubectl get cs              #查看状态NAME                 STATUS    MESSAGE             ERRORscheduler            Healthy   ok                  controller-manager   Healthy   ok                  etcd-1               Healthy   {"health":"true"}   etcd-2               Healthy   {"health":"true"}   etcd-0               Healthy   {"health":"true"}

`3.1.5、重命名并生成配置文件`

[root@localhost kubeconfig]# mv kubeconfig.sh kubeconfig[root@localhost kubeconfig]# lskubeconfig[root@localhost kubeconfig]# vim kubeconfig [root@localhost kubeconfig]# bash kubeconfig 192.168.35.100 /root/k8s/k8s-cert/Cluster "kubernetes" set.User "kubelet-bootstrap" set.Context "default" created.Switched to context "default".Cluster "kubernetes" set.User "kube-proxy" set.Context "default" created.Switched to context "default".[root@localhost kubeconfig]# lsbootstrap.kubeconfig  kubeconfig  kube-proxy.kubeconfig

`3.1.6、拷贝配置文件到node节点`

[root@localhost kubeconfig]# scp bootstrap.kubeconfig kube-proxy.kubeconfig root@192.168.35.101:/opt/kubernetes/cfg/root@192.168.35.101's password: bootstrap.kubeconfig                               100% 2168     2.7MB/s   00:00    kube-proxy.kubeconfig                              100% 6270     5.1MB/s   00:00    [root@localhost kubeconfig]# scp bootstrap.kubeconfig kube-proxy.kubeconfig root@192.168.35.102:/opt/kubernetes/cfg/root@192.168.35.102's password: bootstrap.kubeconfig                               100% 2168     1.8MB/s   00:00    kube-proxy.kubeconfig                              100% 6270     5.9MB/s   00:00

`3.1.7、创建bootstrap角色赋予权限用于连接apiserver请求签名（关键）`

[root@localhost kubeconfig]# kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --user=kubelet-bootstrapclusterrolebinding.rbac.authorization.k8s.io/kubelet-bootstrap created

`3.2、部署kubelet组件`

`在node01节点上操作`

`3.2.1、执行脚本`

[root@localhost ~]# bash kubelet.sh 192.168.35.101Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.

`3.2.2、检查kubelet服务启动`

[root@localhost ~]# ps aux | grep kuberoot      85079  0.0  0.8 300512 16424 ?        Ssl  13:49   0:02 /opt/kubernetes/bin/flanneld --ip-masq --etcd-endpoints=https://192.168.35.100:2379,https://192.168.35.101:2379,https://192.168.35.102:2379 -etcd-cafile=/opt/etcd/ssl/ca.pem -etcd-certfile=/opt/etcd/ssl/server.pem -etcd-keyfile=/opt/etcd/ssl/server-key.pemroot     109665  0.6  2.3 370688 42948 ?        Ssl  18:46   0:00 /opt/kubernetes/bin/kubelet --logtostderr=true --v=4 --hostname-override=192.168.35.101 --kubeconfig=/opt/kubernetes/cfg/kubelet.kubeconfig --bootstrap-kubeconfig=/opt/kubernetes/cfg/bootstrap.kubeconfig --config=/opt/kubernetes/cfg/kubelet.config --cert-dir=/opt/kubernetes/ssl --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0root     109762  0.0  0.0 112676   980 pts/0    S+   18:47   0:00 grep --color=auto kube

`3.2.3、查看服务器的状况`

[root@localhost ~]# systemctl status kubelet.service ● kubelet.service - Kubernetes Kubelet   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)   Active: active (running) since 三 2020-02-05 18:46:35 CST; 3min 1s ago Main PID: 109665 (kubelet)   Memory: 61.6M   CGroup: /system.slice/kubelet.service           └─109665 /opt/kubernetes/bin/kubelet --logtostderr=true --v=4 --hostnam...2月 05 18:46:35 localhost.localdomain kubelet[109665]: I0205 18:46:35.374771  109...2月 05 18:46:35 localhost.localdomain kubelet[109665]: I0205 18:46:35.601115  109...2月 05 18:46:35 localhost.localdomain kubelet[109665]: I0205 18:46:35.619113  109...2月 05 18:46:35 localhost.localdomain kubelet[109665]: I0205 18:46:35.619153  109...2月 05 18:46:35 localhost.localdomain kubelet[109665]: I0205 18:46:35.619192  109...2月 05 18:46:35 localhost.localdomain kubelet[109665]: I0205 18:46:35.619229  109...2月 05 18:46:35 localhost.localdomain kubelet[109665]: I0205 18:46:35.619294  109...2月 05 18:46:35 localhost.localdomain kubelet[109665]: I0205 18:46:35.619302  109...2月 05 18:46:35 localhost.localdomain kubelet[109665]: I0205 18:46:35.620575  109...2月 05 18:46:35 localhost.localdomain kubelet[109665]: I0205 18:46:35.622388  109...Hint: Some lines were ellipsized, use -l to show in full.

`master上操作`

`3.2.4、检查到node1节点的请求`

[root@localhost kubeconfig]# kubectl get csrNAME                                                   AGE    REQUESTOR           CONDITIONnode-csr-HF551kae9hkzNGwirEbd2uAiStXnawpEI-y7nLP4sTU   5m9s   kubelet-bootstrap   PendingPending（等待集群给该节点颁发证书）

`3.2.5、同意请求并颁发证书`

[root@localhost kubeconfig]# kubectl certificate approve node-csr-HF551kae9hkzNGwirEbd2uAiStXnawpEI-y7nLP4sTUcertificatesigningrequest.certificates.k8s.io/node-csr-HF551kae9hkzNGwirEbd2uAiStXnawpEI-y7nLP4sTU approved

`3.2.6、继续查看证书状态`

[root@localhost kubeconfig]# kubectl get csrNAME                                                   AGE     REQUESTOR           CONDITIONnode-csr-HF551kae9hkzNGwirEbd2uAiStXnawpEI-y7nLP4sTU   9m18s   kubelet-bootstrap   Approved,IssuedApproved,Issued（已经被允许加入群集）

`3.2.7、查看群集节点，成功加入node01节点`

[root@localhost kubeconfig]# kubectl get nodeNAME             STATUS   ROLES    AGE     VERSION192.168.35.101   Ready       2m18s   v1.12.3

`3.3、部署kube-proxy组件`

`在node1节点操作`

`3.3.1、启动proxy服务`

[root@localhost ~]# bash proxy.sh 192.168.35.101Created symlink from /etc/systemd/system/multi-user.target.wants/kube-proxy.service to /usr/lib/systemd/system/kube-proxy.service.[root@localhost ~]# systemctl status kube-proxy.service ● kube-proxy.service - Kubernetes Proxy   Loaded: loaded (/usr/lib/systemd/system/kube-proxy.service; enabled; vendor preset: disabled)   Active: active (running) since 三 2020-02-05 18:59:58 CST; 33s ago Main PID: 111731 (kube-proxy)   Memory: 8.9M   CGroup: /system.slice/kube-proxy.service           ‣ 111731 /opt/kubernetes/bin/kube-proxy --logtostderr=true --v=4 --host...2月 05 19:00:28 localhost.localdomain kube-proxy[111731]: I0205 19:00:28.692738  ...2月 05 19:00:28 localhost.localdomain kube-proxy[111731]: I0205 19:00:28.695213  ...2月 05 19:00:28 localhost.localdomain kube-proxy[111731]: I0205 19:00:28.713256  ...2月 05 19:00:28 localhost.localdomain kube-proxy[111731]: I0205 19:00:28.724431  ...2月 05 19:00:28 localhost.localdomain kube-proxy[111731]: I0205 19:00:28.727449  ...2月 05 19:00:28 localhost.localdomain kube-proxy[111731]: I0205 19:00:28.727463  ...2月 05 19:00:29 localhost.localdomain kube-proxy[111731]: I0205 19:00:29.754834  ...2月 05 19:00:29 localhost.localdomain kube-proxy[111731]: I0205 19:00:29.799929  ...2月 05 19:00:31 localhost.localdomain kube-proxy[111731]: I0205 19:00:31.754579  ...2月 05 19:00:31 localhost.localdomain kube-proxy[111731]: I0205 19:00:31.797581  ...Hint: Some lines were ellipsized, use -l to show in full.

`3.3.2、把现成的/opt/kubernetes目录复制到其他节点进行修改即可`

[root@localhost ~]# scp -r /opt/kubernetes/ root@192.168.35.102:/opt/The authenticity of host '192.168.35.102 (192.168.35.102)' can't be established.ECDSA key fingerprint is SHA256:JsLSnAul/dncM/HPvpJWWB09dHLzpIfArHv1fWjQyA8.ECDSA key fingerprint is MD5:d1:b7:d7:74:c6:4a:2a:7b:fc:33:8c:9c:3a:f2:6e:8a.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added '192.168.35.102' (ECDSA) to the list of known hosts.root@192.168.35.102's password: flanneld                                           100%  238   100.8KB/s   00:00    bootstrap.kubeconfig                               100% 2168     5.8MB/s   00:00    kube-proxy.kubeconfig                              100% 6270    11.5MB/s   00:00    kubelet                                            100%  378   885.4KB/s   00:00    kubelet.config                                     100%  268   812.9KB/s   00:00    kubelet.kubeconfig                                 100% 2297     5.7MB/s   00:00    kube-proxy                                         100%  190   546.5KB/s   00:00    mk-docker-opts.sh                                  100% 2139     3.5MB/s   00:00    scp: /opt//kubernetes/bin/flanneld: Text file busykubelet                                            100%  168MB 116.0MB/s   00:01    kube-proxy                                         100%   48MB 122.2MB/s   00:00    kubelet.crt                                        100% 2193   729.3KB/s   00:00    kubelet.key                                        100% 1675     3.5MB/s   00:00    kubelet-client-2020-02-05-18-55-07.pem             100% 1277   488.0KB/s   00:00    kubelet-client-current.pem                         100% 1277   423.3KB/s   00:00

`3.3.3、把kubelet，kube-proxy的service文件拷贝到node2中`

[root@localhost ~]# scp /usr/lib/systemd/system/{kubelet,kube-proxy}.service root@192.168.35.102:/usr/lib/systemd/system/root@192.168.35.102's password: kubelet.service                                    100%  264   113.9KB/s   00:00    kube-proxy.service                                 100%  231   529.7KB/s   00:00

`第四部分（加入群集）`

`4.1、在node02上操作，进行修改`

`4.1.11、首先删除复制过来的证书，等会node02会自行申请证书`

[root@localhost ~]# cd /opt/kubernetes/ssl/[root@localhost ssl]# rm -rf *

`4.1.2、修改配置文件kubelet kubelet.config kube-proxy（三个配置文件）`

[root@localhost ssl]# cd ../cfg/[root@localhost cfg]# lsbootstrap.kubeconfig  kubelet         kubelet.kubeconfig  kube-proxy.kubeconfigflanneld              kubelet.config  kube-proxy[root@localhost cfg]# vim kubeletKUBELET_OPTS="--logtostderr=true \--v=4 \--hostname-override=192.168.35.102 \--kubeconfig=/opt/kubernetes/cfg/kubelet.kubeconfig \--bootstrap-kubeconfig=/opt/kubernetes/cfg/bootstrap.kubeconfig \--config=/opt/kubernetes/cfg/kubelet.config \--cert-dir=/opt/kubernetes/ssl \--pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0"[root@localhost cfg]# vim kubelet.config kind: KubeletConfigurationapiVersion: kubelet.config.k8s.io/v1beta1address: 192.168.35.102port: 10250readOnlyPort: 10255cgroupDriver: cgroupfsclusterDNS:- 10.0.0.2clusterDomain: cluster.local.failSwapOn: falseauthentication:  anonymous:    enabled: true[root@localhost cfg]# vim kube-proxyKUBE_PROXY_OPTS="--logtostderr=true \--v=4 \--hostname-override=192.168.35.102 \--cluster-cidr=10.0.0.0/24 \--proxy-mode=ipvs \--kubeconfig=/opt/kubernetes/cfg/kube-proxy.kubeconfig"

`4.1.3、启动服务`

[root@localhost cfg]#  systemctl start kubelet.service[root@localhost cfg]#  systemctl enable kubelet.serviceCreated symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.[root@localhost cfg]#  systemctl start kube-proxy.service [root@localhost cfg]#  systemctl enable kube-proxy.service Created symlink from /etc/systemd/system/multi-user.target.wants/kube-proxy.service to /usr/lib/systemd/system/kube-proxy.service.

`4.2、在master上操作`

`4.2.1、查看请求`

[root@localhost kubeconfig]# kubectl get csrNAME                                                   AGE     REQUESTOR           CONDITIONnode-csr-HF551kae9hkzNGwirEbd2uAiStXnawpEI-y7nLP4sTU   32m     kubelet-bootstrap   Approved,Issuednode-csr-TszRzIg1T7JcfFKuwrCb82bqv7mcwvX1xZD8KMQJZCY   2m25s   kubelet-bootstrap   Pending

`4.2.2、授权许可加入群集`

[root@localhost kubeconfig]# kubectl certificate approve node-csr-TszRzIg1T7JcfFKuwrCb82bqv7mcwvX1xZD8KMQJZCYcertificatesigningrequest.certificates.k8s.io/node-csr-TszRzIg1T7JcfFKuwrCb82bqv7mcwvX1xZD8KMQJZCY approved

`4.2.3、查看群集中的节点`

[root@localhost kubeconfig]# kubectl get node


[root@localhost kubeconfig]# kubectl get node
NAME             STATUS     ROLES    AGE   VERSION192.168.35.101   Ready         24m   v1.12.3192.168.35.102   Ready         17m     v1.12.3

看完上述内容，你们对Kubernetes的集群部署有进一步的了解吗？如果还想学到更多技能或想了解更多相关内容，请关注行业资讯频道，感谢各位的阅读！




         很赞哦！ 
        

        


        证书
            节点
            文件
            服务
            生成
            组件
            配置
            集群
            拷贝
            脚本
            二进制
            状态
            目录
            部分
            内容
            参数
            地址
            官方
            工具
            网络
    
数据库的安全要保护哪些东西
数据库安全各自的含义是什么
生产安全数据库录入
数据库的安全性及管理
数据库安全策略包含哪些
海淀数据库安全审计系统
建立农村房屋安全信息数据库
易用的数据库客户端支持安全管理
连接数据库失败ssl安全错误
数据库的锁怎样保障安全

vs 数据库建模
受大众喜欢的数据库性能管理
云服务软件开发工程师
国家网络安全平台校园日
网络安全构成什么内容
数据库查询修改
包头软件开发规范
数据库工程师的分类
未转变者服务器花钱么
2021国家网络安全总结
百度网盘的数据库中心在哪里
大学的ftp 服务器
服务器处理器带r什么意思
材料杂志数据库
金融网络安全风采展示
jsp 访问数据库
视频管理服务器功率
服务器安全狗手机app
go做服务器管理
杨浦区即时网络技术费用
isc大会网络安全演讲
流媒体服务器软件生产厂
和rpg一样的服务器
服务器能退吗
虫珀数据库
厦门备件管理软件开发
如何连接服务器数据库
edr网络安全是什么的缩写
战舰世界手游哪个服务器好
数字通讯与网络安全前景怎么样
        
        


        
          
            扫描关注千家信息网微信公众号，第一时间获取内容更新动态
            转载请说明来源于"千家信息网"
            本文地址：https://www.qianjiagd.com/a188904
          
        
        
        
          
            上一篇
              linux执行vi命令编辑文件实现强制退出不保存的方法
              1、打开终端，执行vi命令打开一个文件，例如：vi test.txt2、然后按a或i进入编辑模式，编辑文本内容3、最后按esc键退出编辑模式，输入:q!，例如：:q! #退出不保存以上就是linux使
              
            下一篇
              linux中pfn是什么意思
              这篇文章将为大家详细讲解有关linux中pfn是什么意思，小编觉得挺实用的，因此分享给大家做个参考，希望大家阅读完这篇文章后可以有所收获。在linux中，pfn全称"page frame number




    


    
      
        相关文章
      
      
        recovery是什么意思?电脑开机重启显示recovery蓝屏怎么办
        怎么在Linux中配置SSH和Xshell远程连接服务器
        5分钟入门lsyncd
        基于centos7搭建MySQL数据库
        CentOS 7.7两种方式安装配置JDK 11 LTS
        提高企业开发效率的优质工具：快速开发平台
        Azure实践之如何批量为资源组虚拟机创建alert
        Azure Storage系列之使用AzCopy在blob
        k8s的StatefulSet（有状态服务）实现
        Azure App Service 纵向和横向缩放

 





  
    
      
        站长推荐
        点击排行
      
      
        
          



      recovery是什么意思?电脑开机重启显示recovery蓝屏怎么办
      
        
          怎么在Linux中配置SSH和Xshell远程连接服务器
          
        
          VS2008无法直接查看STL值怎么办
          
        
          什么是RPC框架
          
        
          .net mvc超过了最大请求长度怎么办
          
      
      java怎么实现try/catch异常块
      
        
          PHP中如何处理上传文件
          
        
          php中require_once报错的解决方法
          
        
          PHP如何编写学校网站上新生注册登陆程序
          
        
          php中微信公众号开发模式的示例分析
          
      

          
        
        
          


      在vmware esxi6.5中将硬盘驱动类型由HDD变为SSD类型
     
      
        Vue中的匿名插槽与具名插槽是什么
        vue3与vue2的区别以及vue3的API用法介绍
        录制的横屏视频怎么变成全屏竖屏（录制的横屏怎么变竖屏）
        qq群作业里为什么图片上传不了（qq群作业照片传不上去）
        vscoder如何关闭错误提示
        百度网盘PDF怎么转换成Word格式 PDF转Word操作教程
        老年机号码拉黑怎么解除（老年机号码拉黑怎么解除）
        京东以旧换新评估价和实际一样吗（京东以旧换新估价和成交价一样吗）
      
      拼多多注销后可以重开新用户吗（拼多多注销后重开算新用户吗）
  

          
        
      
    



    
      标签云
      


数据库的安全要保护哪些东西
数据库安全各自的含义是什么
生产安全数据库录入
数据库的安全性及管理
数据库安全策略包含哪些
海淀数据库安全审计系统
建立农村房屋安全信息数据库
易用的数据库客户端支持安全管理
连接数据库失败ssl安全错误
数据库的锁怎样保障安全
数据库安全章节测试
华大基因数据库安全性
数据库es安全性测试工具
数据库与云安全
微生物安全数据库
数据库个人信息安全吗
安全数据库降级
黑龙江数据库安全防护系统
数据库安全性实验例题
在国家公共安全数据库有记录


      
    


    

    
      猜你喜欢
      
        微信登录加载联系人失败怎么弄（微信加载联系人失败 点击重试）
        华为手机按键震动在哪设置关掉 按键振动怎么取消方法
        陌陌无限注册教程（怎么注册陌陌新号）
        win10开机蓝屏终止代码SYSTEM_SERVICE_EXCEPTION的解决方法
        微信看不到朋友圈不显示一条横线（微信看不到朋友圈只有一条横线）
        VS2008无法直接查看STL值怎么办
        快影怎么把视频弄成横屏播放 制作方法分享
        拼多多的多多支付怎么解绑银行卡（拼多多的多多支付怎么解绑银行卡）
        怎么将苹果手机中录音发给好友 iPhone传语音文件方法教程
        iis7.5中如何让html与shtml一样支持include功能

Kubernetes集群部署的方式有哪些？

1.官方提供三种部署方式

minikube

kubeadm

二进制包

2. Kubernetes 架构图

3.自签SSL证书

4. Etcd数据库集群部署

k8s单节点部署思路：

第一部分

第二部分（master）

第三部分（node）

第四部分（加入群集）

k8s部署规划：

负载均衡

Master节点

Node节点

第一部分

1.1、自签ETCD证书

1.1.1、master操作：

下面是etcd.sh脚本内容

下面是 etcd-cert.sh脚本内容

1.1.2、下载官方包

1.2、ETCD部署

1.2.1、定义证书

1.2.2、实现证书签名

1.2.3、生产证书，生成ca-key.pem ca.pem

1.2.4、指定etcd三个节点之间的通信验证

1.2.5、生成ETCD证书 server-key.pem server.pem

1.2.6、复制软件包到centos7中

1.2.7、解压

1.2.8、证书拷贝

1.2.9、此时打开另外一个会话，会发现etcd进程已经开启

1.2.10、在master1、node1、node2上都关闭防火墙

1.2.11、拷贝证书去其他节点

1.2.12、启动脚本拷贝其他节点

1.2.13、在node01节点修改

1.2.14、在node02节点修改

1.2.15、先在master上开启服务

1.2.16、在node1和node2上开启服务

1.2.17、再去master查看，会发现同步成功

1.2.18、检查群集状态

1.3、Node安装docker

部署19版docker

1.3.1、安装依赖包

1.3.2、设置阿里云镜像源

1.3.3、安装Docker-CE

1.3.4、关闭防火墙

1.3.5、开启服务并设为开机自启

1.3.6、镜像加速

1.3.7、网络优化

1.4、Flannel部署

1.4.1、写入分配的子网段到ETCD中，供flannel使用

1.4.2、查看写入的信息

1.4.3、拷贝到所有node节点（只需要部署在node节点即可）

1.4.4、所有node节点操作解压

1.4.5、node1操作

（1）k8s工作目录

（2）开启flannel网络功能

（3）配置docker连接flannel

（4）说明：bip指定启动时的子网

（5）重启docker服务

（6）查看flannel网络

（7）创建容器

1.4.6、node2操作（和上述node1操作一样）

（1）bip指定启动时的子网

（2）查看flannel网络

（3）创建容器

1.4.7、测试ping通对方docker0网卡 证明flannel起到路由作用

1.4.8、再次测试ping通两个node中的centos:7容器

第二部分（master）

2.1、自签APIServer证书

2.1.1、在master上操作，api-server生成证书

2.1.2、生成k8s证书

2.1.3、解压kubernetes压缩包

2.1.4、复制关键命令文件

2.2、部署APIServer组件（token，csv）

2.2.1、随机生成序列号

2.2.2、二进制文件，token，证书都准备好，开启apiserver

2.2.3、检查进程是否启动成功

`1.1.2、下载官方包`

`1.2、ETCD部署`

`1.2.1、定义证书`

`1.2.2、实现证书签名`

`1.2.3、生产证书，生成ca-key.pem ca.pem`

`1.2.4、指定etcd三个节点之间的通信验证`

`1.2.5、生成ETCD证书 server-key.pem server.pem`

`1.2.6、复制软件包到centos7中`

`1.2.7、解压`

`1.2.8、证书拷贝`

`1.2.9、此时打开另外一个会话，会发现etcd进程已经开启`

`1.2.10、在master1、node1、node2上都关闭防火墙`

`1.2.11、拷贝证书去其他节点`

`1.2.12、启动脚本拷贝其他节点`

`1.2.13、在node01节点修改`

`1.2.14、在node02节点修改`

`1.2.15、先在master上开启服务`

`1.2.16、在node1和node2上开启服务`

`1.2.17、再去master查看，会发现同步成功`

`1.2.18、检查群集状态`

`1.3、Node安装docker`

`部署19版docker`

`1.3.1、安装依赖包`

`1.3.2、设置阿里云镜像源`

`1.3.3、安装Docker-CE`

`1.3.4、关闭防火墙`

`1.3.5、开启服务并设为开机自启`

`1.3.6、镜像加速`

`1.3.7、网络优化`

`1.4、Flannel部署`

`1.4.1、写入分配的子网段到ETCD中，供flannel使用`

`1.4.2、查看写入的信息`

`1.4.3、拷贝到所有node节点（只需要部署在node节点即可）`

`1.4.4、所有node节点操作解压`

`1.4.5、node1操作`

`（1）k8s工作目录`

`（2）开启flannel网络功能`

`（3）配置docker连接flannel`

`（4）说明：bip指定启动时的子网`

`（5）重启docker服务`

`（6）查看flannel网络`

`（7）创建容器`

`1.4.6、node2操作（和上述node1操作一样）`

`（1）bip指定启动时的子网`

`（2）查看flannel网络`

`（3）创建容器`

`1.4.7、测试ping通对方docker0网卡证明flannel起到路由作用`

`1.4.8、再次测试ping通两个node中的centos:7容器`

`第二部分（master）`

`2.1、自签APIServer证书`

`2.1.1、在master上操作，api-server生成证书`

`2.1.2、生成k8s证书`

`2.1.3、解压kubernetes压缩包`

`2.1.4、复制关键命令文件`

`2.2、部署APIServer组件（token，csv）`

`2.2.1、随机生成序列号`

`2.2.2、二进制文件，token，证书都准备好，开启apiserver`

`2.2.3、检查进程是否启动成功`

`2.2.4、查看配置文件`

`2.2.5、监听的https端口`

`2.3、部署controller-manager（指定apiserver证书）和scheduler组件`

`2.3.1、启动scheduler服务`

`2.3.2、启动controller-manager`

`2.3.3、查看master 节点状态`

`第三部分（node）`

`3.1、生成kubeconfig（bootstrap，kubeconfig和kube-proxy.kubeconfig）`

`master上操作`

`3.1.1、把 kubelet、kube-proxy拷贝到node节点上去`

`nod01节点操作`

`3.1.2、复制node.zip到/root目录下再解压`

`master操作`

`3.1.3、创建工作目录`

`3.1.2、获取token信息`

`3.1.3、配置文件修改为tokenID`

`3.1.4、设置环境变量（可以写入到/etc/profile中）`

`3.1.5、重命名并生成配置文件`

`3.1.6、拷贝配置文件到node节点`

`3.1.7、创建bootstrap角色赋予权限用于连接apiserver请求签名（关键）`

`3.2、部署kubelet组件`

`在node01节点上操作`