nacos集群部署-k8s环境
发表于：2024-10-23 作者：千家信息网编辑
千家信息网最后更新 2024年10月23日，通过StatefulSets 在Kubernetes上部署Nacos快速部署可以参考官网https://nacos.io/en-us/docs/use-nacos-with-kubernetes.ht
千家信息网最后更新 2024年10月23日nacos集群部署-k8s环境
通过StatefulSets 在Kubernetes上部署Nacos
快速部署可以参考官网https://nacos.io/en-us/docs/use-nacos-with-kubernetes.html
1 快速部署
git clone https://github.com/nacos-group/nacos-k8s.git cd nacos-k8schmod +x quick-startup.sh./quick-startup.sh
1.2服务测试
服务注册curl -X PUT 'http://cluster-ip:8848/nacos/v1/ns/instance?serviceName=nacos.naming.serviceName&ip=20.18.7.10&port=8080'服务发现curl -X GET 'http://cluster-ip:8848/nacos/v1/ns/instances?serviceName=nacos.naming.serviceName'发布配置curl -X POST "http://cluster-ip:8848/nacos/v1/cs/configs?dataId=nacos.cfg.dataId&group=test&content=helloWorld"获取配置curl -X GET "http://cluster-ip:8848/nacos/v1/cs/configs?dataId=nacos.cfg.dataId&group=test"
2 NFS方式部署。
NFS的使用是为了保留数据，数据库的数据以及nacos的数据日志等。
该方式部署需要对官方的yaml进行修改，下面列出实测可用的步骤及yaml文件
2.1部署NFS服务环境
找一台可以与k8s环境通信的内网机器192.168.1.10，在机器上部署nfs服务，选择合适磁盘，作为共享目录。
yum install -y nfs-utils rpcbindmkdir -p /data/nfsmkdir -p /data/mysql_master mkdir -p /data/mysql_slavevim /etc/exports/data/nfs *(insecure,rw,async,no_root_squash)/data/mysql_slave *(insecure,rw,async,no_root_squash)/data/mysql_master *(insecure,rw,async,no_root_squash)systemctl start rpcbindsystemctl start nfssystemctl enable rpcbindsystemctl enable nfs-serverexportfs -ashowmount -e
2.2 k8s上部署nfs
cd nacos-k8s/deploy/nfs/[root@localhost nfs]# ll总用量 12-rw-r--r--. 1 root root 153 10月 15 08:05 class.yaml-rw-r--r--. 1 root root 877 10月 15 14:37 deployment.yaml-rw-r--r--. 1 root root 1508 10月 15 08:05 rbac.yaml
2.2.1 创建rbac,使用默认的rbac.yaml 不用修改，采用的是default命名空间，如果需要部署到特定的命名空间，则修改其中的namespace。
kubectl create -f rbac.yaml
kind: ClusterRoleapiVersion: rbac.authorization.k8s.io/v1metadata: name: nfs-client-provisioner-runnerrules:- apiGroups: [""] resources: ["persistentvolumes"] verbs: ["get", "list", "watch", "create", "delete"]- apiGroups: [""] resources: ["persistentvolumeclaims"] verbs: ["get", "list", "watch", "update"]- apiGroups: [""] resources: ["endpoints"] verbs: ["get", "list", "watch", "create", "update", "patch"]- apiGroups: ["storage.k8s.io"] resources: ["storageclasses"] verbs: ["get", "list", "watch"]- apiGroups: [""] resources: ["events"] verbs: ["create", "update", "patch"]---kind: ClusterRoleBindingapiVersion: rbac.authorization.k8s.io/v1metadata: name: run-nfs-client-provisionersubjects:- kind: ServiceAccount name: nfs-client-provisioner namespace: defaultroleRef: kind: ClusterRole name: nfs-client-provisioner-runner apiGroup: rbac.authorization.k8s.io---kind: RoleapiVersion: rbac.authorization.k8s.io/v1metadata: name: leader-locking-nfs-client-provisionerrules:- apiGroups: [""] resources: ["endpoints"] verbs: ["get", "list", "watch", "create", "update", "patch"]---kind: RoleBindingapiVersion: rbac.authorization.k8s.io/v1metadata: name: leader-locking-nfs-client-provisionersubjects:- kind: ServiceAccount name: nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: defaultroleRef: kind: Role name: leader-locking-nfs-client-provisioner
2.2.2 创建ServiceAccount和部署NFS-Client Provisioner
kubectl create -f deployment.yaml ##修改ip和目录
apiVersion: v1kind: ServiceAccountmetadata: name: nfs-client-provisioner---kind: DeploymentapiVersion: extensions/v1beta1metadata: name: nfs-client-provisionerspec: replicas: 1 strategy: type: Recreate template: metadata: labels: app: nfs-client-provisioner spec: serviceAccount: nfs-client-provisioner containers: - name: nfs-client-provisioner image: quay.io/external_storage/nfs-client-provisioner:latest volumeMounts: - name: nfs-client-root mountPath: /persistentvolumes env: - name: PROVISIONER_NAME value: fuseim.pri/ifs - name: NFS_SERVER value: 192.168.1.10 - name: NFS_PATH value: /data/nfs volumes: - name: nfs-client-root nfs: server: 192.168.1.10 path: /data/nfs
2.2.3 创建NFS StorageClass
kubectl create -f class.yaml ##无需修改yaml
apiVersion: storage.k8s.io/v1kind: StorageClassmetadata: name: managed-nfs-storageprovisioner: fuseim.pri/ifsparameters: archiveOnDelete: "false"
2.3 部署数据库
cd nacos-k8s/deploy/mysql/
2.3.1部署主数据库
kubectl create -f mysql-master-nfs.yaml ##需要修改的是nfs的IP和目录
apiVersion: v1kind: ReplicationControllermetadata: name: mysql-master labels: name: mysql-masterspec: replicas: 1 selector: name: mysql-master template: metadata: labels: name: mysql-master spec: containers: - name: master image: nacos/nacos-mysql-master:latest ports: - containerPort: 3306 volumeMounts: - name: mysql-master-data mountPath: /var/lib/mysql env: - name: MYSQL_ROOT_PASSWORD value: "root" - name: MYSQL_DATABASE value: "nacos_devtest" - name: MYSQL_USER value: "nacos" - name: MYSQL_PASSWORD value: "nacos" - name: MYSQL_REPLICATION_USER value: 'nacos_ru' - name: MYSQL_REPLICATION_PASSWORD value: 'nacos_ru' volumes: - name: mysql-master-data nfs: server: 192.168.1.10 path: /data/mysql_master---apiVersion: v1kind: Servicemetadata: name: mysql-master labels: name: mysql-masterspec: ports: - port: 3306 targetPort: 3306 selector: name: mysql-master
2.3.2 部署从数据库
kubectl create -f mysql-slave-nfs.yaml
apiVersion: v1kind: ReplicationControllermetadata: name: mysql-slave labels: name: mysql-slavespec: replicas: 1 selector: name: mysql-slave template: metadata: labels: name: mysql-slave spec: containers: - name: slave image: nacos/nacos-mysql-slave:latest ports: - containerPort: 3306 volumeMounts: - name: mysql-slave-data mountPath: /var/lib/mysql env: - name: MYSQL_ROOT_PASSWORD value: "root" - name: MYSQL_REPLICATION_USER value: 'nacos_ru' - name: MYSQL_REPLICATION_PASSWORD value: 'nacos_ru' volumes: - name: mysql-slave-data nfs: server: 192.168.1.10 path: /data/mysql_slave---apiVersion: v1kind: Servicemetadata: name: mysql-slave labels: name: mysql-slavespec: ports: - port: 3306 targetPort: 3306 selector: name: mysql-slave
2.4 部署nacos
cd nacos-k8s/deploy/nacos/
kubectl create -f nacos-pvc-nfs.yaml ##该文件需要大修改，主要是改成基于quickstart版本增加挂载，其他无关的内容清理掉，具体如下
注意- name: NACOS_SERVERS这一项，创建时候域名是会自动生成一个这样的cluster.local，nacos-0.nacos-headless.default.svc.cluster.local:8848，我的k8s默认集群名设置成了cluster.test。所以文件中修改成这样的nacos-0.nacos-headless.default.svc.cluster.test:8848。
---apiVersion: v1kind: Servicemetadata: name: nacos-headless labels: app: nacos-headlessspec: ports: - port: 8848 name: server targetPort: 8848 selector: app: nacos---apiVersion: v1kind: ConfigMapmetadata: name: nacos-cmdata: mysql.master.db.name: "nacos_devtest" mysql.master.port: "3306" mysql.slave.port: "3306" mysql.master.user: "nacos" mysql.master.password: "nacos"---apiVersion: apps/v1kind: StatefulSetmetadata: name: nacosspec: serviceName: nacos-headless replicas: 3 template: metadata: labels: app: nacos annotations: pod.alpha.kubernetes.io/initialized: "true" spec: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: "app" operator: In values: - nacos-headless topologyKey: "kubernetes.io/hostname" containers: - name: k8snacos imagePullPolicy: Always image: nacos/nacos-server:latest resources: requests: memory: "2Gi" cpu: "500m" ports: - containerPort: 8848 name: client env: - name: NACOS_REPLICAS value: "3" - name: SERVICE_NAME value: "nacos-headless" - name: POD_NAMESPACE valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.namespace - name: MYSQL_MASTER_SERVICE_DB_NAME valueFrom: configMapKeyRef: name: nacos-cm key: mysql.master.db.name - name: MYSQL_MASTER_SERVICE_PORT valueFrom: configMapKeyRef: name: nacos-cm key: mysql.master.port - name: MYSQL_SLAVE_SERVICE_PORT valueFrom: configMapKeyRef: name: nacos-cm key: mysql.slave.port - name: MYSQL_MASTER_SERVICE_USER valueFrom: configMapKeyRef: name: nacos-cm key: mysql.master.user - name: MYSQL_MASTER_SERVICE_PASSWORD valueFrom: configMapKeyRef: name: nacos-cm key: mysql.master.password - name: NACOS_SERVER_PORT value: "8848" - name: PREFER_HOST_MODE value: "hostname" - name: NACOS_SERVERS value: "nacos-0.nacos-headless.default.svc.cluster.test:8848 nacos-1.nacos-headless.default.svc.cluster.test:8848 nacos-2.nacos-headless.default.svc.cluster.test:8848" volumeMounts: - name: datadir mountPath: /home/nacos/data - name: logdir mountPath: /home/nacos/logs volumeClaimTemplates: - metadata: name: datadir annotations: volume.beta.kubernetes.io/storage-class: "managed-nfs-storage" spec: accessModes: [ "ReadWriteMany" ] resources: requests: storage: 5Gi - metadata: name: logdir annotations: volume.beta.kubernetes.io/storage-class: "managed-nfs-storage" spec: accessModes: [ "ReadWriteMany" ] resources: requests: storage: 5Gi selector: matchLabels: app: nacos
也可以使用这种原始的
---apiVersion: v1kind: Servicemetadata: name: nacos-headless labels: app: nacos annotations: service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"spec: ports: - port: 8848 name: server targetPort: 8848 clusterIP: None selector: app: nacos---apiVersion: v1kind: ConfigMapmetadata: name: nacos-cmdata: mysql.master.db.name: "nacos_devtest" mysql.master.port: "3306" mysql.slave.port: "3306" mysql.master.user: "nacos" mysql.master.password: "nacos"---apiVersion: apps/v1kind: StatefulSetmetadata: name: nacosspec: serviceName: nacos-headless replicas: 2 template: metadata: labels: app: nacos annotations: pod.alpha.kubernetes.io/initialized: "true" spec: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: "app" operator: In values: - nacos topologyKey: "kubernetes.io/hostname" serviceAccountName: nfs-client-provisioner initContainers: - name: peer-finder-plugin-install image: nacos/nacos-peer-finder-plugin:latest imagePullPolicy: Always volumeMounts: - mountPath: "/home/nacos/plugins/peer-finder" name: plugindir containers: - name: nacos imagePullPolicy: Always image: nacos/nacos-server:latest resources: requests: memory: "2Gi" cpu: "1000m" ports: - containerPort: 8848 name: client-port env: - name: NACOS_REPLICAS value: "3" - name: SERVICE_NAME value: "nacos-headless" - name: POD_NAMESPACE valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.namespace - name: MYSQL_MASTER_SERVICE_DB_NAME valueFrom: configMapKeyRef: name: nacos-cm key: mysql.master.db.name - name: MYSQL_MASTER_SERVICE_PORT valueFrom: configMapKeyRef: name: nacos-cm key: mysql.master.port - name: MYSQL_SLAVE_SERVICE_PORT valueFrom: configMapKeyRef: name: nacos-cm key: mysql.slave.port - name: MYSQL_MASTER_SERVICE_USER valueFrom: configMapKeyRef: name: nacos-cm key: mysql.master.user - name: MYSQL_MASTER_SERVICE_PASSWORD valueFrom: configMapKeyRef: name: nacos-cm key: mysql.master.password - name: NACOS_SERVER_PORT value: "8848" - name: PREFER_HOST_MODE value: "hostname" readinessProbe: httpGet: port: client-port path: /nacos/v1/console/health/readiness initialDelaySeconds: 60 timeoutSeconds: 3 livenessProbe: httpGet: port: client-port path: /nacos/v1/console/health/liveness initialDelaySeconds: 60 timeoutSeconds: 3 volumeMounts: - name: plugindir mountPath: /home/nacos/plugins/peer-finder - name: datadir mountPath: /home/nacos/data - name: logdir mountPath: /home/nacos/logs volumeClaimTemplates: - metadata: name: plugindir annotations: volume.beta.kubernetes.io/storage-class: "managed-nfs-storage" spec: accessModes: [ "ReadWriteMany" ] resources: requests: storage: 5Gi - metadata: name: datadir annotations: volume.beta.kubernetes.io/storage-class: "managed-nfs-storage" spec: accessModes: [ "ReadWriteMany" ] resources: requests: storage: 5Gi - metadata: name: logdir annotations: volume.beta.kubernetes.io/storage-class: "managed-nfs-storage" spec: accessModes: [ "ReadWriteMany" ] resources: requests: storage: 5Gi selector: matchLabels: app: nacos
查看结果
[root@localhost nacos]# kubectl get pod NAME READY STATUS RESTARTS AGEmysql-master-hnnzq 1/1 Running 0 43hmysql-slave-jjq98 1/1 Running 0 43hnacos-0 1/1 Running 0 41hnacos-1 1/1 Running 0 41hnacos-2 1/1 Running 0 41hnfs-client-provisioner-57c8c85896-cpxtx 1/1 Running 0 45h[root@localhost nacos]# kubectl get svcNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEkubernetes ClusterIP 172.21.0.1443/TCP 9dmysql-master ClusterIP 172.21.12.11 3306/TCP 43hmysql-slave ClusterIP 172.21.1.9 3306/TCP 43hnacos-headless ClusterIP 172.21.11.220 8848/TCP 41hnginx-svc ClusterIP 172.21.1.104 10080/TCP 8d[root@localhost nacos]# kubectl get storageclassNAME PROVISIONER AGEalicloud-disk-available alicloud/disk 9dalicloud-disk-efficiency alicloud/disk 9dalicloud-disk-essd alicloud/disk 9dalicloud-disk-ssd alicloud/disk 9dmanaged-nfs-storage fuseim.pri/ifs 45h
[root@localhost nacos]# kubectl get pv,pvcNAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGEpersistentvolume/pvc-c920f9cf-f56f-11e9-90dc-da6119823c38 5Gi RWX Delete Bound default/datadir-nacos-0 managed-nfs-storage 43hpersistentvolume/pvc-c921977d-f56f-11e9-90dc-da6119823c38 5Gi RWX Delete Bound default/logdir-nacos-0 managed-nfs-storage 43hpersistentvolume/pvc-c922401f-f56f-11e9-90dc-da6119823c38 5Gi RWX Delete Bound default/plugindir-nacos-0 managed-nfs-storage 43hpersistentvolume/pvc-db3ccda6-f56f-11e9-90dc-da6119823c38 5Gi RWX Delete Bound default/datadir-nacos-1 managed-nfs-storage 43hpersistentvolume/pvc-db3dc25a-f56f-11e9-90dc-da6119823c38 5Gi RWX Delete Bound default/logdir-nacos-1 managed-nfs-storage 43hpersistentvolume/pvc-db3eb86c-f56f-11e9-90dc-da6119823c38 5Gi RWX Delete Bound default/plugindir-nacos-1 managed-nfs-storage 43hpersistentvolume/pvc-fa47ae6e-f57a-11e9-90dc-da6119823c38 5Gi RWX Delete Bound default/logdir-nacos-2 managed-nfs-storage 41hpersistentvolume/pvc-fa489723-f57a-11e9-90dc-da6119823c38 5Gi RWX Delete Bound default/plugindir-nacos-2 managed-nfs-storage 41hpersistentvolume/pvc-fa494137-f57a-11e9-90dc-da6119823c38 5Gi RWX Delete Bound default/datadir-nacos-2 managed-nfs-storage 41hNAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGEpersistentvolumeclaim/datadir-nacos-0 Bound pvc-c920f9cf-f56f-11e9-90dc-da6119823c38 5Gi RWX managed-nfs-storage 43hpersistentvolumeclaim/datadir-nacos-1 Bound pvc-db3ccda6-f56f-11e9-90dc-da6119823c38 5Gi RWX managed-nfs-storage 43hpersistentvolumeclaim/datadir-nacos-2 Bound pvc-fa494137-f57a-11e9-90dc-da6119823c38 5Gi RWX managed-nfs-storage 41hpersistentvolumeclaim/logdir-nacos-0 Bound pvc-c921977d-f56f-11e9-90dc-da6119823c38 5Gi RWX managed-nfs-storage 43hpersistentvolumeclaim/logdir-nacos-1 Bound pvc-db3dc25a-f56f-11e9-90dc-da6119823c38 5Gi RWX managed-nfs-storage 43hpersistentvolumeclaim/logdir-nacos-2 Bound pvc-fa47ae6e-f57a-11e9-90dc-da6119823c38 5Gi RWX managed-nfs-storage 41hpersistentvolumeclaim/plugindir-nacos-0 Bound pvc-c922401f-f56f-11e9-90dc-da6119823c38 5Gi RWX managed-nfs-storage 43hpersistentvolumeclaim/plugindir-nacos-1 Bound pvc-db3eb86c-f56f-11e9-90dc-da6119823c38 5Gi RWX managed-nfs-storage 43hpersistentvolumeclaim/plugindir-nacos-2 Bound pvc-fa489723-f57a-11e9-90dc-da6119823c38 5Gi RWX managed-nfs-storage 41h[root@localhost nacos]#
然后用ingress映射出去就可以访问8848端口
数据
服务
数据库
文件
目录
环境
方式
机器
空间
配置
集群
原始
合适
不用
内容
域名
官方
日志
时候
步骤
数据库的安全要保护哪些东西
数据库安全各自的含义是什么
生产安全数据库录入
数据库的安全性及管理
数据库安全策略包含哪些
海淀数据库安全审计系统
建立农村房屋安全信息数据库
易用的数据库客户端支持安全管理
连接数据库失败ssl安全错误
数据库的锁怎样保障安全
计算机软件网络技术教学视频
网络技术环境复杂多变
四川省高校网络安全大赛
网络安全须知简介
服务器硬盘如何打开
软件开发简历模版word
连接网络加密服务器演示版
东软软件开发是不是假的
厦门装小白网络技术首页
网络安全红队的前途是什么
城管系统软件开发论文
门禁网络技术员
大脑神经网络技术
网络安全防护方向
如何估算软件开发量
网络安全公司能力规划
区块链网络安全建设
通达oa服务器集群
全县网络安全工作情况报告
运满满网络安全检查
网络技术概括
上海华为服务器虚拟化迁移
女性网络安全知识漫画
阿里聚安全 算法 数据库
巴南区网络软件开发流程特点
网络安全法全
dns服务器山东临沂移动云主机
以色列本古里安大学网络安全博士
东方网景网络技术有限公司
重用软件开发分析记录表
