手把手教你写一个通用的helm chart

[TOC]

1. 模板介绍

首先，放上此模板链接：

https://github.com/ygqygq2/charts/tree/master/ygqygq2/mod-chart

此chart可当作POD单image的通用模板，只需要使用sed替换下chart名，并修改下README.md和NOTES.txt就可以了。下文，我通过复制此chart成example-chart来作示范说明。

[root@master1 mod-chart]# tree.├── Chart.yaml  # chart版本信息文件├── README.md  # chart说明文件├── templates  # kubernetes资源yaml模板│   ├── configmap.yaml  # configmap模板│   ├── deployment-statefulset.yaml  # deployment或statefulset模板│   ├── _helpers.tpl  # 辅助模板和 partials│   ├── ingress.yaml  # ingress模板│   ├── NOTES.txt  # 部署chart后输出的帮助文档│   ├── pvc.yaml  # pvc模板│   ├── secret.yaml  # secret模板│   ├── service-headless.yaml  # service headless模板│   └── service.yaml  # service模板└── values.yaml  # 默认设置1 directory, 12 files[root@master1 mod-chart]# helm3 lint --strict .1 chart(s) linted, 0 chart(s) failed

2. 新chart制作

注：
下文中文件内容我保留，只加注释。
注释中需要修改的地方 [*] 标记为必选，[-] 标识为可选。

2.1 目录准备

将模板mod-chart复制成example-chart，并作内容替换。

rsync -avz mod-chart/ example-chart/cd example-chart/sed -i 's@mod-chart@example-chart@g' *.*sed -i 's@mod-chart@example-chart@g' templates/*.*

2.2 修改Chart.yaml

vim Chart.yaml

apiVersion: v1  # 当前helm api版本，不需要修改appVersion: 1.14.2  # 此处为你应用程序的版本号 [*]description: Chart for the nginx server  # 介绍此chart是干嘛的，按需求修改engine: gotpl  # go模板引擎，不需要修改 [-]name: example-chart  # 模板名，对应目录名 [*]version: 1.0.0  # 此chart版本号 [*]home: http://www.nginx.org  # 应用程序官网 [*]icon: https://cache.yisu.com/upload/information/20200309/33/60313.jpg  # 应用程序logo地址 [*]keywords:  # 关键字列表 [*]- nginx- http- web- www- reverse proxymaintainers:  # 维护人员列表 [*]- email: 29ygq@sina.com  name: Chinge Yangsources:  # 应用程序来源 [-]- https://github.com/bitnami/bitnami-docker-nginx

2.3 修改values.yaml

因为values.yaml设置涉及到yaml格式，yaml文件格式说明可以看这篇文章：

http://www.ruanyifeng.com/blog/2016/07/yaml.html

这里提几个常用的地方：

1. 使用2个空格作缩进；
2. 确认数字为字符类型时，使用双引号引起来；
3. 为了迎合helm3的规范，空定义最好将相关符号补上：

   string: ""list: []map: {}

没什么特殊要求，一般需要修改的地方有image、service、healthCheck、persistentVolume.mountPaths

# Default values for mod-chart.# This is a YAML-formatted file.# Declare variables to be passed into your templates.## Global Docker image parameters## Please, note that this will override the image parameters, including dependencies, configured to use the global value## Current available global Docker image parameters: imageRegistry and imagePullSecrets##global:  # 设置后覆盖后面默认的镜像仓库  imageRegistry: ""  imagePullSecrets: []#     - myRegistryKeySecretNamestatefulset:  enabled: false## String to partially override fullname template (will maintain the release name)##nameOverride: ""## String to fully override fullname template##fullnameOverride: ""## By default deploymentStrategy is set to rollingUpdate with maxSurge of 25% and maxUnavailable of 25% .## You can change type to `Recreate` or can uncomment `rollingUpdate` specification and adjust them to your usage.deploymentStrategy: {}  # rollingUpdate:  #   maxSurge: 25%  #   maxUnavailable: 25%  # type: RollingUpdate# 副本个数replicaCount: 1# 容器image及tagimage:  registry: docker.io  repository: bitnami/nginx  tag: latest  pullPolicy: IfNotPresent  # IfNotPresent: 有则不拉（减少流量和操作步骤），Always: 不管tag总拉（适合tag不变时更新）  pullSecrets: []  #  - private-registry-keyservice:  type: ClusterIP  # 一般不用修改  ingressPort: 8080  ports:    http:  # 多端口暴露时，复制一段      port: 8080  # Service port number for client-a port.      protocol: TCP  # Service port protocol for client-a port.## env set## ref: https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/env: []#  - name: DEMO_GREETING#    value: "Hello from the environment"#  - name: DEMO_FAREWELL#    value: "Such a sweet sorrow"## command setstartCommand: []#  - "java -Xdebug -Xnoagent -Djava.compiler=NONE"#  - "-Xrunjdwp:transport=dt_socket,address=5005,server=y,suspend=n"#  - "-Djava.security.egd=file:/dev/urandom"#  - "-jar /test.jar"#  - "-Duser.timezone=GMT+08"## Enable configmap and add data in configmapconfig:  enabled: false  subPath: ""  mountPath: /conf  data: {}############################# 示例 ###################################### 以下示例，挂载文件至 /conf/app.conf#  enabled: true#  mountPath: /conf/app.conf  #  subPath: app.conf    # 使用subPath时，上面mountPath路径写文件完整绝对路径#  data:#    app.conf: |-#      appname = example-chart## 以下示例，挂载多个文件至 /conf/ 下#  enabled: true#  mountPath: /conf    # 不使用subPath#  data:#    app.conf: |-#      appname = example-chart#    bpp.conf: |-#      bppname### 挂载多个文件至多个不同路径，需要相应修改 templates/deployment-statefulset.yaml ############################# 示例 ###################################### To use an additional secret, set enable to true and add data## 用法同上，不另作说明secret:  enabled: false  mountPath: /etc/secret-volume  subPath: ""  readOnly: true  data: {} ## liveness and readiness ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/healthCheck:  enabled: true  type: tcp  # http/tcp  port: http  # 健康检查的端口名或端口  httpPath: '/'  # http时必须设置  livenessInitialDelaySeconds: 10  # 初始延迟秒数  livenessPeriodSeconds: 10  # 检测周期，默认值10，最小为1  readinessInitialDelaySeconds: 10  # 初始延迟秒数  readinessPeriodSeconds: 10   # 检测周期，默认值10，最小为1resources: {}  # 容器资源设置  # We usually recommend not to specify default resources and to leave this as a conscious  # choice for the user. This also increases chances charts run on environments with little  # resources, such as Minikube. If you do want to specify resources, uncomment the following  # lines, adjust them as necessary, and remove the curly braces after 'resources:'.  # limits:  #   cpu: 100m  #   memory: 128Mi  # requests:  #   cpu: 100m  #   memory: 128Mi## Node labels and tolerations for pod assignment### ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector### ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#taints-and-tolerations-beta-featurelabels: {}podAnnotations: {}nodeSelector: {}tolerations: []affinity: {}annotations: {}## Enable persistence using Persistent Volume Claims## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/##persistentVolume:   # 是否存储持久化  enabled: false  ## If defined, storageClassName:   ## If set to "-", storageClassName: "", which disables dynamic provisioning  ## If undefined (the default) or set to null, no storageClassName spec is  ##   set, choosing the default provisioner.  (gp2 on AWS, azure-disk on  ##   Azure, standard on GKE, AWS & OpenStack)  ##  storageClass: "-"  accessMode: ReadWriteOnce  annotations: {}  #   helm.sh/resource-policy: keep  size: 1Gi  # 大小  existingClaim: {}  # 使用已存在的pvc  mountPaths: []  #  - name: data-storage  #    mountPath: /config  #    subPath: config  # 多个路径使用同一个pvc使用subPath，用法同上面config中示例说明  #  - name: data-storage  #    mountPath: /data  #    subPath: dataingress:  # 是否使用nginx暴露域名或端口  enabled: false  annotations: {}    # kubernetes.io/ingress.class: nginx    # kubernetes.io/tls-acme: "true"  path: /  hosts:    - chart-example.local  tls: []  #  - secretName: chart-example-tls  #    hosts:  #      - chart-example.local## Add init containers. e.g. to be used to give specific permissions for data## Add your own init container or uncomment and modify the given example.initContainers: []## Prometheus Exporter / Metrics##metrics:  enabled: false  image:    registry: docker.io    repository: nginx/nginx-prometheus-exporter    tag: 0.1.0    pullPolicy: IfNotPresent    ## Optionally specify an array of imagePullSecrets.    ## Secrets must be manually created in the namespace.    ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/    ##    pullSecrets: []    #   - myRegistrKeySecretName  ## Metrics exporter pod Annotation and Labels  podAnnotations:    # prometheus.io/scrape: "true"    # prometheus.io/port: "9113"    ## Metrics exporter resource requests and limits    ## ref: http://kubernetes.io/docs/user-guide/compute-resources/    ##  resources: {}## Uncomment and modify this to run a command after starting the core container.## ref: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/lifecycle: {}  # preStop:  #   exec:  #     command: ["/bin/bash","/pre-stop.sh"]  # postStart:  #   exec:  #     command: ["/bin/bash","/post-start.sh"]## Deployment additional volumes.deployment:  additionalVolumes: []## init containers## ref: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/## Add init containers. e.g. to be used to give specific permissions for data## Add your own init container or uncomment and modify the given example.initContainers: {}#  - name: fmp-volume-permission#    image: busybox#    imagePullPolicy: IfNotPresent#    command: ['chown','-R', '200', '/extra-data']#    volumeMounts:#      - name: extra-data#        mountPath: /extra-data## Additional containers to be added to the core pod.additionalContainers: {}#  - name: my-sidecar#    image: nginx:latest#  - name: lemonldap-ng-controller#    image: lemonldapng/lemonldap-ng-controller:0.2.0#    args:#      - /lemonldap-ng-controller#      - --alsologtostderr#      - --configmap=$(POD_NAMESPACE)/lemonldap-ng-configuration#    env:#      - name: POD_NAME#        valueFrom:#          fieldRef:#            fieldPath: metadata.name#      - name: POD_NAMESPACE#        valueFrom:#          fieldRef:#            fieldPath: metadata.namespace#    volumeMounts:#    - name: copy-portal-skins#      mountPath: /srv/var/lib/lemonldap-ng/portal/skins

2.4 修改README.md和templates/NOTES.txt

根据 values.yaml中的默认设置相应修改README.md，内容使用markdown语法，这里不作详细说明。
templates/NOTES.txt是部署chart后输出的帮助文档，里面支持go template语法。模板里已经写成了非常通用。必要情况下，适当按应用需求来修改，这样会显得部署后提示非常友好和人性化。

2.5 templates下yaml简要说明

templates目录下为kubernetes资源yaml文件模板，以资源名命名文件名，复杂些的可以加上资源功能或者模块名等。

templates/secret.yaml

{{- if .Values.secret.enabled }}  # if用法。语法代码注意形成一致缩进习惯，便于阅读apiVersion: v1kind: Secretmetadata:  name: {{ template "example-chart.fullname" . }}  # 使用辅助模板时，点号可用 $ 代替，特别是在range下  labels:    app: {{ template "example-chart.name" . }}    chart: {{ template "example-chart.chart" . }}    release: "{{ .Release.Name }}"    heritage: "{{ .Release.Service }}"    {{- if .Values.labels }}{{ toYaml .Values.labels | indent 4 }}  # 使用toYaml时，不缩进，indent接空格数    {{- end }}data:{{- range $key, $value := .Values.secret.data }}  # range用法  {{ $key }}: {{ $value | b64enc | quote }}  # secret中value注意使用base64转换{{- end }}{{- end }}

templates/deployment-statefulset.yaml

{{- if .Values.statefulset.enabled }}  # 判断使用deployment和statefulset资源api类型apiVersion: apps/v1kind: StatefulSet{{- else }}apiVersion: apps/v1kind: Deployment{{- end }}metadata:  name: {{ template "example-chart.fullname" . }}  labels:    app: {{ template "example-chart.name" . }}    chart: {{ template "example-chart.chart" . }}    release: {{ .Release.Name }}    heritage: {{ .Release.Service }}{{- if .Values.labels }}  # 额外的标签{{ toYaml .Values.labels | indent 4 }}{{- end }}{{- if .Values.annotations }}  # 自定义注释  annotations:{{ toYaml .Values.annotations | indent 4 }}{{- end }}spec:  replicas: {{ .Values.replicaCount }}  # 副本数  {{- if .Values.statefulset.enabled }}  # statefulset需要定义serviceName  serviceName: {{ template "example-chart.fullname" . }}-headless  {{- end }}  {{- if .Values.deploymentStrategy }}  strategy:{{ toYaml .Values.deploymentStrategy | indent 4 }}  {{- end }}  selector:    matchLabels:      app: {{ template "example-chart.name" . }}      release: {{ .Release.Name }}  template:    metadata:      annotations:  {{- if .Values.podAnnotations }}{{ toYaml .Values.podAnnotations | indent 8 }}{{- end }}{{- if .Values.metrics.podAnnotations }}{{ toYaml .Values.metrics.podAnnotations | indent 8 }}{{- end }}      labels:        app: {{ template "example-chart.name" . }}        release: {{ .Release.Name }}    spec:{{- include "example-chart.imagePullSecrets" . | indent 6 }}      {{- if .Values.initContainers }}      initContainers:{{ toYaml .Values.initContainers | indent 8 }}      {{- end }}      nodeSelector:{{ toYaml .Values.nodeSelector | indent 8 }}      affinity:{{ toYaml .Values.affinity | indent 8 }}      tolerations:{{ toYaml .Values.tolerations | indent 8 }}      containers:{{- if .Values.metrics.enabled }}  # metrics容器可根据需求修改        - name: metrics          image: {{ template "metrics.image" . }}          imagePullPolicy: {{ .Values.metrics.image.pullPolicy | quote }}          command: [ '/usr/bin/exporter', '-nginx.scrape-uri', 'http://127.0.0.1:8080/status']          ports:          - name: metrics            containerPort: 9113          livenessProbe:            httpGet:              path: /metrics              port: metrics            initialDelaySeconds: 15            timeoutSeconds: 5          readinessProbe:            httpGet:              path: /metrics              port: metrics            initialDelaySeconds: 5            timeoutSeconds: 1          resources:{{ toYaml .Values.metrics.resources | indent 12 }}{{- end }}        - name: {{ .Chart.Name }}          image: {{ template "example-chart.image" . }}          imagePullPolicy: {{ .Values.image.pullPolicy | quote }}          {{- if .Values.lifecycle }}          lifecycle:{{ toYaml .Values.lifecycle | indent 12 }}          {{- end }}          {{- if .Values.startCommand }}          command:{{ toYaml .Values.startCommand |indent 12 }}          {{- end }}          env:{{ toYaml .Values.env | indent 12 }}          resources:{{ toYaml .Values.resources | indent 12 }}          ports:            {{- range $key, $value := .Values.service.ports }}            - name: {{ $key }}              containerPort: {{ $value.port }}              protocol: {{ $value.protocol }}            {{- end }}          {{- if .Values.healthCheck.enabled }}          livenessProbe:            {{- if eq .Values.healthCheck.type "http" }}            httpGet:              path: {{ .Values.healthCheck.httpPath }}              port: {{ .Values.healthCheck.port }}            {{- else }}            tcpSocket:              port: {{ .Values.healthCheck.port }}            {{- end }}            initialDelaySeconds: {{ .Values.healthCheck.livenessInitialDelaySeconds }}            periodSeconds: {{ .Values.healthCheck.livenessPeriodSeconds }}          readinessProbe:            {{- if eq .Values.healthCheck.type "http" }}            httpGet:              path: {{ .Values.healthCheck.httpPath }}              port: {{ .Values.healthCheck.port }}            {{- else }}            tcpSocket:              port: {{ .Values.healthCheck.port }}            {{- end }}            initialDelaySeconds: {{ .Values.healthCheck.readinessInitialDelaySeconds }}            periodSeconds: {{ .Values.healthCheck.readinessPeriodSeconds }}          {{- end }}          volumeMounts:  # 容器挂载点            {{- if .Values.config.enabled }}            - name: {{ template "example-chart.name" . }}-conf              mountPath: {{ .Values.config.mountPath }}              subPath: {{ .Values.config.subPath }}            {{- end }}            {{- if .Values.secret.enabled }}            - name: {{ template "example-chart.name" . }}-secret              mountPath: {{ .Values.secret.mountPath }}              subPath: {{ .Values.secret.subPath }}              readOnly: {{ .Values.secret.readOnly }}            {{- end }}{{- if .Values.persistentVolume.mountPaths }}{{ toYaml .Values.persistentVolume.mountPaths | indent 12 }}{{- end }}        {{- if .Values.additionalContainers }}{{ toYaml .Values.additionalContainers | indent 8 }}        {{- end }}      volumes:  # volume名需要和上文volumeMounts中的名字一一对应        {{- if .Values.config.enabled }}        - name: {{ template "example-chart.name" . }}-conf          configMap:            name: {{ template "example-chart.fullname" . }}        {{- end }}        {{- if .Values.secret.enabled }}        - name: {{ template "example-chart.name" . }}-secret          secret:            secretName: {{ template "example-chart.fullname" . }}        {{- end }}        {{- if .Values.deployment.additionalVolumes }}{{ toYaml .Values.deployment.additionalVolumes | indent 8 }}        {{- end }}{{- if not .Values.statefulset.enabled }}          {{- if .Values.persistentVolume.enabled }}        - name: data-storage          persistentVolumeClaim:            claimName: {{ .Values.persistentVolume.existingClaim | default (include "example-chart.fullname" .) }}          {{- else }}        - name: data-storage          emptyDir: {}          {{- end }}{{- else }}  {{- if .Values.persistentVolume.enabled }}  volumeClaimTemplates:  - metadata:      name: data-storage      labels:        app: {{ template "example-chart.name" . }}        chart: {{ template "example-chart.chart" . }}        release: {{ .Release.Name }}        heritage: {{ .Release.Service }}    spec:      accessModes:        - {{ .Values.persistentVolume.accessMode | quote }}      annotations:      {{- range $key, $value := $.Values.persistentVolume.annotations }}        {{ $key }}: {{ $value }}      {{- end }}      resources:        requests:          storage: {{ .Values.persistentVolume.size }}          {{- if .Values.persistentVolume.storageClass }}            {{- if (eq "-" .Values.persistentVolume.storageClass) }}      storageClassName: ""            {{- else }}      storageClassName: "{{ .Values.persistentVolume.storageClass }}"            {{- end }}          {{- end }}  {{- else }}        - name: data-storage          emptyDir: {}  {{- end }}{{- end -}}

3. 小结

以上yaml中未作详细说明，仔细看其内容都能明白大致意思。以下是对于helm chart新手的一些建议：

1. 刚接触helm chart时，多模仿stable和bianami中charts的写法，特别是values.yaml和templates目录中的一些设计，差不多都已经非常统一了。这样遇到自己有相似需求，可直接使用相应的功能块，写出来的chart也显得非常专业。
2. 使用chart时，最好使用helm命令fetch下来，大概读一遍其chart内容，这样看多了，自然就越来越熟悉，而且出错时，也便于自己排查问题。

参考资料：
[1] https://helm.sh/
[2] https://whmzsu.github.io/helm-doc-zh-cn/