本文分享自华为云社区《nginx-ingress工作原理以及多nginx-ingress部署注意事项》,作者: 可以交个朋友。
nginx-ingress对应的容器中有两个核心组件:nginx-ingress-controller和nginx。其中nginx-ingress-controller负责list-watch kube-apiserver监听ingress、service、endpoint、configmap资源的状态变化,转变为nginx配置,刷新给容器中nginx进程,再由nginx对外保留服务提供转发能力,大致工作流程如下:
nginx-ingress-a实例启动参数
nginx-ingress-b实例启动参数
kubernetes 1.23.x以下版本对应的apiVersion:networking.k8s.io/v1beta1
apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: test-a namespace: nginx-a annotations: kubernetes.io/ingress.class: custom-nginx-a spec: rules: - http: paths: - backend: service: name: nginx port: number: 80 path: / pathType: ImplementationSpecific property: ingress.beta.kubernetes.io/url-match-mode: STARTS_WITHSTARTS_WITH
kubernetes v1.23版本及以上apiVersion 是networking.k8s.io/v1
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: test-a namespace: nginx-a spec: ingressClassName: custom-nginx-a rules: - http: paths: - backend: service: name: nginx port: number: 80 path: / pathType: ImplementationSpecific property: ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH
admission webhook作用域默认是全局的,如果一个集群中有多个nginx-ingress组件(通过ingressClass区域),由于有多个ValidatingAdmissionWebhook,当某个ingress资源创建时,kube-apiserver会调用多个nginx-ingress webhok能力进行校验,当某个nginx-ingress出现故障(可能与该ingress资源并不关联),将会阻塞所有的ingress的写入。
多nginx-ingress场景下建议通过admissionwebhook 自带的namespaceSelector控制每个nginx-ingress的作用域,使其作用到指定的命名空间,只教验作用域命名空间下的ingresses资源。
admissionwebhook: namespaceSelector: matchExpressions: - key: kubernetes.io/metadata.name #需要教验的namespace含有的标签key operator: In values: ["${namespace}"] # namespace含有的标签value(默认命名空间本身) rules: - operations: ["CREATE","UPDATE"] apiGroups: ["*"] apiVersions: ["*"] resources: ["ingresses"] #限制作用资源 scope: "*" #由于限制了作用资源ingresses,无需配置 objectSelector: {}
如果集群版本在1.28及以上,可以通过matchConditions匹配请求,直接筛选ingressClassName字段,官网详情
登录到集群,将ingress-controller helm包下载到集群
wget https://github.com/kubernetes/ingress-nginx/releases/download/helm-chart-4.3.0/ingress-nginx-4.3.0.tgz
新建myvalue.yaml文件,内容如下;安装时指定此配置文件,其它未进行指定的参数会使用安装包中value.yaml默认配置值。
imagePullSecrets: {} #若仓库开启认证需要配置该字段 controller: name: controller #自定义controller名称,不可重复 image: repository: swr.cn-north-4.myhuaweicloud.com/hwofficial/nginx-ingress # controller镜像地址设定 tag: "v1.2.1" ingressClass: custom-nginx # 设定ingress-class值,不可重复 ingressClassResource: name: custom-nginx #设定ingressClass资源名称,不可重复 controllerValue: k8s.io/custom-nginx #设定监测ingressClass资源的controller,不可重复 # 设定controller要使用的loadbalancer service,本例中使用的是独享型ELB service: annotations: { # kubernetes.io/elb.class: performance #开启后turbo集群直通pod(仅限独享型elb) kubernetes.io/elb.id: 3660aa3c-xxxx-xxxx-xxxx-xxxxff97xxxx #ELB ID,不可重复 } # 设定controller的资源限制 resources: requests: cpu: 200m memory: 200Mi # 挂载节点上的/etc/localtime文件,进行时区同步 extraVolumeMounts: - name: localtime mountPath: /etc/localtime readOnly: true extraVolumes: - name: localtime hostPath: path: /etc/localtime admissionWebhooks: enabled: true failurePolicy: Fail port: 8443 certificate: "/usr/local/certificates/cert" key: "/usr/local/certificates/key" namespaceSelector: matchExpressions: - key: kubernetes.io/metadata.name operator: In values: ["${namespace}"] # ${namespace}修改为业务负载所在的命名空间,逗号分割 rules: - operations: ["CREATE","UPDATE"] apiGroups: ["*"] apiVersions: ["*"] resources: ["ingresses"] scope: "*" networkPolicyEnabled: false service: servicePort: 443 type: ClusterIP createSecretJob: resources: limits: cpu: 20m memory: 40Mi requests: cpu: 10m memory: 20Mi patch: enabled: true image: registry: registry.k8s.io # registry.k8s.io,webhook官网镜像仓库需要替换成自己镜像所在仓库地址 image: ingress-nginx/kube-webhook-certgen # ingress-nginx/kube-webhook-certgen tag: v1.1.1 # v20220916-gd32f8c343 pullPolicy: IfNotPresent nodeSelector: kubernetes.io/os: linux securityContext: runAsNonRoot: true runAsUser: 2000 fsGroup: 2000 defaultBackend: enabled: false # 关闭defaultBackend
value配置文件注解如下:
将ingress-controller部署到指定的命名空间${namespace}
helm install ingress-nginx-controller -f myvalue.yaml ./ingress-nginx-a -n ${namespace}
查看controller实例是否部署成功
ingressclasses资源
查看访问入口ELB地址
创建测试工作负载、service和ingress资源
访问测试
|