一文读懂Ingress-Nginx以及实践攻略

目录

• 1 概念
  • 1.1 什么是Ingress？
    • 1.1.1 主要功能：
  • 1.2 Ingress的组件
  • 1.3 什么是ingress-nginx
  • 1.4 ingress-nginx优点和限制
  • 1.5 版本兼容性矩阵
• 2 实践： Ingress nginx部署
  • 2.1 使用helm部署ingress-nginx
    • 2.1.1 安装和配置Helm
    • 2.1.2 配置和创建Ingress-Nginx
  • 2.2 使用yaml文件部署ingress-nginx
  • 2.3 部署后查看ingress状态
  • 2.4 创建实例测试 Ingress
    • 2.4.1 部署ingress对象
    • 2.4.2 访问测试
      • 2.4.2.1 在HOSTS文件里添加解析记录：
    • 2.4.3 pod负载均衡测试
• 3 DaemonSet方式部署Ingress-nginx
  • 3.1 配置DaemonSet
    • 3.1.1 helm部署方式
    • 3.1.2 yaml文件部署方式
• 4 结论
• 5 参考

摘要: Ingress-Nginx 是 Kubernetes 中用于管理 HTTP 和 HTTPS 流量的强大工具。通过配置 Ingress 资源，用户可以灵活地路由流量到后端服务，增强了集群的可访问性和可扩展性。本文将深入探讨 Ingress-Nginx 的工作原理、基本配置及最佳实践，并通过实际案例来验证其功能。

1 概念

1.1 什么是Ingress？

Ingress是Kubernetes的 一种 API 对象，用于管理集群内服务的外部访问。Ingress 可以提供从集群外部到集群内Service的 HTTP 和 HTTPS 路由，并可以基于域名、路径等规则将请求转发到集群内的Service上。

下面是 Ingress 的一个简单示例，可将外部的流量导流到K8S集群的同一 Service， 再由Service将流量发送到后端的Pod：

1.1.1 主要功能：

• HTTP/HTTPS 路由: Ingress 允许外部 HTTP 和 HTTPS 请求根据预定义的路由规则访问集群内部的服务。路由规则可以基于主机名、URL 路径等进行配置。
• 反向代理: Ingress 控制器通常会充当反向代理，将外部请求路由到集群内部的服务。Ingress 不会公开任意端口或协议，通常使用Service.Type=NodePort或Service.Type=LoadBalancer类型的服务。
• 域名支持: 通过 Ingress，可以为集群中的服务配置域名，从而提供更友好的访问方式。例如，可以将 foo.example.com 指向集群内的 foo-service 服务。
• 负载均衡: Ingress 控制器可以将流量分配给多个服务实例，实现负载均衡，从而提高服务的可用性和性能。在云环境中，Ingress 通常会与云提供商的负载均衡器集成，以确保流量能够从外部访问集群。
• TLS/SSL 支持: Ingress 支持 HTTPS 协议，允许为服务配置 TLS 证书，以确保传输的安全性。TLS 证书可以使用 Kubernetes 的 Secret 对象进行管理。
• 虚拟主机: Ingress 可以配置虚拟主机，使得多个服务可以通过不同的域名或子域名进行访问。

1.2 Ingress的组件

• Ingress 控制器: 为了使 Ingress 资源正常工作，集群必须运行一个 Ingress 控制器。类似Kubernetes集群的kube-apiserver服务，负责管理和执行 Ingress 资源定义的路由规则。常见的 Ingress 控制器包括 NGINX Ingress Controller、Traefik、HAProxy 等，还有公有云厂商的Ingress 控制器包括Azure的AKS Application Gateway、阿里云的Alibaba Cloud MSE Ingress。
• Ingress 对象: 用户自定义的 Kubernetes 对象，描述了外部流量访问集群内Service的规则集。

1.3 什么是ingress-nginx

Ingress nginx是 Kubernetes 中最常用的 Ingress 控制器之一，由 NGINX 官方提供支持（备注：nginx和k8s官方各自维护了一套 nginx ingress controller）。

NGINX Ingress Controller 是 NGINX 和 NGINX Plus 的Ingress Controller实现，可以对Websocket、gRPC、TCP 和 UDP 应用程序进行负载平衡。它支持标准Ingress功能，例如基于内容的路由和 TLS/SSL 终止。

1.4 ingress-nginx优点和限制

优点

限制

经过市场广泛使用，社区支持活跃。

需要对 NGINX 配置有一定的理解，以便充分利用其功能。

与 NGINX 的成熟生态系统兼容，提供稳定和高性能的 HTTP/HTTPS 处理。

在非常复杂的路由场景中，配置可能变得繁琐。

丰富的配置选项和灵活的路由机制。

1.5 版本兼容性矩阵

ingress-nginx 项目支持的版本意味着官方已经完成了 E2E 测试，并且它们正在通过列出的版本。 Ingress-Nginx 版本可能适用于旧版本，但该项目不做出这种保证。

支持

Ingress-NGINX版本

k8s支持版本

Alpine 版本

Nginx版本

Helm Chart 版本

??

v1.11.2

1.30,1.29, 1.28, 1.27, 1.26

3.20.0

1.25.5

4.11.2

??

v1.11.1

1.30,1.29, 1.28, 1.27, 1.26

3.20.0

1.25.5

4.11.1

??

v1.11.0

1.30,1.29, 1.28, 1.27, 1.26

3.20.0

1.25.5

4.11.0

??

v1.10.4

1.30,1.29, 1.28, 1.27, 1.26

3.20.0

1.25.5

4.10.4

??

v1.10.3

1.30,1.29, 1.28, 1.27, 1.26

3.20.0

1.25.5

4.10.3

??

v1.10.2

1.30,1.29, 1.28, 1.27, 1.26

3.20.0

1.25.5

4.10.2

??

v1.10.1

1.30,1.29, 1.28, 1.27, 1.26

3.19.1

1.25.3

4.10.1

??

v1.10.0

1.29, 1.28, 1.27, 1.26

3.19.1

1.25.3

4.10.0

2 实践： Ingress nginx部署

本文k8s环境是v1.29.7, ingress-nginx是v1.11.1

注意：提前将ingress-nginx的镜像下载并上传到私有仓库

[root@k8s-master1 ingress-nginx]# grep -n -r image: ingress-nginx.yaml
447:        image:  registry.k8s.io/ingress-nginx/controller:v1.11.1 @sha256:e6439a12b52076965928e83b7b56aae6731231677b01e81818bce7fa5c60161a
548:        image:  registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.1 @sha256:36d05b4077fb8e3d13663702fa337f124675ba8667cbd949c03a8e8ea6fa4366

2.1 使用helm部署ingress-nginx

官方推荐helm方式部署，如果环境没有helm需要先下载： helm下载地址

2.1.1 安装和配置Helm

安装helm

tar -zxvf helm-v3.15.3-linux-amd64.tar.gz --strip-components 1 -C /usr/local/bin linux-amd64/helm

添加ingress-nginx的repo

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx

更新repo信息

helm repo update

搜索ingress-nginx的Chart

[root@k8s-master1 ingress-nginx]# helm search repo ingress-nginx --version 4.11.1
NAME                            CHART VERSION   APP VERSION     DESCRIPTION                      
ingress-nginx/ingress-nginx     4.11.1          1.11.1          Ingress controller for Kubernetes using NGINX a...

说明： --version 4.11.1： 指定Chart版本搜索

下载ingress-nginx的Chart

mkdir helm && cd helm
helm pull ingress-nginx/ingress-nginx --version 4.11.1

解压Charts

# 解压Chart
tar zxvf ingress-nginx-4.11.1.tgz

# 进入Chart目录
cd ingress-nginx

2.1.2 配置和创建Ingress-Nginx

调整Values.yaml的参数

• 镜像仓库和镜像

  21 image:
  22 ## Keep false as default for now!
  23 chroot: false
  24 #注释原仓库地址
  25 #registry: registry.k8s.io
  26 #替换成阿里云仓库地址
  27 registry: registry.cn-hangzhou.aliyuncs.com
  28 image: ingress-nginx-steven/controller
  32 tag: “v1.11.1”
  33 # 注释sha256校验码
  34 #digest: sha256:e6439a12b52076965928e83b7b56aae6731231677b01e81818bce7fa5c60161a

  ---

  805 image:
  806 # 注释原仓库地址
  807 #registry: registry.k8s.io
  808 registry: registry.cn-hangzhou.aliyuncs.com
  809 image: ingress-nginx-steven/kube-webhook-certgen
  813 tag: v1.4.1
  814 # 注释sha256校验码
  815 #digest: sha256:36d05b4077fb8e3d13663702fa337f124675ba8667cbd949c03a8e8ea6fa4366

• 使用本地网络和dns

  启用主机网络

  103 hostNetwork: true
  …

  启用主机dns服务

  78 dnsPolicy: ClusterFirstWithHostNet

• 修改Ingress模式和指定port

  默认是LoadBalancer

  484 #type: LoadBalancer
  485 type: NodePort
  …
  540 appProtocol: true
  541 nodePorts:
  542 # – Node port allocated for the external HTTP listener. If left empty, the service co ntroller allocates one from the configured node port range.
  543 http: “30080”
  544 # – Node port allocated for the external HTTPS listener. If left empty, the service c ontroller allocates one from the configured node port range.
  545 https: “30443”

创建前， 执行以下命令检查配置是否修改成功：

helm install ingress-nginx . --dry-run=client --namespace=ingress-nginx

创建ingress-nignx

helm install ingress-nginx . --namespace=ingress-nginx --create-namespace

查看是否创建成功

[root@k8s-master1 ingress-nginx]# helm list --namespace ingress-nginx
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS  CHART                    APP VERSION
ingress-nginx   ingress-nginx   1               2024-08-27 21:44:34.137344711 +0800 CST deployedingress-nginx-4.11.1     1.11.1

卸载ingress-nginx

[root@k8s-master1 ingress-nginx]# helm uninstall ingress-nginx -n ingress-nginx
release "ingress-nginx" uninstalled

2.2 使用yaml文件部署ingress-nginx

1. 参考官方yaml文件

2. 也可以参考以下配置

   apiVersion: v1
   kind: Namespace
   metadata:
   labels:
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   name: ingress-nginx

   apiVersion: v1
   automountServiceAccountToken: true
   kind: ServiceAccount
   metadata:
   labels:
   app.kubernetes.io/component: controller
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx
   namespace: ingress-nginx

   apiVersion: v1
   automountServiceAccountToken: true
   kind: ServiceAccount
   metadata:
   labels:
   app.kubernetes.io/component: admission-webhook
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx-admission
   namespace: ingress-nginx

   apiVersion: rbac.authorization.k8s.io/v1
   kind: Role
   metadata:
   labels:
   app.kubernetes.io/component: controller
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx
   namespace: ingress-nginx
   rules:

   • apiGroups:
     • “”
       resources:
     • namespaces
       verbs:
     • get
   • apiGroups:
     • “”
       resources:
     • configmaps
     • pods
     • secrets
     • endpoints
       verbs:
     • get
     • list
     • watch
   • apiGroups:
     • “”
       resources:
     • services
       verbs:
     • get
     • list
     • watch
   • apiGroups:
     • networking.k8s.io
       resources:
     • ingresses
       verbs:
     • get
     • list
     • watch
   • apiGroups:
     • networking.k8s.io
       resources:
     • ingresses/status
       verbs:
     • update
   • apiGroups:
     • networking.k8s.io
       resources:
     • ingressclasses
       verbs:
     • get
     • list
     • watch
   • apiGroups:
     • coordination.k8s.io
       resourceNames:
     • ingress-nginx-leader
       resources:
     • leases
       verbs:
     • get
     • update
   • apiGroups:
     • coordination.k8s.io
       resources:
     • leases
       verbs:
     • create
   • apiGroups:
     • “”
       resources:
     • events
       verbs:
     • create
     • patch
   • apiGroups:
     • discovery.k8s.io
       resources:
     • endpointslices
       verbs:
     • list
     • watch
     • get

   ---

   apiVersion: rbac.authorization.k8s.io/v1
   kind: Role
   metadata:
   labels:
   app.kubernetes.io/component: admission-webhook
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx-admission
   namespace: ingress-nginx
   rules:

   • apiGroups:
     • “”
       resources:
     • secrets
       verbs:
     • get
     • create

   ---

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRole
   metadata:
   labels:
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx
   rules:

   • apiGroups:
     • “”
       resources:
     • configmaps
     • endpoints
     • nodes
     • pods
     • secrets
     • namespaces
       verbs:
     • list
     • watch
   • apiGroups:
     • coordination.k8s.io
       resources:
     • leases
       verbs:
     • list
     • watch
   • apiGroups:
     • “”
       resources:
     • nodes
       verbs:
     • get
   • apiGroups:
     • “”
       resources:
     • services
       verbs:
     • get
     • list
     • watch
   • apiGroups:
     • networking.k8s.io
       resources:
     • ingresses
       verbs:
     • get
     • list
     • watch
   • apiGroups:
     • “”
       resources:
     • events
       verbs:
     • create
     • patch
   • apiGroups:
     • networking.k8s.io
       resources:
     • ingresses/status
       verbs:
     • update
   • apiGroups:
     • networking.k8s.io
       resources:
     • ingressclasses
       verbs:
     • get
     • list
     • watch
   • apiGroups:
     • discovery.k8s.io
       resources:
     • endpointslices
       verbs:
     • list
     • watch
     • get

   ---

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRole
   metadata:
   labels:
   app.kubernetes.io/component: admission-webhook
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx-admission
   rules:

   • apiGroups:
     • admissionregistration.k8s.io
       resources:
     • validatingwebhookconfigurations
       verbs:
     • get
     • update

   ---

   apiVersion: rbac.authorization.k8s.io/v1
   kind: RoleBinding
   metadata:
   labels:
   app.kubernetes.io/component: controller
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx
   namespace: ingress-nginx
   roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: Role
   name: ingress-nginx
   subjects:

   • kind: ServiceAccount
     name: ingress-nginx
     namespace: ingress-nginx

   ---

   apiVersion: rbac.authorization.k8s.io/v1
   kind: RoleBinding
   metadata:
   labels:
   app.kubernetes.io/component: admission-webhook
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx-admission
   namespace: ingress-nginx
   roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: Role
   name: ingress-nginx-admission
   subjects:

   • kind: ServiceAccount
     name: ingress-nginx-admission
     namespace: ingress-nginx

   ---

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRoleBinding
   metadata:
   labels:
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx
   roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
   name: ingress-nginx
   subjects:

   • kind: ServiceAccount
     name: ingress-nginx
     namespace: ingress-nginx

   ---

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRoleBinding
   metadata:
   labels:
   app.kubernetes.io/component: admission-webhook
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx-admission
   roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
   name: ingress-nginx-admission
   subjects:

   • kind: ServiceAccount
     name: ingress-nginx-admission
     namespace: ingress-nginx

   ---

   apiVersion: v1
   data:
   allow-snippet-annotations: “false”
   kind: ConfigMap
   metadata:
   labels:
   app.kubernetes.io/component: controller
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx-controller
   namespace: ingress-nginx

   apiVersion: v1
   kind: Service
   metadata:
   labels:
   app.kubernetes.io/component: controller
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx-controller
   namespace: ingress-nginx
   spec:
   externalTrafficPolicy: Local
   ipFamilies:

   • IPv4
     ipFamilyPolicy: SingleStack
     ports:
   • appProtocol: http
     name: http
     port: 80
     protocol: TCP
     targetPort: http
   • appProtocol: https
     name: https
     port: 443
     protocol: TCP
     targetPort: https
     selector:
     app.kubernetes.io/component: controller
     app.kubernetes.io/instance: ingress-nginx
     app.kubernetes.io/name: ingress-nginx
     type: LoadBalancer

   ---

   apiVersion: v1
   kind: Service
   metadata:
   labels:
   app.kubernetes.io/component: controller
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx-controller-admission
   namespace: ingress-nginx
   spec:
   ports:

   • appProtocol: https
     name: https-webhook
     port: 443
     targetPort: webhook
     selector:
     app.kubernetes.io/component: controller
     app.kubernetes.io/instance: ingress-nginx
     app.kubernetes.io/name: ingress-nginx
     type: ClusterIP

   ---

   apiVersion: apps/v1
   kind: Deployment
   metadata:
   labels:
   app.kubernetes.io/component: controller
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx-controller
   namespace: ingress-nginx
   spec:
   minReadySeconds: 0
   revisionHistoryLimit: 10
   selector:
   matchLabels:
   app.kubernetes.io/component: controller
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   strategy:
   rollingUpdate:
   maxUnavailable: 1
   type: RollingUpdate
   template:
   metadata:
   labels:
   app.kubernetes.io/component: controller
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   spec:
   containers:
   - args:
   - /nginx-ingress-controller
   - --publish-service=$(PO{D}_{N}AMESPACE)/ingress-nginx-controller---election-id=ingress-nginx-leader---controller-class=k8s.io/ingress-nginx---ingress-class=nginx---configmap=$(POD_NAMESPACE)/ingress-nginx-controller
   - --validating-webhook=:8443
   - --validating-webhook-certificate=/usr/local/certificates/cert
   - --validating-webhook-key=/usr/local/certificates/key
   - --enable-metrics=false
   env:
   - name: POD_NAME
   valueFrom:
   fieldRef:
   fieldPath: metadata.name
   - name: POD_NAMESPACE
   valueFrom:
   fieldRef:
   fieldPath: metadata.namespace
   - name: LD_PRELOAD
   value: /usr/local/lib/libmimalloc.so
   image: registry.cn-hangzhou.aliyuncs.com/ingress-nginx-steven/controller:v1.11.1
   imagePullPolicy: IfNotPresent
   lifecycle:
   preStop:
   exec:
   command:
   - /wait-shutdown
   livenessProbe:
   failureThreshold: 5
   httpGet:
   path: /healthz
   port: 10254
   scheme: HTTP
   initialDelaySeconds: 10
   periodSeconds: 10
   successThreshold: 1
   timeoutSeconds: 1
   name: controller
   ports:
   - containerPort: 80
   name: http
   protocol: TCP
   - containerPort: 443
   name: https
   protocol: TCP
   - containerPort: 8443
   name: webhook
   protocol: TCP
   readinessProbe:
   failureThreshold: 3
   httpGet:
   path: /healthz
   port: 10254
   scheme: HTTP
   initialDelaySeconds: 10
   periodSeconds: 10
   successThreshold: 1
   timeoutSeconds: 1
   resources:
   requests:
   cpu: 100m
   memory: 90Mi
   securityContext:
   allowPrivilegeEscalation: false
   capabilities:
   add:
   - NET_BIND_SERVICE
   drop:
   - ALL
   readOnlyRootFilesystem: false
   runAsNonRoot: true
   runAsUser: 101
   seccompProfile:
   type: RuntimeDefault
   volumeMounts:
   - mountPath: /usr/local/certificates/
   name: webhook-cert
   readOnly: true
   dnsPolicy: ClusterFirst
   nodeSelector:
   kubernetes.io/os: linux
   serviceAccountName: ingress-nginx
   terminationGracePeriodSeconds: 300
   volumes:
   - name: webhook-cert
   secret:
   secretName: ingress-nginx-admission

   apiVersion: batch/v1
   kind: Job
   metadata:
   labels:
   app.kubernetes.io/component: admission-webhook
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx-admission-create
   namespace: ingress-nginx
   spec:
   template:
   metadata:
   labels:
   app.kubernetes.io/component: admission-webhook
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx-admission-create
   spec:
   containers:
   - args:
   - create
   - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(PO{D}_{N}AMESPACE).svc---namespace=$(POD_NAMESPACE)
   - --secret-name=ingress-nginx-admission
   env:
   - name: POD_NAMESPACE
   valueFrom:
   fieldRef:
   fieldPath: metadata.namespace
   image: registry.cn-hangzhou.aliyuncs.com/ingress-nginx-steven/kube-webhook-certgen:v1.4.1
   imagePullPolicy: IfNotPresent
   name: create
   securityContext:
   allowPrivilegeEscalation: false
   capabilities:
   drop:
   - ALL
   readOnlyRootFilesystem: true
   runAsNonRoot: true
   runAsUser: 65532
   seccompProfile:
   type: RuntimeDefault
   nodeSelector:
   kubernetes.io/os: linux
   restartPolicy: OnFailure
   serviceAccountName: ingress-nginx-admission

   apiVersion: batch/v1
   kind: Job
   metadata:
   labels:
   app.kubernetes.io/component: admission-webhook
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx-admission-patch
   namespace: ingress-nginx
   spec:
   template:
   metadata:
   labels:
   app.kubernetes.io/component: admission-webhook
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx-admission-patch
   spec:
   containers:
   - args:
   - patch
   - --webhook-name=ingress-nginx-admission
   - --namespace=$(POD_NAMESPACE)
   - --patch-mutating=false
   - --secret-name=ingress-nginx-admission
   - --patch-failure-policy=Fail
   env:
   - name: POD_NAMESPACE
   valueFrom:
   fieldRef:
   fieldPath: metadata.namespace
   image: registry.cn-hangzhou.aliyuncs.com/ingress-nginx-steven/kube-webhook-certgen:v1.4.1
   imagePullPolicy: IfNotPresent
   name: patch
   securityContext:
   allowPrivilegeEscalation: false
   capabilities:
   drop:
   - ALL
   readOnlyRootFilesystem: true
   runAsNonRoot: true
   runAsUser: 65532
   seccompProfile:
   type: RuntimeDefault
   nodeSelector:
   kubernetes.io/os: linux
   restartPolicy: OnFailure
   serviceAccountName: ingress-nginx-admission

   apiVersion: networking.k8s.io/v1
   kind: IngressClass
   metadata:
   labels:
   app.kubernetes.io/component: controller
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: nginx
   spec:
   controller: k8s.io/ingress-nginx

   apiVersion: admissionregistration.k8s.io/v1
   kind: ValidatingWebhookConfiguration
   metadata:
   labels:
   app.kubernetes.io/component: admission-webhook
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/part-of: ingress-nginx
   app.kubernetes.io/version: 1.11.1
   name: ingress-nginx-admission
   webhooks:

   • admissionReviewVersions:
     • v1
       clientConfig:
       service:
       name: ingress-nginx-controller-admission
       namespace: ingress-nginx
       path: /networking/v1/ingresses
       failurePolicy: Fail
       matchPolicy: Equivalent
       name: validate.nginx.ingress.kubernetes.io
       rules:
     • apiGroups:
       • networking.k8s.io
         apiVersions:
       • v1
         operations:
       • CREATE
       • UPDATE
         resources:
       • ingresses
         sideEffects: None
         官方yaml文件

• 根据官方yaml，修改镜像地址：

  [root@k8s-master1 ingress-nginx]# grep -n image: ingress-nginx.yaml
  448: image: harbor.zx/hcie/controller:v1.11.1
  549: image: harbor.zx/hcie/kube-webhook-certgen:v1.4.1
  602: image: harbor.zx/hcie/kube-webhook-certgen:v1.4.1

• 修改ingress的端口：

  352 ports:
  353 - appProtocol: http
  354 name: http
  355 port: 80
  356 protocol: TCP
  357 targetPort: http
  358 nodePort: 30080
  359 - appProtocol: https
  360 name: https
  361 port: 443
  362 protocol: TCP
  363 targetPort: https
  364 nodePort: 30443

• 修改ingress的代理方式：

  369 #type: LoadBalancer # 默认是负载均衡，如果是云环境建议使用默认参数
  370 type: NodePort

部署ingress-nginx：

kubectl apply -f ingress-nginx.yaml

部署后输出结果：

namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created

2.3 部署后查看ingress状态

部署后，执行以下命令：

kubectl get svc,pod -n ingress-nginx -owide

查看Pod和SVC的状态：

NAME                                         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE   SELECTOR
service/ingress-nginx-controller              NodePort     10.245.187.253   <none>         80:30080/TCP,443:30443/TCP    18m   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
service/ingress-nginx-controller-admission   ClusterIP   10.245.245.14    <none>        443/TCP                      18m   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx

NAME                                            READY   STATUS      RESTARTS   AGE   IP              NODE          NOMINATED NODE   READINESS GATES
pod/ingress-nginx-admission-create-h6s42        0/1     Completed   0          18m   172.16.126.2    k8s-worker2   <none>           <none>
pod/ingress-nginx-admission-patch-79kv5         0/1     Completed   0          18m   172.16.126.3    k8s-worker2   <none>           <none>
pod/ingress-nginx-controller-746fcdfcdd-6gt4n   1/1     Running     0          18m   172.16.194.85    k8s-worker1    <none>           <none>

说明：

• NodePort: Service的类型是在ingress-nginx.yaml中自定义。
• 80:30080/TCP, 443:30443/TCP: 这两个端口也是在ingress-nginx.yaml中自定义，后面访问使用这两个端口。
• ingress-nginx-controller的Pod为ingress-nginx-controller-746fcdfcdd-6gt4n运行在k8s-worker1节点上。后续访问ingress代理的服务，都要用k8s-worker1节点的IP做DNS解析。

执行以下命令，查看IngressClass：

[root@k8s-master1 ingress-nginx]# kubectl get IngressClass
NAME    CONTROLLER             PARAMETERS   AGE
nginx   k8s.io/ingress-nginx   <none>       25m

说明:

• Ingress类型：nginx是ingress-nginx默认创建的；
• 当创建Ingress对象时，需要指定ingressClass字段(.spec.ingressClassName)
• 可以对ingressClass添加ingressclass.kubernetes.io/is-default-class 注解，这样 IngressClass 会被视为默认的 Ingress 类。 当某个 IngressClass 资源将此注解设置为 true 时， 没有指定类的新 Ingress 资源将被分配到此默认类。

2.4 创建实例测试 Ingress

2.4.1 部署ingress对象

最后，我们打算使用1个3副本的deployment组成nginx-service，然后创建一个ingress，用于代理nginx的服务。yaml配置如下：

# 部署3个nginx实例
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-test
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: registry.cn-hangzhou.aliyuncs.com/hcie/nginx:1.26.1-alpine
        ports:
        - containerPort: 80
---
# 部署nginx的Service
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx-pod
  type: ClusterIP
  ports:
  - port: 80
    targetPort: 80
---
# 创建ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-http
spec:
  # 上面的IngressClass名称
  ingressClassName: "nginx"
  rules:
  # 域名，稍后要通过这个域名访问服务
  - host: nginx.steven.com
    http:
      paths:
      # 路径，访问域名时后面添加的路径
      - path: /
        pathType: Prefix
        backend:
          # 要代理的服务和服务的端口
          service:
            name: nginx-service
            port: 
              number: 80

部署deployment、service、ingress

kubectl apply -f ingress-test.yaml

执行以下命令，查看部署结果：

[root@k8s-master1 ingress-nginx]# kubectl get deployment,svc,ingress
NAME                         READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx-test   3/3     3            3           19s

NAME                    TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
service/nginx-service   ClusterIP   10.245.127.127   <none>        80/TCP    19s

NAME                                     CLASS   HOSTS            ADDRESS          PORTS   AGE
ingress.networking.k8s.io/ingress-http   nginx   nginx.test.com   10.245.187.253   80      19s

查看下ingress-http详细信息：

[root@k8s-master1 ingress-nginx]# kubectl describe ingress ingress-http
Name:             ingress-http
Labels:           <none>
Namespace:        default
Address:          10.245.187.253
Ingress Class:    nginx
Default backend:  <default>
Rules:
  Host            Path  Backends
  ----            ----  --------
  nginx.test.com
                  /   nginx-service:80 (<none>)
Annotations:      <none>
Events:
  Type    Reason  Age                    From                      Message
  ----    ------  ----                   ----                      -------
  Normal  Sync    2m49s (x2 over 2m56s)  nginx-ingress-controller  Scheduled for sync

查看ingress-controller的日志信息：

[root@k8s-master1 ingress-nginx]# kubectl -n ingress-nginx logs -f ingress-nginx-controller-746fcdfcdd-6gt4n
-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       v1.11.1
  Build:         7c44f992012555ff7f4e47c08d7c542ca9b4b1f7
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.25.5

-------------------------------------------------------------------------------

W0826 03:58:30.736251       7 client_config.go:659] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0826 03:58:30.736660       7 main.go:205] "Creating API client" host="https://10.245.0.1:443"
I0826 03:58:37.887271       7 main.go:248] "Running in Kubernetes cluster" major="1" minor="29" git="v1.29.7" state="clean" commit="4e4a18878ce330fefda1dc46acca88ba355e9ce7" platform="linux/amd64"
I0826 03:58:38.432286       7 main.go:101] "SSL fake certificate created" file="/etc/ingress-controller/ssl/default-fake-certificate.pem"
I0826 03:58:38.490727       7 ssl.go:535] "loading tls certificate" path="/usr/local/certificates/cert" key="/usr/local/certificates/key"
I0826 03:58:38.516493       7 nginx.go:271] "Starting NGINX Ingress controller"
I0826 03:58:38.538981       7 event.go:377] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"ingress-nginx-controller", UID:"0dba2870-a96b-4b5e-a908-b985139e5d63", APIVersion:"v1", ResourceVersion:"624593", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/ingress-nginx-controller
I0826 03:58:39.719275       7 nginx.go:317] "Starting NGINX process"
I0826 03:58:39.720802       7 leaderelection.go:250] attempting to acquire leader lease ingress-nginx/ingress-nginx-leader...
I0826 03:58:39.723329       7 nginx.go:337] "Starting validation webhook" address=":8443" certPath="/usr/local/certificates/cert" keyPath="/usr/local/certificates/key"
I0826 03:58:39.724636       7 controller.go:193] "Configuration changes detected, backend reload required"
I0826 03:58:39.748482       7 leaderelection.go:260] successfully acquired lease ingress-nginx/ingress-nginx-leader
I0826 03:58:39.748953       7 status.go:85] "New leader elected" identity="ingress-nginx-controller-746fcdfcdd-6gt4n"
 I0826 03:58:39.826280       7 controller.go:213] "Backend successfully reloaded"
I0826 03:58:39.826390       7 controller.go:224] "Initial sync, sleeping for 1 second" 
I0826 03:58:39.826978       7 event.go:377] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-746fcdfcdd-6gt4n", UID:"68964da5-0da2-4ca7-84f3-742e3d1646ed", APIVersion:"v1", ResourceVersion:"624720", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration

从最后几条日志来看，我们创建的ingress已经与ingress-nginx成功关联上了，下面可以进行访问测试了。

2.4.2 访问测试

2.4.2.1 在HOSTS文件里添加解析记录：

推荐： 如果安装了火绒，可以使用内置安全工具，方便修改。

添加记录：

192.168.3.44 nginx.steven.com

访问测试：

2.4.3 pod负载均衡测试

修改nginx的index.html

[root@k8s-master1 ingress-nginx]# kubectl exec -it nginx-test-6fc95f9f89-dsqkg -- sh
/ # echo "nginx1"> /usr/share/nginx/html/index.html
/ # exit
[root@k8s-master1 ingress-nginx]# kubectl exec -it nginx-test-6fc95f9f89-h7khh -- sh
/ # echo "nginx2"> /usr/share/nginx/html/index.html
/ # exit
[root@k8s-master1 ingress-nginx]# kubectl exec -it nginx-test-6fc95f9f89-z2v6l -- sh
/ # echo "nginx3"> /usr/share/nginx/html/index.html
/ # exit

再次访问测试，默认是轮询方式，负载均衡主要是service的作用。

3 DaemonSet方式部署Ingress-nginx

DaemonSet 确保 Ingress Controller 的副本在集群中的每个节点上运行。这么做的原因如下：

1. 它确保入口控制器保持服务流量高可用，即使某个节点发生故障。
2. 它允许 Ingress Controller 在集群中的所有节点之间均匀分配流量，从而提高性能。
3. 它允许 Ingress Controller 绑定到主机的网络命名空间，这对于某些功能（例如外部 DNS）是必需的。

3.1 配置DaemonSet

3.1.1 helm部署方式

# 切换到Chart目录
cd helm/ingress-nginx
vim values.yaml

• 修改ingress-nginx-controller的kind字段

  222 #kind: Deployment
  223 kind: DaemonSet

更新helm配置

helm upgrade ingress-nginx . -n ingress-nginx

查看部署结果

[root@k8s-master1 ingress-nginx]# kubectl get pod,svc -owide -n ingress-nginx
NAME                                 READY   STATUS    RESTARTS   AGE    IP          NODE          NOMINATED NODE   READINESS GATES
pod/ingress-nginx-controller-2bljr   1/1     Running   0          61s    10.10.0.1   k8s-master1   <none>           <none>
pod/ingress-nginx-controller-2s59q   1/1     Running   0          3m1s   10.10.0.4   k8s-worker1   <none>           <none>
pod/ingress-nginx-controller-dppwt   1/1     Running   0          61s    10.10.0.5   k8s-worker2   <none>           <none>
pod/ingress-nginx-controller-gzshs   1/1     Running   0          61s    10.10.0.3   k8s-master3   <none>           <none>
pod/ingress-nginx-controller-hbj9d   0/1     Running   0          61s    10.10.0.2   k8s-master2   <none>           <none>

NAME                                         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE    SELECTOR
service/ingress-nginx-controller             NodePort    10.245.133.125   <none>        80:30080/TCP,443:30443/TCP   3m2s   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
service/ingress-nginx-controller-admission   ClusterIP   10.245.236.128   <none>        443/TCP                      3m2s   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx

• 看到每个节点都部署了一个ingress-controller的pod

3.1.2 yaml文件部署方式

修改这行即可

396 #kind: Deployment
397 kind: DaemonSet

• 验证方法如4.2节， 这里就不重复叙述了。

4 结论

Ingress-Nginx 是 Kubernetes 中强大而灵活的流量管理工具，能够有效地路由外部请求到内部服务。通过本文的讲解与实践示例，您应该能够快速上手并在自己的集群中实现 Ingress-Nginx 的配置与管理。继续探索 Ingress-Nginx 的更多高级功能，将使您的应用架构更加高效和安全。

5 参考

【1】Kubernetes官方文档-Ingress

【2】Kubernetes官方文档-IngressController

【3】github-ingress-nginx

【4】what-to-deploy-nginx-ingress-controller-as-replicas-set-or-daemon-set