Kubernetes技术详解-从理论到实践-(11)-网关-Ingress
·
1 Ingress网关简介
前面的Service一章我们讲了服务暴露的概念，可以从集群外访问集群内部。NodePort和LoadBalancer服务都可以对外暴露端口，但在实际使用中这两者都有一些问题。
它们都只能暴露IP+端口，无法暴露域名或URL，而端口是集群的稀缺资源，随着服务越来越多，端口最终会被耗尽，并且LB还需要外部负载均衡器的支持。
所以Kubernetes又引入了Ingress资源对象的概念，它是一种“集群入口”资源，用来把集群外的HTTP/HTTPS流量按照域名+URL路径的规则分发到集群内的不同Service，相当于七层负载均衡器+域名路由表。在Gateway API出现之前，Ingress在K8s官方称为Ingress Gateway或者七层入口。为了与Gateway API这种支持多路由、多监听器、多命名空间、有权重、有灰度、有插件链的大而全网关相比，我们可以称Ingress这种仅支持HTTP/HTTPS及域名+URL路由的网关称为七层小网关。
2 Ingress控制器
2.1 Ingress控制器类型
如果想要实现Ingress，需要Ingress控制器(Ingress Controller，K8s文档标准叫法)，泛指任何实现Ingress API的反向代理程序，常见的Ingress控制器有以下几类：
- Nginx系：ingress-nginx、NGINX Gateway Fabric
- Envoy系：Envoy Gateway、Contour和Istio Ingress Gateway
- 国产云网关：Higress阿里开源、MSE Ingress阿里商业、APISIX Ingress
- 轻量级：Traefik(k3s默认)、Caddy Ingress、HAProxy Ingress
- 云厂商：AWS ALB Ingress Controller、Azure Application Gateway Ingress、Aliyun ALB Ingress
我们可以看到，Traefik是K3s默认的，由于我们平时学习开发过程中使用到Ingress场景不多，因此在安装的时候默认关闭了，在本章后面我们会修改k3s.service文件将其打开并演示效果。
下面是k3s.service，可以看到我们把Traefik关掉了，删掉–disable traefik重启k3s服务就可以重新启动Traefik控制器。
另：metrics-server为定时采集各节点、Pod的CPU和内存使用率的工具，关闭之后top/HPA/VPA全部失效，开发测试环境可以关闭，如果是生产环境或需要自动扩缩容场景，则必须启用。
[root@master 11-ingress]# cat /etc/systemd/system/k3s.service
[Unit]
Description=Lightweight Kubernetes
（...部分内容略...）
ExecStart=/usr/local/bin/k3s \
server \
'--docker' \
'--disable' \
'traefik' \
'--disable' \
'metrics-server' \
'--service-node-port-range=0-39999' \
（...部分内容略...）
2.2 部署ingress-nginx控制器
由于Nginx系的ingress-nginx使用较为常见，我们下面就部署这种控制器并进行Ingress演示。
安装镜像和资源清单文件可以从本文配套的资源包中找到。
配套资料包下载地址：https://download.csdn.net/download/field1003/91965690
2.2.1 安装镜像
ingress-nginx镜像需要两个，一个是controller，一个是kube-webhook-certgen，前面的controller负责从外部接收流量、按照Ingress规则转发、热加载nginx.conf文件，后面的kube-webhook-certgen为一次性任务，只在升级安装时用到，负责TLS证书相关。这里需要注意下载的镜像版本要与k3s匹配，否则容易出现兼容性问题。
本专栏配套的资源包有离线镜像文件，可以直接导入。
[root@master 11-ingress]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.k8s.io/ingress-nginx/controller v1.12.0 a4a8af0db089 7 months ago 302MB
registry.k8s.io/ingress-nginx/kube-webhook-certgen v1.5.0 6595b1857d40 7 months ago 66.1MB
2.2.2 下载资源清单文件
ingress-nginx的配置文件可以从官方下载：wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.10.0/deploy/static/provider/cloud/deploy.yaml
如果这个文件中的镜像由于某种原因无法下载，则可以选择替换成国内源，或者下载本专栏配套的资源包用docker load -i xxx离线导入。
修改并重命名后的资源清单文件内容如下（这个文件内容很长，我们会在本章最后一节ingress-nginx实现原理中再介绍）：
[root@master 11-ingress]# cat nginx-ingress-base-deploy.yaml
apiVersion: v1
kind: Namespace
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- endpoints
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
- apiGroups:
- coordination.k8s.io
resourceNames:
- ingress-nginx-leader
resources:
- leases
verbs:
- get
- update
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- create
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- discovery.k8s.io
resources:
- endpointslices
verbs:
- list
- watch
- get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx-admission
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
- namespaces
verbs:
- list
- watch
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
- apiGroups:
- discovery.k8s.io
resources:
- endpointslices
verbs:
- list
- watch
- get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx-admission
rules:
- apiGroups:
- admissionregistration.k8s.io
resources:
- validatingwebhookconfigurations
verbs:
- get
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx-admission
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx-admission
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: v1
data:
allow-snippet-annotations: "true"
kind: ConfigMap
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx-controller
namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
externalTrafficPolicy: Local
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- appProtocol: http
name: http
port: 80
protocol: TCP
targetPort: http
- appProtocol: https
name: https
port: 443
protocol: TCP
targetPort: https
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: NodePort
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx-controller-admission
namespace: ingress-nginx
spec:
ports:
- appProtocol: https
name: https-webhook
port: 443
targetPort: webhook
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
minReadySeconds: 0
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
spec:
hostNetwork: true
containers:
- args:
- /nginx-ingress-controller
- --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
- --election-id=ingress-nginx-leader
- --controller-class=k8s.io/ingress-nginx
- --ingress-class=nginx
- --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
- --validating-webhook=:8443
- --validating-webhook-certificate=/usr/local/certificates/cert
- --validating-webhook-key=/usr/local/certificates/key
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: LD_PRELOAD
value: /usr/local/lib/libmimalloc.so
image: registry.k8s.io/ingress-nginx/controller:v1.12.0
imagePullPolicy: IfNotPresent
lifecycle:
preStop:
exec:
command:
- /wait-shutdown
livenessProbe:
failureThreshold: 5
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: controller
ports:
- containerPort: 80
name: http
protocol: TCP
- containerPort: 443
name: https
protocol: TCP
- containerPort: 8443
name: webhook
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
requests:
cpu: 100m
memory: 90Mi
securityContext:
allowPrivilegeEscalation: true
capabilities:
add:
- NET_BIND_SERVICE
drop:
- ALL
runAsUser: 101
volumeMounts:
- mountPath: /usr/local/certificates/
name: webhook-cert
readOnly: true
dnsPolicy: ClusterFirst
nodeSelector:
kubernetes.io/os: linux
serviceAccountName: ingress-nginx
terminationGracePeriodSeconds: 300
volumes:
- name: webhook-cert
secret:
secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx-admission-create
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx-admission-create
spec:
hostNetwork: true
containers:
- args:
- create
- --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
- --namespace=$(POD_NAMESPACE)
- --secret-name=ingress-nginx-admission
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.5.0
imagePullPolicy: IfNotPresent
name: create
securityContext:
allowPrivilegeEscalation: false
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx-admission-patch
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx-admission-patch
spec:
hostNetwork: true
containers:
- args:
- patch
- --webhook-name=ingress-nginx-admission
- --namespace=$(POD_NAMESPACE)
- --patch-mutating=false
- --secret-name=ingress-nginx-admission
- --patch-failure-policy=Fail
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.5.0
imagePullPolicy: IfNotPresent
name: patch
securityContext:
allowPrivilegeEscalation: false
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: nginx
spec:
controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.12.0
name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:
- v1
clientConfig:
service:
name: ingress-nginx-controller-admission
namespace: ingress-nginx
path: /networking/v1/ingresses
failurePolicy: Fail
matchPolicy: Equivalent
name: validate.nginx.ingress.kubernetes.io
rules:
- apiGroups:
- networking.k8s.io
apiVersions:
- v1
operations:
- CREATE
- UPDATE
resources:
- ingresses
sideEffects: None
2.2.3 创建ingress-nginx资源对象
[root@master 11-ingress]#
# 使用资源清单文件创建资源对象，可以看到资源对象提示created。
[root@master ~]$ kubectl apply -f nginx-ingress-base-deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
# 可以看到创建了ingress-nginx命名空间
[root@master 11-ingress]# kubectl get ns
NAME STATUS AGE
default Active 10h
ingress-nginx Active 2m
kube-node-lease Active 10h
kube-public Active 10h
kube-system Active 10h
# 查看命名空间下是否有ingress-nginx-controller Deployment，以及是否运行正常。如Deployment的READY状态为0/1请等一会再查看。
[root@master 11-ingress]# kubectl get deploy -n ingress-nginx
NAME READY UP-TO-DATE AVAILABLE AGE
ingress-nginx-controller 1/1 1 1 2m
# 查看命名空间下是否有controller的pod，以及是否运行正常。adminssion的两个pod是一次性任务，STATUS为已完成，不用关心READY字段。
[root@master 11-ingress]# kubectl get pod -n ingress-nginx
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-8j29g 0/1 Completed 0 2m
ingress-nginx-admission-patch-5n5z2 0/1 Completed 0 2m
ingress-nginx-controller-688895cf66-xkdvl 1/1 Running 0 2m
# ingress-nginx服务已经创建，NodePort对外暴露的端口为http的17238和https的18054
[root@master 11-ingress]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.43.102.21 <none> 80:17238/TCP,443:18054/TCP 2m
ingress-nginx-controller-admission ClusterIP 10.43.244.169 <none> 443/TCP 2m
[root@master 11-ingress]#
2.3 示例：Ingress域名转发
（1）编写创建两组Deployment和Service对象的资源清单文件
注意Service类型为ClusterIP。
[root@master 11-ingress]# cat nginx-deploy-svc-a.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-dm-a
labels:
app: nginx-dm-a
spec:
selector:
matchLabels:
app: nginx-dm-a
replicas: 3
template:
metadata:
labels:
app: nginx-dm-a
spec:
containers:
- name: nginx
image: nginx:1.27.3
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
volumeMounts:
- name: hostpath-volume
mountPath: /usr/share/nginx/html
volumes:
- name: hostpath-volume
hostPath:
path: /root/k8s-test/11-ingress/data/www.a.com
type: DirectoryOrCreate
---
apiVersion: v1
kind: Service
metadata:
name: nginx-svc-a
spec:
ports:
- port: 8080
targetPort: 80
protocol: TCP
selector:
app: nginx-dm-a
[root@master 11-ingress]# ls
data nginx-deploy-svc-a.yaml nginx-deploy-svc-b.yaml nginx-ingress-base-deploy.yaml nginx-ingress.yaml
[root@master 11-ingress]# cat nginx-deploy-svc-b.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-dm-b
labels:
app: nginx-dm-b
spec:
selector:
matchLabels:
app: nginx-dm-b
replicas: 3
template:
metadata:
labels:
app: nginx-dm-b
spec:
containers:
- name: nginx
image: nginx:1.27.3
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
volumeMounts:
- name: hostpath-volume
mountPath: /usr/share/nginx/html
volumes:
- name: hostpath-volume
hostPath:
path: /root/k8s-test/11-ingress/data/www.b.com
type: DirectoryOrCreate
---
apiVersion: v1
kind: Service
metadata:
name: nginx-svc-b
spec:
ports:
- port: 9090
targetPort: 80
protocol: TCP
selector:
app: nginx-dm-b
(2) 创建Deployment和Service
# 创建deploy和service之前的状态
[root@master 11-ingress]# kubectl get pod
No resources found in default namespace.
[root@master 11-ingress]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 10h
[root@master 11-ingress]# ls
data/ nginx-deploy-svc-b.yaml nginx-deploy-svc-a.yaml nginx-ingress-base-deploy.yaml
# 创建deploy和service
[root@master 11-ingress]# kubectl apply -f nginx-deploy-svc-a.yaml
deployment.apps/nginx-dm-a created
service/nginx-svc-a created
[root@master 11-ingress]# kubectl apply -f nginx-deploy-svc-b.yaml
deployment.apps/nginx-dm-b created
service/nginx-svc-b created
# pod已经创建
[root@master 11-ingress]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-dm-a-5d8f65bc-hhv87 1/1 Running 0 9s
nginx-dm-a-5d8f65bc-m7f2h 1/1 Running 0 9s
nginx-dm-a-5d8f65bc-ttbxl 1/1 Running 0 9s
nginx-dm-b-577f9bdc85-hmqx6 1/1 Running 0 4s
nginx-dm-b-577f9bdc85-mvzht 1/1 Running 0 4s
nginx-dm-b-577f9bdc85-vw6jv 1/1 Running 0 4s
# service也已经创建
[root@master 11-ingress]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 10h
nginx-svc-a ClusterIP 10.43.251.187 <none> 8080/TCP 11s
nginx-svc-b ClusterIP 10.43.177.192 <none> 9090/TCP 6s
# 在对应的挂载目录创建好index.html文件
[root@master 11-ingress]# cat data/www.a.com/index.html
<html><body>Nginx! With k8s ingress www.a.com!</body></html>
[root@master 11-ingress]# cat data/www.b.com/index.html
<html><body>Nginx! With k8s ingress www.b.com!</body></html>
# 查看Pod IP地址
[root@master 11-ingress]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-dm-a-5d8f65bc-hhv87 1/1 Running 0 103s 10.42.0.29 master <none> <none>
nginx-dm-a-5d8f65bc-m7f2h 1/1 Running 0 103s 10.42.0.27 master <none> <none>
nginx-dm-a-5d8f65bc-ttbxl 1/1 Running 0 103s 10.42.0.28 master <none> <none>
nginx-dm-b-577f9bdc85-hmqx6 1/1 Running 0 98s 10.42.0.30 master <none> <none>
nginx-dm-b-577f9bdc85-mvzht 1/1 Running 0 98s 10.42.0.31 master <none> <none>
nginx-dm-b-577f9bdc85-vw6jv 1/1 Running 0 98s 10.42.0.32 master <none> <none>
# 可以通过pod的IP地址访问到nginx服务
[root@master 11-ingress]# curl 10.42.0.27
<html><body>Nginx! With k8s ingress www.a.com!</body></html>
[root@master 11-ingress]# curl 10.42.0.28
<html><body>Nginx! With k8s ingress www.a.com!</body></html>
[root@master 11-ingress]# curl 10.42.0.29
<html><body>Nginx! With k8s ingress www.a.com!</body></html>
[root@master 11-ingress]# curl 10.42.0.30
<html><body>Nginx! With k8s ingress www.b.com!</body></html>
[root@master 11-ingress]# curl 10.42.0.31
<html><body>Nginx! With k8s ingress www.b.com!</body></html>
[root@master 11-ingress]# curl 10.42.0.32
<html><body>Nginx! With k8s ingress www.b.com!</body></html>
# 通过service的IP和端口也可以访问到nginx服务
[root@master 11-ingress]# curl 10.43.251.187:8080
<html><body>Nginx! With k8s ingress www.a.com!</body></html>
[root@master 11-ingress]# curl 10.43.177.192:9090
<html><body>Nginx! With k8s ingress www.b.com!</body></html>
(3) 编写nginx-ingress.yaml配置清单文件
创建nginx-ingress文件，添加两条rules，host分别为www.a.com和www.b.com
[root@master 11-ingress]# cat nginx-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-http
spec:
ingressClassName: "nginx"
rules:
- host: www.a.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx-svc-a
port:
number: 8080
- host: www.b.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx-svc-b
port:
number: 9090
(4) 创建Ingress资源对象
# 当前集群没有Ingress资源对象
[root@master 11-ingress]# kubectl get ingress
No resources found in default namespace.
# 创建Ingress资源对象
[root@master 11-ingress]# kubectl apply -f nginx-ingress.yaml
ingress.networking.k8s.io/ingress-http created
# 创建成功，但可以发现目前ADDRESS为空，尚未就绪，稍等片刻再查
[root@master 11-ingress]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-http nginx www.a.com,www.b.com 80 6s
# 过一会再查看，已经为Ingress分配了地址。ing为ingress的缩写
[root@master 11-ingress]# kubectl get ing
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-http nginx www.a.com,www.b.com 10.43.102.21 80 21s
(5) 在本集群访问Ingress
编辑/etc/hosts文件，添加最后的两行，把www.a.com和www.b.com这两个域名与Ingress的IP 10.43.102.21对应上。
注：这里的地址也可以写成本机IP地址192.168.88.132，这个地址是ingress-http对外暴露的地址，不但集群内节点可以使用，集群外节点也可以使用。但10.43.102.21就只能是集群内才能使用了。
[root@master 11-ingress]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.43.102.21 www.a.com
10.43.102.21 www.b.com
# 访问域名，Ingress转发到对应的服务。在本机，可以访问80端口和17238端口，因为ingress-nginx-controller这个NodePort有对内和对外两类端口。
[root@master 11-ingress]# curl www.a.com
<html><body>Nginx! With k8s ingress www.a.com!</body></html>
[root@master 11-ingress]# curl www.b.com
<html><body>Nginx! With k8s ingress www.b.com!</body></html>
[root@master 11-ingress]# curl www.a.com:17238
<html><body>Nginx! With k8s ingress www.a.com!</body></html>
[root@master 11-ingress]# curl www.b.com:17238
<html><body>Nginx! With k8s ingress www.b.com!</body></html>
[root@master 11-ingress]# curl www.b.com:17239
curl: (7) Failed to connect to www.b.com port 17239 after 0 ms: Connection refused
[root@master 11-ingress]#
(6) 集群外访问Ingress
如果想在集群外访问这两个域名，我们可以按下面的方式操作。
我们之前使用一台IP地址为192.168.88.130的虚拟机创建过nfs服务，这里可以先借用一下。
首先，在刚才的节点上查看ingress服务的IP地址，为192.168.88.132
[root@master 11-ingress]# kubectl get pod -n ingress-nginx -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
ingress-nginx-admission-create-htb7p 0/1 Completed 0 20m 192.168.88.132 master <none> <none>
ingress-nginx-admission-patch-w7nb6 0/1 Completed 1 20m 192.168.88.132 master <none> <none>
ingress-nginx-controller-688895cf66-cmdp8 1/1 Running 0 20m 192.168.88.132 master <none> <none>
然后，在nfs服务器192.168.88.130的/etc/hosts添加两条静态域名-IP映射关系。
[root@nfs-server ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.88.132 www.a.com
192.168.88.132 www.b.com
最后，在集群外的192.168.88.130上访问这两个域名，可以看到能正常访问。
# 集群外部，通过17238访问端口
[root@nfs-server ~]# curl www.a.com:17238
<html><body>Nginx! With k8s ingress www.a.com!</body></html>
[root@nfs-server ~]# curl www.b.com:17238
<html><body>Nginx! With k8s ingress www.b.com!</body></html>
# 在集群外部，无法通过NodePort的80端口访问服务
[root@master ~]# curl www.a.com
curl: (7) Failed to connect to www.a.com port 80 after 0 ms: Connection refused
[root@master ~]# curl www.b.com
curl: (7) Failed to connect to www.b.com port 80 after 0 ms: Connection refused
在正式环境中，我们可以将这两条域名写入搭建的DNS服务器，只需要让待访问这两个域名的设备配置DNS服务器地址就可以了。
(7) 自定义端口访问Ingress
我们在集群外访问www.a.com和www.b.com使用的是17238端口，这个端口不方便记忆，因此我们可以改成自己需要的端口。
有两种方式，一种是修改nginx-ingress-base-deploy.yaml这个文件，把ingress-nginx-controller这个Service，在spec.ports.name下面添加nodePort: 30080，yaml片段如下：
...
spec:
type: NodePort
externalTrafficPolicy: Local
ports:
- name: http
port: 80
targetPort: http
protocol: TCP
nodePort: 30080 # 显式指定http
- name: https
port: 443
targetPort: https
protocol: TCP
nodePort: 30443 # 显式指定https
...
然后kubectl apply -f nginx-ingress-base-deploy.yaml。
或者使用patch方式，patch这种方式由于写入ETCD，所以重启服务或重启设备，配置不会丢失。
# 默认的端口为随机值
[root@master 11-ingress]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.43.190.202 <none> 80:17238/TCP,443:18054/TCP 3h39m
ingress-nginx-controller-admission ClusterIP 10.43.112.84 <none> 443/TCP 3h39m
# 使用patch修改NodePort Service为指定端口号
[root@master 11-ingress] kubectl patch svc ingress-nginx-controller -n ingress-nginx \
--type='json' \
-p='[{"op": "add", "path": "/spec/ports/0/nodePort", "value": 30080},
{"op": "add", "path": "/spec/ports/1/nodePort", "value": 30443}]'
service/ingress-nginx-controller patched
# 再次查看可以看到端口已更改
[root@master 11-ingress]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.43.190.202 <none> 80:30080/TCP,443:30443/TCP 3h39m
ingress-nginx-controller-admission ClusterIP 10.43.112.84 <none> 443/TCP 3h39m
# 此时切换到集群外客户端，访问www.a.com，加30080端口。
[root@nfs-server ~]# curl www.a.com:30080
<html><body>Nginx! With k8s ingress www.a.com!</body></html>
3 Ingress转发原理
我们知道Ingress是一种“集群入口”资源，用来把集群外的HTTP/HTTPS流量按照域名+URL路径的规则分发到集群内的不同Service。下面就来介绍其实现原理。
在ingress-nginx部署时，我们通过nginx-ingress-base-deploy.yaml创建了多个资源对象。
[root@master ~]$ kubectl apply -f nginx-ingress-base-deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
这些资源对象非常多，我们只需要关注最重要的两个，service/ingress-nginx-controller和deployment.apps/ingress-nginx-controller。服务用来做统一入口，Deployment用来做流量转发。
3.1 统一入口
service/ingress-nginx-controller是一个NodePort类型的Service，正是这个Service，接收集群外所有流量，它就是Ingress的统一入口。
[root@master 11-ingress]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.43.190.202 <none> 80:30180/TCP,443:31443/TCP 4h37m
ingress-nginx-controller-admission ClusterIP 10.43.112.84 <none> 443/TCP 4h37m
3.2 流量转发
deployment.apps/ingress-nginx-controller是一个Deployment类型的控制器，它只有一个Pod副本，负责将nginx-ingress.yaml资源对象中的rules规则转换成nginx配置文件规则。
ingress-nginx的Deployment，只有一个Pod。
[root@master 11-ingress]# kubectl get deploy -n ingress-nginx
NAME READY UP-TO-DATE AVAILABLE AGE
ingress-nginx-controller 1/1 1 1 71m
Pod处于Running状态
[root@master 11-ingress]# kubectl get pod -n ingress-nginx
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-8j29g 0/1 Completed 0 71m
ingress-nginx-admission-patch-5n5z2 0/1 Completed 0 67m
ingress-nginx-controller-688895cf66-xkdvl 1/1 Running 0 71m
进入ingress-nginx-controller容器
[root@master 11-ingress]# kubectl exec -it -n ingress-nginx ingress-nginx-controller-688895cf66-xkdvl -- /bin/bash
master:/etc/nginx$
在容器中，查看nginx.conf文件，可以看到有www.a.com及www.b.com相关的字段
master:/etc/nginx$ cat /etc/nginx/nginx.conf | grep 'www.a.com\|www.b.com'
## start server www.a.com
server_name www.a.com ;
## end server www.a.com
## start server www.b.com
server_name www.b.com ;
## end server www.b.com
在Ingress中添加的www.a.com规则转换成nginx配置文件，内容如下。
master:/etc/nginx$ cat /etc/nginx.conf
（部分内容略。。。）
## start server www.a.com
server {
server_name www.a.com ;
http2 on;
listen 80 ;
listen [::]:80 ;
listen 443 ssl;
listen [::]:443 ssl;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_file /etc/nginx/lua/nginx/ngx_conf_certificate.lua;
location / {
set $namespace "default";
set $ingress_name "ingress-http";
set $service_name "nginx-svc-a";
set $service_port "8080";
set $location_path "/";
set $force_ssl_redirect "false";
set $ssl_redirect "true";
set $force_no_ssl_redirect "false";
set $preserve_trailing_slash "false";
set $use_port_in_redirects "false";
rewrite_by_lua_file /etc/nginx/lua/nginx/ngx_rewrite.lua;
header_filter_by_lua_file /etc/nginx/lua/nginx/ngx_conf_srv_hdr_filter.lua;
log_by_lua_file /etc/nginx/lua/nginx/ngx_conf_log_block.lua;
port_in_redirect off;
set $balancer_ewma_score -1;
set $proxy_upstream_name "default-nginx-svc-a-8080";
set $proxy_host $proxy_upstream_name;
set $pass_access_scheme $scheme;
set $pass_server_port $server_port;
set $best_http_host $http_host;
set $pass_port $pass_server_port;
set $proxy_alternative_upstream_name "";
client_max_body_size 1m;
proxy_set_header Host $best_http_host;
# Pass the extracted client certificate to the backend
# Allow websocket connections
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header X-Request-ID $req_id;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Host $best_http_host;
proxy_set_header X-Forwarded-Port $pass_port;
proxy_set_header X-Forwarded-Proto $pass_access_scheme;
proxy_set_header X-Forwarded-Scheme $pass_access_scheme;
proxy_set_header X-Scheme $pass_access_scheme;
# Pass the original X-Forwarded-For
proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
# mitigate HTTPoxy Vulnerability
# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";
# Custom headers to proxied server
proxy_connect_timeout 5s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
proxy_buffering off;
proxy_buffer_size 4k;
proxy_buffers 4 4k;
proxy_max_temp_file_size 1024m;
proxy_request_buffering on;
proxy_http_version 1.1;
proxy_cookie_domain off;
proxy_cookie_path off;
# In case of errors try the next upstream server before returning an error
proxy_next_upstream error timeout;
proxy_next_upstream_timeout 0;
proxy_next_upstream_tries 3;
# Custom Response Headers
proxy_pass http://upstream_balancer;
proxy_redirect off;
}
}
## end server www.a.com
（部分内容略。。。）
可以看到，对于ingress-nginx控制器，首先创建一个NodePort类型的Service用于统一接收外部流量，然后创建一个Deployment用于将Ingress对象的规则转换成Nginx配置，当创建Ingress对象后，Deployment自动将该规则解析成Nginx配置文件。当这些准备工作完成，访问Ingress时，流量先从NodePort Service端口统一进入，然后再根据规则内部转发。
4 示例：K3s自带的Traefic Ingress控制器使用
默认情况下，K3s自带Ingress控制器，Traefik，我们部署K3s时将Traefik禁止掉了，这里再将其打开，演示Traefik Ingress。
(1) 修改K3s启动服务的启动参数
原k3s.service备份一份为k3s.service.disable.traefik，修改k3s.service文件，去掉–disable traefik
[root@master 11-ingress]# cat /etc/systemd/system/k3s.service
（...部分内容略...）
ExecStart=/usr/local/bin/k3s \
server \
'--docker' \
'--disable' \
'metrics-server' \
（...部分内容略...）
修改前后文件差异
[root@master 11-ingress]# diff /etc/systemd/system/k3s.service.disable.traefik /etc/systemd/system/k3s.service
33,34d32
< 'traefik' \
< '--disable' \
(2) 重启K3s
重新加载配置文件[root@master 11-ingress]# systemctl daemon-reload
重启k3s[root@master 11-ingress]# systemctl restart k3s
可以看到traefik相关的Job正在启动，Pod也有两个和traefik相关正在运行
[root@master 11-ingress]# kubectl get all -n kube-system
NAME READY STATUS RESTARTS AGE
pod/coredns-689c5965d-s94lf 1/1 Running 0 2d19h
pod/helm-install-traefik-crd-lbj9b 1/1 Running 0 17s
pod/helm-install-traefik-j7dhb 1/1 Running 0 17s
pod/local-path-provisioner-6f5d79df6-g6qcq 1/1 Running 2 (6d9h ago) 6d9h
pod/nfs-client-provisioner-65d6f566b9-5gz6m 1/1 Running 0 3d10h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kube-dns ClusterIP 10.43.0.10 <none> 53/UDP,53/TCP,9153/TCP 6d9h
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/coredns 1/1 1 1 6d9h
deployment.apps/local-path-provisioner 1/1 1 1 6d9h
deployment.apps/nfs-client-provisioner 1/1 1 1 3d11h
NAME DESIRED CURRENT READY AGE
replicaset.apps/coredns-689c5965d 1 1 1 2d19h
replicaset.apps/local-path-provisioner-6f5d79df6 1 1 1 6d9h
replicaset.apps/nfs-client-provisioner-65d6f566b9 1 1 1 3d11h
NAME COMPLETIONS DURATION AGE
job.batch/helm-install-traefik 0/1 17s 28s
job.batch/helm-install-traefik-crd 0/1 17s 28s
过段时间再次观察，看到traefik已经就绪。外部端口为http的17804和https的14798。
[root@master 11-ingress]# kubectl get all -n kube-system
NAME READY STATUS RESTARTS AGE
pod/coredns-689c5965d-s94lf 1/1 Running 0 2d19h
pod/helm-install-traefik-crd-lbj9b 0/1 Completed 0 77s
pod/helm-install-traefik-j7dhb 0/1 Completed 1 77s
pod/local-path-provisioner-6f5d79df6-g6qcq 1/1 Running 2 (6d9h ago) 6d9h
pod/nfs-client-provisioner-65d6f566b9-5gz6m 1/1 Running 0 3d10h
pod/svclb-traefik-7e9f65a6-qfwrk 2/2 Running 0 57s
pod/traefik-7d5f6474df-vwggs 1/1 Running 0 57s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kube-dns ClusterIP 10.43.0.10 <none> 53/UDP,53/TCP,9153/TCP 6d9h
service/traefik LoadBalancer 10.43.16.188 192.168.88.132 80:17804/TCP,443:14789/TCP 57s
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/svclb-traefik-7e9f65a6 1 1 1 1 1 <none> 57s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/coredns 1/1 1 1 6d9h
deployment.apps/local-path-provisioner 1/1 1 1 6d9h
deployment.apps/nfs-client-provisioner 1/1 1 1 3d11h
deployment.apps/traefik 1/1 1 1 57s
NAME DESIRED CURRENT READY AGE
replicaset.apps/coredns-689c5965d 1 1 1 2d19h
replicaset.apps/local-path-provisioner-6f5d79df6 1 1 1 6d9h
replicaset.apps/nfs-client-provisioner-65d6f566b9 1 1 1 3d11h
replicaset.apps/traefik-7d5f6474df 1 1 1 58s
NAME COMPLETIONS DURATION AGE
job.batch/helm-install-traefik 1/1 25s 89s
job.batch/helm-install-traefik-crd 1/1 22s 89s
(3) 编写traefik Ingress资源对象文件
这里和nginx Ingress的资源对象相似，ingressClassName改为了traefik，其后面的rules都一样，因此要保证nginx-svc-a和nginx-svc-b都正常运行。
[root@master 11-ingress]# cat nginx-traefik-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-traefik-http
spec:
ingressClassName: "traefik"
rules:
- host: www.c.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx-svc-a
port:
number: 8080
- host: www.d.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx-svc-b
port:
number: 9090
(4) 创建Traefik资源对象
# 创建traefik资源对象失败
# 原因是当提交Ingress对象时，API Server仍去调用nginx的校验webhook，而nginx-controller现在没跑，所以webhook调不通，直接500。
# 我们可以让nginx-controller跑起来或者删除ingress-nginx。这里演示后者
[root@master 11-ingress]# kubectl apply -f nginx-traefik-ingress.yaml
Error from server (InternalError): error when creating "nginx-traefik-ingress.yaml": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": failed to call webhook: Post "https://ingress-nginx-controller-admission.ingress-nginx.svc:443/networking/v1/ingresses?timeout=10s": no endpoints available for service "ingress-nginx-controller-admission"
# 删除ingress-nginx相关的所有资源。可以使用创建资源时的资源文件批量删除。
[root@master 11-ingress]# kubectl delete -f nginx-ingress-base-deploy.yaml
namespace "ingress-nginx" deleted
serviceaccount "ingress-nginx" deleted
serviceaccount "ingress-nginx-admission" deleted
role.rbac.authorization.k8s.io "ingress-nginx" deleted
role.rbac.authorization.k8s.io "ingress-nginx-admission" deleted
clusterrole.rbac.authorization.k8s.io "ingress-nginx" deleted
clusterrole.rbac.authorization.k8s.io "ingress-nginx-admission" deleted
rolebinding.rbac.authorization.k8s.io "ingress-nginx" deleted
rolebinding.rbac.authorization.k8s.io "ingress-nginx-admission" deleted
clusterrolebinding.rbac.authorization.k8s.io "ingress-nginx" deleted
clusterrolebinding.rbac.authorization.k8s.io "ingress-nginx-admission" deleted
configmap "ingress-nginx-controller" deleted
service "ingress-nginx-controller" deleted
service "ingress-nginx-controller-admission" deleted
deployment.apps "ingress-nginx-controller" deleted
job.batch "ingress-nginx-admission-create" deleted
job.batch "ingress-nginx-admission-patch" deleted
ingressclass.networking.k8s.io "nginx" deleted
validatingwebhookconfiguration.admissionregistration.k8s.io "ingress-nginx-admission" deleted
# 再次创建Traefik
[root@master 11-ingress]# kubectl apply -f nginx-traefik-ingress.yaml
ingress.networking.k8s.io/ingress-traefik-http created
# Traefik创建成功，端口17804和14789
[root@master 11-ingress]# kubectl get svc -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 10.43.0.10 <none> 53/UDP,53/TCP,9153/TCP 6d9h
traefik LoadBalancer 10.43.16.188 192.168.88.132 80:17804/TCP,443:14789/TCP 30s
(5) 集群外访问
切换到集群外的一台设备，使用traefik服务访问。
先修改hosts文件
[root@master ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.88.132 www.a.com
192.168.88.132 www.b.com
192.168.88.132 www.c.com
192.168.88.132 www.d.com
再访问，可以访问到。
[root@nfs-server ~]# curl www.c.com:17804
<html><body>Nginx! With k8s ingress www.a.com!</body></html>
[root@nfs-server ~]# curl www.d.com:17804
<html><body>Nginx! With k8s ingress www.b.com!</body></html>
开源鸿蒙跨平台开发社区汇聚开发者与厂商，共建“一次开发，多端部署”的开源生态，致力于降低跨端开发门槛，推动万物智联创新。
更多推荐
0
0- 0
已为社区贡献2条内容
所有评论(0)