1. istio 1.6灰度功能测试
istio 1.6新增了istio自身的灰度测试特性，我先测试其灰度功能。
在官网下载并解压istio 1.6，并把istioctl复制到对应目录：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
[root@sh-saas-k8s1-master-dev-01 istio]# tar xfzf istio-1.6.0-linux-amd64.tar.gz [root@sh-saas-k8s1-master-dev-01 istio]# cd istio-1.6.0 [root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# cp bin/istioctl /usr/local/bin/ cp: overwrite ‘/usr/local/bin/istioctl’? y [root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# ll tools/ total 188 -rw-r--r-- 1 root root 2031 May 21 07:12 convert_RbacConfig_to_ClusterRbacConfig.sh -rw-r--r-- 1 root root 10669 May 21 07:12 dump_kubernetes.sh -rw-r--r-- 1 root root 88599 May 21 07:12 _istioctl -rw-r--r-- 1 root root 85301 May 21 07:12 istioctl.bash [root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# cp tools/istioctl.bash /usr/local/bin/ cp: overwrite ‘/usr/local/bin/istioctl.bash’? y [root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# istioctl version client version: 1.6.0 control plane version: 1.5.2 data plane version: 1.5.2 (6 proxies), 1.5.1 (4 proxies) |
1.1 安装控制面板
通过设置revision=canary变量来安装灰度的控制面板：
|
1 2 3 4 5 6 7 8 |
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# istioctl install --set revision=canary Detected that your cluster does not support third party JWT authentication. Falling back to less secure first party JWT. See https://istio.io/docs/ops/best-practices/security/#configure-third-party-service-account-tokens for details. ✔ Istio core installed ✔ Istiod installed ✔ Ingress gateways installed ✔ Addons installed ✔ Installation complete |
安装完后，可以看到带有canary的istiod：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# kubectl get pods -n istio-system NAME READY STATUS RESTARTS AGE grafana-556b649566-twdk2 1/1 Running 0 46d istio-egressgateway-6fdd4f8558-7mnzw 1/1 Running 0 18d istio-ingressgateway-9fd45f487-grz8g 1/1 Running 0 3m55s istio-tracing-7cf5f46848-lhjf7 1/1 Running 0 52d istiod-77fd8dc44c-2krmf 1/1 Running 0 3m42s istiod-77fd8dc44c-9wd95 1/1 Running 0 15d istiod-canary-65fd47bf6f-tcl9n 1/1 Running 0 4m10s kiali-66bcf44569-68nxm 1/1 Running 0 18d prometheus-7ff5fb8799-2hlwr 2/2 Running 0 3m55s [root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# kubectl -n istio-system get svc -lapp=istiod NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE istiod ClusterIP 10.253.252.222 <none> 15012/TCP,443/TCP 52d istiod-canary ClusterIP 10.253.204.108 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP,53/UDP,853/TCP 4m33s |
sidecar injector也有了一个新的灰度版本：
|
1 2 3 4 5 |
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# kubectl get mutatingwebhookconfigurations NAME CREATED AT istio-sidecar-injector 2020-04-04T18:05:07Z istio-sidecar-injector-canary 2020-05-27T07:58:59Z |
1.2 配置数据面板
下面我们使用public-ops-tomcat-dev这个命名空间来做灰度测试。
先看一下命名空间的配置：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# kubectl describe ns public-ops-tomcat-dev Name: public-ops-tomcat-dev Labels: istio-injection=enabled Annotations: <none> Status: Active No resource quota. Resource Limits Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio ---- -------- --- --- --------------- ------------- ----------------------- Container ephemeral-storage - - 256Mi 7Gi - |
可以看到一个标准的注入标签：istio-injection=enabled，要想配置为灰度测试，需要删除istio-injection标签，打上新标签：istio.io/rev=canary
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# kubectl label namespace public-ops-tomcat-dev istio-injection- istio.io/rev=canary namespace/public-ops-tomcat-dev labeled [root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# kubectl describe ns public-ops-tomcat-dev Name: public-ops-tomcat-dev Labels: istio.io/rev=canary Annotations: <none> Status: Active No resource quota. Resource Limits Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio ---- -------- --- --- --------------- ------------- ----------------------- Container ephemeral-storage - - 256Mi 7Gi - |
重启pod，让配置生效，如果kubectl 的版本 >= 1.15:
|
1 2 |
kubectl rollout restart deployment -n public-ops-tomcat-dev |
否则可以一个一个的删除pod：
|
1 2 3 4 5 6 |
[root@sh-saas-k8s1-master-dev-01 ~]# kubectl delete pod -n public-ops-tomcat-dev public-ops-dubbo-demo-web-tomcat-dev-79f758dcf-nwcsg pod "public-ops-dubbo-demo-web-tomcat-dev-79f758dcf-nwcsg" deleted [root@sh-saas-k8s1-master-dev-01 ~]# kubectl delete pod -n public-ops-tomcat-dev public-ops-dubbo-demo-web-tomcat-dev-79f758dcf-wfgsr pod "public-ops-dubbo-demo-web-tomcat-dev-79f758dcf-wfgsr" deleted [root@sh-saas-k8s1-master-dev-01 ~]# |
查看是否是新的版本的sidecar:
|
1 2 3 4 5 6 7 8 |
[root@sh-saas-k8s1-master-dev-01 ~]# kubectl get pods -n public-ops-tomcat-dev -l istio.io/rev=canary NAME READY STATUS RESTARTS AGE public-ops-dubbo-demo-web-tomcat-dev-79f758dcf-m4nm4 2/2 Running 0 79s public-ops-dubbo-demo-web-tomcat-dev-79f758dcf-m6x72 2/2 Running 0 60s [root@sh-saas-k8s1-master-dev-01 ~]# istioctl proxy-config -n public-ops-tomcat-dev endpoints public-ops-dubbo-demo-web-tomcat-dev-79f758dcf-m6x72.public-ops-tomcat-dev --cluster xds-grpc -ojson | grep hostname "hostname": "istiod-canary.istio-system.svc", |
1.3 灰度版本的问题
- 怎么把灰度版本变成主版本？
- 怎么清理旧版本或通过升级旧版本后清理灰度版本？
- 目前发现Ingress Gateway在安装灰度版本时是直接升级了的，需要特别注意。
2. istio 升级到1.6
在创建了灰度的版本后，再升级就有些问题，会报错：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# istioctl manifest versions Binary version is 1.6.0. This version of istioctl can: Install Istio 1.6.0 Update Istio from >=1.5.0 to 1.6.0 Update Istio from <1.7 to 1.6.0 [root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# istioctl upgrade -y -f custom_profile.yaml 2020-05-28T08:18:30.890158Z info proto: tag has too few fields: "-" ! global.proxy.accessLogFile is deprecated; use meshConfig.accessLogFile instead Control Plane - egressgateway pod - istio-egressgateway-6fdd4f8558-7mnzw - version: 1.5.2 Control Plane - ingressgateway pod - istio-ingressgateway-9fd45f487-grz8g - version: 1.6.0 Control Plane - pilot pod - istiod-77fd8dc44c-9wd95 - version: 1.5.2 Control Plane - istiod pod - istiod-canary-65fd47bf6f-tcl9n - version: 1.6.0 2020-05-28T08:18:30.969886Z info Error: failed to read the current Istio version, error: different versions of Istio components found: [egressgateway pod - istio-egressgateway-6fdd4f8558-7mnzw - version: 1.5.2 ingressgateway pod - istio-ingressgateway-9fd45f487-grz8g - version: 1.6.0 pilot pod - istiod-77fd8dc44c-9wd95 - version: 1.5.2 istiod pod - istiod-canary-65fd47bf6f-tcl9n - version: 1.6.0] Error: failed to read the current Istio version, error: different versions of Istio components found: [egressgateway pod - istio-egressgateway-6fdd4f8558-7mnzw - version: 1.5.2 ingressgateway pod - istio-ingressgateway-9fd45f487-grz8g - version: 1.6.0 pilot pod - istiod-77fd8dc44c-9wd95 - version: 1.5.2 istiod pod - istiod-canary-65fd47bf6f-tcl9n - version: 1.6.0] |
istio官网已经有人提交了issue,https://github.com/istio/istio/issues/23923 有人说是istio 1.6+才能使用灰度。原版本1.5.x是不支持的。这样的话只能先删除，再重新安装istio 1.6
删除旧的istio:
|
1 2 3 4 |
istioctl manifest generate --set revision=canary | kubectl delete -f - istioctl manifest generate -f custom_profile.yaml | kubectl delete -f - |
重新安装：
|
1 2 3 4 5 6 7 8 9 10 |
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# istioctl manifest apply -f custom_profile.yaml Detected that your cluster does not support third party JWT authentication. Falling back to less secure first party JWT. See https://istio.io/docs/ops/best-practices/security/#configure-third-party-service-account-tokens for details. ! global.proxy.accessLogFile is deprecated; use meshConfig.accessLogFile instead ✔ Istio core installed ✔ Istiod installed ✔ Ingress gateways installed ✔ Egress gateways installed ✔ Addons installed ✔ Installation complete |
厉害啊，就怕bug出来