1. istio 1.6灰度功能测试
istio 1.6新增了istio自身的灰度测试特性，我先测试其灰度功能。
在官网下载并解压istio 1.6，并把istioctl复制到对应目录：
[root@sh-saas-k8s1-master-dev-01 istio]# tar xfzf istio-1.6.0-linux-amd64.tar.gz
[root@sh-saas-k8s1-master-dev-01 istio]# cd istio-1.6.0
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# cp bin/istioctl /usr/local/bin/
cp: overwrite ‘/usr/local/bin/istioctl’? y
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# ll tools/
total 188
-rw-r--r-- 1 root root 2031 May 21 07:12 convert_RbacConfig_to_ClusterRbacConfig.sh
-rw-r--r-- 1 root root 10669 May 21 07:12 dump_kubernetes.sh
-rw-r--r-- 1 root root 88599 May 21 07:12 _istioctl
-rw-r--r-- 1 root root 85301 May 21 07:12 istioctl.bash
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# cp tools/istioctl.bash /usr/local/bin/
cp: overwrite ‘/usr/local/bin/istioctl.bash’? y
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# istioctl version
client version: 1.6.0
control plane version: 1.5.2
data plane version: 1.5.2 (6 proxies), 1.5.1 (4 proxies)
1.1 安装控制面板
通过设置revision=canary变量来安装灰度的控制面板：
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# istioctl install --set revision=canary
Detected that your cluster does not support third party JWT authentication. Falling back to less secure first party JWT. See https://istio.io/docs/ops/best-practices/security/#configure-third-party-service-account-tokens for details.
✔ Istio core installed
✔ Istiod installed
✔ Ingress gateways installed
✔ Addons installed
✔ Installation complete
安装完后，可以看到带有canary的istiod：
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# kubectl get pods -n istio-system
NAME READY STATUS RESTARTS AGE
grafana-556b649566-twdk2 1/1 Running 0 46d
istio-egressgateway-6fdd4f8558-7mnzw 1/1 Running 0 18d
istio-ingressgateway-9fd45f487-grz8g 1/1 Running 0 3m55s
istio-tracing-7cf5f46848-lhjf7 1/1 Running 0 52d
istiod-77fd8dc44c-2krmf 1/1 Running 0 3m42s
istiod-77fd8dc44c-9wd95 1/1 Running 0 15d
istiod-canary-65fd47bf6f-tcl9n 1/1 Running 0 4m10s
kiali-66bcf44569-68nxm 1/1 Running 0 18d
prometheus-7ff5fb8799-2hlwr 2/2 Running 0 3m55s
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# kubectl -n istio-system get svc -lapp=istiod
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
istiod ClusterIP 10.253.252.222 <none> 15012/TCP,443/TCP 52d
istiod-canary ClusterIP 10.253.204.108 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP,53/UDP,853/TCP 4m33s
sidecar injector也有了一个新的灰度版本：
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# kubectl get mutatingwebhookconfigurations
NAME CREATED AT
istio-sidecar-injector 2020-04-04T18:05:07Z
istio-sidecar-injector-canary 2020-05-27T07:58:59Z
1.2 配置数据面板
下面我们使用public-ops-tomcat-dev这个命名空间来做灰度测试。
先看一下命名空间的配置：
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# kubectl describe ns public-ops-tomcat-dev
Name: public-ops-tomcat-dev
Labels: istio-injection=enabled
Annotations: <none>
Status: Active
No resource quota.
Resource Limits
Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio
---- -------- --- --- --------------- ------------- -----------------------
Container ephemeral-storage - - 256Mi 7Gi -
可以看到一个标准的注入标签：istio-injection=enabled，要想配置为灰度测试，需要删除istio-injection标签，打上新标签：istio.io/rev=canary
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# kubectl label namespace public-ops-tomcat-dev istio-injection- istio.io/rev=canary
namespace/public-ops-tomcat-dev labeled
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# kubectl describe ns public-ops-tomcat-dev
Name: public-ops-tomcat-dev
Labels: istio.io/rev=canary
Annotations: <none>
Status: Active
No resource quota.
Resource Limits
Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio
---- -------- --- --- --------------- ------------- -----------------------
Container ephemeral-storage - - 256Mi 7Gi -
重启pod，让配置生效，如果kubectl 的版本 >= 1.15:
kubectl rollout restart deployment -n public-ops-tomcat-dev
否则可以一个一个的删除pod：
[root@sh-saas-k8s1-master-dev-01 ~]# kubectl delete pod -n public-ops-tomcat-dev public-ops-dubbo-demo-web-tomcat-dev-79f758dcf-nwcsg
pod "public-ops-dubbo-demo-web-tomcat-dev-79f758dcf-nwcsg" deleted
[root@sh-saas-k8s1-master-dev-01 ~]# kubectl delete pod -n public-ops-tomcat-dev public-ops-dubbo-demo-web-tomcat-dev-79f758dcf-wfgsr
pod "public-ops-dubbo-demo-web-tomcat-dev-79f758dcf-wfgsr" deleted
[root@sh-saas-k8s1-master-dev-01 ~]#
查看是否是新的版本的sidecar:
[root@sh-saas-k8s1-master-dev-01 ~]# kubectl get pods -n public-ops-tomcat-dev -l istio.io/rev=canary
NAME READY STATUS RESTARTS AGE
public-ops-dubbo-demo-web-tomcat-dev-79f758dcf-m4nm4 2/2 Running 0 79s
public-ops-dubbo-demo-web-tomcat-dev-79f758dcf-m6x72 2/2 Running 0 60s
[root@sh-saas-k8s1-master-dev-01 ~]# istioctl proxy-config -n public-ops-tomcat-dev endpoints public-ops-dubbo-demo-web-tomcat-dev-79f758dcf-m6x72.public-ops-tomcat-dev --cluster xds-grpc -ojson | grep hostname
"hostname": "istiod-canary.istio-system.svc",
1.3 灰度版本的问题
- 怎么把灰度版本变成主版本？
- 怎么清理旧版本或通过升级旧版本后清理灰度版本？
- 目前发现Ingress Gateway在安装灰度版本时是直接升级了的，需要特别注意。
2. istio 升级到1.6
在创建了灰度的版本后，再升级就有些问题，会报错：
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# istioctl manifest versions
Binary version is 1.6.0.
This version of istioctl can:
Install Istio 1.6.0
Update Istio from >=1.5.0 to 1.6.0
Update Istio from <1.7 to 1.6.0
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# istioctl upgrade -y -f custom_profile.yaml
2020-05-28T08:18:30.890158Z info proto: tag has too few fields: "-"
! global.proxy.accessLogFile is deprecated; use meshConfig.accessLogFile instead
Control Plane - egressgateway pod - istio-egressgateway-6fdd4f8558-7mnzw - version: 1.5.2
Control Plane - ingressgateway pod - istio-ingressgateway-9fd45f487-grz8g - version: 1.6.0
Control Plane - pilot pod - istiod-77fd8dc44c-9wd95 - version: 1.5.2
Control Plane - istiod pod - istiod-canary-65fd47bf6f-tcl9n - version: 1.6.0
2020-05-28T08:18:30.969886Z info Error: failed to read the current Istio version, error: different versions of Istio components found: [egressgateway pod - istio-egressgateway-6fdd4f8558-7mnzw - version: 1.5.2 ingressgateway pod - istio-ingressgateway-9fd45f487-grz8g - version: 1.6.0 pilot pod - istiod-77fd8dc44c-9wd95 - version: 1.5.2 istiod pod - istiod-canary-65fd47bf6f-tcl9n - version: 1.6.0]
Error: failed to read the current Istio version, error: different versions of Istio components found: [egressgateway pod - istio-egressgateway-6fdd4f8558-7mnzw - version: 1.5.2 ingressgateway pod - istio-ingressgateway-9fd45f487-grz8g - version: 1.6.0 pilot pod - istiod-77fd8dc44c-9wd95 - version: 1.5.2 istiod pod - istiod-canary-65fd47bf6f-tcl9n - version: 1.6.0]
istio官网已经有人提交了issue,https://github.com/istio/istio/issues/23923 有人说是istio 1.6+才能使用灰度。原版本1.5.x是不支持的。这样的话只能先删除，再重新安装istio 1.6
删除旧的istio:
istioctl manifest generate --set revision=canary | kubectl delete -f -
istioctl manifest generate -f custom_profile.yaml | kubectl delete -f -
重新安装：
[root@sh-saas-k8s1-master-dev-01 istio-1.6.0]# istioctl manifest apply -f custom_profile.yaml
Detected that your cluster does not support third party JWT authentication. Falling back to less secure first party JWT. See https://istio.io/docs/ops/best-practices/security/#configure-third-party-service-account-tokens for details.
! global.proxy.accessLogFile is deprecated; use meshConfig.accessLogFile instead
✔ Istio core installed
✔ Istiod installed
✔ Ingress gateways installed
✔ Egress gateways installed
✔ Addons installed
✔ Installation complete