coredns解析结果异常的问题

发布在 Kubernetes, coredns

我发现k8s内coredns的解析结果有点问题。经常解析不出来。

/ # nslookup kubernetes-dashboard.kube-system.svc.cluster.local
Server:         10.253.255.10
Address:        10.253.255.10:53

Non-authoritative answer:

*** Can't find kubernetes-dashboard.kube-system.svc.cluster.local: No answer

/ # nslookup kubernetes-dashboard.kube-system.svc.cluster.local
Server:         10.253.255.10
Address:        10.253.255.10:53

Name:   kubernetes-dashboard.kube-system.svc.cluster.local
Address: 10.253.255.40

*** Can't find kubernetes-dashboard.kube-system.svc.cluster.local: No answer

/ # nslookup kubernetes-dashboard.kube-system.svc.cluster.local
Server:         10.253.255.10
Address:        10.253.255.10:53

Name:   kubernetes-dashboard.kube-system.svc.cluster.local
Address: 10.253.255.40

阅读全文

fluentd采集数据到kafka时的版本问题

发布在 Kubernetes, fluentd

在使用fluentd采集数据到kafka时，一直不通，碰到了很多报错。 fluentd版本为：1.2.5 fluent-plugin-kafka版本为：0.7.8 kafka版本为：0.9 最开始碰到了这个报错：

2018-09-05 01:42:06 +0000 [warn]: fluent/log.rb:342:warn: Send exception occurred: unknown topic 
2018-09-05 01:42:06 +0000 [warn]: fluent/log.rb:342:warn: Exception Backtrace : /var/lib/gems/2.3.0/gems/ruby-kafka-0.6.8/lib/kafka/protocol/metadata_response.rb:141:in `partitions_for'
/var/lib/gems/2.3.0/gems/ruby-kafka-0.6.8/lib/kafka/cluster.rb:155:in `partitions_for'
/var/lib/gems/2.3.0/gems/fluent-plugin-kafka-0.7.6/lib/fluent/plugin/kafka_producer_ext.rb:190:in `assign_partitions!'
/var/lib/gems/2.3.0/gems/fluent-plugin-kafka-0.7.6/lib/fluent/plugin/kafka_producer_ext.rb:153:in `block in deliver_messages_with_retries'
/var/lib/gems/2.3.0/gems/fluent-plugin-kafka-0.7.6/lib/fluent/plugin/kafka_producer_ext.rb:148:in `loop'
/var/lib/gems/2.3.0/gems/fluent-plugin-kafka-0.7.6/lib/fluent/plugin/kafka_producer_ext.rb:148:in `deliver_messages_with_retries'
/var/lib/gems/2.3.0/gems/fluent-plugin-kafka-0.7.6/lib/fluent/plugin/kafka_producer_ext.rb:102:in `deliver_messages'
/var/lib/gems/2.3.0/gems/fluent-plugin-kafka-0.7.6/lib/fluent/plugin/out_kafka2.rb:220:in `write'
/var/lib/gems/2.3.0/gems/fluentd-1.2.4/lib/fluent/plugin/output.rb:1110:in `try_flush'
/var/lib/gems/2.3.0/gems/fluentd-1.2.4/lib/fluent/plugin/output.rb:1389:in `flush_thread_run'
/var/lib/gems/2.3.0/gems/fluentd-1.2.4/lib/fluent/plugin/output.rb:444:in `block (2 levels) in start'
/var/lib/gems/2.3.0/gems/fluentd-1.2.4/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
2018-09-05 01:42:06 +0000 [info]: fluent/log.rb:322:info: initialized kafka producer: fluentd
2018-09-05 01:42:06 +0000 [debug]: fluent/log.rb:302:debug: taking back chunk for errors. chunk="57515e0ef787da843836cc864f9d1581"
2018-09-05 01:42:06 +0000 [warn]: fluent/log.rb:342:warn: failed to flush the buffer. retry_time=2 next_retry_seconds=2018-09-05 01:42:06 +0000 chunk="57515e0ef787da843836cc864f9d1581" error_class=Kafka::UnknownTopicOrPartition error="unknown topic "
  2018-09-05 01:42:06 +0000 [warn]: plugin/output.rb:1157:rescue in try_flush: suppressed same stacktrace
2018-09-05 01:42:09 +0000 [debug]: fluent/log.rb:302:debug: 61 messages send.
2018-09-05 01:42:09 +0000 [warn]: fluent/log.rb:342:warn: Send exception occurred: unknown topic 
2018-09-05 01:42:09 +0000 [warn]: fluent/log.rb:342:warn: Exception Backtrace : /var/lib/gems/2.3.0/gems/ruby-kafka-0.6.8/lib/kafka/protocol/metadata_response.rb:141:in `partitions_for'
/var/lib/gems/2.3.0/gems/ruby-kafka-0.6.8/lib/kafka/cluster.rb:155:in `partitions_for'
/var/lib/gems/2.3.0/gems/fluent-plugin-kafka-0.7.6/lib/fluent/plugin/kafka_producer_ext.rb:190:in `assign_partitions!'
/var/lib/gems/2.3.0/gems/fluent-plugin-kafka-0.7.6/lib/fluent/plugin/kafka_producer_ext.rb:153:in `block in deliver_messages_with_retries'
/var/lib/gems/2.3.0/gems/fluent-plugin-kafka-0.7.6/lib/fluent/plugin/kafka_producer_ext.rb:148:in `loop'
/var/lib/gems/2.3.0/gems/fluent-plugin-kafka-0.7.6/lib/fluent/plugin/kafka_producer_ext.rb:148:in `deliver_messages_with_retries'
/var/lib/gems/2.3.0/gems/fluent-plugin-kafka-0.7.6/lib/fluent/plugin/kafka_producer_ext.rb:102:in `deliver_messages'
/var/lib/gems/2.3.0/gems/fluent-plugin-kafka-0.7.6/lib/fluent/plugin/out_kafka2.rb:220:in `write'
/var/lib/gems/2.3.0/gems/fluentd-1.2.4/lib/fluent/plugin/output.rb:1110:in `try_flush'
/var/lib/gems/2.3.0/gems/fluentd-1.2.4/lib/fluent/plugin/output.rb:1389:in `flush_thread_run'
/var/lib/gems/2.3.0/gems/fluentd-1.2.4/lib/fluent/plugin/output.rb:444:in `block (2 levels) in start'
/var/lib/gems/2.3.0/gems/fluentd-1.2.4/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
2018-09-05 01:42:09 +0000 [info]: fluent/log.rb:322:info: initialized kafka producer: fluentd
2018-09-05 01:42:09 +0000 [debug]: fluent/log.rb:302:debug: taking back chunk for errors. chunk="57515e0ef787da843836cc864f9d1581"
2018-09-05 01:42:09 +0000 [warn]: fluent/log.rb:342:warn: failed to flush the buffer. retry_time=3 next_retry_seconds=2018-09-05 01:42:09 +0000

这是因为没有配置default_topic,使用下面的配置指定topic就可以了。

阅读全文

腾讯云环境K8S安装及配置测试

发布在 Kubernetes

目录

1. k8s集群系统规划

1.1. kubernetes 1.10的依赖

k8s V1.10对一些相关的软件包，如etcd,docker并不是全版本支持或全版本测试，建议的版本如下：

  • docker: 1.11.2 to 1.13.1 and 17.03.x
  • etcd: 3.1.12
  • 全部信息如下：

参考：External Dependencies

  • The supported etcd server version is 3.1.12, as compared to 3.0.17 in v1.9 (#60988)
  • The validated docker versions are the same as for v1.9: 1.11.2 to 1.13.1 and 17.03.x (ref)
  • The Go version is go1.9.3, as compared to go1.9.2 in v1.9. (#59012)
  • The minimum supported go is the same as for v1.9: go1.9.1. (#55301)
  • CNI is the same as v1.9: v0.6.0 (#51250)
  • CSI is updated to 0.2.0 as compared to 0.1.0 in v1.9. (#60736)
  • The dashboard add-on has been updated to v1.8.3, as compared to 1.8.0 in v1.9. (#57326)
  • Heapster has is the same as v1.9: v1.5.0. It will be upgraded in v1.11. (ref)
  • Cluster Autoscaler has been updated to v1.2.0. (#60842, @mwielgus)
  • Updates kube-dns to v1.14.8 (#57918, @rramkumar1)
  • Influxdb is unchanged from v1.9: v1.3.3 (#53319)
  • Grafana is unchanged from v1.9: v4.4.3 (#53319)
  • CAdvisor is v0.29.1 (#60867)
  • fluentd-gcp-scaler is v0.3.0 (#61269)
  • Updated fluentd in fluentd-es-image to fluentd v1.1.0 (#58525, @monotek)
  • fluentd-elasticsearch is v2.0.4 (#58525)
  • Updated fluentd-gcp to v3.0.0. (#60722)
  • Ingress glbc is v1.0.0 (#61302)
  • OIDC authentication is coreos/go-oidc v2 (#58544)
  • Updated fluentd-gcp updated to v2.0.11. (#56927, @x13n)
  • Calico has been updated to v2.6.7 (#59130, @caseydavenport)

1.2 测试服务器准备及环境规划

服务器名IP功 能安装服务
sh-saas-cvmk8s-master-0110.12.96.3mastermaster,etcd
sh-saas-cvmk8s-master-0210.12.96.5mastermaster,etcd
sh-saas-cvmk8s-master-0310.12.96.13mastermaster,etcd
sh-saas-cvmk8s-node-0110.12.96.2nodenode
sh-saas-cvmk8s-node-0210.12.96.4nodenode
sh-saas-cvmk8s-node-0310.12.96.6nodenode
bs-ops-test-docker-dev-04172.21.248.242私有镜像仓库harbor
VIP10.12.96.100master vipnetmask:255.255.255.0

netmask都为:255.255.255.0

所有的测试服务器安装centos linux 7.4最新版本.

VIP：10.12.96.100只是用于keepalived的测试，实际本文使用的是腾讯云LB+haproxy的模式，使用的腾讯云LB VIP为：10.12.16.101

容器网段：10.254.0.0/16 容器网段需要避免这些冲突:

  • 同vpc的其它集群的集群网络cidr
  • 所在vpc的cidr
  • 所在vpc的子网路由的cidr
  • route-ctl list 能看到的所有route table 的 cidr 容器网段不要在VPC内创建，也要不在VPC的路由表内，使用一个VPC内不存在的网络。

k8s service cluster网络：10.254.255.0/24

阅读全文

kubernetes 日常使用及操作测试

发布在 Kubernetes

目录

前文已经安装好了一套kubernetes 1.10，下面我们来进行日常使用测试

1. 创建部署及服务

编辑一个yaml文件：

vim nginx.yaml 
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-router
  namespace: test
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx-router
    spec:
      containers:
      - name: nginx-router
        image: 172.21.248.242/base/nginx
        ports:
        - containerPort: 80
		
---
kind: Service
apiVersion: v1
metadata:
  name: nginx-router
  namespace: test
spec:
  selector:
    app: nginx-router
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
	  
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-router-ingress
  namespace: test
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: "nginx.k8s.dev.huilog.com"
    http:
      paths:
      - backend:
          serviceName: nginx-router
          servicePort: 80
		  
# 创建部署及服务：
[root@bs-ops-test-docker-dev-01 dev]# kubectl create -f nginx.yaml 
deployment.extensions "nginx-router" created
service "nginx-router" created
ingress.extensions "nginx-router-ingress" created
[root@bs-ops-test-docker-dev-01 dev]#

阅读全文

kubernetes traefik ingress 安装及配置

发布在 Kubernetes, Traefik

Traefik是一款开源的反向代理与负载均衡工具。它最大的优点是能够与常见的微服务系统直接整合，可以实现自动化动态配置。目前支持Docker, Swarm, Mesos/Marathon, Mesos, Kubernetes, Consul, Etcd, Zookeeper, BoltDB, Rest API等等后端模型。 以下是架构图: 需要指出的是，ingress-controllers其实是kubernetes的一部分，ingress就是从kubernetes集群外访问集群的入口，将用户的URL请求转发到不同的service上。Ingress相当于nginx、apache等负载均衡方向代理服务器，其中还包括规则定义，即URL的路由信息，路由信息得的刷新由Ingress controller来提供。

Ingress Controller 实质上可以理解为是个监视器，Ingress Controller 通过不断地跟 kubernetes API 打交道，实时的感知后端 service、pod 等变化，比如新增和减少 pod，service 增加与减少等；当得到这些变化信息后，Ingress Controller 再结合下文的 Ingress 生成配置，然后更新反向代理负载均衡器，并刷新其配置，达到服务发现的作用。

阅读全文

kubernetes V1.10集群安装及配置

发布在 Kubernetes

目录

1. k8s集群系统规划

1.1. kubernetes 1.10的依赖

k8s V1.10对一些相关的软件包，如etcd,docker并不是全版本支持或全版本测试，建议的版本如下：

  • docker: 1.11.2 to 1.13.1 and 17.03.x
  • etcd: 3.1.12
  • 全部信息如下：

参考：External Dependencies

  • The supported etcd server version is 3.1.12, as compared to 3.0.17 in v1.9 (#60988)
  • The validated docker versions are the same as for v1.9: 1.11.2 to 1.13.1 and 17.03.x (ref)
  • The Go version is go1.9.3, as compared to go1.9.2 in v1.9. (#59012)
  • The minimum supported go is the same as for v1.9: go1.9.1. (#55301)
  • CNI is the same as v1.9: v0.6.0 (#51250)
  • CSI is updated to 0.2.0 as compared to 0.1.0 in v1.9. (#60736)
  • The dashboard add-on has been updated to v1.8.3, as compared to 1.8.0 in v1.9. (#57326)
  • Heapster has is the same as v1.9: v1.5.0. It will be upgraded in v1.11. (ref)
  • Cluster Autoscaler has been updated to v1.2.0. (#60842, @mwielgus)
  • Updates kube-dns to v1.14.8 (#57918, @rramkumar1)
  • Influxdb is unchanged from v1.9: v1.3.3 (#53319)
  • Grafana is unchanged from v1.9: v4.4.3 (#53319)
  • CAdvisor is v0.29.1 (#60867)
  • fluentd-gcp-scaler is v0.3.0 (#61269)
  • Updated fluentd in fluentd-es-image to fluentd v1.1.0 (#58525, @monotek)
  • fluentd-elasticsearch is v2.0.4 (#58525)
  • Updated fluentd-gcp to v3.0.0. (#60722)
  • Ingress glbc is v1.0.0 (#61302)
  • OIDC authentication is coreos/go-oidc v2 (#58544)
  • Updated fluentd-gcp updated to v2.0.11. (#56927, @x13n)
  • Calico has been updated to v2.6.7 (#59130, @caseydavenport)

阅读全文

在K8S 的ingress上配置HTTP认证的方法

发布在 Kubernetes, Traefik

在K8S 的ingress上配置HTTP认证的方法如下：

1 . 使用htpasswd创建一个auth文件：

htpasswd -c ./auth myusername
cat auth
myusername:$apr1$78Jyn/1K$ERHKVRPPlzAX8eBtLuvRZ0
  1. 创建一个K8S的secret:
kubectl create secret generic mysecret --from-file auth --namespace=monitoring 
kubectl --namespace=monitoring get secret mysecret 
NAME      TYPE    DATA    AGE 
mysecret Opaque   1      106d

阅读全文

作者的图片

阿辉

容器技术及容器集群等分布式系统研究

容器平台负责人

上海