eBPF Cilium实战(2) – 底层网络可观测性

在之前的平台中,对于组件之间的网络流向不具备直接的可观测性,用户组件间通信出现问题,只能通过传统命令行工具进行手动排查,而 cilium 的 Hubble 服务可以提供 UI 界面向用户展示实时的流量状态,同时可以将这些指标暴露给 Prometheus 进行聚合整理,让用户可以更直观的对底层网络状态进行观测监控。

开启 Hubble UI 服务

cilium 的网络可观测性由 Hubble 服务提供,在安装 cilium 时,默认不会安装 Hubble ,可以通过以下命令开启 Hubble 服务

helm upgrade cilium cilium/cilium --version 1.11.2 \
   --namespace kube-system \
   --reuse-values \
   --set hubble.relay.enabled=true \
   --set hubble.ui.enabled=true

部署完成后,可以通过以下命令确定状态

$ kubectl get po -n kube-system |grep hubble
hubble-relay-65ff5f9bf6-247pt         1/1     Running     0          5d19h
hubble-ui-5f7cdc86c7-gq5hs            3/3     Running     0          5d19h
$ kubectl get svc -n kube-system | grep hubble
hubble-relay     ClusterIP   10.43.73.95    <none>        80/TCP                   5d19h
hubble-ui        ClusterIP   10.43.20.190   <none>        80/TCP                   5d19h

Hubble 部署完成后,集群外部还无法直接访问,可以通过以下方式开启对外访问

  • 临时开启

    执行命令时可以通过 IP:12000 访问 UI 界面,退出命令后无法继续访问

    cilium hubble ui
    
  • 长期开启

    通过 Rainbond 平台添加第三方组件的方式,随时开启或关闭 UI 界面的访问入口

Hubble UI 展示信息

访问主页面

进入需要查看的名称空间,页面中部展示当前的流量拓扑,下部展示流量记录

点击流量记录可以查看详情

点击组件后只展示相关流量

选择流量记录展示的信息列

选择展示的流量类型

选择展示时是否忽略特殊类型流量

对接 Prometheus 和 Grafana

cilium 提供了部署 Prometheus 和 Grafana 的 yaml 文件,其中包含了 Grafana 的模板文件,但 cilium 默认安装的情况下没有开放监控指标,所以需要先开启监控指标后再部署 Prometheus 和 Grafana

开启监控指标

helm upgrade cilium cilium/cilium --version 1.11.2 \
   --namespace kube-system \
   --reuse-values \
   --set prometheus.enabled=true \
   --set operator.prometheus.enabled=true \
   --set hubble.enabled=true \
   --set hubble.metrics.enabled="{dns,drop,tcp,flow,port-distribution,icmp,http}"

部署 Prometheus 和 Grafana

$ kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/1.11.2/examples/kubernetes/addons/prometheus/monitoring-example.yaml
namespace/cilium-monitoring created
serviceaccount/prometheus-k8s created
configmap/grafana-config created
configmap/grafana-cilium-dashboard created
configmap/grafana-cilium-operator-dashboard created
configmap/grafana-hubble-dashboard created
configmap/prometheus created
clusterrole.rbac.authorization.k8s.io/prometheus unchanged
clusterrolebinding.rbac.authorization.k8s.io/prometheus unchanged
service/grafana created
service/prometheus created
deployment.apps/grafana created
deployment.apps/prometheus created

确认运行状态

$ kubectl get po -n cilium-monitoring
NAME                          READY   STATUS    RESTARTS   AGE
grafana-d69c97b9b-5ztrj       1/1     Running   0          5d20h
prometheus-655fb888d7-456n4   1/1     Running   0          5d20h
$ kubectl get svc -n cilium-monitoring
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
grafana      ClusterIP   10.43.230.15    <none>        3000/TCP   5d20h
prometheus   ClusterIP   10.43.219.180   <none>        9090/TCP   5d20h

开启对外访问

  • 临时开启

    kubectl -n cilium-monitoring port-forward service/grafana --address 0.0.0.0 --address :: 3000:3000
    kubectl -n cilium-monitoring port-forward service/prometheus --address 0.0.0.0 --address :: 9090:9090
    
  • 长期开启

Grafana 展示信息

Cilium Metrics

Cilium Operator

Hubble

K8S中文社区微信公众号

评论 2

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址
  1. #2

    I’m excited to write about Henry Hacker, he is a great and brilliant hacker who penetrated my spouse’s phone without a physical installation app. And I was able to access my spouse’s phone, SMS, Whatsapp, Instagram, Facebook, Wechat, Snapchat, Call Logs, Kik, Twitter and all social media. The most amazing thing there is that he restores all phone deleted text messages. And I also have access to everything including the phone gallery without touching the phone.I can see the whole secret of my spouse. Contact him for any hacking service. He is also a genius in repairing Credit Score, increasing school grade, Clear Criminal Record etc. His service is fast. Contact on: Henryclarkethicalhacker@gmail.com and you can text, call him on whatsapp him on +18134211326…

    Eliott Sharon2周前 (09-19)回复
  2. #1

    Hello Guys, I just completed my divorce with my cheating husband of 10 years with two beautiful kids when i got a solid evidence of his unfaithfulness on extra marital affairs and his infidelity lifestyle, And the various applications he used to hide chats and lot of secret on his cell phone. Then, i decided to hire (Henry clark) an hacker and a PI just to be 100% sure because i don’t want to confront him or take any kinda step without proof, fact and figures because that could leads to defamation… It was with this great, honest, professional and trustworthy Man i got to know my husband is a professional cheater and has been using this app to hide most of his chats. This great hacker helped me broke to into his cell phone activities and wired everything he does on his cell phone directly to my phone and i was able to monitor and track him directly from my phone remotely without him knowing,
    Contact the Ethical hacker via email, Henryclarkethicalhacker@gmail.com you can also reach him on Whatsapp 18134211326..

    Steve Jenna2周前 (09-19)回复