开源镜像网站开发,wordpress开启xmlrppc,网站转回国内,二维码生成器表白文字本文参考
[k8s安装prometheus并持久化数据_/prometheus-config-reloader:-CSDN博客](https://blog.csdn.net/vic_qxz/article/details/119598466)前置要求: 已经部署了NFS或者其他存储的K8s集群. 这里注意networkpolicies网络策略问题#xff0c;可以后面删除这个策略#x…本文参考
[k8s安装prometheus并持久化数据_/prometheus-config-reloader:-CSDN博客](https://blog.csdn.net/vic_qxz/article/details/119598466)前置要求: 已经部署了NFS或者其他存储的K8s集群. 这里注意networkpolicies网络策略问题可以后面删除这个策略这里可以查看我之前的文档。
部署kube-prometheus
这里是配置好才执行这个我们还没有配置存储什么的需要进行修改 $ git clone https://github.com/coreos/kube-prometheus.git #版本最新的是0.13.0
$ kubectl create -f manifests/setup
$ until kubectl get servicemonitors --all-namespaces ; do date; sleep 1; echo ; done
kubectl create -f manifests/ #如果资源已经存在则会报错
kubectl apply -f 跟这个一样 #如果资源已经存在则会进行更新详解一下
2. kubectl create -f manifests/setup: 使用 kubectl 命令创建 Kubernetes 资源这些资源位于 manifests/setup 目录下。一般来说这个命令会创建一些必要的资源比如 ServiceAccount、ClusterRole 和 ClusterRoleBinding 等用于配置 Prometheus 和 Grafana 在 Kubernetes 中的权限。3. until kubectl get servicemonitors --all-namespaces ; do date; sleep 1; echo ; done: 这是一个循环命令它会持续执行 kubectl get servicemonitors --all-namespaces 命令直到能够成功获取到所有命名空间中的 ServiceMonitor 资源。ServiceMonitor 是 Prometheus Operator 中的一种资源类型用于指定 Prometheus 服务器应该如何监控应用程序。在这个命令中通过 until 循环检查是否已经创建了所有的 ServiceMonitor 资源。4. kubectl create -f manifests/: 使用 kubectl 命令创建 Kubernetes 资源这些资源位于 manifests/ 目录下。在这个命令中一般会创建 Prometheus、Alertmanager、Grafana 等监控相关的资源。持久化数据我这里用的是NFS创建动态的pv 我的storageclass名称是nfs-storageclass
rootk8s-master01:~/test/prometheus/kube-prometheus-0.13.0# kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
managed-nfs-storage fuseim.pri/ifs Delete Immediate false 4d20h
nfs-storageclass prometheus-nfs-storage Retain Immediate false 16hkube-prometheus的组件简介及配置变更
1.从整体架构看prometheus 一共四大组件。 exporter 通过接口暴露监控数据 prometheus-server 采集并存储数据 grafana 通过prometheus-server查询并友好展示数据 alertmanager 处理告警对外发送
prometheus-operator
prometheus-operator 服务是deployment方式部署他是整个基础组件的核心他监控我们自定义的 prometheus 和alertmanager并生成对应的 statefulset。 就是prometheus和alertmanager服务是通过他部署出来的。
修改配置文件
grafana-pvc
创建grafana的存储卷. 并修改grafana-deployment.yaml文件, 将官方的emptyDir更换为persistentVolumeClaim
1.创建pvc
$ cd kube-prometheus/manifests/
$ cat grafana-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:# PersistentVolumeClaim 名称name: grafananamespace: monitoringannotations:# 与 nfs-storageClass.yaml metadata.name 保持一致volume.beta.kubernetes.io/storage-class: nfs-storageclass
spec:# 使用的存储类为 nfs-storageclassstorageClassName: nfs-storageclass# 访问模式为 ReadWriteManyaccessModes:- ReadWriteMany#- ReadWriteOnceresources:# 存储请求为 50Girequests:storage: 50Gi $ kubectl apply -f grafana-pvc.yaml2.修改默认的grafana配置文件
$ vim grafana-deployment.yaml...##找到 grafana-storage, 添加上面创建的pvc: grafana. 然后保存.volumes:- name: grafana-storagepersistentVolumeClaim:claimName: grafana
...$ kubectl apply -f grafana-deployment.yamlprometheus-k8s持久化
prometheus-server 获取各端点数据并存储与本地创建方式为自定义资源 crd中的prometheus。 创建自定义资源prometheus后会启动一个statefulset即prometheus-server. 默认是没有配置持久化存储的
1.修改配置文件
$ cd kube-prometheus/manifests/
$ vim prometheus-prometheus.yaml
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:labels:prometheus: k8sname: k8snamespace: monitoring
spec:alerting:alertmanagers:- name: alertmanager-mainnamespace: monitoringport: webstorage: #这部分为持久化配置volumeClaimTemplate:spec:storageClassName: nfs-23 accessModes: [ReadWriteOnce]resources:requests:storage: 100GinodeSelector:kubernetes.io/os: linuxpodMonitorNamespaceSelector: {}podMonitorSelector: {}replicas: 2resources:requests:memory: 400MiruleSelector:matchLabels:prometheus: k8srole: alert-rulessecurityContext:fsGroup: 2000runAsNonRoot: truerunAsUser: 1000serviceAccountName: prometheus-k8sserviceMonitorNamespaceSelector: {}serviceMonitorSelector: {}version: v2.17.2执行变更, 这里会自动创建两个指定大小的pvprometheus-k8s-0prometheus-k8s-1
$ kubectl apply -f manifests/prometheus-prometheus.yaml 修改存储时长
$ vim manifests/setup/prometheus-operator-deployment.yaml
....- args:- --kubelet-servicekube-system/kubelet- --logtostderrtrue- --config-reloader-imagejimmidyson/configmap-reload:v0.3.0- --prometheus-config-reloaderquay.io/coreos/prometheus-config-reloader:v0.39.0- storage.tsdb.retention.time180d ## 修改存储时长
....
$ kubectl apply -f manifests/setup/prometheus-operator-deployment.yaml添加ingress访问grafana和promethues
这里访问是有问题的参照我kubernetes-networkpolicies网络策略问题这篇文章解决
$ cat ingress.yml
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:annotations:k8s.eip.work/workload: grafanak8s.kuboard.cn/workload: grafanageneration: 2labels:app: grafananame: grafananamespace: monitoring
spec:rules:- host: k8s-moni.fenghong.techhttp:paths:- backend:serviceName: grafanaservicePort: httppath: /
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:annotations:k8s.kuboard.cn/workload: prometheus-k8sgeneration: 2labels:app: prometheusprometheus: k8smanagedFields:- apiVersion: networking.k8s.io/v1beta1name: prometheus-k8snamespace: monitoring
spec:rules:- host: k8s-prom.fenghong.techhttp:paths:- backend:serviceName: prometheus-k8sservicePort: webpath: /执行apply
## 安装 ingress controller
$ kubectl apply -f https://kuboard.cn/install-script/v1.18.x/nginx-ingress.yaml## 暴露grafana及prometheus服务
$ kubectl apply -f ingress.ymlweb访问 配置kube-prometheus监控额外的项目
添加additional-scrape-configs配置文件. 例如
$ cat monitor/add.yaml
- job_name: prometheus# metrics_path defaults to /metrics# scheme defaults to http.static_configs:- targets: [192.168.0.23:9100, 192.168.0.21:9101, 192.168.0.61:9100, 192.168.0.62:9100, 192.168.0.63:9100, 192.168.0.64:9100, 192.168.0.89:9100, 192.168.0.11:9100]
- job_name: mysqlstatic_configs:- targets: [192.168.0.21:9104,192.168.0.23:9104]
- job_name: nginxstatic_configs:- targets: [192.168.0.23:9913]- job_name: elasticsearchmetrics_path: /_prometheus/metricsstatic_configs:- targets: [192.168.0.31:9200]创建secret文件, 我这里部署到了monitoring 命名空间.
$ kubectl create secret generic additional-scrape-configs --from-fileadd.yaml --dry-run -oyaml additional-scrape-configs.yaml
$ kubectl apply -f additional-scrape-configs.yaml -n monitoring在prometheus-prometheus.yaml中添加 additionalScrapeConfigs 选项.
$ cat prometheus-prometheus.yaml
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:labels:prometheus: k8sname: k8snamespace: monitoring
spec:alerting:alertmanagers:- name: alertmanager-mainnamespace: monitoringport: webstorage: #这部分为持久化配置volumeClaimTemplate:spec:storageClassName: nfs-23 accessModes: [ReadWriteOnce]resources:requests:storage: 100Giimage: quay.io/prometheus/prometheus:v2.17.2nodeSelector:kubernetes.io/os: linuxpodMonitorNamespaceSelector: {}podMonitorSelector: {}replicas: 3 resources:requests:memory: 400MiruleSelector:matchLabels:prometheus: k8srole: alert-rulessecurityContext:fsGroup: 2000runAsNonRoot: truerunAsUser: 1000serviceAccountName: prometheus-k8sserviceMonitorNamespaceSelector: {}serviceMonitorSelector: {}version: v2.17.2additionalScrapeConfigs:name: additional-scrape-configskey: add.yaml
执行apply即可
$ kubectl apply -f prometheus-prometheus.yaml
其他系统的访问
参考文档
[Kube-prometheus部署Ingress为Prometheus-Grafana开启https_kube-prometheu配置ingress-CSDN博客](https://blog.csdn.net/Happy_Sunshine_Boy/article/details/107955691)Prometheus
基于访问路径过滤
修改yamlkube-prometheus-0.5.0/manifests/prometheus-prometheus.yaml
在参数下image: quay.io/prometheus/prometheus:v2.15.2添加如下参数
externalUrl: https://master170.k8s:30443/prometheuskubectl apply -f prometheus-prometheus.yaml配置ingress-tls.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:annotations:kubernetes.io/ingress.class: nginxnginx.ingress.kubernetes.io/use-regex: truenginx.ingress.kubernetes.io/enable-cors: truenginx.ingress.kubernetes.io/rewrite-target: /$2name: prometheus-k8snamespace: monitoring
spec:rules:- host: #写你的域名http:paths:- path: /prometheus(/|$)(.*)pathType: ImplementationSpecificbackend:service:name: prometheus-k8sport:number: 9090访问prometheus时都要带上“prometheus” 举例 https://master170.k8s:30443/prometheus/graph
AlertManager
修改yamlmanifests/alertmanager-alertmanager.yaml
在参数下image: quay.io/prometheus/alertmanager:v0.20.0添加如下参数
externalUrl: https://master170.k8s:30443/alertmanager配置ingress-tls.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:annotations:kubernetes.io/ingress.class: nginxnginx.ingress.kubernetes.io/use-regex: truenginx.ingress.kubernetes.io/enable-cors: truenginx.ingress.kubernetes.io/rewrite-target: /$2name: prometheus-k8snamespace: monitoring
spec:rules:- host: #写你的域名http:paths:# - path: /prometheus(/|$)(.*)# pathType: ImplementationSpecific# backend:# service:# name: prometheus-k8s# port:# number: 9090- path: /alertmanager(/|$)(.*)pathType: ImplementationSpecificbackend:service:name: alertmanager-mainport:number: 9093 访问alertmanager时都要带上“alertmanager” https://master170.k8s:30443/alertmanager/#/alerts
