环保网站建设开发,西安在线最新招聘信息,应用商店下载 安装正版,wordpress 4.5 安装失败#x1f49d;#x1f49d;#x1f49d;欢迎来到我的博客#xff0c;很高兴能够在这里和您见面#xff01;希望您在这里可以感受到一份轻松愉快的氛围#xff0c;不仅可以获得有趣的内容和知识#xff0c;也可以畅所欲言、分享您的想法和见解。 推荐:Linux运维老纪的首页… 欢迎来到我的博客很高兴能够在这里和您见面希望您在这里可以感受到一份轻松愉快的氛围不仅可以获得有趣的内容和知识也可以畅所欲言、分享您的想法和见解。 推荐:Linux运维老纪的首页,持续学习,不断总结,共同进步,活到老学到老 导航剑指大厂系列:全面总结 运维核心技术:系统基础、数据库、网路技术、系统安全、自动化运维、容器技术、监控工具、脚本编程、云服务等。 常用运维工具系列:常用的运维开发工具, zabbix、nagios、docker、k8s、puppet、ansible等 数据库系列:详细总结了常用数据库 mysql、Redis、MongoDB、oracle 技术点,以及工作中遇到的 mysql 问题等 懒人运维系列:总结好用的命令,解放双手不香吗?能用一个命令完成绝不用两个操作 数据结构与算法系列:总结数据结构和算法,不同类型针对性训练,提升编程思维,剑指大厂 非常期待和您一起在这个小小的网络世界里共同探索、学习和成长。 ✨✨ 欢迎订阅本专栏 ✨✨
基于k8s手动部署rabbitmq集群
1、RabbitMQ介绍
RabbitMQ是实现了高级消息队列协议AMQP的开源消息代理软件亦称面向消息的中间件。RabbitMQ服务器是用Erlang语言编写的而集群和故障转移是构建在开放电信平台框架上的。AMQPAdvanced Message Queue高级消息队列协议。它是应用层协议的一个开放标准为面向消息的中间件设计基于此协议的客户端与消息中间件可传递消息并不受产品、开发语言灯条件的限制
AMQP具有如下的特性
可靠性Reliablity使用了一些机制来保证可靠性比如持久化、传输确认、发布确认灵活的路由Flexible Routing在消息进入队列之前通过Exchange来路由消息。对于典型的路由功能Rabbit已经提供了一些内置的Exchange来实现。针对更复杂的路由功能可以将多个Exchange绑定在一起也通过插件机制实现自己的Exchange消息集群Clustering多个RabbitMQ服务器可以组成一个集群形成一个逻辑Broker高可用Highly Avaliable Queues队列可以在集群中的机器上进行镜像使得在部分节点出问题的情况下队列仍然可用多种协议Multi-protocol支持多种消息队列协议如STOMP、MQTT等多种语言客户端Many Clients几乎支持所有常用语言比如Java、.NET、Ruby等管理界面Management UI提供了易用的用户界面使得用户可以监控和管理消息Broker的许多方面跟踪机制Tracing如果消息异常RabbitMQ提供了消息的跟踪机制使用者可以找出发生了什么插件机制Plugin System提供了许多插件来从多方面进行扩展也可以编辑自己的插件
2、RabbitMQ的持久化和镜像队列
RabbitMQ持久化分为Exchange、Queue、Message
Exchange和Queue持久化指持久化Exchange、Queue元数据持久化的是自身服务宕机Exchange和Queue自身就没有了Message持久化顾名思义就是把每一条消息体持久化服务宕机消息不丢失
RabbitMQ的队列Queue镜像指master node在接受到请求后会同步到其他节点上以此来保证高可用。在confirm模式下具体过程如下
clientpublisher发送消息 – master node接到消息 – master node将消息持久化到磁盘 – 将消息异步发送给其他节点 – master将ack返回给client publisher3、RabbitMQ集群在k8s中的部署
将RabbitMQ以集群的方式部署在k8s中前提是RabbitMQ的每个节点都能像传统方式一样进行相互的服务发现。因此RabbitMQ在k8s集群中通过rabbitmq_peer_discovery_k8s plugin与k8s apiserver进行交互获取各个服务的URL且RabbitMQ在k8s集群中必须用statefulset和headless service进行匹配
需要注意的是rabbitmq_peer_discovery_k8s是RabbitMQ官方基于第三方开源项目rabbitmq-autocluster开发对3.7.X及以上版本提供的Kubernetes下的对等发现插件可实现rabbitmq集群在k8s中的自动化部署因此低于3.7.X版本请使用rabbitmq-autocluster
3.1 环境介绍
本文部署的版本是3.8.3
默认部署在default命名空间下
持久化存储为storageclass动态存储底层为nfs提供
镜像地址rabbitmq:3.8.3-management
以下yaml参考自官方示例https://github.com/rabbitmq/diy-kubernetes-examples
3.2 创建configmap
01-rabbitmq-configmap.yaml
kind: ConfigMap
apiVersion: v1
metadata:name: rabbitmq-cluster-confignamespace: defaultlabels:addonmanager.kubernetes.io/mode: Reconcile
data:enabled_plugins: |[rabbitmq_management,rabbitmq_peer_discovery_k8s].rabbitmq.conf: |default_user admindefault_pass 123!### Cluster formation. See https://www.rabbitmq.com/cluster-formation.html to learn more.cluster_formation.peer_discovery_backend rabbit_peer_discovery_k8scluster_formation.k8s.host kubernetes.default.svc.cluster.local## Should RabbitMQ node name be computed from the pods hostname or IP address?## IP addresses are not stable, so using [stable] hostnames is recommended when possible.## Set to hostname to use pod hostnames.## When this value is changed, so should the variable used to set the RABBITMQ_NODENAME## environment variable.cluster_formation.k8s.address_type hostname## How often should node cleanup checks run?cluster_formation.node_cleanup.interval 30## Set to false if automatic removal of unknown/absent nodes## is desired. This can be dangerous, see## * https://www.rabbitmq.com/cluster-formation.html#node-health-checks-and-cleanup## * https://groups.google.com/forum/#!msg/rabbitmq-users/wuOfzEywHXo/k8z_HWIkBgAJcluster_formation.node_cleanup.only_log_warning truecluster_partition_handling autoheal## See https://www.rabbitmq.com/ha.html#master-migration-data-localityqueue_master_locatormin-masters## See https://www.rabbitmq.com/access-control.html#loopback-usersloopback_users.guest falsecluster_formation.randomized_startup_delay_range.min 0cluster_formation.randomized_startup_delay_range.max 2# default is rabbitmq-clusters namespace# hostname_suffixcluster_formation.k8s.hostname_suffix .rabbitmq-cluster.default.svc.cluster.local# memoryvm_memory_high_watermark.absolute 1GB# diskdisk_free_limit.absolute 2GB部分参数说明
enabled_plugins声明开启的插件名default_pass/default_pass声明用户名和密码虽然有部分文章记录可以通过环境变量的方式声明但是经测试针对此版本如果指定了configmap即rabbitmq的配置文件声明的环境变量是没有用的都需要在配置文件中指定cluster_formation.k8s.address_type从k8s返回的Pod容器列表中计算对等节点列表这里只能使用主机名官方示例中是ip但是默认情况下在k8s中pod的ip都是不固定的因此可能导致节点的配置和数据丢失后面的yaml中会通过引用元数据的方式固定pod的主机名。
更多参数请参考官方文档这里就不赘述了
3.3 创建service
02-rabbitmq-service.yaml
kind: Service
apiVersion: v1
metadata:labels:app: rabbitmq-clustername: rabbitmq-clusternamespace: default
spec:clusterIP: Noneports:- name: rmqportport: 5672targetPort: 5672selector:app: rabbitmq-cluster---
kind: Service
apiVersion: v1
metadata:labels:app: rabbitmq-clustername: rabbitmq-cluster-managenamespace: default
spec:ports:- name: httpport: 15672protocol: TCPtargetPort: 15672selector:app: rabbitmq-clustertype: NodePort
上面定义了两个Service一个是rabbitmq的服务端口一个是管理界面的端口用户外部访问这里通过NodePort方式进行暴露
3.4 创建rbac授权
前面的介绍中提到了RabbitMQ通过插件与k8s apiserver交互获得集群中节点相关信息因此需要对其进行RBAC授权
03-rabbitmq-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:name: rabbitmq-clusternamespace: default
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:name: rabbitmq-clusternamespace: default
rules:
- apiGroups: []resources: [endpoints]verbs: [get]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:name: rabbitmq-clusternamespace: default
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: rabbitmq-cluster
subjects:
- kind: ServiceAccountname: rabbitmq-clusternamespace: default3.5 创建statefulset
RabbitMQ在k8s中作为一个有状态应用进行部署因此控制器类型为StatefulSetyaml中还定义了pvc相关内容
04-rabbitmq-cluster-sts.yaml
kind: StatefulSet
apiVersion: apps/v1
metadata:labels:app: rabbitmq-clustername: rabbitmq-clusternamespace: default
spec:replicas: 3selector:matchLabels:app: rabbitmq-clusterserviceName: rabbitmq-clustertemplate:metadata:labels:app: rabbitmq-clusterspec:containers:- args:- -c- cp -v /etc/rabbitmq/rabbitmq.conf ${RABBITMQ_CONFIG_FILE}; exec docker-entrypoint.shrabbitmq-servercommand:- shenv:- name: TZvalue: Asia/Shanghai- name: RABBITMQ_ERLANG_COOKIEvalue: SWvCP0Hrqv43NG7GybHC95ntCJKoW8UyNFWnBEWG8TY- name: K8S_SERVICE_NAMEvalue: rabbitmq-cluster- name: POD_IPvalueFrom:fieldRef:fieldPath: status.podIP- name: POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: RABBITMQ_USE_LONGNAMEvalue: true- name: RABBITMQ_NODENAMEvalue: rabbit$(POD_NAME).$(K8S_SERVICE_NAME).$(POD_NAMESPACE).svc.cluster.local- name: RABBITMQ_CONFIG_FILEvalue: /var/lib/rabbitmq/rabbitmq.confimage: rabbitmq:3.8.3-managementimagePullPolicy: IfNotPresentlivenessProbe:exec:command:- rabbitmq-diagnostics- status# See https://www.rabbitmq.com/monitoring.html for monitoring frequency recommendations.initialDelaySeconds: 60periodSeconds: 60timeoutSeconds: 15name: rabbitmqports:- containerPort: 15672name: httpprotocol: TCP- containerPort: 5672name: amqpprotocol: TCPreadinessProbe:exec:command:- rabbitmq-diagnostics- statusinitialDelaySeconds: 20periodSeconds: 60timeoutSeconds: 10volumeMounts:- mountPath: /etc/rabbitmqname: config-volumereadOnly: false- mountPath: /var/lib/rabbitmqname: rabbitmq-storagereadOnly: false- name: timezonemountPath: /etc/localtimereadOnly: trueserviceAccountName: rabbitmq-clusterterminationGracePeriodSeconds: 30volumes:- name: config-volumeconfigMap:items:- key: rabbitmq.confpath: rabbitmq.conf- key: enabled_pluginspath: enabled_pluginsname: rabbitmq-cluster-config- name: timezonehostPath:path: /usr/share/zoneinfo/Asia/ShanghaivolumeClaimTemplates:- metadata:name: rabbitmq-storagespec:accessModes:- ReadWriteManystorageClassName: managed-nfs-storageresources:requests:storage: 2Gi3.6 部署检查
➜ rabbitmq-cluster pwd
/Users/ssgeek/Documents/k8s-manifests/rabbitmq-cluster
➜ rabbitmq-cluster ls
01-rabbitmq-configmap.yaml 02-rabbitmq-service.yaml 03-rabbitmq-rbac.yaml 04-rabbitmq-cluster-sts.yaml
➜ rabbitmq-cluster kubectl apply -f .
configmap/rabbitmq-cluster-config created
service/rabbitmq-cluster created
service/rabbitmq-cluster-manage created
serviceaccount/rabbitmq-cluster created
role.rbac.authorization.k8s.io/rabbitmq-cluster created
rolebinding.rbac.authorization.k8s.io/rabbitmq-cluster created
statefulset.apps/rabbitmq-cluster created等待一段时间查看创建的相关资源
➜ rabbitmq-cluster kubectl get po,sts -l apprabbitmq-cluster
NAME READY STATUS RESTARTS AGE
pod/rabbitmq-cluster-0 1/1 Running 0 3m22s
pod/rabbitmq-cluster-1 1/1 Running 0 2m18s
pod/rabbitmq-cluster-2 1/1 Running 0 2m23sNAME READY AGE
statefulset.apps/rabbitmq-cluster 3/3 3m25s查看日志从日志的最后部分观察集群建立的状态
➜ rabbitmq-cluster kubectl logs -f rabbitmq-cluster-0
/etc/rabbitmq/rabbitmq.conf - /var/lib/rabbitmq/rabbitmq.conf
2021-02-17 03:30:39.445 [info] 0.9.0 Feature flags: list of feature flags found:
2021-02-17 03:30:39.445 [info] 0.9.0 Feature flags: [ ] drop_unroutable_metric
2021-02-17 03:30:39.445 [info] 0.9.0 Feature flags: [ ] empty_basic_get_metric
2021-02-17 03:30:39.445 [info] 0.9.0 Feature flags: [ ] implicit_default_bindings
2021-02-17 03:30:39.445 [info] 0.9.0 Feature flags: [ ] quorum_queue
2021-02-17 03:30:39.445 [info] 0.9.0 Feature flags: [ ] virtual_host_metadata
2021-02-17 03:30:39.445 [info] 0.9.0 Feature flags: feature flag states written to disk: yes
2021-02-17 03:30:39.544 [info] 0.269.0 ra: meta data store initialised. 0 record(s) recovered
2021-02-17 03:30:39.547 [info] 0.274.0 WAL: recovering []
2021-02-17 03:31:10.676 [info] 0.313.0 Starting RabbitMQ 3.8.3 on Erlang 22.3.4.1Copyright (c) 2007-2020 Pivotal Software, Inc.Licensed under the MPL 1.1. Website: https://rabbitmq.com## ## RabbitMQ 3.8.3## ############ Copyright (c) 2007-2020 Pivotal Software, Inc.###### ############ Licensed under the MPL 1.1. Website: https://rabbitmq.comDoc guides: https://rabbitmq.com/documentation.htmlSupport: https://rabbitmq.com/contact.htmlTutorials: https://rabbitmq.com/getstarted.htmlMonitoring: https://rabbitmq.com/monitoring.htmlLogs: stdoutConfig file(s): /var/lib/rabbitmq/rabbitmq.confStarting broker...2021-02-17 03:31:10.678 [info] 0.313.0 node : rabbitrabbitmq-cluster-0.rabbitmq-cluster.default.svc.cluster.localhome dir : /var/lib/rabbitmqconfig file(s) : /var/lib/rabbitmq/rabbitmq.confcookie hash : HIQL2spD4MDV4jPi7mMAglog(s) : stdoutdatabase dir : /var/lib/rabbitmq/mnesia/rabbitrabbitmq-cluster-0.rabbitmq-cluster.default.svc.cluster.local
2021-02-17 03:31:10.695 [info] 0.313.0 Running boot step pre_boot defined by app rabbit...省略中间内容2021-02-17 03:31:13.273 [info] 0.824.0 Statistics database started.
2021-02-17 03:31:13.273 [info] 0.823.0 Starting worker pool management_worker_pool with 3 processes in it
2021-02-17 03:31:13.467 [info] 0.9.0 Server startup complete; 5 plugins started.* rabbitmq_peer_discovery_k8s* rabbitmq_management* rabbitmq_web_dispatch* rabbitmq_management_agent* rabbitmq_peer_discovery_commoncompleted with 5 plugins.
2021-02-17 03:32:30.060 [info] 0.566.0 node rabbitrabbitmq-cluster-1.rabbitmq-cluster.default.svc.cluster.local up
2021-02-17 03:32:31.264 [info] 0.566.0 rabbit on node rabbitrabbitmq-cluster-1.rabbitmq-cluster.default.svc.cluster.local up
2021-02-17 03:33:31.280 [info] 0.566.0 node rabbitrabbitmq-cluster-2.rabbitmq-cluster.default.svc.cluster.local up
2021-02-17 03:33:32.627 [info] 0.566.0 rabbit on node rabbitrabbitmq-cluster-2.rabbitmq-cluster.default.svc.cluster.local up进入到pod中通过客户端查看集群状态
➜ rabbitmq-cluster kubectl exec -it rabbitmq-cluster-0 bash
rootrabbitmq-cluster-0:/# rabbitmqctl cluster_status
Cluster status of node rabbitrabbitmq-cluster-0.rabbitmq-cluster.default.svc.cluster.local ...
BasicsCluster name: rabbitrabbitmq-cluster-0.rabbitmq-cluster.default.svc.cluster.localDisk Nodesrabbitrabbitmq-cluster-0.rabbitmq-cluster.default.svc.cluster.local
rabbitrabbitmq-cluster-1.rabbitmq-cluster.default.svc.cluster.local
rabbitrabbitmq-cluster-2.rabbitmq-cluster.default.svc.cluster.localRunning Nodesrabbitrabbitmq-cluster-0.rabbitmq-cluster.default.svc.cluster.local
rabbitrabbitmq-cluster-1.rabbitmq-cluster.default.svc.cluster.local
rabbitrabbitmq-cluster-2.rabbitmq-cluster.default.svc.cluster.localVersionsrabbitrabbitmq-cluster-0.rabbitmq-cluster.default.svc.cluster.local: RabbitMQ 3.8.3 on Erlang 22.3.4.1
rabbitrabbitmq-cluster-1.rabbitmq-cluster.default.svc.cluster.local: RabbitMQ 3.8.3 on Erlang 22.3.4.1
rabbitrabbitmq-cluster-2.rabbitmq-cluster.default.svc.cluster.local: RabbitMQ 3.8.3 on Erlang 22.3.4.1Alarms(none)Network Partitions(none)ListenersNode: rabbitrabbitmq-cluster-0.rabbitmq-cluster.default.svc.cluster.local, interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication
Node: rabbitrabbitmq-cluster-0.rabbitmq-cluster.default.svc.cluster.local, interface: [::], port: 5672, protocol: amqp, purpose: AMQP 0-9-1 and AMQP 1.0
Node: rabbitrabbitmq-cluster-0.rabbitmq-cluster.default.svc.cluster.local, interface: [::], port: 15672, protocol: http, purpose: HTTP API
Node: rabbitrabbitmq-cluster-1.rabbitmq-cluster.default.svc.cluster.local, interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication
Node: rabbitrabbitmq-cluster-1.rabbitmq-cluster.default.svc.cluster.local, interface: [::], port: 5672, protocol: amqp, purpose: AMQP 0-9-1 and AMQP 1.0
Node: rabbitrabbitmq-cluster-1.rabbitmq-cluster.default.svc.cluster.local, interface: [::], port: 15672, protocol: http, purpose: HTTP API
Node: rabbitrabbitmq-cluster-2.rabbitmq-cluster.default.svc.cluster.local, interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication
Node: rabbitrabbitmq-cluster-2.rabbitmq-cluster.default.svc.cluster.local, interface: [::], port: 5672, protocol: amqp, purpose: AMQP 0-9-1 and AMQP 1.0
Node: rabbitrabbitmq-cluster-2.rabbitmq-cluster.default.svc.cluster.local, interface: [::], port: 15672, protocol: http, purpose: HTTP APIFeature flagsFlag: drop_unroutable_metric, state: enabled
Flag: empty_basic_get_metric, state: enabled
Flag: implicit_default_bindings, state: enabled
Flag: quorum_queue, state: enabled
Flag: virtual_host_metadata, state: enabled通过NodePort访问管理界面
kubectl get svc -l apprabbitmq-cluster
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
rabbitmq-cluster ClusterIP None none 5672/TCP 8m47s
rabbitmq-cluster-manage NodePort 10.1.239.191 none 15672:30888/TCP 8m47s到这里在k8s中手动部署一个RabbitMQ集群就完成啦~
