黄石城乡建设网站,项目经理岗位职责,如何修改网站后台,网站布局有哪些背景
基于前面搭建的3节点 Kubernetes 集群#xff0c;今天我们使用 Registry2 搭建私有镜像仓库#xff0c;这在镜像安全性以及离线环境下运维等方面具有重要意义。
Note: 由于是测试环境#xff0c;以下创建了一个 local-storage 的 StorageClass #xff0c;并使用本地…背景
基于前面搭建的3节点 Kubernetes 集群今天我们使用 Registry2 搭建私有镜像仓库这在镜像安全性以及离线环境下运维等方面具有重要意义。
Note: 由于是测试环境以下创建了一个 local-storage 的 StorageClass 并使用本地磁盘的方式创建使用 PV 实际建议使用 NFS 。
虚机资源
共用到了三台虚机1台作为 Master 节点2台 Worker 节点。
主机名IP说明k8s-master172.16.201.25主节点k8s-node1172.16.201.26工作节点k8s-node2172.16.201.27工作节点
[rootk8s-master ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready control-plane,master 37h v1.20.9
k8s-node1 Ready none 35h v1.20.9
k8s-node2 Ready none 35h v1.20.9系统环境
[rootk8s-master ~]# uname -a
Linux k8s-master 3.10.0-1160.71.1.el7.x86_64 #1 SMP Tue Jun 28 15:37:28 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
[rootk8s-master ~]# cat /proc/version
Linux version 3.10.0-1160.71.1.el7.x86_64 (mockbuildkbuilder.bsys.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) ) #1 SMP Tue Jun 28 15:37:28 UTC 2022
[rootk8s-master ~]# cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)使用Registry2搭建私有镜像仓库
采用 YAML 文件的方式搭建私有 Docker Registry 的整体流程
创建一个Docker Registry的命名空间创建一个持久卷来存储Docker Registry的数据创建一个Secret用于存储Docker Registry的凭证创建一个Deployment来部署Docker Registry创建一个Service来暴露Docker Registry登录并推送镜像到私有镜像仓库
创建一个Docker Registry的命名空间
apiVersion: v1
kind: Namespace
metadata:name: docker-registry创建一个持久卷来存储Docker Registry的数据
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:name: local-storagenamespace: docker-registry
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Retain---apiVersion: v1
kind: PersistentVolume
metadata:name: docker-registry-pvlabels:pv: docker-registry-pv
spec:capacity: storage: 5GiaccessModes:- ReadWriteManypersistentVolumeReclaimPolicy: RetainstorageClassName: local-storagelocal:path: /data/dockernodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- k8s-node1---apiVersion: v1
kind: PersistentVolumeClaim
metadata:name: docker-registry-pvcnamespace: docker-registry
spec:resources:requests:storage: 5GiaccessModes:- ReadWriteManystorageClassName: local-storageselector:matchLabels:pv: docker-registry-pvNote
没有必要将namespace字段添加到kind: persistantVolume因为PersistentVolumes绑定(bind)是排他的出于安全考虑Pod不会被调度到Master Node上也就是说默认情况下Master节点taint污点不参与工作负载。如果没有配置nodeAffinity在创建PV时会报错 0/3 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn’t tolerate, 2 node(s) didn’t find available persistent volumes to bind. 解决方法添加 nodeAffinity 这里将 PV 创建到 k8s-node1 节点记得先在 Worker 节点上创建 PV 目录/data/docker
创建一个Secret用于存储Docker Registry的凭证
mkdir authdocker run --entrypoint htpasswd registry:2 -Bbn username password auth/htpasswd报错unable to start container process: exec: “htpasswd”: executable file not found in $PATH: unknown. 原因 registry 镜像在 2.x 相关版本中删除了 /usr/bin/htpasswd 文件导致创建容器时找不到可执行文件参考https://github.com/docker/distribution-library-image/issues/106
使用旧版本的 registry 或者在服务器本地安装 httpd 再执行生成密码的命令。
yum install httpd
htpasswd -Bbn username password auth/htpasswd
kubectl create secret generic docker-registry-secret --from-fileauth/htpasswd -n docker-registry# 创建了一个Opaque类型的Secret
[rootk8s-master ~]# kubectl get secret -n docker-registry
NAME TYPE DATA AGE
default-token-prfsz kubernetes.io/service-account-token 3 11m
docker-registry-secret Opaque 1 20s创建 Docker 私有镜像仓库的 secret 这里是在 docker-registry 命名空间如果新增了 namespace 那么这个 namespace 就需要单独添加一次 secret 而且这个 namespace 下拉取镜像时也需要添加 imagePullSecrets 。
kubectl create secret docker-registry ruoyi-registry-secret --docker-server10.96.198.223:5000 --docker-usernameusername --docker-passwordpassword -n docker-registry[rootk8s-master ~]# kubectl get secret -n docker-registry
NAME TYPE DATA AGE
default-token-prfsz kubernetes.io/service-account-token 3 2d23h
docker-registry-secret Opaque 1 2d23h
ruoyi-registry-secret kubernetes.io/dockerconfigjson 1 3sNote:
docker-registry-secret: 用于配置Registry的账号与域名并在控制台通过username与password登录、推送镜像到私有镜像仓库ruoyi-registry-secret: 用于通过在不同的Worker节点上拉取私有镜像。
将上述创建 NameSpace , StorageClass , PV 以及 PVC 的 YAML 合并为一个 docker-registry-ns-sc-pv-pvc.yaml 并执行。
kubectl apply -f docker-registry-ns-sc-pv-pvc.yaml# 查看sc,pv,pvc状态
[rootk8s-master registry]# kubectl get sc,pv,pvc -n docker-registry
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
storageclass.storage.k8s.io/local-storage kubernetes.io/no-provisioner Retain WaitForFirstConsumer false 4m45sNAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/docker-registry-pv 5Gi RWX Retain Available local-storage 4m45sNAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/docker-registry-pvc Pending local-storage 4m45s看到 PVC 一直是 Pending 状态。
# 查看PVC详细信息
[rootk8s-master registry]# kubectl describe pvc -n docker-registry
Name: docker-registry-pvc
Namespace: docker-registry
StorageClass: local-storage
Status: Pending
Volume:
Labels: none
Annotations: none
Finalizers: [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode: Filesystem
Used By: none
Events:Type Reason Age From Message---- ------ ---- ---- -------Normal WaitForFirstConsumer 9s (x26 over 6m13s) persistentvolume-controller waiting for first consumer to be created before binding提示需要有个消费者进行关联后面应用 Deployment 后 PVC 状态会变为 Binding 。
创建一个Deployment来部署Docker Registry
apiVersion: apps/v1
kind: Deployment
metadata:labels:app: docker-registryname: docker-registrynamespace: docker-registry
spec:replicas: 1revisionHistoryLimit: 5selector:matchLabels:app: docker-registrytemplate:metadata:labels:app: docker-registryspec:securityContext:runAsUser: 0containers:- name: docker-registry image: registry:2 imagePullPolicy: IfNotPresentports:- containerPort: 5000name: webprotocol: TCPresources:requests:memory: 200Micpu: 0.1terminationMessagePath: /dev/termination-logterminationMessagePolicy: Fileenv:- name: REGISTRY_AUTHvalue: htpasswd- name: REGISTRY_AUTH_HTPASSWD_REALMvalue: Registry Realm- name: REGISTRY_AUTH_HTPASSWD_PATHvalue: /auth/htpasswdvolumeMounts:- name: docker-registry-datamountPath: /var/lib/registry/- name: docker-registry-authmountPath: /authreadOnly: truevolumes:- name: docker-registry-datapersistentVolumeClaim:claimName: docker-registry-pvc- name: docker-registry-authsecret:secretName: docker-registry-secret创建一个Service来暴露Docker Registry
apiVersion: v1
kind: Service
metadata:name: docker-registry-servicenamespace: docker-registry
spec:ports:- name: port-nameport: 5000 protocol: TCPtargetPort: 5000selector:app: docker-registrytype: NodePort部署 Deployment 与 Service 。
kubectl apply -f docker-registry-deploy-svc.yaml[rootk8s-master registry]# kubectl get deploy -n docker-registry
NAME READY UP-TO-DATE AVAILABLE AGE
docker-registry 1/1 1 1 21s[rootk8s-master registry]# kubectl get svc -n docker-registry
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
docker-registry-service NodePort 10.96.198.223 none 5000:30858/TCP 15s登录私有镜像仓库
# 直接查看下镜像仓库的内容提示未认证
[rootk8s-master ~]# curl http://10.96.198.223:5000/v2/_catalog
{errors:[{code:UNAUTHORIZED,message:authentication required,detail:[{Type:registry,Class:,Name:catalog,Action:*}]}]}# 使用用户名、密码登录报错http: server gave HTTP response to HTTPS client
[rootk8s-master ~]# docker login -uusername -ppassword 10.96.198.223:5000
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get https://10.96.198.223:5000/v2/: http: server gave HTTP response to HTTPS client# 修改配置文件添加insecure-registries
[rootk8s-master ~]# vi /etc/docker/daemon.json
insecure-registries: [10.96.198.223:5000]# 重启Docker服务
[rootk8s-master ~]# systemctl daemon-reload
[rootk8s-master ~]# systemctl restart docker# 再次使用用户名、密码登录成功
[rootk8s-master registry]# docker login -uusername -ppassword 10.96.198.223:5000
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded# 会自动生成~/.docker/config.json文件
[rootk8s-master ~]# cat ~/.docker/config.json
{auths: {10.96.198.223:5000: {auth: dXNlcm5hbWU6cGFzc3dvcmQ}}
}# 附带用户名、密码获取仓库内镜像为空
[rootk8s-master ~]# curl -u username:password http://10.96.198.223:5000/v2/_catalog
{repositories:[]}推送镜像到私有仓库
# 以hello-world为例测试推送镜像到私有仓库功能
[rootk8s-master ~]# docker run hello-world
[rootk8s-master ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
calico/node v3.20.6 daeec7e26e1f 17 months ago 156MB
calico/pod2daemon-flexvol v3.20.6 39b166f3f936 17 months ago 18.6MB
calico/cni v3.20.6 13b6f63a50d6 17 months ago 138MB
calico/kube-controllers v3.20.6 4dc6e7685020 17 months ago 60.2MB
registry 2 b8604a3fe854 2 years ago 26.2MB
hello-world latest feb5d9fea6a5 2 years ago 13.3kB
registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/kube-proxy v1.20.9 8dbf9a6aa186 2 years ago 99.7MB
registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/kube-scheduler v1.20.9 295014c114b3 2 years ago 47.3MB
registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/kube-controller-manager v1.20.9 eb07fd4ad3b4 2 years ago 116MB
registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/kube-apiserver v1.20.9 0d0d57e4f64c 2 years ago 122MB
registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/etcd 3.4.13-0 0369cf4303ff 3 years ago 253MB
registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/coredns 1.7.0 bfe3a36ebd25 3 years ago 45.2MB
registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/pause 3.2 80d28bedfe5d 3 years ago 683kB# 对hello-world镜像重新打标签
[rootk8s-master ~]# docker tag hello-world 10.96.198.223:5000/hello-world:1
[rootk8s-master ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
calico/node v3.20.6 daeec7e26e1f 17 months ago 156MB
calico/pod2daemon-flexvol v3.20.6 39b166f3f936 17 months ago 18.6MB
calico/cni v3.20.6 13b6f63a50d6 17 months ago 138MB
calico/kube-controllers v3.20.6 4dc6e7685020 17 months ago 60.2MB
registry 2 b8604a3fe854 2 years ago 26.2MB
10.96.198.223:5000/hello-world 1 feb5d9fea6a5 2 years ago 13.3kB
hello-world latest feb5d9fea6a5 2 years ago 13.3kB
registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/kube-proxy v1.20.9 8dbf9a6aa186 2 years ago 99.7MB
registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/kube-scheduler v1.20.9 295014c114b3 2 years ago 47.3MB
registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/kube-controller-manager v1.20.9 eb07fd4ad3b4 2 years ago 116MB
registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/kube-apiserver v1.20.9 0d0d57e4f64c 2 years ago 122MB
registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/etcd 3.4.13-0 0369cf4303ff 3 years ago 253MB
registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/coredns 1.7.0 bfe3a36ebd25 3 years ago 45.2MB
registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/pause 3.2 80d28bedfe5d 3 years ago 683kB# 推送到私有镜像仓库成功
[rootk8s-master ~]# docker push 10.96.198.223:5000/hello-world:1
The push refers to repository [10.96.198.223:5000/hello-world]
e07ee1baac5f: Pushed
1: digest: sha256:f54a58bc1aac5ea1a25d796ae155dc228b3f0e11d046ae276b39c4bf2f13d8c4 size: 525# 附带用户名、密码获取仓库内镜像
[rootk8s-master ~]# curl -u username:password http://10.96.198.223:5000/v2/_catalog
{repositories:[hello-world]}# 退出操作
[rootk8s-master ~]# docker logout 10.96.198.223:5000安装Registry前端
直接通过 Docker 安装 Registry 前端通过用户名与密码登录后可以网页展示已推送的私有镜像。
docker run -d -e ENV_DOCKER_REGISTRY_HOST10.96.198.223 -e ENV_DOCKER_REGISTRY_PORT5000 -p 5001:80 konradkleine/docker-registry-frontend:v2认证 私有镜像列表 小总结
作为测试环境本次使用 Registry2 搭建了私有镜像仓库实际生产环境建议使用 Harbor 。 Registry2 和 Harbor 都是用于 Docker 镜像存储和分发的解决方案但它们各自具有不同的特点
Registry2
基本功能Registry2通常称为Docker Registry是Docker官方的开源镜像仓库提供了存储和分发Docker镜像的基本功能。简单轻量它比较轻量级适合需要快速部署的场景。自主部署可以自主部署在私有环境中满足私有化需求。缺少高级功能相比于HarborRegistry2缺少一些高级功能如图形用户界面、角色基础的访问控制、镜像扫描和签名等。
Harbor
企业级特性Harbor是一个开源的企业级Docker Registry解决方案提供了Registry2的所有基本功能并增加了许多企业级特性。图形用户界面Harbor提供了用户友好的图形界面便于管理和浏览镜像。访问控制支持基于角色的访问控制可以细粒度地管理用户权限。安全性提供了镜像扫描和签名功能增强了镜像的安全性。高可用性支持高可用部署适合企业级应用。
总结来说如果你需要一个简单的、轻量级的 Docker 镜像仓库 Registry2 可能是一个不错的选择。而如果你需要更多的企业级特性如图形界面、细粒度的访问控制、镜像安全扫描等 Harbor 会是更好的选择。 If you have any questions or any bugs are found, please feel free to contact me.
Your comments and suggestions are welcome!
