Air-gappedを想定した環境上にTanzu RabbitMQ for Kubernetesのインストールするメモです。
目次
- Kind Clusterの作成
- Harborのインストール
- Harbor ProjectとRobot Accountsのセットアップ
- CLIのインストール
- Cluster Essentialsのインストール
- Package Repositoryの登録
- Cert Managerパッケージのインストール
- RabbitMQ Operatorパッケージのインストール
- RabbitMQ Clusterの作成
- アプリのデプロイ
- RabbitMQ ClusterのTLS対応
- アプリのRabbitMQ TLS対応
- リソースの削除
Kind Clusterの作成
本記事では、OrbStack上にkindで3 workersのKubernetesクラスタを作成します。
他のKubernetesを使っても基本的な作業は変わりません。
image=kindest/node:v1.30.4@sha256:976ea815844d5fa93be213437e3ff5754cd599b040946b5cca43ca45c2047114
cat <<EOF > kind.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
image: $image
- role: worker
image: $image
- role: worker
image: $image
- role: worker
image: $image
EOF
kind create cluster --config kind.yaml
作成されたnodeを確認します。
$ kubectl get node -A -owide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
kind-control-plane Ready control-plane 5m11s v1.30.4 192.168.228.3 <none> Debian GNU/Linux 12 (bookworm) 6.10.11-orbstack-00280-g1304bd068592 containerd://1.7.18
kind-worker Ready <none> 4m50s v1.30.4 192.168.228.4 <none> Debian GNU/Linux 12 (bookworm) 6.10.11-orbstack-00280-g1304bd068592 containerd://1.7.18
kind-worker2 Ready <none> 4m50s v1.30.4 192.168.228.2 <none> Debian GNU/Linux 12 (bookworm) 6.10.11-orbstack-00280-g1304bd068592 containerd://1.7.18
kind-worker3 Ready <none> 4m50s v1.30.4 192.168.228.5 <none> Debian GNU/Linux 12 (bookworm) 6.10.11-orbstack-00280-g1304bd068592 containerd://1.7.18
OrbStack上にkindでtype=LoadBalancerのServiceを利用できるように、MetalLBをインストールします。
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.8/config/manifests/metallb-native.yaml
kubectl wait --namespace metallb-system \
--for=condition=ready pod \
--selector=app=metallb \
--timeout=90s
SUBNET=$(docker network inspect -f '{{(index (index .IPAM.Config 0) "Subnet")}}' kind)
cat <<EOF > metallb.yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: example
namespace: metallb-system
spec:
addresses:
- $(echo $SUBNET | cut -d. -f1-3).200-$(echo $SUBNET | cut -d. -f1-3).250
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: empty
namespace: metallb-system
EOF
kubectl apply -f metallb.yaml
Harborのインストール
コンテナレジストリとしてHarborをインストールします。
helm repo add harbor https://helm.goharbor.io
helm repo update
SUBNET=$(docker network inspect -f '{{(index (index .IPAM.Config 0) "Subnet")}}' kind)
HARBOR_IP=$(echo $SUBNET | cut -d. -f1-3).210
HelmでHarborをインストールします。
helm upgrade --install \
-n harbor harbor harbor/harbor \
--set expose.type=loadBalancer \
--set expose.tls.auto.commonName=harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io \
--set expose.loadBalancer.IP=${HARBOR_IP} \
--set externalURL=https://harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io \
--set persistence.persistentVolumeClaim.registry.size=100Gi \
--create-namespace \
--wait
HarborのIPを確認します。
$ kubectl get svc -n harbor harbor
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
harbor LoadBalancer 10.96.45.148 192.168.228.210 80:30370/TCP,443:32730/TCP 2m31s
HarborのCA証明書をMac上で信頼させます。
kubectl get secret -n harbor harbor-nginx -otemplate='{{index .data "ca.crt" | base64decode}}' > harbor.ca
sudo security add-trusted-cert -d -r trustRoot -p ssl -p codeSign -k /Library/Keychains/System.keychain harbor.ca
OrbStackを再起動し、kindのコンテナも起動します。
$ kubectl get pod -A
NAMESPACE NAME READY STATUS RESTARTS AGE
harbor harbor-core-7fc48d6f8c-tqzx7 1/1 Running 1 (11m ago) 25m
harbor harbor-database-0 1/1 Running 1 (11m ago) 25m
harbor harbor-jobservice-577c54dfd8-j6r94 1/1 Running 5 (10m ago) 25m
harbor harbor-nginx-57d7bc9f5d-hc9xl 1/1 Running 3 (10m ago) 25m
harbor harbor-portal-d6c99f896-2v8zm 1/1 Running 1 (11m ago) 25m
harbor harbor-redis-0 1/1 Running 1 (11m ago) 25m
harbor harbor-registry-59f445685-7ltfq 2/2 Running 2 (11m ago) 25m
harbor harbor-trivy-0 1/1 Running 1 (11m ago) 25m
kube-system coredns-7db6d8ff4d-hb9x6 1/1 Running 1 (11m ago) 40m
kube-system coredns-7db6d8ff4d-wgxjl 1/1 Running 1 (11m ago) 40m
kube-system etcd-kind-control-plane 1/1 Running 0 11m
kube-system kindnet-ltt8c 1/1 Running 1 (11m ago) 40m
kube-system kindnet-rjxkc 1/1 Running 1 (11m ago) 40m
kube-system kindnet-tfnbl 1/1 Running 1 (11m ago) 40m
kube-system kindnet-vgfs4 1/1 Running 1 (11m ago) 40m
kube-system kube-apiserver-kind-control-plane 1/1 Running 0 11m
kube-system kube-controller-manager-kind-control-plane 1/1 Running 1 (11m ago) 40m
kube-system kube-proxy-4rsjg 1/1 Running 1 (11m ago) 40m
kube-system kube-proxy-8ntlv 1/1 Running 1 (11m ago) 40m
kube-system kube-proxy-qdpnd 1/1 Running 1 (11m ago) 40m
kube-system kube-proxy-w6llt 1/1 Running 1 (11m ago) 40m
kube-system kube-scheduler-kind-control-plane 1/1 Running 1 (11m ago) 40m
local-path-storage local-path-provisioner-7d4d9bdcc5-47d6c 1/1 Running 2 (10m ago) 40m
metallb-system controller-6dd967fdc7-5tlwj 1/1 Running 3 (11m ago) 30m
metallb-system speaker-fs822 1/1 Running 2 (10m ago) 30m
metallb-system speaker-hnqnr 1/1 Running 2 (10m ago) 30m
metallb-system speaker-m4f9x 1/1 Running 2 (10m ago) 30m
metallb-system speaker-q9jhr 1/1 Running 2 (10m ago) 30m
HarborにDockerログインできることを確認します。
$ docker login harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io -u admin -p Harbor12345
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Login Succeeded
HarborのCA証明書をKindの各ノードに信頼させます。
for NODE in $(kubectl get node | grep -v NAME | awk '{print $1}');do
docker exec $NODE mkdir -p /etc/containerd/certs.d
docker cp harbor.ca $NODE:/etc/containerd/certs.d/
cat <<EOF | docker exec -i $NODE tee -a /etc/containerd/config.toml
[plugins."io.containerd.grpc.v1.cri".registry.configs."harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io".tls]
ca_file = "/etc/containerd/certs.d/harbor.ca"
EOF
docker exec $NODE systemctl restart containerd
done
設定を確認します。
$ docker exec kind-control-plane crictl info | jq .config.registry.configs
{
"harbor.192-168-228-210.sslip.io": {
"auth": null,
"tls": {
"insecure_skip_verify": false,
"caFile": "/etc/containerd/certs.d/harbor.ca",
"certFile": "",
"keyFile": ""
}
}
}
HarborからpullしたイメージでPodを起動できるか動作確認します。
imgpkg copy -i nginx:alpine --to-repo harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/library/nginx
kubectl run nginx --image=harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/library/nginx:alpine
Harbor ProjectとRobot Accountsのセットアップ
以下でRelocateする、Cluster Essentials、Tanzu Standard、Tanzu RabbitMQのパッケージ用にそれぞれHarborのプロジェクトを作成します。Tanzu Standardのみpublicアクセスを許可します。
curl -k -u admin:Harbor12345 https://harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/api/v2.0/projects -H "Content-Type: application/json" -d '{"project_name": "tanzu-cluster-essentials", "public": false}'
curl -k -u admin:Harbor12345 https://harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/api/v2.0/projects -H "Content-Type: application/json" -d '{"project_name": "tanzu-rabbitmq", "public": false}'
curl -k -u admin:Harbor12345 https://harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/api/v2.0/projects -H "Content-Type: application/json" -d '{"project_name": "tkg", "public": true}'
Cluster EssentialsとTanzu RabbitMQをpullするRead OnlyのRobot Accountを作成します。
curl -s -k -u admin:Harbor12345 https://harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/api/v2.0/robots -H 'Content-Type: application/json' -d '{"duration":-1,"level":"system","name":"tanzu-ro","permissions":[{"access":[{"action":"pull","resource":"repository"}],"kind":"project","namespace":"tanzu-cluster-essentials"},{"access":[{"action":"pull","resource":"repository"}],"kind":"project","namespace":"tanzu-rabbitmq"}]}' > tanzu-ro.json
CLIのインストール
以下の作業には
imgpkghttps://github.com/carvel-dev/imgpkg/releases/tag/v0.43.1tanzuhttps://github.com/vmware-tanzu/tanzu-cli/releases/tag/v1.5.0
が必要です。
Cluster Essentialsのインストール
Note
参考ドキュメント https://docs.vmware.com/en/Cluster-Essentials-for-VMware-Tanzu/1.10/cluster-essentials/deploy.html
Broadcom Support Portal のアカウントを作成し、ログインし、右上の製品カテゴリからTanzuを選択。
次のURLにアクセス。
"Cluster Essentials for VMware Tanzu"をクリックして、最新版の隣の緑色の"Token Download"ボタンをクリック。
MY-BROADCOM-SUPPORT-ACCESS-TOKEN='eyJ2ZXIi***********************************************************************************************'のeyJ2ZXIi....の部分をコピー。(スクリーンショットはマスクされていますが、実際はもっと長いです。)
コピーしたトークンを次のようにMY_BROADCOM_SUPPORT_ACCESS_TOKEN変数に設定し、次のコマンドを実行し、Cluster Essentialsパッケージのtarをダウンロード。
export BROADCOM_REGISTRY_USERNAME=*****
export MY_BROADCOM_SUPPORT_ACCESS_TOKEN='eyJ2Z...'
docker login cluster-essentials.packages.broadcom.com -u $BROADCOM_REGISTRY_USERNAME -p $MY_BROADCOM_SUPPORT_ACCESS_TOKEN
imgpkg copy \
-b cluster-essentials.packages.broadcom.com/tanzu-cluster-essentials/cluster-essentials-bundle:1.10.1 \
--to-tar tanzu-cluster-essentials-1.10.1.tar \
--include-non-distributable-layers
Cluster EssentialsパッケージのtarをHarborにRelocateします。
IMGPKG_REGISTRY_HOSTNAME=harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io \
IMGPKG_REGISTRY_USERNAME=admin \
IMGPKG_REGISTRY_PASSWORD=Harbor12345 \
imgpkg copy \
--tar tanzu-cluster-essentials-1.10.1.tar \
--to-repo harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/tanzu-cluster-essentials/cluster-essentials-bundle
Broadcom Support PortalのCluster Essentialsの最新版のバージョンをクリックして、I agree toTerms and Conditionsにチェックをいれる。
作業するOSに対応したtanzu-cluster-essentials-****-amd64-1.10.1.tgzをダウンロード。
次のコマンドでCluster Essentialsをインストールします。
mkdir tanzu-cluster-essentials
tar xzvf tanzu-cluster-essentials-*-amd64-*.tgz -C tanzu-cluster-essentials
export INSTALL_BUNDLE=harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/tanzu-cluster-essentials/cluster-essentials-bundle:1.10.1
export INSTALL_REGISTRY_HOSTNAME=harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io
export INSTALL_REGISTRY_USERNAME=$(cat tanzu-ro.json | jq -r .name)
export INSTALL_REGISTRY_PASSWORD=$(cat tanzu-ro.json | jq -r .secret)
export YTT_kappController__config__caCerts=$(cat harbor.ca)
cd tanzu-cluster-essentials
./install.sh --yes
cd ..
Package Repositoryの登録
インストール用のnamespaceを作成します。
kubectl create ns tanzu-install
Tanzu Standard
Tanzu Standard Package Repositoryを登録します。
Note
参考ドキュメント https://docs.vmware.com/en/Cluster-Essentials-for-VMware-Tanzu/1.10/cluster-essentials/deploy.html
Tanzu Standardパッケージのtarをダウンロードします。
imgpkg copy -b projects.registry.vmware.com/tkg/packages/standard/repo:v2024.8.21 \
--to-tar tanzu-packages-standard-v2024.8.21.tar
HarborにRelocateします。
IMGPKG_REGISTRY_HOSTNAME=harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io \
IMGPKG_REGISTRY_USERNAME=admin \
IMGPKG_REGISTRY_PASSWORD=Harbor12345 \
imgpkg copy \
--tar tanzu-packages-standard-v2024.8.21.tar \
--to-repo harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/tkg/packages/standard/repo
Tanzu StandardのPackage Repositoryを作成します。
tanzu package repository add tanzu-standard \
--url harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/tkg/packages/standard/repo:v2024.8.21 \
--namespace tanzu-install --create-namespace
Tanzu RabbitMQ
Tanzu RabbitMQ Package Repositoryの登録
Note
次のURLにアクセス。
"VMware Tanzu RabbitMQ on Kubernetes"をクリックして、最新版の隣の緑色の"Token Download"ボタンをクリック。
Instruction中の"Tanzu RabbitMQ for Kubernetes - Carvel Bundle"を確認します。
以下のように1.と2.のコマンドをそれぞれ実行し、Tanzu RabbitMQパッケージのtarをダウンロードします。
docker login rabbitmq-kubernetes.packages.broadcom.com --username='********' --password='eyJ2ZXIiOiIyIiwi*********'
imgpkg copy -b rabbitmq-kubernetes.packages.broadcom.com/tanzu-rabbitmq-package-repo:4.0.1 --to-tar=rabbitmq-for-kubernetes-4.0.1.tar
HarborにRelocateします。
IMGPKG_REGISTRY_HOSTNAME=harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io \
IMGPKG_REGISTRY_USERNAME=admin \
IMGPKG_REGISTRY_PASSWORD=Harbor12345 \
imgpkg copy \
--tar rabbitmq-for-kubernetes-4.0.1.tar \
--to-repo harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/tanzu-rabbitmq/tanzu-rabbitmq-package-repo
Harbor上のTanzu RabbitMQパッケージをpullするためのSecretを設定します。
tanzu secret registry add tanzu-rabbitmq-registry \
--username $(cat tanzu-ro.json | jq -r .name) \
--password $(cat tanzu-ro.json | jq -r .secret) \
--server harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io \
--export-to-all-namespaces \
--yes \
--namespace tanzu-install
tanzu package repository add tanzu-rabbitmq-repository \
--url harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/tanzu-rabbitmq/tanzu-rabbitmq-package-repo:4.0.1 \
--namespace tanzu-install
利用可能なPackageの確認
$ kubectl get pkg -n tanzu-install
NAME PACKAGEMETADATA NAME VERSION AGE
auditlogger.rabbitmq.tanzu.vmware.com.0.4.3 auditlogger.rabbitmq.tanzu.vmware.com 0.4.3 11s
cert-manager.rabbitmq.tanzu.vmware.com.1.5.3+rmq cert-manager.rabbitmq.tanzu.vmware.com 1.5.3+rmq 11s
cert-manager.tanzu.vmware.com.1.1.0+vmware.1-tkg.2 cert-manager.tanzu.vmware.com 1.1.0+vmware.1-tkg.2 2m5s
cert-manager.tanzu.vmware.com.1.1.0+vmware.2-tkg.1 cert-manager.tanzu.vmware.com 1.1.0+vmware.2-tkg.1 2m5s
cert-manager.tanzu.vmware.com.1.11.1+vmware.1-tkg.1 cert-manager.tanzu.vmware.com 1.11.1+vmware.1-tkg.1 2m5s
cert-manager.tanzu.vmware.com.1.12.10+vmware.2-tkg.2 cert-manager.tanzu.vmware.com 1.12.10+vmware.2-tkg.2 2m5s
cert-manager.tanzu.vmware.com.1.12.2+vmware.2-tkg.2 cert-manager.tanzu.vmware.com 1.12.2+vmware.2-tkg.2 2m5s
cert-manager.tanzu.vmware.com.1.5.3+vmware.2-tkg.1 cert-manager.tanzu.vmware.com 1.5.3+vmware.2-tkg.1 2m5s
cert-manager.tanzu.vmware.com.1.5.3+vmware.4-tkg.1 cert-manager.tanzu.vmware.com 1.5.3+vmware.4-tkg.1 2m5s
cert-manager.tanzu.vmware.com.1.5.3+vmware.7-tkg.1 cert-manager.tanzu.vmware.com 1.5.3+vmware.7-tkg.1 2m5s
cert-manager.tanzu.vmware.com.1.5.3+vmware.7-tkg.3 cert-manager.tanzu.vmware.com 1.5.3+vmware.7-tkg.3 2m5s
cert-manager.tanzu.vmware.com.1.7.2+vmware.1-tkg.1 cert-manager.tanzu.vmware.com 1.7.2+vmware.1-tkg.1 2m4s
cert-manager.tanzu.vmware.com.1.7.2+vmware.3-tkg.1 cert-manager.tanzu.vmware.com 1.7.2+vmware.3-tkg.1 2m5s
cert-manager.tanzu.vmware.com.1.7.2+vmware.3-tkg.3 cert-manager.tanzu.vmware.com 1.7.2+vmware.3-tkg.3 2m5s
cluster-autoscaler.tanzu.vmware.com.1.25.1+vmware.1-tkg.3 cluster-autoscaler.tanzu.vmware.com 1.25.1+vmware.1-tkg.3 2m4s
cluster-autoscaler.tanzu.vmware.com.1.26.2+vmware.1-tkg.3 cluster-autoscaler.tanzu.vmware.com 1.26.2+vmware.1-tkg.3 2m5s
cluster-autoscaler.tanzu.vmware.com.1.27.2+vmware.1-tkg.3 cluster-autoscaler.tanzu.vmware.com 1.27.2+vmware.1-tkg.3 2m5s
cluster-autoscaler.tanzu.vmware.com.1.28.0+vmware.1-tkg.2 cluster-autoscaler.tanzu.vmware.com 1.28.0+vmware.1-tkg.2 2m5s
cluster-autoscaler.tanzu.vmware.com.1.29.0+vmware.2-tkg.1 cluster-autoscaler.tanzu.vmware.com 1.29.0+vmware.2-tkg.1 2m5s
cluster-autoscaler.tanzu.vmware.com.1.30.0+vmware.2-tkg.1 cluster-autoscaler.tanzu.vmware.com 1.30.0+vmware.2-tkg.1 2m5s
contour.tanzu.vmware.com.1.27.4+vmware.1-tkg.1 contour.tanzu.vmware.com 1.27.4+vmware.1-tkg.1 2m5s
contour.tanzu.vmware.com.1.28.5+vmware.1-tkg.1 contour.tanzu.vmware.com 1.28.5+vmware.1-tkg.1 2m5s
contour.tanzu.vmware.com.1.29.1+vmware.1-tkg.1 contour.tanzu.vmware.com 1.29.1+vmware.1-tkg.1 2m5s
external-csi-snapshot-webhook.tanzu.vmware.com.6.1.0+vmware.1-tkg.7 external-csi-snapshot-webhook.tanzu.vmware.com 6.1.0+vmware.1-tkg.7 2m5s
external-dns.tanzu.vmware.com.0.10.0+vmware.1-tkg.1 external-dns.tanzu.vmware.com 0.10.0+vmware.1-tkg.1 2m4s
external-dns.tanzu.vmware.com.0.10.0+vmware.1-tkg.2 external-dns.tanzu.vmware.com 0.10.0+vmware.1-tkg.2 2m5s
external-dns.tanzu.vmware.com.0.11.0+vmware.1-tkg.2 external-dns.tanzu.vmware.com 0.11.0+vmware.1-tkg.2 2m4s
external-dns.tanzu.vmware.com.0.12.2+vmware.8-tkg.1 external-dns.tanzu.vmware.com 0.12.2+vmware.8-tkg.1 2m4s
external-dns.tanzu.vmware.com.0.13.6+vmware.2-tkg.1 external-dns.tanzu.vmware.com 0.13.6+vmware.2-tkg.1 2m4s
external-dns.tanzu.vmware.com.0.14.2+vmware.1-tkg.1 external-dns.tanzu.vmware.com 0.14.2+vmware.1-tkg.1 2m4s
fluent-bit.tanzu.vmware.com.1.7.5+vmware.1-tkg.1 fluent-bit.tanzu.vmware.com 1.7.5+vmware.1-tkg.1 2m5s
fluent-bit.tanzu.vmware.com.1.7.5+vmware.2-tkg.1 fluent-bit.tanzu.vmware.com 1.7.5+vmware.2-tkg.1 2m5s
fluxcd-helm-controller.tanzu.vmware.com.0.21.0+vmware.3-tkg.1 fluxcd-helm-controller.tanzu.vmware.com 0.21.0+vmware.3-tkg.1 2m5s
fluxcd-helm-controller.tanzu.vmware.com.0.28.1+vmware.3-tkg.1 fluxcd-helm-controller.tanzu.vmware.com 0.28.1+vmware.3-tkg.1 2m5s
fluxcd-helm-controller.tanzu.vmware.com.0.36.2+vmware.2-tkg.1-ctrl fluxcd-helm-controller.tanzu.vmware.com 0.36.2+vmware.2-tkg.1-ctrl 2m5s
fluxcd-kustomize-controller.tanzu.vmware.com.0.24.4+vmware.3-tkg.1 fluxcd-kustomize-controller.tanzu.vmware.com 0.24.4+vmware.3-tkg.1 2m5s
fluxcd-kustomize-controller.tanzu.vmware.com.0.32.0+vmware.3-tkg.1 fluxcd-kustomize-controller.tanzu.vmware.com 0.32.0+vmware.3-tkg.1 2m5s
fluxcd-kustomize-controller.tanzu.vmware.com.1.1.1+vmware.2-tkg.1 fluxcd-kustomize-controller.tanzu.vmware.com 1.1.1+vmware.2-tkg.1 2m5s
fluxcd-source-controller.tanzu.vmware.com.0.24.4+vmware.3-tkg.1 fluxcd-source-controller.tanzu.vmware.com 0.24.4+vmware.3-tkg.1 2m5s
fluxcd-source-controller.tanzu.vmware.com.0.33.0+vmware.3-tkg.1 fluxcd-source-controller.tanzu.vmware.com 0.33.0+vmware.3-tkg.1 2m5s
fluxcd-source-controller.tanzu.vmware.com.0.36.1+vmware.3-tkg.1 fluxcd-source-controller.tanzu.vmware.com 0.36.1+vmware.3-tkg.1 2m5s
fluxcd-source-controller.tanzu.vmware.com.1.1.2+vmware.6-tkg.1 fluxcd-source-controller.tanzu.vmware.com 1.1.2+vmware.6-tkg.1 2m5s
grafana.tanzu.vmware.com.10.0.1+vmware.2-tkg.1 grafana.tanzu.vmware.com 10.0.1+vmware.2-tkg.1 2m5s
grafana.tanzu.vmware.com.7.5.16+vmware.1-tkg.1 grafana.tanzu.vmware.com 7.5.16+vmware.1-tkg.1 2m5s
grafana.tanzu.vmware.com.7.5.17+vmware.1-tkg.2 grafana.tanzu.vmware.com 7.5.17+vmware.1-tkg.2 2m5s
grafana.tanzu.vmware.com.7.5.7+vmware.1-tkg.1 grafana.tanzu.vmware.com 7.5.7+vmware.1-tkg.1 2m5s
grafana.tanzu.vmware.com.7.5.7+vmware.2-tkg.1 grafana.tanzu.vmware.com 7.5.7+vmware.2-tkg.1 2m5s
harbor.tanzu.vmware.com.2.10.3+vmware.1-tkg.1 harbor.tanzu.vmware.com 2.10.3+vmware.1-tkg.1 2m5s
harbor.tanzu.vmware.com.2.9.1+vmware.1-tkg.1 harbor.tanzu.vmware.com 2.9.1+vmware.1-tkg.1 2m5s
multus-cni.tanzu.vmware.com.3.7.1+vmware.1-tkg.1 multus-cni.tanzu.vmware.com 3.7.1+vmware.1-tkg.1 2m4s
multus-cni.tanzu.vmware.com.3.7.1+vmware.2-tkg.1 multus-cni.tanzu.vmware.com 3.7.1+vmware.2-tkg.1 2m4s
multus-cni.tanzu.vmware.com.3.7.1+vmware.2-tkg.2 multus-cni.tanzu.vmware.com 3.7.1+vmware.2-tkg.2 2m4s
multus-cni.tanzu.vmware.com.3.8.0+vmware.1-tkg.1 multus-cni.tanzu.vmware.com 3.8.0+vmware.1-tkg.1 2m4s
multus-cni.tanzu.vmware.com.3.8.0+vmware.2-tkg.2 multus-cni.tanzu.vmware.com 3.8.0+vmware.2-tkg.2 2m4s
multus-cni.tanzu.vmware.com.3.8.0+vmware.3-tkg.1 multus-cni.tanzu.vmware.com 3.8.0+vmware.3-tkg.1 2m4s
multus-cni.tanzu.vmware.com.4.0.1+vmware.2-tkg.3 multus-cni.tanzu.vmware.com 4.0.1+vmware.2-tkg.3 2m4s
prometheus.tanzu.vmware.com.2.37.0+vmware.3-tkg.1 prometheus.tanzu.vmware.com 2.37.0+vmware.3-tkg.1 2m4s
prometheus.tanzu.vmware.com.2.43.0+vmware.2-tkg.1 prometheus.tanzu.vmware.com 2.43.0+vmware.2-tkg.1 2m4s
prometheus.tanzu.vmware.com.2.45.0+vmware.1-tkg.2 prometheus.tanzu.vmware.com 2.45.0+vmware.1-tkg.2 2m4s
prometheus.tanzu.vmware.com.2.45.0+vmware.2-tkg.1 prometheus.tanzu.vmware.com 2.45.0+vmware.2-tkg.1 2m4s
rabbitmq.tanzu.vmware.com.4.0.1 rabbitmq.tanzu.vmware.com 4.0.1 11s
vsphere-pv-csi-webhook.tanzu.vmware.com.3.1.0+vmware.1-tkg.4 vsphere-pv-csi-webhook.tanzu.vmware.com 3.1.0+vmware.1-tkg.4 2m4s
whereabouts.tanzu.vmware.com.0.5.1+vmware.2-tkg.1 whereabouts.tanzu.vmware.com 0.5.1+vmware.2-tkg.1 2m4s
whereabouts.tanzu.vmware.com.0.5.4+vmware.1-tkg.1 whereabouts.tanzu.vmware.com 0.5.4+vmware.1-tkg.1 2m4s
whereabouts.tanzu.vmware.com.0.5.4+vmware.2-tkg.1 whereabouts.tanzu.vmware.com 0.5.4+vmware.2-tkg.1 2m4s
whereabouts.tanzu.vmware.com.0.7.0+vmware.2-tkg.1 whereabouts.tanzu.vmware.com 0.7.0+vmware.2-tkg.1 2m4s
Cert Managerパッケージのインストール
RabbitMQ Operatorに必要なCert Managerパッケージをインストールします。
tanzu package install cert-manager --package cert-manager.tanzu.vmware.com --namespace tanzu-install --version 1.12.10+vmware.2-tkg.2
作成されたPodを確認します。
$ kubectl get pod -n cert-manager
NAME READY STATUS RESTARTS AGE
cert-manager-65f6ddc4b5-85dpg 1/1 Running 0 82s
cert-manager-cainjector-84b76dbb8-9r5mv 1/1 Running 0 82s
cert-manager-webhook-f549d4549-67twn 1/1 Running 0 82s
RabbitMQ Operatorパッケージのインストール
次にRabbitMQ Operatorパッケージをインストールします。
kubectl create ns tanzu-rabbitmq
cat <<EOF > tanzu-rabbitmq-values.yaml
namespace: tanzu-rabbitmq
EOF
tanzu package install tanzu-rabbitmq -p rabbitmq.tanzu.vmware.com -n tanzu-install -v 4.0.1 --values-file tanzu-rabbitmq-values.yaml
Important
Tanzu Kubernetes ClustersのようにPod Security Admissionがデフォルトでenforce: restrictedとなっている環境では、Rabbit MQ Operatorのインストールに失敗します。
次のようにoverlayを設定して、インストール先のnamespaceの先約をenforce: baselineに緩める必要があります。
cat <<'EOF' > rabbitmq-operator-psa.yaml
#@ load("@ytt:overlay", "overlay")
#@overlay/match by=overlay.subset({"kind":"Namespace", "metadata": {"name": "tanzu-rabbitmq"}}), expects="0+"
---
metadata:
labels:
#@overlay/match missing_ok=True
pod-security.kubernetes.io/enforce: baseline
EOF
overlayは次のように指定できます。
tanzu package install tanzu-rabbitmq -p rabbitmq.tanzu.vmware.com -n tanzu-install -v 4.0.1 --values-file tanzu-rabbitmq-values.yaml --ytt-overlay-file rabbitmq-operator-psa.yaml
作成されたPodを確認します。
$ kubectl get pod -n tanzu-rabbitmq
NAME READY STATUS RESTARTS AGE
messaging-topology-operator-6c65cc5cbb-ldvpc 1/1 Running 0 14s
rabbitmq-cluster-operator-64c484c587-9vm8q 1/1 Running 0 14s
インストールしたPackageを確認します。
$ kubectl get pkgi -A
NAMESPACE NAME PACKAGE NAME PACKAGE VERSION DESCRIPTION AGE
tanzu-install cert-manager cert-manager.tanzu.vmware.com 1.12.10+vmware.2-tkg.2 Reconcile succeeded 10m
tanzu-install tanzu-rabbitmq rabbitmq.tanzu.vmware.com 4.0.1 Reconcile succeeded 57s
RabbitMQ Clusterの作成
kubectl create ns demo
3 NodesのRabbitMQ Clusterを作成します。Serviceはtype=LoadBalancerで公開します。
cat <<EOF > rabbitmq.yaml
apiVersion: v1
kind: Secret
metadata:
name: tanzu-rabbitmq-registry
namespace: demo
annotations:
secretgen.carvel.dev/image-pull-secret: ""
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: e30K
---
apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
name: demo-rabbitmq
namespace: demo
spec:
imagePullSecrets:
- name: tanzu-rabbitmq-registry
service:
type: LoadBalancer
replicas: 3
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app.kubernetes.io/name
operator: In
values:
- demo-rabbitmq
topologyKey: kubernetes.io/hostname
EOF
kubectl apply -f rabbitmq.yaml
作成されたリソースを確認します。
$ kubectl get pod,sts,rabbitmq,secret,svc -n demo -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/demo-rabbitmq-server-0 1/1 Running 0 27m 10.244.3.35 kind-worker <none> <none>
pod/demo-rabbitmq-server-1 1/1 Running 0 27m 10.244.1.18 kind-worker3 <none> <none>
pod/demo-rabbitmq-server-2 1/1 Running 0 27m 10.244.2.21 kind-worker2 <none> <none>
NAME READY AGE CONTAINERS IMAGES
statefulset.apps/demo-rabbitmq-server 3/3 27m rabbitmq harbor.192-168-228-210.sslip.io/tanzu-rabbitmq/tanzu-rabbitmq-package-repo@sha256:2358d6f9c368fa0436cd83efb2d84a9b3fb743631eebb68f024f139ba488708a
NAME ALLREPLICASREADY RECONCILESUCCESS AGE
rabbitmqcluster.rabbitmq.com/demo-rabbitmq True True 27m
NAME TYPE DATA AGE
secret/demo-rabbitmq-default-user Opaque 7 27m
secret/demo-rabbitmq-erlang-cookie Opaque 1 27m
secret/tanzu-rabbitmq-registry kubernetes.io/dockerconfigjson 1 27m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/demo-rabbitmq LoadBalancer 10.96.171.146 192.168.228.200 5672:32032/TCP,15672:32166/TCP,15692:30211/TCP 27m app.kubernetes.io/name=demo-rabbitmq
service/demo-rabbitmq-nodes ClusterIP None <none> 4369/TCP,25672/TCP 27m app.kubernetes.io/name=demo-rabbitmq
http://192.168.228.200:15672 で管理コンソールにアクセスできます。
ユーザー名とパスワードは次のコマンドで確認できます。
$ kubectl get secret -n demo demo-rabbitmq-default-user -ojson | jq '.data | map_values(@base64d)'
{
"default_user.conf": "default_user = default_user_UgZFbsYLfl9KZEyJnvM\ndefault_pass = wtlfnLKYTWLu4u4y31_vUDOfnrTuUfaI\n",
"host": "demo-rabbitmq.demo.svc",
"password": "wtlfnLKYTWLu4u4y31_vUDOfnrTuUfaI",
"port": "5672",
"provider": "rabbitmq",
"type": "rabbitmq",
"username": "default_user_UgZFbsYLfl9KZEyJnvM"
}
アプリのデプロイ
RabbitMQにサンプルデータを送信するアプリと受信するアプリをデプロイします。ソースコードは
https://github.com/making/demo-rabbitmq
です。
ここではビルド済みのイメージを使用します。HarborにRelocateするためにimgpkgを使ってtarとしてダウンロードします。
imgpkg copy -i ghcr.io/making/demo-rabbitmq-receiver --to-tar demo-rabbitmq-receiver.tar
imgpkg copy -i ghcr.io/making/demo-rabbitmq-sender --to-tar demo-rabbitmq-sender.tar
Harborにアプリ用のプロジェクトを作成します。
curl -k -u admin:Harbor12345 https://harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/api/v2.0/projects -H "Content-Type: application/json" -d '{"project_name": "apps", "public": false}'
アプリ用のプロジェクトのRead Only Robot Accountを作成します。
curl -s -k -u admin:Harbor12345 https://harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/api/v2.0/robots -H 'Content-Type: application/json' -d '{"duration":-1,"level":"system","name":"apps-ro","permissions":[{"access":[{"action":"pull","resource":"repository"}],"kind":"project","namespace":"apps"}]}' > apps-ro.json
このRobot AccountのSecretを登録します。
tanzu secret registry add apps-registry \
--username $(cat apps-ro.json | jq -r .name) \
--password $(cat apps-ro.json | jq -r .secret) \
--server harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io \
--yes \
--namespace demo
IMGPKG_REGISTRY_HOSTNAME=harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io \
IMGPKG_REGISTRY_USERNAME=admin \
IMGPKG_REGISTRY_PASSWORD=Harbor12345 \
imgpkg copy --tar demo-rabbitmq-receiver.tar --to-repo harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/apps/demo-rabbitmq-receiver
IMGPKG_REGISTRY_HOSTNAME=harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io \
IMGPKG_REGISTRY_USERNAME=admin \
IMGPKG_REGISTRY_PASSWORD=Harbor12345 \
imgpkg copy --tar demo-rabbitmq-sender.tar --to-repo harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/apps/demo-rabbitmq-sender
cat <<EOF > demo-rabbitmq-receiver.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: demo-rabbitmq-receiver
labels:
app: demo-rabbitmq-receiver
app.kubernetes.io/part-of: demo-rabbitmq-receiver
spec:
replicas: 2
selector:
matchLabels:
app: demo-rabbitmq-receiver
template:
metadata:
annotations:
prometheus.io/path: /actuator/prometheus
prometheus.io/port: "8081"
prometheus.io/scrape: "true"
labels:
app: demo-rabbitmq-receiver
app.kubernetes.io/part-of: demo-rabbitmq-receiver
spec:
containers:
- name: workload
image: harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/apps/demo-rabbitmq-receiver:latest
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
name: user-port
protocol: TCP
resources:
limits:
cpu: "1"
memory: 1Gi
requests:
cpu: "1"
memory: 1Gi
env:
- name: JAVA_TOOL_OPTIONS
value: -Duser.country=JP -Duser.language=ja -Duser.timezone=Asia/Tokyo -Dfile.encoding=UTF-8 -Dmanagement.endpoint.health.probes.add-additional-paths=true -Dmanagement.endpoint.health.show-details=always -Dmanagement.endpoints.web.base-path=/actuator -Dmanagement.endpoints.web.exposure.include="*" -Dmanagement.health.probes.enabled=true -Dmanagement.server.port=8081 -Dserver.port=8080
- name: SERVICE_BINDING_ROOT
value: /bindings
volumeMounts:
- name: rabbitmq
mountPath: /bindings/rabbitmq
readOnly: true
livenessProbe:
failureThreshold: 5
httpGet:
path: /livez
port: 8080
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
readinessProbe:
httpGet:
path: /readyz
port: 8080
scheme: HTTP
startupProbe:
httpGet:
path: /readyz
port: 8080
scheme: HTTP
failureThreshold: 20
periodSeconds: 5
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
runAsNonRoot: true
runAsUser: 1002
seccompProfile:
type: RuntimeDefault
imagePullSecrets:
- name: apps-registry
volumes:
- name: rabbitmq
secret:
secretName: demo-rabbitmq-default-user
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchLabels:
app: demo-rabbitmq-receiver
topologyKey: kubernetes.io/hostname
weight: 1
EOF
kubectl apply -f demo-rabbitmq-receiver.yaml -n demo
$ kubectl get pod -n demo -l app=demo-rabbitmq-receiver -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
demo-rabbitmq-receiver-5cf79f6cb5-4mbwh 1/1 Running 0 116s 10.244.2.33 kind-worker2 <none> <none>
demo-rabbitmq-receiver-5cf79f6cb5-nl9pt 1/1 Running 0 116s 10.244.3.50 kind-worker <none> <none>
cat <<EOF > demo-rabbitmq-sender.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: demo-rabbitmq-sender
labels:
app: demo-rabbitmq-sender
app.kubernetes.io/part-of: demo-rabbitmq-sender
spec:
replicas: 1
selector:
matchLabels:
app: demo-rabbitmq-sender
template:
metadata:
annotations:
prometheus.io/path: /actuator/prometheus
prometheus.io/port: "8081"
prometheus.io/scrape: "true"
labels:
app: demo-rabbitmq-sender
app.kubernetes.io/part-of: demo-rabbitmq-sender
spec:
containers:
- name: workload
image: harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/apps/demo-rabbitmq-sender:latest
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
name: user-port
protocol: TCP
resources:
limits:
cpu: "1"
memory: 1Gi
requests:
cpu: "1"
memory: 1Gi
env:
- name: JAVA_TOOL_OPTIONS
value: -Duser.country=JP -Duser.language=ja -Duser.timezone=Asia/Tokyo -Dfile.encoding=UTF-8 -Dmanagement.endpoint.health.probes.add-additional-paths=true -Dmanagement.endpoint.health.show-details=always -Dmanagement.endpoints.web.base-path=/actuator -Dmanagement.endpoints.web.exposure.include="*" -Dmanagement.health.probes.enabled=true -Dmanagement.server.port=8081 -Dserver.port=8080
- name: SERVICE_BINDING_ROOT
value: /bindings
volumeMounts:
- name: rabbitmq
mountPath: /bindings/rabbitmq
readOnly: true
livenessProbe:
failureThreshold: 5
httpGet:
path: /livez
port: 8080
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
readinessProbe:
httpGet:
path: /readyz
port: 8080
scheme: HTTP
startupProbe:
httpGet:
path: /readyz
port: 8080
scheme: HTTP
failureThreshold: 20
periodSeconds: 5
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
runAsNonRoot: true
runAsUser: 1002
seccompProfile:
type: RuntimeDefault
imagePullSecrets:
- name: apps-registry
volumes:
- name: rabbitmq
secret:
secretName: demo-rabbitmq-default-user
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchLabels:
app: demo-rabbitmq-sender
topologyKey: kubernetes.io/hostname
weight: 1
EOF
kubectl apply -f demo-rabbitmq-sender.yaml -n demo
$ kubectl get pod -n demo -l app=demo-rabbitmq-sender -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
demo-rabbitmq-sender-7949f97cc7-p6qrc 1/1 Running 0 2m48s 10.244.1.31 kind-worker3 <none> <none>
$ kubectl logs -n demo deploy/demo-rabbitmq-receiver -f
...
2024-10-05T00:23:27.405+09:00 INFO 1 --- [demo-receiver] [rabbit-simple-0] [6700086f62beff530560da9ebaa82e9f-13f334062cf2626c] c.e.rabbitmq.demoreceiver.DemoReceiver : Received event: Event[message=Hello World(4) correlation_id=f7b102f2-1344-4693-8d2c-4ae4862637fa]
2024-10-05T00:23:27.406+09:00 INFO 1 --- [demo-receiver] [rabbit-simple-1] [6700086f62beff530560da9ebaa82e9f-aceb75f9df636f7b] c.e.rabbitmq.demoreceiver.DemoReceiver : Received event: Event[message=Hello World(5) correlation_id=7385c78b-03b2-4095-a165-28f3c7b4a07e]
2024-10-05T00:23:27.408+09:00 INFO 1 --- [demo-receiver] [rabbit-simple-2] [6700086f62beff530560da9ebaa82e9f-8135dc084d972d01] c.e.rabbitmq.demoreceiver.DemoReceiver : Received event: Event[message=Hello World(3) correlation_id=ed824363-0c65-400f-8b18-d8905387b613]
$ kubectl logs -n demo deploy/demo-rabbitmq-sender -f
...
2024-10-05T00:23:50.385+09:00 INFO 1 --- [demo-sender] [.96.209.20:5672] [ ] c.e.rabbitmq.demosender.DemoSender : Received ack for correlation_id=a5fb4737-c2b4-4d8b-8cff-133f7d46474d
2024-10-05T00:23:50.385+09:00 INFO 1 --- [demo-sender] [.96.209.20:5672] [ ] c.e.rabbitmq.demosender.DemoSender : Received ack for correlation_id=4bbd0093-4882-4b90-9263-5a7059fc1653
2024-10-05T00:23:50.387+09:00 INFO 1 --- [demo-sender] [.96.209.20:5672] [ ] c.e.rabbitmq.demosender.DemoSender : Received ack for correlation_id=06f686d0-3683-47da-8b20-6b3bd43bbfe3
RabbitMQ ClusterのTLS対応
TLS対応します。
cat <<EOF > rabbitmq.yaml
apiVersion: v1
kind: Secret
metadata:
name: tanzu-rabbitmq-registry
namespace: demo
annotations:
secretgen.carvel.dev/image-pull-secret: ""
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: e30K
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: selfsigned-issuer
namespace: demo
spec:
selfSigned: { }
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: selfsigned-cert
namespace: demo
spec:
commonName: demo-rabbitmq
dnsNames:
- demo-rabbitmq.demo.svc.cluster.local
- demo-rabbitmq.demo.svc
- "*.demo-rabbitmq-nodes.demo.svc.cluster.local"
secretName: tls-secret
issuerRef:
kind: Issuer
name: selfsigned-issuer
---
apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
name: demo-rabbitmq
namespace: demo
spec:
imagePullSecrets:
- name: tanzu-rabbitmq-registry
service:
type: LoadBalancer
replicas: 3
tls:
secretName: tls-secret
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app.kubernetes.io/name
operator: In
values:
- demo-rabbitmq
topologyKey: kubernetes.io/hostname
EOF
kubectl apply -f rabbitmq.yaml
作成されたリソースを確認します。
$ kubectl get pod,sts,rabbitmq,svc,secret -n demo -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/demo-rabbitmq-receiver-5cf79f6cb5-4mbwh 1/1 Running 0 32m 10.244.2.33 kind-worker2 <none> <none>
pod/demo-rabbitmq-receiver-5cf79f6cb5-nl9pt 1/1 Running 0 32m 10.244.3.50 kind-worker <none> <none>
pod/demo-rabbitmq-sender-7949f97cc7-p6qrc 1/1 Running 0 18m 10.244.1.31 kind-worker3 <none> <none>
pod/demo-rabbitmq-server-0 1/1 Running 0 7m32s 10.244.3.51 kind-worker <none> <none>
pod/demo-rabbitmq-server-1 1/1 Running 0 8m29s 10.244.2.34 kind-worker2 <none> <none>
pod/demo-rabbitmq-server-2 1/1 Running 0 9m28s 10.244.1.32 kind-worker3 <none> <none>
NAME READY AGE CONTAINERS IMAGES
statefulset.apps/demo-rabbitmq-server 3/3 64m rabbitmq harbor.192-168-228-210.sslip.io/tanzu-rabbitmq/tanzu-rabbitmq-package-repo@sha256:2358d6f9c368fa0436cd83efb2d84a9b3fb743631eebb68f024f139ba488708a
NAME ALLREPLICASREADY RECONCILESUCCESS AGE
rabbitmqcluster.rabbitmq.com/demo-rabbitmq True True 64m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/demo-rabbitmq LoadBalancer 10.96.209.20 192.168.228.200 5672:32370/TCP,15672:30518/TCP,15691:31987/TCP,5671:30623/TCP,15671:31360/TCP 64m app.kubernetes.io/name=demo-rabbitmq
service/demo-rabbitmq-nodes ClusterIP None <none> 4369/TCP,25672/TCP 64m app.kubernetes.io/name=demo-rabbitmq
NAME TYPE DATA AGE
secret/apps-registry kubernetes.io/dockerconfigjson 1 34m
secret/demo-rabbitmq-default-user Opaque 7 64m
secret/demo-rabbitmq-erlang-cookie Opaque 1 64m
secret/tanzu-rabbitmq-registry kubernetes.io/dockerconfigjson 1 64m
secret/tls-secret kubernetes.io/tls 3 9m36s
https://192.168.228.200:15671 で管理コンソールにアクセスできます。
ユーザー名とパスワードは次のコマンドで確認できます。
$ kubectl get secret -n demo demo-rabbitmq-default-user -ojson | jq '.data | map_values(@base64d)'
{
"default_user.conf": "default_user = default_user_UgZFbsYLfl9KZEyJnvM\ndefault_pass = wtlfnLKYTWLu4u4y31_vUDOfnrTuUfaI\n",
"host": "demo-rabbitmq.demo.svc",
"password": "wtlfnLKYTWLu4u4y31_vUDOfnrTuUfaI",
"port": "5671",
"provider": "rabbitmq",
"type": "rabbitmq",
"username": "default_user_UgZFbsYLfl9KZEyJnvM"
}
アプリのRabbitMQ TLS対応
cat <<EOF > demo-rabbitmq-receiver.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: demo-rabbitmq-receiver
labels:
app: demo-rabbitmq-receiver
app.kubernetes.io/part-of: demo-rabbitmq-receiver
spec:
replicas: 2
selector:
matchLabels:
app: demo-rabbitmq-receiver
template:
metadata:
annotations:
prometheus.io/path: /actuator/prometheus
prometheus.io/port: "8081"
prometheus.io/scrape: "true"
labels:
app: demo-rabbitmq-receiver
app.kubernetes.io/part-of: demo-rabbitmq-receiver
spec:
containers:
- name: workload
image: harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/apps/demo-rabbitmq-receiver:latest
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
name: user-port
protocol: TCP
resources:
limits:
cpu: "1"
memory: 1Gi
requests:
cpu: "1"
memory: 1Gi
env:
- name: JAVA_TOOL_OPTIONS
value: -Duser.country=JP -Duser.language=ja -Duser.timezone=Asia/Tokyo -Dfile.encoding=UTF-8 -Dmanagement.endpoint.health.probes.add-additional-paths=true -Dmanagement.endpoint.health.show-details=always -Dmanagement.endpoints.web.base-path=/actuator -Dmanagement.endpoints.web.exposure.include="*" -Dmanagement.health.probes.enabled=true -Dmanagement.server.port=8081 -Dserver.port=8080
- name: SERVICE_BINDING_ROOT
value: /bindings
- name: SPRING_RABBITMQ_SSL_ENABLED
value: "true"
- name: SPRING_RABBITMQ_SSL_BUNDLE
value: rabbitmq
- name: SPRING_SSL_BUNDLE_PEM_RABBITMQ_TRUSTSTORE_CERTIFICATE
value: file:///rabbitmq-tls/ca.crt
volumeMounts:
- name: rabbitmq
mountPath: /bindings/rabbitmq
readOnly: true
- name: rabbitmq-tls
mountPath: /rabbitmq-tls
readOnly: true
livenessProbe:
failureThreshold: 5
httpGet:
path: /livez
port: 8080
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
readinessProbe:
httpGet:
path: /readyz
port: 8080
scheme: HTTP
startupProbe:
httpGet:
path: /readyz
port: 8080
scheme: HTTP
failureThreshold: 20
periodSeconds: 5
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
runAsNonRoot: true
runAsUser: 1002
seccompProfile:
type: RuntimeDefault
imagePullSecrets:
- name: apps-registry
volumes:
- name: rabbitmq
secret:
secretName: demo-rabbitmq-default-user
- name: rabbitmq-tls
secret:
secretName: tls-secret
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchLabels:
app: demo-rabbitmq-receiver
topologyKey: kubernetes.io/hostname
weight: 1
EOF
kubectl apply -f demo-rabbitmq-receiver.yaml -n demo
$ kubectl get pod -n demo -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
demo-rabbitmq-receiver-97cc5f46c-pnpb4 1/1 Running 0 2m19s 10.244.1.33 kind-worker3 <none> <none>
demo-rabbitmq-receiver-97cc5f46c-q4mj8 1/1 Running 0 83s 10.244.2.36 kind-worker2 <none> <none>
demo-rabbitmq-sender-7949f97cc7-p6qrc 1/1 Running 0 31m 10.244.1.31 kind-worker3 <none> <none>
demo-rabbitmq-server-0 1/1 Running 0 20m 10.244.3.51 kind-worker <none> <none>
demo-rabbitmq-server-1 1/1 Running 0 21m 10.244.2.34 kind-worker2 <none> <none>
demo-rabbitmq-server-2 1/1 Running 0 22m 10.244.1.32 kind-worker3 <none> <none>
"TLS / SSL"が●
cat <<EOF > demo-rabbitmq-sender.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: demo-rabbitmq-sender
labels:
app: demo-rabbitmq-sender
app.kubernetes.io/part-of: demo-rabbitmq-sender
spec:
replicas: 1
selector:
matchLabels:
app: demo-rabbitmq-sender
template:
metadata:
annotations:
prometheus.io/path: /actuator/prometheus
prometheus.io/port: "8081"
prometheus.io/scrape: "true"
labels:
app: demo-rabbitmq-sender
app.kubernetes.io/part-of: demo-rabbitmq-sender
spec:
containers:
- name: workload
image: harbor.$(echo $HARBOR_IP | sed 's/\./-/g').sslip.io/apps/demo-rabbitmq-sender:latest
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
name: user-port
protocol: TCP
resources:
limits:
cpu: "1"
memory: 1Gi
requests:
cpu: "1"
memory: 1Gi
env:
- name: JAVA_TOOL_OPTIONS
value: -Duser.country=JP -Duser.language=ja -Duser.timezone=Asia/Tokyo -Dfile.encoding=UTF-8 -Dmanagement.endpoint.health.probes.add-additional-paths=true -Dmanagement.endpoint.health.show-details=always -Dmanagement.endpoints.web.base-path=/actuator -Dmanagement.endpoints.web.exposure.include="*" -Dmanagement.health.probes.enabled=true -Dmanagement.server.port=8081 -Dserver.port=8080
- name: SERVICE_BINDING_ROOT
value: /bindings
- name: SPRING_RABBITMQ_SSL_ENABLED
value: "true"
- name: SPRING_RABBITMQ_SSL_BUNDLE
value: rabbitmq
- name: SPRING_SSL_BUNDLE_PEM_RABBITMQ_TRUSTSTORE_CERTIFICATE
value: file:///rabbitmq-tls/ca.crt
volumeMounts:
- name: rabbitmq
mountPath: /bindings/rabbitmq
readOnly: true
- name: rabbitmq-tls
mountPath: /rabbitmq-tls
readOnly: true
livenessProbe:
failureThreshold: 5
httpGet:
path: /livez
port: 8080
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
readinessProbe:
httpGet:
path: /readyz
port: 8080
scheme: HTTP
startupProbe:
httpGet:
path: /readyz
port: 8080
scheme: HTTP
failureThreshold: 20
periodSeconds: 5
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
runAsNonRoot: true
runAsUser: 1002
seccompProfile:
type: RuntimeDefault
imagePullSecrets:
- name: apps-registry
volumes:
- name: rabbitmq
secret:
secretName: demo-rabbitmq-default-user
- name: rabbitmq-tls
secret:
secretName: tls-secret
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchLabels:
app: demo-rabbitmq-sender
topologyKey: kubernetes.io/hostname
weight: 1
EOF
kubectl apply -f demo-rabbitmq-sender.yaml -n demo
リソースの削除
kubectl delete -f demo-rabbitmq-sender.yaml -n demo
kubectl delete -f demo-rabbitmq-receiver.yaml -n demo
kubectl delete -f rabbitmq.yaml