IK.AM

@making's tech note


Tanzu Kubernetes Grid on vSphere 1.5.1を新規インストールするメモ

🗃 {Dev/CaaS/Kubernetes/TKG/vSphere}
🏷 Cluster API 🏷 Kubernetes 🏷 TKG 🏷 Tanzu 🏷 vSphere 
🗓 Updated at 2022-03-04T17:14:26Z  🗓 Created at 2022-03-04T17:12:20Z {✒️️ Edit  ⏰ History  🗑 Delete}  🌎 Translation by Google

⚠️ 本記事の内容はVMwareによってサポートされていません。 記事の内容で生じた問題については自己責任で対応し、 VMwareサポート窓口には問い合わせないでください

Table of contents

CLIのインストール

ドキュメントはこちら。 https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/1.5/vmware-tanzu-kubernetes-grid-15/GUID-install-cli.html

https://customerconnect.vmware.com/downloads/details?downloadGroup=TKG-151&productId=988

から

  • VMware Tanzu CLI for Mac
tar xzvf tanzu-cli-bundle-darwin-amd64.tar.gz

install cli/core/v0.11.1/tanzu-core-darwin_amd64 /usr/local/bin/tanzu

次のバージョンで動作確認します。

$ tanzu version
version: v0.11.1
buildDate: 2022-02-14
sha: 4d578570

初期化します。プラグインがインストールされます。

$ tanzu init
Checking for required plugins...
Installing plugin 'login:v0.11.1'
Installing plugin 'management-cluster:v0.11.1'
Installing plugin 'package:v0.11.1'
Installing plugin 'pinniped-auth:v0.11.1'
Installing plugin 'secret:v0.11.1'
Successfully installed all required plugins
✔  successfully initialized CLI 

プラグイン一覧を確認します。

$ tanzu plugin list
  NAME                DESCRIPTION                                                        SCOPE       DISCOVERY  VERSION  STATUS     
  login               Login to the platform                                              Standalone  default    v0.11.1  installed  
  management-cluster  Kubernetes management-cluster operations                           Standalone  default    v0.11.1  installed  
  package             Tanzu package management                                           Standalone  default    v0.11.1  installed  
  pinniped-auth       Pinniped authentication operations (usually not directly invoked)  Standalone  default    v0.11.1  installed  
  secret              Tanzu secret management                                            Standalone  default    v0.11.1  installed  

バージョンが違う場合は次のコマンドでTKG 1.5.1用のプラグインバージョンに合わせられます。

tanzu plugin sync

vCenterにOVAファイルをアップロード

https://customerconnect.vmware.com/downloads/details?downloadGroup=TKG-151&productId=988

から

  • Photon v3 Kubernetes v1.22.5 OVA

をダウンロードし、~/Downloadsに保存します。

TKGをインストールする環境の情報を次の例のように設定してください。

cat <<EOF > tkg-env.sh
export GOVC_URL=vcsa.v.maki.lol
export GOVC_USERNAME=administrator@vsphere.local
export GOVC_PASSWORD=VMware1!
export GOVC_DATACENTER=Datacenter
export GOVC_NETWORK="VM Network"
export GOVC_DATASTORE=datastore1
export GOVC_RESOURCE_POOL=/Datacenter/host/Cluster/Resources/tkg
export GOVC_INSECURE=1
export TEMPLATE_FOLDER=/Datacenter/vm/tkg
EOF

次のコマンドでOVAファイルをvCenterにアップロードします。

source tkg-env.sh


OVA_FILE=~/Downloads/photon-3-kube-v1.22.5+vmware.1-tkg.2-790a7a702b7fa129fb96be8699f5baa4.ova
govc import.spec ${OVA_FILE} | sed 's/flat/thin/' > options.json
govc import.ova -options options.json -folder ${TEMPLATE_FOLDER} ${OVA_FILE}
govc vm.markastemplate ${TEMPLATE_FOLDER}/photon-3-kube-v1.22.5

Management Clusterのデプロイ

tanzu management-cluster create --ui
image image image image image image image image image image image image image image image image image
$ tanzu management-cluster get
  NAME    NAMESPACE   STATUS   CONTROLPLANE  WORKERS  KUBERNETES        ROLES       PLAN  
  potato  tkg-system  running  1/1           1/1      v1.22.5+vmware.1  management  dev   


Details:

NAME                                                       READY  SEVERITY  REASON  SINCE  MESSAGE
/potato                                                    True                     22m           
├─ClusterInfrastructure - VSphereCluster/potato            True                     22m           
├─ControlPlane - KubeadmControlPlane/potato-control-plane  True                     22m           
│ └─Machine/potato-control-plane-wc6dk                     True                     22m           
└─Workers                                                                                         
  └─MachineDeployment/potato-md-0                          True                     22m           
    └─Machine/potato-md-0-6d59db54c4-mlzxp                 True                     22m           


Providers:

  NAMESPACE                          NAME                    TYPE                    PROVIDERNAME  VERSION  WATCHNAMESPACE  
  capi-kubeadm-bootstrap-system      bootstrap-kubeadm       BootstrapProvider       kubeadm       v1.0.1                   
  capi-kubeadm-control-plane-system  control-plane-kubeadm   ControlPlaneProvider    kubeadm       v1.0.1                   
  capi-system                        cluster-api             CoreProvider            cluster-api   v1.0.1                   
  capv-system                        infrastructure-vsphere  InfrastructureProvider  vsphere       v1.0.2 
tanzu management-cluster kubeconfig get potato --admin
kubectl config use-context potato-admin@potato
$ kubectl get pod -A
NAMESPACE                           NAME                                                             READY   STATUS      RESTARTS      AGE
capi-kubeadm-bootstrap-system       capi-kubeadm-bootstrap-controller-manager-7ffb6dc8fc-v2nx7       1/1     Running     1 (26m ago)   29m
capi-kubeadm-control-plane-system   capi-kubeadm-control-plane-controller-manager-667999fdb8-wtpgq   1/1     Running     0             29m
capi-system                         capi-controller-manager-65c5769c4c-pfwbl                         1/1     Running     1 (27m ago)   29m
capv-system                         capv-controller-manager-5dc759d4d8-gl2qn                         1/1     Running     1 (27m ago)   29m
cert-manager                        cert-manager-cainjector-cc485fcdc-vfgz7                          1/1     Running     1 (27m ago)   35m
cert-manager                        cert-manager-d6b468546-k8hwm                                     1/1     Running     0             35m
cert-manager                        cert-manager-webhook-dd697458d-fb5kc                             1/1     Running     0             35m
kube-system                         antrea-agent-hqshs                                               2/2     Running     0             25m
kube-system                         antrea-agent-rhpkp                                               2/2     Running     0             25m
kube-system                         antrea-controller-5d75c967fd-m66dl                               1/1     Running     0             25m
kube-system                         coredns-6765cf8944-2dw4z                                         1/1     Running     0             35m
kube-system                         coredns-6765cf8944-mc86v                                         1/1     Running     0             35m
kube-system                         etcd-potato-control-plane-wc6dk                                  1/1     Running     0             35m
kube-system                         kube-apiserver-potato-control-plane-wc6dk                        1/1     Running     0             35m
kube-system                         kube-controller-manager-potato-control-plane-wc6dk               1/1     Running     2 (26m ago)   35m
kube-system                         kube-proxy-7sfg4                                                 1/1     Running     0             34m
kube-system                         kube-proxy-sghwb                                                 1/1     Running     0             35m
kube-system                         kube-scheduler-potato-control-plane-wc6dk                        1/1     Running     2 (26m ago)   35m
kube-system                         kube-vip-potato-control-plane-wc6dk                              1/1     Running     2 (30m ago)   35m
kube-system                         metrics-server-6f64fd6d58-cgpf6                                  1/1     Running     0             27m
kube-system                         vsphere-cloud-controller-manager-jpqdl                           1/1     Running     0             26m
kube-system                         vsphere-csi-controller-84d8c5d48f-4p76m                          6/6     Running     3 (26m ago)   27m
kube-system                         vsphere-csi-node-brhxx                                           3/3     Running     2 (27m ago)   27m
kube-system                         vsphere-csi-node-c62ll                                           3/3     Running     1 (27m ago)   27m
pinniped-concierge                  pinniped-concierge-7ccff787d5-6725j                              1/1     Running     0             25m
pinniped-concierge                  pinniped-concierge-7ccff787d5-hb5c5                              1/1     Running     0             25m
pinniped-concierge                  pinniped-concierge-kube-cert-agent-567869f668-jmgt2              1/1     Running     0             25m
pinniped-supervisor                 pinniped-post-deploy-job--1-69zj5                                0/1     Completed   0             25m
pinniped-supervisor                 pinniped-post-deploy-job--1-zvbjj                                0/1     Error       0             25m
pinniped-supervisor                 pinniped-supervisor-8cbcd65cb-678f4                              1/1     Running     0             24m
pinniped-supervisor                 pinniped-supervisor-8cbcd65cb-fmrb2                              1/1     Running     0             24m
tanzu-system-auth                   dex-6956754f84-kqgdc                                             1/1     Running     0             24m
tanzu-system                        secretgen-controller-578558b4b9-chqt8                            1/1     Running     0             26m
tkg-system                          kapp-controller-5dcd4cdcd8-66zv5                                 1/1     Running     0             32m
tkg-system                          tanzu-addons-controller-manager-79c8599956-hthss                 1/1     Running     0             28m
tkg-system                          tanzu-capabilities-controller-manager-65ccbf9d84-cjvlz           1/1     Running     0             34m
tkg-system                          tanzu-featuregates-controller-manager-7bb59d6f8f-ddz7s           1/1     Running     0             27m
tkr-system                          tkr-controller-manager-6879dcfdd8-fwdv9                          1/1     Running     1 (27m ago)   33m
$ kubectl get packageinstall -A
NAMESPACE    NAME                            PACKAGE NAME                               PACKAGE VERSION                 DESCRIPTION           AGE
tkg-system   antrea                          antrea.tanzu.vmware.com                    1.2.3+vmware.4-tkg.1-advanced   Reconcile succeeded   29m
tkg-system   metrics-server                  metrics-server.tanzu.vmware.com            0.5.1+vmware.1-tkg.1            Reconcile succeeded   29m
tkg-system   pinniped                        pinniped.tanzu.vmware.com                  0.12.0+vmware.1-tkg.1           Reconcile succeeded   29m
tkg-system   secretgen-controller            secretgen-controller.tanzu.vmware.com      0.7.1+vmware.1-tkg.1            Reconcile succeeded   29m
tkg-system   tanzu-addons-manager            addons-manager.tanzu.vmware.com            1.5.0+vmware.1-tkg.3            Reconcile succeeded   33m
tkg-system   tanzu-core-management-plugins   core-management-plugins.tanzu.vmware.com   0.11.1-92-g4d578570+vmware.1    Reconcile succeeded   31m
tkg-system   tanzu-featuregates              featuregates.tanzu.vmware.com              0.11.1-92-g4d578570+vmware.1    Reconcile succeeded   33m
tkg-system   vsphere-cpi                     vsphere-cpi.tanzu.vmware.com               1.22.4+vmware.1-tkg.1           Reconcile succeeded   29m
tkg-system   vsphere-csi                     vsphere-csi.tanzu.vmware.com               2.4.1+vmware.1-tkg.1            Reconcile succeeded   29m

Workload Clusterのデプロイ

$ tanzu login --server potato
✔  successfully logged in to management cluster using the kubeconfig potato
Checking for required plugins...
Installing plugin 'cluster:v0.11.1'
Installing plugin 'kubernetes-release:v0.11.1'
Successfully installed all required plugins
$ tanzu plugin list
  NAME                DESCRIPTION                                                        SCOPE       DISCOVERY       VERSION  STATUS     
  cluster             Kubernetes cluster operations                                      Context     default-potato  v0.11.1  installed  
  kubernetes-release  Kubernetes release operations                                      Context     default-potato  v0.11.1  installed  
  login               Login to the platform                                              Standalone  default         v0.11.1  installed  
  management-cluster  Kubernetes management-cluster operations                           Standalone  default         v0.11.1  installed  
  package             Tanzu package management                                           Standalone  default         v0.11.1  installed  
  pinniped-auth       Pinniped authentication operations (usually not directly invoked)  Standalone  default         v0.11.1  installed  
  secret              Tanzu secret management                                            Standalone  default         v0.11.1  installed 
CLUSTER_NAME: strawberry
VSPHERE_CONTROL_PLANE_DISK_GIB: "20"
VSPHERE_CONTROL_PLANE_ENDPOINT: 192.168.11.122
VSPHERE_CONTROL_PLANE_MEM_MIB: "8192"
VSPHERE_CONTROL_PLANE_NUM_CPUS: "4"
VSPHERE_WORKER_DISK_GIB: "50"
VSPHERE_WORKER_MEM_MIB: "16384"
VSPHERE_WORKER_NUM_CPUS: "4"
WORKER_MACHINE_COUNT: "2"
tanzu cluster create -f $HOME/.config/tanzu/tkg/clusterconfigs/strawberry.yaml -v 9
$ tanzu cluster list --include-management-cluster
  NAME        NAMESPACE   STATUS   CONTROLPLANE  WORKERS  KUBERNETES        ROLES       PLAN  
  strawberry  default     running  1/1           2/2      v1.22.5+vmware.1  <none>      dev   
  potato      tkg-system  running  1/1           1/1      v1.22.5+vmware.1  management  dev 

$ tanzu cluster get strawberry --show-group-members                                                                                                                                
  NAME        NAMESPACE  STATUS   CONTROLPLANE  WORKERS  KUBERNETES        ROLES   
  strawberry  default    running  1/1           2/2      v1.22.5+vmware.1  <none>  
ℹ  

Details:

NAME                                                           READY  SEVERITY  REASON  SINCE  MESSAGE
/strawberry                                                    True                     5m34s         
├─ClusterInfrastructure - VSphereCluster/strawberry            True                     6m45s         
├─ControlPlane - KubeadmControlPlane/strawberry-control-plane  True                     5m34s         
│ └─Machine/strawberry-control-plane-bxrqm                     True                     5m35s         
└─Workers                                                                                             
  └─MachineDeployment/strawberry-md-0                          True                     3m29s         
    ├─Machine/strawberry-md-0-7cd97844c6-82tg6                 True                     3m35s         
    └─Machine/strawberry-md-0-7cd97844c6-fxlv4                 True                     3m35s 
$ tanzu cluster node-pool list strawberry                      
  NAME  NAMESPACE  PHASE    REPLICAS  READY  UPDATED  UNAVAILABLE  
  md-0  default    Running  2         2      2        0          
name: md-1
replicas: 2
vsphere:
  datastore: /Datacenter/datastore/datastore01
tanzu cluster node-pool set strawberry -f $HOME/.config/tanzu/tkg/clusterconfigs/strawberry-md-1.yaml
$ tanzu cluster get strawberry --show-group-members     
  NAME        NAMESPACE  STATUS   CONTROLPLANE  WORKERS  KUBERNETES        ROLES   
  strawberry  default    running  1/1           4/4      v1.22.5+vmware.1  <none>  
ℹ  

Details:

NAME                                                           READY  SEVERITY  REASON  SINCE  MESSAGE
/strawberry                                                    True                     26m           
├─ClusterInfrastructure - VSphereCluster/strawberry            True                     28m           
├─ControlPlane - KubeadmControlPlane/strawberry-control-plane  True                     26m           
│ └─Machine/strawberry-control-plane-bxrqm                     True                     26m           
└─Workers                                                                                             
  ├─MachineDeployment/strawberry-md-0                          True                     24m           
  │ ├─Machine/strawberry-md-0-7cd97844c6-82tg6                 True                     24m           
  │ └─Machine/strawberry-md-0-7cd97844c6-fxlv4                 True                     24m           
  └─MachineDeployment/strawberry-md-1                          True                     37s           
    ├─Machine/strawberry-md-1-55847467dc-7vgjf                 True                     2m48s         
    └─Machine/strawberry-md-1-55847467dc-zc6cw                 True                     2m47s 
tanzu cluster kubeconfig get strawberry --admin
kubectl config use-context strawberry-admin@strawberry
$ kubectl get node -owide 
NAME                               STATUS   ROLES                  AGE     VERSION            INTERNAL-IP     EXTERNAL-IP     OS-IMAGE                 KERNEL-VERSION   CONTAINER-RUNTIME
strawberry-control-plane-bxrqm     Ready    control-plane,master   27m     v1.22.5+vmware.1   192.168.11.39   192.168.11.39   VMware Photon OS/Linux   4.19.224-2.ph3   containerd://1.5.9
strawberry-md-0-7cd97844c6-82tg6   Ready    <none>                 25m     v1.22.5+vmware.1   192.168.11.40   192.168.11.40   VMware Photon OS/Linux   4.19.224-2.ph3   containerd://1.5.9
strawberry-md-0-7cd97844c6-fxlv4   Ready    <none>                 25m     v1.22.5+vmware.1   192.168.11.44   192.168.11.44   VMware Photon OS/Linux   4.19.224-2.ph3   containerd://1.5.9
strawberry-md-1-55847467dc-7vgjf   Ready    <none>                 3m23s   v1.22.5+vmware.1   192.168.11.46   192.168.11.46   VMware Photon OS/Linux   4.19.224-2.ph3   containerd://1.5.9
strawberry-md-1-55847467dc-zc6cw   Ready    <none>                 3m23s   v1.22.5+vmware.1   192.168.11.45   192.168.11.45   VMware Photon OS/Linux   4.19.224-2.ph3   containerd://1.5.9
$ kubectl get pod -owide -A 
NAMESPACE             NAME                                                     READY   STATUS      RESTARTS      AGE     IP              NODE                               NOMINATED NODE   READINESS GATES
kube-system           antrea-agent-4tpj2                                       2/2     Running     0             3m43s   192.168.11.45   strawberry-md-1-55847467dc-zc6cw   <none>           <none>
kube-system           antrea-agent-6sk65                                       2/2     Running     0             3m43s   192.168.11.46   strawberry-md-1-55847467dc-7vgjf   <none>           <none>
kube-system           antrea-agent-ljv5h                                       2/2     Running     0             25m     192.168.11.44   strawberry-md-0-7cd97844c6-fxlv4   <none>           <none>
kube-system           antrea-agent-pcsl6                                       2/2     Running     0             26m     192.168.11.39   strawberry-control-plane-bxrqm     <none>           <none>
kube-system           antrea-agent-q8s5r                                       2/2     Running     0             25m     192.168.11.40   strawberry-md-0-7cd97844c6-82tg6   <none>           <none>
kube-system           antrea-controller-7757c65b4d-ggwcd                       1/1     Running     0             26m     192.168.11.39   strawberry-control-plane-bxrqm     <none>           <none>
kube-system           coredns-6765cf8944-l2cj9                                 1/1     Running     0             27m     100.96.0.4      strawberry-control-plane-bxrqm     <none>           <none>
kube-system           coredns-6765cf8944-xrx56                                 1/1     Running     0             27m     100.96.0.2      strawberry-control-plane-bxrqm     <none>           <none>
kube-system           etcd-strawberry-control-plane-bxrqm                      1/1     Running     0             27m     192.168.11.39   strawberry-control-plane-bxrqm     <none>           <none>
kube-system           kube-apiserver-strawberry-control-plane-bxrqm            1/1     Running     0             27m     192.168.11.39   strawberry-control-plane-bxrqm     <none>           <none>
kube-system           kube-controller-manager-strawberry-control-plane-bxrqm   1/1     Running     0             27m     192.168.11.39   strawberry-control-plane-bxrqm     <none>           <none>
kube-system           kube-proxy-5xxxn                                         1/1     Running     0             27m     192.168.11.39   strawberry-control-plane-bxrqm     <none>           <none>
kube-system           kube-proxy-6v9mb                                         1/1     Running     0             25m     192.168.11.40   strawberry-md-0-7cd97844c6-82tg6   <none>           <none>
kube-system           kube-proxy-gxth8                                         1/1     Running     0             3m43s   192.168.11.46   strawberry-md-1-55847467dc-7vgjf   <none>           <none>
kube-system           kube-proxy-l9lml                                         1/1     Running     0             3m43s   192.168.11.45   strawberry-md-1-55847467dc-zc6cw   <none>           <none>
kube-system           kube-proxy-sm9w5                                         1/1     Running     0             25m     192.168.11.44   strawberry-md-0-7cd97844c6-fxlv4   <none>           <none>
kube-system           kube-scheduler-strawberry-control-plane-bxrqm            1/1     Running     0             27m     192.168.11.39   strawberry-control-plane-bxrqm     <none>           <none>
kube-system           kube-vip-strawberry-control-plane-bxrqm                  1/1     Running     0             27m     192.168.11.39   strawberry-control-plane-bxrqm     <none>           <none>
kube-system           metrics-server-865744f7cc-fnn24                          1/1     Running     0             26m     100.96.1.2      strawberry-md-0-7cd97844c6-fxlv4   <none>           <none>
kube-system           vsphere-cloud-controller-manager-mp62j                   1/1     Running     0             26m     192.168.11.39   strawberry-control-plane-bxrqm     <none>           <none>
kube-system           vsphere-csi-controller-6795b58668-m9dk4                  6/6     Running     0             25m     100.96.0.5      strawberry-control-plane-bxrqm     <none>           <none>
kube-system           vsphere-csi-node-4v28r                                   3/3     Running     1 (24m ago)   25m     192.168.11.39   strawberry-control-plane-bxrqm     <none>           <none>
kube-system           vsphere-csi-node-9thk8                                   3/3     Running     1 (24m ago)   25m     192.168.11.40   strawberry-md-0-7cd97844c6-82tg6   <none>           <none>
kube-system           vsphere-csi-node-b929h                                   3/3     Running     0             3m43s   192.168.11.46   strawberry-md-1-55847467dc-7vgjf   <none>           <none>
kube-system           vsphere-csi-node-hpmjh                                   3/3     Running     0             3m43s   192.168.11.45   strawberry-md-1-55847467dc-zc6cw   <none>           <none>
kube-system           vsphere-csi-node-lslc9                                   3/3     Running     1 (24m ago)   25m     192.168.11.44   strawberry-md-0-7cd97844c6-fxlv4   <none>           <none>
pinniped-concierge    pinniped-concierge-76f7d76745-dmltr                      1/1     Running     0             25m     100.96.1.4      strawberry-md-0-7cd97844c6-fxlv4   <none>           <none>
pinniped-concierge    pinniped-concierge-76f7d76745-fx2jp                      1/1     Running     0             25m     100.96.2.2      strawberry-md-0-7cd97844c6-82tg6   <none>           <none>
pinniped-concierge    pinniped-concierge-kube-cert-agent-6444b74d5c-8lt5w      1/1     Running     0             24m     192.168.11.39   strawberry-control-plane-bxrqm     <none>           <none>
pinniped-supervisor   pinniped-post-deploy-job--1-vmjv5                        0/1     Completed   0             25m     100.96.2.3      strawberry-md-0-7cd97844c6-82tg6   <none>           <none>
tanzu-system          secretgen-controller-f88959d8f-8mctj                     1/1     Running     0             26m     100.96.1.3      strawberry-md-0-7cd97844c6-fxlv4   <none>           <none>
tkg-system            kapp-controller-7b89d4579d-k2lvt                         1/1     Running     0             27m     192.168.11.39   strawberry-control-plane-bxrqm     <none>           <none>
tkg-system            tanzu-capabilities-controller-manager-65ccbf9d84-5v8f5   1/1     Running     0             27m     100.96.0.3      strawberry-control-plane-bxrqm     <none>           <none>
$ kubectl get packageinstall -A  
NAMESPACE    NAME                   PACKAGE NAME                            PACKAGE VERSION                 DESCRIPTION           AGE
tkg-system   antrea                 antrea.tanzu.vmware.com                 1.2.3+vmware.4-tkg.1-advanced   Reconcile succeeded   27m
tkg-system   metrics-server         metrics-server.tanzu.vmware.com         0.5.1+vmware.1-tkg.1            Reconcile succeeded   27m
tkg-system   pinniped               pinniped.tanzu.vmware.com               0.12.0+vmware.1-tkg.1           Reconcile succeeded   26m
tkg-system   secretgen-controller   secretgen-controller.tanzu.vmware.com   0.7.1+vmware.1-tkg.1            Reconcile succeeded   27m
tkg-system   vsphere-cpi            vsphere-cpi.tanzu.vmware.com            1.22.4+vmware.1-tkg.1           Reconcile succeeded   27m
tkg-system   vsphere-csi            vsphere-csi.tanzu.vmware.com            2.4.1+vmware.1-tkg.1            Reconcile succeeded   26m