IK.AM

@making's tech note


Tanzu Application Platform 1.1を4 cpu, 8 GB memoryのminikubeにLocal Registryを使ってインストールするメモ

🗃 {Dev/CaaS/Kubernetes/TAP}
🏷 Kubernetes 🏷 Cartographer 🏷 minukube 🏷 Tanzu 🏷 TAP 🏷 Knative 
🗓 Updated at 2022-06-23T12:52:12+09:00  🗓 Created at 2022-06-19T12:24:27+09:00 {✒️️ Edit  ⏰ History  🗑 Delete}

Tanzu Application Platform 1.1 をminikubeにインストールします。

本記事ではTAPをminikubeにLocal Registryを使ってInstallし、"Hello World"なアプリケーションをソースコードからデプロイする機能("Source to URL")を試します。 少ないリソースでも動作するようにインストールするコンポーネントは最小にします。

目次

minikubeクラスタの作成

あえて少ないリソース(4 cpu, 8 GB memory)にします。

minikube start --memory=8192 --cpus=4 --disk-size=70GB  --kubernetes-version='1.22.10' --driver='hyperkit'

Local Registryのインストール

minikubeのregistry addonだとPersistent Volumeがattachされておらず、minikube stopするとデータが失われるので、 カスタマイズした次のマニフェストを使います。

cat <<EOF | kubectl apply -f-
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: kube-registry
  namespace: kube-system
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 30Gi 
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    kubernetes.io/minikube-addons: registry
  name: registry
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      actual-registry: "true"
  strategy:
    type: Recreate
  template:    
    metadata:
      labels:
        actual-registry: "true"
        kubernetes.io/minikube-addons: registry
    spec:
      containers:
      - image: registry:2.7.1@sha256:d5459fcb27aecc752520df4b492b08358a1912fcdfa454f7d2101d4b09991daa
        imagePullPolicy: IfNotPresent
        name: registry
        ports:
        - containerPort: 5000
          protocol: TCP
        env:
        - name: REGISTRY_STORAGE_DELETE_ENABLED
          value: "true"
        - name: REGISTRY_VALIDATION_DISABLED
          value: "true"
        - name: REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY
          value: /var/lib/registry
        volumeMounts:
        - name: image-store
          mountPath: /var/lib/registry        
      volumes:
      - name: image-store
        persistentVolumeClaim:
          claimName: kube-registry      
---
apiVersion: v1
kind: Service
metadata:
  labels:
    kubernetes.io/minikube-addons: registry
  name: registry
  namespace: kube-system
spec:
  type: ClusterIP
  ports:
  - port: 80
    name: http
    targetPort: 5000
  - port: 443
    name: https
    targetPort: 443
  selector:
    actual-registry: "true"
    kubernetes.io/minikube-addons: registry
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: registry-proxy
  namespace: kube-system
spec:
  selector:
    matchLabels:
      registry-proxy: "true"
  template:
    metadata:
      labels:
        registry-proxy: "true"
        kubernetes.io/minikube-addons: registry
    spec:
      containers:
      - image: gcr.io/google_containers/kube-registry-proxy:0.4@sha256:1040f25a5273de0d72c54865a8efd47e3292de9fb8e5353e3fa76736b854f2da
        imagePullPolicy: IfNotPresent
        name: registry-proxy
        ports:
        - name: registry
          containerPort: 80
          hostPort: 5000
        env:
        - name: REGISTRY_HOST
          value: registry.kube-system.svc.cluster.local
        - name: REGISTRY_PORT
          value: "80"
EOF

minikubeのdockerがregistry serviceのDNS名を解決できるように、Node上の/etc/hostsにregistry serviceのClusterIPを明示します。

echo "$(kubectl get svc -n kube-system registry -ojsonpath='{.spec.clusterIP}') registry.kube-system.svc.cluster.local" | minikube ssh --native-ssh=false "sudo tee -a /etc/hosts"

Cluster Essentials for VMware Tanzuのインストール

TAPのインストールに必要なKapp ControllerとSecretgen Controllerをデプロイするために Cluster Essentials for VMware Tanzu をインストールします。

# Mac
pivnet download-product-files --product-slug='tanzu-cluster-essentials' --release-version='1.1.0' --product-file-id=1191985
# Linux
pivnet download-product-files --product-slug='tanzu-cluster-essentials' --release-version='1.1.0' --product-file-id=1191987
# Windows
pivnet download-product-files --product-slug='tanzu-cluster-essentials' --release-version='1.1.0' --product-file-id=1191983

Cluster Essentialsのimgpkg bundleをlocal registryにrelocateします。

まずはTanzuNet Registryにログインします。

TANZUNET_USERNAME=...
TANZUNET_PASSWORD=...

docker login registry.tanzu.vmware.com -u ${TANZUNET_USERNAME} -p ${TANZUNET_PASSWORD}

Cluster Essentialsのimgpkg bundleをtarファイルに保存します。

imgpkg copy -b registry.tanzu.vmware.com/tanzu-cluster-essentials/cluster-essentials-bundle:1.1.0 --to-tar ~/cluster-essentials-bundle-1.1.0.tar

localhost:5000でregistryにアクセスできるようにport-forwardします。

kubectl port-forward --namespace kube-system service/registry 5000:80

tarファイルのimgpkg bundleをlocalhost:5000にrelocateします。

imgpkg copy --tar ~/cluster-essentials-bundle-1.1.0.tar --to-repo localhost:5000/tanzu-cluster-essentials/cluster-essentials-bundle

relocateしたimgpkg bundleを使ってCluster Essentialsをインストールします。

mkdir tanzu-cluster-essentials
tar xzvf tanzu-cluster-essentials-*-amd64-1.1.0.tgz -C tanzu-cluster-essentials

cd tanzu-cluster-essentials

export INSTALL_REGISTRY_HOSTNAME=localhost:5000
export INSTALL_REGISTRY_USERNAME=admin
export INSTALL_REGISTRY_PASSWORD=admin
export INSTALL_BUNDLE=${INSTALL_REGISTRY_HOSTNAME}/tanzu-cluster-essentials/cluster-essentials-bundle:1.1.0

./install.sh --yes
cd ..

Pod一覧を確認します。

$ kubectl get pod -A
NAMESPACE              NAME                                    READY   STATUS    RESTARTS   AGE
kapp-controller        kapp-controller-9475c64b6-tjsq8         1/1     Running   0          28s
kube-system            coredns-78fcd69978-67k9l                1/1     Running   0          8m59s
kube-system            etcd-minikube                           1/1     Running   0          9m14s
kube-system            kube-apiserver-minikube                 1/1     Running   0          9m12s
kube-system            kube-controller-manager-minikube        1/1     Running   0          9m14s
kube-system            kube-proxy-5srjl                        1/1     Running   0          9m
kube-system            kube-scheduler-minikube                 1/1     Running   0          9m12s
kube-system            registry-hdgt5                          1/1     Running   0          8m59s
kube-system            registry-proxy-2hz6x                    1/1     Running   0          8m59s
kube-system            storage-provisioner                     1/1     Running   0          9m11s
secretgen-controller   secretgen-controller-6cfb586cd7-85zzh   1/1     Running   0          8s

Tanzu Application Platformのインストール

TAP用Package Repositoryの登録

TAPのimgpkg bundleをtarファイルに保存します。 約16GBあるのでかなり時間がかかります。

👆 tarファイルを経由せずに直接registryにcopyすることもできますが、
失敗したときにダウンロードからやり直しになってしまうので、
試行錯誤を見越してtarでダウンロードしておいた方が効率的です。

TAP_VERSION=1.1.1
imgpkg copy -b registry.tanzu.vmware.com/tanzu-application-platform/tap-packages:${TAP_VERSION} --to-tar ~/tap-${TAP_VERSION}.tar

tarファイルのimgpkg bundleをlocalhost:5000にrelocateします。

imgpkg copy --tar ~/tap-${TAP_VERSION}.tar --to-repo localhost:5000/tanzu-application-platform/tap-packages

Package Repositoryの設定をします。kapp controllerがlocalhost:5000にアクセスできなかったので、ホスト名にはregistry.kube-system.svc.cluster.localを使用します。

kubectl create ns tap-install

tanzu secret registry add tap-registry \
  --username "${INSTALL_REGISTRY_USERNAME}" \
  --password "${INSTALL_REGISTRY_PASSWORD}" \
  --server registry.kube-system.svc.cluster.local \
  --export-to-all-namespaces \
  --yes \
  --namespace tap-install

tanzu package repository add tanzu-tap-repository \
  --url registry.kube-system.svc.cluster.local/tanzu-application-platform/tap-packages:${TAP_VERSION} \
  --namespace tap-install

利用可能なパッケージを確認します。

$ tanzu package available list --namespace tap-install
  NAME                                                 DISPLAY-NAME                                                              SHORT-DESCRIPTION                                                                                                                                              LATEST-VERSION  
  accelerator.apps.tanzu.vmware.com                    Application Accelerator for VMware Tanzu                                  Used to create new projects and configurations.                                                                                                                1.1.2           
  api-portal.tanzu.vmware.com                          API portal                                                                A unified user interface to enable search, discovery and try-out of API endpoints at ease.                                                                     1.0.15          
  backend.appliveview.tanzu.vmware.com                 Application Live View for VMware Tanzu                                    App for monitoring and troubleshooting running apps                                                                                                            1.1.1           
  build.appliveview.tanzu.vmware.com                   Application Live View Conventions for VMware Tanzu                        Application Live View convention server                                                                                                                        1.0.2           
  buildservice.tanzu.vmware.com                        Tanzu Build Service                                                       Tanzu Build Service enables the building and automation of containerized software workflows securely and at scale.                                             1.5.1           
  cartographer.tanzu.vmware.com                        Cartographer                                                              Kubernetes native Supply Chain Choreographer.                                                                                                                  0.3.0           
  cnrs.tanzu.vmware.com                                Cloud Native Runtimes                                                     Cloud Native Runtimes is a serverless runtime based on Knative                                                                                                 1.2.0           
  connector.appliveview.tanzu.vmware.com               Application Live View Connector for VMware Tanzu                          App for discovering and registering running apps                                                                                                               1.1.1           
  controller.conventions.apps.tanzu.vmware.com         Convention Service for VMware Tanzu                                       Convention Service enables app operators to consistently apply desired runtime configurations to fleets of workloads.                                          0.6.3           
  controller.source.apps.tanzu.vmware.com              Tanzu Source Controller                                                   Tanzu Source Controller enables workload create/update from source code.                                                                                       0.3.3           
  conventions.appliveview.tanzu.vmware.com             Application Live View Conventions for VMware Tanzu                        Application Live View convention server                                                                                                                        1.1.1           
  developer-conventions.tanzu.vmware.com               Tanzu App Platform Developer Conventions                                  Developer Conventions                                                                                                                                          0.6.0           
  fluxcd.source.controller.tanzu.vmware.com            Flux Source Controller                                                    The source-controller is a Kubernetes operator, specialised in artifacts acquisition from external sources such as Git, Helm repositories and S3 buckets.      0.16.4          
  grype.scanning.apps.tanzu.vmware.com                 Grype for Supply Chain Security Tools - Scan                              Default scan templates using Anchore Grype                                                                                                                     1.1.1           
  image-policy-webhook.signing.apps.tanzu.vmware.com   Image Policy Webhook                                                      Image Policy Webhook enables defining of a policy to restrict unsigned container images.                                                                       1.1.2           
  learningcenter.tanzu.vmware.com                      Learning Center for Tanzu Application Platform                            Guided technical workshops                                                                                                                                     0.2.0           
  metadata-store.apps.tanzu.vmware.com                 Supply Chain Security Tools - Store                                       Post SBoMs and query for image, package, and vulnerability metadata.                                                                                           1.1.3           
  ootb-delivery-basic.tanzu.vmware.com                 Tanzu App Platform Out of The Box Delivery Basic                          Out of The Box Delivery Basic.                                                                                                                                 0.7.1           
  ootb-supply-chain-basic.tanzu.vmware.com             Tanzu App Platform Out of The Box Supply Chain Basic                      Out of The Box Supply Chain Basic.                                                                                                                             0.7.1           
  ootb-supply-chain-testing-scanning.tanzu.vmware.com  Tanzu App Platform Out of The Box Supply Chain with Testing and Scanning  Out of The Box Supply Chain with Testing and Scanning.                                                                                                         0.7.1           
  ootb-supply-chain-testing.tanzu.vmware.com           Tanzu App Platform Out of The Box Supply Chain with Testing               Out of The Box Supply Chain with Testing.                                                                                                                      0.7.1           
  ootb-templates.tanzu.vmware.com                      Tanzu App Platform Out of The Box Templates                               Out of The Box Templates.                                                                                                                                      0.7.1           
  run.appliveview.tanzu.vmware.com                     Application Live View for VMware Tanzu                                    App for monitoring and troubleshooting running apps                                                                                                            1.0.3           
  scanning.apps.tanzu.vmware.com                       Supply Chain Security Tools - Scan                                        Scan for vulnerabilities and enforce policies directly within Kubernetes native Supply Chains.                                                                 1.1.1           
  service-bindings.labs.vmware.com                     Service Bindings for Kubernetes                                           Service Bindings for Kubernetes implements the Service Binding Specification.                                                                                  0.7.1           
  services-toolkit.tanzu.vmware.com                    Services Toolkit                                                          The Services Toolkit enables the management, lifecycle, discoverability and connectivity of Service Resources (databases, message queues, DNS records, etc.).  0.6.0           
  spring-boot-conventions.tanzu.vmware.com             Tanzu Spring Boot Conventions Server                                      Default Spring Boot convention server.                                                                                                                         0.4.0           
  tap-auth.tanzu.vmware.com                            Default roles for Tanzu Application Platform                              Default roles for Tanzu Application Platform                                                                                                                   1.0.1           
  tap-gui.tanzu.vmware.com                             Tanzu Application Platform GUI                                            web app graphical user interface for Tanzu Application Platform                                                                                                1.1.1           
  tap-telemetry.tanzu.vmware.com                       Telemetry Collector for Tanzu Application Platform                        Tanzu Application Plaform Telemetry                                                                                                                            0.1.4           
  tap.tanzu.vmware.com                                 Tanzu Application Platform                                                Package to install a set of TAP components to get you started based on your use case.                                                                          1.1.1           
  tekton.tanzu.vmware.com                              Tekton Pipelines                                                          Tekton Pipelines is a framework for creating CI/CD systems.                                                                                                    0.33.5          
  workshops.learningcenter.tanzu.vmware.com            Workshop Building Tutorial                                                Workshop Building Tutorial                                                                                                                                     0.2.0

Iterate Profileのインストール

TAPをインストールするためのtap-values.ymlを作成します。 cnrs.domain_nameには仮のドメインを指定します。あとでenvoyのExternal IPが設定されてから変更します。

4 cpuしかないので使用しないパッケージはexcluded_packages除外します。 また、Cloud Native RuntimesはKnative Servingしか使わないので、それ以外のリソースを削除するoverlayを設定します。

cat <<EOF > tap-values.yml
profile: iterate

ceip_policy_disclosed: true

cnrs:
  domain_name: tap.example.com
  domain_template: "{{.Name}}-{{.Namespace}}.{{.Domain}}"
  default_tls_secret: tanzu-system-ingress/cnrs-default-tls
  provider: local

buildservice:
  kp_default_repository: registry.kube-system.svc.cluster.local/build-service
  kp_default_repository_username: admin
  kp_default_repository_password: admin
  tanzunet_username: ${TANZUNET_USERNAME}
  tanzunet_password: ${TANZUNET_PASSWORD}
  enable_automatic_dependency_updates: true

supply_chain: basic

ootb_supply_chain_basic:
  registry:
    server: registry.kube-system.svc.cluster.local
    repository: supplychain

contour:
  envoy:
    service:
      type: LoadBalancer
      externalTrafficPolicy: Local

package_overlays:
- name: cnrs
  secrets:
  - name: cnrs-default-tls
  - name: cnrs-slim

excluded_packages:
- backend.appliveview.tanzu.vmware.com
- connector.appliveview.tanzu.vmware.com
- image-policy-webhook.signing.apps.tanzu.vmware.com     
EOF

Cloud Native Runtimesで使用するデフォルトのTLS証明書を用意するための次の定義をoverlayで作成します。以下のドキュメントを参考にしました。

cat <<EOF > cnrs-default-tls.yml
#@ load("@ytt:data", "data")
#@ load("@ytt:overlay", "overlay")
#@ namespace = data.values.ingress.external.namespace
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: cnrs-selfsigned-issuer
  namespace: #@ namespace
spec:
  selfSigned: { }
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: cnrs-ca
  namespace: #@ namespace
spec:
  commonName: cnrs-ca
  isCA: true
  issuerRef:
    kind: Issuer
    name: cnrs-selfsigned-issuer
  secretName: cnrs-ca
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: cnrs-ca-issuer
  namespace: #@ namespace
spec:
  ca:
    secretName: cnrs-ca
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: cnrs-default-tls
  namespace: #@ namespace
spec:
  dnsNames:
  - #@ "*.{}".format(data.values.domain_name)
  issuerRef:
    kind: Issuer
    name: cnrs-ca-issuer
  secretName: cnrs-default-tls
---
apiVersion: projectcontour.io/v1
kind: TLSCertificateDelegation
metadata:
  name: contour-delegation
  namespace: #@ namespace
spec:
  delegations:
  - secretName: cnrs-default-tls
    targetNamespaces:
    - "*"
#@overlay/match by=overlay.subset({"metadata":{"name":"config-network"}, "kind": "ConfigMap"})
---
data:
  #@overlay/match missing_ok=True
  default-external-scheme: https
EOF

Cloud Native RuntimesからKnative Serving以外のリソースを削除するoverlayを作成します。

cat <<EOF > cnrs-slim.yml
#@ load("@ytt:overlay", "overlay")
#@overlay/match by=overlay.subset({"metadata":{"namespace":"knative-eventing"}}), expects="1+"
#@overlay/remove
---
#@overlay/match by=overlay.subset({"metadata":{"namespace":"knative-sources"}}), expects="1+"
#@overlay/remove
---
#@overlay/match by=overlay.subset({"metadata":{"namespace":"triggermesh"}}), expects="1+"
#@overlay/remove
---
#@overlay/match by=overlay.subset({"metadata":{"namespace":"vmware-sources"}}), expects="1+"
#@overlay/remove
---
EOF

overlayファイルをSecretとして作成します。

kubectl -n tap-install create secret generic cnrs-default-tls \
  -o yaml \
  --dry-run=client \
  --from-file=cnrs-default-tls.yml \
  | kubectl apply -f-

kubectl -n tap-install create secret generic cnrs-slim \
  -o yaml \
  --dry-run=client \
  --from-file=cnrs-slim.yml \
  | kubectl apply -f-

LoadBalancer Serviceを作成できるように別のコンソールでminikube tunnelを実行しておきます。

minikube tunnel

TAPをインストールします。

tanzu package install tap -p tap.tanzu.vmware.com -v ${TAP_VERSION} --values-file tap-values.yml -n tap-install

インストールの進捗は次のコマンドで確認します。

watch kubectl get app -n tap-install

全てのappが Reconcile succeeded になるまで待ちます。

$ kubectl get app -n tap-install 
NAME                       DESCRIPTION           SINCE-DEPLOY   AGE
appliveview-conventions    Reconcile succeeded   10m            11m
buildservice               Reconcile succeeded   17m            17m
cartographer               Reconcile succeeded   19s            12m
cert-manager               Reconcile succeeded   68s            17m
cnrs                       Reconcile succeeded   10m            11m
contour                    Reconcile succeeded   3m9s           12m
conventions-controller     Reconcile succeeded   12m            12m
developer-conventions      Reconcile succeeded   10m            11m
fluxcd-source-controller   Reconcile succeeded   18s            17m
ootb-delivery-basic        Reconcile succeeded   4s             10m
ootb-supply-chain-basic    Reconcile succeeded   2s             10m
ootb-templates             Reconcile succeeded   18s            11m
service-bindings           Reconcile succeeded   10m            17m
services-toolkit           Reconcile succeeded   19s            17m
source-controller          Reconcile succeeded   28s            17m
spring-boot-conventions    Reconcile succeeded   10m            11m
tap                        Reconcile succeeded   3m3s           18m
tap-auth                   Reconcile succeeded   7m50s          17m
tap-telemetry              Reconcile succeeded   25s            17m
tekton-pipelines           Reconcile succeeded   25s            17m

インストールされたパッケージは次の通りです。

$ kubectl get packageinstall -n tap-install 
NAME                       PACKAGE NAME                                         PACKAGE VERSION   DESCRIPTION           AGE
appliveview-conventions    conventions.appliveview.tanzu.vmware.com       1.1.1             Reconcile succeeded   11m
buildservice               buildservice.tanzu.vmware.com                  1.5.1             Reconcile succeeded   18m
cartographer               cartographer.tanzu.vmware.com                  0.3.0             Reconcile succeeded   12m
cert-manager               cert-manager.tanzu.vmware.com                  1.5.3+tap.2       Reconcile succeeded   18m
cnrs                       cnrs.tanzu.vmware.com                          1.2.0             Reconcile succeeded   11m
contour                    contour.tanzu.vmware.com                       1.18.2+tap.2      Reconcile succeeded   12m
conventions-controller     controller.conventions.apps.tanzu.vmware.com   0.6.3             Reconcile succeeded   12m
developer-conventions      developer-conventions.tanzu.vmware.com         0.6.0             Reconcile succeeded   11m
fluxcd-source-controller   fluxcd.source.controller.tanzu.vmware.com      0.16.4            Reconcile succeeded   18m
ootb-delivery-basic        ootb-delivery-basic.tanzu.vmware.com           0.7.1             Reconcile succeeded   10m
ootb-supply-chain-basic    ootb-supply-chain-basic.tanzu.vmware.com       0.7.1             Reconcile succeeded   10m
ootb-templates             ootb-templates.tanzu.vmware.com                0.7.1             Reconcile succeeded   11m
service-bindings           service-bindings.labs.vmware.com               0.7.1             Reconcile succeeded   18m
services-toolkit           services-toolkit.tanzu.vmware.com              0.6.0             Reconcile succeeded   18m
source-controller          controller.source.apps.tanzu.vmware.com        0.3.3             Reconcile succeeded   18m
spring-boot-conventions    spring-boot-conventions.tanzu.vmware.com       0.4.0             Reconcile succeeded   11m
tap                        tap.tanzu.vmware.com                           1.1.1             Reconcile succeeded   18m
tap-auth                   tap-auth.tanzu.vmware.com                      1.0.1             Reconcile succeeded   18m
tap-telemetry              tap-telemetry.tanzu.vmware.com                 0.1.4             Reconcile succeeded   18m
tekton-pipelines           tekton.tanzu.vmware.com                        0.33.5            Reconcile succeeded   18m

デプロイされたPodは次の通りです。

$ kubectl get pod -A
NAMESPACE                   NAME                                                   READY   STATUS    RESTARTS   AGE
app-live-view-conventions   appliveview-webhook-85cfc47f56-kbwk9                  1/1     Running   0             11m
build-service               build-pod-image-fetcher-bp478                         5/5     Running   0             18m
build-service               dependency-updater-controller-6b76c9fbd7-8r9m6        1/1     Running   0             11m
build-service               secret-syncer-controller-5599c574fc-5vnfr             1/1     Running   0             18m
build-service               smart-warmer-image-fetcher-wr997                      2/2     Running   0             8m14s
build-service               warmer-controller-895db679b-4bt9j                     1/1     Running   0             10m
cartographer-system         cartographer-controller-586456cddb-9vsbt              1/1     Running   1 (11m ago)   13m
cert-injection-webhook      cert-injection-webhook-78d9597c66-npk8h               1/1     Running   0             18m
cert-manager                cert-manager-5d4d657bcf-87574                         1/1     Running   0             18m
cert-manager                cert-manager-cainjector-74c86dc6b7-lklwh              1/1     Running   0             18m
cert-manager                cert-manager-webhook-847968c79b-bh962                 1/1     Running   0             18m
conventions-system          conventions-controller-manager-7866d4f764-86jmg       1/1     Running   0             13m
developer-conventions       webhook-58b85b8877-97xtc                              1/1     Running   0             10m
flux-system                 source-controller-c6b666ffb-cdnxg                     1/1     Running   0             12m
kapp-controller             kapp-controller-9475c64b6-tjsq8                       1/1     Running   0             41m
knative-serving             activator-778d7f5847-wlmmx                            1/1     Running   0             11m
knative-serving             autoscaler-8bd944b87-9mb59                            1/1     Running   0             11m
knative-serving             autoscaler-hpa-7b5f45b48b-z2kff                       1/1     Running   0             11m
knative-serving             controller-5b6f69599-sq7kj                            1/1     Running   0             11m
knative-serving             domain-mapping-bcbdfcb4b-dnzll                        1/1     Running   0             11m
knative-serving             domainmapping-webhook-7658bd7658-fhd8w                1/1     Running   0             11m
knative-serving             net-certmanager-controller-5bc7477865-tqv84           1/1     Running   0             11m
knative-serving             net-certmanager-webhook-7997595d7f-xwzsr              1/1     Running   0             11m
knative-serving             net-contour-controller-6f6cdfcb8d-r2kkc               1/1     Running   0             11m
knative-serving             webhook-6cdd8644cf-hslbm                              1/1     Running   0             11m
kpack                       kpack-controller-6464dc4769-vmg2n                     1/1     Running   0             18m
kpack                       kpack-webhook-9cb9d8b9b-n27hd                         1/1     Running   0             18m
kube-system                 coredns-78fcd69978-67k9l                              1/1     Running   0             49m
kube-system                 etcd-minikube                                         1/1     Running   0             49m
kube-system                 kube-apiserver-minikube                               1/1     Running   0             49m
kube-system                 kube-controller-manager-minikube                      1/1     Running   0             49m
kube-system                 kube-proxy-5srjl                                      1/1     Running   0             49m
kube-system                 kube-scheduler-minikube                               1/1     Running   0             49m
kube-system                 registry-d698l                                        1/1     Running   0             28m
kube-system                 registry-proxy-k7jr4                                  1/1     Running   0             28m
kube-system                 storage-provisioner                                   1/1     Running   0             49m
secretgen-controller        secretgen-controller-6cfb586cd7-85zzh                 1/1     Running   0             40m
service-bindings            manager-6f89b667c7-w2dkd                              1/1     Running   0             10m
services-toolkit            services-toolkit-controller-manager-6497568f5-2qkxf   1/1     Running   0             9m8s
source-system               source-controller-manager-866f69bfcc-xtx6v            1/1     Running   0             8m16s
spring-boot-convention      spring-boot-webhook-5468c975c8-94wcz                  1/1     Running   0             10m
stacks-operator-system      controller-manager-55cfb75c89-ml72d                   1/1     Running   0             18m
tanzu-system-ingress        contour-6d97d46467-jxbv9                              1/1     Running   0             11m
tanzu-system-ingress        contour-6d97d46467-tclwv                              1/1     Running   0             11m
tanzu-system-ingress        envoy-hxln2                                           2/2     Running   0             11m
tap-telemetry               tap-telemetry-controller-6bb5d69d4b-v5lxr             1/1     Running   0             8m14s
tekton-pipelines            tekton-pipelines-controller-67dfb5fc8c-pnb9n          1/1     Running   0             7m35s
tekton-pipelines            tekton-pipelines-webhook-7f4b84b99b-96nw2             1/1     Running   0             7m34s

Envoyに設定されたExternal IPを使って、cnrs.domain_nameを変更します。ドメイン名にはsslip.ioを使用します。 例えば、External IPが10.99.0.147の場合はcnrs.domain_nameに*.10-99-0-147.sslip.ioを指定します。

次のコマンドでtap-values.ymlを更新します。

sed -i."" "s|tap.example.com|$(kubectl get -n tanzu-system-ingress svc envoy -ojsonpath='{.status.loadBalancer.ingress[0].ip}' | sed 's/\./-/g').sslip.io|g" tap-values.yml

TAPを更新します。

tanzu package installed update tap -n tap-install -v ${TAP_VERSION} -f tap-values.yml 

Default TLSのCertificateのDNS名が更新されたことを確認してください。

$ kubectl get certificate -n tanzu-system-ingress cnrs-default-tls -ojsonpath='{.spec.dnsNames[0]}'
*.10-99-0-147.sslip.io

👆 sslip.ioにアクセスできない環境の場合は、
ラップトップ上の/etc/hostsに今後使用する127.0.0.1 <...>.tap.example.comを一つずつ設定してください。

Workloadのデプロイ

Workloadを作成するための事前準備

https://docs.vmware.com/en/Tanzu-Application-Platform/1.1/tap/GUID-install-components.html#setup (一部変更しています)

kubectl create ns demo
tanzu secret registry add registry-credentials --server registry.kube-system.svc.cluster.local --username admin --password admin --namespace demo
cat <<EOF | kubectl -n demo apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: tap-registry
  annotations:
    secretgen.carvel.dev/image-pull-secret: ""
type: kubernetes.io/dockerconfigjson
data:
  .dockerconfigjson: e30K
---
apiVersion: v1
kind: Secret
metadata:
  name: git-ssh
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
secrets:
  - name: registry-credentials
  - name: git-ssh
imagePullSecrets:
  - name: registry-credentials
  - name: tap-registry
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: default-permit-deliverable
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: deliverable
subjects:
  - kind: ServiceAccount
    name: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: default-permit-workload
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: workload
subjects:
  - kind: ServiceAccount
    name: default
EOF

Node.jsアプリのデプロイ

tanzu apps workload apply hello \
  --app hello \
  --git-repo https://github.com/making/hello-nodejs \
  --git-branch master \
  --type web \
  -n demo \
  -y
tanzu apps workload tail hello -n demo   

作成されるリソースを確認したければ次のコマンドをwatchしてください。

watch kubectl get pod,gitrepo,imgs,build,podintent,taskrun,imagerepository,app,ksvc,certificate,httpproxy -n demo -owide
$ tanzu apps workload get -n demo hello
# hello: Ready
---
lastTransitionTime: "2022-06-18T19:12:28Z"
message: ""
reason: Ready
status: "True"
type: Ready

Pods
NAME                                      STATUS      RESTARTS   AGE
hello-00001-deployment-56cccdf645-zs5sm   Running     0          43s
hello-build-1-build-pod                   Succeeded   0          112s
hello-config-writer-lzpz8-pod             Succeeded   0          72s

Knative Services
NAME    READY   URL
hello   Ready   https://hello-demo.10-99-0-147.sslip.io
$ curl -k https://hello-demo.10-99-0-147.sslip.io
Hello Tanzu!!

確認が終わればWorkloadを削除します。

tanzu apps workload delete -n demo hello -y

Javaアプリのデプロイ

tanzu apps workload apply spring-music \
  --app spring-music \
  --git-repo https://github.com/scottfrederick/spring-music \
  --git-branch tanzu \
  --type web \
  --annotation autoscaling.knative.dev/minScale=1 \
  -n demo \
  -y
tanzu apps workload tail spring-music -n demo   

ビルド中に次のようにホスト名が解決できないエラーが出ることがありました。

spring-music-build-1-build-pod[build]             > Could not resolve org.springframework.cloud:spring-cloud-bindings:1.8.1.
spring-music-build-1-build-pod[build]                > Could not get resource 'https://repo.spring.io/release/org/springframework/cloud/spring-cloud-bindings/1.8.1/spring-cloud-bindings-1.8.1.pom'.
spring-music-build-1-build-pod[build]                   > Could not GET 'https://repo.spring.io/release/org/springframework/cloud/spring-cloud-bindings/1.8.1/spring-cloud-bindings-1.8.1.pom'.
spring-music-build-1-build-pod[build]                      > repo.spring.io

その場合はCoreDNSの設定を変更して、8.8.8.8を使って名前解決するようにすれば良いです。

kubectl apply -f https://github.com/categolj/k8s-manifests/raw/main/common/coredns/configmap.yml
$ tanzu apps workload get -n demo spring-music
# spring-music: Ready
---
lastTransitionTime: "2022-06-18T19:22:35Z"
message: ""
reason: Ready
status: "True"
type: Ready

Pods
NAME                                            STATUS      RESTARTS   AGE
spring-music-00001-deployment-d885b656f-cbmlk   Running     0          6m45s
spring-music-build-1-build-pod                  Succeeded   0          10m
spring-music-config-writer-zpt45-pod            Succeeded   0          6m57s

Knative Services
NAME           READY   URL
spring-music   Ready   https://spring-music-demo.10-99-0-147.sslip.io
image

"THIS IS UNSAFE"を入力

image

Spring Bootの場合は自動でmanagement.server.port=8081及びmanagement.endpoints.web.exposure.include=*が設定されます。
またSpring Boot 2.6以上の場合は、management.endpoint.health.probes.add-additional-paths=trueが設定され、readiness probeに /readyz がliveness probeに /livez のpathが設定されます。

$ kubectl get ksvc -n demo spring-music -oyaml | kubectl neat
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  annotations:
    kapp.k14s.io/identity: v1;demo/serving.knative.dev/Service/spring-music;serving.knative.dev/v1
    kapp.k14s.io/original: '{"apiVersion":"serving.knative.dev/v1","kind":"Service","metadata":{"annotations":{"kbld.k14s.io/images":"null\n"},"labels":{"app.kubernetes.io/component":"run","app.kubernetes.io/part-of":"spring-music","apps.tanzu.vmware.com/workload-type":"web","carto.run/workload-name":"spring-music","kapp.k14s.io/app":"1655580167956576323","kapp.k14s.io/association":"v1.d461947476e2f10f282a43f02102e099"},"name":"spring-music","namespace":"demo"},"spec":{"template":{"metadata":{"annotations":{"autoscaling.knative.dev/minScale":"1","boot.spring.io/actuator":"http://:8081/actuator","boot.spring.io/version":"2.6.7","conventions.apps.tanzu.vmware.com/applied-conventions":"spring-boot-convention/spring-boot\nspring-boot-convention/spring-boot-graceful-shutdown\nspring-boot-convention/spring-boot-web\nspring-boot-convention/spring-boot-actuator\nspring-boot-convention/spring-boot-actuator-probes\nspring-boot-convention/service-intent-mysql\nspring-boot-convention/service-intent-postgres\nspring-boot-convention/service-intent-mongodb\nappliveview-sample/app-live-view-connector\nappliveview-sample/app-live-view-appflavours\nappliveview-sample/app-live-view-systemproperties","developer.conventions/target-containers":"workload","services.conventions.apps.tanzu.vmware.com/mongodb":"mongodb-driver-core/4.4.2","services.conventions.apps.tanzu.vmware.com/mysql":"mysql-connector-java/8.0.28","services.conventions.apps.tanzu.vmware.com/postgres":"postgresql/42.3.4"},"labels":{"app.kubernetes.io/component":"run","app.kubernetes.io/part-of":"spring-music","apps.tanzu.vmware.com/workload-type":"web","carto.run/workload-name":"spring-music","conventions.apps.tanzu.vmware.com/framework":"spring-boot","services.conventions.apps.tanzu.vmware.com/mongodb":"workload","services.conventions.apps.tanzu.vmware.com/mysql":"workload","services.conventions.apps.tanzu.vmware.com/postgres":"workload","tanzu.app.live.view":"true","tanzu.app.live.view.application.actuator.port":"8081","tanzu.app.live.view.application.flavours":"spring-boot","tanzu.app.live.view.application.name":"spring-boot-app"}},"spec":{"containers":[{"env":[{"name":"JAVA_TOOL_OPTIONS","value":"-Dmanagement.endpoint.health.probes.add-additional-paths=\"true\"
      -Dmanagement.endpoint.health.show-details=always -Dmanagement.endpoints.web.base-path=\"/actuator\"
      -Dmanagement.endpoints.web.exposure.include=* -Dmanagement.health.probes.enabled=\"true\"
      -Dmanagement.server.port=\"8081\" -Dserver.port=\"8080\" -Dserver.shutdown.grace-period=\"24s\""}],"image":"registry.kube-system.svc.cluster.local/supplychain/spring-music-demo@sha256:100666fc6bce092f826551cb8e7b54a5bfbc110b1d6e487e1dd9871f38cfe7ec","livenessProbe":{"httpGet":{"path":"/livez","port":8080,"scheme":"HTTP"}},"name":"workload","ports":[{"containerPort":8080,"protocol":"TCP"}],"readinessProbe":{"httpGet":{"path":"/readyz","port":8080,"scheme":"HTTP"}},"resources":{},"securityContext":{"runAsUser":1000}}],"serviceAccountName":"default"}}}}'
    kapp.k14s.io/original-diff-md5: 41e2400841d3c59663d988148eb12b73
    kbld.k14s.io/images: |
      null
    serving.knative.dev/creator: system:serviceaccount:demo:default
    serving.knative.dev/lastModifier: system:serviceaccount:demo:default
  labels:
    app.kubernetes.io/component: run
    app.kubernetes.io/part-of: spring-music
    apps.tanzu.vmware.com/workload-type: web
    carto.run/workload-name: spring-music
    kapp.k14s.io/app: "1655580167956576323"
    kapp.k14s.io/association: v1.d461947476e2f10f282a43f02102e099
  name: spring-music
  namespace: demo
spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/minScale: "1"
        boot.spring.io/actuator: http://:8081/actuator
        boot.spring.io/version: 2.6.7
        conventions.apps.tanzu.vmware.com/applied-conventions: |-
          spring-boot-convention/spring-boot
          spring-boot-convention/spring-boot-graceful-shutdown
          spring-boot-convention/spring-boot-web
          spring-boot-convention/spring-boot-actuator
          spring-boot-convention/spring-boot-actuator-probes
          spring-boot-convention/service-intent-mysql
          spring-boot-convention/service-intent-postgres
          spring-boot-convention/service-intent-mongodb
          appliveview-sample/app-live-view-connector
          appliveview-sample/app-live-view-appflavours
          appliveview-sample/app-live-view-systemproperties
        developer.conventions/target-containers: workload
        services.conventions.apps.tanzu.vmware.com/mongodb: mongodb-driver-core/4.4.2
        services.conventions.apps.tanzu.vmware.com/mysql: mysql-connector-java/8.0.28
        services.conventions.apps.tanzu.vmware.com/postgres: postgresql/42.3.4
      creationTimestamp: null
      labels:
        app.kubernetes.io/component: run
        app.kubernetes.io/part-of: spring-music
        apps.tanzu.vmware.com/workload-type: web
        carto.run/workload-name: spring-music
        conventions.apps.tanzu.vmware.com/framework: spring-boot
        services.conventions.apps.tanzu.vmware.com/mongodb: workload
        services.conventions.apps.tanzu.vmware.com/mysql: workload
        services.conventions.apps.tanzu.vmware.com/postgres: workload
        tanzu.app.live.view: "true"
        tanzu.app.live.view.application.actuator.port: "8081"
        tanzu.app.live.view.application.flavours: spring-boot
        tanzu.app.live.view.application.name: spring-boot-app
    spec:
      containerConcurrency: 0
      containers:
      - env:
        - name: JAVA_TOOL_OPTIONS
          value: -Dmanagement.endpoint.health.probes.add-additional-paths="true" -Dmanagement.endpoint.health.show-details=always
            -Dmanagement.endpoints.web.base-path="/actuator" -Dmanagement.endpoints.web.exposure.include=*
            -Dmanagement.health.probes.enabled="true" -Dmanagement.server.port="8081"
            -Dserver.port="8080" -Dserver.shutdown.grace-period="24s"
        image: registry.kube-system.svc.cluster.local/supplychain/spring-music-demo@sha256:100666fc6bce092f826551cb8e7b54a5bfbc110b1d6e487e1dd9871f38cfe7ec
        livenessProbe:
          httpGet:
            path: /livez
            port: 8080
            scheme: HTTP
        name: workload
        ports:
        - containerPort: 8080
          protocol: TCP
        readinessProbe:
          httpGet:
            path: /readyz
            port: 8080
            scheme: HTTP
          successThreshold: 1
        securityContext:
          runAsUser: 1000
      enableServiceLinks: false
      serviceAccountName: default
      timeoutSeconds: 300
  traffic:
  - latestRevision: true
    percent: 100

確認が終わればWorkloadを削除します。

tanzu apps workload delete -n demo spring-music -y