---
title: Tanzu Application Platform 1.3のMulti Cluster構成をAKSにインストールするメモ
tags: ["Kubernetes", "Cartographer", "AKS", "Tanzu", "TAP", "Knative", "Azure", "Grype"]
categories: ["Dev", "CaaS", "Kubernetes", "TAP", "MultiCluster"]
date: 2022-12-16T08:13:48Z
updated: 2022-12-16T14:21:52Z
---

[Tanzu Application Platform 1.3](https://docs.vmware.com/en/Tanzu-Application-Platform/1.3/tap//GUID-overview.html) を [Multi Cluster 構成](https://docs.vmware.com/en/VMware-Tanzu-Application-Platform/1.3/tap/GUID-multicluster-about.html) でAKSにインストールします。

また、Self Signedな証明書でHTTPSを有効にします。


**目次**
<!-- toc -->

### 必要なCLI

以下のCLIは事前にインストール済みとします。

* [kubectl](https://kubernetes.io/docs/tasks/tools/)
* [az](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli)


### Pivnet CLIのインストール

ここでは [`pivnet`](https://github.com/pivotal-cf/pivnet-cli) CLIを使用して必要なソフトウェアをダウンロードします。
`pivnet` CLIはbrewでインストールできます。

```
brew install pivotal/tap/pivnet-cli
```

[VMware Tanzu Network](https://network.tanzu.vmware.com/) のAPI Tokenを取得して、`pivnet` CLIでログインします。

```
pivnet login --api-token=<API Token>
```

### EULAの承諾

初めてインストールする場合は、以下のコンポーネントのEULAをAcceptしてください。

* [Tanzu Application Platform](https://network.tanzu.vmware.com/products/tanzu-application-platform/)
* [Cluster Essentials for VMware Tanzu](https://network.tanzu.vmware.com/products/tanzu-cluster-essentials/)

> ⚠️ EULAで定められている使用期間は30日間です。とは言え、特にソフトウェア的に制限がかけられているわけではありません。


### Tanzu CLIのインストール

```
For Mac
pivnet download-product-files --product-slug='tanzu-application-platform' --release-version='1.3.3' --glob='tanzu-framework-darwin-amd64-*.tar'
# For Linux
pivnet download-product-files --product-slug='tanzu-application-platform' --release-version='1.3.3' --glob='tanzu-framework-linux-amd64-*.tar'
# For Windows
pivnet download-product-files --product-slug='tanzu-application-platform' --release-version='1.3.3' --glob='tanzu-framework-windows-amd64-*.zip'
```

```
tar xvf tanzu-framework-*-amd64-*.tar
install cli/core/v0.25.0/tanzu-core-*_amd64 /usr/local/bin/tanzu
export TANZU_CLI_NO_INIT=true
```

```
$ tanzu version
version: v0.25.0
buildDate: 2022-08-25
sha: 6288c751-dirty
```

プラグインのインストール

```
tanzu plugin install --local cli all
```


```
rm -f tanzu-framework-*-amd64-*.tar
```

### リソースグループ作成

ここではクラスタ毎にリソースグループを作成します。また、ACRは共通のリソースグループに置かれ、各クラスタからアクセスされます。

```
az group create --name tap-common --location japaneast
az group create --name tap-view --location japaneast
az group create --name tap-build --location japaneast
az group create --name tap-run --location japaneast
```

### ACRインスタンスの作成


```
ACR_NAME=tap${RANDOM}
az acr create --resource-group tap-common \
  --location japaneast \
  --name ${ACR_NAME} --sku standard
```

Read OnlyなService Principalの作成

```
ACR_REGISTRY_ID=$(az acr show --name $ACR_NAME --query id --output tsv)
SERVICE_PRINCIPAL_RO_NAME=tap-ro
SERVICE_PRINCIPAL_RO_PASSWORD=$(az ad sp create-for-rbac --name $SERVICE_PRINCIPAL_RO_NAME --scopes $ACR_REGISTRY_ID --years 100 --role acrpull --query password --output tsv)
SERVICE_PRINCIPAL_RO_USERNAME=$(az ad sp list --display-name $SERVICE_PRINCIPAL_RO_NAME --query "[].appId" --output tsv)
```

Read Only Service Principalでログインを確認

```
docker login ${ACR_NAME}.azurecr.io -u ${SERVICE_PRINCIPAL_RO_USERNAME} -p ${SERVICE_PRINCIPAL_RO_PASSWORD}
```


Read WriteなService Principalの作成

```
ACR_REGISTRY_ID=$(az acr show --name $ACR_NAME --query id --output tsv)
SERVICE_PRINCIPAL_RW_NAME=tap-rw
SERVICE_PRINCIPAL_RW_PASSWORD=$(az ad sp create-for-rbac --name $SERVICE_PRINCIPAL_RW_NAME --scopes $ACR_REGISTRY_ID --years 100 --role acrpush --query password --output tsv)
SERVICE_PRINCIPAL_RW_USERNAME=$(az ad sp list --display-name $SERVICE_PRINCIPAL_RW_NAME --query "[].appId" --output tsv)
```

Read Only Service Principalでログインを確認

```
docker login ${ACR_NAME}.azurecr.io -u ${SERVICE_PRINCIPAL_RO_USERNAME} -p ${SERVICE_PRINCIPAL_RO_PASSWORD}
```

Read Write Service Principalでログインを確認

```
docker login ${ACR_NAME}.azurecr.io -u ${SERVICE_PRINCIPAL_RW_USERNAME} -p ${SERVICE_PRINCIPAL_RW_PASSWORD}
```


環境変数をスクリプトに保存します。

```
cat <<EOF > env.sh
export ACR_NAME=${ACR_NAME}
export SERVICE_PRINCIPAL_RO_NAME=${SERVICE_PRINCIPAL_RO_NAME}
export SERVICE_PRINCIPAL_RO_USERNAME=${SERVICE_PRINCIPAL_RO_USERNAME}
export SERVICE_PRINCIPAL_RO_PASSWORD=${SERVICE_PRINCIPAL_RO_PASSWORD}
export SERVICE_PRINCIPAL_RW_NAME=${SERVICE_PRINCIPAL_RW_NAME}
export SERVICE_PRINCIPAL_RW_USERNAME=${SERVICE_PRINCIPAL_RW_USERNAME}
export SERVICE_PRINCIPAL_RW_PASSWORD=${SERVICE_PRINCIPAL_RW_PASSWORD}
EOF
```

### イメージのRelocation

Tanzu Networkのアカウントでregistry.tanzu.vmware.comにログイン

```
TANZUNET_USERNAME=...
TANZUNET_PASSWORD=...

docker login registry.tanzu.vmware.com -u ${TANZUNET_USERNAME} -p ${TANZUNET_PASSWORD}
```

#### Cluster EssentialsのReloaction

```
imgpkg copy \
  -b registry.tanzu.vmware.com/tanzu-cluster-essentials/cluster-essentials-bundle:1.3.0 \
  --to-repo ${ACR_NAME}.azurecr.io/tanzu-cluster-essentials/cluster-essentials-bundle \
  --include-non-distributable-layers
```

#### TAPのRelocation

サイズが大きい(8GB以上)なので、少し時間がかかります

```
imgpkg copy \
  -b registry.tanzu.vmware.com/tanzu-application-platform/tap-packages:1.3.3 \
  --to-repo ${ACR_NAME}.azurecr.io/tanzu-application-platform/tap-packages \
  --include-non-distributable-layers
```

### TBS full dependenciesのRelocation

サイズが大きい(9GB以上)なので、少し時間がかかります

```
imgpkg copy \
  -b registry.tanzu.vmware.com/tanzu-application-platform/full-tbs-deps-package-repo:1.7.4 \
  --to-repo ${ACR_NAME}.azurecr.io/tanzu-application-platform/full-tbs-deps-package-repo \
  --include-non-distributable-layers
```


### AKSクラスタの作成

AKSクラスタにはstandard_f4s_v2 (4 vCPU, 8GB Memory)のWorker Nodeを使用し、cluster-autoscalerを有効にしておきます。またAzure ADとの連携を有効にします。

#### Viewクラスタ用

```
az aks create \
  --resource-group tap-view \
  --location japaneast \
  --name tap-view \
  --node-count 1 \
  --enable-cluster-autoscaler \
  --min-count 1 \
  --max-count 10 \
  --node-vm-size standard_f4s_v2 \
  --load-balancer-sku standard \
  --kubernetes-version 1.24 \
  --generate-ssh-keys \
  --enable-aad
```

後にContourのEnvoyへのIPを静的に設定するための、権限をAKSクラスタに与ます。以下のドキュメントを参考にしました。<br>
https://docs.microsoft.com/en-us/azure/aks/static-ip#create-a-service-using-the-static-ip-address


```
RG_ID=$(az group show --name tap-view -o tsv --query id )
SP_APP_ID=$(az aks show --name tap-view --resource-group tap-view --query "identity.principalId" -o tsv)
az role assignment create --assignee-object-id ${SP_APP_ID} --assignee-principal-type "ServicePrincipal" --role "Network Contributor" --scope ${RG_ID}
```


#### Buildクラスタの作成

```
az aks create \
  --resource-group tap-build \
  --location japaneast \
  --name tap-build \
  --node-count 1 \
  --enable-cluster-autoscaler \
  --min-count 1 \
  --max-count 10 \
  --node-vm-size standard_f4s_v2 \
  --load-balancer-sku standard \
  --kubernetes-version 1.24 \
  --generate-ssh-keys \
  --enable-aad
```

#### Runクラスタ用

```
az aks create \
  --resource-group tap-run \
  --location japaneast \
  --name tap-run \
  --node-count 1 \
  --enable-cluster-autoscaler \
  --min-count 1 \
  --max-count 10 \
  --node-vm-size standard_f4s_v2 \
  --load-balancer-sku standard \
  --kubernetes-version 1.24 \
  --generate-ssh-keys \
  --enable-aad
```

後にContourのEnvoyへのIPを静的に設定するための、権限をAKSクラスタに与ます。以下のドキュメントを参考にしました。<br>
https://docs.microsoft.com/en-us/azure/aks/static-ip#create-a-service-using-the-static-ip-address


```
RG_ID=$(az group show --name tap-run -o tsv --query id )
SP_APP_ID=$(az aks show --name tap-run --resource-group tap-run --query "identity.principalId" -o tsv)
az role assignment create --assignee-object-id ${SP_APP_ID} --assignee-principal-type "ServicePrincipal" --role "Network Contributor" --scope ${RG_ID}
```

クラスタを確認します。

```
$ az aks list --output table
Name       Location    ResourceGroup    KubernetesVersion    CurrentKubernetesVersion    ProvisioningState    Fqdn
---------  ----------  ---------------  -------------------  --------------------------  -------------------  -----------------------------------------------------------
tap-build  japaneast   tap-build        1.24                 1.24.6                      Succeeded            tap-build-tap-build-85cd83-7ed8ba35.hcp.japaneast.azmk8s.io
tap-run    japaneast   tap-run          1.24                 1.24.6                      Succeeded            tap-run-tap-run-85cd83-bc4d3bda.hcp.japaneast.azmk8s.io
tap-view   japaneast   tap-view         1.24                 1.24.6                      Succeeded            tap-view-tap-view-85cd83-27be84e1.hcp.japaneast.azmk8s.io
```

### Cluster Essentials for VMware Tanzuのインストール

TAPのインストールに必要なKapp ControllerとSecretgen Controllerをデプロイするために [Cluster Essentials for VMware Tanzu](https://network.tanzu.vmware.com/products/tanzu-cluster-essentials/#/releases/1130414) をインストールします。

```
# Mac
pivnet download-product-files --product-slug='tanzu-cluster-essentials' --release-version='1.3.0' --glob='tanzu-cluster-essentials-darwin-amd64-*'
# Linux
pivnet download-product-files --product-slug='tanzu-cluster-essentials' --release-version='1.3.0' --glob='tanzu-cluster-essentials-linux-amd64-*'
```

```yaml
mkdir tanzu-cluster-essentials
tar xzvf tanzu-cluster-essentials-*-amd64-*.tgz -C tanzu-cluster-essentials

export INSTALL_BUNDLE=${ACR_NAME}.azurecr.io/tanzu-cluster-essentials/cluster-essentials-bundle:1.3.0
export INSTALL_REGISTRY_HOSTNAME=${ACR_NAME}.azurecr.io
export INSTALL_REGISTRY_USERNAME=${SERVICE_PRINCIPAL_RO_USERNAME}
export INSTALL_REGISTRY_PASSWORD=${SERVICE_PRINCIPAL_RO_PASSWORD}
cd tanzu-cluster-essentials


# For View Cluser
az aks get-credentials --resource-group tap-view --name tap-view --admin --overwrite-existing
./install.sh --yes

# For Build Cluster
az aks get-credentials --resource-group tap-build --name tap-build --admin --overwrite-existing
./install.sh --yes

# For Run Cluster
az aks get-credentials --resource-group tap-run --name tap-run --admin --overwrite-existing
./install.sh --yes

cd ..

rm -f tanzu-cluster-essentials-*-amd64-*.tgz
```

### CA(自己署名)証明書の生成

```
mkdir -p certs
rm -f certs/*
docker run --rm -v ${PWD}/certs:/certs hitch openssl req -new -nodes -out /certs/ca.csr -keyout /certs/ca.key -subj "/CN=default-ca/O=TAP/C=JP"
chmod og-rwx ca.key
docker run --rm -v ${PWD}/certs:/certs hitch openssl x509 -req -in /certs/ca.csr -days 3650 -extfile /etc/ssl/openssl.cnf -extensions v3_ca -signkey /certs/ca.key -out /certs/ca.crt
```

### TAP GUI用のService AccountをBuildとRun Clusterに作成

https://docs.vmware.com/en/VMware-Tanzu-Application-Platform/1.3/tap/GUID-tap-gui-cluster-view-setup.html


```yaml
mkdir -p tap-gui
cat <<EOF > tap-gui/tap-gui-viewer-service-account-rbac.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: tap-gui
---
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: tap-gui
  name: tap-gui-viewer
automountServiceAccountToken: false
---
apiVersion: v1
kind: Secret
metadata:
  name: tap-gui-viewer
  namespace: tap-gui
  annotations:
    kubernetes.io/service-account.name: tap-gui-viewer
type: kubernetes.io/service-account-token
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: tap-gui-read-k8s
subjects:
- kind: ServiceAccount
  namespace: tap-gui
  name: tap-gui-viewer
roleRef:
  kind: ClusterRole
  name: k8s-reader
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: k8s-reader
rules:
- apiGroups: ['']
  resources: ['pods', 'pods/log', 'services', 'configmaps']
  verbs: ['get', 'watch', 'list']
- apiGroups: ['apps']
  resources: ['deployments', 'replicasets']
  verbs: ['get', 'watch', 'list']
- apiGroups: ['autoscaling']
  resources: ['horizontalpodautoscalers']
  verbs: ['get', 'watch', 'list']
- apiGroups: ['networking.k8s.io']
  resources: ['ingresses']
  verbs: ['get', 'watch', 'list']
- apiGroups: ['networking.internal.knative.dev']
  resources: ['serverlessservices']
  verbs: ['get', 'watch', 'list']
- apiGroups: [ 'autoscaling.internal.knative.dev' ]
  resources: [ 'podautoscalers' ]
  verbs: [ 'get', 'watch', 'list' ]
- apiGroups: ['serving.knative.dev']
  resources:
  - configurations
  - revisions
  - routes
  - services
  verbs: ['get', 'watch', 'list']
- apiGroups: ['carto.run']
  resources:
  - clusterconfigtemplates
  - clusterdeliveries
  - clusterdeploymenttemplates
  - clusterimagetemplates
  - clusterruntemplates
  - clustersourcetemplates
  - clustersupplychains
  - clustertemplates
  - deliverables
  - runnables
  - workloads
  verbs: ['get', 'watch', 'list']
- apiGroups: ['source.toolkit.fluxcd.io']
  resources:
  - gitrepositories
  verbs: ['get', 'watch', 'list']
- apiGroups: ['source.apps.tanzu.vmware.com']
  resources:
  - imagerepositories
  - mavenartifacts
  verbs: ['get', 'watch', 'list']
- apiGroups: ['conventions.carto.run']
  resources:
  - podintents
  verbs: ['get', 'watch', 'list']
- apiGroups: ['kpack.io']
  resources:
  - images
  - builds
  verbs: ['get', 'watch', 'list']
- apiGroups: ['scanning.apps.tanzu.vmware.com']
  resources:
  - sourcescans
  - imagescans
  - scanpolicies
  verbs: ['get', 'watch', 'list']
- apiGroups: ['tekton.dev']
  resources:
  - taskruns
  - pipelineruns
  verbs: ['get', 'watch', 'list']
- apiGroups: ['kappctrl.k14s.io']
  resources:
  - apps
  verbs: ['get', 'watch', 'list']
EOF
```

```
kubectl apply -f tap-gui/tap-gui-viewer-service-account-rbac.yaml --context tap-build-admin
kubectl apply -f tap-gui/tap-gui-viewer-service-account-rbac.yaml --context tap-run-admin
```


```
kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}' --context tap-build-admin > tap-gui/cluster-url-build
kubectl -n tap-gui get secret tap-gui-viewer --context tap-build-admin -otemplate='{{index .data "token" | base64decode}}' > tap-gui/cluster-token-build
kubectl -n tap-gui get secret tap-gui-viewer --context tap-build-admin -otemplate='{{index .data "ca.crt"}}'  > tap-gui/cluster-ca-build

kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}' --context tap-run-admin > tap-gui/cluster-url-run
kubectl -n tap-gui get secret tap-gui-viewer --context tap-run-admin -otemplate='{{index .data "token" | base64decode}}' > tap-gui/cluster-token-run
kubectl -n tap-gui get secret tap-gui-viewer --context tap-run-admin -otemplate='{{index .data "ca.crt"}}'  > tap-gui/cluster-ca-run
```

### Tanzu Application Platform (View Cluster) のインストール

コンテキストを変えます

```
kubectl config use-context tap-view-admin
```

#### TAP用Package Repositoryの登録

```
kubectl create ns tap-install

tanzu secret registry add tap-registry \
  --username "${SERVICE_PRINCIPAL_RO_USERNAME}" \
  --password "${SERVICE_PRINCIPAL_RO_PASSWORD}" \
  --server ${ACR_NAME}.azurecr.io \
  --export-to-all-namespaces \
  --yes \
  --namespace tap-install

tanzu package repository add tanzu-tap-repository \
  --url ${ACR_NAME}.azurecr.io/tanzu-application-platform/tap-packages:1.3.3 \
  --namespace tap-install
```

#### Envoy用のPublic IPを作成

```
az network public-ip create --resource-group tap-view --location japaneast --name envoy-ip --sku Standard --allocation-method static
```

```
ENVOY_IP_VIEW=$(az network public-ip show --resource-group tap-view --name envoy-ip --query ipAddress --output tsv)
```

Public IPを確認します。

```
$ az network public-ip list -o table
Name                                  ResourceGroup                     Location    Zones    Address        IdleTimeoutInMinutes    ProvisioningState
------------------------------------  --------------------------------  ----------  -------  -------------  ----------------------  -------------------
046fb436-2a94-4258-98c7-936672972a31  MC_tap-build_tap-build_japaneast  japaneast   123      4.241.144.189  30                      Succeeded
48543263-ed47-41cd-abc3-cb23671407d6  MC_tap-run_tap-run_japaneast      japaneast   123      20.27.128.119  30                      Succeeded
fd09c63c-f277-498e-8338-4cdfda21c7ba  MC_tap-view_tap-view_japaneast    japaneast   123      52.155.119.68  30                      Succeeded
envoy-ip                              tap-view                          japaneast            20.89.90.83    4                       Succeeded
```

#### Overlay filesの作成

インストール後の作業を減らすためにoverlayファイルを作成します。

```yaml
DOMAIN_NAME_VIEW=view.$(echo ${ENVOY_IP_VIEW} | sed 's/\./-/g').sslip.io

mkdir -p overlays/view


cat <<EOF > overlays/view/contour-default-tls.yaml                                                                                                                                                                                                                          
#@ load("@ytt:data", "data")
#@ load("@ytt:overlay", "overlay")
#@ namespace = data.values.namespace
---
apiVersion: v1
kind: Secret
metadata:
  name: default-ca
  namespace: #@ namespace
type: kubernetes.io/tls
stringData:
  tls.crt: |
$(cat certs/ca.crt | sed 's/^/    /g')
  tls.key: |
$(cat certs/ca.key | sed 's/^/    /g')
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: default-ca-issuer
  namespace: #@ namespace
spec:
  ca:
    secretName: default-ca
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: tap-default-tls
  namespace: #@ namespace
spec:
  dnsNames:
  - #@ "*.${DOMAIN_NAME_VIEW}"
  issuerRef:
    kind: Issuer
    name: default-ca-issuer
  secretName: tap-default-tls
---
apiVersion: projectcontour.io/v1
kind: TLSCertificateDelegation
metadata:
  name: contour-delegation
  namespace: #@ namespace
spec:
  delegations:
  - secretName: tap-default-tls
    targetNamespaces:
    - "*"
EOF


cat <<'EOF' > overlays/view/tap-gui-db.yaml
#@ load("@ytt:overlay", "overlay")
#@overlay/match by=overlay.subset({"kind":"Deployment","metadata":{"name":"server"}})
---
spec:
  #@overlay/match missing_ok=True
  template:
    spec:
      containers:
      #@overlay/match by="name"
      - name: backstage
        #@overlay/match missing_ok=True
        envFrom:
         - secretRef:
             name: tap-gui-db
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: tap-gui-db
  namespace: tap-gui
  labels:
    app.kubernetes.io/part-of: tap-gui-db
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tap-gui-db
  namespace: tap-gui
  labels:
    app.kubernetes.io/part-of: tap-gui-db
spec:
  selector:
    matchLabels:
      app.kubernetes.io/part-of: tap-gui-db
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app.kubernetes.io/part-of: tap-gui-db
    spec:
      initContainers:
      - name: remove-lost-found
        image: busybox
        command:
        - sh
        - -c
        - |
          rm -fr /var/lib/postgresql/data/lost+found
        volumeMounts:
        - name: tap-gui-db
          mountPath: /var/lib/postgresql/data
      containers:
      - image: postgres:14-alpine
        name: postgres
        envFrom:
        - secretRef:
            name: tap-gui-db
        ports:
        - containerPort: 5432
          name: tap-gui-db
        volumeMounts:
        - name: tap-gui-db
          mountPath: /var/lib/postgresql/data
      volumes:
      - name: tap-gui-db
        persistentVolumeClaim:
          claimName: tap-gui-db
---
apiVersion: v1
kind: Service
metadata:
  name: tap-gui-db
  namespace: tap-gui
  labels:
    app.kubernetes.io/part-of: tap-gui-db
spec:
  ports:
  - port: 5432
  selector:
    app.kubernetes.io/part-of: tap-gui-db
---
apiVersion: secretgen.k14s.io/v1alpha1
kind: Password
metadata:
  name: tap-gui-db
  namespace: tap-gui
  labels:
    app.kubernetes.io/part-of: tap-gui-db
spec:
  secretTemplate:
    type: Opaque
    stringData:
      POSTGRES_USER: tap-gui
      POSTGRES_PASSWORD: $(value)
EOF

cat <<EOF > overlays/view/metadata-store-read-only-client.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: metadata-store-ready-only
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: metadata-store-read-only
subjects:
- kind: ServiceAccount
  name: metadata-store-read-client
  namespace: metadata-store
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: metadata-store-read-client
  namespace: metadata-store
automountServiceAccountToken: false
---
apiVersion: v1
kind: Secret
metadata:
  name: metadata-store-read-client
  namespace: metadata-store
  annotations:
    kubernetes.io/service-account.name: metadata-store-read-client
type: kubernetes.io/service-account-token
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: metadata-store-read-client-secret-read
  namespace: metadata-store
rules:
- apiGroups: [ "" ]
  resources: [ "secrets" ]
  resourceNames: [ "metadata-store-read-client" ]
  verbs: [ "get" ]
EOF
```

各々のoverlayファイルをSecretとして作成します。

```
kubectl -n tap-install create secret generic contour-default-tls \
  -o yaml \
  --dry-run=client \
  --from-file=overlays/view/contour-default-tls.yaml \
  | kubectl apply -f-

kubectl -n tap-install create secret generic tap-gui-db \
  -o yaml \
  --dry-run=client \
  --from-file=overlays/view/tap-gui-db.yaml \
  | kubectl apply -f-

kubectl -n tap-install create secret generic metadata-store-read-only-client \
  -o yaml \
  --dry-run=client \
  --from-file=overlays/view/metadata-store-read-only-client.yaml \
  | kubectl apply -f-
```

#### View Profileのインストール

tap-values.yamlを作成します。CHANGEMEの部分は後にアップデートします。

```yaml
cat <<EOF > tap-values-view.yaml
profile: view

ceip_policy_disclosed: true

shared:
  ingress_domain: ${DOMAIN_NAME_VIEW}
  ca_cert_data: |
$(cat certs/ca.crt | sed 's/^/    /g')

contour:
  infrastructure_provider: azure
  contour:
    configFileContents:
      accesslog-format: json  
  envoy:
    service:
      type: LoadBalancer
      loadBalancerIP: ${ENVOY_IP_VIEW}      
      annotations:
        service.beta.kubernetes.io/azure-load-balancer-resource-group: tap-view

tap_gui:
  service_type: ClusterIP
  tls:
    secretName: tap-default-tls
    namespace: tanzu-system-ingress
  app_config:
    backend:
      database:
        client: pg
        connection:
          host: \${TAP_GUI_DB_SERVICE_HOST}
          port: \${TAP_GUI_DB_SERVICE_PORT}
          user: \${POSTGRES_USER}
          password: \${POSTGRES_PASSWORD}
    kubernetes:
      serviceLocatorMethod:
        type: multiTenant
      clusterLocatorMethods:
      - type: config
        clusters:
        - url: $(cat tap-gui/cluster-url-run)
          name: run
          authProvider: serviceAccount
          serviceAccountToken: $(cat tap-gui/cluster-token-run)
          skipTLSVerify: false
          caData: $(cat tap-gui/cluster-ca-run)
      - type: config
        clusters:
        - url: $(cat tap-gui/cluster-url-build)
          name: build
          authProvider: serviceAccount
          serviceAccountToken: $(cat tap-gui/cluster-token-build)
          skipTLSVerify: false
          caData: $(cat tap-gui/cluster-ca-build)
    proxy:
      /metadata-store:
        target: https://metadata-store-app.metadata-store:8443/api/v1
        changeOrigin: true
        secure: false
        headers:
          Authorization: "Bearer CHANGEME"
          X-Custom-Source: project-star
appliveview:
  ingressEnabled: true
  tls:
    secretName: tap-default-tls
    namespace: tanzu-system-ingress

accelerator:
  ingress:
    include: true    
    enable_tls: true  
  tls:
    secret_name: tap-default-tls
    namespace: tanzu-system-ingress

metadata_store:
  ns_for_export_app_cert: "*"

package_overlays:
- name: contour
  secrets:
  - name: contour-default-tls
- name: tap-gui
  secrets:
  - name: tap-gui-db
- name: metadata-store
  secrets:
  - name: metadata-store-read-only-client

excluded_packages:
- learningcenter.tanzu.vmware.com
- workshops.learningcenter.tanzu.vmware.com
- api-portal.tanzu.vmware.com
EOF
```

TAPをインストールします。

```
tanzu package install tap \
  -p tap.tanzu.vmware.com \
  -v 1.3.3 \
  --values-file tap-values-view.yaml \
  -n tap-install \
  --wait=false
```

インストールが成功するまで待ちます。5分程度かかります。

```
while [ "$(kubectl -n tap-install get app tap -o=jsonpath='{.status.friendlyDescription}')" != "Reconcile succeeded" ];do
  date
  kubectl get app -n tap-install
  echo "---------------------------------------------------------------------"
  sleep 30
done
echo "✅ Install succeeded"
```

packageinstallを確認します。

```
$ kubectl get packageinstall -n tap-install 
NAME                       PACKAGE NAME                                PACKAGE VERSION   DESCRIPTION           AGE
accelerator                accelerator.apps.tanzu.vmware.com           1.3.2             Reconcile succeeded   3m19s
appliveview                backend.appliveview.tanzu.vmware.com        1.3.1             Reconcile succeeded   3m19s
cert-manager               cert-manager.tanzu.vmware.com               1.7.2+tap.1       Reconcile succeeded   5m19s
contour                    contour.tanzu.vmware.com                    1.22.0+tap.5      Reconcile succeeded   5m5s
fluxcd-source-controller   fluxcd.source.controller.tanzu.vmware.com   0.27.0+tap.1      Reconcile succeeded   5m19s
metadata-store             metadata-store.apps.tanzu.vmware.com        1.3.4             Reconcile succeeded   3m19s
source-controller          controller.source.apps.tanzu.vmware.com     0.5.1             Reconcile succeeded   5m5s
tap                        tap.tanzu.vmware.com                        1.3.3             Reconcile succeeded   5m20s
tap-gui                    tap-gui.tanzu.vmware.com                    1.3.4             Reconcile succeeded   3m19s
tap-telemetry              tap-telemetry.tanzu.vmware.com              0.3.2             Reconcile succeeded   5m19s
```

生成されたMetadata Storeへのアクセストークンを取得して、tap-values.yamlに設定し、packageinstallを更新します。

```
sed -i.bak "s/CHANGEME/$(kubectl get secret -n metadata-store metadata-store-read-client -otemplate='{{.data.token | base64decode}}')/" tap-values-view.yaml
tanzu package installed update -n tap-install tap -f tap-values-view.yaml
```

HTTProxyを取得してアクセス可能なURLを確認します。

```
$ kubectl get httpproxy -A
NAMESPACE            NAME                     FQDN                                       TLS SECRET                             STATUS   STATUS DESCRIPTION
accelerator-system   accelerator              accelerator.view.20-89-90-83.sslip.io      tanzu-system-ingress/tap-default-tls   valid    Valid HTTPProxy
app-live-view        appliveview              appliveview.view.20-89-90-83.sslip.io      tanzu-system-ingress/tap-default-tls   valid    Valid HTTPProxy
metadata-store       metadata-store-ingress   metadata-store.view.20-89-90-83.sslip.io   ingress-cert                           valid    Valid HTTPProxy
tap-gui              tap-gui                  tap-gui.view.20-89-90-83.sslip.io          tanzu-system-ingress/tap-default-tls   valid    Valid HTTPProx
```

TAP GUIにアクセスします。

<img width="1024" alt="image" src="https://user-images.githubusercontent.com/106908/207788758-326a0e7f-6d25-4a0a-9183-61e7d70dfa48.png">

環境変数をenv.shに残します。

```
cat <<EOF >> env.sh
export ENVOY_IP_VIEW=${ENVOY_IP_VIEW}
export DOMAIN_NAME_VIEW=${DOMAIN_NAME_VIEW}
EOF
```

### Tanzu Application Platform (Build Cluster) のインストール


コンテキストを変えます。

```
kubectl config use-context tap-build-admin
```

#### TAP用Package Repositoryの登録

```
kubectl create ns tap-install

tanzu secret registry add tap-registry \
  --username "${SERVICE_PRINCIPAL_RO_USERNAME}" \
  --password "${SERVICE_PRINCIPAL_RO_PASSWORD}" \
  --server ${ACR_NAME}.azurecr.io \
  --export-to-all-namespaces \
  --yes \
  --namespace tap-install

tanzu package repository add tanzu-tap-repository \
  --url ${ACR_NAME}.azurecr.io/tanzu-application-platform/tap-packages:1.3.3 \
  --namespace tap-install

tanzu package repository add tbs-full-deps-repository \
  --url ${ACR_NAME}.azurecr.io/tanzu-application-platform/full-tbs-deps-package-repo:1.7.4 \
  --namespace tap-install
```

#### kpack用のdefault repository secretの作成


```yaml
cat <<'EOF' > create-repository-secret.sh
#!/bin/bash
REGISTRY_SERVER=$1
REGUSTRY_USERNAME=$2
REGUSTRY_PASSWORD=$3
cat <<SCRIPT | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: repository-secret
  namespace: tap-install
type: kubernetes.io/dockerconfigjson
stringData:
  .dockerconfigjson: |
    {
      "auths": {
        "${REGISTRY_SERVER}": {
          "username": "${REGUSTRY_USERNAME}",
          "password": "${REGUSTRY_PASSWORD}"
        }
      }
    }
SCRIPT
EOF
chmod +x create-repository-secret.sh
```

Read WriteのService PrincipalでSecretを作成

```
./create-repository-secret.sh ${ACR_NAME}.azurecr.io ${SERVICE_PRINCIPAL_RW_USERNAME} ${SERVICE_PRINCIPAL_RW_PASSWORD}
```

#### Overlay filesの作成

インストール後の作業を減らすためにoverlayファイルを作成します。

```yaml
mkdir -p overlays/build

cat <<EOF > overlays/build/metadata-store-secrets.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: metadata-store-secrets
---
apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: store-ca-cert
  namespace: metadata-store-secrets
stringData:
  ca.crt: |
$(kubectl get secret -n metadata-store ingress-cert -otemplate='{{index .data "ca.crt" | base64decode}}' --context tap-view-admin | sed 's/^/    /g')
---
apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: store-auth-token
  namespace: metadata-store-secrets
stringData:
  auth_token: $(kubectl get secret -n metadata-store metadata-store-read-write-client -otemplate='{{.data.token | base64decode}}' --context tap-view-admin)
---
apiVersion: secretgen.carvel.dev/v1alpha1
kind: SecretExport
metadata:
  name: store-ca-cert
  namespace: metadata-store-secrets
spec:
  toNamespace: "*"
---
apiVersion: secretgen.carvel.dev/v1alpha1
kind: SecretExport
metadata:
  name: store-auth-token
  namespace: metadata-store-secrets
spec:
  toNamespace: "*"
EOF
```

overlayファイルをsecretとして登録します。

```
kubectl -n tap-install create secret generic metadata-store-secrets \
  -o yaml \
  --dry-run=client \
  --from-file=overlays/build/metadata-store-secrets.yaml \
  | kubectl apply -f-
```

#### Build Profileのインストール

tap-values.yamlを作成します。Grypeパッケージは後ほどnamespace単位でインストールするのでここでは除外します。

```yaml
cat <<EOF > tap-values-build.yaml
profile: build

shared:
  image_registry:
    project_path: "${ACR_NAME}.azurecr.io/tanzu-application-platform"
    secret:
      name: repository-secret
      namespace: tap-install
  ca_cert_data: |
$(cat certs/ca.crt | sed 's/^/    /g')

ceip_policy_disclosed: true

buildservice: 
  exclude_dependencies: true

supply_chain: testing_scanning

scanning:
  metadataStore:
    url: ""

package_overlays:
- name: ootb-supply-chain-testing-scanning
  secrets:
  - name: metadata-store-secrets

excluded_packages:
- grype.scanning.apps.tanzu.vmware.com
- contour.tanzu.vmware.com
EOF
```

TAPとTBSのfull dependenciesをインストールします。


```
tanzu package install tap \
  -p tap.tanzu.vmware.com \
  -v 1.3.3 \
  --values-file tap-values-build.yaml \
  -n tap-install \
  --wait=false

tanzu package install full-tbs-deps \
  -p full-tbs-deps.tanzu.vmware.com \
  -v 1.7.4 \
  -n tap-install \
  --wait=false
```

インストールが成功するまで待ちます。5分程度かかります。

```
while [ "$(kubectl -n tap-install get app tap -o=jsonpath='{.status.friendlyDescription}')" != "Reconcile succeeded" ];do
  date
  kubectl get app -n tap-install
  echo "---------------------------------------------------------------------"
  sleep 30
done
while [ "$(kubectl -n tap-install get app full-tbs-deps -o=jsonpath='{.status.friendlyDescription}')" != "Reconcile succeeded" ];do
  date
  kubectl get app -n tap-install
  echo "---------------------------------------------------------------------"
  sleep 30
done
echo "✅ Install succeeded"
```

packageinstallを確認します。

```
$ tanzu package installed list -n tap-install                              

  NAME                                PACKAGE-NAME                                         PACKAGE-VERSION  STATUS               
  appliveview-conventions             conventions.appliveview.tanzu.vmware.com             1.3.1            Reconcile succeeded  
  buildservice                        buildservice.tanzu.vmware.com                        1.7.4            Reconcile succeeded  
  cartographer                        cartographer.tanzu.vmware.com                        0.5.4            Reconcile succeeded  
  cert-manager                        cert-manager.tanzu.vmware.com                        1.7.2+tap.1      Reconcile succeeded  
  conventions-controller              controller.conventions.apps.tanzu.vmware.com         0.7.1            Reconcile succeeded  
  fluxcd-source-controller            fluxcd.source.controller.tanzu.vmware.com            0.27.0+tap.1     Reconcile succeeded  
  full-tbs-deps                       full-tbs-deps.tanzu.vmware.com                       1.7.4            Reconcile succeeded  
  ootb-supply-chain-testing-scanning  ootb-supply-chain-testing-scanning.tanzu.vmware.com  0.10.5           Reconcile succeeded  
  ootb-templates                      ootb-templates.tanzu.vmware.com                      0.10.5           Reconcile succeeded  
  scanning                            scanning.apps.tanzu.vmware.com                       1.3.1            Reconcile succeeded  
  source-controller                   controller.source.apps.tanzu.vmware.com              0.5.1            Reconcile succeeded  
  spring-boot-conventions             spring-boot-conventions.tanzu.vmware.com             0.5.0            Reconcile succeeded  
  tap                                 tap.tanzu.vmware.com                                 1.3.3            Reconcile succeeded  
  tap-auth                            tap-auth.tanzu.vmware.com                            1.1.0            Reconcile succeeded  
  tap-telemetry                       tap-telemetry.tanzu.vmware.com                       0.3.2            Reconcile succeeded  
  tekton-pipelines                    tekton.tanzu.vmware.com                              0.39.0+tap.2     Reconcile succeeded 
```

ClusterBuilderがREADYであることを確認してください。

```
$ kubectl get clusterbuilder
NAME         LATESTIMAGE                                                                                                                                                     READY
base         tap16979.azurecr.io/tanzu-application-platform/buildservice:clusterbuilder-base@sha256:2f44fc24b1c278dfc8feaa8f19cc33330234a05a414341d9022b468fbd0d6fdb         True
base-jammy   tap16979.azurecr.io/tanzu-application-platform/buildservice:clusterbuilder-base-jammy@sha256:947457234ce80b7aac02e65fdaf2bf28bba39d5ee8fab098eb711f5586d6d633   True
default      tap16979.azurecr.io/tanzu-application-platform/buildservice:clusterbuilder-default@sha256:2f44fc24b1c278dfc8feaa8f19cc33330234a05a414341d9022b468fbd0d6fdb      True
full         tap16979.azurecr.io/tanzu-application-platform/buildservice:clusterbuilder-full@sha256:0f6556997d6d4237197b996f09175f583ea92485637cae7d95655cbfc5a27c8f         True
full-jammy   tap16979.azurecr.io/tanzu-application-platform/buildservice:clusterbuilder-full-jammy@sha256:8926b70651b1066df57e2b22f0f52756e228b3168d28519bdfff3147820274c8   True
tiny         tap16979.azurecr.io/tanzu-application-platform/buildservice:clusterbuilder-tiny@sha256:48294b40310c04e8f3016b969b8a2c725a133ea78e41a08ea5c6a857eb5a8d64         True
tiny-jammy   tap16979.azurecr.io/tanzu-application-platform/buildservice:clusterbuilder-tiny-jammy@sha256:52e8e04310a17fe68e99c5e282a035ab100bf7368f2a2d494912e88bd5696ba8   True
```

### Tanzu Application Platform (Run Cluster) のインストール

Contextを変えます。

```
kubectl config use-context tap-run-admin
```

#### TAP用Package Repositoryの登録

```
kubectl create ns tap-install

tanzu secret registry add tap-registry \
  --username "${SERVICE_PRINCIPAL_RO_USERNAME}" \
  --password "${SERVICE_PRINCIPAL_RO_PASSWORD}" \
  --server ${ACR_NAME}.azurecr.io \
  --export-to-all-namespaces \
  --yes \
  --namespace tap-install

tanzu package repository add tanzu-tap-repository \
  --url ${ACR_NAME}.azurecr.io/tanzu-application-platform/tap-packages:1.3.3 \
  --namespace tap-install
```

#### Envoy用のPublic IPを作成

```
az network public-ip create --resource-group tap-run --location japaneast --name envoy-ip --sku Standard --allocation-method static
```

```
ENVOY_IP_RUN=$(az network public-ip show --resource-group tap-run --name envoy-ip --query ipAddress --output tsv)
```

Public IPを確認します。

```
$ az network public-ip list -o table
Name                                  ResourceGroup                     Location    Zones    Address        IdleTimeoutInMinutes    ProvisioningState
------------------------------------  --------------------------------  ----------  -------  -------------  ----------------------  -------------------
046fb436-2a94-4258-98c7-936672972a31  MC_tap-build_tap-build_japaneast  japaneast   123      4.241.144.189  30                      Succeeded
48543263-ed47-41cd-abc3-cb23671407d6  MC_tap-run_tap-run_japaneast      japaneast   123      20.27.128.119  30                      Succeeded
fd09c63c-f277-498e-8338-4cdfda21c7ba  MC_tap-view_tap-view_japaneast    japaneast   123      52.155.119.68  30                      Succeeded
envoy-ip                              tap-run                           japaneast            20.18.106.167  4                       Succeeded
envoy-ip                              tap-view                          japaneast            20.89.90.83    4                       Succeeded
```

#### Overlay filesの作成

インストール後の作業を減らすために、overlayファイルを作成します。

```yaml
DOMAIN_NAME_RUN=run.$(echo ${ENVOY_IP_RUN} | sed 's/\./-/g').sslip.io

mkdir -p overlays/run


cat <<EOF > overlays/run/contour-default-tls.yaml                                                                                                                                                                                                                          
#@ load("@ytt:data", "data")
#@ load("@ytt:overlay", "overlay")
#@ namespace = data.values.namespace
---
apiVersion: v1
kind: Secret
metadata:
  name: default-ca
  namespace: #@ namespace
type: kubernetes.io/tls
stringData:
  tls.crt: |
$(cat certs/ca.crt | sed 's/^/    /g')
  tls.key: |
$(cat certs/ca.key | sed 's/^/    /g')
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: default-ca-issuer
  namespace: #@ namespace
spec:
  ca:
    secretName: default-ca
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: tap-default-tls
  namespace: #@ namespace
spec:
  dnsNames:
  - #@ "*.${DOMAIN_NAME_RUN}"
  issuerRef:
    kind: Issuer
    name: default-ca-issuer
  secretName: tap-default-tls
---
apiVersion: projectcontour.io/v1
kind: TLSCertificateDelegation
metadata:
  name: contour-delegation
  namespace: #@ namespace
spec:
  delegations:
  - secretName: tap-default-tls
    targetNamespaces:
    - "*"
EOF

cat <<EOF > overlays/run/cnrs-https.yaml
#@ load("@ytt:overlay", "overlay")
#@overlay/match by=overlay.subset({"metadata":{"name":"config-network"}, "kind": "ConfigMap"})
---
data:
  #@overlay/match missing_ok=True
  default-external-scheme: https
  #@overlay/match missing_ok=True
  http-protocol: redirected
EOF
```

各々のoverlayファイルをsecretとして登録します。

```
kubectl -n tap-install create secret generic contour-default-tls \
  -o yaml \
  --dry-run=client \
  --from-file=overlays/run/contour-default-tls.yaml \
  | kubectl apply -f-

kubectl -n tap-install create secret generic cnrs-https \
  -o yaml \
  --dry-run=client \
  --from-file=overlays/run/cnrs-https.yaml \
  | kubectl apply -f-
```

#### Run Profileのインストール

tap-values.yamlを作成します。

```yaml
cat <<EOF > tap-values-run.yaml
profile: run

ceip_policy_disclosed: true

shared:
  ingress_domain: ${DOMAIN_NAME_RUN}
  ca_cert_data: |
$(cat certs/ca.crt | sed 's/^/    /g')

contour:
  infrastructure_provider: azure
  contour:
    configFileContents:
      accesslog-format: json  
  envoy:
    service:
      type: LoadBalancer
      loadBalancerIP: ${ENVOY_IP_RUN}      
      annotations:
        service.beta.kubernetes.io/azure-load-balancer-resource-group: tap-run

cnrs:
  domain_template: "{{.Name}}-{{.Namespace}}.{{.Domain}}"
  default_tls_secret: tanzu-system-ingress/tap-default-tls

appliveview_connector:
  backend:
    ingressEnabled: true
    host: appliveview.${DOMAIN_NAME_VIEW}

api_auto_registration:
  tap_gui_url: https://tap-gui.${DOMAIN_NAME_VIEW}
  cluster_name: run

accelerator:
  ingress:
    include: true    
    enable_tls: true  
  tls:
    secret_name: tap-default-tls
    namespace: tanzu-system-ingress

package_overlays:
- name: contour
  secrets:
  - name: contour-default-tls
- name: cnrs
  secrets:
  - name: cnrs-https

excluded_packages:
- image-policy-webhook.signing.apps.tanzu.vmware.com
- eventing.tanzu.vmware.com
EOF
```

TAPをインストールします。

```
tanzu package install tap \
  -p tap.tanzu.vmware.com \
  -v 1.3.3 \
  --values-file tap-values-run.yaml \
  -n tap-install \
  --wait=false
```

インストールが完了するまで待ちます。5分ほどかかります。

```
while [ "$(kubectl -n tap-install get app tap -o=jsonpath='{.status.friendlyDescription}')" != "Reconcile succeeded" ];do
  date
  kubectl get app -n tap-install
  echo "---------------------------------------------------------------------"
  sleep 30
done
echo "✅ Install succeeded"
```

packageinstallを確認します。

```
$ tanzu package installed list -n tap-install  

  NAME                      PACKAGE-NAME                               PACKAGE-VERSION  STATUS               
  api-auto-registration     apis.apps.tanzu.vmware.com                 0.1.2            Reconcile succeeded  
  appliveview-connector     connector.appliveview.tanzu.vmware.com     1.3.1            Reconcile succeeded  
  appsso                    sso.apps.tanzu.vmware.com                  2.0.0            Reconcile succeeded  
  cartographer              cartographer.tanzu.vmware.com              0.5.4            Reconcile succeeded  
  cert-manager              cert-manager.tanzu.vmware.com              1.7.2+tap.1      Reconcile succeeded  
  cnrs                      cnrs.tanzu.vmware.com                      2.0.2            Reconcile succeeded  
  contour                   contour.tanzu.vmware.com                   1.22.0+tap.5     Reconcile succeeded  
  fluxcd-source-controller  fluxcd.source.controller.tanzu.vmware.com  0.27.0+tap.1     Reconcile succeeded  
  ootb-delivery-basic       ootb-delivery-basic.tanzu.vmware.com       0.10.5           Reconcile succeeded  
  ootb-templates            ootb-templates.tanzu.vmware.com            0.10.5           Reconcile succeeded  
  policy-controller         policy.apps.tanzu.vmware.com               1.1.3            Reconcile succeeded  
  service-bindings          service-bindings.labs.vmware.com           0.8.1            Reconcile succeeded  
  services-toolkit          services-toolkit.tanzu.vmware.com          0.8.1            Reconcile succeeded  
  source-controller         controller.source.apps.tanzu.vmware.com    0.5.1            Reconcile succeeded  
  tap                       tap.tanzu.vmware.com                       1.3.3            Reconcile succeeded  
  tap-auth                  tap-auth.tanzu.vmware.com                  1.1.0            Reconcile succeeded  
  tap-telemetry             tap-telemetry.tanzu.vmware.com             0.3.2            Reconcile succeeded 
```

環境変数をenv.shに残します。

```
cat <<EOF >> env.sh
export ENVOY_IP_RUN=${ENVOY_IP_RUN}
export DOMAIN_NAME_RUN=${DOMAIN_NAME_RUN}
EOF
```

### Namespace毎の設定


https://docs.vmware.com/en/VMware-Tanzu-Application-Platform/1.3/tap/GUID-set-up-namespaces.html


Workloadに対するrbac.yamlを作成します。

```yaml
cat <<EOF > rbac.yaml
apiVersion: v1
kind: Secret
metadata:
  name: tap-registry
  annotations:
    secretgen.carvel.dev/image-pull-secret: ""
type: kubernetes.io/dockerconfigjson
data:
  .dockerconfigjson: e30K
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
secrets:
- name: registry-credentials
imagePullSecrets:
- name: registry-credentials
- name: tap-registry
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: default-permit-deliverable
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: deliverable
subjects:
- kind: ServiceAccount
  name: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: default-permit-workload
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: workload
subjects:
- kind: ServiceAccount
  name: default
EOF
```

#### Run Clusterでのセットアップ

```
kubectl config use-context tap-run-admin
```

```
NAMESPACE=demo
kubectl create ns ${NAMESPACE}
```

```
kubectl apply -f rbac.yaml -n ${NAMESPACE}
```

```
tanzu secret registry add registry-credentials --server ${ACR_NAME}.azurecr.io --username ${SERVICE_PRINCIPAL_RO_USERNAME} --password ${SERVICE_PRINCIPAL_RO_PASSWORD} --namespace ${NAMESPACE}
```

#### Build Clusterでのセットアップ


```
kubectl config use-context tap-build-admin
```

```
NAMESPACE=demo
kubectl create ns ${NAMESPACE}
```

```
kubectl apply -f rbac.yaml -n ${NAMESPACE}
```

```
tanzu secret registry add registry-credentials --server ${ACR_NAME}.azurecr.io --username ${SERVICE_PRINCIPAL_RW_USERNAME} --password ${SERVICE_PRINCIPAL_RW_PASSWORD} --namespace ${NAMESPACE}
```

Tekton Pipelineを作成します(ここではテストをスキップします)

```yaml
cat <<'EOF' > pipeline-maven.yaml
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: maven-test-pipeline
  labels:
    apps.tanzu.vmware.com/pipeline: test
    apps.tanzu.vmware.com/language: java
spec:
  params:
  - name: source-url
  - name: source-revision
  tasks:
  - name: test
    params:
    - name: source-url
      value: $(params.source-url)
    - name: source-revision
      value: $(params.source-revision)
    taskSpec:
      params:
      - name: source-url
      - name: source-revision
      steps:
      - name: test
        image: eclipse-temurin:17
        script: |-
          set -ex
          cd `mktemp -d`
          curl -s $(params.source-url) | tar -m -xzvf -
          # ./mvnw clean test -V --no-transfer-progress
          echo 'Skip'
EOF

kubectl apply -f pipeline-maven.yaml -n ${NAMESPACE}
```

Scan Policyを作成します。

```yaml
cat <<EOF > scan-policy.yaml
apiVersion: scanning.apps.tanzu.vmware.com/v1beta1
kind: ScanPolicy
metadata:
  name: scan-policy
spec:
  regoFile: |
    package main
    # Accepted Values: "Critical", "High", "Medium", "Low", "Negligible", "UnknownSeverity"
    notAllowedSeverities := ["Critical", "High", "UnknownSeverity"]
    # notAllowedSeverities := []
    ignoreCves := ["CVE-2016-1000027", "GHSA-36p3-wjmg-h94x"]
    contains(array, elem) = true {
      array[_] = elem
    } else = false { true }
    isSafe(match) {
      severities := { e | e := match.ratings.rating.severity } | { e | e := match.ratings.rating[_].severity }
      some i
      fails := contains(notAllowedSeverities, severities[i])
      not fails
    }
    isSafe(match) {
      ignore := contains(ignoreCves, match.id)
      ignore
    }
    deny[msg] {
      comps := { e | e := input.bom.components.component } | { e | e := input.bom.components.component[_] }
      some i
      comp := comps[i]
      vulns := { e | e := comp.vulnerabilities.vulnerability } | { e | e := comp.vulnerabilities.vulnerability[_] }
      some j
      vuln := vulns[j]
      ratings := { e | e := vuln.ratings.rating.severity } | { e | e := vuln.ratings.rating[_].severity }
      not isSafe(vuln)
      msg = sprintf("CVE %s %s %s", [comp.name, vuln.id, ratings])
    }
EOF

kubectl apply -f scan-policy.yaml -n ${NAMESPACE}
```

Grype Scannerをインストールします。

```yaml
NAMESPACE=demo
cat <<EOF > grype-${NAMESPACE}-values.yaml
namespace: ${NAMESPACE}
targetImagePullSecret: registry-credentials
metadataStore:
  url: https://metadata-store.${DOMAIN_NAME_VIEW}
  caSecret:
    name: store-ca-cert
    importFromNamespace: metadata-store-secrets
  authSecret:
    name: store-auth-token
    importFromNamespace: metadata-store-secrets
EOF
```

```
tanzu package install -n tap-install grype-${NAMESPACE} -p grype.scanning.apps.tanzu.vmware.com -v 1.3.1 -f grype-${NAMESPACE}-values.yaml
```


### Workloadの作成

Build Clusterにコンテキストを切り替えます。

```
kubectl config use-context tap-build-admin
```

Workloadを作成します。

```
tanzu apps workload apply tanzu-java-web-app \
  --app tanzu-java-web-app \
  --git-repo https://github.com/making/tanzu-java-web-app \
  --git-branch main \
  --type web \
  --label apps.tanzu.vmware.com/has-tests=true \
  --annotation autoscaling.knative.dev/minScale=1 \
  --request-memory 768Mi \
  -n demo \
  -y
```

Supply Chainの全てのResourceがHEALTYになるまで待ちます。

```
$ tanzu apps workload get -n demo tanzu-java-web-app 
📡 Overview
   name:   tanzu-java-web-app
   type:   web

💾 Source
   type:     git
   url:      https://github.com/sample-accelerators/tanzu-java-web-app
   branch:   main

📦 Supply Chain
   name:   source-test-scan-to-url

   RESOURCE           READY   HEALTHY   TIME    OUTPUT
   source-provider    True    True      7m14s   GitRepository/tanzu-java-web-app
   source-tester      True    True      6m42s   Runnable/tanzu-java-web-app
   source-scanner     True    True      5m56s   SourceScan/tanzu-java-web-app
   image-provider     True    True      3m38s   Image/tanzu-java-web-app
   image-scanner      True    True      2m58s   ImageScan/tanzu-java-web-app
   config-provider    True    True      2m57s   PodIntent/tanzu-java-web-app
   app-config         True    True      2m57s   ConfigMap/tanzu-java-web-app
   service-bindings   True    True      2m57s   ConfigMap/tanzu-java-web-app-with-claims
   api-descriptors    True    True      2m56s   ConfigMap/tanzu-java-web-app-with-api-descriptors
   config-writer      True    True      2m39s   Runnable/tanzu-java-web-app-config-writer
   deliverable        True    True      7m15s   ConfigMap/tanzu-java-web-app

🚚 Delivery

   Delivery resources not found.

💬 Messages
   No messages found.

🛶 Pods
   NAME                                         READY   STATUS      RESTARTS   AGE
   scan-tanzu-java-web-app-d2xll-bl7h6          0/1     Completed   0          6m43s
   scan-tanzu-java-web-app-v5m59-8trpp          0/1     Completed   0          3m39s
   tanzu-java-web-app-build-1-build-pod         0/1     Completed   0          5m56s
   tanzu-java-web-app-config-writer-q56zf-pod   0/1     Completed   0          2m56s
   tanzu-java-web-app-qgqlt-test-pod            0/1     Completed   0          7m12s

To see logs: "tanzu apps workload tail tanzu-java-web-app --namespace demo"
```

生成されたDeliverableを取得します。

```
kubectl get cm -n ${NAMESPACE} tanzu-java-web-app -otemplate='{{.data.deliverable}}' > deliverable.yaml
```

Run Clusterにコンテキストを切り替え、このDeliverableをapplyします。

```
kubectl config use-context tap-run-admin
```

```
kubectl apply -f deliverable.yaml -n ${NAMESPACE}
```

TAP 1.3.3の時点ではTAP GUIのSupply ChainプラグインでRun Clusterの情報を表示させるためのlabelが不足しているので、このlabelを追加します。

```
kubectl patch deliverable tanzu-java-web-app -n ${NAMESPACE} --type merge --patch "{\"metadata\":{\"labels\":{\"carto.run/workload-name\":\"tanzu-java-web-app\",\"carto.run/workload-namespace\":\"${NAMESPACE}\"}}}"
```

TAP GUIでWorkloadを確認します。

<img width="1024" alt="image" src="https://user-images.githubusercontent.com/106908/208016081-a16e0134-d14e-4f25-be26-b026644d7afc.png">

Sucurity Analysisを確認します。

<img width="1024" alt="image" src="https://user-images.githubusercontent.com/106908/208016325-85859c56-d288-42cb-9223-9b34ec547b24.png">


KServiceを取得して、アプリのURLを確認します。

```
$ kubectl get ksvc -n demo tanzu-java-web-app 
NAME                 URL                                                          LATESTCREATED              LATESTREADY                READY   REASON
tanzu-java-web-app   https://tanzu-java-web-app-demo.run.20-18-106-167.sslip.io   tanzu-java-web-app-00001   tanzu-java-web-app-00001   True 
```

アプリにアクセスします。



```
$ curl -k $(kubectl get ksvc -n demo tanzu-java-web-app -ojsonpath='{.status.url}')
Greetings from Spring Boot + Tanzu!
```

tanzu-java-web-appのcatalogを登録します。

"REGISTER ENTITY"をクリックします。

<img width="1024" alt="image" src="https://user-images.githubusercontent.com/106908/208018019-d40fea75-5795-42ee-b1c9-19e0a35ec916.png">

"Repository URL" に "https://github.com/making/tanzu-java-web-app/blob/main/catalog/catalog-info.yaml" を入力し、 "ANALYZE" -> "IMPORT" ボタンをクリックします。

<img width="1024" alt="image" src="https://user-images.githubusercontent.com/106908/208018092-8b169687-9a6f-4d6c-981e-678807f1da05.png">

"Runtime Resources" タブをクリックします。

<img width="1024" alt="image" src="https://user-images.githubusercontent.com/106908/208018364-05cb8ef7-33d2-44d9-a721-0d4aefba91af.png">

"Running" なPodをクリックします。

<img width="1024" alt="image" src="https://user-images.githubusercontent.com/106908/208018452-c3e6be64-88ab-43a1-ab21-63ffcd734ce2.png">

"Live View" パネルへスクロールします。

<img width="1024" alt="image" src="https://user-images.githubusercontent.com/106908/208018938-b029cde5-af40-45d8-9f1c-f6008ba35cb8.png">

tanzu-java-web-appに関するlive情報が表示されます。

<img width="1024" alt="image" src="https://user-images.githubusercontent.com/106908/208018767-6a39ec5f-686d-45cc-90db-569bfff77674.png">

"Memory" を選択します。

<img width="1024" alt="image" src="https://user-images.githubusercontent.com/106908/208018867-a0786db7-93e7-4777-bcfb-aef5fb07d3d3.png">