IK.AM

@making's tech note


Tanzu Community EditionのPackage Repositoryをimgpkgでrelocateする

🗃 {Dev/Carvel/imgpkg}
🏷 Kubernetes 🏷 Carvle 🏷 air-gapped 🏷 imgpkg 🏷 TCE 🏷 kapp-controller 🏷 Prometheus 
🗓 Updated at 2021-12-19T00:28:37+09:00  🗓 Created at 2021-12-19T00:28:37+09:00 {✒️️ Edit  ⏰ History  🗑 Delete}

TUNA-JP Advent Calendar 2021 その2の19日目のエントリです。

Tanzu Community Editionで提供されているPackage Repository は次のコマンドで登録できます。

tanzu package repository add tce-repo \
  --url projects.registry.vmware.com/tce/main:0.9.1 \
  --namespace tanzu-package-repo-global

このRepositoryに含まれるPackageのいくつかはDockerHub上のコンテナイメージをmanifestに持っています。 DockerHubはrate limitがあるため、環境によってはPackageをインストールしようとすると409エラーでImagePullBackOffになるかもしれません。 あるいはTCEをAir-gappedな環境にインストールしたい場合もパブリックなコンテナイメージレジストリからは取得できません。

CarvelのPackage Repositoryはレポジトリに含まれるPackage及び、そのPackageで使用される全てのコンテナイメージを別のレジストリにrelocateすることができます。ドキュメントは次の通りです。
https://carvel.dev/kapp-controller/docs/latest/air-gapped-workflow/#copy-packagerepository-bundle-to-new-location

今回はTCEのRepositoryを こちらの記事 で利用したHarborにrelocateしてみます。

次のコマンドで一括でイメージをrelocateできます。

imgpkg copy -b projects.registry.vmware.com/tce/main:0.9.1 --to-repo ${HARBOR_HOST}/library/tce/main --registry-ca-cert-path $HOME/.config/tanzu/tkg/providers/ytt/03_customizations/harbor-ca.pem

次のようなログが出力されます。合計75個のイメージがrelocateされたことが分かります。

copy | exporting 75 images...
copy | will export gcr.io/knative-releases/knative.dev/net-certmanager/cmd/controller@sha256:cab4e2f78c14d200191bef8873b7a37fed7cc7747f767268b30a6dbcdb2f27c0
copy | will export gcr.io/knative-releases/knative.dev/net-certmanager/cmd/webhook@sha256:2ae59e8006ec196fae3ff8b4adce9e12d00076217b0b34431adc134d5de61deb
copy | will export gcr.io/knative-releases/knative.dev/net-contour/cmd/controller@sha256:705ac18c11352a874ed35d7404eed1792d599bbd5fcdaed7b945484b1d567bd9
copy | will export gcr.io/knative-releases/knative.dev/serving/cmd/activator@sha256:91e67a579378fa39d7c941e379db183464c3add3d53b4617f65d9cbc2f0c770a
copy | will export gcr.io/knative-releases/knative.dev/serving/cmd/autoscaler@sha256:761dc36210e69ebef3a64ce72ad9f54f8172e4aed6b97e8a706e3128956ec54d
copy | will export gcr.io/knative-releases/knative.dev/serving/cmd/controller@sha256:d772809059033e437d6e98248a334ded37b6f430c2ca23377875cc2459a3b73e
copy | will export gcr.io/knative-releases/knative.dev/serving/cmd/default-domain@sha256:ee79c1b6c9e07d6bbdf8ff6a53a30b7c288140c1a253e4e545251fe2e8911d44
copy | will export gcr.io/knative-releases/knative.dev/serving/cmd/queue@sha256:6cd0c234bfbf88ac75df5243c2f9213dcc9def610414c506d418f9388187b771
copy | will export gcr.io/knative-releases/knative.dev/serving/cmd/webhook@sha256:268bd1383b56ba7b9acf391c681f7a63780c22dcd4555c2f4a7b61ec6da81cf4
copy | will export index.docker.io/bitnami/external-dns@sha256:1a26ba96760f5f488be15a08070254de9705893bbddd4ec51f40cc6f2cc1ecfd
copy | will export index.docker.io/envoyproxy/envoy@sha256:55d35e368436519136dbd978fa0682c49d8ab99e4d768413510f226762b30b07
copy | will export index.docker.io/envoyproxy/envoy@sha256:ac6a29af5bee160a1b4425d7c7a41a4d8a08a7f9dd7f225f21b5375f6439457a
copy | will export index.docker.io/envoyproxy/envoy@sha256:e5c2bb2870d0e59ce917a5100311813b4ede96ce4eb0c6bfa879e3fbe3e83935
copy | will export index.docker.io/fluent/fluent-bit@sha256:63745ff3b3ced72611f31b9998435721405573a3239d79974f0e3a60e4903d02
copy | will export index.docker.io/goharbor/harbor-core@sha256:b4703fd32a9dc4de208649405ac88de36126122f5d30ea0e7e2eff5626f28397
copy | will export index.docker.io/goharbor/harbor-db@sha256:4fac4598a1066611a7aaca000a56f81b951b5d047d6cefc8643fc4ecc638da67
copy | will export index.docker.io/goharbor/harbor-exporter@sha256:17b33dc8816db4fa214ce1765d5cc9e6ea6e2f7b80f0f64182aa84c9575bc3d2
copy | will export index.docker.io/goharbor/harbor-jobservice@sha256:e555c6736db6383e8147c847c733a36a16400f292e78ee4040c3396bae486da5
copy | will export index.docker.io/goharbor/harbor-portal@sha256:bd5103391fc37ecfa84cb07a4a7ff0dbed88b72f02d92e38d730da7141518b6c
copy | will export index.docker.io/goharbor/harbor-registryctl@sha256:f6764ff0ee618c916568d7bfbf144a9f9ec70c23589985efa451614aa735a371
copy | will export index.docker.io/goharbor/notary-server-photon@sha256:b1f41cfc903ecdf1e5b4efe7e640fe858f70a1c5cd0d3c51d4cc4ab34eb568fc
copy | will export index.docker.io/goharbor/notary-signer-photon@sha256:a0b375fa2e7c27de1cf41e23800e97878adcdeb236ecbf94dea16d03ed4449ef
copy | will export index.docker.io/goharbor/redis-photon@sha256:dc34c1f49c0bffe5fff540d1902bd45f3b6231f08d7abc244885744066085781
copy | will export index.docker.io/goharbor/registry-photon@sha256:c5c65ec7c78ca3965f75536d83d8252780191819b45e8ab5a7e9ff07f9e11046
copy | will export index.docker.io/goharbor/trivy-adapter-photon@sha256:42649438283db6db7e2086058d690be776266a25bea7255b67d24be0352b85e7
copy | will export index.docker.io/jimmidyson/configmap-reload@sha256:904d08e9f701d3d8178cb61651dbe8edc5d08dd5895b56bdcac9e5805ea82b52
copy | will export index.docker.io/library/busybox@sha256:b37dd066f59a4961024cf4bed74cae5e68ac26b48807292bd12198afa3ecb778
copy | will export index.docker.io/nfvpe/multus@sha256:31a0dc9ae164819c86f3ae49af8a01a34d2b1e0ced9340d1fb9fdb0743e3b599
copy | will export index.docker.io/nfvpe/multus@sha256:43d6316cc38d1922d26662d750cc85dd2e1233ded10f64c4ca63dde29d79493c
copy | will export index.docker.io/nfvpe/multus@sha256:78af60d4083dabf6ee6ec762ec0facaff8a05ea58621d45b607b99236105ceb7
copy | will export index.docker.io/nfvpe/multus@sha256:aa849206c510add3e3e1c6a843036bbcab0378312aa9b3e664b5c5139638ecdc
copy | will export index.docker.io/openpolicyagent/gatekeeper@sha256:9cd6e864dcc37d55970e592cc885c129cd7d13e59e3988a337abd77a41852879
copy | will export index.docker.io/projectcontour/contour@sha256:31c376f80f5b80f2ac5558c1c50711438796b22b742285789ea21cd2bba244e2
copy | will export index.docker.io/projectcontour/contour@sha256:3e2eb259697ad21415902436ba7d6544a2da007a8cb6d0a4abd46fa956a95d49
copy | will export index.docker.io/projectcontour/contour@sha256:4ff3e4b23770d80e25915e1450fed692d8490aa491b04f32124908931dd7d5f8
copy | will export index.docker.io/prom/alertmanager@sha256:624c1a5063c7c80635081a504c3e1b020d89809651978eb5d0b652a394f3022d
copy | will export index.docker.io/prom/prometheus@sha256:d1a9a86b9a3e60a9ea3cde141bdc936847456acc497e0affe7e288234383efa5
copy | will export index.docker.io/prom/pushgateway@sha256:ca32c7864bb2573bf27ff6628a03d17b37b1aa3dc367b5d86831e6c0f0761376
copy | will export projects.registry.vmware.com/tce/cert-manager@sha256:31cdadff5b576fb04622b15c9c8fc22006bde0a4da21b9fce0dbf0b99254e2bb
copy | will export projects.registry.vmware.com/tce/cert-manager@sha256:c087c02ac3957486baeb91f9e0713a2d672dd7532c739ed836e9795e4160aeb6
copy | will export projects.registry.vmware.com/tce/cert-manager@sha256:fcd2ea27f3d2c86ef56da7bb92c231537b12824d7c1bc3d7e5cea7d2aed4a9b9
copy | will export projects.registry.vmware.com/tce/contour@sha256:0c3d0f33c171e437268e57bdbe0d83feb1606362d8235f1b656556da8c944e18
copy | will export projects.registry.vmware.com/tce/contour@sha256:7f031ada007ab3ba53a8a71b55a7f2123343a7f180263de13ac72d7de97a16b0
copy | will export projects.registry.vmware.com/tce/contour@sha256:e485ed740ad65c6831ce821d09f419fe92b13f00e1abac40f20f19f5bde9696a
copy | will export projects.registry.vmware.com/tce/external-dns@sha256:8e797800abde577a732d571947d70e1866764c57e00bb587b756911852dd41c3
copy | will export projects.registry.vmware.com/tce/fluent-bit@sha256:e8c0873ef8e1ea04f32b88afbab64fc4338d0c0536ef2f91b9dd54bee2c59ed7
copy | will export projects.registry.vmware.com/tce/gatekeeper@sha256:b7a2102712f4de53f1d46532d468b5af5f1bfba60d5b9c6d1cffe59967b9077d
copy | will export projects.registry.vmware.com/tce/grafana@sha256:53d41d9ac1534fc381efa5bb181aa4cac1ec26fc77c7ffadb34550930112e193
copy | will export projects.registry.vmware.com/tce/harbor@sha256:51c9728301e64763ff872378e86b17d4cedc9cb64305519faa4e90c1ce257e9a
copy | will export projects.registry.vmware.com/tce/knative-serving@sha256:e99da49f3a7515e787ca5278a5d99600e9ee6eaa863e5283e0f6477a7c3d8bc1
copy | will export projects.registry.vmware.com/tce/local-path-provisioner@sha256:6434e827349036958783c1f81b01838b5d7316c1275a25ba0f76ea7a89455231
copy | will export projects.registry.vmware.com/tce/local-path-provisioner@sha256:a2d213121067f0f3df26ec813eb752b4e4d5e9f590b51d0a922c4ecd71541d65
copy | will export projects.registry.vmware.com/tce/local-path-storage@sha256:336f49fb455dae12a03507c1874fdbee38240f4243c512539a2bb1835580462f
copy | will export projects.registry.vmware.com/tce/local-path-storage@sha256:e0db08cc6e83efb1f772ab9714d78900b5634146c266954abc805461a005beb1
copy | will export projects.registry.vmware.com/tce/main@sha256:7e2e49ae8ffec158cf0b9f33701c31d5f70f4ac3a9688fc28523a95c988d0813
copy | will export projects.registry.vmware.com/tce/multus-cni@sha256:c2f76e193746de752a7387f6068293af649a5ddf9492d80b66f46f266af8c770
copy | will export projects.registry.vmware.com/tce/prometheus@sha256:b6d81e04a8ba1d6b5a33fb3f781f8238a26fa9a2add1df301b9e189d58f6682b
copy | will export projects.registry.vmware.com/tce/velero/velero-plugin-for-aws@sha256:b2b82dc47cec1f3d86595b07251efddc870133a4fc89b837502cb8e73e8187e9
copy | will export projects.registry.vmware.com/tce/velero/velero-plugin-for-microsoft-azure@sha256:84e07ce2bf61bed674c32c7f981094c0c3c2f82981ccc459ef33eccd00483c7d
copy | will export projects.registry.vmware.com/tce/velero/velero@sha256:a99d463670667fbef3363785181317e1dd87c44af808a775fc09046b80c3643f
copy | will export projects.registry.vmware.com/tce/velero@sha256:d1334e22e7c3e4e3e8e1b9b7a728f5f4b00c885f5963a74e20eac95484d82313
copy | will export projects.registry.vmware.com/tkg/grafana/grafana@sha256:df8f25cc9ee43d6ea4c22f9c6c46644e2b9a485562dd0dafe831b5b582ac0a71
copy | will export projects.registry.vmware.com/tkg/grafana/k8s-sidecar@sha256:9f1ad1e5e404bc43f9591b1189c187f535d6f61769468c49b4fc97add803d7b9
copy | will export projects.registry.vmware.com/tkg/prometheus/cadvisor@sha256:b4cd4cc0ef05630f70d621420ad1316f631f35cef21edb7a62fff7bd787bbfd3
copy | will export quay.io/coreos/kube-state-metrics@sha256:ace842fc85031688d06c4aa000b5b1e58ba3b9dd13d26e7c8f2547f7ee0bcc84
copy | will export quay.io/jetstack/cert-manager-cainjector@sha256:51c0df411b66aa175e9fc6840f3135d55b52c3781d0b3d4aa10862066d460193
copy | will export quay.io/jetstack/cert-manager-cainjector@sha256:a24ab35640bb9ddd58a0801e8021bec68e0ab247d62ea10c69d86506097e9eaa
copy | will export quay.io/jetstack/cert-manager-cainjector@sha256:de02e3f445cfe7c035f2a9939b948c4d043011713389d9437311a62740f20bef
copy | will export quay.io/jetstack/cert-manager-controller@sha256:22543d32793abc0069680f80ee5be348dcbb3c74c85ba55835b4cf6c76fe18da
copy | will export quay.io/jetstack/cert-manager-controller@sha256:7b039d469ed739a652f3bb8a1ddc122942b66cceeb85bac315449724ee64287f
copy | will export quay.io/jetstack/cert-manager-controller@sha256:85f03d2ffdfd53f4dab53147b3192a5322a3300b5368f60416e7b838e65f42d1
copy | will export quay.io/jetstack/cert-manager-webhook@sha256:41eacd93a30b566b780a6ae525b2547d2a87f1ec5f067fc02840a220aeb0c3f7
copy | will export quay.io/jetstack/cert-manager-webhook@sha256:ed6354190d259524d32ae74471f93bf46bfdcf4df6f73629eedf576cd87e10b8
copy | will export quay.io/jetstack/cert-manager-webhook@sha256:f91867a7769436d72e6f4bb68e4e3d240d93d5bc8cd59742298a2a2b3ccf11b7
copy | will export quay.io/prometheus/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810
copy | exported 75 images
copy | importing 75 images...

 4.05 GiB / 4.05 GiB [==================================================================================================================================================================================================================================================]  99.94% 3.79 MiB/s 18m15s

copy | done uploading images

Succeeded

HarborのUIを見ると沢山のイメージが登録されたことが分かります。

image

relocation先がair-gapped環境の場合は次のようにtarファイルを経由することができます。

imgpkg copy -b projects.registry.vmware.com/tce/main:0.9.1 --to-tar repo.tar --registry-ca-cert-path $HOME/.config/tanzu/tkg/providers/ytt/03_customizations/harbor-ca.pem
imgpkg copy --tar repo.tar --to-repo ${HARBOR_HOST}/library/tce/main --registry-ca-cert-path $HOME/.config/tanzu/tkg/providers/ytt/03_customizations/harbor-ca.pem

relocate後のPackageRepositoryの登録は、次のようにURLを変えるだけです。

tanzu package repository add tce-repo \
  --url ${HARBOR_HOST}/library/tce/main:0.9.1 \
  --namespace tanzu-package-repo-global

次のコマンドでPackageRepositoryを確認できます。

$ kubectl get packagerepository -n tanzu-package-repo-global tce-repo -oyaml
apiVersion: packaging.carvel.dev/v1alpha1
kind: PackageRepository
metadata:
  creationTimestamp: "2021-12-18T14:48:39Z"
  finalizers:
  - finalizers.packagerepository.packaging.carvel.dev/delete
  generation: 1
  name: tce-repo
  namespace: tanzu-package-repo-global
  resourceVersion: "2198551"
  uid: c9d30610-2d0a-478c-aceb-849740efffc1
spec:
  fetch:
    imgpkgBundle:
      image: harbor-10-213-232-22.sslip.io/library/tce/main:0.9.1
status:
  conditions:
  - status: "True"
    type: ReconcileSucceeded
  consecutiveReconcileSuccesses: 1
  deploy:
    exitCode: 0
    finished: true
    startedAt: "2021-12-18T14:49:13Z"
    stdout: |-
      Target cluster 'https://100.64.0.1:443'
      Changes
      Namespace                  Name                                                  Kind             Conds.  Age  Op      Op st.  Wait to  Rs  Ri
      tanzu-package-repo-global  cert-manager.community.tanzu.vmware.com               PackageMetadata  -       -    create  -       -        -   -
      ^                          cert-manager.community.tanzu.vmware.com.1.3.3         Package          -       -    create  -       -        -   -
      ^                          cert-manager.community.tanzu.vmware.com.1.4.4         Package          -       -    create  -       -        -   -
      ^                          cert-manager.community.tanzu.vmware.com.1.5.3         Package          -       -    create  -       -        -   -
      ^                          contour.community.tanzu.vmware.com                    PackageMetadata  -       -    create  -       -        -   -
      ^                          contour.community.tanzu.vmware.com.1.17.1             Package          -       -    create  -       -        -   -
      ^                          contour.community.tanzu.vmware.com.1.17.2             Package          -       -    create  -       -        -   -
      ^                          contour.community.tanzu.vmware.com.1.18.1             Package          -       -    create  -       -        -   -
      ^                          external-dns.community.tanzu.vmware.com               PackageMetadata  -       -    create  -       -        -   -
      ^                          external-dns.community.tanzu.vmware.com.0.8.0         Package          -       -    create  -       -        -   -
      ^                          fluent-bit.community.tanzu.vmware.com                 PackageMetadata  -       -    create  -       -        -   -
      ^                          fluent-bit.community.tanzu.vmware.com.1.7.5           Package          -       -    create  -       -        -   -
      ^                          gatekeeper.community.tanzu.vmware.com                 PackageMetadata  -       -    create  -       -        -   -
      ^                          gatekeeper.community.tanzu.vmware.com.1.0.0           Package          -       -    create  -       -        -   -
      ^                          grafana.community.tanzu.vmware.com                    PackageMetadata  -       -    create  -       -        -   -
      ^                          grafana.community.tanzu.vmware.com.7.5.7              Package          -       -    create  -       -        -   -
      ^                          harbor.community.tanzu.vmware.com                     PackageMetadata  -       -    create  -       -        -   -
      ^                          harbor.community.tanzu.vmware.com.2.2.3               Package          -       -    create  -       -        -   -
      ^                          knative-serving.community.tanzu.vmware.com            PackageMetadata  -       -    create  -       -        -   -
      ^                          knative-serving.community.tanzu.vmware.com.0.22.0     Package          -       -    create  -       -        -   -
      ^                          local-path-storage.community.tanzu.vmware.com         PackageMetadata  -       -    create  -       -        -   -
      ^                          local-path-storage.community.tanzu.vmware.com.0.0.19  Package          -       -    create  -       -        -   -
      ^                          local-path-storage.community.tanzu.vmware.com.0.0.20  Package          -       -    create  -       -        -   -
      ^                          multus-cni.community.tanzu.vmware.com                 PackageMetadata  -       -    create  -       -        -   -
      ^                          multus-cni.community.tanzu.vmware.com.3.7.1           Package          -       -    create  -       -        -   -
      ^                          prometheus.community.tanzu.vmware.com                 PackageMetadata  -       -    create  -       -        -   -
      ^                          prometheus.community.tanzu.vmware.com.2.27.0          Package          -       -    create  -       -        -   -
      ^                          velero.community.tanzu.vmware.com                     PackageMetadata  -       -    create  -       -        -   -
      ^                          velero.community.tanzu.vmware.com.1.6.3               Package          -       -    create  -       -        -   -
      Op:      29 create, 0 delete, 0 update, 0 noop
      Wait to: 0 reconcile, 0 delete, 29 noop
      Pod watching error: pods is forbidden: User "system:serviceaccount:tkg-system:kapp-controller-sa" cannot list resource "pods" in API group "" in the namespace "tanzu-package-repo-global"
      2:49:58PM: ---- applying 29 changes [0/29 done] ----
      2:49:58PM: create package/cert-manager.community.tanzu.vmware.com.1.4.4 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:58PM: create package/velero.community.tanzu.vmware.com.1.6.3 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:58PM: create package/cert-manager.community.tanzu.vmware.com.1.3.3 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:58PM: create packagemetadata/cert-manager.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:58PM: create package/cert-manager.community.tanzu.vmware.com.1.5.3 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:58PM: create package/contour.community.tanzu.vmware.com.1.17.1 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:58PM: create packagemetadata/contour.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:58PM: create package/contour.community.tanzu.vmware.com.1.18.1 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:58PM: create package/contour.community.tanzu.vmware.com.1.17.2 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create packagemetadata/external-dns.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create package/external-dns.community.tanzu.vmware.com.0.8.0 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create package/gatekeeper.community.tanzu.vmware.com.1.0.0 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create packagemetadata/fluent-bit.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create package/fluent-bit.community.tanzu.vmware.com.1.7.5 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create packagemetadata/gatekeeper.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create packagemetadata/local-path-storage.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create packagemetadata/grafana.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create package/grafana.community.tanzu.vmware.com.7.5.7 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create packagemetadata/harbor.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create package/harbor.community.tanzu.vmware.com.2.2.3 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create packagemetadata/knative-serving.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create package/knative-serving.community.tanzu.vmware.com.0.22.0 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create package/multus-cni.community.tanzu.vmware.com.3.7.1 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create package/local-path-storage.community.tanzu.vmware.com.0.0.19 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create package/local-path-storage.community.tanzu.vmware.com.0.0.20 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create packagemetadata/multus-cni.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create package/prometheus.community.tanzu.vmware.com.2.27.0 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create packagemetadata/prometheus.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: create packagemetadata/velero.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ---- waiting on 29 changes [0/29 done] ----
      2:49:59PM: ok: noop packagemetadata/velero.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop packagemetadata/gatekeeper.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop package/fluent-bit.community.tanzu.vmware.com.1.7.5 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop package/cert-manager.community.tanzu.vmware.com.1.4.4 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop packagemetadata/local-path-storage.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop packagemetadata/grafana.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop package/grafana.community.tanzu.vmware.com.7.5.7 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop packagemetadata/harbor.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop package/velero.community.tanzu.vmware.com.1.6.3 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop package/cert-manager.community.tanzu.vmware.com.1.3.3 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop package/harbor.community.tanzu.vmware.com.2.2.3 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop packagemetadata/knative-serving.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop packagemetadata/cert-manager.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop package/cert-manager.community.tanzu.vmware.com.1.5.3 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop package/contour.community.tanzu.vmware.com.1.17.1 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop packagemetadata/contour.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop package/contour.community.tanzu.vmware.com.1.18.1 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop package/contour.community.tanzu.vmware.com.1.17.2 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop package/knative-serving.community.tanzu.vmware.com.0.22.0 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop packagemetadata/external-dns.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop package/external-dns.community.tanzu.vmware.com.0.8.0 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop package/gatekeeper.community.tanzu.vmware.com.1.0.0 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop packagemetadata/fluent-bit.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop package/local-path-storage.community.tanzu.vmware.com.0.0.20 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop package/multus-cni.community.tanzu.vmware.com.3.7.1 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop package/local-path-storage.community.tanzu.vmware.com.0.0.19 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop package/prometheus.community.tanzu.vmware.com.2.27.0 (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop packagemetadata/multus-cni.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ok: noop packagemetadata/prometheus.community.tanzu.vmware.com (data.packaging.carvel.dev/v1alpha1) namespace: tanzu-package-repo-global
      2:49:59PM: ---- applying complete [29/29 done] ----
      2:49:59PM: ---- waiting complete [29/29 done] ----
      Succeeded
    updatedAt: "2021-12-18T14:49:59Z"
  fetch:
    exitCode: 0
    startedAt: "2021-12-18T14:49:11Z"
    stdout: |
      apiVersion: vendir.k14s.io/v1alpha1
      directories:
      - contents:
        - imgpkgBundle:
            image: harbor-10-213-232-22.sslip.io/library/tce/main@sha256:7e2e49ae8ffec158cf0b9f33701c31d5f70f4ac3a9688fc28523a95c988d0813
          path: .
        path: "0"
      kind: LockConfig
    updatedAt: "2021-12-18T14:49:13Z"
  friendlyDescription: Reconcile succeeded
  observedGeneration: 1
  template:
    exitCode: 0
    stderr: |
      resolve | final: projects.registry.vmware.com/tce/cert-manager@sha256:31cdadff5b576fb04622b15c9c8fc22006bde0a4da21b9fce0dbf0b99254e2bb -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:31cdadff5b576fb04622b15c9c8fc22006bde0a4da21b9fce0dbf0b99254e2bb
      resolve | final: projects.registry.vmware.com/tce/cert-manager@sha256:c087c02ac3957486baeb91f9e0713a2d672dd7532c739ed836e9795e4160aeb6 -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:c087c02ac3957486baeb91f9e0713a2d672dd7532c739ed836e9795e4160aeb6
      resolve | final: projects.registry.vmware.com/tce/cert-manager@sha256:fcd2ea27f3d2c86ef56da7bb92c231537b12824d7c1bc3d7e5cea7d2aed4a9b9 -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:fcd2ea27f3d2c86ef56da7bb92c231537b12824d7c1bc3d7e5cea7d2aed4a9b9
      resolve | final: projects.registry.vmware.com/tce/contour@sha256:0c3d0f33c171e437268e57bdbe0d83feb1606362d8235f1b656556da8c944e18 -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:0c3d0f33c171e437268e57bdbe0d83feb1606362d8235f1b656556da8c944e18
      resolve | final: projects.registry.vmware.com/tce/contour@sha256:7f031ada007ab3ba53a8a71b55a7f2123343a7f180263de13ac72d7de97a16b0 -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:7f031ada007ab3ba53a8a71b55a7f2123343a7f180263de13ac72d7de97a16b0
      resolve | final: projects.registry.vmware.com/tce/contour@sha256:e485ed740ad65c6831ce821d09f419fe92b13f00e1abac40f20f19f5bde9696a -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:e485ed740ad65c6831ce821d09f419fe92b13f00e1abac40f20f19f5bde9696a
      resolve | final: projects.registry.vmware.com/tce/external-dns@sha256:8e797800abde577a732d571947d70e1866764c57e00bb587b756911852dd41c3 -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:8e797800abde577a732d571947d70e1866764c57e00bb587b756911852dd41c3
      resolve | final: projects.registry.vmware.com/tce/fluent-bit@sha256:e8c0873ef8e1ea04f32b88afbab64fc4338d0c0536ef2f91b9dd54bee2c59ed7 -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:e8c0873ef8e1ea04f32b88afbab64fc4338d0c0536ef2f91b9dd54bee2c59ed7
      resolve | final: projects.registry.vmware.com/tce/gatekeeper@sha256:b7a2102712f4de53f1d46532d468b5af5f1bfba60d5b9c6d1cffe59967b9077d -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:b7a2102712f4de53f1d46532d468b5af5f1bfba60d5b9c6d1cffe59967b9077d
      resolve | final: projects.registry.vmware.com/tce/grafana@sha256:53d41d9ac1534fc381efa5bb181aa4cac1ec26fc77c7ffadb34550930112e193 -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:53d41d9ac1534fc381efa5bb181aa4cac1ec26fc77c7ffadb34550930112e193
      resolve | final: projects.registry.vmware.com/tce/harbor@sha256:51c9728301e64763ff872378e86b17d4cedc9cb64305519faa4e90c1ce257e9a -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:51c9728301e64763ff872378e86b17d4cedc9cb64305519faa4e90c1ce257e9a
      resolve | final: projects.registry.vmware.com/tce/knative-serving@sha256:e99da49f3a7515e787ca5278a5d99600e9ee6eaa863e5283e0f6477a7c3d8bc1 -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:e99da49f3a7515e787ca5278a5d99600e9ee6eaa863e5283e0f6477a7c3d8bc1
      resolve | final: projects.registry.vmware.com/tce/local-path-storage@sha256:336f49fb455dae12a03507c1874fdbee38240f4243c512539a2bb1835580462f -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:336f49fb455dae12a03507c1874fdbee38240f4243c512539a2bb1835580462f
      resolve | final: projects.registry.vmware.com/tce/local-path-storage@sha256:e0db08cc6e83efb1f772ab9714d78900b5634146c266954abc805461a005beb1 -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:e0db08cc6e83efb1f772ab9714d78900b5634146c266954abc805461a005beb1
      resolve | final: projects.registry.vmware.com/tce/multus-cni@sha256:c2f76e193746de752a7387f6068293af649a5ddf9492d80b66f46f266af8c770 -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:c2f76e193746de752a7387f6068293af649a5ddf9492d80b66f46f266af8c770
      resolve | final: projects.registry.vmware.com/tce/prometheus@sha256:b6d81e04a8ba1d6b5a33fb3f781f8238a26fa9a2add1df301b9e189d58f6682b -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:b6d81e04a8ba1d6b5a33fb3f781f8238a26fa9a2add1df301b9e189d58f6682b
      resolve | final: projects.registry.vmware.com/tce/velero@sha256:d1334e22e7c3e4e3e8e1b9b7a728f5f4b00c885f5963a74e20eac95484d82313 -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:d1334e22e7c3e4e3e8e1b9b7a728f5f4b00c885f5963a74e20eac95484d82313
    updatedAt: "2021-12-18T14:49:13Z"

spec.fetch.imgpkgBundle.imagestatus.template.stderrを見るとイメージがHarborにrelocateされていることが分かります。

Packageのインストールもしてみます。PackageRepositoryが登録されてしまえば、インストール方法はrelocation関係なく行えます。

次のコマンドでPrometheus Packageをインストールします。

cat <<EOF > prometheus-data-values.yaml
prometheus:
  service:
    type: LoadBalancer
EOF
tanzu package install prometheus --package-name prometheus.community.tanzu.vmware.com --version 2.27.0 --namespace default --values-file prometheus-data-values.yaml

インストールが完了したら、次のコマンドで作成されたApp CRを確認します。

$ kubectl get app prometheus -oyaml
apiVersion: kappctrl.k14s.io/v1alpha1
kind: App
metadata:
  creationTimestamp: "2021-12-18T14:55:28Z"
  finalizers:
  - finalizers.kapp-ctrl.k14s.io/delete
  generation: 1
  name: prometheus
  namespace: default
  ownerReferences:
  - apiVersion: packaging.carvel.dev/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: PackageInstall
    name: prometheus
    uid: 6791d34c-c136-4f48-a80d-12f5f349ed94
  resourceVersion: "2200691"
  uid: 947b27a4-67a1-417c-a3df-b8865e301936
spec:
  deploy:
  - kapp: {}
  fetch:
  - imgpkgBundle:
      image: harbor-10-213-232-22.sslip.io/library/tce/main@sha256:b6d81e04a8ba1d6b5a33fb3f781f8238a26fa9a2add1df301b9e189d58f6682b
  serviceAccountName: prometheus-default-sa
  template:
  - ytt:
      paths:
      - config/
      valuesFrom:
      - secretRef:
          name: prometheus-default-values
  - kbld:
      paths:
      - '-'
      - .imgpkg/images.yml
status:
  conditions:
  - status: "True"
    type: ReconcileSucceeded
  consecutiveReconcileSuccesses: 4
  deploy:
    exitCode: 0
    finished: true
    startedAt: "2021-12-18T14:58:37Z"
    stdout: |-
      Target cluster 'https://100.64.0.1:443' (nodes: cheetah-control-plane-xtwcj, 1+)
      02:58:37PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreaclusternetworkpolicystats"}
      02:58:37PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"networkpolicystats"}
      02:58:37PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreanetworkpolicystats"}
      Changes
      Namespace  Name  Kind  Conds.  Age  Op  Op st.  Wait to  Rs  Ri
      Op:      0 create, 0 delete, 0 update, 0 noop
      Wait to: 0 reconcile, 0 delete, 0 noop
      Succeeded
    updatedAt: "2021-12-18T14:58:39Z"
  fetch:
    exitCode: 0
    startedAt: "2021-12-18T14:58:35Z"
    stdout: |
      apiVersion: vendir.k14s.io/v1alpha1
      directories:
      - contents:
        - imgpkgBundle:
            image: harbor-10-213-232-22.sslip.io/library/tce/main@sha256:b6d81e04a8ba1d6b5a33fb3f781f8238a26fa9a2add1df301b9e189d58f6682b
          path: .
        path: "0"
      kind: LockConfig
    updatedAt: "2021-12-18T14:58:36Z"
  friendlyDescription: Reconcile succeeded
  inspect:
    exitCode: 0
    stdout: |-
      Target cluster 'https://100.64.0.1:443' (nodes: cheetah-control-plane-xtwcj, 1+)
      02:58:39PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"networkpolicystats"}
      02:58:39PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreaclusternetworkpolicystats"}
      02:58:39PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreanetworkpolicystats"}
      Resources in app 'prometheus-ctrl'
      Namespace   Name                                            Kind                   Owner    Conds.  Rs  Ri  Age
      (cluster)   alertmanager                                    ClusterRole            kapp     -       ok  -   3m
      ^           alertmanager                                    ClusterRoleBinding     kapp     -       ok  -   3m
      ^           prometheus                                      Namespace              kapp     -       ok  -   3m
      ^           prometheus-cadvisor                             ClusterRole            kapp     -       ok  -   3m
      ^           prometheus-cadvisor                             ClusterRoleBinding     kapp     -       ok  -   3m
      ^           prometheus-kube-state-metrics                   ClusterRole            kapp     -       ok  -   3m
      ^           prometheus-kube-state-metrics                   ClusterRoleBinding     kapp     -       ok  -   3m
      ^           prometheus-node-exporter                        ClusterRole            kapp     -       ok  -   3m
      ^           prometheus-node-exporter                        ClusterRoleBinding     kapp     -       ok  -   3m
      ^           prometheus-node-exporter                        PodSecurityPolicy      kapp     -       ok  -   3m
      ^           prometheus-pushgateway                          ClusterRole            kapp     -       ok  -   3m
      ^           prometheus-pushgateway                          ClusterRoleBinding     kapp     -       ok  -   3m
      ^           prometheus-server                               ClusterRole            kapp     -       ok  -   3m
      ^           prometheus-server                               ClusterRoleBinding     kapp     -       ok  -   3m
      prometheus  alertmanager                                    Deployment             kapp     2/2 t   ok  -   3m
      ^           alertmanager                                    Endpoints              cluster  -       ok  -   3m
      ^           alertmanager                                    PersistentVolumeClaim  kapp     -       ok  -   3m
      ^           alertmanager                                    Secret                 kapp     -       ok  -   3m
      ^           alertmanager                                    Service                kapp     -       ok  -   3m
      ^           alertmanager-548f4ccd66                         ReplicaSet             cluster  -       ok  -   3m
      ^           alertmanager-548f4ccd66-lkhmk                   Pod                    cluster  4/4 t   ok  -   3m
      ^           alertmanager-qlpw9                              EndpointSlice          cluster  -       ok  -   3m
      ^           alertmanager-sa                                 ServiceAccount         kapp     -       ok  -   3m
      ^           prometheus-cadvisor                             DaemonSet              kapp     -       ok  -   3m
      ^           prometheus-cadvisor                             ServiceAccount         kapp     -       ok  -   3m
      ^           prometheus-cadvisor-6bdf8755f                   ControllerRevision     cluster  -       ok  -   3m
      ^           prometheus-cadvisor-s5k5r                       Pod                    cluster  4/4 t   ok  -   3m
      ^           prometheus-kube-state-metrics                   Deployment             kapp     2/2 t   ok  -   3m
      ^           prometheus-kube-state-metrics                   Endpoints              cluster  -       ok  -   3m
      ^           prometheus-kube-state-metrics                   Service                kapp     -       ok  -   3m
      ^           prometheus-kube-state-metrics                   ServiceAccount         kapp     -       ok  -   3m
      ^           prometheus-kube-state-metrics-6b8dd89dcb        ReplicaSet             cluster  -       ok  -   3m
      ^           prometheus-kube-state-metrics-6b8dd89dcb-cdrlf  Pod                    cluster  4/4 t   ok  -   3m
      ^           prometheus-kube-state-metrics-vtspg             EndpointSlice          cluster  -       ok  -   3m
      ^           prometheus-node-exporter                        DaemonSet              kapp     -       ok  -   3m
      ^           prometheus-node-exporter                        Endpoints              cluster  -       ok  -   3m
      ^           prometheus-node-exporter                        Service                kapp     -       ok  -   3m
      ^           prometheus-node-exporter-74f6d656b6             ControllerRevision     cluster  -       ok  -   3m
      ^           prometheus-node-exporter-crc4f                  Pod                    cluster  4/4 t   ok  -   3m
      ^           prometheus-node-exporter-k7c88                  EndpointSlice          cluster  -       ok  -   3m
      ^           prometheus-node-exporter-sa                     ServiceAccount         kapp     -       ok  -   3m
      ^           prometheus-node-exporter-snkqc                  Pod                    cluster  4/4 t   ok  -   3m
      ^           prometheus-pushgateway                          Deployment             kapp     2/2 t   ok  -   3m
      ^           prometheus-pushgateway                          Endpoints              cluster  -       ok  -   3m
      ^           prometheus-pushgateway                          Service                kapp     -       ok  -   3m
      ^           prometheus-pushgateway                          ServiceAccount         kapp     -       ok  -   3m
      ^           prometheus-pushgateway-65bb85d997               ReplicaSet             cluster  -       ok  -   3m
      ^           prometheus-pushgateway-65bb85d997-dz462         Pod                    cluster  4/4 t   ok  -   3m
      ^           prometheus-pushgateway-7wph9                    EndpointSlice          cluster  -       ok  -   3m
      ^           prometheus-server                               ConfigMap              kapp     -       ok  -   3m
      ^           prometheus-server                               Deployment             kapp     2/2 t   ok  -   3m
      ^           prometheus-server                               Endpoints              cluster  -       ok  -   3m
      ^           prometheus-server                               PersistentVolumeClaim  kapp     -       ok  -   3m
      ^           prometheus-server                               Service                kapp     -       ok  -   3m
      ^           prometheus-server-5dffddf9f8                    ReplicaSet             cluster  -       ok  -   3m
      ^           prometheus-server-5dffddf9f8-qjldb              Pod                    cluster  4/4 t   ok  -   3m
      ^           prometheus-server-sa                            ServiceAccount         kapp     -       ok  -   3m
      ^           prometheus-server-xlrbc                         EndpointSlice          cluster  -       ok  -   3m
      Rs: Reconcile state
      Ri: Reconcile information
      58 resources
      Succeeded
    updatedAt: "2021-12-18T14:58:40Z"
  observedGeneration: 1
  template:
    exitCode: 0
    stderr: |
      resolve | final: gcr.io/cadvisor/cadvisor:v0.39.1 -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:b4cd4cc0ef05630f70d621420ad1316f631f35cef21edb7a62fff7bd787bbfd3
      resolve | final: jimmidyson/configmap-reload:v0.5.0 -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:904d08e9f701d3d8178cb61651dbe8edc5d08dd5895b56bdcac9e5805ea82b52
      resolve | final: prom/alertmanager:v0.22.2 -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:624c1a5063c7c80635081a504c3e1b020d89809651978eb5d0b652a394f3022d
      resolve | final: prom/prometheus:v2.27.0 -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:d1a9a86b9a3e60a9ea3cde141bdc936847456acc497e0affe7e288234383efa5
      resolve | final: prom/pushgateway:v1.4.0 -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:ca32c7864bb2573bf27ff6628a03d17b37b1aa3dc367b5d86831e6c0f0761376
      resolve | final: quay.io/coreos/kube-state-metrics:v1.9.8 -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:ace842fc85031688d06c4aa000b5b1e58ba3b9dd13d26e7c8f2547f7ee0bcc84
      resolve | final: quay.io/prometheus/node-exporter:v1.1.2 -> harbor-10-213-232-22.sslip.io/library/tce/main@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810
    updatedAt: "2021-12-18T14:58:37Z"

spec.fetch[0].imgpkgBundle.imagestatus.template.stderrを見るとイメージがHarborにrelocateされていることが分かります。

imgpkgでPackageRepositoryをrelocateしただけで、全てのPackageとイメージが一括でrelocateできるので、とても便利です。