--- title: Tanzu Community Editionのk8sクラスタにPrometheus, Grafana Packageをインストールするメモ tags: ["Kubernetes", "vSphere", "TKG", "Tanzu", "Cluster API", "Prometheus", "Grafana", "TCE", "Docker"] categories: ["Dev", "CaaS", "Kubernetes", "TKG", "TCE", "Docker"] date: 2021-10-17T16:53:12Z updated: 2021-10-17T16:58:33Z --- "[Tanzu Community EditionをDocker上にインストールするメモ](/entries/670)" で構築したクラスタにPrometheus、Grafana Packageをインストールします。 依存PackageとしてCert ManagerとContourもインストールします。 **目次** ### Cert Manager Packageのインストール https://tanzucommunityedition.io/docs/latest/package-readme-cert-manager-1.5.3/ の通り。 次のコマンドでCert Managerをインストールします。 ``` tanzu package install cert-manager --package-name cert-manager.community.tanzu.vmware.com --version 1.5.3 --namespace tce-package-install ``` このPackageでインストールされるリソースは次のコマンドで確認できます。 ``` $ kubectl get app -n tce-package-install cert-manager -oyaml apiVersion: kappctrl.k14s.io/v1alpha1 kind: App metadata: creationTimestamp: "2021-10-17T14:07:48Z" finalizers: - finalizers.kapp-ctrl.k14s.io/delete generation: 1 name: cert-manager namespace: tce-package-install ownerReferences: - apiVersion: packaging.carvel.dev/v1alpha1 blockOwnerDeletion: true controller: true kind: PackageInstall name: cert-manager uid: c4e9e61f-6bff-4685-95fe-e6d35a5a2908 resourceVersion: "62892" uid: a8d083dd-e56a-4186-b677-0677591659fd spec: deploy: - kapp: {} fetch: - imgpkgBundle: image: projects.registry.vmware.com/tce/cert-manager@sha256:fcd2ea27f3d2c86ef56da7bb92c231537b12824d7c1bc3d7e5cea7d2aed4a9b9 serviceAccountName: cert-manager-tce-package-install-sa template: - ytt: paths: - config/ - kbld: paths: - '-' - .imgpkg/images.yml status: conditions: - status: "True" type: ReconcileSucceeded consecutiveReconcileSuccesses: 2 deploy: exitCode: 0 finished: true startedAt: "2021-10-17T14:08:49Z" stdout: |- Target cluster 'https://100.64.0.1:443' (nodes: ikra-control-plane-xpnf2, 3+) 02:08:50PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"networkpolicystats"} 02:08:50PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreanetworkpolicystats"} 02:08:50PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreaclusternetworkpolicystats"} Changes Namespace Name Kind Conds. Age Op Op st. Wait to Rs Ri Op: 0 create, 0 delete, 0 update, 0 noop Wait to: 0 reconcile, 0 delete, 0 noop Succeeded updatedAt: "2021-10-17T14:08:54Z" fetch: exitCode: 0 startedAt: "2021-10-17T14:08:44Z" stdout: | apiVersion: vendir.k14s.io/v1alpha1 directories: - contents: - imgpkgBundle: image: projects.registry.vmware.com/tce/cert-manager@sha256:fcd2ea27f3d2c86ef56da7bb92c231537b12824d7c1bc3d7e5cea7d2aed4a9b9 path: . path: "0" kind: LockConfig updatedAt: "2021-10-17T14:08:49Z" friendlyDescription: Reconcile succeeded inspect: exitCode: 0 stdout: |- Target cluster 'https://100.64.0.1:443' (nodes: ikra-control-plane-xpnf2, 3+) 02:08:54PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreaclusternetworkpolicystats"} 02:08:54PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreanetworkpolicystats"} 02:08:54PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"networkpolicystats"} Resources in app 'cert-manager-ctrl' Namespace Name Kind Owner Conds. Rs Ri Age (cluster) cert-manager Namespace kapp - ok - 59s ^ cert-manager-cainjector ClusterRole kapp - ok - 59s ^ cert-manager-cainjector ClusterRoleBinding kapp - ok - 59s ^ cert-manager-controller-approve:cert-manager-io ClusterRole kapp - ok - 59s ^ cert-manager-controller-approve:cert-manager-io ClusterRoleBinding kapp - ok - 59s ^ cert-manager-controller-certificates ClusterRole kapp - ok - 59s ^ cert-manager-controller-certificates ClusterRoleBinding kapp - ok - 59s ^ cert-manager-controller-certificatesigningrequests ClusterRole kapp - ok - 59s ^ cert-manager-controller-certificatesigningrequests ClusterRoleBinding kapp - ok - 59s ^ cert-manager-controller-challenges ClusterRole kapp - ok - 59s ^ cert-manager-controller-challenges ClusterRoleBinding kapp - ok - 59s ^ cert-manager-controller-clusterissuers ClusterRole kapp - ok - 59s ^ cert-manager-controller-clusterissuers ClusterRoleBinding kapp - ok - 59s ^ cert-manager-controller-ingress-shim ClusterRole kapp - ok - 58s ^ cert-manager-controller-ingress-shim ClusterRoleBinding kapp - ok - 59s ^ cert-manager-controller-issuers ClusterRole kapp - ok - 59s ^ cert-manager-controller-issuers ClusterRoleBinding kapp - ok - 59s ^ cert-manager-controller-orders ClusterRole kapp - ok - 59s ^ cert-manager-controller-orders ClusterRoleBinding kapp - ok - 59s ^ cert-manager-edit ClusterRole kapp - ok - 59s ^ cert-manager-view ClusterRole kapp - ok - 59s ^ cert-manager-webhook MutatingWebhookConfiguration kapp - ok - 59s ^ cert-manager-webhook ValidatingWebhookConfiguration kapp - ok - 59s ^ cert-manager-webhook:subjectaccessreviews ClusterRole kapp - ok - 59s ^ cert-manager-webhook:subjectaccessreviews ClusterRoleBinding kapp - ok - 59s ^ certificaterequests.cert-manager.io CustomResourceDefinition kapp 2/2 t ok - 59s ^ certificates.cert-manager.io CustomResourceDefinition kapp 2/2 t ok - 58s ^ challenges.acme.cert-manager.io CustomResourceDefinition kapp 2/2 t ok - 59s ^ clusterissuers.cert-manager.io CustomResourceDefinition kapp 2/2 t ok - 59s ^ issuers.cert-manager.io CustomResourceDefinition kapp 2/2 t ok - 59s ^ orders.acme.cert-manager.io CustomResourceDefinition kapp 2/2 t ok - 59s cert-manager cert-manager Deployment kapp 2/2 t ok - 56s ^ cert-manager Endpoints cluster - ok - 56s ^ cert-manager Service kapp - ok - 56s ^ cert-manager ServiceAccount kapp - ok - 56s ^ cert-manager-78679d6bbf ReplicaSet cluster - ok - 56s ^ cert-manager-78679d6bbf-7r8q8 Pod cluster 4/4 t ok - 56s ^ cert-manager-cainjector Deployment kapp 2/2 t ok - 56s ^ cert-manager-cainjector ServiceAccount kapp - ok - 56s ^ cert-manager-cainjector-6457db75d4 ReplicaSet cluster - ok - 56s ^ cert-manager-cainjector-6457db75d4-n4n6r Pod cluster 4/4 t ok - 56s ^ cert-manager-cdx5h EndpointSlice cluster - ok - 56s ^ cert-manager-webhook Deployment kapp 2/2 t ok - 56s ^ cert-manager-webhook Endpoints cluster - ok - 56s ^ cert-manager-webhook Service kapp - ok - 56s ^ cert-manager-webhook ServiceAccount kapp - ok - 56s ^ cert-manager-webhook-5t26v EndpointSlice cluster - ok - 56s ^ cert-manager-webhook-7db48df757 ReplicaSet cluster - ok - 56s ^ cert-manager-webhook-7db48df757-45dq5 Pod cluster 4/4 t ok - 56s ^ cert-manager-webhook:dynamic-serving Role kapp - ok - 56s ^ cert-manager-webhook:dynamic-serving RoleBinding kapp - ok - 56s kube-system cert-manager-cainjector:leaderelection Role kapp - ok - 56s ^ cert-manager-cainjector:leaderelection RoleBinding kapp - ok - 56s ^ cert-manager:leaderelection Role kapp - ok - 56s ^ cert-manager:leaderelection RoleBinding kapp - ok - 56s Rs: Reconcile state Ri: Reconcile information 55 resources Succeeded updatedAt: "2021-10-17T14:08:54Z" observedGeneration: 1 template: exitCode: 0 stderr: | resolve | final: quay.io/jetstack/cert-manager-cainjector:v1.5.3 -> quay.io/jetstack/cert-manager-cainjector@sha256:de02e3f445cfe7c035f2a9939b948c4d043011713389d9437311a62740f20bef resolve | final: quay.io/jetstack/cert-manager-controller:v1.5.3 -> quay.io/jetstack/cert-manager-controller@sha256:7b039d469ed739a652f3bb8a1ddc122942b66cceeb85bac315449724ee64287f resolve | final: quay.io/jetstack/cert-manager-webhook:v1.5.3 -> quay.io/jetstack/cert-manager-webhook@sha256:ed6354190d259524d32ae74471f93bf46bfdcf4df6f73629eedf576cd87e10b8 updatedAt: "2021-10-17T14:08:49Z" ``` ### Contour Packageのインストール https://tanzucommunityedition.io/docs/latest/package-readme-contour-1.18.1/ の通り。 次のコマンドでContourをインストールします。 ``` tanzu package install contour --package-name contour.community.tanzu.vmware.com --version 1.18.1 --namespace tce-package-install ``` このPackageでインストールされるリソースは次のコマンドで確認できます。 ``` $ kubectl get app -n tce-package-install contour -oyaml apiVersion: kappctrl.k14s.io/v1alpha1 kind: App metadata: creationTimestamp: "2021-10-17T13:57:30Z" finalizers: - finalizers.kapp-ctrl.k14s.io/delete generation: 1 name: contour namespace: tce-package-install ownerReferences: - apiVersion: packaging.carvel.dev/v1alpha1 blockOwnerDeletion: true controller: true kind: PackageInstall name: contour uid: c0274ef9-f0b7-4362-bb2b-8f54c9403199 resourceVersion: "60970" uid: 60d5c45d-2f58-4434-a47b-099a7c65f218 spec: deploy: - kapp: {} fetch: - imgpkgBundle: image: projects.registry.vmware.com/tce/contour@sha256:7f031ada007ab3ba53a8a71b55a7f2123343a7f180263de13ac72d7de97a16b0 serviceAccountName: contour-tce-package-install-sa template: - ytt: paths: - config/ - kbld: paths: - '-' - .imgpkg/images.yml status: conditions: - status: "True" type: ReconcileSucceeded consecutiveReconcileSuccesses: 2 deploy: exitCode: 0 finished: true startedAt: "2021-10-17T13:58:47Z" stdout: |- Target cluster 'https://100.64.0.1:443' (nodes: ikra-control-plane-xpnf2, 3+) 01:58:47PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"networkpolicystats"} 01:58:47PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreaclusternetworkpolicystats"} 01:58:47PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreanetworkpolicystats"} Changes Namespace Name Kind Conds. Age Op Op st. Wait to Rs Ri Op: 0 create, 0 delete, 0 update, 0 noop Wait to: 0 reconcile, 0 delete, 0 noop Succeeded updatedAt: "2021-10-17T13:58:48Z" fetch: exitCode: 0 startedAt: "2021-10-17T13:58:41Z" stdout: | apiVersion: vendir.k14s.io/v1alpha1 directories: - contents: - imgpkgBundle: image: projects.registry.vmware.com/tce/contour@sha256:7f031ada007ab3ba53a8a71b55a7f2123343a7f180263de13ac72d7de97a16b0 path: . path: "0" kind: LockConfig updatedAt: "2021-10-17T13:58:46Z" friendlyDescription: Reconcile succeeded inspect: exitCode: 0 stdout: |- Target cluster 'https://100.64.0.1:443' (nodes: ikra-control-plane-xpnf2, 3+) 01:58:48PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreaclusternetworkpolicystats"} 01:58:48PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"networkpolicystats"} 01:58:48PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreanetworkpolicystats"} Resources in app 'contour-ctrl' Namespace Name Kind Owner Conds. Rs Ri Age (cluster) contour ClusterRole kapp - ok - 1m ^ contour ClusterRoleBinding kapp - ok - 1m ^ extensionservices.projectcontour.io CustomResourceDefinition kapp 2/2 t ok - 1m ^ httpproxies.projectcontour.io CustomResourceDefinition kapp 2/2 t ok - 1m ^ projectcontour Namespace kapp - ok - 1m ^ tlscertificatedelegations.projectcontour.io CustomResourceDefinition kapp 2/2 t ok - 1m projectcontour contour ConfigMap kapp - ok - 1m ^ contour Deployment kapp 2/2 t ok - 1m ^ contour Endpoints cluster - ok - 1m ^ contour RoleBinding kapp - ok - 1m ^ contour Service kapp - ok - 1m ^ contour ServiceAccount kapp - ok - 1m ^ contour-55d794488b ReplicaSet cluster - ok - 1m ^ contour-55d794488b-bhmwt Pod cluster 4/4 t ok - 1m ^ contour-55d794488b-hcx4z Pod cluster 4/4 t ok - 1m ^ contour-certgen Role kapp - ok - 1m ^ contour-certgen ServiceAccount kapp - ok - 1m ^ contour-certgen-v1.18.1 Job kapp 1/1 t ok Completed 1m ^ contour-certgen-v1.18.1-sf88k Pod cluster 2/4 t ok - 1m ^ contour-l9x6m EndpointSlice cluster - ok - 1m ^ envoy DaemonSet kapp - ok - 1m ^ envoy Endpoints cluster - ok - 1m ^ envoy Service kapp - ok - 1m ^ envoy ServiceAccount kapp - ok - 1m ^ envoy-66kvm Pod cluster 4/4 t ok - 1m ^ envoy-7dbbdbf8c4 ControllerRevision cluster - ok - 1m ^ envoy-8t2h9 Pod cluster 4/4 t ok - 1m ^ envoy-nr4fl Pod cluster 4/4 t ok - 1m ^ envoy-vt994 EndpointSlice cluster - ok - 1m Rs: Reconcile state Ri: Reconcile information 29 resources Succeeded updatedAt: "2021-10-17T13:58:48Z" observedGeneration: 1 template: exitCode: 0 stderr: | resolve | final: docker.io/envoyproxy/envoy:v1.19.1 -> index.docker.io/envoyproxy/envoy@sha256:ac6a29af5bee160a1b4425d7c7a41a4d8a08a7f9dd7f225f21b5375f6439457a resolve | final: docker.io/projectcontour/contour:v1.18.1 -> index.docker.io/projectcontour/contour@sha256:31c376f80f5b80f2ac5558c1c50711438796b22b742285789ea21cd2bba244e2 updatedAt: "2021-10-17T13:58:47Z" ``` 次のコマンドでEnvoyのExternal IPを確認します。 ``` $ kubectl get svc -n projectcontour NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE contour ClusterIP 100.66.102.186 8001/TCP 16m envoy LoadBalancer 100.64.149.8 172.18.0.200 80:30376/TCP,443:30902/TCP 16m ``` ### Prometheus Packageのインストール https://tanzucommunityedition.io/docs/latest/package-readme-prometheus-2.27.0/ の通り。 次のコマンドでPrometheusをインストールします。デフォルトで無効になっているIngress(ContourのHTTPProxy)を有効にするために設定も追加します。 ``` cat < prometheus-values.yaml ingress: enabled: true EOF tanzu package install prometheus --package-name prometheus.community.tanzu.vmware.com --version 2.27.0 --namespace tce-package-install --values-file prometheus-values.yaml ``` なお、設定可能な値一覧は次のコマンドで確認できます。 ``` tanzu package available get prometheus.community.tanzu.vmware.com/2.27.0 --values-schema ``` このPackageでインストールされるリソースは次のコマンドで確認できます。 ``` $ kubectl get app -n tce-package-install prometheus -oyaml apiVersion: kappctrl.k14s.io/v1alpha1 kind: App metadata: creationTimestamp: "2021-10-17T14:30:27Z" finalizers: - finalizers.kapp-ctrl.k14s.io/delete generation: 1 name: prometheus namespace: tce-package-install ownerReferences: - apiVersion: packaging.carvel.dev/v1alpha1 blockOwnerDeletion: true controller: true kind: PackageInstall name: prometheus uid: 21ac07d2-a848-4524-8066-d6fb56e4967b resourceVersion: "69850" uid: 07c3d9ad-365f-4645-b9ff-ef2803dc3d2b spec: deploy: - kapp: {} fetch: - imgpkgBundle: image: projects.registry.vmware.com/tce/prometheus@sha256:b6d81e04a8ba1d6b5a33fb3f781f8238a26fa9a2add1df301b9e189d58f6682b serviceAccountName: prometheus-tce-package-install-sa template: - ytt: paths: - config/ valuesFrom: - secretRef: name: prometheus-tce-package-install-values - kbld: paths: - '-' - .imgpkg/images.yml status: conditions: - status: "True" type: ReconcileSucceeded consecutiveReconcileSuccesses: 5 deploy: exitCode: 0 finished: true startedAt: "2021-10-17T14:34:06Z" stdout: |- Target cluster 'https://100.64.0.1:443' (nodes: ikra-control-plane-xpnf2, 3+) 02:34:07PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"networkpolicystats"} 02:34:07PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreaclusternetworkpolicystats"} 02:34:07PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreanetworkpolicystats"} Changes Namespace Name Kind Conds. Age Op Op st. Wait to Rs Ri Op: 0 create, 0 delete, 0 update, 0 noop Wait to: 0 reconcile, 0 delete, 0 noop Succeeded updatedAt: "2021-10-17T14:34:08Z" fetch: exitCode: 0 startedAt: "2021-10-17T14:34:01Z" stdout: | apiVersion: vendir.k14s.io/v1alpha1 directories: - contents: - imgpkgBundle: image: projects.registry.vmware.com/tce/prometheus@sha256:b6d81e04a8ba1d6b5a33fb3f781f8238a26fa9a2add1df301b9e189d58f6682b path: . path: "0" kind: LockConfig updatedAt: "2021-10-17T14:34:06Z" friendlyDescription: Reconcile succeeded inspect: exitCode: 0 stdout: |- Target cluster 'https://100.64.0.1:443' (nodes: ikra-control-plane-xpnf2, 3+) 02:34:08PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"networkpolicystats"} 02:34:08PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreanetworkpolicystats"} 02:34:08PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreaclusternetworkpolicystats"} Resources in app 'prometheus-ctrl' Namespace Name Kind Owner Conds. Rs Ri Age (cluster) alertmanager ClusterRole kapp - ok - 3m ^ alertmanager ClusterRoleBinding kapp - ok - 3m ^ prometheus Namespace kapp - ok - 3m ^ prometheus-cadvisor ClusterRole kapp - ok - 3m ^ prometheus-cadvisor ClusterRoleBinding kapp - ok - 3m ^ prometheus-kube-state-metrics ClusterRole kapp - ok - 3m ^ prometheus-kube-state-metrics ClusterRoleBinding kapp - ok - 3m ^ prometheus-node-exporter ClusterRole kapp - ok - 3m ^ prometheus-node-exporter ClusterRoleBinding kapp - ok - 3m ^ prometheus-node-exporter PodSecurityPolicy kapp - ok - 3m ^ prometheus-pushgateway ClusterRole kapp - ok - 3m ^ prometheus-pushgateway ClusterRoleBinding kapp - ok - 3m ^ prometheus-server ClusterRole kapp - ok - 3m ^ prometheus-server ClusterRoleBinding kapp - ok - 3m prometheus alertmanager Deployment kapp 2/2 t ok - 3m ^ alertmanager Endpoints cluster - ok - 3m ^ alertmanager PersistentVolumeClaim kapp - ok - 3m ^ alertmanager Secret kapp - ok - 3m ^ alertmanager Service kapp - ok - 3m ^ alertmanager-64df5d576c ReplicaSet cluster - ok - 3m ^ alertmanager-64df5d576c-8kncz Pod cluster 4/4 t ok - 3m ^ alertmanager-d4x6s EndpointSlice cluster - ok - 3m ^ alertmanager-sa ServiceAccount kapp - ok - 3m ^ prometheus-ca Certificate kapp 1/1 t ok - 3m ^ prometheus-ca-8v8b8 CertificateRequest cluster 2/2 t ok - 3m ^ prometheus-ca-issuer Issuer kapp 1/1 t ok - 3m ^ prometheus-cadvisor DaemonSet kapp - ok - 3m ^ prometheus-cadvisor ServiceAccount kapp - ok - 3m ^ prometheus-cadvisor-5b8cdc7f65 ControllerRevision cluster - ok - 3m ^ prometheus-cadvisor-mbmbn Pod cluster 4/4 t ok - 3m ^ prometheus-cadvisor-vkpxs Pod cluster 4/4 t ok - 3m ^ prometheus-cadvisor-xn872 Pod cluster 4/4 t ok - 3m ^ prometheus-httpproxy HTTPProxy kapp 1/1 t ok - 3m ^ prometheus-kube-state-metrics Deployment kapp 2/2 t ok - 3m ^ prometheus-kube-state-metrics Endpoints cluster - ok - 3m ^ prometheus-kube-state-metrics Service kapp - ok - 3m ^ prometheus-kube-state-metrics ServiceAccount kapp - ok - 3m ^ prometheus-kube-state-metrics-b4677f5dd ReplicaSet cluster - ok - 3m ^ prometheus-kube-state-metrics-b4677f5dd-hgrtx Pod cluster 4/4 t ok - 3m ^ prometheus-kube-state-metrics-zr8sc EndpointSlice cluster - ok - 3m ^ prometheus-node-exporter DaemonSet kapp - ok - 3m ^ prometheus-node-exporter Endpoints cluster - ok - 3m ^ prometheus-node-exporter Service kapp - ok - 3m ^ prometheus-node-exporter-5947fcbf4b ControllerRevision cluster - ok - 3m ^ prometheus-node-exporter-5xsjs EndpointSlice cluster - ok - 3m ^ prometheus-node-exporter-f8kbl Pod cluster 4/4 t ok - 3m ^ prometheus-node-exporter-l45rn Pod cluster 4/4 t ok - 3m ^ prometheus-node-exporter-mp52s Pod cluster 4/4 t ok - 3m ^ prometheus-node-exporter-rmsh8 Pod cluster 4/4 t ok - 3m ^ prometheus-node-exporter-sa ServiceAccount kapp - ok - 3m ^ prometheus-pushgateway Deployment kapp 2/2 t ok - 3m ^ prometheus-pushgateway Endpoints cluster - ok - 3m ^ prometheus-pushgateway Service kapp - ok - 3m ^ prometheus-pushgateway ServiceAccount kapp - ok - 3m ^ prometheus-pushgateway-67646d8cfc ReplicaSet cluster - ok - 3m ^ prometheus-pushgateway-67646d8cfc-n4wjf Pod cluster 4/4 t ok - 3m ^ prometheus-pushgateway-6l5lj EndpointSlice cluster - ok - 3m ^ prometheus-self-signed-ca-issuer Issuer kapp 1/1 t ok - 3m ^ prometheus-server ConfigMap kapp - ok - 3m ^ prometheus-server Deployment kapp 2/2 t ok - 3m ^ prometheus-server Endpoints cluster - ok - 3m ^ prometheus-server PersistentVolumeClaim kapp - ok - 3m ^ prometheus-server Service kapp - ok - 3m ^ prometheus-server-5fd6f6d679 ReplicaSet cluster - ok - 3m ^ prometheus-server-5fd6f6d679-m5l5l Pod cluster 4/4 t ok - 3m ^ prometheus-server-9ps7c EndpointSlice cluster - ok - 3m ^ prometheus-server-sa ServiceAccount kapp - ok - 3m ^ prometheus-tls-cert Certificate kapp 1/1 t ok - 3m ^ prometheus-tls-cert-nrj4q CertificateRequest cluster 2/2 t ok - 3m Rs: Reconcile state Ri: Reconcile information 69 resources Succeeded updatedAt: "2021-10-17T14:34:08Z" observedGeneration: 1 template: exitCode: 0 stderr: | resolve | final: gcr.io/cadvisor/cadvisor:v0.39.1 -> projects.registry.vmware.com/tkg/prometheus/cadvisor@sha256:b4cd4cc0ef05630f70d621420ad1316f631f35cef21edb7a62fff7bd787bbfd3 resolve | final: jimmidyson/configmap-reload:v0.5.0 -> index.docker.io/jimmidyson/configmap-reload@sha256:904d08e9f701d3d8178cb61651dbe8edc5d08dd5895b56bdcac9e5805ea82b52 resolve | final: prom/alertmanager:v0.22.2 -> index.docker.io/prom/alertmanager@sha256:624c1a5063c7c80635081a504c3e1b020d89809651978eb5d0b652a394f3022d resolve | final: prom/prometheus:v2.27.0 -> index.docker.io/prom/prometheus@sha256:d1a9a86b9a3e60a9ea3cde141bdc936847456acc497e0affe7e288234383efa5 resolve | final: prom/pushgateway:v1.4.0 -> index.docker.io/prom/pushgateway@sha256:ca32c7864bb2573bf27ff6628a03d17b37b1aa3dc367b5d86831e6c0f0761376 resolve | final: quay.io/coreos/kube-state-metrics:v1.9.8 -> quay.io/coreos/kube-state-metrics@sha256:ace842fc85031688d06c4aa000b5b1e58ba3b9dd13d26e7c8f2547f7ee0bcc84 resolve | final: quay.io/prometheus/node-exporter:v1.1.2 -> quay.io/prometheus/node-exporter@sha256:22fbde17ab647ddf89841e5e464464eece111402b7d599882c2a3393bc0d2810 updatedAt: "2021-10-17T14:34:06Z" ``` PodのリストとHTTPProxyリソースを確認します。 ``` $ kubectl get pod,httpproxy -n prometheus -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod/alertmanager-64df5d576c-8kncz 1/1 Running 0 4m43s 100.96.3.19 ikra-md-0-6b5984c77d-sht5z pod/prometheus-cadvisor-mbmbn 1/1 Running 0 4m46s 100.96.3.15 ikra-md-0-6b5984c77d-sht5z pod/prometheus-cadvisor-vkpxs 1/1 Running 0 4m46s 100.96.1.27 ikra-md-0-6b5984c77d-n7d8f pod/prometheus-cadvisor-xn872 1/1 Running 0 4m46s 100.96.2.15 ikra-md-0-6b5984c77d-8scgf pod/prometheus-kube-state-metrics-b4677f5dd-hgrtx 1/1 Running 0 4m44s 100.96.2.16 ikra-md-0-6b5984c77d-8scgf pod/prometheus-node-exporter-f8kbl 1/1 Running 0 4m43s 100.96.2.18 ikra-md-0-6b5984c77d-8scgf pod/prometheus-node-exporter-l45rn 1/1 Running 0 4m43s 100.96.3.18 ikra-md-0-6b5984c77d-sht5z pod/prometheus-node-exporter-mp52s 1/1 Running 0 4m43s 100.96.0.3 ikra-control-plane-xpnf2 pod/prometheus-node-exporter-rmsh8 1/1 Running 0 4m43s 100.96.1.28 ikra-md-0-6b5984c77d-n7d8f pod/prometheus-pushgateway-67646d8cfc-n4wjf 1/1 Running 0 4m43s 100.96.3.16 ikra-md-0-6b5984c77d-sht5z pod/prometheus-server-5fd6f6d679-m5l5l 2/2 Running 0 4m44s 100.96.2.19 ikra-md-0-6b5984c77d-8scgf NAME FQDN TLS SECRET STATUS STATUS DESCRIPTION httpproxy.projectcontour.io/prometheus-httpproxy prometheus.system.tanzu prometheus-tls valid Valid HTTPProxy ``` デフォルトでは`prometheus.system.tanzu`がPrometheusに対するFQDNです。`/etc/hosts`に次のレコードを追加します。 ``` ENVOY_IP=$(kubectl get svc -n projectcontour envoy -ojsonpath='{.status.loadBalancer.ingress[0].ip}') PROMETHEUS_FQDN=$(kubectl get httpproxy -n prometheus prometheus-httpproxy -ojsonpath='{.spec.virtualhost.fqdn}') cat < grafana-values.yaml grafana: service: type: ClusterIP EOF tanzu package install grafana --package-name grafana.community.tanzu.vmware.com --version 7.5.7 --namespace tce-package-install --values-file prometheus-values.yaml ``` 設定可能な値一覧は次のコマンドで確認できます。 ``` tanzu package available get grafana.community.tanzu.vmware.com/7.5.7 --values-schema ``` このPackageでインストールされるリソースは次のコマンドで確認できます。 ``` $ kubectl get app -n tce-package-install grafana -oyaml apiVersion: kappctrl.k14s.io/v1alpha1 kind: App metadata: creationTimestamp: "2021-10-17T16:15:15Z" finalizers: - finalizers.kapp-ctrl.k14s.io/delete generation: 1 name: grafana namespace: tce-package-install ownerReferences: - apiVersion: packaging.carvel.dev/v1alpha1 blockOwnerDeletion: true controller: true kind: PackageInstall name: grafana uid: 5073b30c-4271-436a-886a-2e9ce31c41de resourceVersion: "96522" uid: f387057f-3036-4c4c-bc61-afe20cab93fb spec: deploy: - kapp: {} fetch: - imgpkgBundle: image: projects.registry.vmware.com/tce/grafana@sha256:53d41d9ac1534fc381efa5bb181aa4cac1ec26fc77c7ffadb34550930112e193 serviceAccountName: grafana-tce-package-install-sa template: - ytt: paths: - config/ valuesFrom: - secretRef: name: grafana-tce-package-install-values - kbld: paths: - '-' - .imgpkg/images.yml status: conditions: - status: "True" type: ReconcileSucceeded consecutiveReconcileSuccesses: 19 deploy: exitCode: 0 finished: true startedAt: "2021-10-17T16:29:23Z" stdout: |- Target cluster 'https://100.64.0.1:443' (nodes: ikra-control-plane-xpnf2, 3+) 04:29:23PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"networkpolicystats"} 04:29:23PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreaclusternetworkpolicystats"} 04:29:23PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreanetworkpolicystats"} Changes Namespace Name Kind Conds. Age Op Op st. Wait to Rs Ri Op: 0 create, 0 delete, 0 update, 0 noop Wait to: 0 reconcile, 0 delete, 0 noop Succeeded updatedAt: "2021-10-17T16:29:24Z" fetch: exitCode: 0 startedAt: "2021-10-17T16:29:17Z" stdout: | apiVersion: vendir.k14s.io/v1alpha1 directories: - contents: - imgpkgBundle: image: projects.registry.vmware.com/tce/grafana@sha256:53d41d9ac1534fc381efa5bb181aa4cac1ec26fc77c7ffadb34550930112e193 path: . path: "0" kind: LockConfig updatedAt: "2021-10-17T16:29:23Z" friendlyDescription: Reconcile succeeded inspect: exitCode: 0 stdout: |- Target cluster 'https://100.64.0.1:443' (nodes: ikra-control-plane-xpnf2, 3+) 04:29:24PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreaclusternetworkpolicystats"} 04:29:24PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"antreanetworkpolicystats"} 04:29:24PM: info: Resources: Ignoring group version: schema.GroupVersionResource{Group:"stats.antrea.tanzu.vmware.com", Version:"v1alpha1", Resource:"networkpolicystats"} Resources in app 'grafana-ctrl' Namespace Name Kind Owner Conds. Rs Ri Age (cluster) grafana Namespace kapp - ok - 14m ^ grafana-clusterrole ClusterRole kapp - ok - 14m ^ grafana-clusterrolebinding ClusterRoleBinding kapp - ok - 14m grafana grafana ConfigMap kapp - ok - 14m ^ grafana Deployment kapp 2/2 t ok - 13m ^ grafana Endpoints cluster - ok - 14m ^ grafana Secret kapp - ok - 14m ^ grafana Service kapp - ok - 14m ^ grafana-86556f498b ReplicaSet cluster - ok - 13m ^ grafana-86556f498b-db25q Pod cluster 4/4 t ok - 13m ^ grafana-ca Certificate kapp 1/1 t ok - 14m ^ grafana-ca-issuer Issuer kapp 1/1 t ok - 14m ^ grafana-ca-zvt7g CertificateRequest cluster 2/2 t ok - 13m ^ grafana-dashboard ConfigMap kapp - ok - 14m ^ grafana-dashboard-apiserver ConfigMap kapp - ok - 14m ^ grafana-dashboard-default ConfigMap kapp - ok - 14m ^ grafana-datasource ConfigMap kapp - ok - 14m ^ grafana-httpproxy HTTPProxy kapp 1/1 t ok - 14m ^ grafana-pvc PersistentVolumeClaim kapp - ok - 14m ^ grafana-sa ServiceAccount kapp - ok - 14m ^ grafana-self-signed-ca-issuer Issuer kapp 1/1 t ok - 13m ^ grafana-tls-cert Certificate kapp 1/1 t ok - 14m ^ grafana-tls-cert-sz4cn CertificateRequest cluster 2/2 t ok - 13m ^ grafana-wbqmh EndpointSlice cluster - ok - 14m Rs: Reconcile state Ri: Reconcile information 24 resources Succeeded updatedAt: "2021-10-17T16:29:24Z" observedGeneration: 1 template: exitCode: 0 stderr: | resolve | final: grafana/grafana:7.5.7 -> projects.registry.vmware.com/tkg/grafana/grafana@sha256:df8f25cc9ee43d6ea4c22f9c6c46644e2b9a485562dd0dafe831b5b582ac0a71 resolve | final: kiwigrid/k8s-sidecar:1.12.1 -> projects.registry.vmware.com/tkg/grafana/k8s-sidecar@sha256:9f1ad1e5e404bc43f9591b1189c187f535d6f61769468c49b4fc97add803d7b9 updatedAt: "2021-10-17T16:29:23Z" ``` PodのリストとHTTPProxyリソースを確認します。 ``` $ kubectl get pod,httpproxy -n grafana NAME READY STATUS RESTARTS AGE pod/grafana-86556f498b-db25q 2/2 Running 0 14m NAME FQDN TLS SECRET STATUS STATUS DESCRIPTION httpproxy.projectcontour.io/grafana-httpproxy grafana.system.tanzu grafana-tls valid Valid HTTPProxy ``` デフォルトでは`grafana.system.tanzu`がGrafanaに対するFQDNです。`/etc/hosts`に次のレコードを追加します。 ``` ENVOY_IP=$(kubectl get svc -n projectcontour envoy -ojsonpath='{.status.loadBalancer.ingress[0].ip}') GRAFANA_FQDN=$(kubectl get httpproxy -n grafana grafana-httpproxy -ojsonpath='{.spec.virtualhost.fqdn}') cat <