--- title: cf-for-k8s(CF4K8s) 0.4.0をKind上にインストールする tags: ["Cloud Foundry", "Kubernetes", "CF4K8s"] categories: ["Dev", "PaaS", "CloudFoundry", "CF4K8s"] date: 2020-07-08T17:23:06Z updated: 2020-07-08T17:28:59Z --- RubyとPythonのBuildpackが追加された[cf-for-k8s (CF4K8s)](https://github.com/cloudfoundry/cf-for-k8s)の0.4.0がリリースされたので試してみます。 [cf-for-k8s(CF4K8s) 0.1.0をKind上にインストールする](https://blog.ik.am/entries/519)同様に、[Kind](https://github.com/kubernetes-sigs/kind)をLaptopにインストールし、その上にCloud Foundryをクリーンインストールします。 本記事の内容は基本的に次のドキュメントの通りです。 https://github.com/cloudfoundry/cf-for-k8s/blob/v0.4.0/docs/deploy-local.md **目次** ### CLIのインストール * [`kubectl`](https://github.com/kubernetes/kubectl) * [`kind`](https://github.com/kubernetes-sigs/kind) * [`kapp`](https://get-kapp.io/) (cf-for-k8sのデプロイに使用します) * [`ytt`](https://get-ytt.io/) (yamlのpatchを適用するために使用します) * [`bosh`](https://github.com/cloudfoundry/bosh-cli) (yaml内のパスワードとTLS証明書を自動生成するために使用します) * [`cf`](https://github.com/cloudfoundry/cli) をインストールします。 ``` brew install kubectl brew install kind brew install k14s/tap/kapp brew install k14s/tap/ytt brew install cloudfoundry/tap/bosh-cli brew install cloudfoundry/tap/cf-cli ``` 動作確認したバージョンは ``` $ kind --version kind version 0.8.1 $ kubectl version --client=true Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.5", GitCommit:"e6503f8d8f769ace2f338794c914a96fc335df0f", GitTreeState:"clean", BuildDate:"2020-07-04T15:01:15Z", GoVersion:"go1.14.4", Compiler:"gc", Platform:"darwin/amd64"} $ kapp version kapp version 0.30.0 $ ytt version ytt version 0.28.0 $ bosh -v version 6.3.0-bb69b142-2020-06-17T21:03:27Z $ cf -v cf version 6.51.0+2acd15650.2020-04-07 ``` です。 ### cf-for-k8sマニフェストの取得 [`0.4.0`](https://github.com/cloudfoundry/cf-for-k8s/releases/tag/v0.4.0)を使用します。 ``` git clone https://github.com/cloudfoundry/cf-for-k8s.git -b v0.4.0 ``` ### Kubernetesクラスターの作成 cf-for-k8sで使用するKubernetesをKindで作成します。Kind用のconfigは[`cf-for-k8s/deploy/kind/cluster.yml`](https://github.com/cloudfoundry/cf-for-k8s/blob/v0.4.0/deploy/kind/cluster.yml)に用意されています。 ``` kind create cluster --config=./cf-for-k8s/deploy/kind/cluster.yml ``` クラスタが作成されたら`kubectl`で動作確認します。 ``` $ kubectl cluster-info Kubernetes master is running at https://127.0.0.1:56303 KubeDNS is running at https://127.0.0.1:56303/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy $ kubectl get node NAME STATUS ROLES AGE VERSION kind-control-plane NotReady master 29s v1.18.2 $ kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-66bff467f8-kg9bp 1/1 Running 0 25s kube-system coredns-66bff467f8-tm4pn 0/1 Running 0 25s kube-system etcd-kind-control-plane 1/1 Running 0 41s kube-system kindnet-wl49f 1/1 Running 0 25s kube-system kube-apiserver-kind-control-plane 1/1 Running 0 41s kube-system kube-controller-manager-kind-control-plane 1/1 Running 0 41s kube-system kube-proxy-gnctc 1/1 Running 0 25s kube-system kube-scheduler-kind-control-plane 1/1 Running 0 41s local-path-storage local-path-provisioner-bd4bb6b75-b2pz5 1/1 Running 0 25s ``` ### Cloud Foundryのインストール いよいよk8s上にCloud Foundryをインストールを行います。 まずは、ソースコードを`cf push`した際にCloud Native Buildpackで作成されるDockerイメージをデプロイするDockerレジストリのアカウント情報を設定します。 ここではDockerHubを使用します。 また、パスワードや証明書を自動生成して[`./cf-for-k8s/hack/generate-values.sh`](https://github.com/cloudfoundry/cf-for-k8s/blob/v0.4.0/hack/generate-values.sh)を実行して設定値を作成します。 CFのドメインには`local.maki.lol`を使用します。 > `*.local.maki.lol`及び`*.apps.local.maki.lol`、`*.sys.local.maki.lol`はAレコードで`127.0.0.1`を返すように登録s亭あります。誰でも利用可能です。 ``` export DOCKERHUB_USERNAME=....... export DOCKERHUB_PASSWORD=.... ./cf-for-k8s/hack/generate-values.sh -d local.maki.lol > /tmp/cf-values.yml cat <> /tmp/cf-values.yml app_registry: hostname: https://index.docker.io/v1/ repository: ${DOCKERHUB_USERNAME} username: ${DOCKERHUB_USERNAME} password: ${DOCKERHUB_PASSWORD} EOF ``` 設定ファイルは`/tmp/cf-value.yml`に出力されます。`kapp`及び`ytt`を使用して次のコマンドでデプロイします。 ``` kapp deploy -a cf \ -f <(ytt -f ./cf-for-k8s/config \ -f /tmp/cf-values.yml \ -f ./cf-for-k8s/config-optional/remove-resource-requirements.yml \ -f ./cf-for-k8s/config-optional/remove-ingressgateway-service.yml \ -f ./cf-for-k8s/config-optional/add-metrics-server-components.yml \ -f ./cf-for-k8s/config-optional/patch-metrics-server.yml) ``` ベースのマニフェストファイルは`cf-for-k8s/config`に含まれます。これに対して、次の差分ファイルを`ytt`で適用しています。 * [`cf-for-k8s/config-optional/remove-resource-requirements.yml`](https://github.com/cloudfoundry/cf-for-k8s/blob/v0.4.0/config-optional/remove-resource-requirements.yml) ... Kindでもデプロイできるようにリソース(CPU、メモリ)の下限の制約を除外する * [`cf-for-k8s/config-optional/remove-ingressgateway-service.yml `](https://github.com/cloudfoundry/cf-for-k8s/blob/v0.4.0/config-optional/remove-ingressgateway-service.yml ) ... IstioのIngress Gateway Serviceを削除する * [`cf-for-k8s/config-optional/add-metrics-server-components.yml`](https://github.com/cloudfoundry/cf-for-k8s/blob/v0.4.0/config-optional/add-metrics-server-components.yml) ... Metrics Serverをインストールする(Kindに同梱されないため) * [`cf-for-k8s/config-optional/patch-metrics-server.yml`](https://github.com/cloudfoundry/cf-for-k8s/blob/v0.4.0/config-optional/patch-metrics-server.yml) ... Metrics Serverの設定変更 `y`を入力してインストールを開始します。次のようなログが流れてインストールが成功します。 ``` 1:46:41AM: ---- applying 80 changes [0/259 done] ---- 1:46:41AM: create clusterrolebinding/kpack-watcher-binding (rbac.authorization.k8s.io/v1) cluster 1:46:41AM: create namespace/cf-workloads-staging (v1) cluster 1:46:41AM: create clusterrole/kpack-watcher (rbac.authorization.k8s.io/v1) cluster 1:46:41AM: create clusterrole/cf:routes-admin (rbac.authorization.k8s.io/v1) cluster 1:46:41AM: create clusterrolebinding/kpack-webhook-certs-mutatingwebhookconfiguration-admin-binding (rbac.authorization.k8s.io/v1) cluster 1:46:41AM: create clusterrole/cf:kpack-builds-admin (rbac.authorization.k8s.io/v1) cluster 1:46:41AM: create clusterrole/cf:kpack-builds-informer (rbac.authorization.k8s.io/v1) cluster 1:46:41AM: create clusterrole/cf:service-accounts-secrets-fetcher (rbac.authorization.k8s.io/v1) cluster 1:46:41AM: create namespace/kpack (v1) cluster ...(略)... 1:52:28AM: ok: reconcile deployment/uaa (apps/v1) namespace: cf-system 1:52:28AM: ---- waiting on 1 changes [258/259 done] ---- 1:53:21AM: ongoing: reconcile deployment/log-cache (apps/v1) namespace: cf-system 1:53:21AM: ^ Waiting for 1 unavailable replicas 1:53:21AM: L ok: waiting on replicaset/log-cache-8555f64db (apps/v1) namespace: cf-system 1:53:21AM: L ongoing: waiting on pod/log-cache-8555f64db-jxrgl (v1) namespace: cf-system 1:53:21AM: ^ Condition Ready is not True (False) 1:53:29AM: ---- waiting on 1 changes [258/259 done] ---- 1:54:18AM: ok: reconcile deployment/log-cache (apps/v1) namespace: cf-system 1:54:18AM: ---- applying complete [259/259 done] ---- 1:54:18AM: ---- waiting complete [259/259 done] ---- Succeeded ``` インストール後の`Pod`一覧は次の通りです。 ``` $ kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE cf-blobstore cf-blobstore-minio-7bc599d776-cl594 2/2 Running 0 12m cf-db cf-db-postgresql-0 2/2 Running 0 12m cf-system ccdb-migrate-frzc6 0/2 Completed 0 12m cf-system cf-api-clock-5c94b5c69b-2xzpt 2/2 Running 0 12m cf-system cf-api-deployment-updater-864df64797-fkn4k 2/2 Running 0 12m cf-system cf-api-kpack-watcher-65bcb8999-htfvt 2/2 Running 0 12m cf-system cf-api-server-78dd7c9cb-7dmj2 5/5 Running 2 12m cf-system cf-api-server-78dd7c9cb-dzp9h 5/5 Running 2 12m cf-system cf-api-worker-7fccd99fb9-ldkvf 2/2 Running 0 12m cf-system eirini-8d468b747-zmw8z 2/2 Running 0 12m cf-system fluentd-87t8b 2/2 Running 0 12m cf-system log-cache-8555f64db-jxrgl 5/5 Running 5 12m cf-system metric-proxy-547f8d6bd6-zhvft 2/2 Running 0 12m cf-system routecontroller-7d5d7d6676-bmvf9 2/2 Running 1 12m cf-system uaa-5dc5db6f4d-4vvvm 2/2 Running 0 12m istio-system istio-citadel-6dbf97cb76-9pbgk 1/1 Running 0 13m istio-system istio-galley-7c95459bbf-95726 2/2 Running 0 13m istio-system istio-ingressgateway-dcxlj 2/2 Running 0 13m istio-system istio-pilot-6dc4d7876c-z66ww 2/2 Running 0 13m istio-system istio-policy-65f9ff4947-zw88k 2/2 Running 1 13m istio-system istio-sidecar-injector-56d6f57bb9-xcch8 1/1 Running 0 13m istio-system istio-telemetry-d88f8bf67-l4w4g 2/2 Running 1 13m kpack kpack-controller-55668d68fb-7cz4n 2/2 Running 0 12m kpack kpack-webhook-68fddc6f79-qbb9d 2/2 Running 0 12m kube-system coredns-66bff467f8-kg9bp 1/1 Running 0 24m kube-system coredns-66bff467f8-tm4pn 1/1 Running 0 24m kube-system etcd-kind-control-plane 1/1 Running 0 25m kube-system kindnet-wl49f 1/1 Running 0 24m kube-system kube-apiserver-kind-control-plane 1/1 Running 0 25m kube-system kube-controller-manager-kind-control-plane 1/1 Running 0 25m kube-system kube-proxy-gnctc 1/1 Running 0 24m kube-system kube-scheduler-kind-control-plane 1/1 Running 0 25m kube-system metrics-server-767fb6ccc4-mgvrl 1/1 Running 0 13m local-path-storage local-path-provisioner-bd4bb6b75-b2pz5 1/1 Running 0 24m ``` インストール後の`Service`一覧は次の通りです。 ``` $ kubectl get svc -A NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE cf-blobstore cf-blobstore-minio ClusterIP 10.104.7.210 9000/TCP 14m cf-db cf-db-postgresql ClusterIP 10.105.233.201 5432/TCP 14m cf-db cf-db-postgresql-headless ClusterIP None 5432/TCP 14m cf-system capi ClusterIP 10.109.78.72 80/TCP 14m cf-system eirini ClusterIP 10.109.197.79 8085/TCP 14m cf-system fluentd-forwarder-ingress ClusterIP 10.109.57.119 24224/TCP 14m cf-system log-cache ClusterIP 10.103.78.242 8083/TCP 14m cf-system log-cache-syslog ClusterIP 10.109.240.104 8082/TCP 14m cf-system metric-proxy ClusterIP 10.106.157.99 8080/TCP 14m cf-system uaa ClusterIP 10.102.224.57 8080/TCP 14m default kubernetes ClusterIP 10.96.0.1 443/TCP 25m istio-system istio-citadel ClusterIP 10.97.208.231 8060/TCP,15014/TCP 14m istio-system istio-galley ClusterIP 10.98.148.137 443/TCP,15014/TCP,9901/TCP,15019/TCP 14m istio-system istio-pilot ClusterIP 10.104.19.227 15010/TCP,15011/TCP,8080/TCP,15014/TCP 14m istio-system istio-policy ClusterIP 10.105.131.226 9091/TCP,15004/TCP,15014/TCP 14m istio-system istio-sidecar-injector ClusterIP 10.103.232.62 443/TCP 14m istio-system istio-telemetry ClusterIP 10.101.223.244 9091/TCP,15004/TCP,15014/TCP,42422/TCP 14m kpack kpack-webhook ClusterIP 10.105.159.252 443/TCP 14m kube-system kube-dns ClusterIP 10.96.0.10 53/UDP,53/TCP,9153/TCP 25m kube-system metrics-server ClusterIP 10.97.244.44 443/TCP 14m 3m34s ``` インストール後の`PersistentVolume`一覧は次の通りです。データベースとしてPostgreSQL、BlobstoreとしてMinioが`PersistentVolume`を使用しています。 ``` $ kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-7d81c027-c94e-4c7f-b35d-ee0152b546b5 10Gi RWO Delete Bound cf-blobstore/cf-blobstore-minio standard 13m pvc-e58a713a-804e-4e6e-a0b4-390cd7b001cf 8Gi RWO Delete Bound cf-db/data-cf-db-postgresql-0 standard 13m ``` また、次のようなCustom Resourceも作成されています。 ``` $ kubectl get $(kubectl get crd -o jsonpath='{range .items[*]}{.metadata.name}{","}{end}')podtemplates -A NAMESPACE NAME AGE istio-system attributemanifest.config.istio.io/istioproxy 17m istio-system attributemanifest.config.istio.io/kubernetes 17m NAMESPACE NAME LATESTIMAGE READY cf-workloads-staging custombuilder.experimental.kpack.pivotal.io/cf-default-builder making/cf-default-builder@sha256:6fe98d20624f29b89ec03b69b12f9b01a1826c0e14158b7dc66bc32bcadcb299 True NAMESPACE NAME HOST AGE istio-system destinationrule.networking.istio.io/istio-policy istio-policy.istio-system.svc.cluster.local 17m istio-system destinationrule.networking.istio.io/istio-telemetry istio-telemetry.istio-system.svc.cluster.local 17m NAMESPACE NAME AGE cf-system gateway.networking.istio.io/istio-ingressgateway 17m istio-system gateway.networking.istio.io/ingressgateway 17m NAMESPACE NAME AGE istio-system handler.config.istio.io/kubernetesenv 17m istio-system handler.config.istio.io/prometheus 17m NAMESPACE NAME AGE istio-system instance.config.istio.io/attributes 17m istio-system instance.config.istio.io/requestcount 17m istio-system instance.config.istio.io/requestduration 17m istio-system instance.config.istio.io/requestsize 17m istio-system instance.config.istio.io/responsesize 17m istio-system instance.config.istio.io/tcpbytereceived 17m istio-system instance.config.istio.io/tcpbytesent 17m istio-system instance.config.istio.io/tcpconnectionsclosed 17m istio-system instance.config.istio.io/tcpconnectionsopened 17m NAMESPACE NAME AGE meshpolicy.authentication.istio.io/default 17m NAMESPACE NAME AGE cf-blobstore policy.authentication.istio.io/cf-blobstore-allow-plaintext 17m kpack policy.authentication.istio.io/kpack-webhook-allow-plaintext 17m NAMESPACE NAME AGE istio-system rule.config.istio.io/kubeattrgenrulerule 17m istio-system rule.config.istio.io/promhttp 17m istio-system rule.config.istio.io/promtcp 17m istio-system rule.config.istio.io/promtcpconnectionclosed 17m istio-system rule.config.istio.io/promtcpconnectionopen 17m istio-system rule.config.istio.io/tcpkubeattrgenrulerule 17m NAMESPACE NAME AGE cf-workloads sidecar.networking.istio.io/default 17m istio-system sidecar.networking.istio.io/default 17m NAMESPACE NAME READY stack.experimental.kpack.pivotal.io/cflinuxfs3-stack True NAMESPACE NAME READY store.experimental.kpack.pivotal.io/cf-buildpack-store True NAMESPACE NAME GATEWAYS HOSTS AGE cf-system virtualservice.networking.istio.io/cf-api-external-virtual-service [cf-system/istio-ingressgateway] [api.local.maki.lol] 17m cf-system virtualservice.networking.istio.io/log-cache-external-virtual-service [cf-system/istio-ingressgateway] [log-cache.local.maki.lol] 17m cf-system virtualservice.networking.istio.io/uaa-external-virtual-service [cf-system/istio-ingressgateway] [uaa.local.maki.lol *.uaa.local.maki.lol login.local.maki.lol *.login.local.maki.lol] 17m ``` `kapp`を使ってインストールした場合は`kapp inspect -a <アプリ名> --tree`コマンドで、作成されたリソース一覧を確認できます。 次のエンドポイントにアクセスして、Cloud Controllerのバージョンを確認できます。 ``` $ curl -s -k https://api.local.maki.lol | jq . { "links": { "self": { "href": "https://api.local.maki.lol" }, "bits_service": null, "cloud_controller_v2": { "href": "https://api.local.maki.lol/v2", "meta": { "version": "2.151.0" } }, "cloud_controller_v3": { "href": "https://api.local.maki.lol/v3", "meta": { "version": "3.86.0" } }, "network_policy_v0": { "href": "https://api.local.maki.lol/networking/v0/external" }, "network_policy_v1": { "href": "https://api.local.maki.lol/networking/v1/external" }, "login": { "href": "https://login.local.maki.lol" }, "uaa": { "href": "https://uaa.local.maki.lol" }, "credhub": null, "routing": { "href": "https://api.local.maki.lol/routing" }, "logging": { "href": "wss://doppler.local.maki.lol:443" }, "log_cache": { "href": "https://log-cache.local.maki.lol" }, "log_stream": { "href": "https://log-stream.local.maki.lol" }, "app_ssh": { "href": "TODO.TODO", "meta": { "host_key_fingerprint": "placeholder", "oauth_client": "placeholder" } } } } ``` ### Cloud Foundryにログイン 早速Cloud Foundryにログインしてみます。 `admin`ユーザーのパスワードは`bosh int /tmp/cf-values.yml --path /cf_admin_password`で取得できます。 次のコマンドでログインします。 ``` cf login -a api.local.maki.lol -u admin -p $(bosh int /tmp/cf-values.yml --path /cf_admin_password) --skip-ssl-validation ``` 次のようなログが出力されればOKです。 ``` API endpoint: api.local.maki.lol Authenticating... OK Targeted org system API endpoint: https://api.local.maki.lol (API version: 3.86.0) User: admin Org: system Space: No space targeted, use 'cf target -s SPACE' ``` Optionalですが、Docker Imageをそのままデプロイできる機能を有効にします。 ``` $ cf enable-feature-flag diego_docker Setting status of diego_docker as admin... OK Feature diego_docker Enabled. ``` UAAにアクセスしてみます。URLは https://uaa.local.maki.lol です。 ![image](https://user-images.githubusercontent.com/106908/79457457-36d37c00-802b-11ea-9e6c-c1000a91a9d8.png) ### Org、Spaceの作成 `demo` Orgと`demo` Spaceを作成します。 ``` cf create-org demo cf create-space demo -o demo cf target -o demo -s demo ``` ### Javaアプリケーションのデプロイ Spring Bootのアプリをデプロイします。 雛形プロジェクトを作成し、ビルドします。 ``` cd /tmp curl https://start.spring.io/starter.tgz \ -d artifactId=hello-cf \ -d baseDir=hello-cf \ -d dependencies=web,actuator \ -d packageName=com.example \ -d applicationName=HelloCfApplication | tar -xzvf - cd hello-cf ./mvnw clean package -DskipTests=true ``` `cf push`します。 ``` $ cf push hello -p target/hello-cf-0.0.1-SNAPSHOT.jar Pushing app hello to org demo / space demo as admin... Getting app info... Creating app with these attributes... + name: hello path: /private/tmp/hello-cf/target/hello-cf-0.0.1-SNAPSHOT.jar routes: + hello.apps.local.maki.lol Creating app hello... Mapping routes... Comparing local files to remote cache... Packaging files to upload... Uploading files... 15.66 MiB / 15.66 MiB [===============================================================================================================================================================================================================================================================] 100.00% 1s Waiting for API to complete processing files... Staging app and tracing logs... Loading secret for "https://index.docker.io/v1/" from secret "cc-kpack-registry-auth-secret-ver-1" at location "/var/build-secrets/cc-kpack-registry-auth-secret-ver-1" Successfully downloaded cf-blobstore-minio.cf-blobstore.svc.cluster.local:9000/cc-packages/cb/83/cb83be62-a59e-44c1-9046-25d662b1a1bd in path "/workspace" 5 of 15 buildpacks participating paketo-buildpacks/bellsoft-liberica 2.8.0 paketo-buildpacks/executable-jar 1.2.7 paketo-buildpacks/apache-tomcat 1.3.0 paketo-buildpacks/dist-zip 1.3.5 paketo-buildpacks/spring-boot 1.6.0 Previous image with name "making/e14ea3c9-0c3d-4b72-99a5-c22b4fbc4e22" not found Paketo BellSoft Liberica Buildpack 2.8.0 https://github.com/paketo-buildpacks/bellsoft-liberica Build Configuration: $BP_JVM_VERSION 11.* the Java version Launch Configuration: $BPL_JVM_HEAD_ROOM 0 the headroom in memory calculation $BPL_JVM_LOADED_CLASS_COUNT 35% of classes the number of loaded classes in memory calculation $BPL_JVM_THREAD_COUNT 250 the number of threads in memory calculation BellSoft Liberica JRE 11.0.7: Contributing to layer Downloading from https://github.com/bell-sw/Liberica/releases/download/11.0.7+10/bellsoft-jre11.0.7+10-linux-amd64.tar.gz Verifying checksum Expanding to /layers/paketo-buildpacks_bellsoft-liberica/jre Writing env.launch/JAVA_HOME.override Writing env.launch/MALLOC_ARENA_MAX.override Writing profile.d/active-processor-count.sh Memory Calculator 4.0.0: Contributing to layer Downloading from https://github.com/cloudfoundry/java-buildpack-memory-calculator/releases/download/v4.0.0/memory-calculator-4.0.0.tgz Verifying checksum Expanding to /layers/paketo-buildpacks_bellsoft-liberica/memory-calculator Writing profile.d/memory-calculator.sh Class Counter: Contributing to layer Copying to /layers/paketo-buildpacks_bellsoft-liberica/class-counter JVMKill Agent 1.16.0: Contributing to layer Downloading from https://github.com/cloudfoundry/jvmkill/releases/download/v1.16.0.RELEASE/jvmkill-1.16.0-RELEASE.so Verifying checksum Copying to /layers/paketo-buildpacks_bellsoft-liberica/jvmkill Writing env.launch/JAVA_OPTS.append Link-Local DNS: Contributing to layer Copying to /layers/paketo-buildpacks_bellsoft-liberica/link-local-dns Writing profile.d/link-local-dns.sh Java Security Properties: Contributing to layer Writing env.launch/JAVA_OPTS.append Writing env.launch/JAVA_SECURITY_PROPERTIES.override Security Providers Configurer: Contributing to layer Copying to /layers/paketo-buildpacks_bellsoft-liberica/security-providers-configurer Writing profile.d/security-providers-classpath.sh Writing profile.d/security-providers-configurer.sh OpenSSL Certificate Loader: Contributing to layer Copying to /layers/paketo-buildpacks_bellsoft-liberica/openssl-security-provider Writing profile.d/openssl-certificate-loader.sh Paketo Executable JAR Buildpack 1.2.7 https://github.com/paketo-buildpacks/executable-jar Writing env.launch/CLASSPATH Process types: executable-jar: java -cp "${CLASSPATH}" ${JAVA_OPTS} org.springframework.boot.loader.JarLauncher task: java -cp "${CLASSPATH}" ${JAVA_OPTS} org.springframework.boot.loader.JarLauncher web: java -cp "${CLASSPATH}" ${JAVA_OPTS} org.springframework.boot.loader.JarLauncher Paketo Spring Boot Buildpack 1.6.0 https://github.com/paketo-buildpacks/spring-boot Build Configuration: $BP_BOOT_NATIVE_IMAGE the build to create a native image (requires GraalVM) $BP_BOOT_NATIVE_IMAGE_BUILD_ARGUMENTS the arguments to pass to the native-image command Image labels: org.opencontainers.image.title org.opencontainers.image.version org.springframework.boot.spring-configuration-metadata.json org.springframework.boot.version Adding layer 'launcher' Adding layer 'paketo-buildpacks/bellsoft-liberica:class-counter' Adding layer 'paketo-buildpacks/bellsoft-liberica:java-security-properties' Adding layer 'paketo-buildpacks/bellsoft-liberica:jre' Adding layer 'paketo-buildpacks/bellsoft-liberica:jvmkill' Adding layer 'paketo-buildpacks/bellsoft-liberica:link-local-dns' Adding layer 'paketo-buildpacks/bellsoft-liberica:memory-calculator' Adding layer 'paketo-buildpacks/bellsoft-liberica:openssl-security-provider' Adding layer 'paketo-buildpacks/bellsoft-liberica:security-providers-configurer' Adding layer 'paketo-buildpacks/executable-jar:class-path' Adding 1/1 app layer(s) Adding layer 'config' *** Images (sha256:5cf43b2d676e15a09aa6845ce107d46a0807cabb24e051643191e771a9a4747a): making/e14ea3c9-0c3d-4b72-99a5-c22b4fbc4e22 index.docker.io/making/e14ea3c9-0c3d-4b72-99a5-c22b4fbc4e22:b1.20200708.170540 Build successful Waiting for app to start... name: hello requested state: started isolation segment: placeholder routes: hello.apps.local.maki.lol last uploaded: Thu 09 Jul 02:08:50 JST 2020 stack: buildpacks: type: web instances: 1/1 memory usage: 1024M start command: java -cp "${CLASSPATH}" ${JAVA_OPTS} org.springframework.boot.loader.JarLauncher state since cpu memory disk details #0 running 2020-07-08T17:08:55Z 0.0% 0 of 1G 0 of 1G ``` デプロイが成功しました。初回は5分くらいかかるかもしれません。 `cf apps`コマンドでアプリケーション一覧を確認します。 ``` $ cf apps Getting apps in org demo / space demo as admin... OK name requested state instances memory disk urls hello started 1/1 1G 1G hello.local.maki.lol ``` `cf app`コマンドでアプリの詳細を確認できます。 ``` $ cf app hello Showing health and status for app hello in org demo / space demo as admin... name: hello requested state: started isolation segment: placeholder routes: hello.apps.local.maki.lol last uploaded: Thu 09 Jul 02:08:50 JST 2020 stack: buildpacks: type: web instances: 1/1 memory usage: 1024M state since cpu memory disk details #0 running 2020-07-08T17:08:54Z 0.2% 152.4M of 1G 0 of 1G ``` `/actuator/health`エンドポイントにアクセスします。 ``` $ curl -k https://hello.apps.local.maki.lol/actuator/health {"status":"UP"} ``` また、これまでのCloud FoundryではBuildpack v2が使用されていましたが、cf-for-k8sでは[kpack](https://github.com/pivotal/kpack)経由で[Cloud Native Buildpacks](https://buildpacks.io/)が使用されます。 `cf push`で作成されたコンテナイメージはインストール時に設定したDocker Registryにデプロイされます。実際にDockerHubにイメージがデプロイされていました。 ![image](https://user-images.githubusercontent.com/106908/79461235-d9422e00-8030-11ea-945d-fbbeddbb7a54.png) なお、Kpackが使用するCloud Native Buildpacks一覧は次のコマンドで確認できます。 ``` $ kubectl get -n cf-workloads-staging custombuilder cf-default-builder -o yaml | bosh int - --path /spec order: - group: - id: paketo-community/ruby - group: - id: paketo-community/python - group: - id: paketo-buildpacks/java - group: - id: paketo-buildpacks/nodejs - group: - id: paketo-buildpacks/go - group: - id: paketo-buildpacks/dotnet-core - group: - id: paketo-buildpacks/php - group: - id: paketo-buildpacks/httpd - group: - id: paketo-buildpacks/nginx - group: - id: paketo-buildpacks/procfile serviceAccount: cc-kpack-registry-service-account stack: cflinuxfs3-stack store: cf-buildpack-store tag: making/cf-default-builder ``` サポートされているBuildpackはJava, Nodejs, Go, DotNet, PHP, HTTPD, Nginx, Procfileです。 `cf logs hello --recent`コマンドでアプリログを確認できます。 cf-for-k8sでは、EiriniというコンポーネントがCloud Controllerからのリクエストを受けてKubernetesにアプリを`Statefulset`としてデプロイします。Eiriniが作成した`StatefulSet`及び、それにより作られた`Pod`は次のコマンドで確認できます。 ``` $ kubectl get statefulset,pod -n cf-workloads NAME READY AGE statefulset.apps/hello-demo-9e22263f53 1/1 7m24s NAME READY STATUS RESTARTS AGE pod/hello-demo-9e22263f53-0 2/2 Running 0 7m24s ``` 0.4.0からは`cf map-route`で`Route` Custom Resourceが作成されるようになりました。 ``` $ kubectl get route -A NAMESPACE NAME AGE cf-workloads 749a3751-6aac-4b8d-a022-82f8449dc6b0 14m ``` 次のような内容が定義されています。 ``` $ kubectl get route -n cf-workloads 749a3751-6aac-4b8d-a022-82f8449dc6b0 -o yaml apiVersion: networking.cloudfoundry.org/v1alpha1 kind: Route metadata: creationTimestamp: "2020-07-08T17:05:31Z" finalizers: - routes.networking.cloudfoundry.org generation: 3 labels: app.kubernetes.io/component: cf-networking app.kubernetes.io/managed-by: cloudfoundry app.kubernetes.io/name: 749a3751-6aac-4b8d-a022-82f8449dc6b0 app.kubernetes.io/part-of: cloudfoundry app.kubernetes.io/version: 0.0.0 cloudfoundry.org/domain_guid: d7858c4f-52dc-4a39-8375-94a960ebc6b9 cloudfoundry.org/org_guid: bd7c98ba-a78c-45d8-9efd-8da760852add cloudfoundry.org/route_guid: 749a3751-6aac-4b8d-a022-82f8449dc6b0 cloudfoundry.org/space_guid: ab55dd1a-fbc9-46df-8073-b85406081c6e managedFields: - apiVersion: networking.cloudfoundry.org/v1alpha1 fieldsType: FieldsV1 fieldsV1: f:metadata: f:finalizers: .: {} v:"routes.networking.cloudfoundry.org": {} f:status: .: {} f:conditions: {} manager: routecontroller operation: Update time: "2020-07-08T17:05:31Z" - apiVersion: networking.cloudfoundry.org/v1alpha1 fieldsType: FieldsV1 fieldsV1: f:metadata: f:labels: .: {} f:app.kubernetes.io/component: {} f:app.kubernetes.io/managed-by: {} f:app.kubernetes.io/name: {} f:app.kubernetes.io/part-of: {} f:app.kubernetes.io/version: {} f:cloudfoundry.org/domain_guid: {} f:cloudfoundry.org/org_guid: {} f:cloudfoundry.org/route_guid: {} f:cloudfoundry.org/space_guid: {} f:spec: .: {} f:destinations: {} f:domain: .: {} f:internal: {} f:name: {} f:host: {} f:url: {} manager: rest-client operation: Update time: "2020-07-08T17:05:32Z" name: 749a3751-6aac-4b8d-a022-82f8449dc6b0 namespace: cf-workloads resourceVersion: "7366" selfLink: /apis/networking.cloudfoundry.org/v1alpha1/namespaces/cf-workloads/routes/749a3751-6aac-4b8d-a022-82f8449dc6b0 uid: 429fa1bd-19a1-4928-a2fb-4ac134263b05 spec: destinations: - app: guid: e14ea3c9-0c3d-4b72-99a5-c22b4fbc4e22 process: type: web guid: 8399a02d-1115-4873-abe4-e7fc719aa61d port: 8080 selector: matchLabels: cloudfoundry.org/app_guid: e14ea3c9-0c3d-4b72-99a5-c22b4fbc4e22 cloudfoundry.org/process_type: web domain: internal: false name: apps.local.maki.lol host: hello url: hello.apps.local.maki.lol ``` > 現時点でcf-for-k8sでDeveloper(`cf push`を使うユーザー)が`kubectl`を使うことは想定していません。`kubectl`はPlatformエンジニアが使う想定です。 ### Smoke Testの実行 `cf push`は成功しましたが、インストールが適切にできているか確認するために、Smoke Testを実行してみます。 Smoke Testを実行するには[ginkgo](https://github.com/onsi/ginkgo)をインストールする必要があります。 ``` brew install go go get -u github.com/onsi/ginkgo/ginkgo export PATH=$PATH:$HOME/go/bin ``` 次のコマンドでSmoke Testを実行します。 ``` export SMOKE_TEST_APPS_DOMAIN=apps.local.maki.lol export SMOKE_TEST_API_ENDPOINT=api.local.maki.lol export SMOKE_TEST_USERNAME=admin export SMOKE_TEST_PASSWORD=$(bosh int /tmp/cf-values.yml --path /cf_admin_password) export SMOKE_TEST_SKIP_SSL=true ./cf-for-k8s/hack/run-smoke-tests.sh ``` 次のような結果が出力されれば成功です。 ``` Running Suite: Smoke Tests Suite ================================ Random Seed: 1594228649 Will run 1 of 1 specs ..(略).. Ran 1 of 1 Specs in 164.904 seconds SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped PASS Ginkgo ran 1 suite in 2m51.151127314s Test Suite Passed ``` ### Cloud Foundryのアンインストール アンインストールは`kapp`で一発です。 ``` kapp delete -a cf ``` 次のコマンドでKindのK8sクラスタも削除できます。 ``` kind delete cluster ``` --- cf-for-k8s 0.4.0を試しました。0.1.0に比べて機能が増えてきました。 リリースノートも合わせて確認してみてください。 https://github.com/cloudfoundry/cf-for-k8s/releases/tag/v0.4.0