--- title: BOSH v2 CLIでConcourseを自宅サーバーのBOSH Lite on VirtualBox上にインストールする tags: ["BOSH", "BOSH-Lite", "Concourse CI"] categories: ["Dev", "CI", "ConcourseCI"] date: 2017-04-22T21:27:36Z updated: 2017-06-04T05:48:46Z --- [前記事](https://blog.ik.am/entries/415)でVirtualBoxにBOSH v2 CLIを使ってBOSH Liteをインストールした。 以下の記事はこの設定が済んでいる前提である。 今度はこのBOSH Lite上にConcourseをインストールする。 最終的に作りたいのは次のような環境。 ![image](https://cloud.githubusercontent.com/assets/106908/25311223/ebeb411e-2835-11e7-9d6e-ca298a7a4061.png) まずはHA Proxy抜きな状態からセットアップする。 **目次** ### Cloud Configの修正 ConcourseのWEB UIをStatic IPで固定するためにcloud configを修正する。 次のコマンドで`cloud-config.yml`をダウンロード。 ``` bosh2 cloud-config > ~/cloud-config.yml ``` Static IPに`10.244.0.200`を追下。 ``` yml networks: - name: default subnets: - azs: - z1 - z2 - z3 gateway: 10.244.0.1 range: 10.244.0.0/24 reserved: [] static: - 10.244.0.34 - 10.244.0.200 # ここ追加 type: manual ``` 適用。 ``` bosh2 update-cloud-config ~/cloud-config.yml ``` ### Concourseのデプロイ manifestは[こちら](https://raw.githubusercontent.com/making/bosh-manifests/master/concourse.yml)に穴埋め状態で用意しておいた。 Concourseに必要なBOSH Releaseのurlはmanifestに記載してあり、明示的に`bosh upload-release`をする必要はない。 ``` wget https://raw.githubusercontent.com/making/bosh-manifests/master/concourse.yml ``` BOSH v2 CLIの機能でpasswordやTLS証明書といったcredentials情報は自動生成させ、manifest上には記述しない。 生成されたcredentials情報は次の`bosh2 interpolate`コマンドで確認できる。 ``` bosh2 interpolate concourse.yml --vars-store ./creds-concourse.yml -v internal_ip=10.244.0.200 ``` `creds-concourse.yml`に生成されたcredentials情報が保存される。 ``` bosh2 deploy -d concourse --vars-store ./creds-concourse.yml -v internal_ip=10.244.0.200 concourse.yml ``` ログは次の通り。 ``` console $ bosh2 deploy -d concourse --vars-store ./creds-concourse.yml -v internal_ip=10.244.0.200 concourse.yml Using environment '192.168.50.6' as client 'admin' Using deployment 'concourse' Release 'concourse/2.7.3' already exists. Release 'garden-runc/1.4.0' already exists. + azs: + - name: z1 + - name: z2 + - name: z3 + vm_types: + - name: default + compilation: + az: z1 + network: default + reuse_compilation_vms: true + vm_type: default + workers: 5 + networks: + - name: default + subnets: + - azs: + - z1 + - z2 + - z3 + gateway: 10.244.0.1 + range: 10.244.0.0/24 + reserved: [] + static: + - 10.244.0.34 + - 10.244.0.200 + type: manual + disk_types: + - disk_size: 1024 + name: default + stemcells: + - alias: trusty + os: ubuntu-trusty + version: '3363.19' + releases: + - name: concourse + sha1: ca38944353c893660a428cc6678941734b51b036 + url: https://bosh.io/d/github.com/concourse/concourse?v=2.7.3 + version: 2.7.3 + - name: garden-runc + sha1: 1d6020e761806d7f355ceda06c889c582b47dc32 + url: https://bosh.io/d/github.com/cloudfoundry/garden-runc-release?v=1.4.0 + version: 1.4.0 + update: + canaries: 1 + canary_watch_time: 1000-60000 + max_in_flight: 1 + serial: false + update_watch_time: 1000-60000 + instance_groups: + - azs: + - z1 + instances: 1 + jobs: + - name: atc + properties: + basic_auth_password: "" + basic_auth_username: "" + bind_port: "" + external_url: "" + postgresql_database: "" + tls_bind_port: "" + tls_cert: "" + tls_key: "" + release: concourse + - name: tsa + properties: {} + release: concourse + name: web + networks: + - name: default + static_ips: + - 10.244.0.200 + stemcell: trusty + vm_type: default + - azs: + - z1 + instances: 1 + jobs: + - name: postgresql + properties: + databases: + - name: "" + password: "" + role: "" + release: concourse + name: db + networks: + - name: default + persistent_disk_type: default + stemcell: trusty + vm_type: default + - azs: + - z1 + instances: 1 + jobs: + - name: groundcrew + properties: {} + release: concourse + - name: baggageclaim + properties: {} + release: concourse + - name: garden + properties: + garden: + listen_address: "" + listen_network: "" + release: garden-runc + name: worker + networks: + - name: default + stemcell: trusty + vm_type: default + name: concourse + variables: + - name: default_ca + options: + common_name: ca + is_ca: true + type: certificate + - name: atc_ssl + options: + alternative_names: + - 10.244.0.200 + ca: default_ca + common_name: 10.244.0.200 + type: certificate + - name: postgres_password + type: password + - name: ui_password + type: password Continue? [yN]: y Task 11 20:29:07 | Preparing deployment: Preparing deployment (00:00:00) 20:29:07 | Preparing package compilation: Finding packages to compile (00:00:00) 20:29:07 | Compiling packages: tar/f2ea61c537d8eb8cb2d691ce51e8516b28fa5bb7 20:29:07 | Compiling packages: busybox/fc652425c32d0dad62f45bca18e1899671e2e570 20:29:07 | Compiling packages: shadow/7a5e46357a33cafc8400a8e3e2e1f6d3a1159cb6 20:29:07 | Compiling packages: libseccomp/7a54b27a61b42980935e863d7060dc5a076b44d0 20:29:07 | Compiling packages: apparmor/d2a9bb24b85a144e874e7627a5fefb4e9b8b30f3 20:29:17 | Compiling packages: busybox/fc652425c32d0dad62f45bca18e1899671e2e570 (00:00:10) 20:29:17 | Compiling packages: iptables/70cd40ad87371de73d6bdfc34967e94422fc2cc4 20:29:29 | Compiling packages: libseccomp/7a54b27a61b42980935e863d7060dc5a076b44d0 (00:00:22) 20:29:29 | Compiling packages: golang/63a243be32451af083a062ba2c929c3f2b34f132 20:29:46 | Compiling packages: shadow/7a5e46357a33cafc8400a8e3e2e1f6d3a1159cb6 (00:00:39) 20:29:46 | Compiling packages: btrfs_tools/6856973d0bc2dc673b6740f5e164ba77a77fd4a6 (00:00:01) 20:29:47 | Compiling packages: vagrant_cloud_resource/965f230db2311645cbbf9d240db6ee5fdb43a983 (00:00:02) 20:29:49 | Compiling packages: tracker_resource/caedeb694f9b4945b12a3ce9731d5cfbf98a0e56 20:29:51 | Compiling packages: golang/63a243be32451af083a062ba2c929c3f2b34f132 (00:00:22) 20:29:51 | Compiling packages: time_resource/9c9481e92eeb038d3621854e325412fdea5a6511 (00:00:02) 20:29:53 | Compiling packages: semver_resource/d012d4c6d21de36c3a576b20dfc3b134d10ec42c 20:29:53 | Compiling packages: tracker_resource/caedeb694f9b4945b12a3ce9731d5cfbf98a0e56 (00:00:04) 20:29:53 | Compiling packages: s3_resource/2af4d1cea8d5b617e6d4f580818b414db7e54d42 (00:00:05) 20:29:58 | Compiling packages: pool_resource/7bb393617fa62ee257c4e56c1660cbf99fa7ee62 20:29:58 | Compiling packages: semver_resource/d012d4c6d21de36c3a576b20dfc3b134d10ec42c (00:00:05) 20:29:58 | Compiling packages: hg_resource/ad85955734d8ce0a813218102dcc4d339d779ecc 20:29:59 | Compiling packages: apparmor/d2a9bb24b85a144e874e7627a5fefb4e9b8b30f3 (00:00:52) 20:29:59 | Compiling packages: github_release_resource/905a156da0707f87155b1390c05722d3e6a7532f 20:30:01 | Compiling packages: iptables/70cd40ad87371de73d6bdfc34967e94422fc2cc4 (00:00:44) 20:30:01 | Compiling packages: git_resource/9eab8c35f89630fefce5e3bcd9cb39697ee835c0 20:30:02 | Compiling packages: pool_resource/7bb393617fa62ee257c4e56c1660cbf99fa7ee62 (00:00:04) 20:30:02 | Compiling packages: docker_image_resource/b7ad85a4aa0e63bb89fc58682de209429a0db1ef 20:30:02 | Compiling packages: github_release_resource/905a156da0707f87155b1390c05722d3e6a7532f (00:00:03) 20:30:02 | Compiling packages: cf_resource/6c95b8277b3310d3f4fee49efdca7b7b61dbd602 20:30:03 | Compiling packages: hg_resource/ad85955734d8ce0a813218102dcc4d339d779ecc (00:00:05) 20:30:03 | Compiling packages: bosh_io_stemcell_resource/4aa948b8442e7f2395984931398d330000c4604f 20:30:05 | Compiling packages: git_resource/9eab8c35f89630fefce5e3bcd9cb39697ee835c0 (00:00:04) 20:30:05 | Compiling packages: bosh_io_release_resource/980773e5718d0f4e80778735af1560c0438ccfe6 20:30:06 | Compiling packages: bosh_io_stemcell_resource/4aa948b8442e7f2395984931398d330000c4604f (00:00:03) 20:30:06 | Compiling packages: bosh_deployment_resource/56c0e245445bab5097dd6725fd99e98fa638b130 20:30:07 | Compiling packages: bosh_io_release_resource/980773e5718d0f4e80778735af1560c0438ccfe6 (00:00:02) 20:30:07 | Compiling packages: archive_resource/d95bdd7a1d7ab0467da4645786e0e2962e3687c7 20:30:07 | Compiling packages: cf_resource/6c95b8277b3310d3f4fee49efdca7b7b61dbd602 (00:00:05) 20:30:07 | Compiling packages: postgresql_9.3/f8b7a26f105b921843fa5565818b50cc45ace6af 20:30:09 | Compiling packages: archive_resource/d95bdd7a1d7ab0467da4645786e0e2962e3687c7 (00:00:02) 20:30:09 | Compiling packages: generated_worker_key/00b5b02050bc6588cdfe9e523b2ba24a6c9de3c7 (00:00:01) 20:30:10 | Compiling packages: generated_tsa_host_key/00b5b02050bc6588cdfe9e523b2ba24a6c9de3c7 20:30:10 | Compiling packages: bosh_deployment_resource/56c0e245445bab5097dd6725fd99e98fa638b130 (00:00:04) 20:30:10 | Compiling packages: generated_signing_key/00b5b02050bc6588cdfe9e523b2ba24a6c9de3c7 20:30:11 | Compiling packages: generated_tsa_host_key/00b5b02050bc6588cdfe9e523b2ba24a6c9de3c7 (00:00:01) 20:30:11 | Compiling packages: concourse_version/33e3d7d5da79cadb3d6e09880eaa196df0ddea54 20:30:12 | Compiling packages: generated_signing_key/00b5b02050bc6588cdfe9e523b2ba24a6c9de3c7 (00:00:02) 20:30:12 | Compiling packages: golang/b1b2135422b3691f62b4dfd38d2f960759155697 20:30:12 | Compiling packages: docker_image_resource/b7ad85a4aa0e63bb89fc58682de209429a0db1ef (00:00:10) 20:30:12 | Compiling packages: pid_utils/a1f0590ea02d938b933a101c7438985721cd0ab4 20:30:12 | Compiling packages: concourse_version/33e3d7d5da79cadb3d6e09880eaa196df0ddea54 (00:00:01) 20:30:12 | Compiling packages: maximus/d37f10465e2adbfcb708773ab5f540f81bbaf884 20:30:13 | Compiling packages: pid_utils/a1f0590ea02d938b933a101c7438985721cd0ab4 (00:00:01) 20:30:13 | Compiling packages: guardian/d2b0ce380e1ddb59d71223894723055f1d31acd8 20:30:18 | Compiling packages: tar/f2ea61c537d8eb8cb2d691ce51e8516b28fa5bb7 (00:01:11) 20:30:18 | Compiling packages: runc-rootless/13f90a80fb14a002a60a9f7db518b9340422eb24 20:30:20 | Compiling packages: maximus/d37f10465e2adbfcb708773ab5f540f81bbaf884 (00:00:08) 20:30:20 | Compiling packages: runc/27c25b1fbc49b7cf5b4d35958ec9b1d290b9601a 20:30:34 | Compiling packages: golang/b1b2135422b3691f62b4dfd38d2f960759155697 (00:00:22) 20:30:34 | Compiling packages: baggageclaim/56f351a1e6394de4001a4d6e2af321f701b38719 20:30:35 | Compiling packages: runc-rootless/13f90a80fb14a002a60a9f7db518b9340422eb24 (00:00:17) 20:30:35 | Compiling packages: groundcrew/70115c7b6030f03e55f3b2e7404b1197442c574b 20:30:39 | Compiling packages: runc/27c25b1fbc49b7cf5b4d35958ec9b1d290b9601a (00:00:19) 20:30:39 | Compiling packages: resource_discovery/624b6ecbed549981d6aed42fcc99f8c35f3a0e59 20:30:41 | Compiling packages: guardian/d2b0ce380e1ddb59d71223894723055f1d31acd8 (00:00:28) 20:30:41 | Compiling packages: tsa/2dfac0651773697ea82974553960911c75925aba 20:30:44 | Compiling packages: baggageclaim/56f351a1e6394de4001a4d6e2af321f701b38719 (00:00:10) 20:30:44 | Compiling packages: fly/fa6033d0990d8d08458a3a04fb8dd7d9ccdf874b 20:30:45 | Compiling packages: groundcrew/70115c7b6030f03e55f3b2e7404b1197442c574b (00:00:10) 20:30:45 | Compiling packages: atc/3f8900840b6e83a458c92a38868822d7a98f8769 20:30:49 | Compiling packages: resource_discovery/624b6ecbed549981d6aed42fcc99f8c35f3a0e59 (00:00:10) 20:30:53 | Compiling packages: tsa/2dfac0651773697ea82974553960911c75925aba (00:00:12) 20:30:55 | Compiling packages: fly/fa6033d0990d8d08458a3a04fb8dd7d9ccdf874b (00:00:11) 20:31:02 | Compiling packages: atc/3f8900840b6e83a458c92a38868822d7a98f8769 (00:00:17) 20:32:56 | Compiling packages: postgresql_9.3/f8b7a26f105b921843fa5565818b50cc45ace6af (00:02:49) 20:32:56 | Creating missing vms: web/21ca23ff-57cf-4917-8f29-4c3cd65bca76 (0) 20:32:56 | Creating missing vms: worker/f2b9d34c-8691-4744-9213-093af0b1c08a (0) 20:32:56 | Creating missing vms: db/7894b167-7d44-4d45-99ca-36e397213441 (0) 20:33:03 | Creating missing vms: web/21ca23ff-57cf-4917-8f29-4c3cd65bca76 (0) (00:00:07) 20:33:04 | Creating missing vms: db/7894b167-7d44-4d45-99ca-36e397213441 (0) (00:00:08) 20:33:04 | Creating missing vms: worker/f2b9d34c-8691-4744-9213-093af0b1c08a (0) (00:00:08) 20:33:04 | Updating instance web: web/21ca23ff-57cf-4917-8f29-4c3cd65bca76 (0) (canary) 20:33:04 | Updating instance db: db/7894b167-7d44-4d45-99ca-36e397213441 (0) (canary) 20:33:04 | Updating instance worker: worker/f2b9d34c-8691-4744-9213-093af0b1c08a (0) (canary) 20:33:23 | Updating instance db: db/7894b167-7d44-4d45-99ca-36e397213441 (0) (canary) (00:00:19) 20:33:24 | Updating instance web: web/21ca23ff-57cf-4917-8f29-4c3cd65bca76 (0) (canary) (00:00:20) 20:33:40 | Updating instance worker: worker/f2b9d34c-8691-4744-9213-093af0b1c08a (0) (canary) (00:00:36) Started Sat Apr 22 20:29:07 UTC 2017 Finished Sat Apr 22 20:33:40 UTC 2017 Duration 00:04:33 Task 11 done Succeeded ``` できた。簡単。 ``` console $ curl -v -k https://10.244.0.200 * Rebuilt URL to: https://10.244.0.200/ * Trying 10.244.0.200... * Connected to 10.244.0.200 (10.244.0.200) port 443 (#0) * found 173 certificates in /etc/ssl/certs/ca-certificates.crt * found 692 certificates in /etc/ssl/certs * ALPN, offering http/1.1 * SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256 * server certificate verification SKIPPED * server certificate status verification SKIPPED * common name: 10.244.0.200 (matched) * server certificate expiration date OK * server certificate activation date OK * certificate public key: RSA * certificate version: #3 * subject: C=USA,O=Cloud Foundry,CN=10.244.0.200 * start date: Sat, 22 Apr 2017 20:39:36 GMT * expire date: Sun, 22 Apr 2018 20:39:36 GMT * issuer: C=USA,O=Cloud Foundry,CN=ca * compression: NULL * ALPN, server did not agree to a protocol > GET / HTTP/1.1 > Host: 10.244.0.200 > User-Agent: curl/7.47.0 > Accept: */* > < HTTP/1.1 200 OK < X-Content-Type-Options: nosniff < X-Download-Options: noopen < X-Xss-Protection: 1; mode=block < Date: Sat, 22 Apr 2017 21:08:10 GMT < Content-Type: text/html; charset=utf-8 < Transfer-Encoding: chunked < Concourse
* Connection #0 to host 10.244.0.200 left intact ``` ### HA Proxyで外部からアクセス可能にする 作りたいのは次のような環境(再掲)。 ![image](https://cloud.githubusercontent.com/assets/106908/25311223/ebeb411e-2835-11e7-9d6e-ca298a7a4061.png) 前述のインストールではConcourseのATCでTLS Terminationを行っていたが、今回はHA Proxy側でTLS Terminationを行い、Concourseには80番(HTTP)で転送する。 自己証明書の生成はBOSH v2 CLIで簡単にできる。 まずはConsourseにExternal IPを設定する。ここでは`concourse-192-168-11-210.sslip.io` (DNSで正引きすると`192.168.11.210`を返すIPアドレス)を使用する。 `concourse.yml`の次の2箇所を修正。 ``` yaml - name: atc release: concourse properties:      external_url: &external_url https://((external_ip)) # ここを変更       basic_auth_username: admin basic_auth_password: ((ui_password)) ``` と ``` yaml - name: atc_ssl type: certificate options: ca: default_ca common_name: ((external_ip)) # ここを変更 alternative_names: [((external_ip))] # ここを変更 ``` 次に`creds-concourse.yml`に先ほど生成した証明書が既に設定されているため、再生成するために、`creds-concourse.yml`から`atc_ssl.certificate`と`atc_ssl.private_key`、`atc_ssl.ca`を削除する。 削除したら、 ``` bosh2 interpolate concourse.yml --vars-store ./creds-concourse.yml -v internal_ip=10.244.0.200 -v external_ip=concourse-192-168-11-210.sslip.io ``` で再度証明書を生成。 生成された証明書(`atc_ssl.certificate`)と秘密鍵(`atc_ssl.private_key`)はそれぞれ ``` bosh2 interpolate --path=/atc_ssl/certificate creds-concourse.yml bosh2 interpolate --path=/atc_ssl/private_key creds-concourse.yml ``` で取得できる。これらを順番に並べた`server.pem`とファイルを作成して`/etc/haproxy/server.pem`におく。(空行を入れないように) Concourse側はもうTLSの設定は不要なので、`atc` jobの`properties`から`tls_cert`、`tls_key`、`tls_bind_port`を削除。 ``` yaml jobs: - name: atc release: concourse properties: external_url: &external_url https://((external_ip)) basic_auth_username: admin basic_auth_password: ((ui_password)) # tls_cert: ((atc_ssl.certificate)) # コメントアウトまたは削除 # tls_key: ((atc_ssl.private_key)) # コメントアウトまたは削除 bind_port: 80 # tls_bind_port: 443 # コメントアウトまたは削除 postgresql_database: &atc_db atc ``` これで`external_ip`も設定してBOSHにデプロイ。 ``` bosh2 deploy -d concourse --vars-store ./creds-concourse.yml -v internal_ip=10.244.0.200 -v external_ip=concourse-192-168-11-210.sslip.io concourse.yml ``` デプロイが完了すれば80番でATCにアクセスできることを確認。 ``` curl 10.244.0.200 ``` 最後にHA Proxyの設定。`/etc/haproxy/haproxy.cfg`に次の設定を追加。 今後、Concourse以外にもBOSH管理でHTTPSを受け付けるものがあると想定して、`Host`ヘッダーが`concourse-192-168-11-210.sslip.io`のものだけをConcourseの80番に転送する。 ``` frontend https_frontend bind *:443 ssl crt /etc/haproxy/server.pem reqadd X-Forwarded-Proto:\ https acl host_concourse_sslip hdr(host) concourse-192-168-11-210.sslip.io use_backend concourse_backend if host_concourse_sslip backend concourse_backend balance roundrobin server concourse 10.244.0.200:80 ``` BOSHでHTTPSをConcourseにしか使わない場合は`default_backend`でもOK。 ``` frontend https_frontend bind *:443 ssl crt /etc/haproxy/server.pem reqadd X-Forwarded-Proto:\ https default_backend concourse_backend backend concourse_backend balance roundrobin server concourse 10.244.0.200:80 ``` あとは ``` sudo service haproxy restart ``` これで [https://concourse-192-168-11-210.sslip.io](https://concourse-192-168-11-210.sslip.io)にアクセスできる。 ``` curl -k https://concourse-192-168-11-210.sslip.io ``` `Host`ヘッダーのチェックをしていれば、他のホスト名だとNGになる。 ``` console $ curl -k https://foo-192-168-11-210.sslip.io

503 Service Unavailable

No server is available to handle this request. ``` これでセットアップ環境。 HA Proxy版のmanifestは[こちら](https://github.com/making/bosh-manifests/blob/master/concourse-haproxy.yml)にpushしておいた。 Happy BOSH Life🐚を送れそう。