--- title: New BOSH CLI (Beta)でBOSH Lite環境作成 tags: ["BOSH", "BOSH-Lite"] categories: ["Dev", "Infrastructure", "BOSH"] date: 2017-01-22T16:49:19Z updated: 2017-06-04T05:47:08Z --- 分散ソフトウェアプロビジョニングツールである[BOSH](https://bosh.io)の[CLI](https://github.com/cloudfoundry/bosh-cli)がGo言語で書き直されていて、使い方が新しくなっていた。まだ正式リリース前だけど試したみた。 新BOSH CLIでBOSH Liteをセットアップしたメモ。 **目次** ### 事前準備 * VirtualBox 5.1.xをインストール https://www.virtualbox.org/wiki/Downloads * New BOSH CLI (Beta)のインストール https://bosh.io/docs/cli-v2.html バージョンは`2.0.13`を使用した。 ### BOSH Directorのセットアップ 旧CLIの`bosh-init`に相当する作業を行う。 以前はBOSH Liteの場合、Vagrantのboxファイルが用意されていたので`vagrant up`で起動したが、 新CLIではBOSH Lite用のDirectorインスタンスも`bosh create-env`コマンドで作成する。他のIaaSと同じようになって、セットアップ方法が統一された模様。 Directorセットアップ用のマニフェストファイルは https://github.com/cloudfoundry/bosh-deployment にある。 この記事では[`506260980fb43f60da3e405f5377b7d593950236`](https://github.com/cloudfoundry/bosh-deployment/tree/506260980fb43f60da3e405f5377b7d593950236)を使用した。 ``` git clone https://github.com/cloudfoundry/bosh-deployment ~/workspace/bosh-deployment mkdir -p ~/deployments/vbox cd ~/deployments/vbox bosh create-env ~/workspace/bosh-deployment/bosh.yml \ --state ./state.json \ -o ~/workspace/bosh-deployment/virtualbox/cpi.yml \ -o ~/workspace/bosh-deployment/virtualbox/outbound-network.yml \ -o ~/workspace/bosh-deployment/bosh-lite.yml \ -o ~/workspace/bosh-deployment/bosh-lite-runc.yml \ -o ~/workspace/bosh-deployment/jumpbox-user.yml \ --vars-store ./creds.yml \ -v director_name="Bosh Lite Director" \ -v internal_ip=192.168.50.6 \ -v internal_gw=192.168.50.1 \ -v internal_cidr=192.168.50.0/24 \ -v outbound_network_name=NatNetwork ``` cpiとstemcesllのダウンロードが遅くて、`bosh create-env`が完了するのに3時間以上かかった・・ `bosh.yml`が基本的なマニフェストで、環境差分は`-o`で指定するopsfileで基本の設定内容を部分的に書き換えている。 ちなみにどんなマニフェストファイルが使われているかは次のように`bosh interpolate`で補完結果が見れる。 ``` bosh interpolate ~/workspace/bosh-deployment/bosh.yml \ -o ~/workspace/bosh-deployment/virtualbox/cpi.yml \ -o ~/workspace/bosh-deployment/virtualbox/outbound-network.yml \ -o ~/workspace/bosh-deployment/bosh-lite.yml \ -o ~/workspace/bosh-deployment/bosh-lite-runc.yml \ -o ~/workspace/bosh-deployment/jumpbox-user.yml \ --vars-store ./creds.yml \ -v director_name="Bosh Lite Director" \ -v internal_ip=192.168.50.6 \ -v internal_gw=192.168.50.1 \ -v internal_cidr=192.168.50.0/24 \ -v outbound_network_name=NatNetwork ``` ちなみに`bosh create-env`のログはこんな感じだった。 ``` vbox $ bosh create-env ~/workspace/bosh-deployment/bosh.yml \ > --state ./state.json \ > -o ~/workspace/bosh-deployment/virtualbox/cpi.yml \ > -o ~/workspace/bosh-deployment/virtualbox/outbound-network.yml \ > -o ~/workspace/bosh-deployment/bosh-lite.yml \ > -o ~/workspace/bosh-deployment/bosh-lite-runc.yml \ > -o ~/workspace/bosh-deployment/jumpbox-user.yml \ > --vars-store ./creds.yml \ > -v director_name="Bosh Lite Director" \ > -v internal_ip=192.168.50.6 \ > -v internal_gw=192.168.50.1 \ > -v internal_cidr=192.168.50.0/24 \ > -v outbound_network_name=NatNetwork Deployment manifest: '/Users/makit/workspace/bosh-deployment/bosh.yml' Deployment state: './state.json' Started validating Downloading release 'bosh'... Finished (00:45:46) Validating release 'bosh'... Finished (00:00:00) Downloading release 'bosh-virtualbox-cpi'... Finished (00:03:24) Validating release 'bosh-virtualbox-cpi'... Finished (00:00:01) Downloading release 'bosh-warden-cpi'... Finished (00:23:51) Validating release 'bosh-warden-cpi'... Finished (00:00:00) Downloading release 'os-conf'... Finished (00:00:04) Validating release 'os-conf'... Finished (00:00:00) Downloading release 'garden-runc'... Finished (00:21:25) Validating release 'garden-runc'... Finished (00:00:00) Validating cpi release... Finished (00:00:00) Validating deployment manifest... Finished (00:00:00) Downloading stemcell... Finished (00:30:48) Validating stemcell... Finished (00:00:01) Finished validating (02:05:27) Started installing CPI Compiling package 'golang_1.7/21609f611781e8586e713cfd7ceb389cee429c5a'... Finished (00:00:11) Compiling package 'virtualbox_cpi/e293cbbb8359fd2cbbb9777b7b91fd142ab6c688'... Finished (00:00:08) Installing packages... Finished (00:00:01) Rendering job templates... Finished (00:00:00) Installing job 'virtualbox_cpi'... Finished (00:00:00) Finished installing CPI (00:00:02) Starting registry... Finished (00:00:00) Uploading stemcell 'bosh-vsphere-esxi-ubuntu-trusty-go_agent/3312.15'... Finished (00:00:09) Started deploying Creating VM for instance 'bosh/0' from stemcell 'sc-069969e4-a28f-4537-5f66-5ef7413851c2'... Finished (00:00:01) Waiting for the agent on VM 'vm-f3dd1dd7-961e-409d-7e23-2059bfb6177a' to be ready... Finished (00:00:35) Creating disk... Finished (00:00:00) Attaching disk 'disk-920919cf-f1c3-415d-5c52-3f32a4978ad3' to VM 'vm-f3dd1dd7-961e-409d-7e23-2059bfb6177a'... Finished (00:00:04) Rendering job templates... Finished (00:00:02) Compiling package 'libseccomp/7a54b27a61b42980935e863d7060dc5a076b44d0'... Skipped [Package already compiled] (00:00:00) Compiling package 'golang_1.7.1/91909d54d203acc915a4392b52c37716e15b5aff'... Skipped [Package already compiled] (00:00:00) Compiling package 'apparmor/c8e25d84146677878c699ddc5cdd893030acb26f'... Skipped [Package already compiled] (00:00:00) Compiling package 'ruby/589d4b05b422ac6c92ee7094fc2a402db1f2d731'... Skipped [Package already compiled] (00:00:00) Compiling package 'golang_1.7/21609f611781e8586e713cfd7ceb389cee429c5a'... Finished (00:00:15) Compiling package 's3cli/8cbc6ee1b5acaac18c63fafc5989bd6911c9be83'... Skipped [Package already compiled] (00:00:00) Compiling package 'mysql/b7e73acc0bfe05f1c6cbfd97bf92d39b0d3155d5'... Skipped [Package already compiled] (00:00:00) Compiling package 'libpq/09c8f60b87c9bd41b37b0f62159c9d77163f52b8'... Skipped [Package already compiled] (00:00:00) Compiling package 'davcli/5f08f8d5ab3addd0e11171f739f072b107b30b8c'... Skipped [Package already compiled] (00:00:00) Compiling package 'golang_1.7/c82ff355bb4bd412a4397dba778682293cd4f392'... Skipped [Package already compiled] (00:00:00) Compiling package 'runc/68f36fbe363fefa5ec8d44b48ee30a56ac6e1e0e'... Skipped [Package already compiled] (00:00:00) Compiling package 'nats/0155cf6be0305c9f98ba2e9e2503cd72da7c05c3'... Skipped [Package already compiled] (00:00:00) Compiling package 'virtualbox_cpi/e293cbbb8359fd2cbbb9777b7b91fd142ab6c688'... Finished (00:00:12) Compiling package 'shadow/7a5e46357a33cafc8400a8e3e2e1f6d3a1159cb6'... Skipped [Package already compiled] (00:00:00) Compiling package 'iptables/24e83997945f8817627223c6cee78ca9064f42d5'... Skipped [Package already compiled] (00:00:00) Compiling package 'busybox/fc652425c32d0dad62f45bca18e1899671e2e570'... Skipped [Package already compiled] (00:00:00) Compiling package 'nginx/21e909d27fa69b3b2be036cdf5b8b293c6800158'... Skipped [Package already compiled] (00:00:00) Compiling package 'postgres/4b9f6514001f7c3f7d4394920d6aced9435a3bbd'... Skipped [Package already compiled] (00:00:00) Compiling package 'director/3451fde97191ac240d10ea180b659ed55ee0ccba'... Skipped [Package already compiled] (00:00:00) Compiling package 'health_monitor/884a822dc2547735ac42b889654ddf9f074bb7e7'... Skipped [Package already compiled] (00:00:00) Compiling package 'tar/f2ea61c537d8eb8cb2d691ce51e8516b28fa5bb7'... Skipped [Package already compiled] (00:00:00) Compiling package 'warden_cpi/29ac97b841a747dc238277ffc7d6bf59a278fa37'... Skipped [Package already compiled] (00:00:00) Compiling package 'postgres-9.4/6c9e820cdfe15267c8f864f482c7fbed0943c6de'... Skipped [Package already compiled] (00:00:00) Compiling package 'guardian/c4acb6073abb4e17165253935c923dfbdfbfb188'... Skipped [Package already compiled] (00:00:00) Updating instance 'bosh/0'... Finished (00:00:20) Waiting for instance 'bosh/0' to be running... Finished (00:00:09) Running the post-start scripts 'bosh/0'... Finished (00:00:00) Finished deploying (00:01:48) Stopping registry... Finished (00:00:00) Cleaning up rendered CPI jobs... Finished (00:00:00) Succeeded ``` VirtualBoxを見るとDirector用のVMが起動しているのとStemcell用のテンプレートファイルがいることがわかる。 ![image](https://qiita-image-store.s3.amazonaws.com/0/1852/4c49bc69-7b36-2369-e84c-9ef4d6837297.png) また、`NatNetwork`という名前のNATネットワーク(`10.0.2.0/24`)と`vboxnet0`という名前のホストオンリーネットワークができていることがわかる。 ![image](https://qiita-image-store.s3.amazonaws.com/0/1852/e111d077-5f03-e087-11c5-a633ad773d7a.png) ![image](https://qiita-image-store.s3.amazonaws.com/0/1852/9d07c0e0-0c54-4751-d8b0-253056bba736.png) ![image](https://qiita-image-store.s3.amazonaws.com/0/1852/0028e3d6-14e4-fa46-2a01-9b5d7a766b1b.png) ![image](https://qiita-image-store.s3.amazonaws.com/0/1852/c2bf155f-a15f-1637-e68e-62facb858ead.png) `VBoxManage list runningvms --long`コマンドでも確認できる。 Credentials情報は`cred.yml`に自動生成されている。 Directorインスタンスは`192.168.50.6`であり、この環境のエイリアスを作成する。これは以前の`bosh target`相当。 ``` bosh alias-env vbox -e 192.168.50.6 --ca-cert <(bosh int ./creds.yml --path /director_ssl/ca) ``` これ以降は`bosh -e vbox [command]`を実行すれば良い。 ログインというか認証情報の環境変数設定をする。 ``` export BOSH_CLIENT=admin export BOSH_CLIENT_SECRET=`bosh int ./creds.yml --path /admin_password` ``` `-e vbox`を毎回つけるのが面倒くさい場合は、 ``` export BOSH_ENVIRONMENT=192.168.50.6 ``` も設定しておく。 BOSH Lite用のStemcellをDirectorに登録する。 ``` bosh upload-stemcell https://bosh.io/d/stemcells/bosh-warden-boshlite-ubuntu-trusty-go_agent ``` これもダウンロード速度が遅いととても時間がかかる。進捗が出なくて不安になる場合、次のように実行するとどのくらいダウンロードされているか分かって良い。 ``` wget --content-disposition https://bosh.io/d/stemcells/bosh-warden-boshlite-ubuntu-trusty-go_agent bosh -e vbox upload-stemcell ./bosh-stemcell-3312.15-warden-boshlite-ubuntu-trusty-go_agent.tgz ``` Cloud Config (BOSH Liteの環境情報)を設定。 ``` bosh -n update-cloud-config ~/workspace/bosh-deployment/warden/cloud-config.yml ``` ここまででBOSH Liteのセットアップ完了。 `vagrant up`より手間は多いが、細かい設定が可能なのと、他のIaaSでBOSHをセットアップするのとあまり変わらないので練習にも良いと思う。 ### サンプルBOSH Releaseのデプロイ BOSH Directorが起動したので、[サンプルアプリ](https://github.com/making-dev/hello-legacy-tomcat-boshrelease)をデプロイする。 ``` bosh -n -d hello-legacy-tomcat deploy <(wget -O- https://raw.githubusercontent.com/making-dev/hello-legacy-tomcat-boshrelease/master/manifest/simple.yml) ``` しばらくするとデプロイ完了する。 ログは ``` console --2017-04-23 03:18:44-- https://raw.githubusercontent.com/making-dev/hello-legacy-tomcat-boshrelease/master/manifest/simple.yml Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.88.133 Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.88.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 687 [text/plain] Saving to: 'STDOUT' - 100%[=========================================================================>] 687 --.-KB/s in 0s 2017-04-23 03:18:44 (130 MB/s) - written to stdout [687/687] Using environment '192.168.50.6' as client 'admin' Using deployment 'hello-legacy-tomcat' Task 2 18:18:45 | Downloading remote release: Downloading remote release (00:01:57) 18:20:42 | Verifying remote release: Verifying remote release (00:00:00) 18:20:42 | Extracting release: Extracting release (00:00:01) 18:20:43 | Verifying manifest: Verifying manifest (00:00:00) 18:20:43 | Resolving package dependencies: Resolving package dependencies (00:00:00) 18:20:43 | Creating new packages: hello-legacy-tomcat/9a3f61ae2d9ac62bed2d6ac8daf7c5e9d87b75a4 (00:00:00) 18:20:43 | Creating new packages: java/36c812837a1e61d7393c84a0a2c5270eb1ae51af (00:00:01) 18:20:44 | Creating new packages: tomcat/3055087aee0f5ee08ea6412b921c6d92f19797b5 (00:00:00) 18:20:44 | Creating new jobs: hello-legacy-tomcat/49a453ed9c90a68b4102a2adf616311fac196278 (00:00:00) 18:20:44 | Release has been created: hello-legacy-tomcat/0.2 (00:00:00) Started Sat Apr 22 18:18:45 UTC 2017 Finished Sat Apr 22 18:20:44 UTC 2017 Duration 00:01:59 Task 2 done + azs: + - name: z1 + - name: z2 + - name: z3 + vm_types: + - name: default + compilation: + az: z1 + network: default + reuse_compilation_vms: true + vm_type: default + workers: 5 + networks: + - name: default + subnets: + - azs: + - z1 + - z2 + - z3 + gateway: 10.244.0.1 + range: 10.244.0.0/24 + reserved: [] + static: + - 10.244.0.34 + type: manual + disk_types: + - disk_size: 1024 + name: default + stemcells: + - alias: ubuntu + os: ubuntu-trusty + version: '3363.19' + releases: + - name: hello-legacy-tomcat + sha1: a401254722d4b032f0170b1d17ca3546fd611524 + url: https://github.com/making-dev/hello-legacy-tomcat-boshrelease/releases/download/0.2/hello-legacy-tomcat-0.2.tgz + version: '0.2' + update: + canaries: 2 + canary_watch_time: 5000-60000 + max_in_flight: 1 + update_watch_time: 5000-60000 + instance_groups: + - azs: + - z1 + instances: 1 + jobs: + - name: hello-legacy-tomcat + properties: {} + release: hello-legacy-tomcat + name: hello-legacy-tomcat + networks: + - name: default + persistent_disk_type: default + stemcell: ubuntu + vm_type: default + name: hello-legacy-tomcat Task 3 18:20:45 | Preparing deployment: Preparing deployment (00:00:00) 18:20:45 | Preparing package compilation: Finding packages to compile (00:00:00) 18:20:45 | Compiling packages: java/36c812837a1e61d7393c84a0a2c5270eb1ae51af 18:20:45 | Compiling packages: tomcat/3055087aee0f5ee08ea6412b921c6d92f19797b5 (00:00:14) 18:20:59 | Compiling packages: hello-legacy-tomcat/9a3f61ae2d9ac62bed2d6ac8daf7c5e9d87b75a4 (00:00:01) 18:21:14 | Compiling packages: java/36c812837a1e61d7393c84a0a2c5270eb1ae51af (00:00:29) 18:21:15 | Creating missing vms: hello-legacy-tomcat/3677118a-33f4-41ce-bca0-174867a733fe (0) (00:00:14) 18:21:29 | Updating instance hello-legacy-tomcat: hello-legacy-tomcat/3677118a-33f4-41ce-bca0-174867a733fe (0) (canary) (00:00:30) Started Sat Apr 22 18:18:45 UTC 2017 Finished Sat Apr 22 18:21:59 UTC 2017 Duration 00:03:14 Task 3 done Succeeded ``` インスタンスの確認。 ``` bash $ bosh vms Using environment '192.168.50.6' as client 'admin' Task 7. Done Deployment 'hello-legacy-tomcat' Instance Process State AZ IPs VM CID VM Type hello-legacy-tomcat/6099534d-509b-4542-a529-1c399d789078 running z1 10.244.0.2 44356df0-0484-458c-6c19-f3d92bc86c21 default 1 vms Succeeded ``` `10.244.0.2`で起動していることがわかる。 サンプルアプリにアクセスする。`10.244.0.0/16`にアクセスするためにルーティングテーブルと追加する。 Macの場合 ``` sudo route add -net 10.244.0.0/16 192.168.50.6 ``` Linuxの場合 ``` sudo route add -net 10.244.0.0/16 gw 192.168.50.6 ``` Windowsの場合 ``` route add 10.244.0.0/16 192.168.50.6 ``` `hello-legacy-tomcat`にアクセス ``` bash $ curl 10.244.0.2:8080/env PATH: /bin:/usr/bin:/sbin:/usr/sbin:/var/vcap/packages/java/bin MONIT_SERVICE: hello-legacy-tomcat MONIT_PROCESS_PID: 0 JAVA_HOME: /var/vcap/packages/java MONIT_PROCESS_CHILDREN: 0 XFILESEARCHPATH: /usr/dt/app-defaults/%L/Dt MONIT_EVENT: Started MONIT_DESCRIPTION: Started NLSPATH: /usr/dt/lib/nls/msg/%L/%N.cat MONIT_PROCESS_CPU_PERCENT: 0 MONIT_HOST: localhost PWD: /etc/sv/monit MONIT_PROCESS_MEMORY: 0 SHLVL: 0 MONIT_DATE: Sun, 22 Jan 2017 16:07:04 +0000 ``` できた。 このブログの[BOSH系の記事](https://blog.ik.am/tags/BOSH/entries)、更新しないと・・・