<div dir="ltr">Hi Roman,<div><br></div><div>Thanks a lot for looking into this issue.</div><div><br></div><div>I re-ran the same process again and this time it successfully completed Genesis phase. The postgresql-0 and nfs-provisioner logs did not show any apparent errors.</div><div>So It seems to me there is a consistency issue because all I did was destroy the n0 vm and "vagrant up" again. It finally succeeded once out of 5 tries.</div><div><br></div><div>Btw, my virtualization environment is as below.</div><div><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:12pt;font-family:Calibri,sans-serif"><span lang="EN-US"> vagrant@n0:~$ sudo virt-what</span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:12pt;font-family:Calibri,sans-serif"><span lang="EN-US"> virtualbox</span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:12pt;font-family:Calibri,sans-serif"><span lang="EN-US"> kvm</span></p></div><div><br></div><div>However, the subsequent ./run_shipyard.sh commit configdocs failed as below.</div><div>How can this failure be fixed? </div><div><br></div><div>Since Genesis is complete, can we re-run the script without the Genesis part - i.e., skipping the Genesis part ?</div><div><br></div><div>Btw, can you describe your environment setup, so we can try to follow your exact execution environment? </div><div><br></div><div> (My environment: a physical server, installed ubuntu 16.04.5 , install virtualbox and vagrant, and run "vagrant up")</div><div><br></div><div>Thanks again,</div><div>Calvin</div><div><br></div><div><div>================================================= </div><div><div>==> n0: + export max_shipyard_count=60</div><div>==> n0: + max_shipyard_count=60</div><div>==> n0: + export shipyard_query_time=90</div><div>==> n0: + shipyard_query_time=90</div><div>==> n0: + bash execute_shipyard_action.sh deploy_site</div><div>==> n0: + run_action deploy_site</div><div>==> n0: + action=deploy_site</div><div>==> n0: + action_args=</div><div>==> n0: + NC='\033[0m'</div><div>==> n0: + RED='\033[0;31m'</div><div>==> n0: + GREEN='\033[0;32m'</div><div>==> n0: +++ dirname execute_shipyard_action.sh</div><div>==> n0: ++ cd .</div><div>==> n0: ++ pwd</div><div>==> n0: + DIR=/root/deploy/site</div><div>==> n0: + cd /root/deploy/site</div><div>==> n0: + source shipyard_docker_base_command.sh</div><div>==> n0: ++ NAMESPACE=ucp</div><div>==> n0: ++ SHIPYARD_IMAGE=<a href="http://quay.io/airshipit/shipyard:master" target="_blank">quay.io/airshipit/shipyard:master</a></div><div>==> n0: +++ cat</div><div>==> n0: Execute deploy_site Dag...</div><div>==> n0: ++ base_docker_command='sudo -E docker run -t --rm --net=host</div><div>==> n0: -e http_proxy=</div><div>==> n0: -e https_proxy=</div><div>==> n0: -e no_proxy=</div><div>==> n0: -e OS_AUTH_URL=<a href="http://keystone.ucp.svc.cluster.local/v3" target="_blank">http://keystone.ucp.svc.cluster.local:80/v3</a></div><div>==> n0: -e OS_USERNAME=shipyard</div><div>==> n0: -e OS_USER_DOMAIN_NAME=default</div><div>==> n0: -e OS_PASSWORD</div><div>==> n0: -e OS_PROJECT_DOMAIN_NAME=default</div><div>==> n0: -e OS_PROJECT_NAME=service'</div><div>==> n0: + echo -e 'Execute deploy_site Dag...\n'</div><div>==> n0: + sudo -E docker run -t --rm --net=host -e http_proxy= -e https_proxy= -e no_proxy= -e OS_AUTH_URL=<a href="http://keystone.ucp.svc./v3" target="_blank">http://keystone.ucp.svc./v3</a> -e OS_USERNAME=shipyard -e OS_USER_DOMAIN_NAME=default -e OS_PASSWORD -e OS_PROJECT_DOMAIN_NAME=default -e OS_PROJECT_NAME=servhipit/shipyard:master create action deploy_site</div><div>==> n0: Error: Unable to complete request to Airflow <======================== Failed</div><div>==> n0: Reason: Airflow could not be contacted properly by Shipyard.</div><div>==> n0: - Error: <class 'requests.exceptions.ConnectTimeout'></div><div>==> n0:</div><div>==> n0: #### Errors: 1, Warnings: 0, Infos: 0, Other: 0 ####</div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, May 13, 2019 at 10:13 PM Roman Gorshunov <<a href="mailto:paye600@gmail.com">paye600@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello Calvin,<br>
<br>
Seems like PosgreSQL database was not able to properly write data onto<br>
the disk. PostgreSQL runs as a postgresql-0 pod in ucp namespace, uses<br>
a persistent volume claim postgresql-data-postgresql-0, and persistent<br>
volume mounted via NFS.<br>
<br>
kubectl describe pod postgresql-0 -n ucp<br>
kubectl logs -n ucp postgresql-0<br>
kubectl -n ucp describe pvc postgresql-data-postgresql-0<br>
kubectl describe pv pvc-0382c985-7572-11e9-b431-525400681552 # volume<br>
name could be different)<br>
<br>
NFS is provisioned by nfs-provisioner-7799d64d59-ptsgk (last two parts<br>
would be different in your case):<br>
kubectl get pods -n kube-system | grep nfs<br>
kubectl -n kube-system describe pod nfs-provisioner-7799d64d59-ptsgk<br>
kubectl -n kube-system logs nfs-provisioner-7799d64d59-ptsgk<br>
<br>
Check if there are any problems with it (e.g. unable to mount NFS<br>
share, or lack of free storage space - `df -h`).<br>
<br>
Also running kubectl get events --all-namespaces could help to<br>
understand what went wrong.<br>
<br>
I have run an AIAB installation today twice, and it all worked fine. I<br>
use `vagrant up` and my hypervisor is KVM, if that could help you.<br>
<br>
I hope it helps.<br>
<br>
Best regards,<br>
-- Roman Gorshunov<br>
<br>
On Fri, May 10, 2019 at 7:01 AM calvin whole <<a href="mailto:calvinwhole@gmail.com" target="_blank">calvinwhole@gmail.com</a>> wrote:<br>
><br>
> Hi Roman,<br>
><br>
> Not sure if my last email were out properly, its size is too big. Here is a short one. Thanks for responding in advance.<br>
><br>
> I re-ran the "vagrant up" and looking into the logs for "deckhand-db-init-zs499" as showed below.<br>
> It showed ERROR: checkpoint request failed<br>
> HINT: Consult recent messages in the server log for details.<br>
><br>
> What is the specific "server" log we should look into for details?<br>
><br>
> Thanks for help.<br>
><br>
> Sincerely,<br>
> Calvin<br>
><br>
><br>
> On Thu, May 9, 2019 at 12:17 PM calvin whole <<a href="mailto:calvinwhole@gmail.com" target="_blank">calvinwhole@gmail.com</a>> wrote:<br>
>><br>
>> Hi Roman,<br>
>><br>
>> Btw, continue my last post, the kubectl describe pod deckhand-db-init-zs499 output is as follows.<br>
>><br>
>> Thanks,<br>
>> Calvin<br>
>> =========== kubectl describe pod deckhand-db-init-zs499 =================<br>
>> root@n0:/home/vagrant# kubectl describe pod deckhand-db-init-zs499 -n ucp<br>
>> Name: deckhand-db-init-zs499<br>
>> Namespace: ucp<br>
>> Node: n0/<a href="http://10.0.2.15" rel="noreferrer" target="_blank">10.0.2.15</a><br>
>> Start Time: Thu, 09 May 2019 03:48:48 +0000<br>
>> Labels: application=deckhand<br>
>> component=db-init<br>
>> controller-uid=59f1bee0-720d-11e9-92ac-080027fc876e<br>
>> job-name=deckhand-db-init<br>
>> release_group=airship-ucp-deckhand<br>
>> Annotations: <none><br>
>> Status: Running<br>
>> IP: 10.97.26.50<br>
>> Controlled By: Job/deckhand-db-init<br>
>> Init Containers:<br>
>> init:<br>
>> Container ID: docker://b58e8b6b7296df618cb8120b5226370afeba2a4e79dd70ee6894b5afd853c0db<br>
>> Image: <a href="http://quay.io/stackanetes/kubernetes-entrypoint:v0.3.1" rel="noreferrer" target="_blank">quay.io/stackanetes/kubernetes-entrypoint:v0.3.1</a><br>
>> Image ID: docker-pullable://<a href="http://quay.io/stackanetes/kubernetes-entrypoint@sha256:32b1b657ee4bcc9cc7a1529e31d8e1a06376172373ee020f97f3e78168fde4b6" rel="noreferrer" target="_blank">quay.io/stackanetes/kubernetes-entrypoint@sha256:32b1b657ee4bcc9cc7a1529e31d8e1a06376172373ee020f97f3e78168fde4b6</a><br>
>> Port: <none><br>
>> Host Port: <none><br>
>> Command:<br>
>> kubernetes-entrypoint<br>
>> State: Terminated<br>
>> Reason: Completed<br>
>> Exit Code: 0<br>
>> Started: Thu, 09 May 2019 03:48:52 +0000<br>
>> Finished: Thu, 09 May 2019 03:48:54 +0000<br>
>> Ready: True<br>
>> Restart Count: 0<br>
>> Environment:<br>
>> POD_NAME: deckhand-db-init-zs499 (v1:<a href="http://metadata.name" rel="noreferrer" target="_blank">metadata.name</a>)<br>
>> NAMESPACE: ucp (v1:metadata.namespace)<br>
>> INTERFACE_NAME: eth0<br>
>> PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/<br>
>> DEPENDENCY_SERVICE: ucp:postgresql<br>
>> DEPENDENCY_DAEMONSET:<br>
>> DEPENDENCY_CONTAINER:<br>
>> DEPENDENCY_POD_JSON:<br>
>> COMMAND: echo done<br>
>> Mounts:<br>
>> /var/run/secrets/<a href="http://kubernetes.io/serviceaccount" rel="noreferrer" target="_blank">kubernetes.io/serviceaccount</a> from deckhand-db-init-token-gczr5 (ro)<br>
>> Containers:<br>
>> deckhand-db-init:<br>
>> Container ID: docker://5dea2aa975c3718ca298536005b9cc0b21de47e08b2260cc73005e3455bb1350<br>
>> Image: <a href="http://docker.io/postgres:9.5" rel="noreferrer" target="_blank">docker.io/postgres:9.5</a><br>
>> Image ID: docker-pullable://postgres@sha256:0605b4b20a205c09ddd10eeeddd3ed7bf3cc442a8e9896ec34862ca882658be4<br>
>> Port: <none><br>
>> Host Port: <none><br>
>> Command:<br>
>> /tmp/db-init.sh<br>
>> State: Waiting<br>
>> Reason: CrashLoopBackOff<br>
>> Last State: Terminated<br>
>> Reason: Error <========<br>
>> Exit Code: 1<br>
>> Started: Thu, 09 May 2019 04:10:29 +0000<br>
>> Finished: Thu, 09 May 2019 04:10:30 +0000<br>
>> Ready: False<br>
>> Restart Count: 9<br>
>> Environment:<br>
>> DECKHAND_DB_URL: <set to the key 'DATABASE_URI' in secret 'deckhand-db-user'> Optional: false<br>
>> DB_NAME: <set to the key 'DATABASE_NAME' in secret 'deckhand-db-user'> Optional: false<br>
>> DB_SERVICE_USER: <set to the key 'DATABASE_USERNAME' in secret 'deckhand-db-user'> Optional: false<br>
>> DB_SERVICE_PASSWORD: <set to the key 'DATABASE_PASSWORD' in secret 'deckhand-db-user'> Optional: false<br>
>> DB_FQDN: <set to the key 'DATABASE_HOST' in secret 'deckhand-db-user'> Optional: false<br>
>> DB_PORT: <set to the key 'DATABASE_PORT' in secret 'deckhand-db-user'> Optional: false<br>
>> DB_ADMIN_USER: <set to the key 'DATABASE_USERNAME' in secret 'deckhand-db-admin'> Optional: false<br>
>> PGPASSWORD: <set to the key 'DATABASE_PASSWORD' in secret 'deckhand-db-admin'> Optional: false<br>
>> Mounts:<br>
>> /etc/deckhand from etc-deckhand (rw)<br>
>> /etc/deckhand/deckhand.conf from deckhand-etc (ro)<br>
>> /tmp/db-init.sh from deckhand-bin (ro)<br>
>> /var/run/secrets/<a href="http://kubernetes.io/serviceaccount" rel="noreferrer" target="_blank">kubernetes.io/serviceaccount</a> from deckhand-db-init-token-gczr5 (ro)<br>
>> Conditions:<br>
>> Type Status<br>
>> Initialized True<br>
>> Ready False<br>
>> PodScheduled True<br>
>> Volumes:<br>
>> etc-deckhand:<br>
>> Type: EmptyDir (a temporary directory that shares a pod's lifetime)<br>
>> Medium:<br>
>> deckhand-etc:<br>
>> Type: Secret (a volume populated by a Secret)<br>
>> SecretName: deckhand-etc<br>
>> Optional: false<br>
>> deckhand-bin:<br>
>> Type: ConfigMap (a volume populated by a ConfigMap)<br>
>> Name: deckhand-bin<br>
>> Optional: false<br>
>> deckhand-db-init-token-gczr5:<br>
>> Type: Secret (a volume populated by a Secret)<br>
>> SecretName: deckhand-db-init-token-gczr5<br>
>> Optional: false<br>
>> QoS Class: BestEffort<br>
>> Node-Selectors: ucp-control-plane=enabled<br>
>> Tolerations: <a href="http://node.kubernetes.io/not-ready:NoExecute" rel="noreferrer" target="_blank">node.kubernetes.io/not-ready:NoExecute</a> for 300s<br>
>> <a href="http://node.kubernetes.io/unreachable:NoExecute" rel="noreferrer" target="_blank">node.kubernetes.io/unreachable:NoExecute</a> for 300s<br>
>> Events:<br>
>> Type Reason Age From Message<br>
>> ---- ------ ---- ---- -------<br>
>> Normal Scheduled 24m default-scheduler Successfully assigned deckhand-db-init-zs499 to n0<br>
>> Normal SuccessfulMountVolume 24m kubelet, n0 MountVolume.SetUp succeeded for volume "etc-deckhand"<br>
>> Normal SuccessfulMountVolume 24m kubelet, n0 MountVolume.SetUp succeeded for volume "deckhand-bin"<br>
>> Normal SuccessfulMountVolume 24m kubelet, n0 MountVolume.SetUp succeeded for volume "deckhand-etc"<br>
>> Normal SuccessfulMountVolume 24m kubelet, n0 MountVolume.SetUp succeeded for volume "deckhand-db-init-token-gczr5"<br>
>> Normal Pulled 24m kubelet, n0 Container image "<a href="http://quay.io/stackanetes/kubernetes-entrypoint:v0.3.1" rel="noreferrer" target="_blank">quay.io/stackanetes/kubernetes-entrypoint:v0.3.1</a>" already present on machine<br>
>> Normal Created 24m kubelet, n0 Created container<br>
>> Normal Started 24m kubelet, n0 Started container<br>
>> Normal Pulled 23m (x4 over 24m) kubelet, n0 Container image "<a href="http://docker.io/postgres:9.5" rel="noreferrer" target="_blank">docker.io/postgres:9.5</a>" already present on machine<br>
>> Normal Created 23m (x4 over 24m) kubelet, n0 Created container<br>
>> Normal Started 23m (x4 over 24m) kubelet, n0 Started container<br>
>> Warning BackOff 4m (x90 over 24m) kubelet, n0 Back-off restarting failed container<br>
>> root@n0:/home/vagrant#<br>
>><br>
>> On Thu, May 9, 2019 at 12:08 PM calvin whole <<a href="mailto:calvinwhole@gmail.com" target="_blank">calvinwhole@gmail.com</a>> wrote:<br>
>>><br>
>>> Hi Roman，<br>
>>><br>
>>> Thanks for looking into this and gave us suggestions.<br>
>>><br>
>>> I re-ran the "vagrant up" and looking into the logs for "deckhand-db-init-zs499" as showed below.<br>
>>> It showed ERROR: checkpoint request failed<br>
>>> HINT: Consult recent messages in the server log for details.<br>
>>><br>
>>> What is the specific "server" log we should look into for details?<br>
>>><br>
>>> Thanks for help.<br>
>>><br>
>>> Sincerely,<br>
>>> Calvin<br>
>>><br>
>>> ================== log for deckhand-db-init-zs499 ==================================<br>
>>> root@n0:/home/vagrant# kubectl logs deckhand-db-init-zs499 -n ucp<br>
>>> + export HOME=/tmp<br>
>>> + HOME=/tmp<br>
>>> + pgsql_superuser_cmd 'SELECT 1 FROM pg_database WHERE datname = '\''deckhand'\'''<br>
>>> + grep -q 1<br>
>>> + DB_COMMAND='SELECT 1 FROM pg_database WHERE datname = '\''deckhand'\'''<br>
>>> + [[ ! -z '' ]]<br>
>>> + psql -h postgresql.ucp.svc.cluster.local -p 5432 -U postgres '--command=SELECT 1 FROM pg_database WHERE datname = '\''deckhand'\'''<br>
>>> + pgsql_superuser_cmd 'CREATE DATABASE deckhand'<br>
>>> + DB_COMMAND='CREATE DATABASE deckhand'<br>
>>> + [[ ! -z '' ]]<br>
>>> + psql -h postgresql.ucp.svc.cluster.local -p 5432 -U postgres '--command=CREATE DATABASE deckhand'<br>
>>> ERROR: checkpoint request failed<br>
>>> HINT: Consult recent messages in the server log for details.<br>
>>><br>
>>> =====================================================================================<br>
>>> ==> n0: NAMESPACE NAME READY STATUS RESTARTS AGE<br>
>>> ==> n0: kube-system auxiliary-etcd-n0 3/3 Running 0 49m<br>
>>> ==> n0: kube-system bootstrap-armada-n0 4/4 Running 0 49m<br>
>>> ==> n0: kube-system calico-etcd-anchor-ncl2p 1/1 Running 0 47m<br>
>>> ==> n0: kube-system calico-etcd-n0 1/1 Running 0 46m<br>
>>> ==> n0: kube-system calico-kube-controllers-56c54d8cf8-5csnn 1/1 Running 0 46m<br>
>>> ==> n0: kube-system calico-node-m4rtf 1/1 Running 0 46m<br>
>>> ==> n0: kube-system calico-settings-tkp6r 0/1 Completed 0 46m<br>
>>> ==> n0: kube-system coredns-84bdd76f4d-hhbcs 1/1 Running 0 44m<br>
>>> ==> n0: kube-system coredns-84bdd76f4d-k8tcc 1/1 Running 0 44m<br>
>>> ==> n0: kube-system coredns-84bdd76f4d-qp2xd 1/1 Running 0 44m<br>
>>> ==> n0: kube-system haproxy-n0 1/1 Running 0 50m<br>
>>> ==> n0: kube-system ingress-error-pages-7c65f766d-dn2tw 1/1 Running 0 41m<br>
>>> ==> n0: kube-system ingress-gtvp8 2/2 Running 0 41m<br>
>>> ==> n0: kube-system kubernetes-apiserver-anchor-99jhn 1/1 Running 0 42m<br>
>>> ==> n0: kube-system kubernetes-apiserver-n0 1/1 Running 0 41m<br>
>>> ==> n0: kube-system kubernetes-controller-manager-anchor-vqddp 1/1 Running 0 42m<br>
>>> ==> n0: kube-system kubernetes-controller-manager-n0 1/1 Running 0 41m<br>
>>> ==> n0: kube-system kubernetes-etcd-anchor-9jcpl 1/1 Running 0 44m<br>
>>> ==> n0: kube-system kubernetes-etcd-n0 1/1 Running 0 42m<br>
>>> ==> n0: kube-system kubernetes-proxy-2m9t2 1/1 Running 0 47m<br>
>>> ==> n0: kube-system kubernetes-scheduler-anchor-nl9fb 1/1 Running 0 42m<br>
>>> ==> n0: kube-system kubernetes-scheduler-n0 1/1 Running 0 41m<br>
>>> ==> n0: kube-system nfs-provisioner-7799d64d59-vtkbd 1/1 Running 0 40m<br>
>>> ==> n0: kube-system tiller-deploy-7d88c6f956-qwfzb 1/1 Running 0 27m<br>
>>> ==> n0: ucp airship-ucp-keystone-memcached-memcached-74d79d8896-vfl69 1/1 Running 0 34m<br>
>>> ==> n0: ucp airship-ucp-rabbitmq-rabbitmq-0 1/1 Running 0 39m<br>
>>> ==> n0: ucp armada-api-d5f757d5-6wl98 1/1 Running 0 15m<br>
>>> ==> n0: ucp armada-ks-endpoints-vl9rs 0/3 Completed 0 15m<br>
>>> ==> n0: ucp armada-ks-service-vpcjd 0/1 Completed 0 15m<br>
>>> ==> n0: ucp armada-ks-user-rv4gs 0/1 Completed 0 15m<br>
>>> ==> n0: ucp barbican-api-5d7b88d8ff-8dd6w 1/1 Running 0 13m<br>
>>> ==> n0: ucp barbican-db-init-gqvt4 0/1 Completed 0 13m<br>
>>> ==> n0: ucp barbican-db-sync-tqtgq 0/1 Completed 0 13m<br>
>>> ==> n0: ucp barbican-ks-endpoints-rwtql 0/3<br>
>>> ==> n0: Completed 0 13m<br>
>>> ==> n0: ucp barbican-ks-service-l2h6h 0/1 Completed 0 13m<br>
>>> ==> n0: ucp barbican-ks-user-wwvc7 0/1 Completed 0 13m<br>
>>> ==> n0: ucp barbican-rabbit-init-6spq4 0/1 Completed 0 13m<br>
>>> ==> n0: ucp deckhand-api-78b9644f96-5686f 0/1 Running 0 11m<br>
>>> ==> n0: ucp deckhand-db-init-zs499 0/1 CrashLoopBackOff 7 11m <===<br>
>>> ==> n0: ucp deckhand-db-sync-ct7wl 0/1 Init:0/1 0 11m<br>
>>> ==> n0: ucp deckhand-ks-endpoints-x4hd9 0/3 Completed 0 11m<br>
>>> ==> n0: ucp deckhand-ks-service-ms6n5 0/1 Completed 0 11m<br>
>>> ==> n0: ucp deckhand-ks-user-7fnvt 0/1 Completed 0 11m<br>
>>> ==> n0: ucp divingbell-apparmor-default-hth8z 1/1 Running 0 27m<br>
>>> ==> n0: ucp divingbell-apt-default-r965m 1/1 Running 0 27m<br>
>>> ==> n0: ucp divingbell-ethtool-default-ldcmc 1/1 Running 0 27m<br>
>>> ==> n0: ucp divingbell-exec-default-f7h7x 1/1 Running 0 27m<br>
>>> ==> n0: ucp divingbell-limits-default-sp9mj 1/1 Running 0 27m<br>
>>> ==> n0: ucp divingbell-mounts-default-8f5a00a2-frbl2 1/1 Running 0 27m<br>
>>> ==> n0: ucp divingbell-perm-default-d7wxp 1/1 Running 0 27m<br>
>>> ==> n0: ucp divingbell-sysctl-default-c8pnp 1/1 Running 0 27m<br>
>>> ==> n0: ucp divingbell-uamlite-default-rfct6 1/1 Running 0 27m<br>
>>> ==> n0: ucp ingress-86576d6599-mdgj4 1/1 Running 0 39m<br>
>>> ==> n0: ucp ingress-error-pages-5c97bb46bb-7lg5l 1/1 Running 0 39m<br>
>>> ==> n0: ucp keystone-api-678fc44bdd-594bb 1/1 Running 0 34m<br>
>>> ==> n0: ucp keystone-bootstrap-rprr6 0/1 Completed 0 34m<br>
>>> ==> n0: ucp keystone-credential-setup-zkjgs 0/1 Completed 0 34m<br>
>>> ==> n0: ucp keystone-db-init-xkgxm 0/1 Completed 0 34m<br>
>>> ==> n0: ucp keystone-db-sync-lm6xs 0/1 Completed 0 34m<br>
>>> ==> n0: ucp keystone-domain-manage-9pzjq 0/1 Completed 0 34m<br>
>>> ==> n0: ucp keystone-fernet-setup-q7t8p 0/1 Completed 0 34m<br>
>>> ==> n0: ucp keystone-rabbit-init-qpvgt 0/1 Completed 0 34m<br>
>>> ==> n0: ucp maas-bootstrap-admin-user-8npgw 0/1 Completed 0 26m<br>
>>> ==> n0: ucp maas-db-init-9z86n 0/1 Completed 0 26m<br>
>>> ==> n0: ucp maas-db-sync-r7rkg 0/1 Completed 0 26m<br>
>>> ==> n0: ucp maas-export-api-key-n2gz4 0/1 Completed 1 26m<br>
>>> ==> n0: ucp maas-import-resources-prlml 0/1 Completed 0 26m<br>
>>> ==> n0: ucp maas-ingress-756f6f9d6-h65nj 2/2 Running 0 26m<br>
>>> ==> n0: ucp maas-ingress-errors-8686d56d98-swfg9<br>
>>> ==> n0: 1/1 Running 0 26m<br>
>>> ==> n0: ucp maas-rack-0 1/1 Running 0 26m<br>
>>> ==> n0: ucp maas-region-0 1/1 Running 0 26m<br>
>>> ==> n0: ucp mariadb-ingress-55794d94c8-dsw5w 1/1 Running 0 39m<br>
>>> ==> n0: ucp mariadb-ingress-55794d94c8-jczmh 1/1 Running 0 39m<br>
>>> ==> n0: ucp mariadb-ingress-error-pages-85f96fbd-jrqsg 1/1 Running 0 39m<br>
>>> ==> n0: ucp mariadb-server-0 1/1 Running 0 39m<br>
>>> ==> n0: ucp postgresql-0 1/1 Running 1 39m<br>
>>><br>
>>><br>
>>> On Tue, May 7, 2019 at 6:25 PM Roman Gorshunov <<a href="mailto:paye600@gmail.com" target="_blank">paye600@gmail.com</a>> wrote:<br>
>>>><br>
>>>> Hello Calvin,<br>
>>>><br>
>>>> Try to get some kubectl logs and describe deckhand-db-init-r9jvg pod.<br>
>>>> kubectl describe pod deckhand-db-init-r9jvg -u ucp<br>
>>>> May be it would help to understand what is happening there.<br>
>>>><br>
>>>> Thank you for trying Airship.<br>
>>>><br>
>>>> Best regards,<br>
>>>> -- Roman Gorshunov<br>
>>>><br>
>>>> On Tue, May 7, 2019 at 7:51 AM calvin whole <<a href="mailto:calvinwhole@gmail.com" target="_blank">calvinwhole@gmail.com</a>> wrote:<br>
>>>> ><br>
>>>> > Hi,<br>
>>>> ><br>
>>>> > We are trying to deploy AIIB.<br>
>>>> ><br>
>>>> > I have a physical server with Ubuntu 16.04.5 OS, installed virtualbox and vagrant.<br>
>>>> > The process is straightforward by following <a href="https://opendev.org/airship/in-a-bottle/" rel="noreferrer" target="_blank">https://opendev.org/airship/in-a-bottle/</a><br>
>>>> > We created ~/deploy directory, downloaded Vagrantfile, and do "vagrant up".<br>
>>>> ><br>
>>>> > However it stuck in the error below:<br>
>>>> > deckhand-db-init-r9jvg 0/1 CrashLoopBackOff 16<br>
>>>> ><br>
>>>> > Could anyone help to resolve this? Many thanks in advance.<br>
>>>> ><br>
>>>> > Sincerely,<br>
>>>> > Calvin<br>
>>>> ><br>
>>>> > ==> n0: NAMESPACE NAME READY STATUS RESTARTS AGE<br>
>>>> > ==> n0: kube-system auxiliary-etcd-n0 3/3 Running 0 1h<br>
>>>> > ==> n0: kube-system bootstrap-armada-n0 4/4 Running 0 1h<br>
>>>> > ==> n0: kube-system calico-etcd-anchor-5tqhk 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system calico-etcd-n0 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system calico-kube-controllers-56c54d8cf8-5ssl6 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system calico-node-pbsxh 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system calico-settings-lzpk9 0/1 Completed 0 1h<br>
>>>> > ==> n0: kube-system coredns-84bdd76f4d-6cwnl 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system coredns-84bdd76f4d-d4p8c 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system coredns-84bdd76f4d-xrknz 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system haproxy-n0 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system ingress-9pkmx 2/2 Running 0 1h<br>
>>>> > ==> n0: kube-system ingress-error-pages-7c65f766d-2pqfx 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system kubernetes-apiserver-anchor-hszbf 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system kubernetes-apiserver-n0 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system kubernetes-controller-manager-anchor-h49vz 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system kubernetes-controller-manager-n0 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system kubernetes-etcd-anchor-nnjbb 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system kubernetes-etcd-n0 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system kubernetes-proxy-vgzjp 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system kubernetes-scheduler-anchor-bq2gk 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system kubernetes-scheduler-n0 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system nfs-provisioner-7799d64d59-jx7hq 1/1 Running 0 1h<br>
>>>> > ==> n0: kube-system tiller-deploy-7d88c6f956-d9kzg 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp airship-ucp-keystone-memcached-memcached-74d79d8896-q9wqx 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp airship-ucp-rabbitmq-rabbitmq-0 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp armada-api-d5f757d5-d9l9h 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp armada-ks-endpoints-qwbtg 0/3 Completed 0 1h<br>
>>>> > ==> n0: ucp armada-ks-service-lg8kq 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp armada-ks-user-g2j6v 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp barbican-api-84665dd99d-qv5fz 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp barbican-db-init-ndx58 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp barbican-db-sync-sh7c9 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp barbican-ks-endpoints-bv7xv 0/3 Completed 0 1h<br>
>>>> > ==> n0: ucp barbican-ks-service-46hjk 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp barbican-ks-user-6df74 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp barbican-rabbit-init-gnvfl 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp deckhand-api-6cd9c4479d-wc5cw 0/1 Running 0 1h<br>
>>>> > ==> n0: ucp deckhand-db-init-r9jvg 0/1 CrashLoopBackOff 17 1h <=====<br>
>>>> > ==> n0: ucp deckhand-db-sync-llstv 0/1 Init:0/1 0 1h<br>
>>>> > ==> n0: ucp deckhand-ks-endpoints-4gqfj 0/3 Completed 0 1h<br>
>>>> > ==> n0: ucp deckhand-ks-service-c6gbq 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp deckhand-ks-user-5skng 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp divingbell-apparmor-default-lkcl6 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp divingbell-apt-default-7jgtv 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp divingbell-ethtool-default-tm2w4 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp divingbell-exec-default-l45m8 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp divingbell-limits-default-q84pr 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp divingbell-mounts-default-29420945-nrdsz 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp divingbell-perm-default-wdgld 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp divingbell-sysctl-default-t7f2m 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp divingbell-uamlite-default-fc4jx 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp ingress-86576d6599-q8ng4 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp ingress-error-pages-5c97bb46bb-pjz9m 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp keystone-api-678fc44bdd-ncxc2 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp keystone-bootstrap-28l4g 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp keystone-credential-setup-rq5d4 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp keystone-db-init-z8x4w 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp keystone-db-sync-9hvb5 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp keystone-domain-manage-tzcnf 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp keystone-fernet-setup-bzdpb 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp keystone-rabbit-init-cxpc6 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp maas-bootstrap-admin-user-g99rl 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp maas-db-init-h4llm 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp maas-db-sync-6tsqj 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp maas-export-api-key-c8rdb 0/1 Completed 0 1h<br>
>>>> > ==> n0: ucp maas-import-resources-hhq7f 0/1 Completed 1 1h<br>
>>>> > ==> n0: ucp maas-ingress-756f6f9d6-dpcp9 2/2 Running 0 1h<br>
>>>> > ==> n0: ucp maas-ingress-errors-8686d56d98-jr6xx 1/1 Running 0 1h<br>
>>>> > ==> n0: u<br>
>>>> > ==> n0: cp maas-rack-0 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp maas-region-0 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp mariadb-ingress-55794d94c8-mhjjf 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp mariadb-ingress-55794d94c8-vglbv 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp mariadb-ingress-error-pages-85f96fbd-28cdv 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp mariadb-server-0 1/1 Running 0 1h<br>
>>>> > ==> n0: ucp postgresql-0 1/1 Running 1 1h<br>
>>>> > _______________________________________________<br>
>>>> > Airship-discuss mailing list<br>
>>>> > <a href="mailto:Airship-discuss@lists.airshipit.org" target="_blank">Airship-discuss@lists.airshipit.org</a><br>
>>>> > <a href="http://lists.airshipit.org/cgi-bin/mailman/listinfo/airship-discuss" rel="noreferrer" target="_blank">http://lists.airshipit.org/cgi-bin/mailman/listinfo/airship-discuss</a><br>
</blockquote></div>