[Airship-discuss] scaled down version of site/airship-seaworthy <airship-treasuremap problems>
MCEUEN, MATT
MM9745 at att.com
Thu Sep 6 17:44:50 UTC 2018
Hey Jan-Erik!
If your lab is still in this state, could you check the pod logs for ceph-rgw-694b44bd5d-ckv8v and share what you see? Looks like that guy is Running but not completed (as of the time of the logs below).
Please check `kubectl -n ceph exec ceph-mon-lv7hw -- ceph -s` to see the ceph status.
We are in progress of updating the treasuremap readthedocs content, stay tuned!
Thanks,
Matt
-----Original Message-----
From: Jan-Erik Mångs <kire at kth.se>
Sent: Monday, September 3, 2018 3:50 PM
To: airship-discuss at lists.airshipit.org
Subject: [Airship-discuss] scaled down version of site/airship-seaworthy <airship-treasuremap problems>
(reposting again in plain text, my first attempt was too big >.40 KB due to formatting)
We're playing around with https://urldefense.proofpoint.com/v2/url?u=http-3A__git.openstack.org_cgit_openstack_airship-2Dtreasuremap_&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=MyMpkoXeJDDM5X1b0oakc6awcXcRf74kWEKFp5iQp9w&s=i-lEaji6J8bUrRsLcZQNEzciBuYCTkRQAdOil7Lajow&e= and we're having some problems.
We're trying to deploy a scaled down version of site/airship-seaworthy (a smaller "template" could come in handy, who has this many disks in their lab?). We're only using four servers (Dell 630s), with three SSDs/server, networking according to seaworthy etc. We've copied site/airship-seaworthy to our own site and tried to modify things to the best of our knowledge.
When we run the genesis-script on the genesis node we see an error related to mariadb, see below. Any hints on what could be wrong are much appreciated. (Storage is always among the most difficult things, I really need to learn ceph :-)).
(BTW, instructions here https://urldefense.proofpoint.com/v2/url?u=https-3A__treasuremap.readthedocs.io_en_latest_deployment.html&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=MyMpkoXeJDDM5X1b0oakc6awcXcRf74kWEKFp5iQp9w&s=MKTFp8gJE_JdUc2rKHp0oTUezN9h1rlsW0FU-RXDbXA&e= does not seem to be in sync with the repo above)
Thanks for your help
/Jan-Erik (et al.)
Output from the genesis script:
NAMESPACE NAME READY STATUS RESTARTS AGE
ceph ceph-bootstrap-vb495 0/1 Completed 0 32m
ceph ceph-cephfs-client-key-generator-jvc62 0/1 Completed 0 21m
ceph ceph-cephfs-provisioner-cdfb7cfbf-5pb88 1/1 Running 0 21m
ceph ceph-cephfs-provisioner-cdfb7cfbf-789vh 1/1 Running 0 21m
ceph ceph-mds-6c757f9549-fqw9v 1/1 Running 0 21m
ceph ceph-mds-keyring-generator-p2hzh 0/1 Completed 0 32m
ceph ceph-mgr-5f985c796-fj6ql 1/1 Running 0 21m
ceph ceph-mgr-keyring-generator-2xvvm 0/1 Completed 0 32m
ceph ceph-mon-check-84c94dc8ff-wgblt 1/1 Running 0 32m
ceph ceph-mon-keyring-generator-rln66 0/1 Completed 0 32m
ceph ceph-mon-lv7hv 2/2 Running 0 32m
ceph ceph-osd-default-83945928-xl2g5 2/2 Running 0 29m
ceph ceph-osd-keyring-generator-zn6dg 0/1 Completed 0 32m
ceph ceph-rbd-pool-t7b8d 0/1 Completed 0 21m
ceph ceph-rbd-provisioner-86c99578d8-2689g 1/1 Running 0 21m
ceph ceph-rbd-provisioner-86c99578d8-jtfd5 1/1 Running 0 21m
ceph ceph-rgw-694b44bd5d-ckv8v 0/1 Running 6 21m
ceph ceph-rgw-keyring-generator-chlhr 0/1 Completed 0 32m
ceph ceph-storage-keys-generator-5g58r 0/1 Completed 0 32m
ceph ingress-65dc849968-68nfm 1/1 Running 0 32m
ceph ingress-65dc849968-gfwqq 1/1 Running 0 32m
ceph ingress-error-pages-768bd7c654-jx8tb 1/1 Running 0 32m
ceph ingress-error-pages-768bd7c654-kfd42 1/1 Running 0 32m
kube-system auxiliary-etcd-radio-st-11 3/3 Running 0 39m
kube-system bootstrap-armada-radio-st-11 4/4 Running 0 39m
kube-system calico-etcd-anchor-rvghv 1/1 Running 0 37m
kube-system calico-etcd-radio-st-11 1/1 Running 0 37m
kube-system calico-kube-policy-controllers-5df8bcbfc6-swxn4 1/1 Running 0 37m
kube-system calico-node-s9qfh 2/2 Running 0 37m
kube-system calico-settings-ljc9s 0/1 Completed 0 37m
kube-system coredns-5545b74458-6kmmf 1/1 Running 0 36m
kube-system coredns-5545b74458-nfqwq 1/1 Running 0 36m
kube-system haproxy-anchor-9zh6b 1/1 Running 0 34m
kube-system haproxy-radio-st-11 1/1 Running 1 34m
kube-system ingress-error-pages-5ccf96bf7d-2mzsl 1/1 Running 0 32m
kube-system ingress-error-pages-5ccf96bf7d-x9p64 1/1 Running 0 32m
kube-system ingress-kqxck 1/1 Running 0 32m
kube-system kubernetes-apiserver-anchor-ppv9m 1/1 Running 0 34m
kube-system kubernetes-apiserver-radio-st-11 1/1 Running 0 32m
kube-system kubernetes-controller-manager-anchor-m2mm4 1/1 Running 0 33m
kube-system kubernetes-controller-manager-radio-st-11 1/1 Running 0 33m
kube-system kubernetes-etcd-anchor-v7nr9 1/1 Running 0 36m
kube-system kubernetes-etcd-radio-st-11 1/1 Running 0 34m
kube-system kubernetes-proxy-86gmf 1/1 Running 0 38m
kube-system kubernetes-scheduler-anchor-dztff 1/1 Running 0 33m
kube-system kubernetes-scheduler-radio-st-11 1/1 Running 0 33m
ucp ceph-namespace-client-key-generator-tnng5 0/1 Completed 0 5m
ucp ingress-6cd5b89d5d-2pw7x 1/1 Running 0 5m
ucp ingress-6cd5b89d5d-z74xt 1/1 Running 0 5m
ucp ingress-error-pages-6bff886f96-24rjr 1/1 Running 0 5m
ucp ingress-error-pages-6bff886f96-qvbmt 1/1 Running 0 5m
ucp mariadb-ingress-74f744b54c-n7wzt 0/1 Running 0 4m
ucp mariadb-ingress-error-pages-5dc7dd974c-lbdh4 1/1 Running 0 4m
ucp mariadb-server-0 0/1 Pending 0 4m
2018-09-03 15:40:50.523 2392 ERROR armada.handlers.tiller [-] Error while installing release airship-ucp-mariadb: grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.UNKNOWN, release airship-ucp-mariadb failed: timed out waiting for the condition)>
2018-09-03 15:40:50.523 2392 ERROR armada.handlers.tiller Traceback (most recent call last):
2018-09-03 15:40:50.523 2392 ERROR armada.handlers.tiller File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 401, in install_release
2018-09-03 15:40:50.523 2392 ERROR armada.handlers.tiller metadata=self.metadata)
2018-09-03 15:40:50.523 2392 ERROR armada.handlers.tiller File "/usr/local/lib/python3.5/site-packages/grpc/_channel.py", line 487, in __call__
2018-09-03 15:40:50.523 2392 ERROR armada.handlers.tiller return _end_unary_response_blocking(state, call, False, deadline)
2018-09-03 15:40:50.523 2392 ERROR armada.handlers.tiller File "/usr/local/lib/python3.5/site-packages/grpc/_channel.py", line 437, in _end_unary_response_blocking
2018-09-03 15:40:50.523 2392 ERROR armada.handlers.tiller raise _Rendezvous(state, None, None, deadline)
2018-09-03 15:40:50.523 2392 ERROR armada.handlers.tiller grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.UNKNOWN, release airship-ucp-mariadb failed: timed out waiting for the condition)>
2018-09-03 15:40:50.523 2392 ERROR armada.handlers.tiller
2018-09-03 15:40:50.635 2392 ERROR armada.cli [-] Caught internal exception: armada.exceptions.tiller_exceptions.ReleaseException: Failed to Install release: airship-ucp-mariadb - Tiller Message: b'Release "airship-ucp-mariadb" failed: timed out waiting for the condition'
2018-09-03 15:40:50.635 2392 ERROR armada.cli Traceback (most recent call last):
2018-09-03 15:40:50.635 2392 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 401, in install_release
2018-09-03 15:40:50.635 2392 ERROR armada.cli metadata=self.metadata)
2018-09-03 15:40:50.635 2392 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/grpc/_channel.py", line 487, in __call__
2018-09-03 15:40:50.635 2392 ERROR armada.cli return _end_unary_response_blocking(state, call, False, deadline)
2018-09-03 15:40:50.635 2392 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/grpc/_channel.py", line 437, in _end_unary_response_blocking
2018-09-03 15:40:50.635 2392 ERROR armada.cli raise _Rendezvous(state, None, None, deadline)
2018-09-03 15:40:50.635 2392 ERROR armada.cli grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.UNKNOWN, release airship-ucp-mariadb failed: timed out waiting for the condition)>
2018-09-03 15:40:50.635 2392 ERROR armada.cli
2018-09-03 15:40:50.635 2392 ERROR armada.cli During handling of the above exception, another exception occurred:
2018-09-03 15:40:50.635 2392 ERROR armada.cli
2018-09-03 15:40:50.635 2392 ERROR armada.cli Traceback (most recent call last):
2018-09-03 15:40:50.635 2392 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/cli/__init__.py", line 39, in safe_invoke
2018-09-03 15:40:50.635 2392 ERROR armada.cli self.invoke()
2018-09-03 15:40:50.635 2392 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/cli/apply.py", line 217, in invoke
2018-09-03 15:40:50.635 2392 ERROR armada.cli resp = armada.sync()
2018-09-03 15:40:50.635 2392 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/armada.py", line 480, in sync
2018-09-03 15:40:50.635 2392 ERROR armada.cli timeout=timer)
2018-09-03 15:40:50.635 2392 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 414, in install_release
2018-09-03 15:40:50.635 2392 ERROR armada.cli raise ex.ReleaseException(release, status, 'Install')
2018-09-03 15:40:50.635 2392 ERROR armada.cli armada.exceptions.tiller_exceptions.ReleaseException: Failed to Install release: airship-ucp-mariadb - Tiller Message: b'Release "airship-ucp-mariadb" failed: timed out waiting for the condition'
2018-09-03 15:40:50.635 2392 ERROR armada.cli ^[[C 2018-09-03 15:41:04.365 3391 INFO armada.handlers.armada [-] Performing pre-flight operations.
2018-09-03 15:41:04.529 3391 INFO armada.handlers.armada [-] Cloning repo: https://urldefense.proofpoint.com/v2/url?u=https-3A__git.openstack.org_openstack_airship-2Dpromenade&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=MyMpkoXeJDDM5X1b0oakc6awcXcRf74kWEKFp5iQp9w&s=6DCFL5kcvZL7DFfRT81Zbyi6wbxOwwNQCel0W4pLQ8o&e= from branch: f60f6d7a8569c5006d32c44c27f4584f10ffe57a
_______________________________________________
Airship-discuss mailing list
Airship-discuss at lists.airshipit.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.airshipit.org_cgi-2Dbin_mailman_listinfo_airship-2Ddiscuss&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=MyMpkoXeJDDM5X1b0oakc6awcXcRf74kWEKFp5iQp9w&s=DzkWxm3aSJdAH_c6Kq0pn2OJ_pEbX15oIVdMlklUCs4&e=
More information about the Airship-discuss
mailing list