<div dir="ltr">Hello Roman,<div><br></div><div>Thanks for your answer.</div><div><br></div><div>What is the state of promenade + armada right now?</div><div>We are pretty excited about it.</div><div><br></div><div>Here is the output you required:</div><div><br></div><div>1) kubectl get pods --all-namespaces -o wide</div><div><a href="https://pastebin.com/b4kWGpBs">https://pastebin.com/b4kWGpBs</a><br></div><div><br></div><div>The only thing I see is the pod ldap-0 from the namespace osh-infra</div><div>Here is the output of kubectl describe pod -n osh-infra ldap-0</div><div><a href="https://pastebin.com/sSBVZnch">https://pastebin.com/sSBVZnch</a><br></div><div><br></div><div>I believe the interesting part is here:</div><div><ol class="gmail-text" style="color:rgb(172,172,172);background:rgb(247,247,247);margin:0px;padding:0px 0px 0px 55px;font-family:Consolas,Menlo,Monaco,"Lucida Console","Liberation Mono","DejaVu Sans Mono","Bitstream Vera Sans Mono",monospace,serif;font-size:12px;text-decoration-style:initial;text-decoration-color:initial"><li class="gmail-li2" style="background:rgb(255,255,255)"><div class="gmail-de2" style="padding:0px 8px;vertical-align:top;color:rgb(51,51,51);border-left:1px solid rgb(221,221,221);background:rgb(255,255,255);font-family:Consolas,Menlo,Monaco,"Lucida Console","Liberation Mono","DejaVu Sans Mono","Bitstream Vera Sans Mono",monospace,serif">Events:</div></li><li class="gmail-li1" style="background:rgb(255,255,255)"><div class="gmail-de1" style="padding:0px 8px;vertical-align:top;color:rgb(51,51,51);border-left:1px solid rgb(221,221,221);background:rgb(255,255,255);font-family:Consolas,Menlo,Monaco,"Lucida Console","Liberation Mono","DejaVu Sans Mono","Bitstream Vera Sans Mono",monospace,serif"> Type Reason Age From Message</div></li><li class="gmail-li1" style="background:rgb(255,255,255)"><div class="gmail-de1" style="padding:0px 8px;vertical-align:top;color:rgb(51,51,51);border-left:1px solid rgb(221,221,221);background:rgb(255,255,255);font-family:Consolas,Menlo,Monaco,"Lucida Console","Liberation Mono","DejaVu Sans Mono","Bitstream Vera Sans Mono",monospace,serif"> ---- ------ ---- ---- -------</div></li><li class="gmail-li1" style="background:rgb(255,255,255)"><div class="gmail-de1" style="padding:0px 8px;vertical-align:top;color:rgb(51,51,51);border-left:1px solid rgb(221,221,221);background:rgb(255,255,255);font-family:Consolas,Menlo,Monaco,"Lucida Console","Liberation Mono","DejaVu Sans Mono","Bitstream Vera Sans Mono",monospace,serif"> Warning FailedMount 12m (x576 over 21h) kubelet, n3 Unable to mount volumes for pod "ldap-0_osh-infra(85e73018-6b4c-11e8-b6e5-080027ee1df7)": timeout expired waiting for volumes to attach or mount for pod "osh-infra"/"ldap-0". list of unmounted volumes=[ldap-config ldap-data]. list of unattached volumes=[ldap-config ldap-data ldap-token-txdqp]</div></li><li class="gmail-li1" style="background:rgb(255,255,255)"><div class="gmail-de1" style="padding:0px 8px;vertical-align:top;color:rgb(51,51,51);border-left:1px solid rgb(221,221,221);background:rgb(255,255,255);font-family:Consolas,Menlo,Monaco,"Lucida Console","Liberation Mono","DejaVu Sans Mono","Bitstream Vera Sans Mono",monospace,serif"> Warning FailedMount 6m (x649 over 21h) kubelet, n3 MountVolume.WaitForAttach failed for volume "pvc-85dc46a5-6b4c-11e8-b6e5-080027ee1df7" : fail to check rbd image status with: (executable file not found in $PATH), rbd output: ()</div></li><li class="gmail-li2" style="background:rgb(255,255,255)"><div class="gmail-de2" style="padding:0px 8px;vertical-align:top;color:rgb(51,51,51);border-left:1px solid rgb(221,221,221);background:rgb(255,255,255);font-family:Consolas,Menlo,Monaco,"Lucida Console","Liberation Mono","DejaVu Sans Mono","Bitstream Vera Sans Mono",monospace,serif"> Warning FailedMount 2m (x651 over 21h) kubelet, n3 MountVolume.WaitForAttach failed for volume "pvc-85d6770b-6b4c-11e8-b6e5-080027ee1df7" : fail to check rbd image status with: (executable file not found in $PATH), rbd output: ()</div></li></ol><br></div><div><br></div><div>2) docker ps on all nodes</div><div><a href="https://pastebin.com/CaqLEv70">https://pastebin.com/CaqLEv70</a><br></div><div><br></div><div>3) docker ps -a on all nodes</div><div><a href="https://pastebin.com/wvB0qkbh">https://pastebin.com/wvB0qkbh</a><br></div><div><br></div><div>4) docker images on all nodes</div><div><a href="https://pastebin.com/H07kEwqe">https://pastebin.com/H07kEwqe</a><br></div><div><br></div><div>Best,</div><div>James</div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Jun 9, 2018 at 1:51 PM, Roman Gorshunov <span dir="ltr"><<a href="mailto:paye600@gmail.com" target="_blank">paye600@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello James,<br>
<br>
I didn’t see it before. Could you provide output of the following commands, if you still have this environment running. It would allow to get understanding of what has been deployed and what has not:<br>
kubectl get pods --all-namespaces -o wide # on n0<br>
docker ps # on all nodes<br>
docker ps -a # on all nodes<br>
docker images # on all nodes<br>
<br>
Also note that <a href="http://artifacts-aic.atlantafoundry.com" rel="noreferrer" target="_blank">artifacts-aic.atlantafoundry.<wbr>com</a> host where some of docker images are hosted is inaccessible at the moment, it could be that some images have not been downloaded successfully, team started mirroring images to dockerhub & <a href="http://quay.io" rel="noreferrer" target="_blank">quay.io</a>. Right before the summit development was focused on making airship-in-a-bottle work, and migration from github/gerrithub to openstack foundation infrastructure is still ongoing.<br>
<br>
Apologize for the inconvenience, James.<br>
<br>
Thank you.<br>
<div><div class="h5"><br>
> On 8 Jun 2018, at 21:18, James Devon <<a href="mailto:jr8586335@gmail.com">jr8586335@gmail.com</a>> wrote:<br>
> <br>
> Hi,<br>
> <br>
> After having tried airship-in-a-bottle, I'm trying to setup a Kubernetes cluster and install openstack on it using promenade and armada.<br>
> <br>
> I'm using the basic example : <a href="https://github.com/openstack/airship-promenade/tree/master/examples/basic" rel="noreferrer" target="_blank">https://github.com/openstack/<wbr>airship-promenade/tree/master/<wbr>examples/basic</a>, I've changed the ip addresses to match my environment of 4 nodes (n0 for genesis and then n1, n2, n3).<br>
> <br>
> At this point I'm facing only one problem: promenade-api.ucp.svc.cluster.<wbr>local is resolvable but the resolution is bad (192.168.150.165).<br>
> So instead of promenade-api.ucp.svc.cluster.<wbr>local, I used the pod address with the good port and it is working fine.<br>
> <br>
> Then, I am able to make n1,n2 and n3 join the cluster and I use the scripts from <a href="https://github.com/openstack/openstack-helm/tree/master/tools/deployment/armada" rel="noreferrer" target="_blank">https://github.com/openstack/<wbr>openstack-helm/tree/master/<wbr>tools/deployment/armada</a> to apply armada manifests.<br>
> <br>
> osh-infra is not able to start the ldap pod.<br>
> <br>
> Events:<br>
> Type Reason Age From Message<br>
> ---- ------ ---- ---- -------<br>
> Warning FailedScheduling 6m (x4 over 6m) default-scheduler pod has unbound PersistentVolumeClaims (repeated 3 times)<br>
> Normal Scheduled 6m default-scheduler Successfully assigned ldap-0 to n3<br>
> Normal SuccessfulAttachVolume 6m attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-85d6770b-6b4c-11e8-b6e5-<wbr>080027ee1df7"<br>
> Normal SuccessfulAttachVolume 6m attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-85dc46a5-6b4c-11e8-b6e5-<wbr>080027ee1df7"<br>
> Normal SuccessfulMountVolume 6m kubelet, n3 MountVolume.SetUp succeeded for volume "ldap-token-txdqp"<br>
> Warning FailedMount 13s (x11 over 6m) kubelet, n3 MountVolume.WaitForAttach failed for volume "pvc-85dc46a5-6b4c-11e8-b6e5-<wbr>080027ee1df7" : fail to check rbd image status with: (executable file not found in $PATH), rbd output: ()<br>
> Warning FailedMount 12s (x11 over 6m) kubelet, n3 MountVolume.WaitForAttach failed for volume "pvc-85d6770b-6b4c-11e8-b6e5-<wbr>080027ee1df7" : fail to check rbd image status with: (executable file not found in $PATH), rbd output: ()<br>
> Warning FailedMount 2s (x3 over 4m) kubelet, n3 Unable to mount volumes for pod "ldap-0_osh-infra(85e73018-<wbr>6b4c-11e8-b6e5-080027ee1df7)": timeout expired waiting for volumes to attach or mount for pod "osh-infra"/"ldap-0". list of unmounted volumes=[ldap-config ldap-data]. list of unattached volumes=[ldap-config ldap-data ldap-token-txdqp]<br>
> <br>
> I've checked the logs of ceph-osd and there's also something:<br>
> <br>
> ceph-2.1 | starting osd.2 at - osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/journal/journal.<wbr>2<br>
> ceph-2.1 | 2018-06-08 18:41:03.854234 7f7cd7d58e00 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway<br>
> ceph-2.1 | 2018-06-08 18:41:03.855499 7f7cd7d58e00 -1 journal do_read_entry(8192): bad header magic<br>
> ceph-2.1 | 2018-06-08 18:41:03.855567 7f7cd7d58e00 -1 journal do_read_entry(8192): bad header magic<br>
> ceph-2.1 | 2018-06-08 18:41:03.901804 7f7cd7d58e00 -1 osd.2 0 log_to_monitors {default=true}<br>
> ceph-2.1 | 2018-06-08 18:41:04.529480 7f7cbb2c7700 -1 osd.2 0 waiting for initial osdmap<br>
> <br>
> Did anybody encountered this error and how to fix it? <br>
> <br>
> Best,<br>
> James<br>
> <br>
</div></div>> ______________________________<wbr>_________________<br>
> Airship-discuss mailing list<br>
> <a href="mailto:Airship-discuss@lists.airshipit.org">Airship-discuss@lists.<wbr>airshipit.org</a><br>
> <a href="http://lists.airshipit.org/cgi-bin/mailman/listinfo/airship-discuss" rel="noreferrer" target="_blank">http://lists.airshipit.org/<wbr>cgi-bin/mailman/listinfo/<wbr>airship-discuss</a><br>
<br>
</blockquote></div><br></div></div>