[Airship-discuss] Airship in a bottle install failure...

Alex Arnoldy alex.arnoldy at suse.com
Mon Dec 10 17:16:39 UTC 2018


Thanks for the reply Daniel.

I won't have a chance to troubleshoot it for at least two more days. When I get a chance, I'll try to nsenter the namespaces occupied by that pod and have a look around.

Best,

--Alex.

Alex Arnoldy
Software Defined Infrastructure Solutions Architect
SUSE
Alex.Arnoldy at SUSE.com<mailto:Alex.Arnoldy at SUSE.com>
(P) +1 (408) 828-9797

From: Balsiger Daniel, INI-EAI-INO-MBL <Daniel.Balsiger at swisscom.com>
Sent: Monday, December 10, 2018 7:59 AM
To: airship-discuss at lists.airshipit.org; Alex Arnoldy <alex.arnoldy at suse.com>
Subject: Re: Airship in a bottle install failure...


Hi there Alex.



Thanks for the reply.

>From your logs it seems to me like  maas-db-init is not able to connect to postgres DB with psql client to the other container. (clients should be listed by IP in pg_hba.conf, wildcards should be possible....)



I will try another install from a fresh VM and let you know if I have the same one.



Thanks

________________________________
From: Alex Arnoldy <alex.arnoldy at suse.com<mailto:alex.arnoldy at suse.com>>
Sent: Monday, December 10, 2018 4:26 PM
To: Balsiger Daniel, INI-EAI-INO-MBL; airship-discuss at lists.airshipit.org<mailto:airship-discuss at lists.airshipit.org>
Subject: RE: Airship in a bottle install failure...


Hello Daniel,



I had launched the same thing but then got pulled away from it and totally forgot to check it. I thought I'd be able to tell you I got it running, but I don't think that's the case. For my installation, the maas-db-init pod seems to be stuck in CrashLoopBackOff:



root at airship:~# kubectl get pods -o wide --all-namespaces

NAMESPACE     NAME                                                        READY     STATUS             RESTARTS   AGE       IP              NODE

kube-system   auxiliary-etcd-airship                                      3/3       Running            0          3d        172.16.231.60   airship

kube-system   bootstrap-armada-airship                                    4/4       Running            0          3d        172.16.231.60   airship

kube-system   calico-etcd-airship                                         1/1       Running            0          3d        172.16.231.60   airship

kube-system   calico-etcd-anchor-nqntm                                    1/1       Running            0          3d        172.16.231.60   airship

kube-system   calico-kube-controllers-6d5997c8c4-bm9rk                    1/1       Running            0          3d        172.16.231.60   airship

kube-system   calico-node-2sgkk                                           2/2       Running            0          3d        172.16.231.60   airship

kube-system   calico-settings-bqs5v                                       0/1       Completed          0          3d        172.16.231.60   airship

kube-system   coredns-d4587b7f8-bbpr6                                     1/1       Running            13         3d        10.97.196.130   airship

kube-system   coredns-d4587b7f8-tdl8v                                     1/1       Running            11         3d        10.97.196.131   airship

kube-system   coredns-d4587b7f8-ts9lr                                     1/1       Running            12         3d        10.97.196.129   airship

kube-system   haproxy-airship                                             1/1       Running            0          3d        172.16.231.60   airship

kube-system   ingress-error-pages-5ccf96bf7d-8q6g5                        1/1       Running            0          3d        10.97.196.133   airship

kube-system   ingress-fmrmk                                               1/1       Running            0          3d        172.16.231.60   airship

kube-system   kubernetes-apiserver-airship                                1/1       Running            0          3d        172.16.231.60   airship

kube-system   kubernetes-apiserver-anchor-rkfpz                           1/1       Running            0          3d        172.16.231.60   airship

kube-system   kubernetes-controller-manager-airship                       1/1       Running            0          3d        172.16.231.60   airship

kube-system   kubernetes-controller-manager-anchor-75k62                  1/1       Running            0          3d        172.16.231.60   airship

kube-system   kubernetes-etcd-airship                                     1/1       Running            0          3d        172.16.231.60   airship

kube-system   kubernetes-etcd-anchor-hzcrz                                1/1       Running            0          3d        172.16.231.60   airship

kube-system   kubernetes-proxy-994kg                                      1/1       Running            0          3d        172.16.231.60   airship

kube-system   kubernetes-scheduler-airship                                1/1       Running            0          3d        172.16.231.60   airship

kube-system   kubernetes-scheduler-anchor-gsqtj                           1/1       Running            0          3d        172.16.231.60   airship

kube-system   nfs-provisioner-7799d64d59-g2g6n                            1/1       Running            0          3d        10.97.196.134   airship

kube-system   tiller-deploy-7d88c6f956-9jjk7                              1/1       Running            0          3d        10.97.196.153   airship

ucp           airship-ucp-keystone-memcached-memcached-74d79d8896-dd8k2   1/1       Running            0          3d        10.97.196.144   airship

ucp           airship-ucp-rabbitmq-rabbitmq-0                             1/1       Running            2          3d        10.97.196.178   airship

ucp           ingress-6cd5b89d5d-xx592                                    1/1       Running            0          3d        10.97.196.135   airship

ucp           ingress-error-pages-5c97bb46bb-ljmw6                        1/1       Running            0          3d        10.97.196.136   airship

ucp           keystone-api-7657986b8c-b49r6                               1/1       Running            0          3d        10.97.196.145   airship

ucp           keystone-bootstrap-8494p                                    0/1       Completed          0          3d        10.97.196.147   airship

ucp           keystone-credential-setup-r5twm                             0/1       Completed          0          3d        10.97.196.146   airship

ucp           keystone-db-init-pxlzr                                      0/1       Completed          0          3d        10.97.196.148   airship

ucp           keystone-db-sync-zx665                                      0/1       Completed          0          3d        10.97.196.151   airship

ucp           keystone-domain-manage-rbfl5                                0/1       Completed          0          3d        10.97.196.149   airship

ucp           keystone-fernet-rotate-1544356800-kcgg9                     0/1       Completed          0          1d        10.97.196.148   airship

ucp           keystone-fernet-rotate-1544400000-78qc8                     0/1       Completed          0          14h       10.97.196.163   airship

ucp           keystone-fernet-rotate-1544443200-v8bkn                     0/1       Completed          0          2h        10.97.196.159   airship

ucp           keystone-fernet-setup-srrqj                                 0/1       Completed          0          3d        10.97.196.152   airship

ucp           keystone-rabbit-init-x4stm                                  0/1       Completed          0          3d        10.97.196.150   airship

ucp           maas-bootstrap-admin-user-26lhf                             0/1       Init:0/1           0          3d        10.97.196.132   airship

ucp           maas-db-init-v7z6r                                          0/1       CrashLoopBackOff   944        3d        10.97.196.143   airship

ucp           maas-db-sync-5c24z                                          0/1       Init:0/1           0          3d        10.97.196.162   airship

ucp           maas-export-api-key-zhfxp                                   0/1       Init:0/1           0          3d        10.97.196.147   airship

ucp           maas-import-resources-j6tzd                                 0/1       Init:0/1           0          3d        10.97.196.152   airship

ucp           maas-rack-0                                                 0/1       Init:0/1           0          3d        172.16.231.60   airship

ucp           maas-region-0                                               0/1       Init:0/1           0          3d        10.97.196.146   airship

ucp           mariadb-ingress-6c4f9c76f-sg8cx                             1/1       Running            0          3d        10.97.196.137   airship

ucp           mariadb-ingress-6c4f9c76f-sz26m                             1/1       Running            0          3d        10.97.196.138   airship

ucp           mariadb-ingress-error-pages-5dd6fbb75-x2xbn                 1/1       Running            0          3d        10.97.196.139   airship

ucp           mariadb-server-0                                            1/1       Running            0          3d        10.97.196.142   airship

ucp           postgresql-0                                                1/1       Running            1          3d        10.97.196.140   airship



The only output from that pod is this:



root at airship:~# kubectl logs -n ucp maas-db-init-v7z6r

+ export HOME=/tmp

+ HOME=/tmp

+ [[ ! -v DB_HOST ]]

+ [[ ! -v ROOT_DB_USER ]]

+ [[ ! -v PGPASSWORD ]]

+ [[ ! -v USER_DB_USER ]]

+ [[ ! -v USER_DB_PASS ]]

+ [[ ! -v USER_DB_NAME ]]

+ echo 'Got DB connection info'

Got DB connection info

+ pgsql_superuser_cmd 'SELECT 1 FROM pg_database WHERE datname = '\''maasdb'\'''

+ DB_COMMAND='SELECT 1 FROM pg_database WHERE datname = '\''maasdb'\'''

+ [[ ! -z '' ]]

+ grep -q 1

+ psql -h postgresql.ucp.svc.cluster.local -p 5432 -U postgres '--command=SELECT 1 FROM pg_database WHERE datname = '\''maasdb'\'''

psql: FATAL:  no pg_hba.conf entry for host "10.97.196.143", user "postgres", database "postgres", SSL off

+ pgsql_superuser_cmd 'CREATE DATABASE maasdb'

+ DB_COMMAND='CREATE DATABASE maasdb'

+ [[ ! -z '' ]]

+ psql -h postgresql.ucp.svc.cluster.local -p 5432 -U postgres '--command=CREATE DATABASE maasdb'

psql: FATAL:  no pg_hba.conf entry for host "10.97.196.143", user "postgres", database "postgres", SSL off



Unfortunately, I haven't been able to troubleshoot this any further.



Best,

--Alex.



Alex Arnoldy
Software Defined Infrastructure Solutions Architect
SUSE
Alex.Arnoldy at SUSE.com<mailto:Alex.Arnoldy at SUSE.com>
(P) +1 (408) 828-9797



From: Balsiger Daniel, INI-EAI-INO-MBL <Daniel.Balsiger at swisscom.com<mailto:Daniel.Balsiger at swisscom.com>>
Sent: Monday, December 10, 2018 6:32 AM
To: airship-discuss at lists.airshipit.org<mailto:airship-discuss at lists.airshipit.org>
Subject: [Airship-discuss] Airship in a bottle install failure...



Hi there Airshippers



I wanted to try installing airhip-in-a-bottle in a fresh Ubuntu16.04 VM running on OpenStack over the weekend. I followed the instructions on airshipit.org



--snip

sudo -i

mkdir -p /root/deploy && cd "$_"

git clone https://github.com/openstack/airship-in-a-bottle

cd /root/deploy/airship-in-a-bottle/manifests/dev_single_node

./airship-in-a-bottle.sh

--snip



The script is still running after 2 days so I think something went wrong.

The output is repeating itself, listing running/restarted containers (I think postgres is the one not coming up properly)

--snip

NAMESPACE     NAME                                          READY     STATUS      RESTARTS   AGE
kube-system   auxiliary-etcd-airship                        3/3       Running     0          2d
kube-system   bootstrap-armada-airship                      4/4       Running     0          2d
kube-system   calico-etcd-airship                           1/1       Running     0          2d
kube-system   calico-etcd-anchor-2jrqf                      1/1       Running     0          2d
kube-system   calico-kube-controllers-6d5997c8c4-zxftw      1/1       Running     0          2d
kube-system   calico-node-l885v                             2/2       Running     0          2d
kube-system   calico-settings-kbq79                         0/1       Completed   0          2d
kube-system   coredns-d4587b7f8-7wlpm                       1/1       Running     0          2d
kube-system   coredns-d4587b7f8-98t62                       1/1       Running     0          2d
kube-system   coredns-d4587b7f8-frx5w                       1/1       Running     0          2d
kube-system   haproxy-airship                               1/1       Running     0          2d
kube-system   ingress-8w29l                                 1/1       Running     0          2d
kube-system   ingress-error-pages-5ccf96bf7d-jsrww          1/1       Running     0          2d
kube-system   kubernetes-apiserver-airship                  1/1       Running     0          2d
kube-system   kubernetes-apiserver-anchor-55x59             1/1       Running     0          2d
kube-system   kubernetes-controller-manager-airship         1/1       Running     1          2d
kube-system   kubernetes-controller-manager-anchor-2wsg8    1/1       Running     0          2d
kube-system   kubernetes-etcd-airship                       1/1       Running     0          2d
kube-system   kubernetes-etcd-anchor-64wgx                  1/1       Running     0          2d
kube-system   kubernetes-proxy-9s68d                        1/1       Running     0          2d
kube-system   kubernetes-scheduler-airship                  1/1       Running     1          2d
kube-system   kubernetes-scheduler-anchor-npmn7             1/1       Running     0          2d
kube-system   nfs-provisioner-7799d64d59-scgdc              1/1       Running     0          2d
ucp           airship-ucp-rabbitmq-rabbitmq-0               1/1       Running     0          2d
ucp           ingress-6cd5b89d5d-gp98m                      1/1       Running     0          2d
ucp           ingress-error-pages-5c97bb46bb-zq5cw          1/1       Running     0          2d
ucp           mariadb-ingress-6c4f9c76f-bjd99               0/1       Running     0          2d
ucp           mariadb-ingress-6c4f9c76f-xjrn9               0/1       Running     0          2d
ucp           mariadb-ingress-error-pages-5dd6fbb75-89qmx   1/1       Running     0          2d
ucp           mariadb-server-0                              0/1       Running     0          2d
ucp           postgresql-0                                  1/1       Running     1          2d
--snip



Anybody out there with the same problem or (even better) a solution for it?

Should I pick a particular branch or commit?

Any help is appreciated.



Best Regards, Daniel


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.airshipit.org/pipermail/airship-discuss/attachments/20181210/3b92ba8e/attachment-0001.html>


More information about the Airship-discuss mailing list