[Airship-discuss] Airship-in-a-Bottle

Evgeny L eli at mirantis.com
Fri Apr 12 14:49:51 UTC 2019


Hi,

Can you copy the logs of ingress pod? (use https://paste.ubuntu.com/ to
share the logs)
You can get the logs by running "kubectl logs -n ucp
maas-ingress-567777887c-v9fzp".

Have you used this instruction to configure your behind the proxy airship?
https://github.com/openstack/airship-in-a-bottle/blob/master/manifests/dev_minimal/README.txt

Are you running the latest master? Can you check a commit that you have in
your cloned airship-in-a-bottle repo?

Thanks,



On Thu, Apr 11, 2019 at 3:41 PM MUEEN, MUZAMMIL <mm621n at att.com> wrote:

> Hi Team,
>
>
>
> While installing airship in a bottle on a physical node, behind a proxy,
> with dev minimal configuration.
>
> The setup is stuck with maas pod in a crashloop … Below mentioned are the
> display from the installation script and the *kubectl describe *for maas
> pod.
>
>
>
> I’ll be grateful if anyone can help me solve how to get maas out of this
> loop?
>
>
>
>
>
> *NAMESPACE     NAME
> READY     STATUS              RESTARTS   AGE*
>
> kube-system   auxiliary-etcd-mtn15r07c001
> 3/3       Running             12         23h
>
> kube-system   bootstrap-armada-mtn15r07c001
> 4/4       Running             16         23h
>
> kube-system   calico-etcd-anchor-56fjc
> 1/1       Running             0          23h
>
> kube-system   calico-etcd-mtn15r07c001
>                              1/1       Running             0          23h
>
> kube-system   calico-kube-controllers-56c54d8cf8-r955j
> 1/1       Running             0          23h
>
> kube-system   calico-node-r9bdv
>            1/1       Running             0          23h
>
> kube-system   calico-settings-866vn
> 0/1       Completed           0          23h
>
> kube-system   coredns-764f47bb78-5tsc4
> 1/1       Running             0          23h
>
> kube-system   coredns-764f47bb78-hv5j6
> 1/1       Running             0          23h
>
> kube-system   coredns-764f47bb78-zsjst
> 1/1       Running             0          23h
>
> kube-system   haproxy-mtn15r07c001
> 1/1       Running             4          23h
>
> kube-system   ingress-error-pages-5ccf96bf7d-d7955
> 1/1       Running             0          23h
>
> kube-system   ingress-error-pages-5ccf96bf7d-mxz68
> 1/1       Running             0          23h
>
> kube-system   ingress-ggdvw
>                     1/1       Running             0          23h
>
> kube-system   kubernetes-apiserver-anchor-8btb7
> 1/1       Running             0          23h
>
> kube-system   kubernetes-apiserver-mtn15r07c001
>   1/1       Running             0          23h
>
> kube-system   kubernetes-controller-manager-anchor-b4ztr
> 1/1       Running             0          23h
>
> kube-system   kubernetes-controller-manager-mtn15r07c001
> 1/1       Running             0          23h
>
> kube-system   kubernetes-etcd-anchor-n9fwl
> 1/1       Running             0          23h
>
> kube-system   kubernetes-etcd-mtn15r07c001
> 1/1       Running             0          23h
>
> kube-system   kubernetes-proxy-2x2l9
> 1/1       Running             0          23h
>
> kube-system   kubernetes-scheduler-anchor-mqw5j
> 1/1       Running             0          23h
>
> kube-system   kubernetes-scheduler-mtn15r07c001
> 1/1       Running             0          23h
>
> kube-system   nfs-provisioner-7799d64d59-z2wwd
> 1/1       Running             0          23h
>
> ucp           airship-ucp-keystone-memcached-memcached-74d79d8896-cff8q
> 1/1       Running             0          23h
>
> ucp           airship-ucp-rabbitmq-rabbitmq-0
> 1/1       Running             0          23h
>
> ucp           divingbell-apparmor-default-7czhv
> 1/1       Running             0          22h
>
> ucp           divingbell-apt-default-trvxq
> 1/1       Running             0          22h
>
> ucp           divingbell-ethtool-default-mdkdv
>              1/1       Running             0          22h
>
> ucp           divingbell-exec-default-n8z7r
> 1/1       Running             0          22h
>
> ucp           divingbell-limits-default-wp2p9
> 1/1       Running             0          22h
>
> ucp           divingbell-mounts-default-fc036fae-vrwmv
> 1/1       Running             0          22h
>
> ucp           divingbell-perm-default-rv4ff
> 1/1       Running             0          22h
>
> ucp           divingbell-sysctl-default-x8jpz
> 1/1       Running             0          22h
>
> ucp           divingbell-uamlite-default-g9dnv
> 1/1       Running             0          22h
>
> ucp           ingress-86576d6599-zdf7q
> 1/1       Running             0          23h
>
> ucp           ingress-error-pages-5c97bb46bb-c9jhz
> 1/1       Running             0          23h
>
> ucp           keystone-api-7c99d578b7-fvf6j
> 1/1       Running             0          23h
>
> ucp           keystone-bootstrap-qsf4g
> 0/1       Completed           0          23h
>
> ucp           keystone-credential-setup-6xsjs
> 0/1       Completed           0          23h
>
> ucp           keystone-db-init-lpj2t
> 0/1       Completed           0          23h
>
> ucp           keystone-db-sync-m28br
> 0/1       Completed           0          23h
>
> ucp           keystone-domain-manage-rj66q
> 0/1       Completed           0          23h
>
> ucp           keystone-fernet-rotate-1553644800-m2gx8
>                     0/1       Init:0/1            0          21h
>
> ucp           keystone-fernet-setup-8ffkj
> 0/1       Completed           0          23h
>
> ucp           keystone-rabbit-init-95xht
>   0/1       Completed           0          23h
>
> ucp           maas-bootstrap-admin-user-7bfgx
> 0/1       Init:0/1            0          21h
>
> ucp           maas-db-init-r5mkj
> 0/1       Init:0/1            0          21h
>
> ucp           maas-db-sync-gpwq2
> 0/1       Init:0/1            0          21h
>
> ucp           maas-export-api-key-tk9c8
> 0/1       Init:0/1            0          21h
>
> ucp           maas-import-resources-kftsl
> 0/1       Init:0/1            0          21h
>
> ucp           maas-ingress-567777887c-v9fzp
> 1/2       CrashLoopBackOff    261        21h
>
> ucp           maas-ingress-errors-798cbd8c4c-q987s
> 0/1       ContainerCreating   0          21h
>
> ucp           maas-rack-0
> 0/1       Init:0/1            0          21h
>
> ucp           maas-rack-1
> 0/1       Pending             0          21h
>
> ucp           maas-region-0
> 0/1       Init:0/1            0          21h
>
> ucp           mariadb-ingress-6c9dcbc755-bp5r6
> 1/1       Running             0          23h
>
> ucp           mariadb-ingress-error-pages-b7f9f85c7-qbnh8
> 1/1       Running             0          23h
>
> ucp           mariadb-server-0
>              1/1       Running             0          23h
>
> ucp           postgresql-0
> 1/1       Running             0          23h
>
>
>
>
>
>
>
> *root at mtn15r07c001:~# kubectl describe pods maas-ingress-567777887c-v9fzp
> --namespace=ucp*
>
> Name:           maas-ingress-567777887c-v9fzp
>
> Namespace:      ucp
>
> Node:           mtn15r07c001/135.21.35.69
>
> Start Time:     Tue, 26 Mar 2019 23:55:29 +0000
>
> Labels:         application=maas
>
>                 component=ingress
>
>                 pod-template-hash=1233334437
>
>                 release_group=airship-maas
>
> Annotations:
> configmap-bin-hash=4b7d801062899a7d636c1f3e1c943bfad865192b964fea74b9057d85eec28d52
>
>
>              configmap-etc-hash=6ae6a39a1a7edf9c6d28d6952cf28fa1673062dc655a10ed04484f4ac80ffb8a
>
> Status:         Running
>
> IP:             135.21.35.69
>
> Controlled By:  ReplicaSet/maas-ingress-567777887c
>
> Init Containers:
>
>   init:
>
>     Container ID:
> docker://6abdaa7b1adc8a1c3e1a3e7cb39c20684c63aa19a72b3261298cfdff21cd0188
>
>     Image:         quay.io/stackanetes/kubernetes-entrypoint:v0.3.1
>
>     Image ID:      docker-pullable://
> quay.io/stackanetes/kubernetes-entrypoint at sha256:32b1b657ee4bcc9cc7a1529e31d8e1a06376172373ee020f97f3e78168fde4b6
>
>     Port:          <none>
>
>     Host Port:     <none>
>
>     Command:
>
>       kubernetes-entrypoint
>
>     State:          Terminated
>
>       Reason:       Completed
>
>       Exit Code:    0
>
>       Started:      Tue, 26 Mar 2019 23:55:33 +0000
>
>       Finished:     Tue, 26 Mar 2019 23:55:33 +0000
>
>     Ready:          True
>
>     Restart Count:  0
>
>     Environment:
>
>       POD_NAME:              maas-ingress-567777887c-v9fzp (v1:
> metadata.name)
>
>       NAMESPACE:             ucp (v1:metadata.namespace)
>
>       INTERFACE_NAME:        eth0
>
>       PATH:
> /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/
>
>       DEPENDENCY_SERVICE:
>
>       DEPENDENCY_DAEMONSET:
>
>       DEPENDENCY_CONTAINER:
>
>       DEPENDENCY_POD_JSON:
>
>       COMMAND:               echo done
>
>     Mounts:
>
>       /var/run/secrets/kubernetes.io/serviceaccount from
> maas-ingress-token-8qlcl (ro)
>
>   maas-ingress-vip-init:
>
>     Container ID:
> docker://195e75792c7592c90cff15703a7a977c721b4f863e0834033a72302e683a3a7d
>
>     Image:         docker.io/busybox:latest
>
>     Image ID:      docker-pullable://busybox@sha256
> :061ca9704a714ee3e8b80523ec720c64f6209ad3f97c0ff7cb9ec7d19f15149f
>
>     Port:          <none>
>
>     Host Port:     <none>
>
>     Command:
>
>       /tmp/maas-vip-configure.sh
>
>       start
>
>     State:          Terminated
>
>       Reason:       Completed
>
>       Exit Code:    0
>
>       Started:      Tue, 26 Mar 2019 23:55:34 +0000
>
>       Finished:     Tue, 26 Mar 2019 23:55:34 +0000
>
>     Ready:          True
>
>     Restart Count:  0
>
>     Environment:
>
>       addr:       192.169.1.5/32
>
>       interface:  maas-vip
>
>       mode:       routed
>
>     Mounts:
>
>       /mnt/host-rootfs from host-rootfs (ro)
>
>       /tmp/maas-vip-configure.sh from maas-bin (ro)
>
>       /var/run/secrets/kubernetes.io/serviceaccount from
> maas-ingress-token-8qlcl (ro)
>
> Containers:
>
>   maas-ingress-vip:
>
>     Container ID:
> docker://8be5e78def6312c70d48f0bbe57c77e4b95ab3e5a417416cc0c6d1d3d44ff536
>
>     Image:         docker.io/busybox:latest
>
>     Image ID:      docker-pullable://busybox@sha256
> :061ca9704a714ee3e8b80523ec720c64f6209ad3f97c0ff7cb9ec7d19f15149f
>
>     Port:          <none>
>
>     Host Port:     <none>
>
>     Command:
>
>       /bin/init
>
>     State:          Running
>
>       Started:      Tue, 26 Mar 2019 23:55:35 +0000
>
>     Ready:          True
>
>     Restart Count:  0
>
>     Environment:
>
>       addr:       192.169.1.5/32
>
>       interface:  maas-vip
>
>       mode:       routed
>
>     Mounts:
>
>       /etc/inittab from maas-etc (ro)
>
>       /tmp/maas-vip-configure.sh from maas-bin (ro)
>
>       /var/run/secrets/kubernetes.io/serviceaccount from
> maas-ingress-token-8qlcl (ro)
>
>   maas-ingress:
>
>     Container ID:
> docker://a892c56049d267479c0298e3fe077eed4cc0a73968d069ad5b4eee8fb6f9431f
>
>     Image:
> quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.20.0
>
>     Image ID:      docker-pullable://
> quay.io/kubernetes-ingress-controller/nginx-ingress-controller at sha256:f6180c5397d2361c317aff1314dc192ab0f9f515346a5319422cdc264f05d2d9
>
>     Port:          <none>
>
>     Host Port:     <none>
>
>     Command:
>
>       /tmp/maas-ingress.sh
>
>       start
>
>     State:          Waiting
>
>       Reason:       CrashLoopBackOff
>
>     Last State:     Terminated
>
>       Reason:       Error
>
>       Exit Code:    255
>
>       Started:      Wed, 27 Mar 2019 20:13:16 +0000
>
>       Finished:     Wed, 27 Mar 2019 20:13:16 +0000
>
>     Ready:          False
>
>     Restart Count:  243
>
>     Environment:
>
>       POD_NAMESPACE:       ucp (v1:metadata.namespace)
>
>       POD_NAME:            maas-ingress-567777887c-v9fzp (v1:metadata.name
> )
>
>       RELEASE_NAME:        airship-maas
>
>       HTTP_PORT:           80
>
>       HTTPS_PORT:          443
>
>       HEALTHZ_PORT:        10254
>
>       STATUS_PORT:         18080
>
>       ERROR_PAGE_SERVICE:  maas-ingress-error
>
>       DEFAULT_ERROR_PORT:  8181
>
>     Mounts:
>
>       /tmp/maas-ingress.sh from maas-bin (ro)
>
>       /var/run/secrets/kubernetes.io/serviceaccount from
> maas-ingress-token-8qlcl (ro)
>
> Conditions:
>
>   Type           Status
>
>   Initialized    True
>
>   Ready          False
>
>   PodScheduled   True
>
> Volumes:
>
>   maas-bin:
>
>     Type:      ConfigMap (a volume populated by a ConfigMap)
>
>     Name:      maas-bin
>
>     Optional:  false
>
>   maas-etc:
>
>     Type:      ConfigMap (a volume populated by a ConfigMap)
>
>     Name:      maas-etc
>
>     Optional:  false
>
>   host-rootfs:
>
>     Type:          HostPath (bare host directory volume)
>
>     Path:          /
>
>     HostPathType:
>
>   maas-ingress-token-8qlcl:
>
>     Type:        Secret (a volume populated by a Secret)
>
>     SecretName:  maas-ingress-token-8qlcl
>
>     Optional:    false
>
> QoS Class:       BestEffort
>
> Node-Selectors:  ucp-control-plane=enabled
>
> Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
>
>                  node.kubernetes.io/unreachable:NoExecute for 300s
>
> Events:
>
>   Type     Reason   Age                  From                   Message
>
>   ----     ------   ----                 ----                   -------
>
>   Warning  BackOff  2m (x5500 over 20h)  kubelet, mtn15r07c001  Back-off
> restarting failed container
>
> root at mtn15r07c001:~#
>
>
>
> *Regards,*
> Muzammil
> Architect
> Network & Shared Services
> 469-600-0477
>
> [image: cid:image001.png at 01D32007.C3B5AE40]
>
>
>
>
> _______________________________________________
> Airship-discuss mailing list
> Airship-discuss at lists.airshipit.org
> http://lists.airshipit.org/cgi-bin/mailman/listinfo/airship-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.airshipit.org/pipermail/airship-discuss/attachments/20190412/37403318/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 11051 bytes
Desc: not available
URL: <http://lists.airshipit.org/pipermail/airship-discuss/attachments/20190412/37403318/attachment-0001.png>


More information about the Airship-discuss mailing list