Hi, Can you copy the logs of ingress pod? (use https://paste.ubuntu.com/ to share the logs) You can get the logs by running "kubectl logs -n ucp maas-ingress-567777887c-v9fzp". Have you used this instruction to configure your behind the proxy airship? https://github.com/openstack/airship-in-a-bottle/blob/master/manifests/dev_m... Are you running the latest master? Can you check a commit that you have in your cloned airship-in-a-bottle repo? Thanks, On Thu, Apr 11, 2019 at 3:41 PM MUEEN, MUZAMMIL <mm621n@att.com> wrote:
Hi Team,
While installing airship in a bottle on a physical node, behind a proxy, with dev minimal configuration.
The setup is stuck with maas pod in a crashloop … Below mentioned are the display from the installation script and the *kubectl describe *for maas pod.
I’ll be grateful if anyone can help me solve how to get maas out of this loop?
*NAMESPACE NAME READY STATUS RESTARTS AGE*
kube-system auxiliary-etcd-mtn15r07c001 3/3 Running 12 23h
kube-system bootstrap-armada-mtn15r07c001 4/4 Running 16 23h
kube-system calico-etcd-anchor-56fjc 1/1 Running 0 23h
kube-system calico-etcd-mtn15r07c001 1/1 Running 0 23h
kube-system calico-kube-controllers-56c54d8cf8-r955j 1/1 Running 0 23h
kube-system calico-node-r9bdv 1/1 Running 0 23h
kube-system calico-settings-866vn 0/1 Completed 0 23h
kube-system coredns-764f47bb78-5tsc4 1/1 Running 0 23h
kube-system coredns-764f47bb78-hv5j6 1/1 Running 0 23h
kube-system coredns-764f47bb78-zsjst 1/1 Running 0 23h
kube-system haproxy-mtn15r07c001 1/1 Running 4 23h
kube-system ingress-error-pages-5ccf96bf7d-d7955 1/1 Running 0 23h
kube-system ingress-error-pages-5ccf96bf7d-mxz68 1/1 Running 0 23h
kube-system ingress-ggdvw 1/1 Running 0 23h
kube-system kubernetes-apiserver-anchor-8btb7 1/1 Running 0 23h
kube-system kubernetes-apiserver-mtn15r07c001 1/1 Running 0 23h
kube-system kubernetes-controller-manager-anchor-b4ztr 1/1 Running 0 23h
kube-system kubernetes-controller-manager-mtn15r07c001 1/1 Running 0 23h
kube-system kubernetes-etcd-anchor-n9fwl 1/1 Running 0 23h
kube-system kubernetes-etcd-mtn15r07c001 1/1 Running 0 23h
kube-system kubernetes-proxy-2x2l9 1/1 Running 0 23h
kube-system kubernetes-scheduler-anchor-mqw5j 1/1 Running 0 23h
kube-system kubernetes-scheduler-mtn15r07c001 1/1 Running 0 23h
kube-system nfs-provisioner-7799d64d59-z2wwd 1/1 Running 0 23h
ucp airship-ucp-keystone-memcached-memcached-74d79d8896-cff8q 1/1 Running 0 23h
ucp airship-ucp-rabbitmq-rabbitmq-0 1/1 Running 0 23h
ucp divingbell-apparmor-default-7czhv 1/1 Running 0 22h
ucp divingbell-apt-default-trvxq 1/1 Running 0 22h
ucp divingbell-ethtool-default-mdkdv 1/1 Running 0 22h
ucp divingbell-exec-default-n8z7r 1/1 Running 0 22h
ucp divingbell-limits-default-wp2p9 1/1 Running 0 22h
ucp divingbell-mounts-default-fc036fae-vrwmv 1/1 Running 0 22h
ucp divingbell-perm-default-rv4ff 1/1 Running 0 22h
ucp divingbell-sysctl-default-x8jpz 1/1 Running 0 22h
ucp divingbell-uamlite-default-g9dnv 1/1 Running 0 22h
ucp ingress-86576d6599-zdf7q 1/1 Running 0 23h
ucp ingress-error-pages-5c97bb46bb-c9jhz 1/1 Running 0 23h
ucp keystone-api-7c99d578b7-fvf6j 1/1 Running 0 23h
ucp keystone-bootstrap-qsf4g 0/1 Completed 0 23h
ucp keystone-credential-setup-6xsjs 0/1 Completed 0 23h
ucp keystone-db-init-lpj2t 0/1 Completed 0 23h
ucp keystone-db-sync-m28br 0/1 Completed 0 23h
ucp keystone-domain-manage-rj66q 0/1 Completed 0 23h
ucp keystone-fernet-rotate-1553644800-m2gx8 0/1 Init:0/1 0 21h
ucp keystone-fernet-setup-8ffkj 0/1 Completed 0 23h
ucp keystone-rabbit-init-95xht 0/1 Completed 0 23h
ucp maas-bootstrap-admin-user-7bfgx 0/1 Init:0/1 0 21h
ucp maas-db-init-r5mkj 0/1 Init:0/1 0 21h
ucp maas-db-sync-gpwq2 0/1 Init:0/1 0 21h
ucp maas-export-api-key-tk9c8 0/1 Init:0/1 0 21h
ucp maas-import-resources-kftsl 0/1 Init:0/1 0 21h
ucp maas-ingress-567777887c-v9fzp 1/2 CrashLoopBackOff 261 21h
ucp maas-ingress-errors-798cbd8c4c-q987s 0/1 ContainerCreating 0 21h
ucp maas-rack-0 0/1 Init:0/1 0 21h
ucp maas-rack-1 0/1 Pending 0 21h
ucp maas-region-0 0/1 Init:0/1 0 21h
ucp mariadb-ingress-6c9dcbc755-bp5r6 1/1 Running 0 23h
ucp mariadb-ingress-error-pages-b7f9f85c7-qbnh8 1/1 Running 0 23h
ucp mariadb-server-0 1/1 Running 0 23h
ucp postgresql-0 1/1 Running 0 23h
*root@mtn15r07c001:~# kubectl describe pods maas-ingress-567777887c-v9fzp --namespace=ucp*
Name: maas-ingress-567777887c-v9fzp
Namespace: ucp
Node: mtn15r07c001/135.21.35.69
Start Time: Tue, 26 Mar 2019 23:55:29 +0000
Labels: application=maas
component=ingress
pod-template-hash=1233334437
release_group=airship-maas
Annotations: configmap-bin-hash=4b7d801062899a7d636c1f3e1c943bfad865192b964fea74b9057d85eec28d52
configmap-etc-hash=6ae6a39a1a7edf9c6d28d6952cf28fa1673062dc655a10ed04484f4ac80ffb8a
Status: Running
IP: 135.21.35.69
Controlled By: ReplicaSet/maas-ingress-567777887c
Init Containers:
init:
Container ID: docker://6abdaa7b1adc8a1c3e1a3e7cb39c20684c63aa19a72b3261298cfdff21cd0188
Image: quay.io/stackanetes/kubernetes-entrypoint:v0.3.1
Image ID: docker-pullable:// quay.io/stackanetes/kubernetes-entrypoint@sha256:32b1b657ee4bcc9cc7a1529e31d8e1a06376172373ee020f97f3e78168fde4b6
Port: <none>
Host Port: <none>
Command:
kubernetes-entrypoint
State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 26 Mar 2019 23:55:33 +0000
Finished: Tue, 26 Mar 2019 23:55:33 +0000
Ready: True
Restart Count: 0
Environment:
POD_NAME: maas-ingress-567777887c-v9fzp (v1: metadata.name)
NAMESPACE: ucp (v1:metadata.namespace)
INTERFACE_NAME: eth0
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/
DEPENDENCY_SERVICE:
DEPENDENCY_DAEMONSET:
DEPENDENCY_CONTAINER:
DEPENDENCY_POD_JSON:
COMMAND: echo done
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from maas-ingress-token-8qlcl (ro)
maas-ingress-vip-init:
Container ID: docker://195e75792c7592c90cff15703a7a977c721b4f863e0834033a72302e683a3a7d
Image: docker.io/busybox:latest
Image ID: docker-pullable://busybox@sha256 :061ca9704a714ee3e8b80523ec720c64f6209ad3f97c0ff7cb9ec7d19f15149f
Port: <none>
Host Port: <none>
Command:
/tmp/maas-vip-configure.sh
start
State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 26 Mar 2019 23:55:34 +0000
Finished: Tue, 26 Mar 2019 23:55:34 +0000
Ready: True
Restart Count: 0
Environment:
addr: 192.169.1.5/32
interface: maas-vip
mode: routed
Mounts:
/mnt/host-rootfs from host-rootfs (ro)
/tmp/maas-vip-configure.sh from maas-bin (ro)
/var/run/secrets/kubernetes.io/serviceaccount from maas-ingress-token-8qlcl (ro)
Containers:
maas-ingress-vip:
Container ID: docker://8be5e78def6312c70d48f0bbe57c77e4b95ab3e5a417416cc0c6d1d3d44ff536
Image: docker.io/busybox:latest
Image ID: docker-pullable://busybox@sha256 :061ca9704a714ee3e8b80523ec720c64f6209ad3f97c0ff7cb9ec7d19f15149f
Port: <none>
Host Port: <none>
Command:
/bin/init
State: Running
Started: Tue, 26 Mar 2019 23:55:35 +0000
Ready: True
Restart Count: 0
Environment:
addr: 192.169.1.5/32
interface: maas-vip
mode: routed
Mounts:
/etc/inittab from maas-etc (ro)
/tmp/maas-vip-configure.sh from maas-bin (ro)
/var/run/secrets/kubernetes.io/serviceaccount from maas-ingress-token-8qlcl (ro)
maas-ingress:
Container ID: docker://a892c56049d267479c0298e3fe077eed4cc0a73968d069ad5b4eee8fb6f9431f
Image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.20.0
Image ID: docker-pullable:// quay.io/kubernetes-ingress-controller/nginx-ingress-controller@sha256:f6180c5397d2361c317aff1314dc192ab0f9f515346a5319422cdc264f05d2d9
Port: <none>
Host Port: <none>
Command:
/tmp/maas-ingress.sh
start
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 255
Started: Wed, 27 Mar 2019 20:13:16 +0000
Finished: Wed, 27 Mar 2019 20:13:16 +0000
Ready: False
Restart Count: 243
Environment:
POD_NAMESPACE: ucp (v1:metadata.namespace)
POD_NAME: maas-ingress-567777887c-v9fzp (v1:metadata.name )
RELEASE_NAME: airship-maas
HTTP_PORT: 80
HTTPS_PORT: 443
HEALTHZ_PORT: 10254
STATUS_PORT: 18080
ERROR_PAGE_SERVICE: maas-ingress-error
DEFAULT_ERROR_PORT: 8181
Mounts:
/tmp/maas-ingress.sh from maas-bin (ro)
/var/run/secrets/kubernetes.io/serviceaccount from maas-ingress-token-8qlcl (ro)
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
Volumes:
maas-bin:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: maas-bin
Optional: false
maas-etc:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: maas-etc
Optional: false
host-rootfs:
Type: HostPath (bare host directory volume)
Path: /
HostPathType:
maas-ingress-token-8qlcl:
Type: Secret (a volume populated by a Secret)
SecretName: maas-ingress-token-8qlcl
Optional: false
QoS Class: BestEffort
Node-Selectors: ucp-control-plane=enabled
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning BackOff 2m (x5500 over 20h) kubelet, mtn15r07c001 Back-off restarting failed container
root@mtn15r07c001:~#
*Regards,* Muzammil Architect Network & Shared Services 469-600-0477
[image: cid:image001.png@01D32007.C3B5AE40]
_______________________________________________ Airship-discuss mailing list Airship-discuss@lists.airshipit.org http://lists.airshipit.org/cgi-bin/mailman/listinfo/airship-discuss
