From santosh.thapamager at as.ntt-at.co.jp Wed Dec 5 00:26:11 2018 From: santosh.thapamager at as.ntt-at.co.jp (santosh.thapamager) Date: Wed, 5 Dec 2018 09:26:11 +0900 Subject: [Airship-discuss] =?utf-8?q?Airship-in-a-bottle=EF=BC=9Adev=5Fmin?= =?utf-8?q?imal=2Cdev=5Fsingle=5Fnode_behind_proxy?= In-Reply-To: <7C64A75C21BB8D43BD75BB18635E4D896CE31033@MOSTLS1MSGUSRFF.ITServices.sbc.com> References: <001e01d487ac$d6963580$83c2a080$@as.ntt-at.co.jp> <7C64A75C21BB8D43BD75BB18635E4D896CE31033@MOSTLS1MSGUSRFF.ITServices.sbc.com> Message-ID: <003201d48c31$1fe90cc0$5fbb2640$@as.ntt-at.co.jp> Hi Matt、 Thanks for reaching out. Is there any update regarding the issue. I would be greatly thankful if I could get any help to fix the issue. Best Regards, Santosh -----Original Message----- From: MCEUEN, MATT Sent: Thursday, November 29, 2018 11:00 PM To: santosh.thapamager ; airship-discuss at lists.airshipit.org Subject: RE: [Airship-discuss] Airship-in-a-bottle:dev_minimal,dev_single_node behind proxy Hi Santosh! I heard yesterday that there may be an issue with the Airship-in-a-Bottle multinode gate behind the proxy as well. I suspect it's the same issue - I'll try to replicate it today. Will let you know. Thanks, Matt -----Original Message----- From: santosh.thapamager Sent: Thursday, November 29, 2018 12:29 AM To: airship-discuss at lists.airshipit.org Subject: [Airship-discuss] Airship-in-a-bottle:dev_minimal,dev_single_node behind proxy Hi all!! I am trying to install dev_minimal and dev_single_node behind the proxy. I could find the installation instruction of dev_minimal behind proxy however I could not find instruction for installing dev_single_node behind proxy. I would be thankful if you could answer me if we can deploy dev_single_node behind proxy. Currently I have be trying to install dev_minimal behind proxy. During installation I encountered following issues. Issue 1: During installation I found that helm test for airship-maas has failed. This is the snippet of installation log. ************************************************************** ************************************** **** 2018-11-26 04:41:37.536 8 INFO armada.handlers.armada [-] Install completed with results from Tiller: {'namespace': 'ucp', 'status': 'DEPLOYED', 'description': 'Install complete', 'version': 1, 'release': 'airship-maas'} 2018-11-26 04:41:37.537 8 INFO armada.handlers.armada [-] Running sequenced test, timeout remaining: 385s. 2018-11-26 04:41:37.537 8 INFO armada.handlers.tiller [-] Running Helm test: release=airship-maas, timeout=385 2018-11-26 04:41:37.556 8 INFO armada.handlers.tiller [-] RUNNING: airship-maas-api-test 2018-11-26 04:41:42.078 8 INFO armada.handlers.tiller [-] FAILED: airship-maas-api-test, run `kubectl logs airship-maas-api-test --namespace ucp` for more info 2018-11-26 04:41:42.136 8 INFO armada.handlers.tiller [-] 1 test(s) failed 2018-11-26 04:41:42.275 8 INFO armada.handlers.armada [-] Test failed for release: airship-maas 2018-11-26 04:41:42.275 8 ERROR armada.cli [-] Caught internal exception: armada.exceptions.tiller_exceptions.TestFailedException: Test failed for release: airship-maas 2018-11-26 04:41:42.275 8 ERROR armada.cli Traceback (most recent call last): 2018-11-26 04:41:42.275 8 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/cli/__init__.py", line 39, in safe_invoke 2018-11-26 04:41:42.275 8 ERROR armada.cli self.invoke() 2018-11-26 04:41:42.275 8 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/cli/apply.py", line 217, in invoke 2018-11-26 04:41:42.275 8 ERROR armada.cli resp = armada.sync() 2018-11-26 04:41:42.275 8 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/armada.py", line 494, in sync 2018-11-26 04:41:42.275 8 ERROR armada.cli self._test_chart(*test_chart_args) 2018-11-26 04:41:42.275 8 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/armada.py", line 591, in _test_chart 2018-11-26 04:41:42.275 8 ERROR armada.cli raise tiller_exceptions.TestFailedException(release_name) 2018-11-26 04:41:42.275 8 ERROR armada.cli armada.exceptions.tiller_exceptions.TestFailedException: Test failed for release: airship-maas 2018-11-26 04:41:42.275 8 ERROR armada.cli ************************************************************** ************************************** **** Issue 2: During installation While deploying site using bash execute_shipyard_action.sh 'deploy_site‘, the output of the installation log said the site deployment has been completed however there are some steps which has failed status. Here is the snippet of the installation log. ************************************************************** ************************************** **** + describe_action='Name: deploy_site Action: action/01CX75ZXFH9RRREF3AQSS8NVXS Lifecycle: Complete Parameters: {} Datetime: 2018-11-26 04:56:10.993587+00:00 Dag Status: success Context Marker: e552d7bb-a345-43bf-8875-139ec02ba715 User: shipyard Steps Index State Footnotes step/01CX75ZXFH9RRREF3AQSS8NVXS/action_xcom 1 success step/01CX75ZXFH9RRREF3AQSS8NVXS/dag_concurrency_check 2 success step/01CX75ZXFH9RRREF3AQSS8NVXS/preflight 3 success step/01CX75ZXFH9RRREF3AQSS8NVXS/get_rendered_doc 4 success step/01CX75ZXFH9RRREF3AQSS8NVXS/deployment_configuration 5 success step/01CX75ZXFH9RRREF3AQSS8NVXS/validate_site_design 6 success step/01CX75ZXFH9RRREF3AQSS8NVXS/drydock_build 7 success step/01CX75ZXFH9RRREF3AQSS8NVXS/verify_site 8 success step/01CX75ZXFH9RRREF3AQSS8NVXS/armada_get_status 9 success step/01CX75ZXFH9RRREF3AQSS8NVXS/prepare_site 10 success step/01CX75ZXFH9RRREF3AQSS8NVXS/armada_build 11 failed step/01CX75ZXFH9RRREF3AQSS8NVXS/armada_post_apply 12 failed step/01CX75ZXFH9RRREF3AQSS8NVXS/ucp_preflight_check 13 success step/01CX75ZXFH9RRREF3AQSS8NVXS/deckhand_validate_site_design 14 success step/01CX75ZXFH9RRREF3AQSS8NVXS/armada_validate_site_design 15 success step/01CX75ZXFH9RRREF3AQSS8NVXS/promenade_validate_site_design 16 success step/01CX75ZXFH9RRREF3AQSS8NVXS/drydock_validate_site_design 17 success step/01CX75ZXFH9RRREF3AQSS8NVXS/prepare_and_deploy_nodes 18 success step/01CX75ZXFH9RRREF3AQSS8NVXS/armada_get_releases 19 upstream_failed step/01CX75ZXFH9RRREF3AQSS8NVXS/create_action_tag 20 success Commands User Datetime invoke shipyard 2018-11-26 04:56:12.256963+00:00 Validations: None ' ++ awk '{print $8}' ++ echo Name: deploy_site $'\r' Action: action/01CX75ZXFH9RRREF3AQSS8NVXS $'\r' Lifecycle: Complete $'\r' Parameters: '{}' $'\r' Datetime: 2018-11-26 04:56:10.993587+00:00 $'\r' Dag Status: success $'\r' Context Marker: e552d7bb-a345-43bf-8875- 139ec02ba715 $'\r' User: shipyard $'\r' $'\r' Steps Index State Footnotes $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/action_xcom 1 success $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/dag_concurrency_check 2 success $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/preflight 3 success $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/get_rendered_doc 4 success $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/deployment_configuration 5 success $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/validate_site_design 6 success $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/drydock_build 7 success $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/verify_site 8 success $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/armada_get_status 9 success $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/prepare_site 10 success $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/armada_build 11 failed $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/armada_post_apply 12 failed $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/ucp_preflight_check 13 success $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/deckhand_validate_site_design 14 success $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/armada_validate_site_design 15 success $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/promenade_validate_site_design 16 success $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/drydock_validate_site_design 17 success $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/prepare_and_deploy_nodes 18 success $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/armada_get_releases 19 upstream_failed $'\r' step/01CX75ZXFH9RRREF3AQSS8NVXS/create_action_tag 20 success $'\r' $'\r' $'\r' $'\r' Commands User Datetime $'\r' invoke shipyard 2018-11-26 04:56:12.256963+00:00 $'\r' $'\r' Validations: $'None\r' $'\r' $'\r' $'\r' + action_lifecycle=Complete + [[ Complete == \C\o\m\p\l\e\t\e ]] + echo -e '\nSuccessfully performed' deploy_site Successfully performed deploy_site + echo -e '\n' + break + [[ Complete == \C\o\m\p\l\e\t\e ]] + exit 0 + [[ 30 -ge 40 ]] + clean + set +x To remove files generated during this script's execution, delete /root/deploy. This VM is disposable. Re-deployment in this same VM will lead to unpredictable results. ************************************************************** ************************************** **** I have been stuck in these issues. I would be very greatful if I can get answers regarding these issues. Best Regards, Santosh Thapa Magar _______________________________________________ Airship-discuss mailing list Airship-discuss at lists.airshipit.org https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.airshipit.org_cgi-2Dbin_mailman_listinfo_airship-2Ddiscuss&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=87Zg_ovdhJYwPJjp2ZFeXKqWHTR4JzsXSoC2NVLRfw4&s=iru3CxcNSwfW_QABEGYTVk2EdOcNNNrHwWxTFh5KIx0&e= From claire at openstack.org Wed Dec 5 22:33:08 2018 From: claire at openstack.org (Claire Massey) Date: Wed, 5 Dec 2018 16:33:08 -0600 Subject: [Airship-discuss] [Newsletter] What's Happening in the Open Infrastructure Community Message-ID: Hi everyone, This week, the OSF team distributed the first Open Infrastructure Community Newsletter. Our goal with the bi-weekly newsletter is to provide a digest of the latest developments and activities across open infrastructure projects, events and users. This week, the newsletter highlighted the Berlin Summit as well as brief updates from Airship and other OSF projects - OpenStack, StarlingX, Kata Containers and Zuul. You can checkout the full newsletter on Superuser [1], and if you are interested in receiving the upcoming newsletters, you can subscribe here [2]. If you would like to contribute to a future newsletter or have feedback, please reach out to community at openstack.org . Thanks, Claire [1] http://superuser.openstack.org/articles/inside-open-infrastructure-newsletter1 [2] https://www.openstack.org/community/email-signup -------------- next part -------------- An HTML attachment was scrubbed... URL: From Daniel.Balsiger at swisscom.com Mon Dec 10 14:31:42 2018 From: Daniel.Balsiger at swisscom.com (Balsiger Daniel, INI-EAI-INO-MBL) Date: Mon, 10 Dec 2018 14:31:42 +0000 Subject: [Airship-discuss] Airship in a bottle install failure... Message-ID: Hi there Airshippers I wanted to try installing airhip-in-a-bottle in a fresh Ubuntu16.04 VM running on OpenStack over the weekend. I followed the instructions on airshipit.org --snip sudo -i mkdir -p /root/deploy && cd "$_" git clone https://github.com/openstack/airship-in-a-bottle cd /root/deploy/airship-in-a-bottle/manifests/dev_single_node ./airship-in-a-bottle.sh --snip The script is still running after 2 days so I think something went wrong. The output is repeating itself, listing running/restarted containers (I think postgres is the one not coming up properly) --snip NAMESPACE NAME READY STATUS RESTARTS AGE kube-system auxiliary-etcd-airship 3/3 Running 0 2d kube-system bootstrap-armada-airship 4/4 Running 0 2d kube-system calico-etcd-airship 1/1 Running 0 2d kube-system calico-etcd-anchor-2jrqf 1/1 Running 0 2d kube-system calico-kube-controllers-6d5997c8c4-zxftw 1/1 Running 0 2d kube-system calico-node-l885v 2/2 Running 0 2d kube-system calico-settings-kbq79 0/1 Completed 0 2d kube-system coredns-d4587b7f8-7wlpm 1/1 Running 0 2d kube-system coredns-d4587b7f8-98t62 1/1 Running 0 2d kube-system coredns-d4587b7f8-frx5w 1/1 Running 0 2d kube-system haproxy-airship 1/1 Running 0 2d kube-system ingress-8w29l 1/1 Running 0 2d kube-system ingress-error-pages-5ccf96bf7d-jsrww 1/1 Running 0 2d kube-system kubernetes-apiserver-airship 1/1 Running 0 2d kube-system kubernetes-apiserver-anchor-55x59 1/1 Running 0 2d kube-system kubernetes-controller-manager-airship 1/1 Running 1 2d kube-system kubernetes-controller-manager-anchor-2wsg8 1/1 Running 0 2d kube-system kubernetes-etcd-airship 1/1 Running 0 2d kube-system kubernetes-etcd-anchor-64wgx 1/1 Running 0 2d kube-system kubernetes-proxy-9s68d 1/1 Running 0 2d kube-system kubernetes-scheduler-airship 1/1 Running 1 2d kube-system kubernetes-scheduler-anchor-npmn7 1/1 Running 0 2d kube-system nfs-provisioner-7799d64d59-scgdc 1/1 Running 0 2d ucp airship-ucp-rabbitmq-rabbitmq-0 1/1 Running 0 2d ucp ingress-6cd5b89d5d-gp98m 1/1 Running 0 2d ucp ingress-error-pages-5c97bb46bb-zq5cw 1/1 Running 0 2d ucp mariadb-ingress-6c4f9c76f-bjd99 0/1 Running 0 2d ucp mariadb-ingress-6c4f9c76f-xjrn9 0/1 Running 0 2d ucp mariadb-ingress-error-pages-5dd6fbb75-89qmx 1/1 Running 0 2d ucp mariadb-server-0 0/1 Running 0 2d ucp postgresql-0 1/1 Running 1 2d --snip Anybody out there with the same problem or (even better) a solution for it? Should I pick a particular branch or commit? Any help is appreciated. Best Regards, Daniel -------------- next part -------------- An HTML attachment was scrubbed... URL: From MM9745 at att.com Mon Dec 10 15:29:01 2018 From: MM9745 at att.com (MCEUEN, MATT) Date: Mon, 10 Dec 2018 15:29:01 +0000 Subject: [Airship-discuss] Airship in a bottle install failure... In-Reply-To: References: Message-ID: <7C64A75C21BB8D43BD75BB18635E4D896CEAD143@MOSTLS1MSGUSRFF.ITServices.sbc.com> Hi Daniel, Are you running from behind a proxy? A few fixes have gone in recently, so I want to try to reproduce your setup. Thanks, Matt From: Balsiger Daniel, INI-EAI-INO-MBL Sent: Monday, December 10, 2018 8:32 AM To: airship-discuss at lists.airshipit.org Subject: [Airship-discuss] Airship in a bottle install failure... Hi there Airshippers I wanted to try installing airhip-in-a-bottle in a fresh Ubuntu16.04 VM running on OpenStack over the weekend. I followed the instructions on airshipit.org --snip sudo -i mkdir -p /root/deploy && cd "$_" git clone https://github.com/openstack/airship-in-a-bottle cd /root/deploy/airship-in-a-bottle/manifests/dev_single_node ./airship-in-a-bottle.sh --snip The script is still running after 2 days so I think something went wrong. The output is repeating itself, listing running/restarted containers (I think postgres is the one not coming up properly) --snip NAMESPACE NAME READY STATUS RESTARTS AGE kube-system auxiliary-etcd-airship 3/3 Running 0 2d kube-system bootstrap-armada-airship 4/4 Running 0 2d kube-system calico-etcd-airship 1/1 Running 0 2d kube-system calico-etcd-anchor-2jrqf 1/1 Running 0 2d kube-system calico-kube-controllers-6d5997c8c4-zxftw 1/1 Running 0 2d kube-system calico-node-l885v 2/2 Running 0 2d kube-system calico-settings-kbq79 0/1 Completed 0 2d kube-system coredns-d4587b7f8-7wlpm 1/1 Running 0 2d kube-system coredns-d4587b7f8-98t62 1/1 Running 0 2d kube-system coredns-d4587b7f8-frx5w 1/1 Running 0 2d kube-system haproxy-airship 1/1 Running 0 2d kube-system ingress-8w29l 1/1 Running 0 2d kube-system ingress-error-pages-5ccf96bf7d-jsrww 1/1 Running 0 2d kube-system kubernetes-apiserver-airship 1/1 Running 0 2d kube-system kubernetes-apiserver-anchor-55x59 1/1 Running 0 2d kube-system kubernetes-controller-manager-airship 1/1 Running 1 2d kube-system kubernetes-controller-manager-anchor-2wsg8 1/1 Running 0 2d kube-system kubernetes-etcd-airship 1/1 Running 0 2d kube-system kubernetes-etcd-anchor-64wgx 1/1 Running 0 2d kube-system kubernetes-proxy-9s68d 1/1 Running 0 2d kube-system kubernetes-scheduler-airship 1/1 Running 1 2d kube-system kubernetes-scheduler-anchor-npmn7 1/1 Running 0 2d kube-system nfs-provisioner-7799d64d59-scgdc 1/1 Running 0 2d ucp airship-ucp-rabbitmq-rabbitmq-0 1/1 Running 0 2d ucp ingress-6cd5b89d5d-gp98m 1/1 Running 0 2d ucp ingress-error-pages-5c97bb46bb-zq5cw 1/1 Running 0 2d ucp mariadb-ingress-6c4f9c76f-bjd99 0/1 Running 0 2d ucp mariadb-ingress-6c4f9c76f-xjrn9 0/1 Running 0 2d ucp mariadb-ingress-error-pages-5dd6fbb75-89qmx 1/1 Running 0 2d ucp mariadb-server-0 0/1 Running 0 2d ucp postgresql-0 1/1 Running 1 2d --snip Anybody out there with the same problem or (even better) a solution for it? Should I pick a particular branch or commit? Any help is appreciated. Best Regards, Daniel -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.arnoldy at suse.com Mon Dec 10 15:40:15 2018 From: alex.arnoldy at suse.com (Alex Arnoldy) Date: Mon, 10 Dec 2018 15:40:15 +0000 Subject: [Airship-discuss] Airship in a bottle install failure... References: Message-ID: <545AF708D41B364BA4146E47647D2D2AE3A2DF@prvxmb04.microfocus.com> Hello Daniel, I had launched the same thing but then got pulled away from it and totally forgot to check it. I thought I'd be able to tell you I got it running, but I don't think that's the case. For my installation all of the containers are either Running or Completed except for the following: ucp maas-bootstrap-admin-user-26lhf 0/1 Init:0/1 0 3d 10.97.196.132 airship ucp maas-db-init-v7z6r 0/1 CrashLoopBackOff 953 3d 10.97.196.143 airship ucp maas-db-sync-5c24z 0/1 Init:0/1 0 3d 10.97.196.162 airship ucp maas-export-api-key-zhfxp 0/1 Init:0/1 0 3d 10.97.196.147 airship ucp maas-import-resources-j6tzd 0/1 Init:0/1 0 3d 10.97.196.152 airship ucp maas-rack-0 0/1 Init:0/1 0 3d 172.16.231.60 airship ucp maas-region-0 0/1 Init:0/1 0 3d 10.97.196.146 airship The only output from the maas-db-init pod is this: root at airship:~# kubectl logs -n ucp maas-db-init-v7z6r + export HOME=/tmp + HOME=/tmp + [[ ! -v DB_HOST ]] + [[ ! -v ROOT_DB_USER ]] + [[ ! -v PGPASSWORD ]] + [[ ! -v USER_DB_USER ]] + [[ ! -v USER_DB_PASS ]] + [[ ! -v USER_DB_NAME ]] + echo 'Got DB connection info' Got DB connection info + pgsql_superuser_cmd 'SELECT 1 FROM pg_database WHERE datname = '\''maasdb'\''' + DB_COMMAND='SELECT 1 FROM pg_database WHERE datname = '\''maasdb'\''' + [[ ! -z '' ]] + grep -q 1 + psql -h postgresql.ucp.svc.cluster.local -p 5432 -U postgres '--command=SELECT 1 FROM pg_database WHERE datname = '\''maasdb'\''' psql: FATAL: no pg_hba.conf entry for host "10.97.196.143", user "postgres", database "postgres", SSL off + pgsql_superuser_cmd 'CREATE DATABASE maasdb' + DB_COMMAND='CREATE DATABASE maasdb' + [[ ! -z '' ]] + psql -h postgresql.ucp.svc.cluster.local -p 5432 -U postgres '--command=CREATE DATABASE maasdb' psql: FATAL: no pg_hba.conf entry for host "10.97.196.143", user "postgres", database "postgres", SSL off Unfortunately, I haven't been able to troubleshoot this any further. Best, --Alex. Alex Arnoldy Software Defined Infrastructure Solutions Architect SUSE Alex.Arnoldy at SUSE.com (P) +1 (408) 828-9797 From: Balsiger Daniel, INI-EAI-INO-MBL > Sent: Monday, December 10, 2018 6:32 AM To: airship-discuss at lists.airshipit.org Subject: [Airship-discuss] Airship in a bottle install failure... Hi there Airshippers I wanted to try installing airhip-in-a-bottle in a fresh Ubuntu16.04 VM running on OpenStack over the weekend. I followed the instructions on airshipit.org --snip sudo -i mkdir -p /root/deploy && cd "$_" git clone https://github.com/openstack/airship-in-a-bottle cd /root/deploy/airship-in-a-bottle/manifests/dev_single_node ./airship-in-a-bottle.sh --snip The script is still running after 2 days so I think something went wrong. The output is repeating itself, listing running/restarted containers (I think postgres is the one not coming up properly) --snip NAMESPACE NAME READY STATUS RESTARTS AGE kube-system auxiliary-etcd-airship 3/3 Running 0 2d kube-system bootstrap-armada-airship 4/4 Running 0 2d kube-system calico-etcd-airship 1/1 Running 0 2d kube-system calico-etcd-anchor-2jrqf 1/1 Running 0 2d kube-system calico-kube-controllers-6d5997c8c4-zxftw 1/1 Running 0 2d kube-system calico-node-l885v 2/2 Running 0 2d kube-system calico-settings-kbq79 0/1 Completed 0 2d kube-system coredns-d4587b7f8-7wlpm 1/1 Running 0 2d kube-system coredns-d4587b7f8-98t62 1/1 Running 0 2d kube-system coredns-d4587b7f8-frx5w 1/1 Running 0 2d kube-system haproxy-airship 1/1 Running 0 2d kube-system ingress-8w29l 1/1 Running 0 2d kube-system ingress-error-pages-5ccf96bf7d-jsrww 1/1 Running 0 2d kube-system kubernetes-apiserver-airship 1/1 Running 0 2d kube-system kubernetes-apiserver-anchor-55x59 1/1 Running 0 2d kube-system kubernetes-controller-manager-airship 1/1 Running 1 2d kube-system kubernetes-controller-manager-anchor-2wsg8 1/1 Running 0 2d kube-system kubernetes-etcd-airship 1/1 Running 0 2d kube-system kubernetes-etcd-anchor-64wgx 1/1 Running 0 2d kube-system kubernetes-proxy-9s68d 1/1 Running 0 2d kube-system kubernetes-scheduler-airship 1/1 Running 1 2d kube-system kubernetes-scheduler-anchor-npmn7 1/1 Running 0 2d kube-system nfs-provisioner-7799d64d59-scgdc 1/1 Running 0 2d ucp airship-ucp-rabbitmq-rabbitmq-0 1/1 Running 0 2d ucp ingress-6cd5b89d5d-gp98m 1/1 Running 0 2d ucp ingress-error-pages-5c97bb46bb-zq5cw 1/1 Running 0 2d ucp mariadb-ingress-6c4f9c76f-bjd99 0/1 Running 0 2d ucp mariadb-ingress-6c4f9c76f-xjrn9 0/1 Running 0 2d ucp mariadb-ingress-error-pages-5dd6fbb75-89qmx 1/1 Running 0 2d ucp mariadb-server-0 0/1 Running 0 2d ucp postgresql-0 1/1 Running 1 2d --snip Anybody out there with the same problem or (even better) a solution for it? Should I pick a particular branch or commit? Any help is appreciated. Best Regards, Daniel -------------- next part -------------- An HTML attachment was scrubbed... URL: From Daniel.Balsiger at swisscom.com Mon Dec 10 15:54:05 2018 From: Daniel.Balsiger at swisscom.com (Balsiger Daniel, INI-EAI-INO-MBL) Date: Mon, 10 Dec 2018 15:54:05 +0000 Subject: [Airship-discuss] Airship in a bottle install failure... In-Reply-To: <7C64A75C21BB8D43BD75BB18635E4D896CEAD143@MOSTLS1MSGUSRFF.ITServices.sbc.com> References: , <7C64A75C21BB8D43BD75BB18635E4D896CEAD143@MOSTLS1MSGUSRFF.ITServices.sbc.com> Message-ID: Hi Matt Thanks for the quick reply. No, no proxy is in place. VM has a private tenant network address and one floating IP assigned. The floating IP is regularly NATed (Masquerade) to the outside world. I access the VM by floating IP via SSH. KR Daniel ________________________________ From: MCEUEN, MATT Sent: Monday, December 10, 2018 4:29 PM To: Balsiger Daniel, INI-EAI-INO-MBL; airship-discuss at lists.airshipit.org Subject: RE: Airship in a bottle install failure... Hi Daniel, Are you running from behind a proxy? A few fixes have gone in recently, so I want to try to reproduce your setup. Thanks, Matt From: Balsiger Daniel, INI-EAI-INO-MBL Sent: Monday, December 10, 2018 8:32 AM To: airship-discuss at lists.airshipit.org Subject: [Airship-discuss] Airship in a bottle install failure... Hi there Airshippers I wanted to try installing airhip-in-a-bottle in a fresh Ubuntu16.04 VM running on OpenStack over the weekend. I followed the instructions on airshipit.org --snip sudo -i mkdir -p /root/deploy && cd "$_" git clone https://github.com/openstack/airship-in-a-bottle cd /root/deploy/airship-in-a-bottle/manifests/dev_single_node ./airship-in-a-bottle.sh --snip The script is still running after 2 days so I think something went wrong. The output is repeating itself, listing running/restarted containers (I think postgres is the one not coming up properly) --snip NAMESPACE NAME READY STATUS RESTARTS AGE kube-system auxiliary-etcd-airship 3/3 Running 0 2d kube-system bootstrap-armada-airship 4/4 Running 0 2d kube-system calico-etcd-airship 1/1 Running 0 2d kube-system calico-etcd-anchor-2jrqf 1/1 Running 0 2d kube-system calico-kube-controllers-6d5997c8c4-zxftw 1/1 Running 0 2d kube-system calico-node-l885v 2/2 Running 0 2d kube-system calico-settings-kbq79 0/1 Completed 0 2d kube-system coredns-d4587b7f8-7wlpm 1/1 Running 0 2d kube-system coredns-d4587b7f8-98t62 1/1 Running 0 2d kube-system coredns-d4587b7f8-frx5w 1/1 Running 0 2d kube-system haproxy-airship 1/1 Running 0 2d kube-system ingress-8w29l 1/1 Running 0 2d kube-system ingress-error-pages-5ccf96bf7d-jsrww 1/1 Running 0 2d kube-system kubernetes-apiserver-airship 1/1 Running 0 2d kube-system kubernetes-apiserver-anchor-55x59 1/1 Running 0 2d kube-system kubernetes-controller-manager-airship 1/1 Running 1 2d kube-system kubernetes-controller-manager-anchor-2wsg8 1/1 Running 0 2d kube-system kubernetes-etcd-airship 1/1 Running 0 2d kube-system kubernetes-etcd-anchor-64wgx 1/1 Running 0 2d kube-system kubernetes-proxy-9s68d 1/1 Running 0 2d kube-system kubernetes-scheduler-airship 1/1 Running 1 2d kube-system kubernetes-scheduler-anchor-npmn7 1/1 Running 0 2d kube-system nfs-provisioner-7799d64d59-scgdc 1/1 Running 0 2d ucp airship-ucp-rabbitmq-rabbitmq-0 1/1 Running 0 2d ucp ingress-6cd5b89d5d-gp98m 1/1 Running 0 2d ucp ingress-error-pages-5c97bb46bb-zq5cw 1/1 Running 0 2d ucp mariadb-ingress-6c4f9c76f-bjd99 0/1 Running 0 2d ucp mariadb-ingress-6c4f9c76f-xjrn9 0/1 Running 0 2d ucp mariadb-ingress-error-pages-5dd6fbb75-89qmx 1/1 Running 0 2d ucp mariadb-server-0 0/1 Running 0 2d ucp postgresql-0 1/1 Running 1 2d --snip Anybody out there with the same problem or (even better) a solution for it? Should I pick a particular branch or commit? Any help is appreciated. Best Regards, Daniel -------------- next part -------------- An HTML attachment was scrubbed... URL: From Daniel.Balsiger at swisscom.com Mon Dec 10 15:59:19 2018 From: Daniel.Balsiger at swisscom.com (Balsiger Daniel, INI-EAI-INO-MBL) Date: Mon, 10 Dec 2018 15:59:19 +0000 Subject: [Airship-discuss] Airship in a bottle install failure... In-Reply-To: <545AF708D41B364BA4146E47647D2D2AE3A29F@prvxmb04.microfocus.com> References: , <545AF708D41B364BA4146E47647D2D2AE3A29F@prvxmb04.microfocus.com> Message-ID: Hi there Alex. Thanks for the reply. >From your logs it seems to me like maas-db-init is not able to connect to postgres DB with psql client to the other container. (clients should be listed by IP in pg_hba.conf, wildcards should be possible....) I will try another install from a fresh VM and let you know if I have the same one. Thanks ________________________________ From: Alex Arnoldy Sent: Monday, December 10, 2018 4:26 PM To: Balsiger Daniel, INI-EAI-INO-MBL; airship-discuss at lists.airshipit.org Subject: RE: Airship in a bottle install failure... Hello Daniel, I had launched the same thing but then got pulled away from it and totally forgot to check it. I thought I’d be able to tell you I got it running, but I don’t think that’s the case. For my installation, the maas-db-init pod seems to be stuck in CrashLoopBackOff: root at airship:~# kubectl get pods -o wide --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE kube-system auxiliary-etcd-airship 3/3 Running 0 3d 172.16.231.60 airship kube-system bootstrap-armada-airship 4/4 Running 0 3d 172.16.231.60 airship kube-system calico-etcd-airship 1/1 Running 0 3d 172.16.231.60 airship kube-system calico-etcd-anchor-nqntm 1/1 Running 0 3d 172.16.231.60 airship kube-system calico-kube-controllers-6d5997c8c4-bm9rk 1/1 Running 0 3d 172.16.231.60 airship kube-system calico-node-2sgkk 2/2 Running 0 3d 172.16.231.60 airship kube-system calico-settings-bqs5v 0/1 Completed 0 3d 172.16.231.60 airship kube-system coredns-d4587b7f8-bbpr6 1/1 Running 13 3d 10.97.196.130 airship kube-system coredns-d4587b7f8-tdl8v 1/1 Running 11 3d 10.97.196.131 airship kube-system coredns-d4587b7f8-ts9lr 1/1 Running 12 3d 10.97.196.129 airship kube-system haproxy-airship 1/1 Running 0 3d 172.16.231.60 airship kube-system ingress-error-pages-5ccf96bf7d-8q6g5 1/1 Running 0 3d 10.97.196.133 airship kube-system ingress-fmrmk 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-apiserver-airship 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-apiserver-anchor-rkfpz 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-controller-manager-airship 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-controller-manager-anchor-75k62 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-etcd-airship 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-etcd-anchor-hzcrz 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-proxy-994kg 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-scheduler-airship 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-scheduler-anchor-gsqtj 1/1 Running 0 3d 172.16.231.60 airship kube-system nfs-provisioner-7799d64d59-g2g6n 1/1 Running 0 3d 10.97.196.134 airship kube-system tiller-deploy-7d88c6f956-9jjk7 1/1 Running 0 3d 10.97.196.153 airship ucp airship-ucp-keystone-memcached-memcached-74d79d8896-dd8k2 1/1 Running 0 3d 10.97.196.144 airship ucp airship-ucp-rabbitmq-rabbitmq-0 1/1 Running 2 3d 10.97.196.178 airship ucp ingress-6cd5b89d5d-xx592 1/1 Running 0 3d 10.97.196.135 airship ucp ingress-error-pages-5c97bb46bb-ljmw6 1/1 Running 0 3d 10.97.196.136 airship ucp keystone-api-7657986b8c-b49r6 1/1 Running 0 3d 10.97.196.145 airship ucp keystone-bootstrap-8494p 0/1 Completed 0 3d 10.97.196.147 airship ucp keystone-credential-setup-r5twm 0/1 Completed 0 3d 10.97.196.146 airship ucp keystone-db-init-pxlzr 0/1 Completed 0 3d 10.97.196.148 airship ucp keystone-db-sync-zx665 0/1 Completed 0 3d 10.97.196.151 airship ucp keystone-domain-manage-rbfl5 0/1 Completed 0 3d 10.97.196.149 airship ucp keystone-fernet-rotate-1544356800-kcgg9 0/1 Completed 0 1d 10.97.196.148 airship ucp keystone-fernet-rotate-1544400000-78qc8 0/1 Completed 0 14h 10.97.196.163 airship ucp keystone-fernet-rotate-1544443200-v8bkn 0/1 Completed 0 2h 10.97.196.159 airship ucp keystone-fernet-setup-srrqj 0/1 Completed 0 3d 10.97.196.152 airship ucp keystone-rabbit-init-x4stm 0/1 Completed 0 3d 10.97.196.150 airship ucp maas-bootstrap-admin-user-26lhf 0/1 Init:0/1 0 3d 10.97.196.132 airship ucp maas-db-init-v7z6r 0/1 CrashLoopBackOff 944 3d 10.97.196.143 airship ucp maas-db-sync-5c24z 0/1 Init:0/1 0 3d 10.97.196.162 airship ucp maas-export-api-key-zhfxp 0/1 Init:0/1 0 3d 10.97.196.147 airship ucp maas-import-resources-j6tzd 0/1 Init:0/1 0 3d 10.97.196.152 airship ucp maas-rack-0 0/1 Init:0/1 0 3d 172.16.231.60 airship ucp maas-region-0 0/1 Init:0/1 0 3d 10.97.196.146 airship ucp mariadb-ingress-6c4f9c76f-sg8cx 1/1 Running 0 3d 10.97.196.137 airship ucp mariadb-ingress-6c4f9c76f-sz26m 1/1 Running 0 3d 10.97.196.138 airship ucp mariadb-ingress-error-pages-5dd6fbb75-x2xbn 1/1 Running 0 3d 10.97.196.139 airship ucp mariadb-server-0 1/1 Running 0 3d 10.97.196.142 airship ucp postgresql-0 1/1 Running 1 3d 10.97.196.140 airship The only output from that pod is this: root at airship:~# kubectl logs -n ucp maas-db-init-v7z6r + export HOME=/tmp + HOME=/tmp + [[ ! -v DB_HOST ]] + [[ ! -v ROOT_DB_USER ]] + [[ ! -v PGPASSWORD ]] + [[ ! -v USER_DB_USER ]] + [[ ! -v USER_DB_PASS ]] + [[ ! -v USER_DB_NAME ]] + echo 'Got DB connection info' Got DB connection info + pgsql_superuser_cmd 'SELECT 1 FROM pg_database WHERE datname = '\''maasdb'\''' + DB_COMMAND='SELECT 1 FROM pg_database WHERE datname = '\''maasdb'\''' + [[ ! -z '' ]] + grep -q 1 + psql -h postgresql.ucp.svc.cluster.local -p 5432 -U postgres '--command=SELECT 1 FROM pg_database WHERE datname = '\''maasdb'\''' psql: FATAL: no pg_hba.conf entry for host "10.97.196.143", user "postgres", database "postgres", SSL off + pgsql_superuser_cmd 'CREATE DATABASE maasdb' + DB_COMMAND='CREATE DATABASE maasdb' + [[ ! -z '' ]] + psql -h postgresql.ucp.svc.cluster.local -p 5432 -U postgres '--command=CREATE DATABASE maasdb' psql: FATAL: no pg_hba.conf entry for host "10.97.196.143", user "postgres", database "postgres", SSL off Unfortunately, I haven’t been able to troubleshoot this any further. Best, --Alex. Alex Arnoldy Software Defined Infrastructure Solutions Architect SUSE Alex.Arnoldy at SUSE.com (P) +1 (408) 828-9797 From: Balsiger Daniel, INI-EAI-INO-MBL Sent: Monday, December 10, 2018 6:32 AM To: airship-discuss at lists.airshipit.org Subject: [Airship-discuss] Airship in a bottle install failure... Hi there Airshippers I wanted to try installing airhip-in-a-bottle in a fresh Ubuntu16.04 VM running on OpenStack over the weekend. I followed the instructions on airshipit.org --snip sudo -i mkdir -p /root/deploy && cd "$_" git clone https://github.com/openstack/airship-in-a-bottle cd /root/deploy/airship-in-a-bottle/manifests/dev_single_node ./airship-in-a-bottle.sh --snip The script is still running after 2 days so I think something went wrong. The output is repeating itself, listing running/restarted containers (I think postgres is the one not coming up properly) --snip NAMESPACE NAME READY STATUS RESTARTS AGE kube-system auxiliary-etcd-airship 3/3 Running 0 2d kube-system bootstrap-armada-airship 4/4 Running 0 2d kube-system calico-etcd-airship 1/1 Running 0 2d kube-system calico-etcd-anchor-2jrqf 1/1 Running 0 2d kube-system calico-kube-controllers-6d5997c8c4-zxftw 1/1 Running 0 2d kube-system calico-node-l885v 2/2 Running 0 2d kube-system calico-settings-kbq79 0/1 Completed 0 2d kube-system coredns-d4587b7f8-7wlpm 1/1 Running 0 2d kube-system coredns-d4587b7f8-98t62 1/1 Running 0 2d kube-system coredns-d4587b7f8-frx5w 1/1 Running 0 2d kube-system haproxy-airship 1/1 Running 0 2d kube-system ingress-8w29l 1/1 Running 0 2d kube-system ingress-error-pages-5ccf96bf7d-jsrww 1/1 Running 0 2d kube-system kubernetes-apiserver-airship 1/1 Running 0 2d kube-system kubernetes-apiserver-anchor-55x59 1/1 Running 0 2d kube-system kubernetes-controller-manager-airship 1/1 Running 1 2d kube-system kubernetes-controller-manager-anchor-2wsg8 1/1 Running 0 2d kube-system kubernetes-etcd-airship 1/1 Running 0 2d kube-system kubernetes-etcd-anchor-64wgx 1/1 Running 0 2d kube-system kubernetes-proxy-9s68d 1/1 Running 0 2d kube-system kubernetes-scheduler-airship 1/1 Running 1 2d kube-system kubernetes-scheduler-anchor-npmn7 1/1 Running 0 2d kube-system nfs-provisioner-7799d64d59-scgdc 1/1 Running 0 2d ucp airship-ucp-rabbitmq-rabbitmq-0 1/1 Running 0 2d ucp ingress-6cd5b89d5d-gp98m 1/1 Running 0 2d ucp ingress-error-pages-5c97bb46bb-zq5cw 1/1 Running 0 2d ucp mariadb-ingress-6c4f9c76f-bjd99 0/1 Running 0 2d ucp mariadb-ingress-6c4f9c76f-xjrn9 0/1 Running 0 2d ucp mariadb-ingress-error-pages-5dd6fbb75-89qmx 1/1 Running 0 2d ucp mariadb-server-0 0/1 Running 0 2d ucp postgresql-0 1/1 Running 1 2d --snip Anybody out there with the same problem or (even better) a solution for it? Should I pick a particular branch or commit? Any help is appreciated. Best Regards, Daniel -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.arnoldy at suse.com Mon Dec 10 17:16:39 2018 From: alex.arnoldy at suse.com (Alex Arnoldy) Date: Mon, 10 Dec 2018 17:16:39 +0000 Subject: [Airship-discuss] Airship in a bottle install failure... In-Reply-To: References: , <545AF708D41B364BA4146E47647D2D2AE3A29F@prvxmb04.microfocus.com> Message-ID: <545AF708D41B364BA4146E47647D2D2AE3A366@prvxmb04.microfocus.com> Thanks for the reply Daniel. I won't have a chance to troubleshoot it for at least two more days. When I get a chance, I'll try to nsenter the namespaces occupied by that pod and have a look around. Best, --Alex. Alex Arnoldy Software Defined Infrastructure Solutions Architect SUSE Alex.Arnoldy at SUSE.com (P) +1 (408) 828-9797 From: Balsiger Daniel, INI-EAI-INO-MBL Sent: Monday, December 10, 2018 7:59 AM To: airship-discuss at lists.airshipit.org; Alex Arnoldy Subject: Re: Airship in a bottle install failure... Hi there Alex. Thanks for the reply. >From your logs it seems to me like maas-db-init is not able to connect to postgres DB with psql client to the other container. (clients should be listed by IP in pg_hba.conf, wildcards should be possible....) I will try another install from a fresh VM and let you know if I have the same one. Thanks ________________________________ From: Alex Arnoldy > Sent: Monday, December 10, 2018 4:26 PM To: Balsiger Daniel, INI-EAI-INO-MBL; airship-discuss at lists.airshipit.org Subject: RE: Airship in a bottle install failure... Hello Daniel, I had launched the same thing but then got pulled away from it and totally forgot to check it. I thought I'd be able to tell you I got it running, but I don't think that's the case. For my installation, the maas-db-init pod seems to be stuck in CrashLoopBackOff: root at airship:~# kubectl get pods -o wide --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE kube-system auxiliary-etcd-airship 3/3 Running 0 3d 172.16.231.60 airship kube-system bootstrap-armada-airship 4/4 Running 0 3d 172.16.231.60 airship kube-system calico-etcd-airship 1/1 Running 0 3d 172.16.231.60 airship kube-system calico-etcd-anchor-nqntm 1/1 Running 0 3d 172.16.231.60 airship kube-system calico-kube-controllers-6d5997c8c4-bm9rk 1/1 Running 0 3d 172.16.231.60 airship kube-system calico-node-2sgkk 2/2 Running 0 3d 172.16.231.60 airship kube-system calico-settings-bqs5v 0/1 Completed 0 3d 172.16.231.60 airship kube-system coredns-d4587b7f8-bbpr6 1/1 Running 13 3d 10.97.196.130 airship kube-system coredns-d4587b7f8-tdl8v 1/1 Running 11 3d 10.97.196.131 airship kube-system coredns-d4587b7f8-ts9lr 1/1 Running 12 3d 10.97.196.129 airship kube-system haproxy-airship 1/1 Running 0 3d 172.16.231.60 airship kube-system ingress-error-pages-5ccf96bf7d-8q6g5 1/1 Running 0 3d 10.97.196.133 airship kube-system ingress-fmrmk 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-apiserver-airship 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-apiserver-anchor-rkfpz 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-controller-manager-airship 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-controller-manager-anchor-75k62 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-etcd-airship 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-etcd-anchor-hzcrz 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-proxy-994kg 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-scheduler-airship 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-scheduler-anchor-gsqtj 1/1 Running 0 3d 172.16.231.60 airship kube-system nfs-provisioner-7799d64d59-g2g6n 1/1 Running 0 3d 10.97.196.134 airship kube-system tiller-deploy-7d88c6f956-9jjk7 1/1 Running 0 3d 10.97.196.153 airship ucp airship-ucp-keystone-memcached-memcached-74d79d8896-dd8k2 1/1 Running 0 3d 10.97.196.144 airship ucp airship-ucp-rabbitmq-rabbitmq-0 1/1 Running 2 3d 10.97.196.178 airship ucp ingress-6cd5b89d5d-xx592 1/1 Running 0 3d 10.97.196.135 airship ucp ingress-error-pages-5c97bb46bb-ljmw6 1/1 Running 0 3d 10.97.196.136 airship ucp keystone-api-7657986b8c-b49r6 1/1 Running 0 3d 10.97.196.145 airship ucp keystone-bootstrap-8494p 0/1 Completed 0 3d 10.97.196.147 airship ucp keystone-credential-setup-r5twm 0/1 Completed 0 3d 10.97.196.146 airship ucp keystone-db-init-pxlzr 0/1 Completed 0 3d 10.97.196.148 airship ucp keystone-db-sync-zx665 0/1 Completed 0 3d 10.97.196.151 airship ucp keystone-domain-manage-rbfl5 0/1 Completed 0 3d 10.97.196.149 airship ucp keystone-fernet-rotate-1544356800-kcgg9 0/1 Completed 0 1d 10.97.196.148 airship ucp keystone-fernet-rotate-1544400000-78qc8 0/1 Completed 0 14h 10.97.196.163 airship ucp keystone-fernet-rotate-1544443200-v8bkn 0/1 Completed 0 2h 10.97.196.159 airship ucp keystone-fernet-setup-srrqj 0/1 Completed 0 3d 10.97.196.152 airship ucp keystone-rabbit-init-x4stm 0/1 Completed 0 3d 10.97.196.150 airship ucp maas-bootstrap-admin-user-26lhf 0/1 Init:0/1 0 3d 10.97.196.132 airship ucp maas-db-init-v7z6r 0/1 CrashLoopBackOff 944 3d 10.97.196.143 airship ucp maas-db-sync-5c24z 0/1 Init:0/1 0 3d 10.97.196.162 airship ucp maas-export-api-key-zhfxp 0/1 Init:0/1 0 3d 10.97.196.147 airship ucp maas-import-resources-j6tzd 0/1 Init:0/1 0 3d 10.97.196.152 airship ucp maas-rack-0 0/1 Init:0/1 0 3d 172.16.231.60 airship ucp maas-region-0 0/1 Init:0/1 0 3d 10.97.196.146 airship ucp mariadb-ingress-6c4f9c76f-sg8cx 1/1 Running 0 3d 10.97.196.137 airship ucp mariadb-ingress-6c4f9c76f-sz26m 1/1 Running 0 3d 10.97.196.138 airship ucp mariadb-ingress-error-pages-5dd6fbb75-x2xbn 1/1 Running 0 3d 10.97.196.139 airship ucp mariadb-server-0 1/1 Running 0 3d 10.97.196.142 airship ucp postgresql-0 1/1 Running 1 3d 10.97.196.140 airship The only output from that pod is this: root at airship:~# kubectl logs -n ucp maas-db-init-v7z6r + export HOME=/tmp + HOME=/tmp + [[ ! -v DB_HOST ]] + [[ ! -v ROOT_DB_USER ]] + [[ ! -v PGPASSWORD ]] + [[ ! -v USER_DB_USER ]] + [[ ! -v USER_DB_PASS ]] + [[ ! -v USER_DB_NAME ]] + echo 'Got DB connection info' Got DB connection info + pgsql_superuser_cmd 'SELECT 1 FROM pg_database WHERE datname = '\''maasdb'\''' + DB_COMMAND='SELECT 1 FROM pg_database WHERE datname = '\''maasdb'\''' + [[ ! -z '' ]] + grep -q 1 + psql -h postgresql.ucp.svc.cluster.local -p 5432 -U postgres '--command=SELECT 1 FROM pg_database WHERE datname = '\''maasdb'\''' psql: FATAL: no pg_hba.conf entry for host "10.97.196.143", user "postgres", database "postgres", SSL off + pgsql_superuser_cmd 'CREATE DATABASE maasdb' + DB_COMMAND='CREATE DATABASE maasdb' + [[ ! -z '' ]] + psql -h postgresql.ucp.svc.cluster.local -p 5432 -U postgres '--command=CREATE DATABASE maasdb' psql: FATAL: no pg_hba.conf entry for host "10.97.196.143", user "postgres", database "postgres", SSL off Unfortunately, I haven't been able to troubleshoot this any further. Best, --Alex. Alex Arnoldy Software Defined Infrastructure Solutions Architect SUSE Alex.Arnoldy at SUSE.com (P) +1 (408) 828-9797 From: Balsiger Daniel, INI-EAI-INO-MBL > Sent: Monday, December 10, 2018 6:32 AM To: airship-discuss at lists.airshipit.org Subject: [Airship-discuss] Airship in a bottle install failure... Hi there Airshippers I wanted to try installing airhip-in-a-bottle in a fresh Ubuntu16.04 VM running on OpenStack over the weekend. I followed the instructions on airshipit.org --snip sudo -i mkdir -p /root/deploy && cd "$_" git clone https://github.com/openstack/airship-in-a-bottle cd /root/deploy/airship-in-a-bottle/manifests/dev_single_node ./airship-in-a-bottle.sh --snip The script is still running after 2 days so I think something went wrong. The output is repeating itself, listing running/restarted containers (I think postgres is the one not coming up properly) --snip NAMESPACE NAME READY STATUS RESTARTS AGE kube-system auxiliary-etcd-airship 3/3 Running 0 2d kube-system bootstrap-armada-airship 4/4 Running 0 2d kube-system calico-etcd-airship 1/1 Running 0 2d kube-system calico-etcd-anchor-2jrqf 1/1 Running 0 2d kube-system calico-kube-controllers-6d5997c8c4-zxftw 1/1 Running 0 2d kube-system calico-node-l885v 2/2 Running 0 2d kube-system calico-settings-kbq79 0/1 Completed 0 2d kube-system coredns-d4587b7f8-7wlpm 1/1 Running 0 2d kube-system coredns-d4587b7f8-98t62 1/1 Running 0 2d kube-system coredns-d4587b7f8-frx5w 1/1 Running 0 2d kube-system haproxy-airship 1/1 Running 0 2d kube-system ingress-8w29l 1/1 Running 0 2d kube-system ingress-error-pages-5ccf96bf7d-jsrww 1/1 Running 0 2d kube-system kubernetes-apiserver-airship 1/1 Running 0 2d kube-system kubernetes-apiserver-anchor-55x59 1/1 Running 0 2d kube-system kubernetes-controller-manager-airship 1/1 Running 1 2d kube-system kubernetes-controller-manager-anchor-2wsg8 1/1 Running 0 2d kube-system kubernetes-etcd-airship 1/1 Running 0 2d kube-system kubernetes-etcd-anchor-64wgx 1/1 Running 0 2d kube-system kubernetes-proxy-9s68d 1/1 Running 0 2d kube-system kubernetes-scheduler-airship 1/1 Running 1 2d kube-system kubernetes-scheduler-anchor-npmn7 1/1 Running 0 2d kube-system nfs-provisioner-7799d64d59-scgdc 1/1 Running 0 2d ucp airship-ucp-rabbitmq-rabbitmq-0 1/1 Running 0 2d ucp ingress-6cd5b89d5d-gp98m 1/1 Running 0 2d ucp ingress-error-pages-5c97bb46bb-zq5cw 1/1 Running 0 2d ucp mariadb-ingress-6c4f9c76f-bjd99 0/1 Running 0 2d ucp mariadb-ingress-6c4f9c76f-xjrn9 0/1 Running 0 2d ucp mariadb-ingress-error-pages-5dd6fbb75-89qmx 1/1 Running 0 2d ucp mariadb-server-0 0/1 Running 0 2d ucp postgresql-0 1/1 Running 1 2d --snip Anybody out there with the same problem or (even better) a solution for it? Should I pick a particular branch or commit? Any help is appreciated. Best Regards, Daniel -------------- next part -------------- An HTML attachment was scrubbed... URL: From MM9745 at att.com Mon Dec 10 17:32:45 2018 From: MM9745 at att.com (MCEUEN, MATT) Date: Mon, 10 Dec 2018 17:32:45 +0000 Subject: [Airship-discuss] Airship in a bottle install failure... In-Reply-To: References: , <545AF708D41B364BA4146E47647D2D2AE3A29F@prvxmb04.microfocus.com> Message-ID: <7C64A75C21BB8D43BD75BB18635E4D896CEAD525@MOSTLS1MSGUSRFF.ITServices.sbc.com> Weird, the maas-db-init job is running fine for me with a fresh deployment of airship in a bottle. Alex, are you seeing that "no pg_hba.conf" error current aiab code, or can you give git commit where you're hitting the issue? Daniel, can you please run "sudo kubectl -n ucp logs mariadb-server-0" and share the results? Looks like MariaDB isn't starting cleanly for some reason. Thanks, Matt From: Balsiger Daniel, INI-EAI-INO-MBL Sent: Monday, December 10, 2018 9:59 AM To: Alex Arnoldy ; airship-discuss at lists.airshipit.org Subject: Re: [Airship-discuss] Airship in a bottle install failure... Hi there Alex. Thanks for the reply. >From your logs it seems to me like maas-db-init is not able to connect to postgres DB with psql client to the other container. (clients should be listed by IP in pg_hba.conf, wildcards should be possible....) I will try another install from a fresh VM and let you know if I have the same one. Thanks ________________________________ From: Alex Arnoldy > Sent: Monday, December 10, 2018 4:26 PM To: Balsiger Daniel, INI-EAI-INO-MBL; airship-discuss at lists.airshipit.org Subject: RE: Airship in a bottle install failure... Hello Daniel, I had launched the same thing but then got pulled away from it and totally forgot to check it. I thought I'd be able to tell you I got it running, but I don't think that's the case. For my installation, the maas-db-init pod seems to be stuck in CrashLoopBackOff: root at airship:~# kubectl get pods -o wide --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE kube-system auxiliary-etcd-airship 3/3 Running 0 3d 172.16.231.60 airship kube-system bootstrap-armada-airship 4/4 Running 0 3d 172.16.231.60 airship kube-system calico-etcd-airship 1/1 Running 0 3d 172.16.231.60 airship kube-system calico-etcd-anchor-nqntm 1/1 Running 0 3d 172.16.231.60 airship kube-system calico-kube-controllers-6d5997c8c4-bm9rk 1/1 Running 0 3d 172.16.231.60 airship kube-system calico-node-2sgkk 2/2 Running 0 3d 172.16.231.60 airship kube-system calico-settings-bqs5v 0/1 Completed 0 3d 172.16.231.60 airship kube-system coredns-d4587b7f8-bbpr6 1/1 Running 13 3d 10.97.196.130 airship kube-system coredns-d4587b7f8-tdl8v 1/1 Running 11 3d 10.97.196.131 airship kube-system coredns-d4587b7f8-ts9lr 1/1 Running 12 3d 10.97.196.129 airship kube-system haproxy-airship 1/1 Running 0 3d 172.16.231.60 airship kube-system ingress-error-pages-5ccf96bf7d-8q6g5 1/1 Running 0 3d 10.97.196.133 airship kube-system ingress-fmrmk 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-apiserver-airship 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-apiserver-anchor-rkfpz 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-controller-manager-airship 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-controller-manager-anchor-75k62 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-etcd-airship 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-etcd-anchor-hzcrz 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-proxy-994kg 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-scheduler-airship 1/1 Running 0 3d 172.16.231.60 airship kube-system kubernetes-scheduler-anchor-gsqtj 1/1 Running 0 3d 172.16.231.60 airship kube-system nfs-provisioner-7799d64d59-g2g6n 1/1 Running 0 3d 10.97.196.134 airship kube-system tiller-deploy-7d88c6f956-9jjk7 1/1 Running 0 3d 10.97.196.153 airship ucp airship-ucp-keystone-memcached-memcached-74d79d8896-dd8k2 1/1 Running 0 3d 10.97.196.144 airship ucp airship-ucp-rabbitmq-rabbitmq-0 1/1 Running 2 3d 10.97.196.178 airship ucp ingress-6cd5b89d5d-xx592 1/1 Running 0 3d 10.97.196.135 airship ucp ingress-error-pages-5c97bb46bb-ljmw6 1/1 Running 0 3d 10.97.196.136 airship ucp keystone-api-7657986b8c-b49r6 1/1 Running 0 3d 10.97.196.145 airship ucp keystone-bootstrap-8494p 0/1 Completed 0 3d 10.97.196.147 airship ucp keystone-credential-setup-r5twm 0/1 Completed 0 3d 10.97.196.146 airship ucp keystone-db-init-pxlzr 0/1 Completed 0 3d 10.97.196.148 airship ucp keystone-db-sync-zx665 0/1 Completed 0 3d 10.97.196.151 airship ucp keystone-domain-manage-rbfl5 0/1 Completed 0 3d 10.97.196.149 airship ucp keystone-fernet-rotate-1544356800-kcgg9 0/1 Completed 0 1d 10.97.196.148 airship ucp keystone-fernet-rotate-1544400000-78qc8 0/1 Completed 0 14h 10.97.196.163 airship ucp keystone-fernet-rotate-1544443200-v8bkn 0/1 Completed 0 2h 10.97.196.159 airship ucp keystone-fernet-setup-srrqj 0/1 Completed 0 3d 10.97.196.152 airship ucp keystone-rabbit-init-x4stm 0/1 Completed 0 3d 10.97.196.150 airship ucp maas-bootstrap-admin-user-26lhf 0/1 Init:0/1 0 3d 10.97.196.132 airship ucp maas-db-init-v7z6r 0/1 CrashLoopBackOff 944 3d 10.97.196.143 airship ucp maas-db-sync-5c24z 0/1 Init:0/1 0 3d 10.97.196.162 airship ucp maas-export-api-key-zhfxp 0/1 Init:0/1 0 3d 10.97.196.147 airship ucp maas-import-resources-j6tzd 0/1 Init:0/1 0 3d 10.97.196.152 airship ucp maas-rack-0 0/1 Init:0/1 0 3d 172.16.231.60 airship ucp maas-region-0 0/1 Init:0/1 0 3d 10.97.196.146 airship ucp mariadb-ingress-6c4f9c76f-sg8cx 1/1 Running 0 3d 10.97.196.137 airship ucp mariadb-ingress-6c4f9c76f-sz26m 1/1 Running 0 3d 10.97.196.138 airship ucp mariadb-ingress-error-pages-5dd6fbb75-x2xbn 1/1 Running 0 3d 10.97.196.139 airship ucp mariadb-server-0 1/1 Running 0 3d 10.97.196.142 airship ucp postgresql-0 1/1 Running 1 3d 10.97.196.140 airship The only output from that pod is this: root at airship:~# kubectl logs -n ucp maas-db-init-v7z6r + export HOME=/tmp + HOME=/tmp + [[ ! -v DB_HOST ]] + [[ ! -v ROOT_DB_USER ]] + [[ ! -v PGPASSWORD ]] + [[ ! -v USER_DB_USER ]] + [[ ! -v USER_DB_PASS ]] + [[ ! -v USER_DB_NAME ]] + echo 'Got DB connection info' Got DB connection info + pgsql_superuser_cmd 'SELECT 1 FROM pg_database WHERE datname = '\''maasdb'\''' + DB_COMMAND='SELECT 1 FROM pg_database WHERE datname = '\''maasdb'\''' + [[ ! -z '' ]] + grep -q 1 + psql -h postgresql.ucp.svc.cluster.local -p 5432 -U postgres '--command=SELECT 1 FROM pg_database WHERE datname = '\''maasdb'\''' psql: FATAL: no pg_hba.conf entry for host "10.97.196.143", user "postgres", database "postgres", SSL off + pgsql_superuser_cmd 'CREATE DATABASE maasdb' + DB_COMMAND='CREATE DATABASE maasdb' + [[ ! -z '' ]] + psql -h postgresql.ucp.svc.cluster.local -p 5432 -U postgres '--command=CREATE DATABASE maasdb' psql: FATAL: no pg_hba.conf entry for host "10.97.196.143", user "postgres", database "postgres", SSL off Unfortunately, I haven't been able to troubleshoot this any further. Best, --Alex. Alex Arnoldy Software Defined Infrastructure Solutions Architect SUSE Alex.Arnoldy at SUSE.com (P) +1 (408) 828-9797 From: Balsiger Daniel, INI-EAI-INO-MBL > Sent: Monday, December 10, 2018 6:32 AM To: airship-discuss at lists.airshipit.org Subject: [Airship-discuss] Airship in a bottle install failure... Hi there Airshippers I wanted to try installing airhip-in-a-bottle in a fresh Ubuntu16.04 VM running on OpenStack over the weekend. I followed the instructions on airshipit.org --snip sudo -i mkdir -p /root/deploy && cd "$_" git clone https://github.com/openstack/airship-in-a-bottle cd /root/deploy/airship-in-a-bottle/manifests/dev_single_node ./airship-in-a-bottle.sh --snip The script is still running after 2 days so I think something went wrong. The output is repeating itself, listing running/restarted containers (I think postgres is the one not coming up properly) --snip NAMESPACE NAME READY STATUS RESTARTS AGE kube-system auxiliary-etcd-airship 3/3 Running 0 2d kube-system bootstrap-armada-airship 4/4 Running 0 2d kube-system calico-etcd-airship 1/1 Running 0 2d kube-system calico-etcd-anchor-2jrqf 1/1 Running 0 2d kube-system calico-kube-controllers-6d5997c8c4-zxftw 1/1 Running 0 2d kube-system calico-node-l885v 2/2 Running 0 2d kube-system calico-settings-kbq79 0/1 Completed 0 2d kube-system coredns-d4587b7f8-7wlpm 1/1 Running 0 2d kube-system coredns-d4587b7f8-98t62 1/1 Running 0 2d kube-system coredns-d4587b7f8-frx5w 1/1 Running 0 2d kube-system haproxy-airship 1/1 Running 0 2d kube-system ingress-8w29l 1/1 Running 0 2d kube-system ingress-error-pages-5ccf96bf7d-jsrww 1/1 Running 0 2d kube-system kubernetes-apiserver-airship 1/1 Running 0 2d kube-system kubernetes-apiserver-anchor-55x59 1/1 Running 0 2d kube-system kubernetes-controller-manager-airship 1/1 Running 1 2d kube-system kubernetes-controller-manager-anchor-2wsg8 1/1 Running 0 2d kube-system kubernetes-etcd-airship 1/1 Running 0 2d kube-system kubernetes-etcd-anchor-64wgx 1/1 Running 0 2d kube-system kubernetes-proxy-9s68d 1/1 Running 0 2d kube-system kubernetes-scheduler-airship 1/1 Running 1 2d kube-system kubernetes-scheduler-anchor-npmn7 1/1 Running 0 2d kube-system nfs-provisioner-7799d64d59-scgdc 1/1 Running 0 2d ucp airship-ucp-rabbitmq-rabbitmq-0 1/1 Running 0 2d ucp ingress-6cd5b89d5d-gp98m 1/1 Running 0 2d ucp ingress-error-pages-5c97bb46bb-zq5cw 1/1 Running 0 2d ucp mariadb-ingress-6c4f9c76f-bjd99 0/1 Running 0 2d ucp mariadb-ingress-6c4f9c76f-xjrn9 0/1 Running 0 2d ucp mariadb-ingress-error-pages-5dd6fbb75-89qmx 1/1 Running 0 2d ucp mariadb-server-0 0/1 Running 0 2d ucp postgresql-0 1/1 Running 1 2d --snip Anybody out there with the same problem or (even better) a solution for it? Should I pick a particular branch or commit? Any help is appreciated. Best Regards, Daniel -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.arnoldy at suse.com Mon Dec 10 18:48:25 2018 From: alex.arnoldy at suse.com (Alex Arnoldy) Date: Mon, 10 Dec 2018 18:48:25 +0000 Subject: [Airship-discuss] Airship in a bottle install failure... In-Reply-To: <7C64A75C21BB8D43BD75BB18635E4D896CEAD525@MOSTLS1MSGUSRFF.ITServices.sbc.com> References: , <545AF708D41B364BA4146E47647D2D2AE3A29F@prvxmb04.microfocus.com> <7C64A75C21BB8D43BD75BB18635E4D896CEAD525@MOSTLS1MSGUSRFF.ITServices.sbc.com> Message-ID: <545AF708D41B364BA4146E47647D2D2AE3A439@prvxmb04.microfocus.com> Hi Matt, Here is the commit I'm using: commit 221441f1bff351ab366b02d6fd7856b70b06108f Author: Bryan Strassner Date: Mon Nov 19 17:12:10 2018 -0600 Update apiserver for admission controller Update the apiserver's command prefix to match between the apiserver chart and the genesis api server. Includes an optimization of virsh commands to parallelize and wait for cloud init to complete (requires Ubuntu 16.04.4) Change-Id: I793d210e15c3c13e8b662bd2459ed8bcba30d434 I'm happy to start over to see if a new pull resolves the issue, but it happened to me twice in a row so I thought I'd preserve the environment for examination. Best, --Alex. Alex Arnoldy Software Defined Infrastructure Solutions Architect SUSE Alex.Arnoldy at SUSE.com (P) +1 (408) 828-9797 From: MCEUEN, MATT Sent: Monday, December 10, 2018 9:33 AM To: airship-discuss at lists.airshipit.org; Alex Arnoldy ; INI-EAI-INO-MBL Balsiger Daniel Subject: RE: Airship in a bottle install failure... Weird, the maas-db-init job is running fine for me with a fresh deployment of airship in a bottle. Alex, are you seeing that "no pg_hba.conf" error current aiab code, or can you give git commit where you're hitting the issue? Daniel, can you please run "sudo kubectl -n ucp logs mariadb-server-0" and share the results? Looks like MariaDB isn't starting cleanly for some reason. Thanks, Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From Daniel.Balsiger at swisscom.com Tue Dec 11 15:28:57 2018 From: Daniel.Balsiger at swisscom.com (Balsiger Daniel, INI-EAI-INO-MBL) Date: Tue, 11 Dec 2018 15:28:57 +0000 Subject: [Airship-discuss] Airship in a bottle install failure... In-Reply-To: <545AF708D41B364BA4146E47647D2D2AE3A439@prvxmb04.microfocus.com> References: , <545AF708D41B364BA4146E47647D2D2AE3A29F@prvxmb04.microfocus.com> <7C64A75C21BB8D43BD75BB18635E4D896CEAD525@MOSTLS1MSGUSRFF.ITServices.sbc.com>, <545AF708D41B364BA4146E47647D2D2AE3A439@prvxmb04.microfocus.com> Message-ID: Hi there everyone Thanks for the quick replies, I tried a new install on a fresh VM, behaves the same, problem is persisting: I see in Postgresql-0 logs: --snip FATAL: the database system is in recovery mode LOG: database system was not properly shut down; automatic recovery in progress LOG: invalid record length at 0/102A400 LOG: redo is not required LOG: MultiXact member wraparound protections are now enabled LOG: database system is ready to accept connections LOG: autovacuum launcher started FATAL: could not open relation mapping file "global/pg_filenode.map": No such file or directory LOG: autovacuum launcher process (PID 79) exited with exit code 1 LOG: terminating any other active server processes LOG: all server processes terminated; reinitializing LOG: database system was interrupted; last known up at 2018-12-11 15:25:38 UTC LOG: database system was not properly shut down; automatic recovery in progress --snip and in mariadb-server-0 logs: --snip ubuntu at airship:~$ sudo kubectl -n ucp logs mariadb-server-0 2018-12-11 15:25:35,704 - OpenStack-Helm Mariadb - INFO - This instance hostname: mariadb-server-0 2018-12-11 15:25:36,310 - OpenStack-Helm Mariadb - INFO - Kubernetes API Version: v1.10.2 2018-12-11 15:25:36,313 - OpenStack-Helm Mariadb - INFO - Will use "airship-ucp-mariadb-mariadb-state" configmap for cluster state info 2018-12-11 15:25:36,313 - OpenStack-Helm Mariadb - INFO - Getting cluster state 2018-12-11 15:25:36,348 - OpenStack-Helm Mariadb - INFO - The cluster is currently in "init" state. 2018-12-11 15:25:36,348 - OpenStack-Helm Mariadb - INFO - Getting cluster state 2018-12-11 15:25:36,455 - OpenStack-Helm Mariadb - INFO - The cluster is currently in "init" state. 2018-12-11 15:25:36,456 - OpenStack-Helm Mariadb - INFO - Waiting for cluster to start running 2018-12-11 15:25:36,456 - OpenStack-Helm Mariadb - INFO - Getting cluster state 2018-12-11 15:25:36,474 - OpenStack-Helm Mariadb - INFO - The cluster is currently in "init" state. 2018-12-11 15:25:56,491 - OpenStack-Helm Mariadb - INFO - Getting cluster state 2018-12-11 15:25:56,637 - OpenStack-Helm Mariadb - INFO - The cluster is currently in "init" state. 2018-12-11 15:26:16,657 - OpenStack-Helm Mariadb - INFO - Getting cluster state 2018-12-11 15:26:16,678 - OpenStack-Helm Mariadb - INFO - The cluster is currently in "init" state. 2018-12-11 15:26:36,684 - OpenStack-Helm Mariadb - INFO - Getting cluster state 2018-12-11 15:26:36,697 - OpenStack-Helm Mariadb - INFO - The cluster is currently in "init" state. 2018-12-11 15:26:56,717 - OpenStack-Helm Mariadb - INFO - Getting cluster state 2018-12-11 15:26:56,749 - OpenStack-Helm Mariadb - INFO - The cluster is currently in "init" state. 2018-12-11 15:27:16,761 - OpenStack-Helm Mariadb - INFO - Getting cluster state 2018-12-11 15:27:16,876 - OpenStack-Helm Mariadb - INFO - The cluster is currently in "init" state. --snip Best Regards, D ________________________________ From: Alex Arnoldy Sent: Monday, December 10, 2018 7:48 PM To: MCEUEN, MATT ; airship-discuss at lists.airshipit.org; Balsiger Daniel, INI-EAI-INO-MBL Subject: RE: Airship in a bottle install failure... Hi Matt, Here is the commit I’m using: commit 221441f1bff351ab366b02d6fd7856b70b06108f Author: Bryan Strassner Date: Mon Nov 19 17:12:10 2018 -0600 Update apiserver for admission controller Update the apiserver's command prefix to match between the apiserver chart and the genesis api server. Includes an optimization of virsh commands to parallelize and wait for cloud init to complete (requires Ubuntu 16.04.4) Change-Id: I793d210e15c3c13e8b662bd2459ed8bcba30d434 I’m happy to start over to see if a new pull resolves the issue, but it happened to me twice in a row so I thought I’d preserve the environment for examination. Best, --Alex. Alex Arnoldy Software Defined Infrastructure Solutions Architect SUSE Alex.Arnoldy at SUSE.com (P) +1 (408) 828-9797 From: MCEUEN, MATT Sent: Monday, December 10, 2018 9:33 AM To: airship-discuss at lists.airshipit.org; Alex Arnoldy ; INI-EAI-INO-MBL Balsiger Daniel Subject: RE: Airship in a bottle install failure... Weird, the maas-db-init job is running fine for me with a fresh deployment of airship in a bottle. Alex, are you seeing that “no pg_hba.conf” error current aiab code, or can you give git commit where you’re hitting the issue? Daniel, can you please run “sudo kubectl -n ucp logs mariadb-server-0” and share the results? Looks like MariaDB isn’t starting cleanly for some reason. Thanks, Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From eli at mirantis.com Wed Dec 12 21:54:54 2018 From: eli at mirantis.com (Evgeny L) Date: Wed, 12 Dec 2018 13:54:54 -0800 Subject: [Airship-discuss] Armada API connectivity to Tiller (proxy problem) Message-ID: Hi, I've been working on getting behind the proxy configuration for AIAB to work [1] and I stumbled across a problem related to GRPC [2], which does not support CIDR notation for no_proxy entries, this make Armada API to fail to connect to Tiller, because it tries to use proxy configured with environment variables. While looking for a solution other than doing some manual reconfiguration [2], I started to have the following questions: 1. Why does Shipyard uses [4] Tiller Pod IP? Can we create a ClusterIP and use a DNS name? 2. Why Shipyard is the one who provides Armada API with Tiller IP address? If we do 1st we can possibly make endpoint a part of Armada API configuration. Or is Armada API supposed to work with multiple Tillers at the same time? Any feedback and historical background would be really helpful! Thanks, [1] https://review.openstack.org/#/c/624122/ [2] https://review.openstack.org/#/c/624504/2/manifests/dev_minimal/README.rst [3] https://github.com/grpc/grpc/blob/9e9cae7839a362936228cf333045e5da877ace40/src/core/ext/filters/client_channel/http_proxy.cc#L146-L149 [4] https://github.com/openstack/airship-shipyard/blob/a235f8e6db6e791550727e3575e045e26f177252/src/bin/shipyard_airflow/shipyard_airflow/plugins/armada_base_operator.py#L147-L153 -------------- next part -------------- An HTML attachment was scrubbed... URL: From santosh.thapamager at as.ntt-at.co.jp Thu Dec 13 04:03:03 2018 From: santosh.thapamager at as.ntt-at.co.jp (santosh.thapamager) Date: Thu, 13 Dec 2018 13:03:03 +0900 Subject: [Airship-discuss] =?iso-2022-jp?b?QWlyc2hpcC1pbi1hLWJvdHRsZV9E?= =?iso-2022-jp?b?ZXZfTWluaW1hbF9CZWRoaW5kIFByb3h5GyRCIScbKEJIb3cg?= =?iso-2022-jp?b?dG8gVXBkYXRlIFNpdGU/?= Message-ID: <008001d49298$bed0db70$3c729250$@as.ntt-at.co.jp> Hi all, I was able to install dev_minimal and deploy site behind proxy. Because of Kubernetes apiserver vulnerability (CVE-2018-1002105)I wanted to update Kubernetes version in my currently deployed site. However I could not find any documentations regarding updating deployed site. I would be thankful to all of you, if I could get any support. Best Regards, Santosh Thapa Magar From rp2723 at att.com Thu Dec 13 14:17:33 2018 From: rp2723 at att.com (PACHECO, RODOLFO J) Date: Thu, 13 Dec 2018 14:17:33 +0000 Subject: [Airship-discuss] Canceled: Airship - Open Design Call - PM EST Call Message-ID: <99088997CCAD0C4BA20008FD094344963267F11C@MISOUT7MSGUSRDI.ITServices.sbc.com> I am cancelling the call today. Most of Airship members that attend this call are out . Will resume this calls on January 10. 2019 Happy Holidays to all the Airship Community · Etherpad for the Airship Open Design discussion · https://etherpad.openstack.org/p/Airship_OpenDesignDiscussions · Storyboard in flight Specs · https://storyboard.openstack.org/#!/project/openstack/airship-specs · Github Airship Specs · https://github.com/openstack/airship-specs/tree/master/specs · Inflight/reviewing specs · https://review.openstack.org/#/q/status:open+airship-specs __________________________________________ To join the Meeting: https://bluejeans.com/421706689 To join via Room System: Video Conferencing System: bjn.vc -or-199.48.152.152 Meeting ID : 421706689 To join via phone : 1) Dial: +1.408.317.9254 (BlueJeans U.S. Toll) +1.866.226.4650 (US Toll Free) (see all numbers - http://bluejeans.com/numbers) 2) Enter Conference ID : 421706689 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 2671 bytes Desc: not available URL: From rp2723 at att.com Thu Dec 13 16:22:47 2018 From: rp2723 at att.com (PACHECO, RODOLFO J) Date: Thu, 13 Dec 2018 16:22:47 +0000 Subject: [Airship-discuss] Airship - Open Design Call - AM EST Call Message-ID: <99088997CCAD0C4BA20008FD0943449632680B5B@MISOUT7MSGUSRDI.ITServices.sbc.com> When: Occurs every Thursday from 11:00 AM to 12:00 PM effective 8/30/2018 until 2/28/2019. (UTC-05:00) Eastern Time (US & Canada) Where: https://bluejeans.com/421706689 *~*~*~*~*~*~*~*~*~* Changing recurrence to only be the 11 EST Time. Very low attendance in the EST PM meeting. We can change this when the community needs it. · Etherpad for the Airship Open Design discussion · https://etherpad.openstack.org/p/Airship_OpenDesignDiscussions · Storyboard in flight Specs · https://storyboard.openstack.org/#!/project/openstack/airship-specs · Github Airship Specs · https://github.com/openstack/airship-specs/tree/master/specs · Inflight/reviewing specs · https://review.openstack.org/#/q/status:open+airship-specs __________________________________________ To join the Meeting: https://bluejeans.com/421706689 To join via Room System: Video Conferencing System: bjn.vc -or-199.48.152.152 Meeting ID : 421706689 To join via phone : 1) Dial: +1.408.317.9254 (BlueJeans U.S. Toll) +1.866.226.4650 (US Toll Free) (see all numbers - http://bluejeans.com/numbers) 2) Enter Conference ID : 421706689 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 3165 bytes Desc: not available URL: From claire at openstack.org Thu Dec 13 22:26:42 2018 From: claire at openstack.org (Claire Massey) Date: Thu, 13 Dec 2018 16:26:42 -0600 Subject: [Airship-discuss] Creation of OSF Project Confirmation Guidelines Message-ID: <95128D2C-F25D-4CC1-9CB7-7673FECC3C06@openstack.org> Hi everyone, As you know, Airship is a pilot project supported by the OpenStack Foundation (OSF). As such, we wanted to make sure you’re aware of an effort recently kicked off a sub group of the OSF Board of Directors to draft guidelines[1] for confirming pilot projects as full, top-level, open infrastructure project of the Foundation. The current pilot projects that will be eligible for confirmation review in 2019-2020 are Airship, Kata Containers, StarlingX and Zuul. Currently the OpenStack project is the only confirmed project. The discussion about drafting these guidelines is just getting started, and *we invite you to participate in this effort* by reviewing the working draft in this etherpad and adding in your feedback and thoughts starting on line 139: https://etherpad.openstack.org/p/BrainstormingOSFProjectConfirmationGuidelines There will most likely be an open call scheduled in early 2019 where additional feedback will also be welcomed. I will share that invitation with you once it’s schedule. The goal is to come up with a set of points that the Board can use when reviewing pilot projects for confirmation. This topic has previously been discussed at a very high-level at previous Board meetings, including the September 18th meeting[2] (starting around slide 38). 1. https://etherpad.openstack.org/p/ProjectConfirmationGuidelines 2. https://docs.google.com/presentation/d/10UyCpxkjPqC3kT-dYRpBxzNT39i2OhlggJvzGDosMz0/edit#slide=id.g4274351d5e_1_392 Thanks, Claire -------------- next part -------------- An HTML attachment was scrubbed... URL: From claire at openstack.org Tue Dec 18 16:18:35 2018 From: claire at openstack.org (Claire Massey) Date: Tue, 18 Dec 2018 10:18:35 -0600 Subject: [Airship-discuss] CFP Open Until January 23, Open Infrastructure Summit in Denver Message-ID: Hi everyone, FYI - the CFP is now open for the first Open Infrastructure Summit (formerly the OpenStack Summit) which will be held in Denver, Colorado April 29 - May 1, 2019. Wednesday, *January 23* is the deadline to Submit presentations . The Open Infrastructure Summit is organized by OSF and designed to be a place where open source infrastructure communities can come together and collaborate in the open. Airship will have a large and prominent presence at the event so please submit talks to the CFP and plan to attend! SUBMIT YOUR PRESENTATION Important info: Based on previous Program Committee and attendee feedback, we have added / updated three Tracks: Security, Getting Started, and Open Development (previously Open Source Community). You can find the Track descriptions here . All of the OSF pilot projects —including Airship, Kata Containers, StarlingX and Zuul — will be front and center alongside other open source communities like Ansible, Cloud Foundry, Docker, Kubernetes, and many more. The Open Infrastructure Summit (formerly the OpenStack Summit), has evolved to recognize our diverse audience, and to signal to the market that the event is relevant for all IT infrastructure decision makers. If you’re interested in influencing the Summit content, apply to be a Programming Committee member *, where you can also find a full list of time requirements and expectations. Nominations will close on January 4, 2019. The content submission process for the Forum and Project Teams Gathering will be managed separately in the upcoming months. *OSF Staff will serve as the Programming Committee for the Getting Started Track. Denver Summit registration and sponsor sales are currently open. Learn more and email summit at openstack.org with any questions. Please email speakersupport at openstack.org with any questions or feedback. Thanks, Claire -------------- next part -------------- An HTML attachment was scrubbed... URL: From MM9745 at att.com Wed Dec 19 19:37:02 2018 From: MM9745 at att.com (MCEUEN, MATT) Date: Wed, 19 Dec 2018 19:37:02 +0000 Subject: [Airship-discuss] =?utf-8?q?Airship-in-a-bottle=5FDev=5FMinimal?= =?utf-8?q?=5FBedhind_Proxy=EF=BC=9AHow_to_Update_Site=3F?= In-Reply-To: <008001d49298$bed0db70$3c729250$@as.ntt-at.co.jp> References: <008001d49298$bed0db70$3c729250$@as.ntt-at.co.jp> Message-ID: <7C64A75C21BB8D43BD75BB18635E4D896CF0E916@MOSTLS1MSGUSRFF.ITServices.sbc.com> Hi Santosh - in general, the right way to update an airship site is by: 1) supplying updated deployment manifests to Shipyard, including the change 2) calling Shipyard's update_site API We don't have that as part of Aiship-in-a-Bottle, because AiaB is not intended to be used for "real life" sites -- it's a demo environment. I suggest taking a look at the Treasuremap documentation [1] and example manifests [2] as a way to stand up a fully lifecycled-site. Update_site (along with the rest of the Shipyard interface) is documented here [3] for the CLI and here [4] for the API. 1: https://airship-treasuremap.readthedocs.io/en/latest/authoring_and_deployment.html 2: https://github.com/openstack/airship-treasuremap 3: https://airship-shipyard.readthedocs.io/en/latest/CLI.html 4: https://airship-shipyard.readthedocs.io/en/latest/API.html The latest version of Promenade provisions a Kubernetes cluster that patches for the CVE you mention. Hope that helps - let me know if you have any other questions! Matt -----Original Message----- From: santosh.thapamager Sent: Wednesday, December 12, 2018 10:03 PM To: airship-discuss at lists.airshipit.org Subject: [Airship-discuss] Airship-in-a-bottle_Dev_Minimal_Bedhind Proxy:How to Update Site? Hi all, I was able to install dev_minimal and deploy site behind proxy. Because of Kubernetes apiserver vulnerability (CVE-2018-1002105)I wanted to update Kubernetes version in my currently deployed site. However I could not find any documentations regarding updating deployed site. I would be thankful to all of you, if I could get any support. Best Regards, Santosh Thapa Magar _______________________________________________ Airship-discuss mailing list Airship-discuss at lists.airshipit.org https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.airshipit.org_cgi-2Dbin_mailman_listinfo_airship-2Ddiscuss&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=Kp_9DVZcWk0qumyITPRn31pyOnpfXyno0HbJJJul7Xw&s=j6f8UNYzAnQn2PEYvDhC0N3F4i2vFF8qXyVC7Arx3hQ&e= From james at openstack.org Wed Dec 19 19:54:00 2018 From: james at openstack.org (James Cole) Date: Wed, 19 Dec 2018 11:54:00 -0800 Subject: [Airship-discuss] Airship Website Refresh Message-ID: <3605CB09-4575-462C-9BF9-E826A294CE20@openstack.org> Greetings Airship Team! I’m James Cole, a graphic designer with the OpenStack Foundation. We’ve been working on an updated look for the Airship website and wanted to get your feedback. Please view the updated site on Netlify: https://airship-website-dev.netlify.com/ A few things to note: The blog does not currently have any blog posts and we’d probably want to have something real published there before going live. If it isn’t possible to publish something before this launches, we can remove it from the navigation and add it back in once we have some content. Any guidance is welcomed if links on the Community page need to point to other locations. The Gerrit repo link does not currently link to anything useful, for instance. We’ve been considering adding a subhead under “Elevate Your Infrastructure” on the hero image of the homepage for clarity. One of the ideas we’re floating around is "Airship makes operating open infrastructure simple, repeatable, and resilient.” Does this sound good to you? Please let us know if you notice something that should change—spelling/grammar errors, factual errors, broken links, etc.—or if you have any other type of feedback. Once everyone is happy with the content and layout we’ll point the official URL (airshipit.org ) to this version of the site. Thank you! James Cole Graphic Designer OpenStack Foundation -------------- next part -------------- An HTML attachment was scrubbed... URL: From eli at mirantis.com Wed Dec 19 20:59:06 2018 From: eli at mirantis.com (Evgeny L) Date: Wed, 19 Dec 2018 12:59:06 -0800 Subject: [Airship-discuss] Airship Website Refresh In-Reply-To: <3605CB09-4575-462C-9BF9-E826A294CE20@openstack.org> References: <3605CB09-4575-462C-9BF9-E826A294CE20@openstack.org> Message-ID: Hi James, This looks awesome! Just some thoughts: it may be useful to have "Try it!" somewhere alongside "VIEW THE CODE" on the main page which would lead to "Airship in a bottle" docs, so newcomers can quickly find how to get it running. On a blog page there is "CONNECT" section which seems to have some broken links: https://airship-website-dev.netlify.com/blog/ Thanks you! On Wed, Dec 19, 2018 at 11:54 AM James Cole wrote: > Greetings Airship Team! > > > I’m James Cole, a graphic designer with the OpenStack Foundation. We’ve > been working on an updated look for the Airship website and wanted to > get your feedback. > > *Please view the updated site on Netlify:* > https://airship-website-dev.netlify.com/ > > A few things to note: > > The blog does not currently have any blog posts and we’d probably want to > have something real published there before going live. If it isn’t possible > to publish something before this launches, we can remove it from the > navigation and add it back in once we have some content. > > Any guidance is welcomed if links on the Community page need to point to > other locations. The Gerrit repo link does not currently link to anything > useful, for instance. > > We’ve been considering adding a subhead under “Elevate Your > Infrastructure” on the hero image of the homepage for clarity. One of the > ideas we’re floating around is "Airship makes operating open infrastructure > simple, repeatable, and resilient.” Does this sound good to you? > > Please let us know if you notice something that should > change—spelling/grammar errors, factual errors, broken links, etc.—or if > you have any other type of feedback. Once everyone is happy with the > content and layout we’ll point the official URL (airshipit.org) to this > version of the site. > > > Thank you! > > > *James Cole* > Graphic Designer > OpenStack Foundation > > > _______________________________________________ > Airship-discuss mailing list > Airship-discuss at lists.airshipit.org > http://lists.airshipit.org/cgi-bin/mailman/listinfo/airship-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From drewwalters96 at gmail.com Thu Dec 20 21:41:48 2018 From: drewwalters96 at gmail.com (Drew Walters) Date: Thu, 20 Dec 2018 15:41:48 -0600 Subject: [Airship-discuss] Armada API connectivity to Tiller (proxy problem) In-Reply-To: References: Message-ID: Hi Eli, The Armada API accepts a Tiller hostname as a safeguard in the event that Armada is unable to find a running Tiller pod. Shipyard uses the Python Kubernetes client to retrieve the IP address that it provides to Armada. 1. It is possible to create a clusterIP with DNS and provide a domain name to Armada. 2. Armada only communicates with one Tiller pod at a time. This can be changed per API request by providing the hostname of a different Tiller pod. In the event that a Tiller hostname is not provided, Armada selects the first running Tiller pod it encounters. We invite you to join our IRC team meeting (#airshipit on freenode) January 8th to help us better understand your desired configuration and discuss in more detail. In the meantime, please let us know if you have any additional questions! Best, Drew Walters On Wed, Dec 12, 2018 at 3:55 PM Evgeny L wrote: > Hi, > > I've been working on getting behind the proxy configuration for AIAB to > work [1] and I stumbled across a problem related to GRPC [2], which does > not support CIDR notation for no_proxy entries, this make Armada API to > fail to connect to Tiller, because it tries to use proxy configured with > environment variables. > > While looking for a solution other than doing some manual reconfiguration > [2], I started to have the following questions: > > 1. Why does Shipyard uses [4] Tiller Pod IP? Can we create a ClusterIP > and use a DNS name? > 2. Why Shipyard is the one who provides Armada API with Tiller IP > address? If we do 1st we can possibly make endpoint a part of Armada API > configuration. Or is Armada API supposed to work with multiple Tillers at > the same time? > > > Any feedback and historical background would be really helpful! > > Thanks, > > [1] https://review.openstack.org/#/c/624122/ > [2] > https://review.openstack.org/#/c/624504/2/manifests/dev_minimal/README.rst > [3] > https://github.com/grpc/grpc/blob/9e9cae7839a362936228cf333045e5da877ace40/src/core/ext/filters/client_channel/http_proxy.cc#L146-L149 > [4] > https://github.com/openstack/airship-shipyard/blob/a235f8e6db6e791550727e3575e045e26f177252/src/bin/shipyard_airflow/shipyard_airflow/plugins/armada_base_operator.py#L147-L153 > _______________________________________________ > Airship-discuss mailing list > Airship-discuss at lists.airshipit.org > http://lists.airshipit.org/cgi-bin/mailman/listinfo/airship-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From james at openstack.org Thu Dec 20 21:59:01 2018 From: james at openstack.org (James Cole) Date: Thu, 20 Dec 2018 13:59:01 -0800 Subject: [Airship-discuss] Airship Website Refresh In-Reply-To: References: <3605CB09-4575-462C-9BF9-E826A294CE20@openstack.org> Message-ID: Thanks for taking a look, Evgeny! We currently have the Airship in a bottle link on the community page, but maybe there can be a place on the homepage for it too. Good catch on the blog links—will update those. -James > On Dec 19, 2018, at 12:59 PM, Evgeny L wrote: > > Hi James, > > This looks awesome! > > Just some thoughts: it may be useful to have "Try it!" somewhere alongside "VIEW THE CODE" on the main page which would lead to "Airship in a bottle" docs, so newcomers can quickly find how to get it running. > > On a blog page there is "CONNECT" section which seems to have some broken links: https://airship-website-dev.netlify.com/blog/ > > Thanks you! > > On Wed, Dec 19, 2018 at 11:54 AM James Cole > wrote: > Greetings Airship Team! > > > I’m James Cole, a graphic designer with the OpenStack Foundation. We’ve been working on an updated look for the Airship website and wanted to get your feedback. > > Please view the updated site on Netlify: https://airship-website-dev.netlify.com/ > > A few things to note: > > The blog does not currently have any blog posts and we’d probably want to have something real published there before going live. If it isn’t possible to publish something before this launches, we can remove it from the navigation and add it back in once we have some content. > > Any guidance is welcomed if links on the Community page need to point to other locations. The Gerrit repo link does not currently link to anything useful, for instance. > > We’ve been considering adding a subhead under “Elevate Your Infrastructure” on the hero image of the homepage for clarity. One of the ideas we’re floating around is "Airship makes operating open infrastructure simple, repeatable, and resilient.” Does this sound good to you? > > Please let us know if you notice something that should change—spelling/grammar errors, factual errors, broken links, etc.—or if you have any other type of feedback. Once everyone is happy with the content and layout we’ll point the official URL (airshipit.org ) to this version of the site. > > > Thank you! > > > James Cole > Graphic Designer > OpenStack Foundation > > > _______________________________________________ > Airship-discuss mailing list > Airship-discuss at lists.airshipit.org > http://lists.airshipit.org/cgi-bin/mailman/listinfo/airship-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From santosh.thapamager at as.ntt-at.co.jp Thu Dec 20 23:59:13 2018 From: santosh.thapamager at as.ntt-at.co.jp (santosh.thapamager) Date: Fri, 21 Dec 2018 08:59:13 +0900 Subject: [Airship-discuss] =?utf-8?q?Airship-in-a-bottle=5FDev=5FMinimal?= =?utf-8?q?=5FBedhind_Proxy=EF=BC=9AHow_to_Update_Site=3F?= In-Reply-To: <7C64A75C21BB8D43BD75BB18635E4D896CF0E916@MOSTLS1MSGUSRFF.ITServices.sbc.com> References: <008001d49298$bed0db70$3c729250$@as.ntt-at.co.jp> <7C64A75C21BB8D43BD75BB18635E4D896CF0E916@MOSTLS1MSGUSRFF.ITServices.sbc.com> Message-ID: <002101d498c0$020cc0c0$06264240$@as.ntt-at.co.jp> Hi Matt, Thanks for the instruction. I have been following airship-treasuremap document. As you mentioned that we need to update the deployment manifest and supply to Shipyard and call Shipyard update_site API. *************** 1) supplying updated deployment manifests to Shipyard, including the change 2) calling Shipyard's update_site API *************** However I could not find instruction for 1) as you mentioned. I would be greatly thankful to you if I can get some instruction for updating the deployment manifests and how to supply updated manifests to Shipyard. Best Regards, Santosh Thapa Magar -----Original Message----- From: MCEUEN, MATT Sent: Thursday, December 20, 2018 4:37 AM To: santosh.thapamager ; airship-discuss at lists.airshipit.org Subject: RE: [Airship-discuss] Airship-in-a-bottle_Dev_Minimal_Bedhind Proxy:How to Update Site? Hi Santosh - in general, the right way to update an airship site is by: 1) supplying updated deployment manifests to Shipyard, including the change 2) calling Shipyard's update_site API We don't have that as part of Aiship-in-a-Bottle, because AiaB is not intended to be used for "real life" sites -- it's a demo environment. I suggest taking a look at the Treasuremap documentation [1] and example manifests [2] as a way to stand up a fully lifecycled-site. Update_site (along with the rest of the Shipyard interface) is documented here [3] for the CLI and here [4] for the API. 1: https://airship-treasuremap.readthedocs.io/en/latest/authoring_and_deployment.html 2: https://github.com/openstack/airship-treasuremap 3: https://airship-shipyard.readthedocs.io/en/latest/CLI.html 4: https://airship-shipyard.readthedocs.io/en/latest/API.html The latest version of Promenade provisions a Kubernetes cluster that patches for the CVE you mention. Hope that helps - let me know if you have any other questions! Matt -----Original Message----- From: santosh.thapamager Sent: Wednesday, December 12, 2018 10:03 PM To: airship-discuss at lists.airshipit.org Subject: [Airship-discuss] Airship-in-a-bottle_Dev_Minimal_Bedhind Proxy:How to Update Site? Hi all, I was able to install dev_minimal and deploy site behind proxy. Because of Kubernetes apiserver vulnerability (CVE-2018-1002105)I wanted to update Kubernetes version in my currently deployed site. However I could not find any documentations regarding updating deployed site. I would be thankful to all of you, if I could get any support. Best Regards, Santosh Thapa Magar _______________________________________________ Airship-discuss mailing list Airship-discuss at lists.airshipit.org https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.airshipit.org_cgi-2Dbin_mailman_listinfo_airship-2Ddiscuss&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=Kp_9DVZcWk0qumyITPRn31pyOnpfXyno0HbJJJul7Xw&s=j6f8UNYzAnQn2PEYvDhC0N3F4i2vFF8qXyVC7Arx3hQ&e= From eli at mirantis.com Wed Dec 26 18:56:06 2018 From: eli at mirantis.com (Evgeny L) Date: Wed, 26 Dec 2018 10:56:06 -0800 Subject: [Airship-discuss] Armada API connectivity to Tiller (proxy problem) In-Reply-To: References: Message-ID: Hi Drew, Thank you for the response! >> The Armada API accepts a Tiller hostname as a safeguard in the event that Armada is unable to find a running Tiller pod Looking at the code [1] it seems to be that Armada will try to find Tiller pod in the event Shipyard doesn't find one. Was there some problem with Armada API not being able to retrieve Tiller endpoint so it required a fallback? They both seem to use k8s API for that. >> This can be changed per API request by providing the hostname of a different Tiller pod. >From architecture point of view, do we expect it to work with multiple Tillers within a single Airship installation? I'm asking these questions to explore the options for resolving GRPC (which is used by Tiller client) [2] constraints, it does not handle CIDR in no_proxy environment variables [3]. One of the options is: 1. [high priority] Expose Tiller to create a DNS name for the pod, and use it when making requests from Armada API to Tiller, in this case Aramada API POD can have ".cluster.local" in NO_PROXY and everything would work just fine (i.e. requests to Tiller won't go through the proxy). 2. [low priority] Reduce coupling between Tiller/k8s and Shipyard by configuring Tiller endpoints in Armada API directly without requiring Shipyard to pass one. Prior to submitting any changes I would like to verify with you if this makes sense or if I should explore other options. Thanks, [1] https://github.com/openstack/airship-armada/blob/307f1318c4e83f247f1e3838478957e7555d6ce0/armada/handlers/tiller.py#L158-L164 [2] https://review.openstack.org/#/c/624504/2/manifests/dev_minimal/README.rst [3] https://github.com/grpc/grpc/blob/9e9cae7839a362936228cf333045e5da877ace40/src/core/ext/filters/client_channel/http_proxy.cc#L146-L149 On Thu, Dec 20, 2018 at 1:42 PM Drew Walters wrote: > Hi Eli, > > The Armada API accepts a Tiller hostname as a safeguard in the event that > Armada is unable to find a running Tiller pod. Shipyard uses the Python > Kubernetes client to retrieve the IP address that it provides to Armada. > > 1. It is possible to create a clusterIP with DNS and provide a domain name > to Armada. > > 2. Armada only communicates with one Tiller pod at a time. This can be > changed per API request by providing the hostname of a different Tiller > pod. In the event that a Tiller hostname is not provided, Armada selects > the first running Tiller pod it encounters. > > We invite you to join our IRC team meeting (#airshipit on freenode) > January 8th to help us better understand your desired configuration and > discuss in more detail. In the meantime, please let us know if you have any > additional questions! > > Best, > Drew Walters > > > On Wed, Dec 12, 2018 at 3:55 PM Evgeny L wrote: > >> Hi, >> >> I've been working on getting behind the proxy configuration for AIAB to >> work [1] and I stumbled across a problem related to GRPC [2], which does >> not support CIDR notation for no_proxy entries, this make Armada API to >> fail to connect to Tiller, because it tries to use proxy configured with >> environment variables. >> >> While looking for a solution other than doing some manual reconfiguration >> [2], I started to have the following questions: >> >> 1. Why does Shipyard uses [4] Tiller Pod IP? Can we create a >> ClusterIP and use a DNS name? >> 2. Why Shipyard is the one who provides Armada API with Tiller IP >> address? If we do 1st we can possibly make endpoint a part of Armada API >> configuration. Or is Armada API supposed to work with multiple Tillers at >> the same time? >> >> >> Any feedback and historical background would be really helpful! >> >> Thanks, >> >> [1] https://review.openstack.org/#/c/624122/ >> [2] >> https://review.openstack.org/#/c/624504/2/manifests/dev_minimal/README.rst >> [3] >> https://github.com/grpc/grpc/blob/9e9cae7839a362936228cf333045e5da877ace40/src/core/ext/filters/client_channel/http_proxy.cc#L146-L149 >> [4] >> https://github.com/openstack/airship-shipyard/blob/a235f8e6db6e791550727e3575e045e26f177252/src/bin/shipyard_airflow/shipyard_airflow/plugins/armada_base_operator.py#L147-L153 >> _______________________________________________ >> Airship-discuss mailing list >> Airship-discuss at lists.airshipit.org >> http://lists.airshipit.org/cgi-bin/mailman/listinfo/airship-discuss > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Shoaib.Nasir at windriver.com Thu Dec 13 18:36:03 2018 From: Shoaib.Nasir at windriver.com (Nasir, Shoaib) Date: Thu, 13 Dec 2018 18:36:03 -0000 Subject: [Airship-discuss] Armada Compatibility with Keystone Message-ID: <25B6D7837B9EB04495DAD8D02D71DFF0203E9DF7@ALA-MBD.corp.ad.wrs.com> Hi I have a kubernetes cluster that is setup with Keystone as its Authentication/Authorization Backend (via the k8s-keystone-auth Webhook pod) With that, any application such as helm now needs to pass user credentials or a keystone token when talking to the kubernetes-apiserver If I set up env OS_ variables and use any helm commands (such as helm init, or helm lists etc), everything works fine as these OS_ user credentials are authorized via the kubernetes keystone backend However, I run into problems with Armada. After bringing up the armada docker container, and logging into it to execute any armada apply commands, and using the -token argument to pass along a valid keystone token, armada still fails to access the kubernetes cluster."User "system:anonymous" suggests that the token that armada cli gets, does not pass along to the kubernetes-apiserver. I'm very new to armada and need some assistance in figuring out how to setup the armada container or any other configurations required to make it pass the keystone token to the kubernetes-apiserver. Each and every armada command fails similar to the one below. Any assistance in this regard would be greatly appreciated ? armada at c7f42c9263ad:~$ armada --token $TOKEN --debug apply 2018-12-13 18:10:13.656 72 ERROR armada.cli [-] Caught unexpected exception: kubernetes.client.rest.ApiException: (403) Reason: Forbidden HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 13 Dec 2018 18:10:13 GMT', 'Content-Length': '265', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff'}) HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User \"system:anonymous\" cannot list resource \"pods\" in API group \"\" in the namespace \"kube-system\"","reason":"Forbidden","details":{"kind":"pods"},"code":403} 2018-12-13 18:10:13.656 72 ERROR armada.cli Traceback (most recent call last): 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/cli/__init__.py", line 39, in safe_invoke 2018-12-13 18:10:13.656 72 ERROR armada.cli self.invoke() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/cli/apply.py", line 215, in invoke 2018-12-13 18:10:13.656 72 ERROR armada.cli target_manifest=self.target_manifest) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/armada.py", line 103, in __init__ 2018-12-13 18:10:13.656 72 ERROR armada.cli dry_run=dry_run) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 95, in __init__ 2018-12-13 18:10:13.656 72 ERROR armada.cli self.channel = self.get_channel() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 117, in get_channel 2018-12-13 18:10:13.656 72 ERROR armada.cli tiller_ip = self._get_tiller_ip() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 163, in _get_tiller_ip 2018-12-13 18:10:13.656 72 ERROR armada.cli pod = self._get_tiller_pod() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 141, in _get_tiller_pod 2018-12-13 18:10:13.656 72 ERROR armada.cli CONF.tiller_pod_labels).items 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/k8s.py", line 215, in get_namespace_pod 2018-12-13 18:10:13.656 72 ERROR armada.cli namespace, label_selector=label_selector) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/apis/core_v1_api.py", line 12310, in list_namespaced_pod 2018-12-13 18:10:13.656 72 ERROR armada.cli (data) = self.list_namespaced_pod_with_http_info(namespace, **kwargs) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/apis/core_v1_api.py", line 12413, in list_namespaced_pod_with_http_info 2018-12-13 18:10:13.656 72 ERROR armada.cli collection_formats=collection_formats) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 321, in call_api 2018-12-13 18:10:13.656 72 ERROR armada.cli _return_http_data_only, collection_formats, _preload_content, _request_timeout) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 155, in __call_api 2018-12-13 18:10:13.656 72 ERROR armada.cli _request_timeout=_request_timeout) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 342, in request 2018-12-13 18:10:13.656 72 ERROR armada.cli headers=headers) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/rest.py", line 231, in GET 2018-12-13 18:10:13.656 72 ERROR armada.cli query_params=query_params) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/rest.py", line 222, in request 2018-12-13 18:10:13.656 72 ERROR armada.cli raise ApiException(http_resp=r) 2018-12-13 18:10:13.656 72 ERROR armada.cli kubernetes.client.rest.ApiException: (403) 2018-12-13 18:10:13.656 72 ERROR armada.cli Reason: Forbidden 2018-12-13 18:10:13.656 72 ERROR armada.cli HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 13 Dec 2018 18:10:13 GMT', 'Content-Length': '265', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff'}) 2018-12-13 18:10:13.656 72 ERROR armada.cli HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User \"system:anonymous\" cannot list resource \"pods\" in API group \"\" in the namespace \"kube-system\"","reason":"Forbidden","details":{"kind":"pods"},"code":403} 2018-12-13 18:10:13.656 72 ERROR armada.cli 2018-12-13 18:10:13.656 72 ERROR armada.cli 2018-12-13 18:10:13.656 72 ERROR armada.cli -------------- next part -------------- An HTML attachment was scrubbed... URL: