[Airship-discuss] Airship in a bottle install failure...

Tue Dec 11 15:28:57 UTC 2018

Hi there everyone

Thanks for the quick replies, I tried a new install on a fresh VM, behaves the same, problem is persisting:

I see in Postgresql-0 logs:

--snip

FATAL:  the database system is in recovery mode
LOG:  database system was not properly shut down; automatic recovery in progress
LOG:  invalid record length at 0/102A400
LOG:  redo is not required
LOG:  MultiXact member wraparound protections are now enabled
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
FATAL:  could not open relation mapping file "global/pg_filenode.map": No such file or directory
LOG:  autovacuum launcher process (PID 79) exited with exit code 1
LOG:  terminating any other active server processes
LOG:  all server processes terminated; reinitializing
LOG:  database system was interrupted; last known up at 2018-12-11 15:25:38 UTC
LOG:  database system was not properly shut down; automatic recovery in progress
--snip

and in mariadb-server-0 logs:

--snip

ubuntu at airship:~$ sudo kubectl -n ucp logs mariadb-server-0
2018-12-11 15:25:35,704 - OpenStack-Helm Mariadb - INFO - This instance hostname: mariadb-server-0
2018-12-11 15:25:36,310 - OpenStack-Helm Mariadb - INFO - Kubernetes API Version: v1.10.2
2018-12-11 15:25:36,313 - OpenStack-Helm Mariadb - INFO - Will use "airship-ucp-mariadb-mariadb-state" configmap for cluster state info
2018-12-11 15:25:36,313 - OpenStack-Helm Mariadb - INFO - Getting cluster state
2018-12-11 15:25:36,348 - OpenStack-Helm Mariadb - INFO - The cluster is currently in "init" state.
2018-12-11 15:25:36,348 - OpenStack-Helm Mariadb - INFO - Getting cluster state
2018-12-11 15:25:36,455 - OpenStack-Helm Mariadb - INFO - The cluster is currently in "init" state.
2018-12-11 15:25:36,456 - OpenStack-Helm Mariadb - INFO - Waiting for cluster to start running
2018-12-11 15:25:36,456 - OpenStack-Helm Mariadb - INFO - Getting cluster state
2018-12-11 15:25:36,474 - OpenStack-Helm Mariadb - INFO - The cluster is currently in "init" state.
2018-12-11 15:25:56,491 - OpenStack-Helm Mariadb - INFO - Getting cluster state
2018-12-11 15:25:56,637 - OpenStack-Helm Mariadb - INFO - The cluster is currently in "init" state.
2018-12-11 15:26:16,657 - OpenStack-Helm Mariadb - INFO - Getting cluster state
2018-12-11 15:26:16,678 - OpenStack-Helm Mariadb - INFO - The cluster is currently in "init" state.
2018-12-11 15:26:36,684 - OpenStack-Helm Mariadb - INFO - Getting cluster state
2018-12-11 15:26:36,697 - OpenStack-Helm Mariadb - INFO - The cluster is currently in "init" state.
2018-12-11 15:26:56,717 - OpenStack-Helm Mariadb - INFO - Getting cluster state
2018-12-11 15:26:56,749 - OpenStack-Helm Mariadb - INFO - The cluster is currently in "init" state.
2018-12-11 15:27:16,761 - OpenStack-Helm Mariadb - INFO - Getting cluster state
2018-12-11 15:27:16,876 - OpenStack-Helm Mariadb - INFO - The cluster is currently in "init" state.
--snip

Best Regards, D

________________________________
From: Alex Arnoldy <alex.arnoldy at suse.com>
Sent: Monday, December 10, 2018 7:48 PM
To: MCEUEN, MATT ; airship-discuss at lists.airshipit.org; Balsiger Daniel, INI-EAI-INO-MBL
Subject: RE: Airship in a bottle install failure...

Hi Matt,

Here is the commit I’m using:

commit 221441f1bff351ab366b02d6fd7856b70b06108f

Author: Bryan Strassner <bryan.strassner at gmail.com>

Date:   Mon Nov 19 17:12:10 2018 -0600

    Update apiserver for admission controller

    Update the apiserver's command prefix to match between the apiserver

    chart and the genesis api server.

    Includes an optimization of virsh commands to parallelize and wait for

    cloud init to complete (requires Ubuntu 16.04.4)

    Change-Id: I793d210e15c3c13e8b662bd2459ed8bcba30d434

I’m happy to start over to see if a new pull resolves the issue, but it happened to me twice in a row so I thought I’d preserve the environment for examination.

Best,

--Alex.

Alex Arnoldy
Software Defined Infrastructure Solutions Architect
SUSE
Alex.Arnoldy at SUSE.com<mailto:Alex.Arnoldy at SUSE.com>
(P) +1 (408) 828-9797

From: MCEUEN, MATT <MM9745 at att.com>
Sent: Monday, December 10, 2018 9:33 AM
To: airship-discuss at lists.airshipit.org; Alex Arnoldy <alex.arnoldy at suse.com>; INI-EAI-INO-MBL Balsiger Daniel <Daniel.Balsiger at swisscom.com>
Subject: RE: Airship in a bottle install failure...

Weird, the maas-db-init job is running fine for me with a fresh deployment of airship in a bottle.  Alex, are you seeing that “no pg_hba.conf” error current aiab code, or can you give git commit where you’re hitting the issue?

Daniel, can you please run “sudo kubectl -n ucp logs mariadb-server-0” and share the results?  Looks like MariaDB isn’t starting cleanly for some reason.

Thanks,
Matt

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.airshipit.org/pipermail/airship-discuss/attachments/20181211/f64fa683/attachment-0001.html>