[Airship-discuss] Error installing Airship In A Bottle

Mon Mar 18 08:54:21 UTC 2019

Hi Nick,

I think I face a similar problem in my first attempts to install AiaB (I'm
a bit further but also stuck too).
Note that I'm behind a proxy and that some filters are activated by our IT
so I need to replace the default google nameservers.
I fix this by adding 2 dns nameservers in /etc/resolv.conf and in the
'common-addresses.yaml' file:

  echo "nameserver __dns1__" >>/etc/resolv.conf
  echo "nameserver __dns2__" >>/etc/resolv.conf

  export
common_adr_file=/root/deploy/airship-in-a-bottle/deployment_files/site/demo/networks/common-addresses.yaml
  sed -i 's/8.8.8.8/__dns1__/g' $common_adr_file
  sed -i 's/8.8.4.4/__dns2__/g' $common_adr_file

Hope it helps, BR,
Loïc

Le dim. 17 mars 2019 à 18:46, Nick Chase <nchase at mirantis.com> a écrit :

> Very nice!  Capital letters was indeed the problem -- or at least, THAT
> problem.  I created a new VM without them and started again, getting past
> that point.  However, I'm now stuck with CrashLoopBack on the coredns
> pods.  I've been trying to grab logs without success.  However, I am seeing
> this:
>
> 2019-03-17 17:31:29.142 3396 INFO armada.handlers.armada [-] Processing
> ChartGroup: kubernetes-proxy (Kubernetes proxy), sequenced=True
> 2019-03-17 17:31:29.143 3396 INFO armada.handlers.chart_deploy [-]
> [chart=proxy]: Processing Chart, release=airship-kubernetes-proxy
> 2019-03-17 17:31:29.145 3396 INFO armada.handlers.chartbuilder [-]
> [chart=proxy]: Building dependency chart kubernetes-proxy-htk for release
> kubernetes-proxy.
> 2019-03-17 17:31:29.196 3396 INFO armada.handlers.chart_deploy [-]
> [chart=proxy]: Existing release airship-kubernetes-proxy found in namespace
> kube-system
> 2019-03-17 17:31:29.200 3396 INFO armada.handlers.chart_deploy [-]
> [chart=proxy]: Checking for updates to chart release inputs.
> 2019-03-17 17:31:29.238 3396 INFO armada.handlers.chart_deploy [-]
> [chart=proxy]: Found no updates to chart release inputs
> 2019-03-17 17:31:29.239 3396 INFO armada.handlers.wait [-] [chart=proxy]:
> Waiting for resource type=job, namespace=kube-system
> labels=release_group=airship-kubernetes-proxy for 600s (k8s wait 1 times,
> sleep 1s)
> 2019-03-17 17:31:29.279 3396 INFO armada.handlers.wait [-] [chart=proxy]:
> Waiting for resource type=pod, namespace=kube-system
> labels=release_group=airship-kubernetes-proxy for 600s (k8s wait 1 times,
> sleep 1s)
> 2019-03-17 17:31:29.297 3396 INFO armada.handlers.armada [-] All Charts
> applied in ChartGroup kubernetes-proxy.
> 2019-03-17 17:31:29.298 3396 INFO armada.handlers.armada [-] Processing
> ChartGroup: kubernetes-container-networking (Container networking via
> Calico), sequenced=True
> 2019-03-17 17:31:29.298 3396 INFO armada.handlers.chart_deploy [-]
> [chart=etcd]: Processing Chart, release=airship-kubernetes-calico-etcd
> 2019-03-17 17:31:29.298 3396 INFO armada.handlers.chartbuilder [-]
> [chart=etcd]: Building dependency chart kubernetes-calico-etcd-htk for
> release kubernetes-calico-etcd.
> 2019-03-17 17:31:29.312 3396 INFO armada.handlers.chart_deploy [-]
> [chart=etcd]: Existing release airship-kubernetes-calico-etcd found in
> namespace kube-system
> 2019-03-17 17:31:29.347 3396 INFO armada.handlers.chart_deploy [-]
> [chart=etcd]: Checking for updates to chart release inputs.
> 2019-03-17 17:31:29.421 3396 INFO armada.handlers.chart_deploy [-]
> [chart=etcd]: Found no updates to chart release inputs
> 2019-03-17 17:31:29.422 3396 INFO armada.handlers.wait [-] [chart=etcd]:
> Waiting for resource type=job, namespace=kube-system
> labels=release_group=airship-kubernetes-calico-etcd for 600s (k8s wait 1
> times, sleep 1s)
> 2019-03-17 17:31:29.425 3396 INFO armada.handlers.wait [-] [chart=etcd]:
> Waiting for resource type=pod, namespace=kube-system
> labels=release_group=airship-kubernetes-calico-etcd for 600s (k8s wait 1
> times, sleep 1s)
> 2019-03-17 17:31:29.432 3396 INFO armada.handlers.chart_deploy [-]
> [chart=calico]: Processing Chart, release=airship-kubernetes-calico
> 2019-03-17 17:31:29.432 3396 INFO armada.handlers.chartbuilder [-]
> [chart=calico]: Building dependency chart calico-htk for release
> kubernetes-calico.
> 2019-03-17 17:31:29.456 3396 INFO armada.handlers.chart_deploy [-]
> [chart=calico]: Existing release airship-kubernetes-calico found in
> namespace kube-system
> 2019-03-17 17:31:29.498 3396 INFO armada.handlers.chart_deploy [-]
> [chart=calico]: Checking for updates to chart release inputs.
> 2019-03-17 17:31:29.642 3396 INFO armada.handlers.chart_deploy [-]
> [chart=calico]: Found no updates to chart release inputs
> 2019-03-17 17:31:29.642 3396 INFO armada.handlers.wait [-] [chart=calico]:
> Waiting for resource type=job, namespace=kube-system
> labels=release_group=airship-kubernetes-calico for 600s (k8s wait 1 times,
> sleep 1s)
> 2019-03-17 17:31:29.648 3396 INFO armada.handlers.wait [-] [chart=calico]:
> Waiting for resource type=pod, namespace=kube-system
> labels=release_group=airship-kubernetes-calico for 600s (k8s wait 1 times,
> sleep 1s)
> 2019-03-17 17:31:29.667 3396 INFO armada.handlers.armada [-] All Charts
> applied in ChartGroup kubernetes-container-networking.
> 2019-03-17 17:31:29.667 3396 INFO armada.handlers.armada [-] Processing
> ChartGroup: kubernetes-dns (Cluster DNS), sequenced=False
> 2019-03-17 17:31:29.668 3396 INFO armada.handlers.chart_deploy [-]
> [chart=coredns]: Processing Chart, release=airship-coredns
> 2019-03-17 17:31:29.668 3396 INFO armada.handlers.chartbuilder [-]
> [chart=coredns]: Building dependency chart coredns-htk for release coredns.
> 2019-03-17 17:31:29.685 3396 INFO armada.handlers.chart_deploy [-]
> [chart=coredns]: Purging FAILED release airship-coredns before deployment.
> 2019-03-17 17:31:29.685 3396 INFO armada.handlers.tiller [-]
> [chart=coredns]: Uninstall airship-coredns release with
> disable_hooks=False, purge=True flags
> 2019-03-17 17:31:31.016 3396 INFO armada.handlers.chart_deploy [-]
> [chart=coredns]: Installing release airship-coredns in namespace
> kube-system, wait=True, timeout=599s
> 2019-03-17 17:31:31.018 3396 INFO armada.handlers.tiller [-]
> [chart=coredns]: Helm install release: wait=True, timeout=599
>
> Any thoughts?
>
> ----  Nick
>
> On Wed, Mar 13, 2019 at 10:56 PM Hemanth N <mail2hemanth.n at gmail.com>
> wrote:
>
>> Hi Nick
>>
>> First thing i would do, is look for apiserver and etcd container errors
>>
>> Can you check the following as well
>>
>> 1. Verify if etcd service is up or not
>>     On VM, run netstat -ntulp | grep 2379
>>
>> 2. journalctl -u kubelet.service | grep "Invalid value"
>>    Look for errros, it might point out any pod spec issues
>>
>> Also please provide following information
>> 3. cat /etc/kubernetes/manifests/kubernetes-etcd.yaml
>>     Check under volumes section if the names have any CAPITAL letters
>>
>> 4. hostname -s
>>
>> My hunch is you have hostname with a CAPS letter which is triggering
>> this failure. Above commands should give an idea if that is the cause.
>> If thats the case, change your hostname to all lowercase and then run
>> airship-in-a-bottle.
>>
>> Regards,
>> Hemanth
>>
>>
>>
>>
>>
>>
>> > Message: 3
>> > Date: Tue, 12 Mar 2019 17:01:51 -0400
>> > From: Nick Chase <nchase at mirantis.com>
>> > To: "MCEUEN, MATT" <MM9745 at att.com>
>> > Cc: Evgeny L <eli at mirantis.com>,
>> >         "airship-discuss at lists.airshipit.org"
>> >         <airship-discuss at lists.airshipit.org>
>> > Subject: Re: [Airship-discuss] Error installing Airship In A Bottle
>> > Message-ID:
>> >         <CA+p99G=RzD9+4qAOF5bv-Z=fG0phye=
>> 55CxMPnC7bytG+RSg4w at mail.gmail.com>
>> > Content-Type: text/plain; charset="utf-8"
>> >
>> > OK, so if it IS a timeout thing, if you can tell me how to increase it,
>> > I'll appreciate it, because the script finishes, it just finishes with
>> an
>> > error.
>> >
>> > So the Airship script log is here:
>> http://paste.openstack.org/show/747758/
>> > Docker processes here: http://paste.openstack.org/show/747759/
>> >
>> > Thanks....
>> >
>> > ----  Nick
>> >
>> >
>> > On Wed, Mar 13, 2019 at 6:39 PM MCEUEN, MATT <MM9745 at att.com> wrote:
>> >
>> > > Agree, and I’d add to check the docker logs of the kube apiserver to
>> make
>> > > sure it’s starting up healthy.
>> > >
>> > >
>> > >
>> > > If you’re not seeing a smoking gun, can you do a `docker ps -a` and
>> > > pastebin the results?
>> > >
>> > >
>> > >
>> > > Thanks,
>> > >
>> > > Matt
>> > >
>> > >
>> > >
>> > > *From:* Evgeny L <eli at mirantis.com>
>> > > *Sent:* Wednesday, March 13, 2019 12:59 PM
>> > > *To:* Nick Chase <nchase at mirantis.com>
>> > > *Cc:* airship-discuss at lists.airshipit.org
>> > > *Subject:* Re: [Airship-discuss] Error installing Airship In A Bottle
>> > >
>> > >
>> > >
>> > > Hi Nick,
>> > >
>> > >
>> > >
>> > > As far as I remember these errors are just retries that wait until
>> > > Kubernetes is up and running, have you tried to wait a bit?
>> > >
>> > > If it still fails for you, can you please send us a complete log? You
>> can
>> > > use http://paste.openstack.org/
>> > > <
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__paste.openstack.org_&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=WhNp-VjTw70e2z_QbunOZc8fyS-zMWVbK70DnNjJMS0&s=8BzvCwZPVXNqnW1cYOCO0Av7scqQCiZlumjUr7A2dk8&e=
>> >
>> > > for that.
>> > >
>> > >
>> > >
>> > > Thanks!
>> > >
>> > >
>> > >
>> > > On Mon, Mar 11, 2019 at 8:59 PM Nick Chase <nchase at mirantis.com>
>> wrote:
>> > >
>> > > I'm trying to install AIB on a Unbutu Virtualbox VM with 48GB of RAM,
>> 4
>> > > CPU and a 300GB hard drive.  Everything seems to go along fine, until
>> we
>> > > get here:
>> > >
>> > >
>> > >
>> > > Digest:
>> > >
>> sha256:badd2b1da29d4d530b10b920f64bf66a1b41150db46c3c99b49d56f3f18a82db
>> > > Status: Downloaded newer image for
>> > > gcr.io/google_containers/hyperkube-amd64:v1.10.2
>> > > <
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__gcr.io_google-5Fcontainers_hyperkube-2Damd64-3Av1.10.2&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=WhNp-VjTw70e2z_QbunOZc8fyS-zMWVbK70DnNjJMS0&s=RON0xE-JpIMxDTL_Ja4E7LJK_asUyNLjZiaJmHyci3M&e=
>> >
>> > > Unable to connect to the server: EOF
>> > > .Unable to connect to the server: net/http: request canceled
>> > > (Client.Timeout exceeded while awaiting headers)
>> > >
>> > >
>> > >
>> > > From there, it just keeps giving these two errors.
>> > >
>> > >
>> > >
>> > > Any thoughts?
>> > >
>> > >
>> > >
>> > > Thanks...
>> > >
>> > >
>> > >
>> > > ----  Nick
>> > >
>> > > _______________________________________________
>> > > Airship-discuss mailing list
>> > > Airship-discuss at lists.airshipit.org
>> > > http://lists.airshipit.org/cgi-bin/mailman/listinfo/airship-discuss
>> > > <
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.airshipit.org_cgi-2Dbin_mailman_listinfo_airship-2Ddiscuss&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=WhNp-VjTw70e2z_QbunOZc8fyS-zMWVbK70DnNjJMS0&s=nBDu-ohIh1kHRZSZCF0aSC5mIryxhGUOu9y-I8hj7dc&e=
>> >
>> > >
>> > >
>> > -------------- next part --------------
>> > An HTML attachment was scrubbed...
>> > URL: <
>> http://lists.airshipit.org/pipermail/airship-discuss/attachments/20190312/5c30236d/attachment.html
>> >
>> >
>> > ------------------------------
>> >
>> > Subject: Digest Footer
>> >
>> > _______________________________________________
>> > Airship-discuss mailing list
>> > Airship-discuss at lists.airshipit.org
>> > http://lists.airshipit.org/cgi-bin/mailman/listinfo/airship-discuss
>> >
>> > ------------------------------
>> >
>> > End of Airship-discuss Digest, Vol 10, Issue 12
>> > ***********************************************
>>
> _______________________________________________
> Airship-discuss mailing list
> Airship-discuss at lists.airshipit.org
> http://lists.airshipit.org/cgi-bin/mailman/listinfo/airship-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.airshipit.org/pipermail/airship-discuss/attachments/20190318/be731f8b/attachment-0001.html>