<div dir="ltr"><div class="gmail_quote"><div dir="ltr">Thanks again Kaspars,<div><br></div><div>Adding all these charts seem too huge even for a 32 GB RAM VM (I'll try again, in case the failure came from a latency of our infra).</div><div>Anyway, I succeed to add only Prometheus, AlertManager and Grafana by doing this:</div><div>A - add these chartgroups in aiab bootstrap manifest </div><div><font face="monospace"> - osh-infra-monitoring</font></div><font face="monospace"> - osh-infra-ingress-controller<br> - osh-infra-mariadb<br> - osh-infra-dashboards</font><br><div><br></div><div>B - Overload the osh-infra-dashboard chartgroup to include only grafana (using deckhand overloading mechanism instead of brute removal )</div><div> B1. add a label at the global layer to avoid Deckhand complains in B2</div><div><font face="monospace">+++ b/global/software/charts/osh-infra/osh-infra-dashboards/chart-group.yaml<br>@@ -6,6 +6,8 @@ metadata:<br> layeringDefinition:<br> abstract: false<br> layer: global<br>+ labels:<br>+ name: osh-infra-dashboards-global<br> storagePolicy: cleartext</font><br></div><div> </div><div> B2. create a new file at the type layer that only contains grafana:</div><div><font face="monospace">+++ b/type/sloop/charts/osh-infra/dashboards-chartgroup.yaml<br>@@ -0,0 +1,20 @@<br>+---<br>+schema: armada/ChartGroup/v1<br>+metadata:<br>+ schema: metadata/Document/v1<br>+ replacement: true<br>+ name: osh-infra-dashboards<br>+ layeringDefinition:<br>+ abstract: false<br>+ layer: type<br>+ parentSelector:<br>+ name: osh-infra-dashboards-global<br>+ actions:<br>+ - method: merge<br>+ path: .<br>+ storagePolicy: cleartext<br>+data:<br>+ # disable kibana by changing the chartgroup content<br>+ chart_group:<br>+ - grafana<br>+...</font><br>C - Expose the services:</div><div><p style="margin:0in"><font face="monospace">kubectl expose pod/prometheus-0 -n osh-infra
--type=NodePort --name=prometheus-dashboard</font></p></div><div><font face="monospace">kubectl expose pod/alertmanager-0 -n osh-infra --type=NodePort --name=alertmgr-dashboard<br></font></div><div><font face="monospace">kubectl expose service/grafana-dashboard -n osh-infra --type=NodePort --name=grafana-web</font><br></div><div><br></div><div>Not every Grafana Dashboards are operational in aiab but those that are are really good !</div><div><br></div><div><div>BR, Loïc<br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Le mer. 7 août 2019 à 17:27, SKELS, KASPARS <<a href="mailto:ks3019@att.com" target="_blank">ks3019@att.com</a>> a écrit :<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div style="direction:ltr;font-family:Tahoma;color:rgb(0,0,0);font-size:10pt">
<div>Hi, <br>
</div>
<div><br>
</div>
<div>these charts are used in Airsloop/Seaworthy runs, so they should be tested, and especially since site/aiab uses sloop type, the resources are scaled down (less replicas, less storage requests), see
<a href="https://github.com/airshipit/treasuremap/tree/master/type/sloop/charts/osh-infra." target="_blank">
https://github.com/airshipit/treasuremap/tree/master/type/sloop/charts/osh-infra.</a> You should be able to deploy entire LMA (logging, monitoring, alerting) stack by adding this to full-site manifest<br>
</div>
<div><br>
</div>
<div><font face="Courier New"> - osh-infra-ingress-controller<br>
- osh-infra-ceph-config<br>
- osh-infra-radosgw<br>
- osh-infra-logging<br>
- osh-infra-monitoring<br>
- osh-infra-mariadb<br>
- osh-infra-dashboards</font><br>
<br>
</div>
<div>This will add things like promethius, elasticsearch, kibana, etc.</div>
<div><br>
</div>
<div>Part of the reasoning for not including these charts in the default AIAB site was to be able to limit resources to 20G of memory (so people can get it up and running in VMs on their laptops, and other resources constrained places).</div>
<div><br>
</div>
<div>I wonder if you could investigate a way we could run AIAB (airship in a bottle) with an option (say something like --with-lma) or similar.
<br>
</div>
<div><br>
</div>
<div>We are having community IRC meetings every Tuesday 9AM CST and it would be a good topic to discuss.</div>
<div><br>
</div>
<div>Kindly, Kaspars<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div style="font-family:"Times New Roman";color:rgb(0,0,0);font-size:16px">
<hr>
<div id="m_-372394373163320096gmail-m_2840745304031202398divRpF197709" style="direction:ltr"><font size="2" face="Tahoma" color="#000000"><b>From:</b> Loic Le Gal [<a href="mailto:l.legal.astellia@gmail.com" target="_blank">l.legal.astellia@gmail.com</a>]<br>
<b>Sent:</b> Wednesday, August 07, 2019 10:06 AM<br>
<b>To:</b> <a href="mailto:airship-discuss@lists.airshipit.org" target="_blank">airship-discuss@lists.airshipit.org</a><br>
<b>Subject:</b> Re: [Airship-discuss] Activating Prometheus (& other monitoring tools) for aiab<br>
</font><br>
</div>
<div></div>
<div>
<div dir="ltr">Hi,<br>
<div><br>
</div>
<div>Replying myself to close this thread.</div>
<div>I finally succeed in creating prometheus-0 and alerter-0 pods by adding also -osh-infra-monitoring in the chart_groups: section of the file site/aiab/manifests/bootstrap.yaml.</div>
<div>It seems to list the available charts, the deployment process looks in to match labels set on nodes.</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">Le mer. 7 août 2019 à 14:39, Loic Le Gal <<a href="mailto:l.legal.astellia@gmail.com" rel="noopener noreferrer" target="_blank">l.legal.astellia@gmail.com</a>> a écrit :<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Hi airshipers,
<div><br>
</div>
<div>I'd like to add Prometheus monitoring (+fluentbit, ...) to the new treasuremap/site/aiab; </div>
<div>It seems that some charts already exist but I'm not sure the integration is finished because nothing usefull found in seaworthy site def.</div>
<div> </div>
<div>Server/client labels do not exist on the Genesis node so I've added them to site/aiab/profiles/genesis.yaml and launch again the whole genesis process from scratch but no new pods appear. (I also tried adding prometheus-global=true and prometheus-type=true
labels)</div>
<div><br>
</div>
<div>I've already searched in Armada read-the-docs. </div>
<div>Even if the WIP in armada regarding automation of metrics collection is still a WIP [1), I just want to get prometheus pods (then add grafana, fluentbit, ...) </div>
<div><br>
</div>
<div>
<div>Do you know where I can find information to add/activate the monitoring layer (osh-infra-monitoring) or just add a Prometheus pod ?</div>
<br class="m_-372394373163320096gmail-m_2840745304031202398gmail-m_6755943800310498891gmail-Apple-interchange-newline">
</div>
<div>[1] <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__review.opendev.org_-23_c_668980_&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=L89wsG0LNn57q1mQMoKsdP6Ws59EGdCArXPqN62UF7Y&s=8-IWFHCVVTKqC2CvGawDzxq2wQDSw4xCVexRf0-EhAE&e=" rel="noopener noreferrer" target="_blank">https://review.opendev.org/#/c/668980/</a><br>
</div>
<div><br>
</div>
<div>BR, Loïc<br>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</blockquote></div>
</div></div>