Thanks again Kaspars,

Adding all these charts seem too huge even for a 32 GB RAM VM (I'll try again, in case the failure came from a latency of our infra).
Anyway, I succeed to add only Prometheus, AlertManager and Grafana by doing this:
A - add these chartgroups in aiab bootstrap manifest 
    - osh-infra-monitoring
    - osh-infra-ingress-controller
    - osh-infra-mariadb
    - osh-infra-dashboards

B - Overload the osh-infra-dashboard chartgroup to include only grafana (using deckhand overloading mechanism instead of brute removal )
  B1. add a label at the global layer to avoid Deckhand complains in B2
+++ b/global/software/charts/osh-infra/osh-infra-dashboards/chart-group.yaml
@@ -6,6 +6,8 @@ metadata:
   layeringDefinition:
     abstract: false
     layer: global
+  labels:
+    name: osh-infra-dashboards-global
   storagePolicy: cleartext
  
  B2. create a new file at the type layer that only contains grafana:
+++ b/type/sloop/charts/osh-infra/dashboards-chartgroup.yaml
@@ -0,0 +1,20 @@
+---
+schema: armada/ChartGroup/v1
+metadata:
+  schema: metadata/Document/v1
+  replacement: true
+  name: osh-infra-dashboards
+  layeringDefinition:
+    abstract: false
+    layer: type
+    parentSelector:
+      name: osh-infra-dashboards-global
+    actions:
+      - method: merge
+        path: .
+  storagePolicy: cleartext
+data:
+  # disable kibana by changing the chartgroup content
+  chart_group:
+    - grafana
+...
C - Expose the services:

kubectl expose pod/prometheus-0 -n osh-infra --type=NodePort --name=prometheus-dashboard

kubectl expose pod/alertmanager-0 -n osh-infra --type=NodePort --name=alertmgr-dashboard
kubectl expose service/grafana-dashboard -n osh-infra --type=NodePort --name=grafana-web

Not every Grafana Dashboards are operational in aiab but those that are are really good !

BR, Loïc

Le mer. 7 août 2019 à 17:27, SKELS, KASPARS <ks3019@att.com> a écrit :
Hi,

these charts are used in Airsloop/Seaworthy runs, so they should be tested, and especially since site/aiab uses sloop type, the resources are scaled down (less replicas, less storage requests), see https://github.com/airshipit/treasuremap/tree/master/type/sloop/charts/osh-infra. You should be able to deploy entire LMA (logging, monitoring, alerting) stack by adding this to full-site manifest

    - osh-infra-ingress-controller
    - osh-infra-ceph-config
    - osh-infra-radosgw
    - osh-infra-logging
    - osh-infra-monitoring
    - osh-infra-mariadb
    - osh-infra-dashboards

This will add things like promethius, elasticsearch, kibana, etc.

Part of the reasoning for not including these charts in the default AIAB site was to be able to limit resources to 20G of memory (so people can get it up and running in VMs on their laptops, and other resources constrained places).

I wonder if you could investigate a way we could run AIAB (airship in a bottle) with an option (say something like --with-lma) or similar.

We are having community IRC meetings every Tuesday 9AM CST and it would be a good topic to discuss.

Kindly, Kaspars




From: Loic Le Gal [l.legal.astellia@gmail.com]
Sent: Wednesday, August 07, 2019 10:06 AM
To: airship-discuss@lists.airshipit.org
Subject: Re: [Airship-discuss] Activating Prometheus (& other monitoring tools) for aiab

Hi,

Replying myself to close this thread.
I finally succeed in creating prometheus-0 and alerter-0 pods by adding also -osh-infra-monitoring in the chart_groups: section of the file site/aiab/manifests/bootstrap.yaml.
It seems to list the available charts, the deployment process looks in to match labels set on nodes.

Le mer. 7 août 2019 à 14:39, Loic Le Gal <l.legal.astellia@gmail.com> a écrit :
Hi airshipers, 

I'd like to add Prometheus monitoring (+fluentbit, ...) to the new treasuremap/site/aiab; 
It seems that some charts already exist but I'm not sure the integration is finished because nothing usefull found in seaworthy site def.
 
Server/client labels do not exist on the Genesis node so I've added them to site/aiab/profiles/genesis.yaml and launch again the whole genesis process from scratch but no new pods appear. (I also tried adding prometheus-global=true and prometheus-type=true labels)

I've already searched in Armada read-the-docs. 
Even if the WIP in armada regarding automation of metrics collection is still a WIP [1), I just want to get prometheus pods (then add grafana, fluentbit, ...) 

Do you know where I can find information to add/activate the monitoring layer (osh-infra-monitoring) or just add a Prometheus pod ?


BR, Loïc