Add prometheus scraping and grafana support to charmed katib-controller operator (#1839)

* Add prometheus scraping and grafana support to charmed operator

* Upgrade black version to 22.3.0 to fix issue with click dependency

* fix: unpin `black`, fix formatting errors

* fix: minor refactor of prometheus integration

Revert to defaults for relation names and paths, where appropriate.

* fix: apply operator linting checks only to source code

* chore: point katib-db to charm in charmhub

* fix: remove unneeded handling of prometheus relation event

* feat: Add template dashboard and alert rules

These are not working properly.  When connecting to grafana, the dashboard shows up but does not populate properly with data.  The data source appears wrong

* fix: handle leader-elected events

Without this, upgrade-charm does not work.

* fix: correctly template the sample grafana dashboard

* fix: remove placeholder grafana/prometheus files

* fix: bump wait time to avoid flaky test failure

Co-authored-by: Andrew Scribner <[email protected]>

Author: jardon

.github/workflows/test-charmed-katib.yaml

@@ -21,10 +21,10 @@ jobs:
           sudo pip3 install black flake8
 
       - name: Check black
-        run: black --check operators
+        run: black --check operators/*/src
 
       - name: Check flake8
-        run: cd operators && flake8
+        run: cd operators && flake8 ./katib*/src
 
   build:
     name: Test
@@ -73,7 +73,7 @@ jobs:
           sg microk8s -c 'juju bootstrap microk8s uk8s'
           juju add-model kubeflow
           juju bundle deploy --build --destructive-mode --serial
-          juju wait -wvt 300
+          juju wait -wvt 600
 
       - name: Test Katib
         run: kubectl apply -f examples/v1beta1/hp-tuning/random.yaml

operators/bundle.yaml

@@ -1,7 +1,7 @@
 bundle: kubernetes
 applications:
   katib-controller: { charm: katib-controller,            scale: 1 }
-  katib-db:         { charm: cs:~charmed-osm/mariadb-k8s, scale: 1, options: { database: katib } }
+  katib-db:         { charm: charmed-osm-mariadb-k8s,     scale: 1, options: { database: katib } }
   katib-db-manager: { charm: katib-db-manager,            scale: 1 }
   katib-ui:         { charm: katib-ui,                    scale: 1 }
 relations: