k3s: Pod-Monitoring in Grafana (Task 6.5)

kube-state-metrics (NodePort 30808) und ServiceAccount prometheus-infra (nur nodes/metrics) im Cluster; Prometheus auf infra liest kubelet/cAdvisor per TLS gegen die k3s-CA. Ziele per file_sd aus kubectl get nodes, cAdvisor auf die nötigen Reihen begrenzt. Dashboard "k3s – Pods" (Vorgabe nex), Alerts K3s* nur mit notify. Playbook monitoring-k3s.yml. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 16:31:03 +02:00
parent abf4a75063
commit 4ac251715e
17 changed files with 1271 additions and 13 deletions
@@ -66,3 +66,39 @@ scrape_configs:
      - {source_labels: [__address__], target_label: __param_target}
      - {source_labels: [__param_target], target_label: instance}
      - {target_label: __address__, replacement: "pve-exporter:9221"}
+
+  # k3s (Task 6.5): kubelet direkt auf jedem Knoten (10250), TLS gegen die k3s-CA, Token des ServiceAccounts
+  # prometheus-infra (nur nodes/metrics). Ziele per file_sd aus `kubectl get nodes` (monitoring-targets.py auf ops).
+  # infra hat 4 GB RAM: von kubelet und cAdvisor nur die Reihen, die Dashboard „k3s – Pods“ und Alerts brauchen.
+  - job_name: kubelet
+    scheme: https
+    authorization: {credentials_file: /etc/knaster-k3s/token}
+    tls_config: {ca_file: /etc/knaster-k3s/ca.crt}
+    file_sd_configs: [{files: [/etc/knaster-sd/kubelet.json]}]
+    metric_relabel_configs:
+      - source_labels: [__name__]
+        regex: kubelet_running_pods|kubelet_running_containers|kubelet_volume_stats_(used|capacity)_bytes
+        action: keep
+
+  - job_name: cadvisor
+    scheme: https
+    metrics_path: /metrics/cadvisor
+    authorization: {credentials_file: /etc/knaster-k3s/token}
+    tls_config: {ca_file: /etc/knaster-k3s/ca.crt}
+    file_sd_configs: [{files: [/etc/knaster-sd/kubelet.json]}]
+    metric_relabel_configs:
+      - source_labels: [__name__]
+        regex: container_cpu_usage_seconds_total|container_memory_working_set_bytes|container_memory_rss|container_oom_events_total|container_network_(receive|transmit)_bytes_total
+        action: keep
+      # nur Pods (keine systemd-Slices, keine Docker-Container der Compose-Stacks)
+      - source_labels: [pod]
+        regex: ""
+        action: drop
+      # CPU/Speicher je Container: Pod-Summe und Pause-Container verwerfen (Netz gibt es nur je Pod)
+      - source_labels: [__name__, container]
+        regex: "container_(cpu|memory|oom)[^;]*;(POD)?"
+        action: drop
+
+  - job_name: kube-state-metrics
+    honor_labels: true   # namespace/pod/node kommen aus den Metriken selbst
+    file_sd_configs: [{files: [/etc/knaster-sd/kube-state-metrics.json]}]