0011 — Self-hosted Grafana LGTM monitoring¶
Status: reconstructed — needs confirmation
Context¶
The platform needs metrics, logs, and (eventually) traces without a SaaS observability bill, on the same single VPS.
Decision¶
monitoring/docker-compose.yml runs Grafana 12, Prometheus 3, Loki 3,
Tempo 2, Grafana Alloy, node-exporter, and cAdvisor — all memory-capped for
the 1-vCPU box, none with published ports. Grafana is reached only through
Caddy at monitor.aimgold.org. Alloy tails every container's logs into
Loki; Prometheus scrapes the NestJS apps' prom-client metrics at
/v1/metrics (libs/common/src/metrics/). Dashboards and datasources are
file-provisioned from git (monitoring/grafana/).
Consequences¶
- Monitoring config is code-reviewed and redeployable
(
.github/workflows/deploy-monitoring.yml); dashboard edits made in the Grafana UI are not persisted to git. - Metric retention 30 d / 10 GB, logs 14 d, traces 7 d — bounded by VPS disk.
- Tempo receives OTLP but the deployed apps don't ship traces yet.