Prometheus and Grafana for Next.js 15, Tracking LLM Token Burn

If you only log token counts to stdout, you will not notice a bad loop until the invoice arrives. Prometheus scrapes a number every 15 seconds. Grafana draws it. That is enough to see burn, latency, and whether a deploy made chat slower.

AgenticPacks OSS has no metrics stack: Agentic-Packs/agenticpacks-oss. Pro ships Prometheus, Grafana, /api/metrics, and dashboards for token counts and USD. Ports 9090 and 3005 in Compose. This is observability for the sold starter, not a third-party analytics script on a marketing site.

The problem

An AI SaaS has two bills. The VPS is small and flat. The model vendor is usage. Founders watch the VPS and ignore the model until something in LangGraph retries a tool too often.

You want at least:

• Prompt tokens and completion tokens per model
• Estimated USD (your own rates, not the vendor dashboard delay)
• Request duration for /api/chat and /api/agent
• Process RSS, so you see a leak before OOM. See next build SIGKILL for compile-time memory. This post is runtime.

Expose metrics from Node, not Edge

prom-client needs a process that lives. A serverless isolate that dies after the request cannot scrape well. This is the same long-lived node server.js as the self-hosting post.

import client from "prom-client";
import { NextResponse } from "next/server";

export const runtime = "nodejs";

const register = new client.Registry();
client.collectDefaultMetrics({ register });

export const tokenCounter = new client.Counter({
  name: "llm_tokens_total",
  help: "LLM tokens by model and type",
  labelNames: ["model", "kind"],
  registers: [register],
});

export async function GET() {
  return new NextResponse(await register.metrics(), {
    headers: { "Content-Type": register.contentType },
  });
}

After each chat or graph run, increment:

tokenCounter.inc({ model, kind: "prompt" }, promptTokens);
tokenCounter.inc({ model, kind: "completion" }, completionTokens);

Do not put this route on the public internet without a network policy. In Compose, Prometheus scrapes web:3000 on the internal network. Caddy does not need to publish /api/metrics.

Compose scrape config

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: next
    static_configs:
      - targets: ["web:3000"]
    metrics_path: /api/metrics

Grafana points at http://prometheus:9090. Pro keeps dashboards as JSON under grafana/dashboards/ so a new container is not an empty UI.

A second Compose file for monitoring is fine. You do not want Grafana on port 3005 on a public NIC without auth. Bind it to localhost or put basic auth in front. Caddy can do that on a separate hostname if you need it. Caddy itself is Pro.

What to graph first

If USD in Grafana disagrees with Stripe, trust Stripe for money and Grafana for shape. The ledger is the billing source. Grafana is how you see a spike at 2am.

Tired of discovering token burn on the vendor invoice?

AgenticPacks OSS is the app without Prometheus: Agentic-Packs/agenticpacks-oss.

Prometheus, Grafana, and the token dashboards are in Prometheus token dashboards in Pro.

What this is not

It is not a replacement for Stripe. It is not host metrics for the kernel (you can add node_exporter later). It is the minimum so an AI SaaS can see model spend next to request latency.

---

Where to Go From Here

You do not have to buy a hosted APM to watch token burn.

Option A: Free open-source baseline (MIT)

• GitHub: github.com/Agentic-Packs/agenticpacks-oss
• What is included: Next.js 15, Compose, Postgres, Auth.js, streaming chat, single-plan Stripe. No Prometheus, no Grafana.
• Cost: free, MIT.

---

Option B: Ship with AgenticPacks Pro ($149 one-time)

If you want scrape, dashboards, and the ledger together:

• Automated Hetzner provisioning: Terraform, Caddy, Compose, monitoring stack.
• Production multi-agent core: LangGraph, pgvector, Redis.
• Usage & Unit Economics: Stripe, team pools, token-to-dollar ledger, Grafana panels.