Building a Grafana Dashboard for Chunk Health

A chunk-health dashboard answers one operational question at a glance — are chunks accumulating, compressing, and refreshing on schedule? — and it does that by pairing three stat tiles (chunk count, compression coverage, refresh lag) with four time-series panels whose thresholds turn a panel red before a human notices anything is wrong. This page lays out those panels, gives the exact PromQL each one needs, and sets threshold numbers you can copy for an IoT fleet. It assumes the metrics already reach Prometheus through the collector described in exporting TimescaleDB job metrics to Prometheus.

# Input Profiling: Which Series Each Panel Needs

Every panel on this dashboard reads a Prometheus series scraped from the TimescaleDB exporter. Before laying out a single panel, confirm the exporter is publishing the following gauges and counters — each is labelled by hypertable so one dashboard can serve an entire cluster through a template variable:

• timescaledb_hypertable_chunks_total — gauge, total chunk count per hypertable. Feeds the chunk-count tile and its trend panel.
• timescaledb_hypertable_compressed_chunks_total — gauge, count of chunks currently in the columnar store. Combined with the total, it yields compression coverage.
• timescaledb_compression_before_bytes and timescaledb_compression_after_bytes — two gauges holding pre- and post-compression byte totals. Their quotient is the compression ratio.
• timescaledb_cagg_watermark_seconds — gauge, the Unix-epoch timestamp of the newest fully materialized bucket in a continuous aggregate. Subtracted from time() it becomes refresh lag.
• timescaledb_job_total_failures and timescaledb_job_last_run_status — a counter and a status gauge (1 = success, 0 = failure) per background job. They drive the job-failures panel.

If any of these is missing from up-healthy targets, fix the exporter before building panels — a panel bound to a non-existent series renders “No data”, which reads identically to a genuinely idle system and hides the outage. The watermark and job series come directly from the job-metrics exporter; the byte gauges come from hypertable_compression_stats.

# Per-Panel Implementation

Each panel below names its Grafana panel type, the PromQL expression bound to it, and the threshold that flips it red. All expressions filter on $hypertable, the dashboard template variable, so the same JSON serves every table in the fleet.

# Stat tiles

The three tiles across the top are Grafana Stat panels — a single big number with a coloured background driven by threshold steps.

# Chunk count tile
timescaledb_hypertable_chunks_total{hypertable="$hypertable"}

# Compression coverage tile (percent)
100 * timescaledb_hypertable_compressed_chunks_total{hypertable="$hypertable"}
     / timescaledb_hypertable_chunks_total{hypertable="$hypertable"}

# Refresh lag tile (seconds; format the field unit as "duration")
time() - timescaledb_cagg_watermark_seconds{hypertable="$hypertable"}

On the coverage tile set the threshold steps so green is the high end (coverage should be near 100%): red below 60, amber 60–80, green above 80. On the refresh-lag tile the polarity flips — green below 300 s, amber 300–600 s, red above 600 s — because here a large number is the failure.

# Compression ratio (derived panel)

Ratio is not a stored series; you derive it from the two byte gauges. Given $B_{before}$ uncompressed bytes and $B_{after}$ compressed bytes for a hypertable, the ratio is

$$R_{comp} = \frac{B_{before}}{B_{after}}$$

so a ratio of $12$ means the columnar store holds the same data in one-twelfth the space. In PromQL that is a direct division, guarded so the panel does not divide by zero before any chunk has compressed:

timescaledb_compression_before_bytes{hypertable="$hypertable"}
  / (timescaledb_compression_after_bytes{hypertable="$hypertable"} > 0)

The > 0 acts as a filter: when after_bytes is zero the sample is dropped rather than producing +Inf, leaving a clean gap instead of a spike that wrecks the panel’s y-axis. Render this as a Time series panel and set a threshold line at your expected floor — an unexpectedly low ratio signals that segmentby/orderby ordering has drifted, which the companion guide on tracking compression ratio trends over time explores in depth.

# Refresh lag and job failures

Refresh lag reuses the tile expression as a Time series with a red threshold line, so you see not just the current lag but whether it is trending toward the ceiling. Job failures are best drawn as a Time series in bar mode over the delta of the failure counter:

# Refresh lag over time (seconds)
time() - timescaledb_cagg_watermark_seconds{hypertable="$hypertable"}

# New job failures per scrape window
increase(timescaledb_job_total_failures{hypertable="$hypertable"}[5m])

Using increase(...) rather than the raw counter means the panel shows new failures in each window and resets to zero cleanly, so a single old failure does not paint the panel red forever.

# Panel-to-threshold reference

Grafana’s own alert rules can attach to these same queries, so the panel colour and the alert fire from one source of truth rather than two subtly different expressions.

# Worked Example: An IoT Fleet

Take a fleet of 50,000 devices writing to a sensor_readings hypertable on a 30-minute chunk interval, with a compression policy at 7 days and a 5-minute continuous aggregate. At steady state that table holds roughly 1,200 chunks, of which the newest 7 days (~336 chunks) stay uncompressed and the rest compress — so compression coverage sits near 88% and anything below 80% means the compression job is falling behind ingestion.

The compression ratio for this workload — narrow numeric telemetry with good segmentby locality — lands around 12:1, so a threshold at 8 flags a real regression without firing on normal variance. Refresh lag should stay under one aggregate cycle plus a margin: with a 5-minute refresh, a 600-second ceiling catches a genuinely stuck job while tolerating one skipped run. Chunk count grows predictably at roughly 48 new chunks per day; an alert on the count rate exceeding twice that baseline catches a runaway backfill or a retention job that stopped dropping old chunks. Job failures use the strictest threshold of all — any value above zero in a 5-minute window is worth a look, because a healthy fleet produces exactly none.

Reading the tiles together tells a story a single metric cannot: coverage at 88%, ratio at 12, lag at 20 seconds, zero failures is a healthy fleet; coverage sliding to 70% while lag climbs past 600 s points at overloaded background workers, not a compression bug.

# Edge Cases & When to Deviate

• Exporter down. When the exporter or its target goes away, Prometheus stops receiving samples and every panel shows “No data” — indistinguishable from idle. Add a companion panel or alert on up{job="timescaledb"} == 0 so the dashboard reports its own blindness rather than pretending all is well. Set panel “No value” mapping to render as a distinct state, not zero.
• Ratio undefined before first compression. On a freshly created hypertable, after_bytes is zero until the first chunk compresses, and before / after would be +Inf. The (after_bytes > 0) guard drops those samples so the ratio panel simply starts drawing once real compression has happened, instead of opening with a meaningless infinity spike.
• Counter resets on restart. timescaledb_job_total_failures is a counter that resets to zero when the database restarts. Always wrap it in increase() or rate(), which are reset-aware, rather than graphing the raw value — otherwise a restart looks like failures being magically “repaired”.
• Per-hypertable variable. A database with many hypertables should not need one dashboard each. Define a Grafana template variable $hypertable from label_values(timescaledb_hypertable_chunks_total, hypertable) and interpolate it into every query. Add an All option only if your panels use aggregating functions such as sum by (hypertable), otherwise the tiles will silently sum unrelated tables.
• Sparse or paused fleets. A hypertable that legitimately stops receiving data will show a frozen watermark and rising lag. If pauses are expected (maintenance windows, seasonal sensors), gate the lag alert behind an ingestion check so a deliberately quiet table does not page anyone.

# Verification

Confirm each panel reads live data before you trust it in an incident. Open the dashboard against a known-healthy table and check that every panel resolves to a real number rather than “No data”. Then verify each series independently from the Prometheus expression browser:

# Every hypertable the exporter knows about should return a series here.
timescaledb_hypertable_chunks_total

# Coverage should be a plausible 0-100 value, never NaN.
100 * timescaledb_hypertable_compressed_chunks_total
     / timescaledb_hypertable_chunks_total

# Lag should be small and positive on a healthy cagg (seconds since last bucket).
time() - timescaledb_cagg_watermark_seconds

Cross-check the numbers against the database itself so the dashboard and the catalog agree:

-- Chunk count and compression coverage straight from the catalog.
SELECT
    hypertable_name,
    count(*)                                         AS chunks_total,
    count(*) FILTER (WHERE is_compressed)            AS chunks_compressed,
    round(100.0 * count(*) FILTER (WHERE is_compressed) / count(*), 1) AS coverage_pct
FROM timescaledb_information.chunks
GROUP BY hypertable_name
ORDER BY hypertable_name;

-- Compression ratio inputs, to match the derived panel.
SELECT
    hypertable_name,
    pg_size_pretty(before_compression_total_bytes)   AS before,
    pg_size_pretty(after_compression_total_bytes)    AS after,
    round(before_compression_total_bytes::numeric
          / nullif(after_compression_total_bytes, 0), 1) AS ratio
FROM hypertable_compression_stats('sensor_readings');

If a panel’s number and the SQL disagree by more than a scrape interval’s worth of drift, the exporter query is stale or mislabelled — fix it there, not by patching the dashboard. Once the tile numbers track the catalog and the thresholds fire on a deliberately induced lag (pause the refresh job and watch the tile go red), the dashboard is trustworthy.

# Related

• Prometheus & Grafana Integration — the parent guide on wiring TimescaleDB metrics into a Prometheus and Grafana stack
• Exporting TimescaleDB Job Metrics to Prometheus — where the chunk, job, and watermark series on this dashboard come from
• Tracking Compression Ratio Trends Over Time — going deeper on the compression-ratio panel and its long-run drift
• Monitoring, Observability & Alerting for TimescaleDB Automation — the broader monitoring program these dashboards fit into

← Back to Prometheus & Grafana Integration