Building observability dashboards that actually get used
Most observability dashboards become wallpaper. Here's how to build ones that drive decisions — what to put on them, what to cut, and which widgets matter.
Every team has at least one dashboard nobody looks at. It was important the week it was built, then it quietly became wallpaper — a wall of green numbers that no one checks until something is already on fire. The problem usually isn't the data; it's that the dashboard was built to display metrics instead of to answer questions. Good observability dashboards earn their place on a second monitor because they help you decide something. This post is about building those: what to put on one, what to cut, and how to keep it honest.
Start with a question, not a metric
A dashboard is a tool for answering a recurring question fast. So write the question first: "Is the checkout flow healthy right now?" or "Did last night's deploy make anything worse?" Then add only the widgets that answer it. If a chart doesn't change a decision you'd make, it's decoration. This single discipline kills more dashboard clutter than any design system. When you find yourself adding a metric "because we have it," stop and ask what you'd do differently if the number doubled.
Three numbers beat thirty
A dashboard with thirty panels has no signal — your eye has nowhere to land. Lead with the two or three numbers that actually mean "things are okay" for the system this dashboard covers:
- Error rate — new and total issues over the window.
- Latency — a high percentile (p95/p99), not the average, which hides the pain.
- Throughput — so a drop in errors caused by a drop in traffic doesn't fool you.
Everything else is a drill-down you reach for after one of those three looks wrong.
Match the dashboard to the audience
The dashboard an on-call engineer needs at 2 a.m. is not the one you show in a Monday review. Build for the reader:
- On-call wants "what's broken and where" — recent error spikes, the worst endpoints, links straight to the offending issue.
- A team lead wants trends — is reliability improving week over week, are SLOs holding.
- A whole-org view wants the rollup — health per project, not per query.
One generic dashboard tries to serve all three and serves none. Build a few focused ones instead.
Make it span the whole request, not just errors
Errors tell you something broke; they don't always tell you why. The dashboards that get used connect the dots — error rate next to slow queries, next to the deploy markers, next to throughput. When the error line jumps at the same minute as a release, you've found your culprit without opening a single log. Lookout's custom dashboards let you compose widgets across watchers — errors, performance, queries, releases — with a widget and query builder, so one view can show the spike and the likely cause side by side.
Build it, then prune it
Dashboards rot. A panel that mattered during a migration is noise three months later. Put a recurring 15 minutes on the calendar to delete widgets nobody used and promote the ones people keep drilling into. A dashboard you trim is a dashboard you trust — and the act of pruning forces you to re-ask the original question. If you can't remember why a panel is there, that's your answer.
The short version
- Write the question before you add the widget.
- Lead with two or three numbers, not thirty.
- Build a focused dashboard per audience, not one for everyone.
- Put errors next to their likely causes (queries, latency, deploys).
- Prune regularly so signal stays signal.
A dashboard that answers a real question gets opened every day; one that merely displays data gets ignored by Friday. Compose yours around decisions, keep it lean, and connect errors to their causes. Lookout's custom dashboards make that composition quick, and the free Starter tier (10,000 events/month, one project, no credit card) is plenty to build your first useful one. Start free and build a dashboard you'll actually open.