How to Build a Data Quality Dashboard Your Stakeholders Will Actually Use

Most data quality dashboards are built in a weekend and abandoned within a month. Not because the team lacked effort or the tooling was wrong, but because the dashboard was designed for the person who built it — usually a data engineer — not for the people who needed to act on it.

A dashboard that shows Airflow DAG success rates and dbt test pass/fail counts is genuinely useful to a data engineer. It is meaningless to a compliance director preparing a quarterly submission. It is invisible to a department head who wants to know whether the coverage quality trends in their program data are moving in the right direction.

The result is a pattern that plays out in nearly every data organization that tries to build a data quality dashboard: engineers have one (usually in Datadog or a dbt docs page), directors have another (usually a manually-updated Confluence page or a spreadsheet someone emails on Fridays), and compliance officers have nothing — they call the engineering team two weeks before every submission and ask for a status report.

This guide solves that by doing something obvious that almost nobody does: designing the dashboard from the persona outward, not from the metrics inward. Three distinct views, three different sets of questions, three different update cadences. All drawing from the same underlying data quality infrastructure.

The Three Personas — and What Each One Actually Needs

Before building any dashboard, you need to answer the question that most teams skip: who is this for, and what decision does it help them make? A dashboard that tries to serve everyone simultaneously serves nobody well.

The Engineer View

Who: Data engineers, analytics engineers, pipeline owners Decision being made: Which pipeline needs my attention right now? Is there a data quality issue I need to fix before it reaches stakeholders? Update cadence: Real-time to hourly Key characteristics: High granularity, actionable, failure-first sorting, links to specific jobs and tests

Engineers need a triage surface. The dashboard is a queue: what broke, when, where, and how bad. Status-only views are useless to them. Every item needs to be actionable — a link to the failing job, the test that failed, the table affected.

The Director View

Who: Data directors, analytics managers, program directors Decision being made: Is our data quality getting better or worse over time? Are there systemic issues I need to escalate or resource? Update cadence: Daily to weekly Key characteristics: Trend lines, coverage rates, SLA compliance, program-level aggregation

Directors need a trend surface. They are not going to fix a failing dbt test themselves, but they need to know if dbt test failure rates have trended up 15% over the past quarter. They need to see coverage quality by program or data domain, not by individual pipeline. A director who opens a dashboard and sees 247 individual pass/fail checks has learned nothing.

The Compliance View

Who: Compliance officers, quality assurance leads, submission managers Decision being made: Are we ready to submit? What is the risk level of this submission? Where do we need to remediate before the deadline? Update cadence: Weekly to on-demand, with a spike in the two weeks before submission Key characteristics: Submission-readiness score, critical field completeness, edit check pass rates against the receiving agency’s published specs, open issues with owner and resolution date

Compliance leads need a readiness surface. They don’t care about infrastructure. They care about one thing: will this data pass when it hits CMS, CDC, or their state agency’s validation layer? They need to know the answer to that question with enough lead time to fix anything that won’t.

The Metrics: What Goes on Each View

Engineer View Metrics

Metric	Definition	Threshold / Alert
Pipeline health score	% of jobs completing within SLA window, last 24h	< 95% = yellow; < 90% = red
Test pass rate	% of dbt/validation tests passing, by domain	< 98% triggers investigation
Rows loaded vs. expected	Actual row count vs. 30-day rolling average per source	< 85% of average = alert
Data freshness	Time since most recent record per critical table	> 26h for daily feeds = alert
Null rate drift	Change in null rate for critical fields vs. 7-day baseline	> 5pp increase = flag
Failing tests queue	Open test failures sorted by severity + table criticality	Actionable list, linked to job
Reprocessing backlog	Count of records awaiting reprocessing or manual review	Should trend to zero daily

Director View Metrics

Metric	Definition	Visualization
Coverage quality score	% of records meeting all defined quality rules, by program/domain	Weekly trend line, 13-week rolling
SLA compliance rate	% of pipelines meeting their SLA, by month	Bar chart, monthly
Critical issue velocity	# of P1/P2 data quality issues opened vs. closed per week	Line chart (opened vs. closed)
Provider submission coverage	% of expected providers/sources that submitted in the period	Heat map by source
Time to resolution	Median hours from data quality issue detection to resolution	Weekly trend; target < 48h
Pre-submission remediation rate	% of detected issues resolved before submission deadline	% score; target > 95%

Compliance View Metrics

Metric	Definition	Format
Submission readiness score	Overall score against target agency edit specs	% score + red/yellow/green status
Critical field completeness	Null rate on required fields (NPI, MRN, diagnosis codes, dates)	% complete per field, table
Edit check pass rate	% of records passing the receiving agency’s published edit checks	Run against CMS/CDC/state specs
Open issues register	All open data quality issues, owner, due date, severity	Tracker table, sortable
Days to deadline	Countdown to next submission with status RAG rating	Single large KPI tile
Resubmission history	Count and reason for prior resubmissions by period	Table, last 8 periods

Dashboard Anti-Patterns That Kill Adoption

The failure modes of data quality dashboards are consistent enough to treat as a checklist. Before you build, check these off.

1. The everything dashboard. One view, 200 metrics, all personas addressed simultaneously. Engineers can’t find their failing jobs. Directors see nothing they can act on. Compliance leads stop opening it within three weeks. Fix: Three views, separated. A tab per persona at minimum; ideally a separate URL or access level so each persona sees only what they need.

2. Vanity metrics at the top. Leading the dashboard with “98.3% overall test pass rate” gives the impression of quality without revealing anything about risk. A single P1 issue in a federal submission table is more important than 1,000 passing tests on reference data nobody uses. Fix: Failure-first sorting, critical-domain weighting, risk-adjusted scores.

3. No ownership layer. A dashboard that shows 47 open data quality issues with no owner assigned is a dashboard that generates anxiety, not action. Within two weeks, the team stops looking at it because the open issues are never resolved and the number just grows. Fix: Every issue must have an owner. The compliance view’s open issues register is a live tracker, not a read-only report.

4. Static screenshot dashboards. A PDF or screenshot of a dashboard sent in a Friday email. By Monday, it’s already stale. By the time someone acts on it, the data has moved. Fix: Live dashboards, linked directly from the tools teams actually use (Slack, email, Jira). Automated alerting supplements the dashboard; the dashboard is not the alert mechanism.

5. Built for the builder. The dashboard reflects the schema of the underlying data quality tables rather than the questions stakeholders ask. Fix: Start with a user interview. Ask three stakeholders: “what is the one question you wish you could answer about our data quality right now?” Build those three answers first. Add everything else later.

6. No historical baseline. A dashboard that only shows current state tells you nothing about trend direction. Is 94% test pass rate good or bad? If it was 89% two months ago, it’s good. If it was 99%, you have a problem. Fix: Every metric on the director and compliance views should show a trend, minimum 13 weeks.

Implementation Guide

Building a three-persona data quality dashboard does not require a BI tool rebuild or a new data platform. The practical path depends on where your data quality metrics currently live.

If you have a dbt-based stack:

Your test results are already in dbt_test_results or the equivalent exposure in your data warehouse. Build a mart layer on top:

-- mart: data_quality.test_results_daily
SELECT
  DATE(executed_at) AS test_date,
  node_name AS model,
  test_name,
  status,  -- pass / fail / warn
  failures AS failing_row_count,
  rows_tested
FROM {{ ref('dbt_test_results') }}

From this mart, your engineer view queries by status = 'fail' sorted by executed_at DESC. Your director view aggregates SUM(CASE WHEN status = 'pass' THEN 1 ELSE 0 END) / COUNT(*) by domain, by week. Your compliance view filters by the tables tagged with your submission_critical: true tag.

If you use a validation platform (Vexdata, Great Expectations, Soda):

Most platforms expose a results API or write results to a target schema. Map their output to your three-persona mart layer using the same pattern above. The key is centralizing results regardless of where the validation ran.

Frontend options by team maturity:

Maturity Level	Tool	Best for
Early-stage / low BI tooling	Metabase, Redash	Fast to set up, open source, easy embedding
Mid-maturity	Tableau, Looker, Power BI	Already in use for other dashboards, persona tabs built natively
Platform-native	Vexdata pipeline health view	Purpose-built for data quality, no separate BI layer needed
Advanced / custom	React + your data warehouse API	Full control, integrates with your existing developer portal

Design principles for all tiers:

1. Mobile-readable titles. Your director will check this from their phone. If the headline metric isn’t readable at mobile size, nobody checks it.

2. Color only for status, not aesthetics. Red = action required. Yellow = watch. Green = clear. Never use red for branding.

3. Every metric links to its source. A director who sees “provider submission coverage: 87%” should be able to click through to which providers haven’t submitted.

4. An automated weekly digest. The dashboard is a pull surface; most stakeholders won’t open it unprompted. A weekly Monday morning digest sent to their inbox — three metrics, one RAG status, one open issue — drives consistent engagement and pre-submission awareness.

Conclusion

A data quality dashboard that nobody opens is worse than no dashboard at all — it creates the false impression that data quality is being managed while the actual work still happens in spreadsheets and frantic pre-submission emails.

The fix is straightforward: design from persona outward, not from metric inward. Three views, three different questions, three different cadences. Engineers get a triage queue. Directors get a trend surface. Compliance leads get a readiness score. All three views draw from the same underlying validation results.

The teams that have built this well share one characteristic: they started with a single, honest user interview per persona. “What is the one question you wish you could answer about our data right now?” The answer to that question becomes the headline KPI on that persona’s view. Everything else is secondary.

See what a Vexdata pipeline health dashboard looks like in practice — including the coverage quality trending view that the Massachusetts DPH data team uses before every federal submission. Request a demo or start with the Data Fitness Assessment to see where your current coverage stands.