Zero ETL Doesn’t Mean Zero Data Quality Problems
The Zero ETL narrative is compelling: eliminate the extract-transform-load pipeline layer, replicate data directly from source to destination in near-real-time, reduce latency, remove a maintenance burden, and let your data engineers focus on analytics rather than plumbing. AWS, Snowflake, Databricks, and Google are all racing to offer it.
The pitch is not wrong. Zero ETL genuinely removes friction from certain categories of data movement. But the marketing consistently conflates “remove the ETL pipeline” with “remove the data quality problem,” and those are not the same thing.
The ETL layer, despite its costs, did one thing that Zero ETL architectures do not replace by default: it was a natural checkpoint. Data moved through a defined transformation process, and if you were disciplined about it, you inserted validation logic into that transformation. Bad data was caught — or at least visible — before it landed in your warehouse.
When you remove the ETL layer, you remove the checkpoint. The data still moves. The quality problems don’t disappear — they relocate upstream, arrive faster, and are harder to catch because there’s no single inspection point anymore.
This article is for data architects and engineers evaluating Zero ETL adoption. It is not an argument against Zero ETL. It is an argument for understanding exactly what you are trading away in a Zero ETL architecture, and what you need to put in place instead.
What Zero ETL Actually Is
Zero ETL refers to a class of data integration architectures that replicate data directly from a source system to a destination (typically a cloud data warehouse) without requiring a separately managed ETL pipeline. The key implementations:
AWS Aurora Zero-ETL to Redshift: Transactional data from Aurora MySQL or PostgreSQL replicates directly into Redshift in near-real-time, without Glue jobs or custom connectors.
Snowflake Streaming Ingest / Auto-Ingest: Event data from Kafka or S3 streams directly into Snowflake tables with sub-minute latency.
Databricks Delta Sharing / Unity Catalog: Data shares between Databricks workspaces or to external consumers without copying the underlying files.
Google BigQuery Omni / Dataplex: Query across data stores without moving the data at all.
What changes in each of these architectures: the transformation step is either eliminated entirely or deferred to the destination (ELT rather than ETL). Data lands in the warehouse in its source format, and analysts or dbt models run transformations after the data arrives.
This is the architectural shift that carries the data quality implication. In a traditional ETL architecture:
Extract from source
Transform (clean, standardize, validate, enforce business rules)
Load to warehouse
In a Zero ETL architecture:
Replicate from source to warehouse (fast, automated, no transformation)
Transform in warehouse (dbt models, SQL, Spark)
Step 2 in Zero ETL is not the same as Step 2 in traditional ETL. Transformation-in-warehouse is primarily about business logic and data modeling, not data quality enforcement. The validation layer that existed in the ETL pipeline — whether explicit (Great Expectations checks, custom validation scripts) or implicit (transformation code that would fail on bad input) — no longer exists at the point of ingestion.
Data quality problems in Zero ETL arrive in your warehouse faster, at higher volume, and without a single inspection point.
5 Data Quality Problems Zero ETL Creates or Makes Worse
1. Bad Data Arrives Faster
In a traditional ETL pipeline, there is an inherent latency — typically minutes to hours — between when bad data enters your source system and when it reaches your warehouse. That latency is a detection window. An alert system monitoring the ETL layer has time to catch a problem before it propagates.
Zero ETL eliminates that latency. AWS Aurora Zero-ETL achieves sub-second replication. When your source system generates a bad record — a null patient ID, a future date, a diagnosis code that doesn’t exist in the current ICD-10 specification — that record is in your warehouse before any scheduled validation job can run.
In a high-throughput environment (tens of thousands of records per minute), a data quality issue at the source can pollute millions of warehouse records before it’s detected. The pipeline didn’t create the problem. Zero ETL just removed the speed bump that would have caught it.
What to do: Implement real-time streaming validation at the point of replication, not after. AWS EventBridge Pipes, Snowflake Dynamic Tables with quality assertions, or a Kafka Streams validation layer before the warehouse connector are all viable approaches. The key is that validation must be concurrent with replication, not downstream of it.
2. Source System Changes Are Invisible Until They Break Something
Traditional ETL pipelines have an inherent schema enforcement mechanism: if a source field changes in a way that breaks the transformation, the job fails. The failure is noisy, but it’s visible.
Zero ETL replication is schema-flexible by design. AWS Aurora Zero-ETL handles column additions and type widening automatically. Snowflake Schema Evolution picks up new columns without intervention. This is marketed as a feature. For data quality, it is also a risk.
When a source developer renames a column, adds a new required field, or changes a date format from YYYY-MM-DD to MM/DD/YYYY, Zero ETL often handles the change gracefully at the replication layer. The data still arrives. Your downstream dbt models and reports may silently consume the changed data without a failure — until a business user asks why their revenue metric is wrong.
What to do: Implement schema registry monitoring. Track column-level metadata (name, type, null rate, cardinality) on every replication cycle. Alert on unexpected changes. Tools like dbt schema tests, Soda, or Vexdata’s schema drift detection can be applied post-replication to catch changes before they propagate to reporting models.
3. No Single Place to Enforce Business Rules
In ETL, business rules lived in the transform step. A rule like “claim amounts must be positive” or “patient age must be between 0 and 130” was encoded in the transformation logic. When a record violated the rule, the transform caught it.
In Zero ETL, the transformation step at ingestion doesn’t exist. Business rules must now be enforced either at the source system (which you usually don’t control) or in the warehouse after replication (which catches the problem after it’s already landed, not before).
This is not fatal — post-landing validation is better than no validation — but it changes the cost model. In ETL, a bad record that fails a business rule is excluded or quarantined before landing. In Zero ETL, a bad record lands in the warehouse and propagates into every downstream model that runs before the post-landing validation catches it.
What to do: Build a quarantine layer. Instead of validating directly against your consumption tables, replicate into a raw/staging schema, run business rule validation against staging, and promote only passing records to your consumption layer. This adds latency back into Zero ETL, but it restores the enforcement checkpoint.
4. Deduplication Is No Longer Free
ETL pipelines typically include deduplication logic — a merge/upsert step that ensures a source record doesn’t arrive twice. In Zero ETL, especially for event-based streaming architectures, at-least-once delivery semantics are common. Duplicates are expected and handled at the destination.
If your destination doesn’t have explicit deduplication configured — and many default Zero ETL setups do not — you will accumulate duplicates in your warehouse. For operational data, duplicates cause double-counting. For clinical or claims data, duplicates create patient-level distortions that affect every downstream analysis.
What to do: Explicitly configure deduplication at the destination. For Snowflake, use QUALIFY ROW_NUMBER() OVER (PARTITION BY record_id ORDER BY loaded_at DESC) = 1 in your staging view. For Redshift, use MERGE with an explicit unique key. For Databricks Delta, use MERGE INTO ... WHEN MATCHED THEN UPDATE. Never assume Zero ETL deduplicates by default.
5. Lineage Becomes Opaque
In ETL pipelines, lineage was often implicit in the pipeline code: you could read the transformation to understand where a field came from and what rules it passed through. In Zero ETL, the path from source to warehouse is a black box managed by the platform.
When a downstream report shows an unexpected number, tracing it back to the source requires querying the platform’s replication logs (when available) and cross-referencing with your warehouse audit tables. The lineage that existed in your ETL code is now distributed across platform-managed metadata that may not be easily queryable or visible to your data team.
What to do: Instrument lineage at the warehouse layer using a dedicated lineage tool (dbt’s lineage graph, OpenLineage, Datahub, or Atlan) that tracks which consumption tables were built from which replication sources. Accept that the source-to-warehouse leg of lineage will be platform-managed and document it separately from your warehouse-internal lineage.
What Validation Looks Like in a Zero ETL World
Validation in a Zero ETL architecture cannot live in the pipeline layer, because there is no pipeline layer. It must be rebuilt in three new locations:
Location 1: Pre-replication (at the source) This is the highest-value placement but the hardest to implement, because it requires either writing validation logic into the source system or deploying a change data capture (CDC) stream processor that sits between the source and the replication mechanism. Teams with Kafka-based Zero ETL can insert a Kafka Streams or Flink job that validates records before they reach the warehouse connector. Teams using AWS Aurora Zero-ETL have fewer native hooks — Lambda triggers on the source Aurora instance can fire on record creation, but at the cost of source system overhead.
Location 2: Post-replication, pre-consumption (staging schema validation) The most practical location for most teams. Configure your Zero ETL replication to land in a raw/staging schema rather than directly in your consumption layer. Run a validation job against the staging schema — this can be dbt tests, Great Expectations, Soda, or a Vexdata validation pipeline — before promoting records to the consumption schema. This adds latency (typically 2–10 minutes depending on validation complexity and data volume) but restores the inspection checkpoint.
-- Example: conditional promotion from staging to consumption
INSERT INTO consumption.claims
SELECT *
FROM staging.claims_raw
WHERE
claim_id IS NOT NULL
AND patient_id IS NOT NULL
AND claim_amount > 0
AND service_date <= CURRENT_DATE
AND service_date >= '2020-01-01'
AND provider_npi ~ '^[0-9]{10}$' -- NPI format validation
AND icd10_code IN (SELECT code FROM reference.valid_icd10_codes)
;Location 3: Continuous monitoring at the consumption layer Even with pre-consumption validation, distribution drift, lineage breaks, and aggregate anomalies require ongoing monitoring at the consumption layer. This is the data observability layer described in the previous blog — volume baselines, freshness checks, null rate tracking — running continuously against the tables that power your reports.
The critical insight: in Zero ETL, validation is not lighter than in traditional ETL. It is more important, because the natural inspection checkpoint has been removed. Teams that adopt Zero ETL and reduce their validation investment on the assumption that simpler architecture means fewer problems will discover the opposite: faster pipelines surface data quality problems faster, to more stakeholders, with less time to fix them before executives notice.
Conclusion
Zero ETL is a genuine architectural improvement for teams that need low-latency data movement and are willing to manage the data quality implications thoughtfully. The latency reduction is real. The operational simplicity is real. The elimination of the checkpoint is also real.
The data engineering teams that will succeed with Zero ETL are the ones that treat the removal of the ETL layer as a reason to invest more in validation, not less. Faster data movement and higher validation rigor are not in tension — they are requirements that coexist in any Zero ETL architecture that you trust for production reporting.
The five problems above — faster bad data arrival, invisible schema changes, no business rule enforcement point, deduplication gaps, and opaque lineage — each have solutions. None of them require abandoning Zero ETL. They require building the validation layer that Zero ETL moves but does not eliminate.
If your team is evaluating Zero ETL or has already adopted it, see how Vexdata’s validation layer integrates with streaming and Zero ETL architectures — including pre-consumption staging validation and real-time distribution monitoring that runs alongside your replication layer.
Comments