ElixirData Blog Best Data Quality Software for Ingestion Checks (2026)

In brief

The best data quality software for ingestion checks is the one that can stop or divert a bad load before anyone downstream sees it, and tell the right owner why. For the hard rules (schema, duplicates, required fields, accepted values), that usually means a test layer inside the pipeline: dbt tests, Soda contracts, Great Expectations or pipeline-native expectations such as Databricks’. For the surprises nobody wrote a rule for (a feed that’s late, a table that’s half its usual size), add an observability platform. Enterprise suites add a central rule library and lineage.

This guide is for data engineers, platform owners and data leaders choosing how to validate data at the point it lands. It compares categories of tools, names vendors only where their own pages back the point, and ends with a short test you can run in a pilot.

Disclosure: ElixirData sells one of the options in the wider stack, Context OS. It is not an ingestion check tool, and we say below where it fits and where it doesn’t. We read vendor pages on 9 November 2026 and tested nothing, so treat this as a reading of documentation, not a benchmark.

What should an ingestion check actually catch?

An ingestion check runs when data arrives, before reports, models or AI agents rely on it. Seven things come up most often:

• Schema: the columns and data types are the ones you agreed to.
• Volume: the row count is in the range you expect.
• Freshness: the newest record is recent enough.
• Duplicates: keys that should be unique are unique.
• Referential checks: a value that points at another table exists there.
• Rule checks: required fields are filled and values sit inside allowed ranges or lists.
• Lineage: when something fails, you can see which upstream source it came from and which dashboards, models or agents depend on it.

Then there’s what happens to the failure: block the load, quarantine the bad rows in a holding area, or alert and carry on. Freshness and volume deserve special attention because a missing feed produces no failing row for a rule to catch. Something has to watch the clock.

How do the categories of tool compare?

The table groups tools by how they work. “Not described” means the pages we read didn’t say, not that the tool can’t do it.

What do the vendors say, according to their own sites?

dbt. According to its documentation, dbt has four built-in generic tests: unique, not_null, accepted_values and relationships (the referential check). Each test can be set to warn or error severity, and dbt can save failing rows to a table, which is the nearest thing to a quarantine table in its documentation [1]. Source freshness compares the newest value in a timestamp column you name against warn and error thresholds [2]. The tests page doesn’t describe a native row-count test, so a volume rule is something you write yourself [1].

Great Expectations. According to its documentation, GX Core is a Python library where you write Expectations (assertions about data), group them into suites and run them with Checkpoints, which can trigger actions such as email, Slack or Microsoft Teams alerts. It also produces readable Data Docs of results [3]. Its public gallery has entries named ExpectTableRowCountToBeBetween, ExpectColumnValuesToBeUnique, ExpectColumnValuesToBeBetween and ExpectTableColumnsToMatchSet, but the pages we could open didn’t print descriptions, so check what each one does before relying on it [4].

Soda. According to its documentation, a contract verifies the schema (columns and types) and data checks such as missing values, duplicates, invalid values and custom rules [5]. In Python, a failed verification can raise an exception that stops a pipeline run [6]. Soda’s April 2026 blog describes three responses: block for violations that would cause silent downstream errors, quarantine to a holding table for threshold breaches on non-critical fields, and alert-only for things like freshness breaches [7]. The pages we read didn’t name row-count or failed-row capture checks.

Databricks. According to its documentation, an expectation is a rule on a pipeline dataset with three actions: warn (keep the row and record the metric), drop (remove the row) or fail (stop the update) [8]. A separate page shows a quarantine pattern that flags each row and routes valid and invalid rows to different views, plus patterns for count comparison, primary-key uniqueness and missing records [9]. One caveat on that page: a validation table doesn’t block downstream tables by itself, so you have to arrange the dependency [9]. These are pipeline features, so they apply to Databricks pipelines, not to other platforms.

Monte Carlo. According to its site, it generates baseline monitors for freshness, volume and schema, offers field-level lineage that maps tables to the dashboards, models and agents that depend on them, and routes alerts by domain with related incidents grouped through lineage [10]. The page doesn’t say in general whether checks run before or after load, though it says that for Salesforce and Data Cloud issues can be caught at the source [10]. Treat it as watching data once it exists unless a demo shows otherwise.

Informatica. According to its site, observability tracks freshness, volume, distribution and schema changes, and the platform can autogenerate common rules and profile data continuously [11]. The page we read didn’t address ingestion-time checking or exception handling.

Ataccama. According to its site, quality rules sit in a central library with versioning and approvals, checks can be embedded in pipelines, data can be validated at entry through REST or GraphQL APIs, and lineage shows data from source to consumption with quality insights. It also lists instant alerts and a remediation list of issues to share with data owners [12]. Quarantine isn’t mentioned.

Should a failed check block, quarantine or only alert?

Decide this per dataset before you pick a tool. Soda’s rule of thumb is useful: if a violation would cause silent errors downstream rather than obvious ones, block [7]. Money movements, regulatory feeds and anything an AI agent acts on usually belong there. Quarantine fits when a partial load is acceptable and the bad rows can be fixed later. Alert-only fits soft thresholds and freshness checks where a late feed is better than no feed.

Two habits help. First, start new rules in warn mode. Soda suggests two to four weeks to see how often a check fires before it blocks anything [7]. Second, give every quarantine an owner and a time limit. A holding table nobody reads is just a slower way of losing data.

For each failure, keep the rule that fired, the rule version, the record or a pointer to it, the time, and who resolved it. That record is what makes lineage and alerts useful: the alert says what broke, the lineage says what it touches, and the record says who dealt with it.

Where does ElixirData fit, and where doesn’t it?

We sell one option in the wider stack, and it isn’t an ingestion check tool. According to our Context OS page, Context OS sits between enterprise data and AI agents. It maps entities and relationships, sets boundaries through policy checks, authority rules and approval thresholds, and records each decision with the information used and the checks performed. It doesn’t validate schemas, count rows or check freshness at ingestion. Use a tool from the categories above for that.

Context OS fits after the checks, when an agent acts on data and you need to show which policy applied and who approved it. Our post on transformation drift in agentic ETL pipelines covers a related problem: pipelines that pass every check while their meaning shifts. For the question of who may fix a quarantined record, see our guide to AI data quality software for compliance. This guide stays on checks at the point of arrival.

How do you choose?

• You already run dbt: start with its tests and source freshness, and write a row-count test for volume.
• You want checks in code, independent of one warehouse: compare Great Expectations and Soda, and ask for a demo of the failed-record output.
• Your pipelines run on Databricks: use expectations first, and plan the quarantine pattern and downstream dependency deliberately.
• You’ve been hit by late or missing feeds: add an observability platform for freshness, volume and schema, after the hard rules are in place.
• You need one rule library, approvals and lineage across many systems: look at enterprise suites and ask exactly where their checks execute.

What can you test in a pilot?

Run the same short trial on every shortlisted tool, using a copy of one real feed:

1. Drop a column from a test file. Does the load stop, and how quickly?
2. Change a data type (a number to text). Is it caught at arrival or later?
3. Send a file at a tenth of normal size. Does anything notice the volume drop?
4. Hold back a delivery past its due time. Does a freshness check fire without a failing row?
5. Add duplicate keys, a null in a required field and an out-of-range value. Which does the tool block, flag or miss?
6. Check where the bad rows go. Can you find them, and do they carry the rule and the time?
7. Follow the alert. Who gets it, how fast, and does it show which downstream tables, dashboards or agents are affected?

Score each result as blocked, diverted, flagged or missed. What a tool leaves for you to build matters as much as what it does.

Frequently asked questions

What is the difference between ingestion checks and data observability? Ingestion checks apply rules you wrote when data lands and can block a load. Observability watches data and pipelines over time to catch problems you didn’t write a rule for, and it often works on data that has already loaded.

Which checks should run first at ingestion? Start with schema, required fields, duplicate keys, volume and freshness. They’re cheap to write and catch the failures that break most downstream work.

Is quarantine a feature or a design choice? Mostly a design choice. Databricks documents a pattern for it, and dbt can store failing rows, but in the pages we read quarantine is something you build from a tool’s failure output.

Can one tool cover everything? Rarely. Many teams pair a rule layer in the pipeline with a monitoring tool outside it. Check each vendor’s documentation for the specific check you need.

Is Context OS a data quality tool? No. It governs how AI agents use data, with policy, approvals and a record of each decision. It works alongside the tools above rather than replacing them.

If your agents act on data that has already passed its checks and you need to show the policy, approval and context behind each decision, book a demo of Context OS.

Sources

All pages opened 9 November 2026.

1. dbt, Data tests: https://docs.getdbt.com/docs/build/data-tests
2. dbt, Sources and freshness: https://docs.getdbt.com/docs/build/sources
3. Great Expectations, GX Core overview: https://docs.greatexpectations.io/docs/core/introduction/gx_overview
4. Great Expectations, Expectation gallery: https://greatexpectations.io/expectations/
5. Soda, Data testing and contracts: https://docs.soda.io/data-testing
6. Soda, Verify a data contract: https://docs.soda.io/data-contracts/data-contracts-verify
7. Soda, Enforce data contracts in CI/CD (7 April 2026): https://soda.io/blog/enforce-data-contracts-ci-cd-pipelines
8. Azure Databricks documentation on Microsoft Learn, Manage data quality with pipeline expectations (page updated 26 October 2026): https://learn.microsoft.com/en-us/azure/databricks/ldp/expectations
9. Azure Databricks documentation on Microsoft Learn, Expectation recommendations and advanced patterns (page updated 9 October 2026): https://learn.microsoft.com/en-us/azure/databricks/ldp/expectation-patterns
10. Monte Carlo, Data quality: https://montecarlo.ai/platform/data-quality
11. Informatica, Data Quality: https://www.informatica.com/products/data-quality.html
12. Ataccama, Data quality: https://www.ataccama.com/platform/data-quality
13. ElixirData, Context OS: https://www.elixirdata.co/platform/context-os/
14. ElixirData, Transformation drift in agentic ETL pipelines: https://www.elixirdata.co/blog/transformation-drift-agentic-etl-pipelines
15. ElixirData, Best AI Data Quality Software for Compliance in 2026: https://www.elixirdata.co/blog/best-ai-data-quality-software-compliance-2026

This article is published by XenonStack, which develops ElixirData. ElixirData sells Context OS, which is not one of the ingestion check tools compared here.