Summary
- Open-source data lineage tools like OpenMetadata, Marquez, Atlas, Egeria, and Spline cost nothing to license but need your own hosting.
- Platform lineage in Databricks Unity Catalog and Google Knowledge Catalog is automatic inside the platform and limited outside it.
- Collibra and Atlan automatically map data dependencies across many systems, and both support OpenLineage.
- Google keeps Knowledge Catalog lineage for 30 days, while Unity Catalog keeps it in Catalog Explorer indefinitely.
- OpenLineage is the common standard, so instrumenting pipelines once lets you change the tools later.
Nothing brings a data team together quite like a revenue dashboard that suddenly stops making sense. Everyone has a theory. Nobody knows which upstream table to blame. After an hour of digging through dbt and Spark, the investigation reaches two Power BI datasets. The answer? A column renamed earlier in the week. Not exactly the mystery of the century.
Data lineage tools are supposed to make that investigation considerably shorter. Instead of following every dependency by hand, you can trace where the data came from and who’ll panic if it disappears. The options range from self-hosted open-source projects to lineage built into platforms like Databricks and Google Cloud. Commercial data catalogs promise a broader view across the stack.
We put ten tools side by side on the same points. How does lineage get in? Does it go down to column level, does it speak OpenLineage, and where does it run? Recent changes are here as well (Google’s Dataplex is now Knowledge Catalog, and a handful of lineage products have ended up inside bigger vendors), plus a table to help you pick.
So What Are Data Lineage Tools?
Data lineage tools keep a kind of diary for each dataset, noting where it came from, which jobs changed it, and where it ended up. That diary is saved as a searchable graph. Is a dashboard number off? It points to the tables, jobs, and queries sitting behind it. Planning to drop a column? It lists every report that still reads from it.
Knowing where a dataset came from is a start. But suppose you need to figure out why one particular revenue column looks wrong. Suddenly, a map of tables isn’t nearly enough.
That’s where column-level lineage earns its keep. You can follow a field through joins and calculations, rather than guessing which transformation touched it. At the broader table level, you’re simply seeing how datasets connect. Some platforms also translate the technical machinery into business terms. Engineers can examine the underlying jobs and code; analysts follow the same trail through reports.
The benefits aren’t limited to fixing broken numbers, either. Imagine renaming a table without realizing six dashboards depend on it. Lineage can show those dependencies beforehand. And when compliance asks where customer information travels, you have a record of its path rather than a reconstruction based on everyone’s increasingly unreliable memory. Particularly handy when GDPR or CCPA enters the conversation.
How Data Lineage Tools Capture Lineage
Where the lineage comes from is what separates these tools most.
| Method | How it works | Tools that rely on it |
|---|---|---|
| Built into the platform | The engine records lineage for every query it runs | Databricks Unity Catalog, Google Knowledge Catalog |
| Pushed by jobs at runtime | ETL tools and schedulers send lineage events as jobs execute | Microsoft Purview (from Data Factory, Synapse, Azure Databricks), OpenLineage with Marquez, Spline |
| Extracted from SQL, logs and tool APIs | The tool reads query logs, SQL code and BI tool metadata, then rebuilds the graph | Atlan, OpenMetadata, Collibra (automated extraction across sources, ETL and BI tools) |
| Added by hand or through an API | People or scripts register lineage that no connector can see | Most tools, as a fallback |
Inside its own platform, platform lineage is the most accurate option you’ll get, though it can’t see a thing outside it. Parsing reaches the most systems, as long as the connectors are there. Runtime events tell you what actually ran. The catch is that every job has to be instrumented.
Why Data Lineage Can Be Incomplete

Automated lineage is only as good as what the tool can parse or observe, and a graph can look finished while relationships are missing. dbt says its column-level traceability relies on SQL parsing, and that lineage can be incomplete when parsing fails or a column’s origin is unknown. Its examples include JSON unpacking, lateral joins, and hardcoded table references. Ambiguous or overly complex SQL, including complex lateral joins, can cause parsing errors with the same result.
Column-level lineage also may not capture every way a column is used. dbt says its column-level lineage reflects SELECT logic in the model SQL, but not other usage such as joins and filters. So if a column only appears in a join condition or a filter, don’t assume the graph shows it.
BI tools have the same problem. Google Cloud says LookML that uses complex custom SQL, such as Liquid templates or derived tables with complex joins, may not be fully parsed. The result can be disconnected lineage nodes.
The practical takeaway: when you evaluate a tool, ask which systems, SQL patterns, and object types it covers, and how it shows you the gaps.
Data Lineage Tools Compared
| Tool | Type | How lineage gets in | Column level | OpenLineage | Where it runs |
|---|---|---|---|---|---|
| OpenMetadata | Open source | Ingestion from warehouses and BI tools, manual edits, API | Yes | Accepts events | Self-hosted |
| OpenLineage + Marquez | Open source | Run-time events from integrations such as dbt | Depends on the integration | It is the standard | Self-hosted |
| Apache Atlas | Open source | Hooks in the Hadoop ecosystem, REST API | Yes, but column-level support varies by system | No | Self-hosted |
| Egeria | Open source | Metadata exchange between connected tools | Yes, through DataMapping relationships and OpenLineage | Yes, natively supports OpenLineage events and Airflow integration | Self-hosted |
| Spline | Open source | Spark agent | Yes, primarily for Apache Spark execution plans | Adapter is a proof of concept | Self-hosted, Docker |
| Microsoft Purview | Commercial | Pushed by Azure services and Power BI, scanned from databases | Varies by source | Yes, specifically as an ingestion path for lineage events | Microsoft cloud |
| Databricks Unity Catalog | Platform | Automatic for queries on Databricks | Yes | No native support; works separately via Spark | Databricks |
| Google Knowledge Catalog | Platform | Automatic for BigQuery, Dataflow and other services; API | Only BigQuery SQL and Spark jobs | Recommended format for custom sources | Google Cloud |
| Collibra | Commercial | Automated extraction from sources, ETL and BI tools | Yes | Supported | Collibra cloud |
| Atlan | Commercial | SQL parsing, tool APIs, run-time events | Yes | Consumes events | Atlan cloud |
Open Source Data Lineage Tools
OpenMetadata

OpenMetadata is an open-source metadata platform, and lineage comes built in. You set up its ingestion workflows from the UI, and they pull lineage out of warehouses such as Snowflake and out of dashboard tools. Did a connector miss something? Edit the table and column lineage by hand. OpenLineage events are welcome too, through its REST API. The 2.0 line is the current one.
Best for: teams that host their own tools and would like discovery, governance, and lineage in one place instead of three.
Limits to know: you operate it yourself, ingestion and all, so lineage only reaches as far as the connectors someone has set up.
OpenLineage and Marquez

Strictly speaking, OpenLineage is an open standard for capturing lineage while jobs are running, built on datasets, jobs, and runs, which you can extend with custom “facets”. Integrations for tools like dbt emit those events. Marquez is the reference implementation of the standard. It’s a metadata server whose OpenLineage-compatible endpoint receives events in real time, keeps them, and renders the graph. WeWork created Marquez and later open-sourced it.
Best for: engineers who trust lineage from real job runs, kept in a format that other tools can read later.
Limits to know: you have to instrument every pipeline, and Marquez is one more service for you to host.
Apache Atlas

Apache Atlas was designed for the Hadoop ecosystem and delivers metadata and governance services there, while still connecting to the broader enterprise stack. It defines metadata types for Hadoop as well as for other systems. Assets can be classified (PII, data quality, expiry dates, for example), and the classifications follow the data along its lineage. Atlas ships under the Apache 2.0 license.
Best for: companies with a big Hadoop estate already in place.
Limits to know: beyond Hadoop, expect to write a larger share of the integration yourself.
Egeria

Egeria comes from the Linux Foundation, and rather than acting as one more catalog, it focuses on letting metadata move between the tools a company already has. At the base, there’s an open metadata schema covering more than 600 types. APIs, connectors and protocols are layered over it, which is how separate repositories end up sharing and linking metadata, lineage among it.
Best for: large organizations that need all their catalogs and tools to work from a single metadata model.
Limits to know: as an integration framework, it takes a dedicated project to deploy and then to maintain.
Spline

Spline started out tracking lineage for Apache Spark (Spark Lineage is where the name comes from), and an IEEE paper covered it in 2018. Here’s the basic flow. An agent running in Spark collects lineage from jobs written in Scala, Python, or Spark SQL. That lineage travels over REST or Kafka to the Spline Server, which saves the graph in ArangoDB and shows it in a web UI. There’s a quick start that runs in Docker. No license fee: Spline is open source under Apache 2.0.
Best for: teams whose transformations mostly run on Spark.
Limits to know: the OpenLineage adapter is still only a proof of concept, so getting Spline to work alongside other tools takes effort.
Commercial and Platform Data Lineage Tools
Microsoft Purview

Microsoft Purview pulls lineage from Azure services together into a single map. While they run, Data Factory, Data Share, Synapse, and Azure Databricks push their lineage to it. Power BI and Azure Machine Learning report on how they use datasets. Databases such as Oracle, Teradata, and SAP add lineage from their views and stored procedures.
These details describe lineage in Purview’s classic Data Catalog. In the newer Unified Catalog, you open it by searching for a data product and then viewing the data asset’s details.
Best for: organizations built on Azure and Power BI.
Limits to know: how much you get depends on the source, and stored procedures with DROP or CREATE statements inside them don’t get captured.
Databricks Unity Catalog

For queries run on Databricks, Unity Catalog captures lineage on its own, all the way to column level and across every workspace attached to the metastore. Outside systems can join the same graph if you register them as external assets, say Salesforce or MySQL upstream and Tableau or Power BI downstream. And retention? For data captured since September 1, 2024, Catalog Explorer holds lineage indefinitely, while the system tables roll over after one year.
Best for: teams whose transformations mostly happen in Databricks.
Limits to know: queries must run through Spark DataFrames or Databricks SQL to be captured, and outside systems show up only after you register them.
Google Knowledge Catalog (formerly Dataplex)

Dataplex Universal Catalog got a new name from Google: Knowledge Catalog. Turn on the Data Lineage API and BigQuery, Dataflow, and the other supported services start reporting lineage without further setup. Other systems send their events through that same API, and Google’s recommendation is the OpenLineage format.
Best for: stacks on Google Cloud with BigQuery at the center.
Limits to know: lineage sticks around for just 30 days by default, and you pay for it through Knowledge Catalog’s premium processing SKU, in Data Compute Units. It isn’t instant either: Google says lineage can take from 30 minutes up to 24 hours to appear, depending on processing volume and complexity. Column-level lineage is supported only for BigQuery SQL transformations and Managed Service for Apache Spark jobs. Load jobs, copy jobs, routines, external tables, custom OpenLineage events, and nested fields in STRUCT or JSON types are excluded. If one transformation job creates more than 1,500 column-level links, Knowledge Catalog skips column-level collection and keeps table-level lineage only.
Collibra

Lineage in Collibra is extracted automatically from sources, ETL tools and BI tools, and impact can be traced at table, column and report level. It also supports OpenLineage. Audit coming up? The diagrams can be exported to PDF, PNG, or CSV.
Best for: enterprises whose main reason for lineage is compliance and the data governance team.
Limits to know: since it’s a complete governance platform, lineage earns its keep inside a governance program and less so as a fast bolt-on.
Atlan

Atlan gets to column-level lineage by parsing SQL and calling each tool’s API, and it claims coverage of more than 80 systems. It takes in OpenLineage events from Airflow, Spark, dbt Cloud, and Astronomer as well. For lineage that no connector reaches, there are APIs, SDKs, CSV import, and a visual builder. AI agents can query the lineage graph too, through an MCP server.
Best for: cloud-first companies that want business users, and not just engineers, to use the catalog.
Limits to know: the 80+ systems number is Atlan’s own count, so look for your specific connectors on its list.
Data Lineage Tools: Acquisitions and Name Changes
On older lists, you’ll still spot Manta and Octopai as independent vendors, but they are not independent anymore.
- IBM acquired Manta Software in October 2023 and incorporated its lineage technology into its watsonx.data Intelligence portfolio; the product is now positioned as IBM Data Lineage, rather than “IBM Manta Data Lineage.”
- Cloudera agreed to acquire Octopai’s platform in November 2024 and now sells it as Cloudera Data Lineage (formerly Octopai).
- Talend was acquired by Qlik in 2023 and retained the Talend brand within its portfolio. The combined platform is now marketed as Qlik Talend Cloud, with Talend-generated datasets and job lineage publishable into Qlik Cloud’s catalog, where users can trace transformations back to source. By the way, here is our article on Talend alternatives if you’re still looking for something to replace the tool.
- Salesforce completed its acquisition of Informatica in November 2025 and is integrating its catalog, governance, metadata-management, and MDM capabilities into the Salesforce platform. Informatica’s lineage is being positioned as enterprise-wide context for Salesforce’s trusted data foundation and Agentforce, extending metadata and traceability beyond the Salesforce ecosystem to systems across the business. Check our article on Informatica alternatives if you’re looking for something to replace it in your stack.
OpenLineage and Why It Matters
Look across this list, and OpenLineage keeps turning up. Marquez is built on it. Both OpenMetadata and Atlan accept its events, Collibra supports it, and Google recommends it for custom sources in Knowledge Catalog. That means one round of OpenLineage instrumentation lets you replace or combine the tools later without repeating the work.
On dbt already? Then you already have a head start. dbt documents each model and builds a DAG of the project, and the OpenLineage integration for dbt reads that metadata, so dbt with OpenLineage and Marquez is one of the least expensive ways to get working lineage. Keep dbt’s limits in mind, though. Its column-level lineage follows SELECT logic and can be incomplete when SQL parsing fails, so check the graph for models that use lateral joins, JSON unpacking, or hardcoded table references.
How to Pick a Data Lineage Tool
| Question | If the answer is… | Look at |
|---|---|---|
| Where do most transformations run? | Databricks | Unity Catalog |
| BigQuery | Google Knowledge Catalog | |
| Azure services and Power BI | Microsoft Purview | |
| Spark jobs you write | Spline, or OpenLineage with Marquez | |
| Many tools across clouds | Atlan, Collibra or OpenMetadata | |
| Do you need column-level impact analysis? | Yes | A tool with column-level lineage for your sources |
| Who will run it? | Your own engineers | OpenMetadata, Marquez, Atlas, Egeria, Spline |
| A vendor | Collibra, Atlan, Purview, or your platform’s catalog | |
| How long must the history last? | Longer than 30 days | Check retention: Knowledge Catalog keeps 30 days |
| Will you change tools later? | Possibly | Prefer tools that read OpenLineage events |
Pick the systems where bad numbers do the most damage. Prove lineage there, then roll it out wider.
Where Skyvia Fits
Skyvia doesn’t do lineage itself. What it handles is the earlier step of pulling data from scattered apps into one warehouse, so platform lineage and dbt have data to look at.
- Replication brings data from 200+ cloud apps and databases into BigQuery, Snowflake, Redshift, Azure Synapse, Databricks, or a database, and incremental updates keep it current. With history mode switched on, older versions of each record are kept, which is handy when an audit asks about a past value. With history mode switched on, older versions of each record are kept, which is handy when an audit asks about a past value.

Reading about data pipelines is fine, but watching one run is better. Here’s a quick video of Skyvia Replication at work:
- dbt Core runs. Your dbt project from Git can run on a schedule or straight after replication in a Control Flow, which keeps the models in dbt’s DAG built on current data.
- Automation logs each execution and keeps version history for its workflows.
Does your lineage stop at the warehouse because half your sources never get there? Then that’s the place to start: try Skyvia free and replicate one app into your warehouse.
FAQ for Data Lineage Tools
Are there free data lineage tools?
Yes, several of them. OpenMetadata, OpenLineage with Marquez, Apache Atlas, Egeria, and Spline are all open source, which means there’s no license to pay for. The hosting and maintenance are on you, though, and the lineage only covers what your integrations send it.
What separates table-level lineage from column-level lineage?
At table level, you see which datasets feed which. Column-level goes further and follows each field through joins and calculations, so you can see exactly which reports would break if one column changed. You’ll get column-level lineage from Unity Catalog, Collibra, Atlan, and OpenMetadata.
What is OpenLineage?
It’s an open standard for recording lineage as jobs run, and it’s organized around datasets, jobs, and runs. Marquez is the reference implementation. Because OpenMetadata and Atlan accept its events, Collibra supports the standard, and Google recommends the format for custom sources in Knowledge Catalog, one round of instrumentation can feed several tools at once.
Does Google Dataplex still exist?
Not by that name. Google renamed Dataplex Universal Catalog to Knowledge Catalog, but lineage there works just as it did: once the Data Lineage API is switched on, supported services like BigQuery and Dataflow report lineage, and the records stay for 30 days.
Can I get data lineage from dbt?
Partly, yes. Since dbt documents every model and builds a DAG of the project, you can already see how models depend on one another. For lineage that goes past dbt itself, the OpenLineage integration for dbt passes that metadata along to a lineage server such as Marquez.

