Each time dbt runs, it generates a trove of metadata about datasets and the work it performs with them. In this post, I’d like to show you how to harvest this metadata and put it to good use.
Introducing OpenLineage 0.1.0
We are pleased to announce the initial release of OpenLineage. This release includes the core specification, data model, clients, and integrations with common data tools.
Expecting Great Quality with OpenLineage Facets
Good data is paramount to making good decisions- but how can you trust the quality of your data and its dependencies?
Extending OpenLineage with Facets
Facets are a self-contained definition of one aspect of a job, dataset, or run at the time the event happened. They make the OpenLineage model extensible.
OpenLineage joins the LF AI & Data Foundation
Becoming a LF AI & Data project ensures that OpenLineage can never belong to a company, or even a group of developers; it belongs to us all.
Exploring Lineage History via the Marquez API
Taking advantage of recent changes to the Marquez API, this post shows how to diagnose job failures and explore the impact of code changes on downstream dependents.
Backfilling Airflow DAGs using Marquez
In this blog post, we'll discuss how lineage metadata can be used to automatically backfill DAGs with complex upstream and downstream dependencies.
How OpenLineage takes inspiration from OpenTelemetry
The data world and the service world have many similarities but also a few crucial differences.