Skip to content

feat: OpenLineage lineage server and operator support - #6759

Open
ntkathole wants to merge 1 commit into
feast-dev:masterfrom
ntkathole:lineage_operator
Open

feat: OpenLineage lineage server and operator support#6759
ntkathole wants to merge 1 commit into
feast-dev:masterfrom
ntkathole:lineage_operator

Conversation

@ntkathole

@ntkathole ntkathole commented Aug 18, 2026

Copy link
Copy Markdown
Member

What this PR does / why we need it:

  • Introduces automatic producer discovery via a centralized feast-openlineage-config ConfigMap in the controller namespace, so external producers can discover the consumer endpoint without manual configuration.
  • Adds feast serve_lineage CLI command to run the OpenLineage consumer as an independent FastAPI server, enabling separate scaling and deployment from the Feast registry/UI server.
  • Extends the Feast Operator CRD with lineageServer configuration under openlineage.consumer, allowing the operator to deploy a dedicated lineage server Deployment and Service in Kubernetes.
  • Adds standalone_server config flag to delegate the retention background task exclusively to the lineage server while keeping the full consumer API available on both the UI/registry and lineage servers.

@ntkathole
feat: Support Lineage configs via operator
2483a48 
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
@ntkathole ntkathole self-assigned this Aug 18, 2026
@ntkathole
ntkathole requested a review from a team as a code owner August 18, 2026 17:16
@codecov-commenter

codecov-commenter commented Aug 18, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 46.97%. Comparing base (6a4690a) to head (2483a48).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #6759   +/-   ##
=======================================
  Coverage   46.96%   46.97%           
=======================================
  Files         418      418           
  Lines       51675    51667    -8     
  Branches     7485     7483    -2     
=======================================
  Hits        24268    24268           
+ Misses      25673    25665    -8     
  Partials     1734     1734
Flag Coverage Δ *Carryforward flag
go-feature-server 30.58% <ø> (ø)
python-unit 48.27% <ø> (+<0.01%) ⬆️ Carriedforward from 6a4690a

*This pull request uses carry forward flags. Click here to find out more.

Files with missing lines Coverage Δ
sdk/python/feast/api/registry/rest/__init__.py 39.86% <ø> (+2.11%) ⬆️
sdk/python/feast/cli/cli.py 55.17% <ø> (ø)
sdk/python/feast/cli/serve.py 56.41% <ø> (ø)
sdk/python/feast/constants.py 100.00% <ø> (ø)
sdk/python/feast/openlineage/config.py 63.88% <ø> (ø)
sdk/python/feast/openlineage/consumer.py 68.36% <ø> (ø)
sdk/python/feast/repo_config.py 78.93% <ø> (ø)

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6a4690a...2483a48. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@ntkathole ntkathole

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ntkathole @codecov-commenter