me/dags
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
No description
43 commits 1 branch 0 tags 123 KiB
  • Python 68.6%
  • Shell 15.5%
  • Nix 9.5%
  • Dockerfile 6.4%
Find a file
Exact
Christian Lengert 66d5fbd8cb
All checks were successful
Build Airflow Image / build_airflow (push) Successful in 43s
Details
Deduplicate bronze locator by submission_id 
Kafka retries and topic replays can deliver the same submission multiple times.
QUALIFY keeps the first occurrence per submission_id so downstream silver/gold
models and the uniqueness test always see clean data.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 08:26:44 +02:00
.forgejo/workflows Add dbt/ to CI trigger paths 2026-06-27 00:46:56 +02:00
dags Switch to singular dbt tests for external parquet models 2026-06-28 09:30:11 +02:00
dbt Deduplicate bronze locator by submission_id 2026-06-30 08:26:44 +02:00
scripts Switch raw layer to aws s3 sync with capturedAt filtering in bronze 2026-06-27 16:30:39 +02:00
src/clients Restructure: src/clients library, dags/, tests/, pyproject.toml 2026-06-26 20:31:32 +02:00
tests Restructure: src/clients library, dags/, tests/, pyproject.toml 2026-06-26 20:31:32 +02:00
.gitignore Ignore local data/ directory 2026-06-26 23:52:11 +02:00
Dockerfile recfx base image 2026-06-28 08:02:19 +02:00
flake.lock Add unified locator_pipeline DAG; remove superseded individual DAGs 2026-06-27 00:30:28 +02:00
flake.nix Add flake.nix dev shell with kaf and duckdb 2026-06-26 23:53:54 +02:00
pyproject.toml Replace Soda with dbt tests on bronze locator model 2026-06-28 09:25:56 +02:00