Skip to content

Release 1.0#177

Merged
FusRoman merged 130 commits into
release-1.0.0from
release_1.0
Mar 19, 2026
Merged

Release 1.0#177
FusRoman merged 130 commits into
release-1.0.0from
release_1.0

Conversation

@FusRoman
Copy link
Copy Markdown
Owner

Complete rewrite of Fink-FAT in Rust

The previous Python implementation relied heavily on Pandas and lacked the
performance and architectural clarity needed to scale to the volumes produced
by ZTF and Vera Rubin. This rewrite is a clean break from that codebase —
nothing has been carried over as-is.

What changed

  • The entire codebase is now written in Rust.
  • The linking pipeline is built around a graph-based architecture: photometric
    alerts are modeled as nodes, and candidate trajectories emerge from edge
    construction and resolution across nightly observation windows.
  • No more Pandas. No more Python overhead at the algorithmic level.

Why a full rewrite

The Python version was not salvageable for the target scale. Rewriting in Rust
allows tight control over memory, parallelism (via Rayon), and latency — all
of which matter when processing millions of alerts per night.

…d also the struct managing the track hypothesis
…ding of fink-fat-engine. add a first seeding evaluation reports on the seeding parameters
@FusRoman FusRoman added this to the fink-fat 1.0 milestone Feb 23, 2026
@FusRoman FusRoman self-assigned this Feb 23, 2026
FusRoman and others added 5 commits February 23, 2026 13:00
* add support for file compression, add progress bar as a cli option

* run cargo fmt

* clippy fixes

* fix doc

* fix ci for coverage report

* fix non positive log score, add tests for log score, make edge new return a result instead of assert

* fix error message

* fix edge_builder progress bar, fix manifest test

* fix multiple seed building over night of the build_edge stage for single night mode

* improve progress bar of loading data stage

* optimize save edge compaction

* optimize loading and saving by doing async and concurrency

* add solver and fit orbit stage to the command line

* add new cost function for edges

* add new cost function for edges

* precision on doc

* change pre-commit to trigger workspace doc

* add logging

* add logging and tests

* add new config parameters to control edge generation

* fix chi2 position blow-up, replace gnomonic residuals for spherical residuals

* add data volume logs

* deactivate edges after successfull fit orbit, add unit and integration tests for that

* fix a huge bug regarding edge generation

* optimize memory consumption for edges
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 11, 2026

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 88.41363% with 493 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (release-1.0.0@8c69762). Learn more about missing BASE report.

Files with missing lines Patch % Lines
crates/fink-fat-engine/src/graph/edge/mod.rs 73.04% 138 Missing ⚠️
...-fat-engine/src/engine_config/propagator_config.rs 76.67% 59 Missing ⚠️
...ink-fat-engine/src/engine_config/triplet_config.rs 36.23% 44 Missing ⚠️
.../fink-fat-engine/src/graph/edge/edge_prediction.rs 87.53% 43 Missing ⚠️
...s/fink-fat-engine/src/engine_config/pair_config.rs 32.14% 38 Missing ⚠️
crates/fink-fat-engine/src/alerts/store.rs 94.71% 37 Missing ⚠️
crates/fink-fat-engine/src/engine_config/mod.rs 90.44% 32 Missing ⚠️
crates/fink-fat-engine/src/alerts/mod.rs 55.93% 26 Missing ⚠️
crates/fink-fat-engine/src/display_format.rs 0.00% 14 Missing ⚠️
crates/fink-fat-engine/src/engine_config/units.rs 93.29% 13 Missing ⚠️
... and 9 more
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@               Coverage Diff                @@
##             release-1.0.0     #177   +/-   ##
================================================
  Coverage                 ?   91.29%           
================================================
  Files                    ?       68           
  Lines                    ?    13963           
  Branches                 ?        0           
================================================
  Hits                     ?    12748           
  Misses                   ?     1215           
  Partials                 ?        0           
Flag Coverage Δ
rust 91.29% <88.41%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

FusRoman and others added 9 commits March 18, 2026 18:09
* big refactor of the fink-fat-eval crate

* update gitignore

* enabling seed parallel construction

* add seeding evaluation

* add plots for seeding

* fix an overflow error bug

* add edges stats

* move dev profile in the workspace cargo toml

* improve edge generation

* forgotten files

* optimize edge generation

* add tests, add solver evaluation

* add an edge features exporter

* fix docs and missing system dep

* clean edge ml prediction project, add a ml model evaluation in rust

* add rust ml eval plots

* add optuna optimization

* add fink-fat-eval readme

* add evaluation compilation profile note in readme

* add a new edges generation benchmark

* change ml behavior in the pipeline, ml is not a ranking anymore, it is now a filtering

* add engine readme, remove units file, put content in lib.rs

* add main doc page

* refactor fink fat main

* skip code coverage for fink-fat-eval
@FusRoman FusRoman merged commit efa0b58 into release-1.0.0 Mar 19, 2026
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants