Skip to content

Issue/372/new extreme state blazar#373

Open
JulianHamo wants to merge 18 commits into
masterfrom
issue/372/new_extreme_state_blazar
Open

Issue/372/new extreme state blazar#373
JulianHamo wants to merge 18 commits into
masterfrom
issue/372/new_extreme_state_blazar

Conversation

@JulianHamo
Copy link
Copy Markdown
Collaborator

Summary

This PR refactor the filtering of extreme states, by creating both low- and high-state filters.

Changes

  • Create new blazar_low_state filter to use blazar_extreme_state science module.
  • Introduce new blazar_high_state filter to identify extreme high states using blazar_extreme_state.
  • Rename former blazar_low_state to keep track of both for overlap verifications.
  • Remove deprecated blazar_new_low_state filter.

Motivation

Closes issue #372 by merging refactoring old blazar_low_state filter into new blazar_low_state and blazarhigh_state`` filters.

@JulianHamo
Copy link
Copy Markdown
Collaborator Author

Rubin Sentinel failed before ZTF Sentinel started. I think there will still be an issue because to datatest/CTAO_blazar/CTAO_blazar_datatest_v20-12-24.parquet has not been modified to be processed with the new blazar_extreme_state module. I am not sure how to update it though.

@JulienPeloton
Copy link
Copy Markdown
Member

CI problem solved -- now it is failing in a relevant manner :-)

test-suite (ztf)	UNKNOWN STEP	2026-04-08T19:24:05.3653895Z File "/__w/fink-filters/fink-filters/fink_filters/ztf/filter_blazar_high_state/filter.py", line ?, in __main__.high_state_filter
test-suite (ztf)	UNKNOWN STEP	2026-04-08T19:24:05.3654794Z Failed example:
test-suite (ztf)	UNKNOWN STEP	2026-04-08T19:24:05.3655098Z     print(df.count())
test-suite (ztf)	UNKNOWN STEP	2026-04-08T19:24:05.3655410Z Expected:
test-suite (ztf)	UNKNOWN STEP	2026-04-08T19:24:05.3655684Z     12
test-suite (ztf)	UNKNOWN STEP	2026-04-08T19:24:05.3655927Z Got:
test-suite (ztf)	UNKNOWN STEP	2026-04-08T19:24:05.3656918Z     0

@JulianHamo
Copy link
Copy Markdown
Collaborator Author

Done. All tests passed.

Still keeping the former low state module and filter for comparison. Please check that you do not call them under the same column name blazar_stats (if that is the way to store them and not just ap refix for the name of a column in the alert package), though I believe it should be okay. If it is not, there will be some crash due to not found key in the dictionnary blazar_stats. Once that check is done, there should not be any issue.

Will remove the old module and filter two months after this one has been set to production. Two months of data to analyze should be enough to check a coherent behaviour.

@JulienPeloton
Copy link
Copy Markdown
Member

Thanks for the addition! I will think about the naming scheme.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants