Glows L2 - Apply calibration#2918
Glows L2 - Apply calibration#2918vmartinez-cu merged 9 commits intoIMAP-Science-Operations-Center:devfrom
Conversation
vmartinez-cu
requested review from
Copilot,
lacoak21,
maxinelasp,
sdhoyt and
tech3371
vmartinez-cu
added
enhancement
There was a problem hiding this comment.
Pull request overview
This PR adds support for applying a time-dependent GLOWS L2 calibration factor (counts/s per Rayleigh) when converting daily lightcurve histogram rates and uncertainties into physical units (Rayleighs), using an ancillary “l2-calibration” dataset.
Changes:
- Plumbs a calibration dataset through
glows_l2()→HistogramL2→DailyLightcurveand applies it in flux/uncertainty calculations. - Adds
HistogramL2.get_calibration_factor()to select an appropriate factor based on the mid-epoch of the observation day. - Refactors the GLOWS ancillary calibration
.datreader to store calibration times as coordinates and sort by time; updates/extends tests and fixtures accordingly.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| imap_processing/glows/l2/glows_l2.py | Passes calibration dataset into HistogramL2 during L2 generation. |
| imap_processing/glows/l2/glows_l2_data.py | Adds calibration-factor selection and applies calibration in DailyLightcurve flux math. |
| imap_processing/ancillary/ancillary_dataset_combiner.py | Refactors l2-calibration .dat ingestion to use start_time_utc as a coordinate and sorts by it. |
| imap_processing/tests/glows/conftest.py | Adds a mock calibration dataset fixture. |
| imap_processing/tests/glows/test_glows_l2.py | Updates tests to provide calibration dataset to glows_l2()/HistogramL2. |
| imap_processing/tests/glows/test_glows_l2_data.py | Adds coverage for calibration-factor selection; updates flux/uncertainty expectations with calibration. |
| imap_processing/tests/ancillary/test_ancillary_dataset_combiner.py | Extends calibration combiner test and asserts coordinate usage/sorting. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
tech3371
left a comment
tech3371
left a comment
There was a problem hiding this comment.
The code changes looks good! Thank you for getting this in so quick!
| good_data["epoch"].data, calibration_dataset | ||
| ) | ||
| else: | ||
| calibration_factor = None |
There was a problem hiding this comment.
Can you add comment about why we can continue if data is empty and calibration can be None? I am guessing, you carry bad data?
There was a problem hiding this comment.
Bad data is carried through and processing returns an empty dataset rather than raising an error
| # The mid-epoch is after calibration timestamps, | ||
| # so the most recent (1.020) is selected. |
There was a problem hiding this comment.
(1.020), Is that the calibration data for the mid-time?
There was a problem hiding this comment.
Correct! I updated the comments to make this clearer
| between_epoch = np.array([369808281184000000]) | ||
| assert HistogramL2.get_calibration_factor( | ||
| between_epoch, mock_calibration_dataset | ||
| ) == pytest.approx(0.849) |
…r calibration factors and sort data by time. Update corresponding unit test to reflect changes
…t for the calibration factor needed for flux calculations. Update existing tests where needed to fix broken tests
…-epoch is extracted from numpy array
…afer comparison with l1b dataset epoch
vmartinez-cu
force-pushed
the
glows-l2-calibration
branch
from
3dd30fb to
020c7fe
Compare
vmartinez-cu
merged commit d5f3d78
into
IMAP-Science-Operations-Center:dev
This pull request implements the use of a time-dependent calibration factor for GLOWS L2 data processing, ensuring that photon flux and uncertainties are correctly converted to physical units (Rayleighs) using ancillary calibration data. The changes include updating data structures, processing logic, and tests to support calibration lookup and application.
Key changes:
GLOWS L2 Data Processing and Calibration
The
HistogramL2andDailyLightcurveclasses were updated to accept and use a calibration factor (counts per second per Rayleigh), which is required for correct photon flux and uncertainty calculations. The calibration factor is selected based on the mid-epoch of the observational day using a new static method,get_calibration_factor.The main
glows_l2processing function passes a calibration dataset through to theHistogramL2data class.Ancillary Calibration Data Handling
convert_file_to_dataset) was refactored to store calibration times as coordinates and ensure the dataset is sorted by time, improving reliability when selecting calibration factors.Testing Enhancements
Closes #2886