v0.12.0

This is the March 2025 release.

Highlights

• Instrument Processing
  • Improvements to CoDICE processing
    • Implemented code to validate the following L1a data products: hskp, hi-counters-aggregated, hi-counters-singles, hi-sectored, hi-priority, and lo-counters-aggregated
    • Added unit tests to validate L1a data product 'support' variables (e.g. energy tables, acquisition times, etc.)
  • Improvements to Hi processing
    • Implemented pset find_second_de_packet_data for L1c
    • Implemented exposure times for L1c
    • Implemented pset get_de_clock_ticks_for_esa_step for L1c
    • Validated hist data for L1c
    • Implemented bin L1b counts into pset bins for L1c
    • Implemented decommutation of DIAG_FEE packets for L1a
    • Implemented conversion of strings to coincidence type values in L1c pset calibration product config
    • Validated L1a DE data
  • Improvements to HIT processing
    • Validated L1b housekeeping data
    • Implemented summed rates algorithm for L1b
    • Added ancillary tables and table handling for L2
    • Added check for housekeeping APID in L1b
    • Implemented summed fluxes algorithm for L2
    • Updated logical sources
    • Validated L1a raw housekeeping CDF
    • Added sectorate angles for L0
    • Implemented standard fluxes algorithm for L2
    • Refactored processing for summed intensity for L2
    • Updated sectored rates dimensions for L1a
  • Improvements to IDEX processing
    • Implemented initial L2a processes with unit tests
    • Validated IDEX L1a CDFs
  • Improvements to Lo processing
    • Implemented initial L1b processing and DE dataset
    • Implemented calculation for average spin duration for each ASC
    • Added SHCOARSE and MET fields to L1a DE
    • Implemented Spin bin for L1b DE
  • Improvements to MAG processing
    • Implemented L1b processing
    • Implemented initial processing for L1c
  • Improvements to SWAPI processing
    • Fixed processing with duplicate SHCOARSE value in housekeeping data
  • Improvements to SWE processing
    • Minor improvements to L2 processing
    • Created L2 CDF attrs file
    • Added XTCE back in non-science data processing
    • Added acq_duration to L1b and L2
    • Created CDF for non-science packets and defined their CDF attrs
  • Improvements to Ultra processing
    • Fixed bug in theta and phi calculations for L1b
    • Fixed bug in extendedspin calculation for L1b
    • Added unique ID in L1a processing
    • Created spacecraft histogram for L1c healpix
    • Enabled spacecraft exposure in healpix
    • Implemented structure for L1c sensitivity
    • Updated file parsing for L1a
    • Implemented spacecraft pset data processing for L1c
    • Reordered Ultra map code dimensions to (epoch, energy, az, el)
    • Implemented L2 CDF structure
    • Implemented mock L1c data on healpix grid
• Infrastructure
  • Added support for spacecraft quanternion processing
  • Added script that enables webpoda download capability
  • Added support for segmented polynomials in XTCE files
  • Created lambda and API endpoint for dependency tracking
  • Updated dependencies for IDEX L2a, Ultra L1b, SWE L3, and HIT L1-L3
  • Converted ENA map time from TT J2000ns to TBD ET
  • Implemented "pull" index matching and value projection method for ENA maps
  • Replaced internal angles in radians with degrees for ENA maps
  • Implemented index mapping util for rectangular grids
  • Now producing xarray datasets from ENA maps
  • Removed ISTP check for L1 files and set terminate_on_warning to False
  • Added ancillary file to dependency tree
  • Updated indexer lambda to ingest ancillary files
  • Improvements to SPICE
    • Changed default angular unit to degrees
    • Implemented pointing table query code
  • Improvements to I-ALiRT processing
    • Implemented I-ALiRT MAG decommutation
    • Added IP address for security group in Kiel
    • Reconfigured lambda to use non-proxy integration and queryStringParameters
    • Renamed I-ALiRT download endpoint
• Data Access API
  • Added support for querying for ancillary files including unit tests
  • Added I-ALiRT and spacecraft as valid instruments
  • Fixed bug in ancillary filename convention
  • Updated get_files function and added a download function
• Misc
  • Fixed bug in release GitHub Actions build for imap_processing and imap_data_access repos
  • Removed some unused test files from the imap_processing repo
  • Removed pyyaml dependency from imap_processing repo
  • Fixed poetry/psycopg import issue

What's Changed

• MAG L1B by @maxinelasp in #1376
• HIT L1B - add algorithm for summed rates product by @vmartinez-cu in #1369
• SWE: L2 update from SWE and I-ALiRT Discussion and added SPASE metadata by @tech3371 in #1379
• Mag l1B timeshift by @maxinelasp in #1380
• 1401 hi l1c pset find second de packet data by @subagonsouth in #1403
• Fix CoDICE-lo L1a validation discrepencies by @bourque in #1400
• Exclude certain variables in CoDICE housekeeping products by @bourque in #1399
• Ultra l1a - Add unique id by @laspsandoval in #1390
• SWE: adding back in SWE non-science APID to processing pipeline by @tech3371 in #1388
• Reorder ultra map code dimensions -> (epoch, energy, az, el) by @nkerman in #1414
• Ultra l1b - phi theta corrections by @laspsandoval in #1405
• Add repoint module, test coverage, and general conftest fixture for g… by @subagonsouth in #1415
• 1402 hi l1c pset get de clock ticks for esa step by @subagonsouth in #1404
• 1416 hi l1a decom diag fee packets by @subagonsouth in #1420
• TST: Remove unused test files by @greglucas in #1422
• Implement "Pull" Index matching method and value binning/projection; Convert TT -> TDB by @nkerman in #1396
• ENH: XTCE: Add support for segmented polynomials by @greglucas in #1409
• Add auto generation of calibration product integer values column from… by @subagonsouth in #1408
• MNT: Remove pyyaml from our direct dependencies by @greglucas in #1425
• SWE: L1 and L2 CDF attrs fixes identified by SKTEditor by @tech3371 in #1428
• SWE: Add CDF attrs for non-science apid by @tech3371 in #1430
• Mag cdf metadata from instrument team by @maxinelasp in #1411
• HIT L2 - add summed flux algorithm by @vmartinez-cu in #1419
• XTCE - Strip spaces from strings by @vmartinez-cu in #1434
• 1151 hi l1c exposure times by @subagonsouth in #1424
• add segmented polynomial handling to convert_raw_to_eu() by @lacoak21 in #1429
• SWE: Added acq_duration in L1B and L2 for L3 purposes by @tech3371 in #1432
• HIT L1B - validate housekeeping data by @vmartinez-cu in #1433
• Changes to rename hi l1b delta_t variables to tof by @subagonsouth in #1438
• IDEX l2a processing by @lacoak21 in #1382
• MNT: Removing ISTP check for l1 data files by @tech3371 in #1443
• HIT L1B: add check for housekeeping apid by @vmartinez-cu in #1444
• Ultra l1b aux quality flags by @laspsandoval in #1412
• SWAPI: Fixing case when there is duplicate value in HK data by @tech3371 in #1445
• HIT L1B: Fix key error by @vmartinez-cu in #1446
• Healpix Sky Maps by @nkerman in #1423
• Add energy table data to CoDICE-Hi L1a products by @bourque in #1421
• Ultra l1b - remove aux dependency by @laspsandoval in #1439
• I-ALiRT - decom mag by @laspsandoval in #1406
• Use degrees internally for ENA Maps by @nkerman in #1450
• Ultra l1c healpix - create spacecraft histogram by @laspsandoval in #1453
• Set terminate_on_warning to False for L1 products by @vmartinez-cu in #1458
• HIT - Update logical sources by @vmartinez-cu in #1468
• Ultra l1b - Spacecraft exposure in HEALPIX by @laspsandoval in #1462
• CoDICE L1a hi-sectored Validation by @bourque in #1456
• Ultra l1c pset - Structure for Ultra l1c Sensitivity by @laspsandoval in #1477
• 1150 hi l1c bin l1b counts into pset bins by @subagonsouth in #1452
• CoDICE L1a hi-priority implementation by @bourque in #1465
• Update parsing of DE binary blob to match updated order of DE fields by @subagonsouth in #1494
• HIT L0 - Add sectorate angles by @vmartinez-cu in #1493
• ENA Maps: Change UltraPointingSet to Healpix by @nkerman in #1484
• Ultra l1a - updates to parsing by @laspsandoval in #1499
• CoDICE L1a hi-counters-singles validation by @bourque in #1466
• HIT L1 - update housekeeping attributes by @vmartinez-cu in #1503
• Mag L1C first pass by @maxinelasp in #1455
• Lo - Add SHCOARSE and MET to L1A Direct Events by @sdhoyt in #1511
• HIT L2 - add algorithm for standard intensity by @vmartinez-cu in #1490
• Lo - Initialize the L1B DE Processing by @sdhoyt in #1522
• IDEX L1A CDF Validation by @lacoak21 in #1479
• CoDICE L1a hi-counters-aggregated validation by @bourque in #1467
• Lo L1B - Average Spin Duration Calculation by @sdhoyt in #1524
• Remove bin_center from coord by @nkerman in #1505
• MNT: Update SWAPI DataLevel metadata by @greglucas in #1527
• Update release build to ignore metadata by @bourque in #1526
• ENH: Add spacecraft quaternion processing by @greglucas in #1525
• Conversions for ena map dictionaries to xarray DataArray/Datasets by @nkerman in #1485
• HIT L2 - refactor summed intensity processing by @vmartinez-cu in #1534
• MNT: Add test_data directory to gitignore file by @greglucas in #1536
• Codice l1a support variable validation by @bourque in #1488
• HIT L1A - update sector rates data by @vmartinez-cu in #1540
• MNT: Remove Data_level as a field from the CDF variables by @greglucas in #1535
• FIX: L0 is not a valid processing step, we only process l1a+ by @greglucas in #1532
• Ultra l1c - Basic spacecraft pset data by @laspsandoval in #1486
• CoDICE L1a lo-counters-aggregated Validation by @bourque in #1544
• Lo L1B - Create Lo Spin Bin Direct Event Field by @sdhoyt in #1538
• Updating conftest by @maxinelasp in #1549

Full Changelog: v0.11.0...v0.12.0