Skip to content

Release: v0.1.0#4

Open
RemingtonRohel wants to merge 72 commits intomainfrom
develop
Open

Release: v0.1.0#4
RemingtonRohel wants to merge 72 commits intomainfrom
develop

Conversation

@RemingtonRohel
Copy link
Copy Markdown
Collaborator

@RemingtonRohel RemingtonRohel commented Sep 27, 2024

Installation

pip install git+https://github.com/SuperDARNCanada/procDARN.git@develop

Python API

In-memory fitting

Use FITACF3 algorithm to fit a list of RAWACF records

fitacf3(recs: list[dict]) -> list[dict]

Example

import procdarn
import dmap
infile = "path/to/rawacf"
rawacf_records = dmap.read_rawacf(infile)
fitacf_records = procdarn.fitacf3(rawacf_records)
dmap.write_fitacf(fitacf_records, "path/to/fitacf")

File-to-file fitting

Fit a RAWACF file into a FITACF file

file_fitacf3(rawacf_file: str, fitacf_file: str)

Example

import procdarn
procdarn.file_fitacf3("path/to/rawacf", "path/to/fitacf")

Rust API

  • procdarn::file_fitacf3(raw_file: PathBuf, fit_file: PathBuf) -> Result<(), Fitacf3Error>: Fits a RAWACF file into a FITACF file
  • procdarn::fitting::fitacf3::fitacf_v3::fitacf3(Vec<RawacfRecord>) -> Result<Vec<FitacfRecord>, Fitacf3Error>: single-threaded FITACFv3 on collection of RawacfRecords (from dmap crate)
  • procdarn::fitting::fitacf3::fitacf_v3::par_fitacf3(Vec<RawacfRecord>) -> Result<Vec<FitacfRecord>, Fitacf3Error>: multi-threaded FITACFv3 implementation

Binary

fit_fitacf3: Command-line tool for fitting a rawacf file

Usage: fit_fitacf3 <INFILE> <OUTFILE>

Arguments:
  <INFILE>   Rawacf file to fit
  <OUTFILE>  Output fitacf file path

Options:
  -h, --help     Print help
  -V, --version  Print version

RemingtonRohel and others added 30 commits November 9, 2023 22:41
* So far have copied all of the command line arguments from RST that are relevant
* Have yet to implement any functionality
* Added default arguments and types for some cli arguments
* Adding structs for gridding
* Adding RadarScan struct
* Adding in rpos utilities
* Finished GridTable::map() method, working through GridTable::add_beam()
* Using geodesy crate for coordinate transformations.
* Including IGRF calls is going to be extremely tedious.
…Records.

* Method used in the grid.rs binary.
* Added function to find the first record after a timestamp in an array of FitacfRecords
* Added more functionality to bin_main() of grid.rs binary, up to calculating end_time for the grid.
* Need to iterate through GridPoints of Table and convert them to vectors suitable for GridRecord
* Rounded out the rest of bin_main in bin/grid.rs, still lots of TODOs
* Corrected RadarScan::reset_beams() after reviewing equivalent RST function
* Updated dmap dependency
* Created new file utils/dmap.rs and put function there, as it will have shared usage by gridding.
* Updated dmap dependency to fix some types in GridRecord
* Removed unused use statements
* Fixed moved value errors in bin/grid.rs
* Completed TODOs around filtering data from scans when gridding in bin/grid.rs
* Still have to apply boxcar median filter, and load AACGM coefficients
* Write the GridRecords to file once done looping through files
Started working on median filter kernel for gridding.

* Still very much a work in progress, only about 1/8 implemented.
* Updated dmap crate dependency to use new error_refactor branch
* Still untested, but IDE not throwing any warnings in the file.
* fieldpoint calculations now mostly added. Still have some functions to add but pretty close to complete.
* Still have some differences in numerical results vs. RST, working through those.
…` command.

* Uses new DMAP API
* Benchmarked faster than RST, 8x on my machine with 8 cores
* Fixed clippy warnings
* Made fit_fitacf3 arguments positional
* Created a convenience function to get the hdw info associated with a rawacf record, reducing copied code in bench and tests.
Update following `dmap` API change
* Removed Rust binary `fit_fitacf3`, exposed `fit_fitacf3` as a Python CLI tool using maturin.
* Created another Python-available function `fitacf3` which takes a list of rawacf records, like what is read in by `dmap.read_rawacf()`
* Created two rust functions, `fitacf3` and `par_fitacf3` which are single- and multi-threaded fitacf3 callers on `Vec<RawacfRecord>`, used in bench and test files and Python API functions.
* Avoids passing data through the Python-Rust interface
RemingtonRohel and others added 30 commits February 27, 2025 17:25
Bring `develop` into `gridding` branch
* Now compiles, currently working through more bugs that cause errors at runtime.
* AACGM_v2_DAT_PREFIX not set - need to figure out how to get this from dependency, if possible.
* Currently, stumped at GridTable::map() panicking with an index error.
* median_filter calculated max number of beams correctly; skips averaging beams with no cells
* Made all GridArgs members public; removed `old_aacgm_flag` member
* Fixed some mixups between `no_limits_flag` and `op_param_flag`
* let `end_time` be an Option; doesn't error out now if not specified
* Gated some printing with `args.verbose`
* Used `extended_mode_flag` to determine whether pwr + spectral width written to file
* Fixed `RadarScan::add_beam()` so it initializes vectors of correct size
* Handle partial records (no slist + other vectors in fitacf record)
* Ran cargo fmt
* Testing shows differences vs. RST, but is able to produce files. Need
  to investigate where the differences arise.
* Added a suite of gridding tests, all passing so far.
* Added structs for different coordinate systems.
* Now present two fit2grid functions - one for providing file paths, and another for providing FITACF records.
* GridArgs no longer requires `infiles` or `outfile` - these are provided with `GridArgsCLI` struct that wraps `GridArgs` for exposed `fit2grid` binary.
Added RST's `make_grid`, calling it `fit2grid`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant