Merge pull request #12 from Caltech-IPAC/spherex-cutouts-update

Documentation on cutouts, mostly

Author: vandesai1

documentation/spherex_data_access.md

@@ -5,15 +5,15 @@ IRSA provides layered access to these data to support a variety of use cases and
 These layers include:
 
 * **Browsable Directories:** SPHEREx on-premises data products are laid out in directories that can be navigated with standard web browsers.
-* **Application Program Interfaces:** IRSA provides SPHEREx data access APIs that are compliant with International Virtual Observatory Alliance (IVOA) standards.
+* **Application Program Interfaces:** IRSA provides SPHEREx data access APIs, most of which are compliant with International Virtual Observatory Alliance (IVOA) standards.
 * **Python Packages:** SPHEREx data at IRSA are accessible via the Python packages pyvo and astroquery
 * **SPHEREx Data Explorer:** IRSA provides a web-based Graphical User Interface (GUI) that makes it easy to search for, visualize, and download SPHEREx data.
 
 Each of these data access layers is described in greater detail in the subsections below.
 
 ## Browsable Directories
 SPHEREx data products are laid out in directories that can be navigated with standard web browsers.
-This is convenient for users to get a quick sense of the types of data products that are available, to quickly download some examples by clicking through the directory tree, and to script bulk downloads using wget or curl.
+This is convenient for users to get a quick sense of the types of data products that are available, to quickly download some examples by clicking through the directory tree, and to script bulk downloads using `wget` or `curl`.
 
 The root of the SPHEREx data quick release data directories is:
 https://irsa.ipac.caltech.edu/ibe/data/spherex/qr
@@ -35,6 +35,8 @@ The content of each subdirectory is described in greater detail in the Data Prod
 
 ## Application Program Interfaces (APIs)
 
+### SPHEREx Spectral Image MEFs and Calibration Files are available through the IVOA Simple Image Access Protocol
+
 IRSA provides API access to SPHEREx Spectral Images through version 2 of the VO Simple Image Access (SIA2) protocol.
 SIA2 allows users to query for a list of images that satisfy constraints based on position(s) on the sky, band, time, ID, and instrument.
 The list returned by the service includes data access URLs, which can be used to retrieve some or all of the images in the list using wget or curl.
@@ -47,12 +49,12 @@ Due to the nature of the ingestion process, new SPHEREx data will first be avail
 Availability via SIA2 and Python libraries like Astroquery and PyVO will lag on the order of a day.
 :::
 
-IRSA's generric SIA2 endpoint is:
+IRSA's generic SIA2 endpoint is:
 
 `https://irsa.ipac.caltech.edu/SIA?`
 
 Users must add a `COLLECTION` parameter to this endpoint to specify which dataset to search.
- There are three SPHEREx-related SIA2 collections:
+There are three SPHEREx-related SIA2 collections:
 
 * SPHEREx Quick Release Spectral Image MEFs that are part of the SPHEREx **Wide Survey** can be accessed with: `COLLECTION=spherex_qr`
 
@@ -63,9 +65,21 @@ Users must add a `COLLECTION` parameter to this endpoint to specify which datase
 You can use `wget` or `curl` to submit SIA2 queries from the command line.
 For example:
 
-* `wget -O example1.html "https://irsatest.ipac.caltech.edu/SIA?COLLECTION=spherex_qr&POS=circle+127.69444+-39.17760+0.01&RESPONSEFORMAT=HTML"`
+* `wget -O example1.html "https://irsa.ipac.caltech.edu/SIA?COLLECTION=spherex_qr&POS=circle+127.69444+-39.17760+0.01&RESPONSEFORMAT=HTML"`
+
+See the section on Python packages to learn how to use Python wrappers around IRSA’s SIA2 service.
+
+### Cutouts of SPHEREx Spectral Image MEFs
+
+SPHEREx Spectral Image MEFs are available on premises at IPAC and on the cloud.
+
+If you have identified the access URL for an on-premises Spectral Image, you can request a cutout of this MEF by appending a query string containing the center and size parameters. The parameters are described in more detail here:
+
+`https://irsa.ipac.caltech.edu/ibe/cutouts.html`
+
+**Example:**
 
-See the next section to learn how to use Python wrappers around IRSA’s SIA2 service.
+`curl -o cutout.fits "https://irsa.ipac.caltech.edu/ibe/data/spherex/qr/level2/2025W19_2B/l2b-v11-2025-163/3/level2_2025W19_2B_0073_2D3_spx_l2b-v11-2025-163.fits?center=156.09328159,-41.64466331&size=0.1"`
 
 ## Python packages: PyVO & Astroquery
 

documentation/spherex_data_products.md

@@ -1,4 +1,4 @@
-# SPHEREx Data Products Available at IRSA
+ SPHEREx Data Products Available at IRSA
 
 A detailed description of SPHEREx data products available to the public is provided in the [SPHEREx Explanatory Supplement](https://irsa.ipac.caltech.edu/data/SPHEREx/docs/SPHEREx_Expsupp_QR.pdf).
 Here we provide a concise summary of the science, calibration, and additional data products available at IRSA.
@@ -7,7 +7,10 @@ This summary includes filenaming conventions, for which we adopt the following d
 - `Planning Period` designates the survey plan uploaded to the spacecraft, e.g. `2025W18_2B`.
 Each planning period covers approximately 3.5 days of operation.
 
-- `Observation ID` includes the survey planning period and the large and small slew counters, e.g. `2025W18_2B_0001_1`.
+- `Observation ID` includes the survey planning period and the large and small slew counters.
+For example, `2025W18_2B_0001_1` contains the planning period (`2025W18_2B`), the large slew counter (`0001`), and the small slew counter (`1`).
+Each large slew has a maximum of 4 small slews, so the allowed small slew counter range is 1 to 4.
+Some large slews will have fewer than 4 small slews.
 
 - `Detector` is an integer from 1 through 6.
 
@@ -18,8 +21,10 @@ Each planning period covers approximately 3.5 days of operation.
 
 ## Main Science Data Product: Spectral Image Multi-Extension FITS Files (MEF)
 
-The main Quick Release data product is the Level 2 Spectral Image MEF.
+The main Quick Release data product is the Level 2 Spectral Image MEF,as described in Section 2.1 of the [Explanatory Supplement](https://irsa.ipac.caltech.edu/data/SPHEREx/docs/SPHEREx_Expsupp_QR.pdf).
+
 There are 6 Spectral MEFs (one for each detector) for each sky pointing.
+Because data quality assessments are evaluated per spectral image band, some observations will not include all 6 bands in the archive.
 Each Spectral MEF is approximately 70 MB and contains 6 extensions:
 
 HDU 1: IMAGE
@@ -40,7 +45,7 @@ HDU 1: IMAGE
 
 HDU 2: FLAGS
  : Bitmap of per-pixel status and processing flags, stored as a 2040 x 2040 image.
-   The definition of the flags are provided in Table 8 of the [SPHEREx Explanatory Supplement](https://irsa.ipac.caltech.edu/data/SPHEREx/docs/SPHEREx_Expsupp_QR.pdf).
+   The definition of the flags are provided in Table 10 of the [SPHEREx Explanatory Supplement](https://irsa.ipac.caltech.edu/data/SPHEREx/docs/SPHEREx_Expsupp_QR.pdf).
 
 HDU 3: VARIANCE
  : Variance of calibrated surface brightness flux in units of (MJy/sr)^2, stored as a 2,040 x 2,040 image.
@@ -78,6 +83,30 @@ HDU 6: WCS-WAVE
 
 - `level2_2025W18_2B_0001_1D1_spx_l2b-v12-2025-164.fits`
 
+## Cutouts of Spectral Image MEFs
+
+IRSA enables users to access rectangular cutouts of a SPHEREx Spectral Image MEF by simply appending a [query string](https://irsa.ipac.caltech.edu/ibe/cutouts.html) containing center and size parameters to the image URL.
+These cutout MEFs contain the same HDUs as the original Spectral Images (IMAGE, FLAGS, VARIANCE, ZODI, PSF, WCS-WAVE). However, the IMAGE, FLAGS, VARIANCE, AND ZODI HDUs have been modified to include only those pixels within the specified cutout size.
+The WCS-WAVE HDU has also modified to provide the correct mapping between the pixels in the cutout to wavelength.
+The PSF HDU from the original spectral image is included unmodified in the cutout MEF.
+
+The spatially-varying PSF is represented as an image cube with 121 planes.
+Each plane is a 101x101 pixel image representing a PSF for a different region of the detector. Users interested in performing photometry on a cutout using the information in the cutout PSF HDU will need to understand how to find the most applicable PSF cube plane for each pixel in the cutout.
+The basic steps are described below, and a [Python notebook tutorial](https://caltech-ipac.github.io/irsa-tutorials/) is provided to help users get started with a simple implementation.
+
+1. Determine the 0-based pixel coordinates of the position of interest in the IMAGE HDU of the cutout.
+
+2. Determine the 0-based pixel coordinates of the position of interest in the IMAGE HDU of the original Spectral Image.
+
+```
+xpix_orig = 1 + xpix_cutout - CRPIX1A
+ypix_orig = 1 + ypix_cutout - CRPIX2A
+```
+
+3. Examine the header of the PSF HDU of the cutout to determine the PSF zone and cube plane corresponding to the pixel of interest in the original Spectral Image.
+
+The PSF HDU has a header containing the keywords `XCTR_*`, `YCTR_*`, `XWID_*`, and `YWID_*`, where * goes from [1 to 121].
+To determine if a pixel in the original Spectral Image falls within a PSF zone, simply find the closest `XCTR_*` and `YCTR_*` to determine the cube plane that contains the corresponding PSF for this zone.
 
 ## Calibration Product: Absolute Gain Matrix