Export Full GuPPy Analysis Outputs to Integrated NWB Files

[pauladkisson]

Summary

The lab requires all analysis outputs — raw photometry traces, processed signals (z-score, dF/F), event timestamps (TTLs and externally imported behavioral events), and transient detections — exported as a single integrated NWB file per session. GuPPy currently outputs bespoke HDF5 and CSV files; no NWB conversion exists. NWB export is a requirement of the ASAP program that funds this work, and the lab specifically emphasized that photometry and behavioral data must be co-located in the same file.

Motivation

• Alexandra Nelson stated the lab's priority clearly: "we're very interested in making sure outputs are in .nwb" and confirmed that having them as a "whole package" — photometry plus raw behavioral data — is the key requirement
• Xiaowen Zhuang asked explicitly: "is it possible to convert for NWB as a whole package, including the photometry as well as the raw behavior data, so that it comes in as a whole?"
• Alexandra followed up: "to try to make it interoperable for other people, you'd like to have all the data streams integrated" — NWB interoperability for downstream users is the stated goal, not just archival
• Venus confirmed NWB export is in progress but not yet implemented: "our NWB code is still in progress, we haven't done it yet, but eventually that's the goal"
• The ASAP program has NWB as a data sharing standard; this makes NWB export a program-level requirement, not a lab-specific request
• CatalystNeuro (the organization developing GuPPy with the Lerner lab) maintains neuroconv, an existing NWB conversion toolbox, which may provide reusable infrastructure

Proposed Solution

• Add an src/guppy/exporters/nwb_exporter.py module following the same extractor/exporter pattern as src/guppy/extractors/
• Use pynwb and the ndx-fiber-photometry NWB extension to represent photometry data in the community-standard schema:
  • Raw fluorescence traces as FiberPhotometryResponseSeries (or TimeSeries if the extension is not used)
  • dF/F and z-score as additional TimeSeries objects referencing the same fiber/channel metadata
  • Event timestamps (TTLs, imported behavioral timestamps) as TimeIntervals or AnnotationSeries objects in the same file
  • Transient peak times and amplitudes as a TimeIntervals table if Step 5 has been run
• NWB session-level metadata (subject ID, genotype, experimenter, institution, session description, implant location) should be configurable via a new metadata section in GuPPyParamtersUsed.json and a corresponding GUI form
• Add a step7_export_nwb keyword-only function to src/guppy/testing/api.py following the existing step function pattern
• Add a GUI step for invoking NWB export, accessible after Step 5 is complete
• Produce one NWB file per session (the NWB convention for single-session data), placed in the session folder alongside existing outputs

Open Questions

• Should the NWB export use the ndx-fiber-photometry community extension (actively maintained and semantically richer) or base pynwb TimeSeries objects (no additional dependency, more broadly compatible)? The extension is preferred for interoperability but requires users to have it installed
• Should NWB export reuse infrastructure from CatalystNeuro's neuroconv tool, or be implemented standalone within GuPPy? Reuse reduces maintenance burden but adds a dependency; standalone keeps GuPPy self-contained
• How should multi-fiber / multi-channel recordings be represented in the NWB schema? The ndx-fiber-photometry extension has explicit support for multi-fiber setups, but the mapping from GuPPy storenames to NWB fiber/channel objects needs to be defined
• Should export fail with a clear error if required NWB metadata fields (subject, institution) are not filled in, or produce a valid but incomplete NWB file with a warning?
• How should sessions be linked if a lab runs multi-session experiments (e.g., baseline + drug + washout across days)? NWB supports subject as a cross-session identifier; GuPPy would need a subject ID field to enable this
• Should behavioral data imported via the external timestamp wrapper (Issue 04) be automatically included in NWB export, and if so, how should it be labeled and attributed in the NWB file?