Mem3DG converter

Justfile

@@ -23,7 +23,7 @@ clean:
 # install with all deps (and setup conda env with readdy)
 install:
 	conda env update --file environment.yml
-	pip install -e .[lint,test,docs,dev,mcell,physicell,md,cellpack]
+	pip install -e .[lint,test,docs,dev,mcell,physicell,md,cellpack,mem3dg]
 
 # lint, format, and check all files
 lint:

examples/Tutorial_mem3dg.ipynb

@@ -0,0 +1,170 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Simularium Conversion Tutorial : Mem3DG .nc Data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from IPython.display import Image\n",
+    "\n",
+    "import numpy as np\n",
+    "\n",
+    "from simulariumio.mem3dg import Mem3dgConverter, Mem3dgData\n",
+    "from simulariumio import MetaData, CameraData, UnitData"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This notebook provides example python code for converting your own simulation trajectories into the format consumed by the Simularium Viewer. It creates a .simularium file which you can drag and drop onto the viewer like this:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "![title](img/drag_drop.gif)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# _Note:_\n",
+    "To install simulariumio with all depencies needed for Mem3DG conversion, use `pip install simulariumio[mem3dg]`"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "***\n",
+    "## Prepare your spatial data"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The Simularium `Mem3dgConverter` consumes spatiotemporal data from Mem3DG .nc output files using the netCDF4 Python package. \n",
+    "\n",
+    "The converter requires a `Mem3dgData` object as a parameter.\n",
+    "\n",
+    "Unlike other simulariumio converters, the `Mem3dgConverter` will generate several `.obj` files (one per frame) in addition to the `.simularium` file. Use `output_obj_file_path` to specify where the generated obj files should be saved.\n",
+    "\n",
+    "The test input .nc data for this example was created using [Mem3DG's Jupyter Notebook tutorials](https://github.com/RangamaniLabUCSD/Mem3DG/blob/main/tests/python/tutorial/tutorial1.ipynb)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "box_size = 5.\n",
+    "scale_factor = 10\n",
+    "\n",
+    "example_data = Mem3dgData(\n",
+    "    input_file_path=\"../simulariumio/tests/data/mem3dg/traj.nc\",\n",
+    "    output_obj_file_path=\".\",\n",
+    "    meta_data=MetaData(\n",
+    "        box_size=np.array([box_size, box_size, box_size]),\n",
+    "        trajectory_title=\"Some parameter set\",\n",
+    "        camera_defaults=CameraData(position=np.array([0, 0, 200])),\n",
+    "        scale_factor=scale_factor\n",
+    "    ),\n",
+    "    agent_color=\"#a38fba\",\n",
+    "    agent_name=\"my-object\",\n",
+    "    time_units=UnitData(\"us\", 0.2),\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Convert and save as .simularium file"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Once your data is shaped like in the `example_data` object, you can use the converter to generate the file at the given path"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Reading Mem3DG Data -------------\n",
+      "Converting Trajectory Data to JSON -------------\n",
+      "Writing JSON -------------\n",
+      "saved to example_mem3dg.simularium\n"
+     ]
+    }
+   ],
+   "source": [
+    "converter = Mem3dgConverter(example_data).save(\"example_mem3dg\", binary=False)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Visualize in the Simularium viewer"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In a supported web-browser (Firefox or Chrome), navigate to https://simularium.allencell.org/ and import your file into the view.\n",
+    "\n",
+    "**Note:** In order to view your data in simularium, you must import the generated .simularium file _and_ all of the generated .obj files!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.16"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

pyproject.toml

@@ -57,6 +57,9 @@ cellpack = [
 nerdss = [
   "MDAnalysis>=2.0.0",
 ]
+mem3dg = [
+  "netCDF4",
+]
 tutorial = [
   "jupyter",
   "scipy>=1.5.2",

simulariumio/mem3dg/__init__.py

@@ -0,0 +1,5 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+from .mem3dg_converter import Mem3dgConverter  # noqa: F401
+from .mem3dg_data import Mem3dgData  # noqa: F401

simulariumio/mem3dg/mem3dg_converter.py

@@ -0,0 +1,104 @@
+from netCDF4 import Dataset
+import numpy as np
+from pathlib import Path
+
+from ..trajectory_converter import TrajectoryConverter
+from ..data_objects import (
+    AgentData,
+    TrajectoryData,
+    DimensionData,
+    DisplayData,
+)
+from ..constants import DISPLAY_TYPE
+from ..exceptions import InputDataError
+from .mem3dg_data import Mem3dgData
+
+
+class Mem3dgConverter(TrajectoryConverter):
+    def __init__(
+        self,
+        input_data: Mem3dgData,
+    ):
+        """
+        Parameters
+        ----------
+        input_data : Mem3dgData
+            An object containing info for reading
+            Mem3DG simulation trajectory output
+        """
+        self._data = self._read(input_data)
+
+    def write_to_obj(self, filepath, data, frame):
+        # Extract XYZ coordinates for vertices
+        coordinates = np.array(
+            data.groups["Trajectory"].variables["coordinates"][frame]
+        )
+        coordinates = np.reshape(coordinates, (-1, 3))
+
+        # Extract indices of vertices to make faces (all triangles)
+        topology = np.array(data.groups["Trajectory"].variables["topology"][frame])
+        topology = np.reshape(topology, (-1, 3))
+        # change indices to be 1 indexed instead of 0 indexed for .obj files
+        topology += 1
+
+        # Generate one .obj file per frame
+        with open(filepath, "w") as file:
+            file.write(f"# Frame {frame}\n")
+            for v in coordinates:
+                file.write(f"v {v[0]} {v[1]} {v[2]}\n")
+            for t in topology:
+                file.write(f"f {t[0]} {t[1]} {t[2]}\n")
+
+    def _read_traj_data(self, input_data: Mem3dgData) -> AgentData:
+        try:
+            data = Dataset(input_data.input_file_path, "r")
+            n_frames = np.size(data.groups["Trajectory"].variables["time"])
+        except Exception as e:
+            raise InputDataError(f"Error reading input Mem3DG data: {e}")
+
+        # for now, we are representing converted Mem3DG trajectories as one
+        # unique mesh agent per frame
+        dimensions = DimensionData(total_steps=n_frames, max_agents=1)
+        agent_data = AgentData.from_dimensions(dimensions)
+        agent_data.n_timesteps = n_frames
+
+        base_agent_name = input_data.agent_name or "object"
+        for frame in range(n_frames):
+            agent_data.times[frame] = data.groups["Trajectory"].variables["time"][frame]
+            agent_data.n_agents[frame] = 1
+
+            output_file_path = Path(input_data.output_obj_file_path) / f"{frame}.obj"
+            self.write_to_obj(output_file_path, data, frame)
+
+            agent_data.radii[frame][0] = input_data.meta_data.scale_factor
+            agent_data.unique_ids[frame][0] = frame
+
+            name = str(frame)
+            agent_data.types[frame].append(name)
+            object_display_data = DisplayData(
+                name=f"{base_agent_name}#frame{frame}",
+                display_type=DISPLAY_TYPE.OBJ,
+                url=f"{frame}.obj",
+                color=input_data.agent_color,
+            )
+            agent_data.display_data[name] = object_display_data
+        return agent_data
+
+    def _read(self, input_data: Mem3dgData) -> TrajectoryData:
+        """
+        Return a TrajectoryData object containing the Mem3DG data
+        """
+        print("Reading Mem3DG Data -------------")
+        if input_data.meta_data.scale_factor is None:
+            input_data.meta_data.scale_factor = 1.0
+        agent_data = self._read_traj_data(input_data)
+        input_data.spatial_units.multiply(1.0 / input_data.meta_data.scale_factor)
+        input_data.meta_data._set_box_size()
+        result = TrajectoryData(
+            meta_data=input_data.meta_data,
+            agent_data=agent_data,
+            time_units=input_data.time_units,
+            spatial_units=input_data.spatial_units,
+            plots=input_data.plots,
+        )
+        return result

simulariumio/mem3dg/mem3dg_data.py

@@ -0,0 +1,65 @@
+from ..data_objects import MetaData, UnitData
+from typing import List, Dict, Any
+
+
+class Mem3dgData:
+    input_file_path: str
+    output_obj_file_path: str
+    meta_data: MetaData
+    agent_name: str
+    agent_color: str
+    time_units: UnitData
+    spatial_units: UnitData
+    plots: List[Dict[str, Any]]
+
+    def __init__(
+        self,
+        input_file_path: str,
+        output_obj_file_path: str = None,
+        meta_data: MetaData = None,
+        agent_name: str = None,
+        agent_color: str = None,
+        time_units: UnitData = None,
+        spatial_units: UnitData = None,
+        plots: List[Dict[str, Any]] = None,
+    ):
+        """
+        Parameters
+        ----------
+        input_file_path : str
+            The path to the .nc file output by Mem3DG for this trajectory.
+        output_obj_file_path : str (optional)
+            The path to the directory where output .obj files will be saved
+            to. If nothing is provided, the output .obj files will be saved
+            to the current directory.
+        meta_data : MetaData
+            An object containing metadata for the trajectory
+            including box size, scale factor, and camera defaults
+        agent_name: str (optional)
+            This converter generates it's own DisplayData, but the agent name
+            can optionally be overridden here. This will change the agent
+            name that is displayed on the side column in simularium
+        agent_color: string (optional)
+            This converter generates it's own DisplayData, but the agent color
+            can optionally be overridden here with a hex value for the color to
+            display, e.g "#FFFFFF"
+            Default: Use default colors from Simularium Viewer
+        time_units: UnitData (optional)
+            multiplier and unit name for time values
+            Default: 1.0 second
+        spatial_units: UnitData (optional)
+            multiplier and unit name for spatial values
+            (including positions, radii, and box size)
+            Default: 1.0 meter
+        plots : List[Dict[str, Any]] (optional)
+            An object containing plot data already
+            in Simularium format
+        """
+        self.input_file_path = input_file_path
+        self.output_obj_file_path = output_obj_file_path or "."
+        self.meta_data = meta_data or MetaData()
+        self.agent_name = agent_name
+        self.agent_color = agent_color
+        self.time_units = time_units or UnitData("s")
+        self.spatial_units = spatial_units or UnitData("m")
+        self.plots = plots or []