Improve doc

docs/_static/my_style.css

@@ -12,15 +12,11 @@
   text-decoration: line-through;
 }
 
-/* Hiding captions on the main page. Did not allow to add comments under the
-caption. Adding subtitles instead, and we don't have duplicate caption. But we
-still want the caption in the sidebar.
- */
-.wy-nav-content .caption-text {
-    display: none;
-}
+/* -------------------
+   Changes prepared by Emmanuelle Renauld:
+   -------------------*/
 
-/* Code shows in red when writing with `code`. Changing to gray */
+/* Code shows in red when writing with ``code``. Changing to gray */
 code.literal {
     color: dimgray !important;
 }
@@ -46,7 +42,7 @@ div class rst-content
         background-color: #eeefef;
 }
 
-/*Now tables colors are off*/
+/*Now tables colors are off. Fixing.*/
 table .row-odd {
     background-color: white
 }
@@ -55,6 +51,15 @@ table .row-even {
     background-color: #bfbdbd;
 }
 
+/* Creating a "centered" container.*/
+.centered {
+    text-align: center;
+}
+.centered table {
+    margin-left: auto;
+    margin-right: auto;
+}
+
 /* Adding the same padding to footers */
 footer {
     padding: 1em;

docs/for_developers/data_management/Advanced_data_containers.rst

@@ -6,7 +6,8 @@ The MultisubjectDataset
 Here is how our data is organized to allow torch to use them through Dataloaders. All of the following can be used as lazy instead. Then, the data is only loaded when needed.
 
 
-- **(Lazy)MultisubjectDataset**
+(Lazy)MultisubjectDataset
+-------------------------
 
     - subcontainers:
 
@@ -20,9 +21,10 @@ Here is how our data is organized to allow torch to use them through Dataloaders
 
     - methods:
 
-        - *.load_data()*: loads the training and validation sets.
+        - *.load_data()*: loads the training and validation sets (if not lazy), or loads only the information about each subject.
 
-- **Multisubjectsubset**
+Multisubjectsubset
+------------------
 
     - subcontainers:
 
@@ -36,12 +38,13 @@ Here is how our data is organized to allow torch to use them through Dataloaders
 
     - methods:
 
-        - *.get_volume()*: gets a specific mri volume (ID corresponds to the group ID in the config_file) from a specific subject.
+        - *.get_volume(id)*: gets a specific mri volume (contained in the hdf5) from a specific subject.
         - *.get_volume_verify_cache()*: same, but if data was lazy, checks the volume cache first. If it was not cached, loads it and sends it to the cache.
         - *.__getitem__()*: used by the dataloader. Does not do anything per say, simply returns the sampled streamline id. The batch sampler will do the job of actually loading the data.
 
 
-- **(Lazy)SubjectsDataList**
+(Lazy)SubjectsDataList
+----------------------
 
     - subcontainers:
 
@@ -59,14 +62,16 @@ Here is how our data is organized to allow torch to use them through Dataloaders
         - *.getitem_with_handle()*: same, but in the lazy case, adds a hdf5 handle first to allow loading. You probably won't need this method: used in the get_volume() method of the MultisubjectSubset.
 
 
-- **(Lazy)SubjectData**
+(Lazy)SubjectData
+-----------------
 
     - subcontainers:
 
         - .volume_list: list of **MRIData** (lazy or not)
         - .sft_data_list: list of **SFTData** (lazy or not)
 
     - other attributes:
+
         - .volume_groups, .streamline_group, .subject_id: general attributes found in the hdf5 file.
 
     - methods:
@@ -75,7 +80,8 @@ Here is how our data is organized to allow torch to use them through Dataloaders
         - *.with_handle()*: useful only in the lazy case. Adds hdf_handle to the subject to allow loading.
 
 
-**MRIData**
+MRIData
+-------
 
     - attributes:
 
@@ -89,7 +95,8 @@ Here is how our data is organized to allow torch to use them through Dataloaders
         - *.as_tensor()*: gets the data.
 
 
-**SFTData**
+SFTData
+-------
 
     - attributes:
 

docs/for_developers/data_management/BatchLoader.rst

@@ -3,4 +3,23 @@
 Batch loader
 ============
 
-toDo
+These classes define how batches of streamlines are loaded from a MultiSubjectDataset and how data augmentation is applied. Two main types of batch loaders are implemented:
+
+DWIMLStreamlinesBatchLoader
+---------------------------
+
+Loads augmented streamlines only (no MRI volumes). Loads streamlines from a dataset, applies optional data augmentation (resampling, cutting, reversing, noise), and returns them in voxel/corner space.
+
+Methods:
+    - ``set_context``: Sets whether the loader operates on training or validation data. Also configures which noise augmentation applies.
+    - ``load_batch_streamlines(streamline_ids_per_subj)``: Loads the streamlines for each subject, applies: resampling or compression, splitting, reversing, conversion to voxel + corner coordinates
+
+DWIMLBatchLoaderOneInput
+------------------------
+
+Child class of DWIMLStreamlinesBatchLoader. Additionnally communicates with the model to prepare input volume(s) under each point of each streamline and performs trilinear interpolation, and, optionnally, neighborhood extraction.
+
+Methods:
+    - ``load_batch_inputs(batch_streamlines, ids_per_subj)``
+
+Note: Must be used with a model with inputs, uses the model’s method: ``prepare_batch_one_input()``.

docs/for_developers/data_management/BatchSampler.rst

@@ -3,45 +3,11 @@
 Batch sampler
 =============
 
-These classes defines how to sample the streamlines available in the
-MultiSubjectData.
+These classes defines how to sample the streamlines available in the MultiSubjectData. You are encouraged to contribute to dwi_ml by adding any child class here.
 
-**AbstractBatchSampler:**
 
-- Defines the __iter__ method:
+DWIMLBatchIDSampler
+-------------------
 
-    - Finds a list of streamlines ids and associated subj that you can later load in your favorite way.
+- Defines the __iter__ method: It finds a list of streamlines ids and associated subjects that you can later load in your favorite way. It limits the number of subjects per batch and orders streamlines by subjects to make sure you don't need to load a full new volume at each new streamline.
 
-- Define the load_batch method:
-
-    - Loads the streamlines associated to sampled ids. Can resample them.
-
-    - Performs data augmentation (on-the-fly to avoid having to multiply data on disk) (ex: splitting, reversing, adding noise).
-
-Child class : **BatchStreamlinesSamplerOneInput:**
-
-- Redefines the load_batch method:
-
-    - Now also loads the input data under each point of the streamline (and possibly its neighborhood), for one input volume.
-
-You are encouraged to contribute to dwi_ml by adding any child class here.
-
-
-
-        - For instance, the BatchSequenceSampler creates batches of streamline ids that will be used for each batch iteratively through *__iter__*. Those chosen streamlines can then be loaded and processed with *load_batch*, which also uses data augmentation.
-
-        - For instance, the BatchSequencesSamplerOneInputVolume then uses the generated streamlines to load one underlying input for each timestep of each streamline.
-
-        - The BatchSampler uses the **MultiSubjectDataset** (or lazy)
-
-            - Creates a list of subjects. Using *self.load_data()*, it loops on all subjects (for either the training set or the validation set) and loads the data from the hdf5 (lazily or not).
-
-            - The list is a **DataListForTorch** (or lazy). It contains the subjects but also common parameters such as the feature sizes of each input.
-
-            - The elements are **SubjectData** (or lazy)
-
-                - They contain both the volumes and the streamlines, with the subject ID.
-
-                - Volumes are **MRIData** (or lazy). They contain the data and affine.
-
-                - Streamlines are **SFTData** (or lazy). They contain all information necessary to create a Stateful Tractogram from the streamlines. In the lazy version, streamlines are not loaded all together but read when needed from the **LazyStreamlinesGetter**.

docs/for_developers/models/directionGetters.rst

@@ -1,9 +1,9 @@
 .. _direction_getters:
 
-Create a tractography model: use a DirectionGetter
-==================================================
+The DirectionGetter Layer
+=========================
 
-Direction getter layers should be used as last layer of any streamline generation model for the tractography task. They define the format of the output and possible associated loss functions.
+``DirectionGetter`` layers should be used as last layer of any streamline generation model for the tractography task. They define the format of the output and possible associated loss functions.
 
 General architecture
 --------------------
@@ -21,6 +21,8 @@ Regression models
 
 These models use regression to learn directly a direction, formatted as a coordinate [x, y, z]. If EOS is used, then the direction is formatted as [x, y, z, eos], where eos is a probability between 0 and 1.
 
+.. container:: centered
+
         +------------------------+---------------------------------+
         | Shape of the output    | A vector of length 3 or 4       |
         +------------------------+---------------------------------+
@@ -51,6 +53,7 @@ Classification models
 
 This model uses classification by formatting directions as a choice of direction among a list of discrete points on the sphere (Ex: ``dipy.data.get_sphere('symmetric724')``). Each point is a class. If EOS is used, it represents an additional class.
 
+.. container:: centered
 
         +------------------------+----------------------------------+
         | Shape of the output    | A vector of length K (nb class)  |
@@ -80,6 +83,8 @@ This is a regression model that learns parameters representing a *probability fu
 
 - **SingleGaussianDG**: the model is a 2-layer NN for the means and a 2-layer NN for the variances. The output is 6 parameters: 3 means (x, y, z) and 3 variances (x, y, z). If EOS is used, it is a 7th learned value.
 
+.. container:: centered
+
         +------------------------+-------------------------------------------------+
         | Shape of the output    | A vector of length 6 or 7                       |
         +------------------------+-------------------------------------------------+
@@ -92,6 +97,8 @@ This is a regression model that learns parameters representing a *probability fu
 
 - **GaussianMixtureDG**: In this case, the models learns to represent the function probability as a mixture of N Gaussians, possibly representing direction choices in the case of fiber crossing and other special configurations. The loss is again the negative log-likelihood. Note that the model is a 2-layer NN for the mean and a 2-layer NN for the variance, for each of N Gaussians. The output is N * (6 parameters: 3 means (x, y, z) and 3 variances (x, y, z) plus a mixture parameter for each, giving the probability that the right direction would be given by this Gaussian.
 
+.. container:: centered
+
         +------------------------+-------------------------------------------------+
         | Shape of the output    | A vector of length N * (7 or 8)                 |
         +------------------------+-------------------------------------------------+
@@ -109,6 +116,12 @@ Note that tyically, in the literature, Gaussian mixtures are used with expectati
 .. [1] Some code is available `in this blog <See for example here https://blog.dominodatalab.com/fitting-gaussian-process-models-python/>`_.
 .. [2] See the `Tractoinferno paper <https://www.nature.com/articles/s41597-022-01833-1>`_
 
+.. toctree::
+    :maxdepth: 1
+    :caption: Full formulas (Gaussian + FvM)
+    :hidden:
+
+    formulas_fisher_gaussian
 
 
 Fisher von mises models
@@ -120,6 +133,7 @@ See the detailed mathematics in :ref:`ref_formulas`.
 
 - **FisherVonMisesDG**: The loss is the negative log-likelihood. Note that the model is a 2-layer NN for the means and a 2-layer NN for the variances. See :ref:`ref_formulas` for the complete formulas. The output is 4 parameters: 3 for the means and one for kappa. If EOS is used, it is a 5th learned value.
 
+.. container:: centered
 
         +------------------------+-------------------------------------------------+
         | Shape of the output    | A vector of length 4 or 5.                      |
@@ -133,12 +147,6 @@ See the detailed mathematics in :ref:`ref_formulas`.
 
 **FisherVonMisesMixtureDG**: Not implemented yet.
 
-Other ideas
-'''''''''''
-
-An equivalent model could learn to represent the direction on the sphere to learn a normalized direction. The means would be 2D (phi, rho), and the variances too. This has not been implemented yet.
-
-
 %%%%%%%%%%%%%%%%%%%
 
 .. [3] Directional Statistics (Mardia and Jupp, 1999)), implemented `here <https://github.com/jasonlaska/spherecluster>`_.

docs/for_developers/models/formulas_fisher_gaussian.rst

@@ -10,9 +10,9 @@ Log-likelihood of a single Gaussian
 
     - N(x) = The normal distribution, with mean mu and variance sigma^2.
     - C = the covariance matrix. It indicates the relations between each all dimensions x,y,z. C is diagonal if the axis are independant, with the variances on the diagonal.
-    - d = the dimension of data (here, 3: x,y,z)
+    - d = the dimension of data (here, 3: [x,y,z])
 
-    We do pose that the axis are independant. WHY CAN WE POSE THAT????
+    We pose that the axis are independant. We assume the x, y, and z components are independent because the model predicts each directional axis separately, and there is no prior reason to expect correlations between them in the learned representation. This simplifies the covariance structure to a diagonal matrix without significantly affecting the model’s performance.
 
 **Formulas:**
 

docs/for_developers/models/index.rst

@@ -1,7 +1,9 @@
 .. _create_your_model:
 
-Create your own model
-=====================
+Creating your own model
+=======================
+
+You are welcome to add your own project's model in dwi_ml!
 
     .. toctree::
         :maxdepth: 1
@@ -11,4 +13,3 @@ Create your own model
         main_abstract_model
         other_main_models
         directionGetters
-        formulas_fisher_gaussian

docs/for_developers/models/main_abstract_model.rst

@@ -1,11 +1,9 @@
 .. _main_abstract_model:
 
-Create your own model: use MainModelAbstract
-============================================
+Inherit from MainModelAbstract
+==============================
 
-You are welcome to add your own project's model in dwi_ml!
-
-Projects using diffusion imaging and tractography streamlines are usually quite heavy in memory. For this reason, dwi_ml is not only a space for models, but it also includes smart management of data loading during training (see :ref:`trainers` for more information). Our training objects and model objects are thus intertwined. For this reason, you should always make your model a child class of our **MainModelAbstract**.
+Projects using diffusion imaging and tractography streamlines are usually quite heavy in memory. For this reason, dwi_ml is not only a space for models, but it also includes smart management of data loading during training. Our training objects (see :ref:`trainers` for more information) and model objects are thus intertwined. For this reason, you should always make your model a child class of our **MainModelAbstract**.
 
 
 The MainModelAbstract class
@@ -48,17 +46,18 @@ Our model:
 Where to start?
 ---------------
 
-As a first step, try to implement a child class of ``dwi.models.main_models.MainModelAbstract`` and see how you would implement the two following methods: ``forward()`` (the core of your model) and ``compute_loss()``.
-
-Once you are more experimented, you can explore how to use methods already implemented in dwi_ml to improve your model, using, for instance, our models used for tractographie generation, or our models able to add a neighborhood to each input point, etc.
+As a first step, try to implement a child class of ``dwi.general.models.main_models.main_abstract_model.MainModelAbstract`` and see how you would implement the two following methods: ``forward()`` (the core of your model) and ``compute_loss()``.
 
-1. Create a new file in ``src/dwi_ml/models/projects`` named my_project.py
+1. Create a new file in ``src/dwi_ml/projects/my_project`` named my_project.py
 
 2. Start your project like this:
 
             .. image:: /_static/images/create_your_model.png
+               :align: center
                :width: 500
 
 3. Learn to use your model in our Trainer (see page :ref:`trainers`).
 
-4. Before coding everything from scratch in our model, verify if it could inherit from our other models (see page :ref:`other_main_models`) to benefit from their methods.
+4. Discover the optional parameters of the ``MainModelAbstract`` model.
+
+5. Before coding everything else from scratch in our model, verify if it could inherit from our other models (see page :ref:`other_main_models`) to benefit from their methods.