Add validation and optimization pages. Fixes #30.

Author: Travis Jacobs

docs/digitization.md

@@ -1,13 +1,11 @@
 Digitization
 ============
 
-Digitized figures from papers in the literature will be used to validate and refine the ion channel models of ChannelWorm.
+Digitized figures from papers in the literature will be used to validate the ion channel models of ChannelWorm.
 
 For example, a study which conducts [patch clamp recordings](https://en.wikipedia.org/wiki/Patch_clamp) will produce data that describe electrical properties of ion channels.
 This data is valuable to anyone modeling these ion channels; we can use actual ion channel data to guide our efforts toward biological realism, and incorporate it in [confirming that we are modeling what we are intending to](validation/).
 
-If you want to help digitize some figures for this part of the OpenWorm project, check out the [detailed walkthrough](#Walkthrough-of-the-process) below.
-
 ## So why digitization?
 
 In a published paper, data is included in an expressive form, such as a visual plot, rather than as raw numerical data. This is not to say that we cannot *extract* the information and make use of it, however.
@@ -30,12 +28,9 @@ These image files can then be processed by [digitization software](http://arohat
 
 ## Walkthrough of the process
 
-1. Select a paper.
-    * For example, let's use [this Jospin et al. article](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2173050/).
-    * [Here](https://drive.google.com/folderview?id=0B4qffTA1q81rfmhoZzBva2xKRTJIVTlhR2plYTF2S2JCNXBLODJWN09KUHU4X09FN0hEQlE&usp=sharing) is a list of papers *you* can choose from
-2. Select the figures that describe ion channel properties.
-    * In our example, [figure 2B](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2173050/figure/fig2/) is looking promising.
-    * There will likely be a number of relevant figures in each paper you process.
+1. Select a paper and a figure from the list.
+    * For example, let's use [figure 2B](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2173050/figure/fig2/) from [this Jospin et al. article](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2173050/).
+    * [Here](https://docs.google.com/spreadsheets/d/1jTXDHsLsdK-T_d-RwCb4gmQgMhjumvZE7C_2PEXFYAk/edit#gid=0) is a Google sheet you can choose from
 3. Download the image, and crop the relevant portion using image editing software.
     * In our example case we just right-click on the image, save it, and crop out 2B.
       * [The result](https://cloud.githubusercontent.com/assets/7369273/7684968/e5d974c4-fd64-11e4-9cf0-fae6656bc4eb.png)
@@ -46,8 +41,10 @@ These image files can then be processed by [digitization software](http://arohat
 6. Now enter the values you selected on the axes.
     * In our case, the values [looked like this](https://cloud.githubusercontent.com/assets/7369273/7684983/03f827fc-fd65-11e4-8dc0-f433000a24ed.png)
 7. Now add a data point for each data point on the graph.
+    * Again, be sure to use the magnifying glass to place the points accurately.
     * Your plot should look something like [this](https://cloud.githubusercontent.com/assets/6655104/6027361/b509831c-abf6-11e4-95a4-372b911533fb.png) by now.
 8. Finally, you can click "view data > graph in Plotly" to get something easily sharable with the community.
+    * Paste the link to your Plotly graph back into the Google sheet
     * **Tada!**
 
 <div>

docs/optimization.md

@@ -0,0 +1,35 @@
+Optimization
+============
+
+As described in the [general OpenWorm documentation](http://docs.openworm.org/en/latest/Projects/optimization/), optimization is being employed at several levels of the project. By tuning the parameters of the models to experimental data, a high degree of biological realism can be attained.
+
+## How ChannelWorm uses optimization
+
+[![](https://docs.google.com/drawings/d/13JvpUktlTXN2GKH9fXzacXQWudm5MQUMXXY94cr6S50/pub?w=778&h=370)](https://docs.google.com/drawings/d/13JvpUktlTXN2GKH9fXzacXQWudm5MQUMXXY94cr6S50/edit)
+
+At a high level, the modeling-validation-optimization loop relies on the optimization process to consume inaccurate models and return more accurate models, given some experimental objective data.
+
+A model is chosen for optimization if it fails the [validation stage](validation/) of this loop. Inside the optimization process, the model's parameters are then tuned to minimize their difference with the experimental data used for validation.
+
+A new model with slightly different parameters is produced, and is pulled into the validation stage once again. This loop continues until the validation stage is passed, and the model is finally [stored](information-management/#data-management).
+
+## The process in detail
+
+*Note:*Steps that are not yet streamlined will be tagged with their relevant Github issues. The details of these steps will be added as the issues are resolved.
+
+### Inputs
+
+* A hook must be established to import models that fail [validation](validation/)
+* Will the optimization itself take place locally or on a remote machine?
+
+### Optimization
+
+* Selection of free parameters
+* Selection of training data from simulated data
+* What type of optimization algorithm will be used? There is material [here](https://drive.google.com/open?id=0B_t3mQaA-HaMbXA1M0s1a25KSTA&authuser=0) about genetic algorithms, and information [here](https://optimal-neuron.readthedocs.org/en/latest/architecture.html) on NeuroTune
+* Scripts to perform the optimization will differ depending on the above choice
+
+### Outputs
+
+* Update the model's parameters with the results of the optimization
+* Save the new model in a form that can be simulated again, to continue the loop

docs/validation.md

@@ -0,0 +1,40 @@
+Validation
+==========
+
+The process of validation is *central* to OpenWorm's tenet of modeling with biological realism.
+Only through comparing data produced by our simulations with data from the actual organism can we make the claim that we are modeling what we say we are.
+
+This is as true for small-scale components of the worm as it is for the worm itself, and is thus important for ChannelWorm's goal of modeling *C. elegans* ion channels.
+
+## The process - High level
+
+[![](https://docs.google.com/drawings/d/13JvpUktlTXN2GKH9fXzacXQWudm5MQUMXXY94cr6S50/pub?w=778&h=370)](https://docs.google.com/drawings/d/13JvpUktlTXN2GKH9fXzacXQWudm5MQUMXXY94cr6S50/edit)
+
+The modeling-validation-optimization loop is outlined in the diagram above, but the focus of this article is the validation step (green in the diagram).
+
+At its core, the validation process relies on **comparing two sets of data**, one being the output of our [*simulated* ion channel experiments](simulation/), the other from *actual* ion channel experiments (ex: [digitized I/V plots](digitization/)).
+
+Using the [SciUnit](https://github.com/scidash/sciunit) framework, tests are constructed to judge whether simulations of our models are close enough, on various parameters, to the observed data.
+
+Depending on the results of these validation tests, the model will either be accepted or rejected. If the model is accepted, we [store it in a database](information-management/#data-management) with the supporting experimental evidence. If the model is rejected, we subject it to [further optimization](optimization/) and another round of simulation/validation testing. This loop continues until validation is passed, and the model is kept.
+
+## Implementation
+
+*Note:*Steps that are not yet streamlined will be tagged with their relevant Github issues. The details of these steps will be added as the issues are resolved.
+
+### Inputs
+
+* [Ion channel model data must be comparable to experimental data](https://github.com/VahidGh/ChannelWorm/issues/28)
+* Model and experimental data should be pulled from the database (PyOpenWorm)
+
+### Tests
+
+* [A list of tests that can be run on the collection of models](https://github.com/VahidGh/ChannelWorm/issues/32)
+* [A validation test template including testing (CI) environment](https://github.com/VahidGh/ChannelWorm/issues/27)
+* Comparison of two sets of data using statistical tests, with numerical output
+
+### Outputs
+
+* Comparison of test output with "acceptable range", to determine whether the model is a good fit
+* Ability to export data to database
+* Ability to export data to optimization process

mkdocs.yml

@@ -5,3 +5,5 @@ pages:
   - [index.md, Introduction]
   - [information-management.md, Information Management]
   - [digitization.md, Digitization]
+  - [validation.md, Validation]
+  - [optimization.md, Optimization]