Skip to content

Commit

Permalink
Merge pull request #746 from ArnoStrouwen/canon
Browse files Browse the repository at this point in the history
canonize docs
  • Loading branch information
ChrisRackauckas authored Oct 22, 2022
2 parents 133e08b + 1efac7e commit 8f02ca7
Show file tree
Hide file tree
Showing 16 changed files with 45 additions and 45 deletions.
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
# SciMLSensitivity.jl

[![Join the chat at https://julialang.zulipchat.com #sciml-bridged](https://img.shields.io/static/v1?label=Zulip&message=chat&color=9558b2&labelColor=389826)](https://julialang.zulipchat.com/#narrow/stream/279055-sciml-bridged)
[![Stable](https://img.shields.io/badge/docs-stable-blue.svg)](http://sensitivity.sciml.ai/stable/)
[![Global Docs](https://img.shields.io/badge/docs-SciML-blue.svg)](https://docs.sciml.ai/dev/modules/SciMLSensitivity/)
[![Global Docs](https://img.shields.io/badge/docs-SciML-blue.svg)](https://docs.sciml.ai/SciMLSensitivity/stable/)

[![codecov](https://codecov.io/gh/SciML/SciMLSensitivity.jl/branch/master/graph/badge.svg)](https://codecov.io/gh/SciML/SciMLSensitivity.jl)
[![Build Status](https://github.com/SciML/SciMLSensitivity.jl/workflows/CI/badge.svg)](https://github.com/SciML/SciMLSensitivity.jl/actions?query=workflow%3ACI)
Expand All @@ -13,10 +12,11 @@

SciMLSensitivity.jl is a component package in the [SciML Scientific Machine Learning ecosystem](https://sciml.ai/).
It holds the sensitivity analysis utilities. Users interested in using this
functionality should check out [DifferentialEquations.jl](https://github.com/JuliaDiffEq/DifferentialEquations.jl).
functionality should check out [DifferentialEquations.jl](https://docs.sciml.ai/DiffEqDocs/stable/).

## Tutorials and Documentation

For information on using the package, see the [SciMLSensitivity](https://docs.sciml.ai/dev/modules/SciMLSensitivity/) section of the
[SciML docs](docs.sciml.ai). For information on specific previous versions of this package, see the
[see the stable SciMLSensitivity-only documentation](https://sensitivity.sciml.ai/stable/).
For information on using the package,
[see the stable documentation](https://docs.sciml.ai/SciMLSensitivity/stable/). Use the
[in-development documentation](https://docs.sciml.ai/SciMLSensitivity/dev/) for the version of
the documentation, which contains the unreleased features.
2 changes: 1 addition & 1 deletion docs/make.jl
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ makedocs(sitename = "SciMLSensitivity.jl",
# :autodocs_block, :cross_references, :docs_block, :eval_block, :example_block, :footnote, :meta_block, :missing_docs, :setup_block
],
format = Documenter.HTML(assets = ["assets/favicon.ico"],
canonical = "https://sensitivity.sciml.ai/stable/"),
canonical = "https://docs.sciml.ai/SciMLSensitivity/stable/"),
pages = pages)

deploydocs(repo = "github.com/SciML/SciMLSensitivity.jl.git";
Expand Down
2 changes: 1 addition & 1 deletion docs/src/ad_examples/adjoint_continuous_functional.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ sol = solve(prob,DP8())

gives a continuous solution `sol(t)` with the derivative at each time point. This
can then be used to define a continuous cost function via
[Integrals.jl](https://github.com/SciML/Integrals.jl), though the derivative would
[Integrals.jl](https://docs.sciml.ai/Integrals/stable/), though the derivative would
need to be defined by hand using the extra sensitivity terms.

## Example: Continuous Adjoints on an Energy Functional
Expand Down
8 changes: 4 additions & 4 deletions docs/src/ad_examples/differentiating_ode.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

This tutorial assumes familiarity with DifferentialEquations.jl
If you are not familiar with DifferentialEquations.jl, please consult
[the DifferentialEquations.jl documentation](https://diffeq.sciml.ai/stable/)
[the DifferentialEquations.jl documentation](https://docs.sciml.ai/DiffEqDocs/stable/)

In this tutorial we will introduce how to use local sensitivity analysis via
automatic differentiation. The automatic differentiation interfaces are the
Expand Down Expand Up @@ -38,7 +38,7 @@ differentiation methods.
Let's say we need the derivative of the solution with respect to the initial condition
`u0` and its parameters `p`. One of the simplest ways to do this is via ForwardDiff.jl.
To do this, all that one needs to do is use
[the ForwardDiff.jl library](https://github.com/JuliaDiff/ForwardDiff.jl) to differentiate
[the ForwardDiff.jl library](https://juliadiff.org/ForwardDiff.jl/stable/) to differentiate
some function `f` which uses a differential equation `solve` inside of it. For example,
let's say we want the derivative of the first component of ODE solution with respect to
these quantities at evenly spaced time points of `dt = 1`. We can compute this via:
Expand Down Expand Up @@ -67,11 +67,11 @@ solution at time `t=1` with respect to `p[1]`.

!!! note

Since [the global error is 1-2 orders of magnitude higher than the local error](https://diffeq.sciml.ai/stable/basics/faq/#What-does-tolerance-mean-and-how-much-error-should-I-expect), we use accuracies of 1e-6 (instead of the default 1e-3) to get reasonable sensitivities
Since [the global error is 1-2 orders of magnitude higher than the local error](https://docs.sciml.ai/DiffEqDocs/stable/basics/faq/#What-does-tolerance-mean-and-how-much-error-should-I-expect), we use accuracies of 1e-6 (instead of the default 1e-3) to get reasonable sensitivities

## Reverse-Mode Automatic Differentiation

[The `solve` function is automatically compatible with AD systems like Zygote.jl](https://diffeq.sciml.ai/latest/analysis/sensitivity/)
[The `solve` function is automatically compatible with AD systems like Zygote.jl](https://docs.sciml.ai/SciMLSensitivity/stable/)
and thus there is no machinery that is necessary to use other than to put `solve` inside of
a function that is differentiated by Zygote. For example, the following computes the solution
to an ODE and computes the gradient of a loss function (the sum of the ODE's output at each
Expand Down
2 changes: 1 addition & 1 deletion docs/src/dae_fitting/physical_constraints.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Enforcing Physical Constraints via Universal Differential-Algebraic Equations

As shown in the [stiff ODE tutorial](https://docs.juliadiffeq.org/latest/tutorials/advanced_ode_example/#Handling-Mass-Matrices-1),
As shown in the [stiff ODE tutorial](https://docs.sciml.ai/SciMLTutorialsOutput/stable/advanced/02-advanced_ODE_solving/#Handling-Mass-Matrices),
differential-algebraic equations (DAEs) can be used to impose physical
constraints. One way to define a DAE is through an ODE with a singular mass
matrix. For example, if we make `Mu' = f(u)` where the last row of `M` is all
Expand Down
2 changes: 1 addition & 1 deletion docs/src/hybrid_jump_fitting/bouncing_ball.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Bouncing Ball Hybrid ODE Optimization

The bouncing ball is a classic hybrid ODE which can be represented in
the [DifferentialEquations.jl event handling system](https://diffeq.sciml.ai/stable/features/callback_functions/). This can be applied to ODEs, SDEs, DAEs, DDEs,
the [DifferentialEquations.jl event handling system](https://docs.sciml.ai/DiffEqDocs/stable/features/callback_functions/). This can be applied to ODEs, SDEs, DAEs, DDEs,
and more. Let's now add the DiffEqFlux machinery to this
problem in order to optimize the friction that's required to match
data. Assume we have data for the ball's height after 15 seconds. Let's
Expand Down
2 changes: 1 addition & 1 deletion docs/src/hybrid_jump_fitting/hybrid_diffeq.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Training Neural Networks in Hybrid Differential Equations

Hybrid differential equations are differential equations with implicit or
explicit discontinuities as specified by [callbacks](https://diffeq.sciml.ai/stable/features/callback_functions/).
explicit discontinuities as specified by [callbacks](https://docs.sciml.ai/DiffEqDocs/stable/features/callback_functions/).
In the following example, explicit dosing times are given for a pharmacometric
model and the universal differential equation is trained to uncover the missing
dynamical equations.
Expand Down
22 changes: 11 additions & 11 deletions docs/src/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ Thus, what SciMLSensitivity.jl provides is:

This documentation assumes familiarity with the solver packages for the respective problem
types. If one is not familiar with the solver packages, please consult the documentation
for pieces like [DifferentialEquations.jl](https://diffeq.sciml.ai/stable/),
[NonlinearSolve.jl](https://nonlinearsolve.sciml.ai/dev/),
[LinearSolve.jl](http://linearsolve.sciml.ai/dev/), etc. first.
for pieces like [DifferentialEquations.jl](https://docs.sciml.ai/DiffEqDocs/stable/),
[NonlinearSolve.jl](https://docs.sciml.ai/NonlinearSolve/stable/),
[LinearSolve.jl](https://docs.sciml.ai/LinearSolve/stable/), etc. first.

## Installation

Expand All @@ -46,10 +46,10 @@ solve(prob,args...;sensealg=InterpolatingAdjoint(),
`solve` is fully compatible with automatic differentiation libraries
like:

- [Zygote.jl](https://github.com/FluxML/Zygote.jl)
- [ReverseDiff.jl](https://github.com/JuliaDiff/ReverseDiff.jl)
- [Zygote.jl](https://fluxml.ai/Zygote.jl/stable/)
- [ReverseDiff.jl](https://juliadiff.org/ReverseDiff.jl/)
- [Tracker.jl](https://github.com/FluxML/Tracker.jl)
- [ForwardDiff.jl](https://github.com/JuliaDiff/ForwardDiff.jl)
- [ForwardDiff.jl](https://juliadiff.org/ForwardDiff.jl/stable/)

and will automatically replace any calculations of the solution's derivative
with a fast method. The keyword argument `sensealg` controls the dispatch to the
Expand All @@ -68,7 +68,7 @@ is used, i.e. going back to the AD mechanism.
## Equation Scope

SciMLSensitivity.jl supports all of the equation types of the
[SciML Common Interface](https://scimlbase.sciml.ai/dev/), extending the problem
[SciML Common Interface](https://docs.sciml.ai/SciMLBase/stable/), extending the problem
types by adding overloads for automatic differentiation to improve the performance
and flexibility of the differentiation system. This includes:

Expand Down Expand Up @@ -148,15 +148,15 @@ Many different training techniques are supported by this package, including:
- Discretize-then-optimize (forward and reverse mode discrete sensitivity analysis)
- This is a generalization of [ANODE](https://arxiv.org/pdf/1902.10298.pdf) and
[ANODEv2](https://arxiv.org/pdf/1906.04596.pdf) to all
[DifferentialEquations.jl ODE solvers](https://diffeq.sciml.ai/latest/solvers/ode_solve/)
[DifferentialEquations.jl ODE solvers](https://docs.sciml.ai/DiffEqDocs/stable/solvers/ode_solve/)
- Hybrid approaches (adaptive time stepping + AD for adaptive discretize-then-optimize)
- O(1) memory backprop of ODEs via BacksolveAdjoint, and Virtual Brownian Trees for O(1) backprop of SDEs
- [Continuous adjoints for integral loss functions](@ref continuous_loss)
- Probabilistic programming and variational inference on ODEs/SDEs/DAEs/DDEs/hybrid
equations etc. is provided by integration with [Turing.jl](https://turing.ml/dev/)
equations etc. is provided by integration with [Turing.jl](https://turing.ml/stable/docs/using-turing/)
and [Gen.jl](https://github.com/probcomp/Gen.jl). Reproduce
[variational loss functions](https://arxiv.org/abs/2001.01328) by plugging
[composible libraries together](https://turing.ml/dev/tutorials/9-variationalinference/).
[composible libraries together](https://turing.ml/stable/tutorials/09-variational-inference/).

all while mixing forward mode and reverse mode approaches as appropriate for the
most speed. For more details on the adjoint sensitivity analysis methods for
Expand All @@ -182,7 +182,7 @@ use and swap out the ODE solver between any common interface compatible library,
- LSODA.jl
- [IRKGaussLegendre.jl](https://github.com/mikelehu/IRKGaussLegendre.jl)
- [SciPyDiffEq.jl](https://github.com/SciML/SciPyDiffEq.jl)
- [... etc. many other choices!](https://diffeq.sciml.ai/stable/solvers/ode_solve/)
- [... etc. many other choices!](https://docs.sciml.ai/DiffEqDocs/stable/solvers/ode_solve/)

In addition, due to the composability of the system, none of the components are directly
tied to the Flux.jl machine learning framework. For example, you can [use SciMLSensitivity.jl
Expand Down
14 changes: 7 additions & 7 deletions docs/src/manual/differential_equation_sensitivities.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ requiring the user to do any of the setup.
Current AD libraries whose calls are captured by the sensitivity
system are:

- [Zygote.jl](https://github.com/FluxML/Zygote.jl)
- [Zygote.jl](https://fluxml.ai/Zygote.jl/stable/)
- [Diffractor.jl](https://github.com/JuliaDiff/Diffractor.jl)

## Using and Controlling Sensitivity Algorithms within AD
Expand Down Expand Up @@ -167,7 +167,7 @@ When this is done, the choice of `ZygoteVJP` will utilize your VJP
function during the internal steps of the adjoint. This is useful for
models where automatic differentiation may have trouble producing
optimal code. This can be paired with
[ModelingToolkit.jl](https://github.com/SciML/ModelingToolkit.jl)
[ModelingToolkit.jl](https://docs.sciml.ai/ModelingToolkit/stable/)
for producing hyper-optimized, sparse, and parallel VJP functions utilizing
the automated symbolic conversions.

Expand Down Expand Up @@ -238,11 +238,11 @@ practice.

To avoid the issues of backwards solving the ODE, `InterpolatingAdjoint`
and `QuadratureAdjoint` utilize information from the forward pass.
By default these methods utilize the [continuous solution](https://diffeq.sciml.ai/latest/basics/solution/#Interpolations-1)
By default these methods utilize the [continuous solution](https://docs.sciml.ai/DiffEqDocs/stable/basics/solution/#Interpolations-1)
provided by DifferentialEquations.jl in the calculations of the
adjoint pass. `QuadratureAdjoint` uses this to build a continuous
function for the solution of adjoint equation and then performs an
adaptive quadrature via [Quadrature.jl](https://github.com/SciML/Quadrature.jl),
adaptive quadrature via [Integrals.jl](https://docs.sciml.ai/Integrals/stable/),
while `InterpolatingAdjoint` appends the integrand to the ODE so it's
computed simultaneously to the Lagrange multiplier. When memory is
not an issue, we find that the `QuadratureAdjoint` approach tends to
Expand Down Expand Up @@ -278,14 +278,14 @@ is done on the discretized system. While traditionally this can be
done discrete sensitivity analysis, this is can be equivalently done
by automatic differentiation on the solver itself. `ReverseDiffAdjoint`
performs reverse-mode automatic differentiation on the solver via
[ReverseDiff.jl](https://github.com/JuliaDiff/ReverseDiff.jl),
[ReverseDiff.jl](https://juliadiff.org/ReverseDiff.jl/),
`ZygoteAdjoint` performs reverse-mode automatic
differentiation on the solver via
[Zygote.jl](https://github.com/FluxML/Zygote.jl), and `TrackerAdjoint`
[Zygote.jl](https://fluxml.ai/Zygote.jl/latest/), and `TrackerAdjoint`
performs reverse-mode automatic differentiation on the solver via
[Tracker.jl](https://github.com/FluxML/Tracker.jl). In addition,
`ForwardDiffSensitivty` performs forward-mode automatic differentiation
on the solver via [ForwardDiff.jl](https://github.com/JuliaDiff/ForwardDiff.jl).
on the solver via [ForwardDiff.jl](https://juliadiff.org/ForwardDiff.jl/stable/).

We note that many studies have suggested that [this approach produces
more accurate gradients than the optimize-than-discretize approach](https://arxiv.org/abs/2005.13420)
2 changes: 1 addition & 1 deletion docs/src/neural_ode/simplechains.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ end

The next step is to minimize the loss, so that the NeuralODE gets trained. But in order to be able to do that, we have to be able to backpropagate through the NeuralODE model. Here the backpropagation through the neural network is the easy part and we get that out of the box with any deep learning package(although not as fast as SimpleChains for the small nn case here). But we have to find a way to first propagate the sensitivities of the loss back, first through the ODE solver and then to the neural network.

The adjoint of a neural ODE can be calculated through the various AD algorithms available in SciMLSensitivity.jl. But for working with [StaticArrays](https://github.com/JuliaArrays/StaticArrays.jl) in SimpleChains.jl we require a special adjoint method as StaticArrays do not allow any mutation. All the adjoint methods make heavy use of in-place mutation to be performant with the heap allocated normal arrays. For our statically sized, stack allocated StaticArrays, in order to be able to compute the ODE adjoint we need to do everything out of place. Hence we have specifically used `QuadratureAdjoint(autojacvec=ZygoteVJP())` adjoint algorithm in the solve call inside `predict_neuralode(p)` which computes everything out-of-place when u0 is a StaticArray. Hence we can move forward with the training of the NeuralODE
The adjoint of a neural ODE can be calculated through the various AD algorithms available in SciMLSensitivity.jl. But for working with [StaticArrays](https://docs.sciml.ai/StaticArrays/stable/) in SimpleChains.jl we require a special adjoint method as StaticArrays do not allow any mutation. All the adjoint methods make heavy use of in-place mutation to be performant with the heap allocated normal arrays. For our statically sized, stack allocated StaticArrays, in order to be able to compute the ODE adjoint we need to do everything out of place. Hence we have specifically used `QuadratureAdjoint(autojacvec=ZygoteVJP())` adjoint algorithm in the solve call inside `predict_neuralode(p)` which computes everything out-of-place when u0 is a StaticArray. Hence we can move forward with the training of the NeuralODE

```@example sc_neuralode
callback = function (p, l, pred; doplot = true)
Expand Down
4 changes: 2 additions & 2 deletions docs/src/ode_fitting/data_parallel.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Instead of parallelizing within an ODE solve, one can parallelize the
solves to the ODE itself. While this will be less effective on very
large ODEs, like big neural ODE image classifiers, this method be effective
even if the ODE is small or the `f` function is not well-parallelized.
This kind of parallelism is done via the [DifferentialEquations.jl ensemble interface](https://diffeq.sciml.ai/stable/features/ensemble/). The following examples
This kind of parallelism is done via the [DifferentialEquations.jl ensemble interface](https://docs.sciml.ai/DiffEqDocs/stable/features/ensemble/). The following examples
showcase multithreaded, cluster, and (multi)GPU parallelism through this
interface.

Expand Down Expand Up @@ -139,7 +139,7 @@ prob = ODEProblem((u, p, t) -> 1.01u .* p, [θ[1]], (0.0, 1.0), [θ[2]])
In the `prob_func` we define how to build a new problem based on the
base problem. In this case, we want to change `u0` by a constant, i.e.
`0.5 .+ i/100 .* prob.u0` for different trajectories labelled by `i`.
Thus we use the [remake function from the problem interface](https://diffeq.sciml.ai/stable/basics/problem/#Modification-of-problem-types) to do so:
Thus we use the [remake function from the problem interface](https://docs.sciml.ai/DiffEqDocs/stable/basics/problem/#Modification-of-problem-types) to do so:

```@example dataparallel
function prob_func(prob, i, repeat)
Expand Down
4 changes: 2 additions & 2 deletions docs/src/ode_fitting/optimization_ode.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ result_ode = Optimization.solve(optprob, PolyOpt(),
## Explanation

First let's create a Lotka-Volterra ODE using DifferentialEquations.jl. For
more details, [see the DifferentialEquations.jl documentation](http://docs.juliadiffeq.org/dev/). The Lotka-Volterra equations have the form:
more details, [see the DifferentialEquations.jl documentation](https://docs.sciml.ai/DiffEqDocs/stable/). The Lotka-Volterra equations have the form:

```math
\begin{aligned}
Expand Down Expand Up @@ -106,7 +106,7 @@ savefig("LV_ode.png")

For this first example, we do not yet include a neural network. We take
[AD-compatible `solve`
function](https://docs.juliadiffeq.org/latest/analysis/sensitivity/) function
function](https://docs.sciml.ai/SciMLSensitivity/stable/manual/differential_equation_sensitivities/) function
that takes the parameters and an initial condition and returns the solution of
the differential equation. Next we choose a loss function. Our goal will be to
find parameters that make the Lotka-Volterra solution constant `x(t)=1`, so we
Expand Down
Loading

0 comments on commit 8f02ca7

Please sign in to comment.