-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathextensions.html
340 lines (322 loc) · 45.8 KB
/
extensions.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Extending DiffSharp
</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="author" content="Atılım Güneş Baydin, Don Syme, Barak A. Pearlmutter, Jeffrey Siskind, and DiffSharp contributors">
<meta name="description" content="DiffSharp is a tensor library with support for differentiable programming. It is designed for use in machine learning, probabilistic programming, optimization and other domains.">
<script src="https://code.jquery.com/jquery-1.8.0.js"></script>
<script src="https://code.jquery.com/ui/1.8.23/jquery-ui.js"></script>
<script src="https://netdna.bootstrapcdn.com/twitter-bootstrap/2.2.1/js/bootstrap.min.js"></script>
<script type="text/javascript" async src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<link href="https://netdna.bootstrapcdn.com/twitter-bootstrap/2.2.1/css/bootstrap-combined.min.css" rel="stylesheet">
<link type="text/css" rel="stylesheet" href="https://diffsharp.github.io/content/fsdocs-default.css" />
<script src="https://diffsharp.github.io/content/fsdocs-tips.js" type="text/javascript"></script>
<!-- BEGIN SEARCH BOX: this adds support for the search box -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/JavaScript-autoComplete/1.0.4/auto-complete.css" />
<!-- END SEARCH BOX: this adds support for the search box -->
</head>
<body>
<div class="container">
<!-- <div class="masthead">
<ul class="nav nav-pills pull-right">
<li><a href="https://fsharp.org">fsharp.org</a></li>
</ul>
<h3 class="muted">DiffSharp</h3>
</div> -->
<!-- <hr /> -->
<div class="row">
<div class="col-xs-12" style="height:10px;"></div>
</div>
<div class="row">
<div class="span3" id="fsdocs-nav">
<a href="index.html"><img class="logo" src="https://diffsharp.github.io/img/diffsharp-logo-text.png"/></a>
<!-- BEGIN SEARCH BOX: this adds support for the search box -->
<div id="header">
<div class="searchbox">
<label for="search-by">
<i class="fas fa-search"></i>
</label>
<input data-search-input="" id="search-by" type="search" placeholder="Search..." />
<span data-search-clear="">
<i class="fas fa-times"></i>
</span>
</div>
</div>
<!-- END SEARCH BOX: this adds support for the search box -->
<ul class="nav nav-list" id="menu">
<!-- <li class="nav-header">DiffSharp</li> -->
<!-- <li class="divider"></li> -->
<li><a href="https://diffsharp.github.io/index.html">Home</a></li>
<li><a href="https://github.com/DiffSharp/DiffSharp/">GitHub</a></li>
<li><a href="https://github.com/DiffSharp/DiffSharp/blob/dev/LICENSE">License (BSD)</a></li>
<li class="nav-header">Getting Started</li>
<!-- <li class="divider"></li> -->
<li><a href="https://diffsharp.github.io/install.html">Install</a></li>
<li><a href="https://diffsharp.github.io/quickstart.html">Quickstart</a></li>
<!-- <li><a href="https://diffsharp.github.io/tensors.html">Tensors</a></li> -->
<!-- <li><a href="https://diffsharp.github.io/differentiable-programming.html">Differentiable Programming</a></li> -->
<!-- <li><a href="https://diffsharp.github.io/nested-derivatives.html">Nested Derivatives</a></li> -->
<!-- <li><a href="https://diffsharp.github.io/models.html">Models</a></li> -->
<!-- <li><a href="https://diffsharp.github.io/optimization.html">Optimization</a></li> -->
<!-- <li><a href="https://diffsharp.github.io/probability-distributions.html">Probability Distributions</a></li> -->
<li class="nav-header">Tutorials</li>
<!-- <li class="divider"></li> -->
<!-- <li><a href="https://diffsharp.github.io/tutorial-classifier.html">Classifier</a></li> -->
<!-- <li><a href="https://diffsharp.github.io/tutorial-gan.html">GAN</a></li> -->
<!-- <li><a href="https://diffsharp.github.io/tutorial-vae.html">VAE</a></li> -->
<!-- <li><a href="https://diffsharp.github.io/tutorial-language.html">Language Models</a></li> -->
<li><a href="https://github.com/DiffSharp/DiffSharp/tree/dev/examples">More Examples</a></li>
<li class="nav-header">API Documentation</li>
<li><a href="https://diffsharp.github.io/reference/index.html">API Reference</a></li>
<li><a href="https://diffsharp.github.io/extensions.html">Extensions</a></li>
<!-- <li class="nav-header">Examples</li> -->
<!-- <li class="divider"></li> -->
<!-- <li class="nav-header">Machine Learning</li> -->
<!-- <li><a href="https://diffsharp.github.io/examples-topic1.html">Topic 1</a></li> -->
<!-- <li class="divider"></li>
<li class="nav-header">Authors</li>
<li><a href="http://www.robots.ox.ac.uk/~gunes/">Atılım Güneş Baydin</a></li>
<li><a href="http://www.bcl.hamilton.ie/~barak/">Barak A. Pearlmutter</a></li>
<li><a href="https://www.microsoft.com/en-us/research/people/dsyme/">Don Syme</a></li> -->
</ul>
</div>
<div class="span9" id="fsdocs-content">
<p><a href="https://colab.research.google.com/github/DiffSharp/diffsharp.github.io/blob/master/extensions.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Binder" /></a> 
<a href="https://mybinder.org/v2/gh/diffsharp/diffsharp.github.io/master?filepath=extensions.ipynb"><img src="img/badge-binder.svg" alt="Binder" /></a> 
<a href="extensions.fsx"><img src="img/badge-script.svg" alt="Script" /></a> 
<a href="extensions.ipynb"><img src="img/badge-notebook.svg" alt="Script" /></a></p>
<h1><a name="Extending-DiffSharp" class="anchor" href="#Extending-DiffSharp">Extending DiffSharp</a></h1>
<p>DiffSharp provides most of the essential operations found in tensor libraries such as <a href="https://numpy.org/">NumPy</a>, <a href="https://pytorch.org/">PyTorch</a>, and <a href="https://www.tensorflow.org/">TensorFlow</a>. All differentiable operations support the forward, reverse, and nested differentiation modes.</p>
<p>When implementing new operations, you should prefer to implement these as compositions of existing DiffSharp <a href="https://diffsharp.github.io/reference/diffsharp-tensor.html">Tensor</a> operations, which would give you differentiability out of the box.</p>
<p>In the rare cases where you need to extend DiffSharp with a completely new differentiable operation that cannot be implemented as a composition of existing operations, you can use the provided extension API.</p>
<h2><a name="Simple-elementwise-functions" class="anchor" href="#Simple-elementwise-functions">Simple elementwise functions</a></h2>
<p>If the function you would like to implement is a simple elementwise function, you can use the <a href="https://diffsharp.github.io/reference/diffsharp-unaryopelementwise.html">UnaryOpElementwise</a> or <a href="https://diffsharp.github.io/reference/diffsharp-binaryopelementwise.html">BinaryOpElementwise</a> types to define your function and its derivatives. The forward, reverse, and nested differentiation rules for the function are automatically generated by the type. The documentation of these two types detail how they should be instantiated.</p>
<p>Let's see several examples.</p>
<p><span class="math">\(f(a) = \mathrm{sin}(a)\)</span>, with derivative <span class="math">\(\frac{\partial f(a)}{\partial a} = \mathrm{cos}(a) \;\)</span>.</p>
<pre class="fssnip highlighted"><code lang="fsharp"><span class="k">open</span> <span onmouseout="hideTip(event, 'fs1', 10)" onmouseover="showTip(event, 'fs1', 10)" class="id">DiffSharp</span>
<span class="k">type</span> <span onmouseout="hideTip(event, 'fs7', 11)" onmouseover="showTip(event, 'fs7', 11)" class="rt">Tensor</span> <span class="k">with</span>
<span class="k">member</span> <span onmouseout="hideTip(event, 'fs8', 12)" onmouseover="showTip(event, 'fs8', 12)" class="fn">a</span><span class="pn">.</span><span class="fn">sin</span><span class="pn">(</span><span class="pn">)</span> <span class="o">=</span>
<span onmouseout="hideTip(event, 'fs7', 13)" onmouseover="showTip(event, 'fs7', 13)" class="rt">Tensor</span><span class="pn">.</span><span onmouseout="hideTip(event, 'fs9', 14)" onmouseover="showTip(event, 'fs9', 14)" class="id">Op</span>
<span class="pn">{</span> <span class="k">new</span> <span onmouseout="hideTip(event, 'fs10', 15)" onmouseover="showTip(event, 'fs10', 15)" class="rt">UnaryOpElementwise</span><span class="pn">(</span><span class="s">"sin"</span><span class="pn">)</span> <span class="k">with</span>
<span class="k">member</span> <span class="id">_</span><span class="pn">.</span><span class="fn">fRaw</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs11', 16)" onmouseover="showTip(event, 'fs11', 16)" class="fn">a</span><span class="pn">)</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs11', 17)" onmouseover="showTip(event, 'fs11', 17)" class="fn">a</span><span class="pn">.</span><span onmouseout="hideTip(event, 'fs12', 18)" onmouseover="showTip(event, 'fs12', 18)" class="id">SinT</span><span class="pn">(</span><span class="pn">)</span>
<span class="k">member</span> <span class="id">_</span><span class="pn">.</span><span class="fn">dfda</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs8', 19)" onmouseover="showTip(event, 'fs8', 19)" class="fn">a</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs13', 20)" onmouseover="showTip(event, 'fs13', 20)" class="fn">f</span><span class="pn">)</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs8', 21)" onmouseover="showTip(event, 'fs8', 21)" class="fn">a</span><span class="pn">.</span><span class="id">cos</span><span class="pn">(</span><span class="pn">)</span>
<span class="pn">}</span>
<span class="pn">(</span><span onmouseout="hideTip(event, 'fs8', 22)" onmouseover="showTip(event, 'fs8', 22)" class="fn">a</span><span class="pn">)</span>
</code></pre>
<p><span class="math">\(f(a) = \mathrm{log}(a)\)</span>, with derivative <span class="math">\(\frac{\partial f(a)}{\partial a} = 1/a \;\)</span>.</p>
<pre class="fssnip highlighted"><code lang="fsharp"><span class="k">type</span> <span onmouseout="hideTip(event, 'fs7', 23)" onmouseover="showTip(event, 'fs7', 23)" class="rt">Tensor</span> <span class="k">with</span>
<span class="k">member</span> <span onmouseout="hideTip(event, 'fs8', 24)" onmouseover="showTip(event, 'fs8', 24)" class="fn">a</span><span class="pn">.</span><span class="fn">log</span><span class="pn">(</span><span class="pn">)</span> <span class="o">=</span>
<span onmouseout="hideTip(event, 'fs7', 25)" onmouseover="showTip(event, 'fs7', 25)" class="rt">Tensor</span><span class="pn">.</span><span onmouseout="hideTip(event, 'fs9', 26)" onmouseover="showTip(event, 'fs9', 26)" class="id">Op</span>
<span class="pn">{</span> <span class="k">new</span> <span onmouseout="hideTip(event, 'fs10', 27)" onmouseover="showTip(event, 'fs10', 27)" class="rt">UnaryOpElementwise</span><span class="pn">(</span><span class="s">"log"</span><span class="pn">)</span> <span class="k">with</span>
<span class="k">member</span> <span class="id">_</span><span class="pn">.</span><span class="fn">fRaw</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs11', 28)" onmouseover="showTip(event, 'fs11', 28)" class="fn">a</span><span class="pn">)</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs11', 29)" onmouseover="showTip(event, 'fs11', 29)" class="fn">a</span><span class="pn">.</span><span onmouseout="hideTip(event, 'fs14', 30)" onmouseover="showTip(event, 'fs14', 30)" class="id">LogT</span><span class="pn">(</span><span class="pn">)</span>
<span class="k">member</span> <span class="id">_</span><span class="pn">.</span><span class="fn">dfda</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs8', 31)" onmouseover="showTip(event, 'fs8', 31)" class="fn">a</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs13', 32)" onmouseover="showTip(event, 'fs13', 32)" class="fn">f</span><span class="pn">)</span> <span class="o">=</span> <span class="n">1</span><span class="o">/</span><span onmouseout="hideTip(event, 'fs8', 33)" onmouseover="showTip(event, 'fs8', 33)" class="fn">a</span>
<span class="pn">}</span>
<span class="pn">(</span><span onmouseout="hideTip(event, 'fs8', 34)" onmouseover="showTip(event, 'fs8', 34)" class="fn">a</span><span class="pn">)</span>
</code></pre>
<p><span class="math">\(f(a, b) = ab\)</span>, with derivatives <span class="math">\(\frac{\partial f(a, b)}{\partial a} = b\)</span>, <span class="math">\(\frac{\partial f(a, b)}{\partial b} = a \;\)</span>.</p>
<pre class="fssnip highlighted"><code lang="fsharp"><span class="k">type</span> <span onmouseout="hideTip(event, 'fs7', 35)" onmouseover="showTip(event, 'fs7', 35)" class="rt">Tensor</span> <span class="k">with</span>
<span class="k">member</span> <span onmouseout="hideTip(event, 'fs8', 36)" onmouseover="showTip(event, 'fs8', 36)" class="fn">a</span><span class="pn">.</span><span class="fn">mul</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs15', 37)" onmouseover="showTip(event, 'fs15', 37)" class="fn">b</span><span class="pn">)</span> <span class="o">=</span>
<span onmouseout="hideTip(event, 'fs7', 38)" onmouseover="showTip(event, 'fs7', 38)" class="rt">Tensor</span><span class="pn">.</span><span onmouseout="hideTip(event, 'fs9', 39)" onmouseover="showTip(event, 'fs9', 39)" class="id">Op</span>
<span class="pn">{</span> <span class="k">new</span> <span onmouseout="hideTip(event, 'fs16', 40)" onmouseover="showTip(event, 'fs16', 40)" class="rt">BinaryOpElementwise</span><span class="pn">(</span><span class="s">"mul"</span><span class="pn">)</span> <span class="k">with</span>
<span class="k">member</span> <span class="id">_</span><span class="pn">.</span><span class="fn">fRaw</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs11', 41)" onmouseover="showTip(event, 'fs11', 41)" class="fn">a</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs17', 42)" onmouseover="showTip(event, 'fs17', 42)" class="fn">b</span><span class="pn">)</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs11', 43)" onmouseover="showTip(event, 'fs11', 43)" class="fn">a</span><span class="pn">.</span><span onmouseout="hideTip(event, 'fs18', 44)" onmouseover="showTip(event, 'fs18', 44)" class="id">MulTT</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs17', 45)" onmouseover="showTip(event, 'fs17', 45)" class="fn">b</span><span class="pn">)</span>
<span class="k">member</span> <span class="id">_</span><span class="pn">.</span><span class="fn">dfda</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs8', 46)" onmouseover="showTip(event, 'fs8', 46)" class="fn">a</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs15', 47)" onmouseover="showTip(event, 'fs15', 47)" class="fn">b</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs13', 48)" onmouseover="showTip(event, 'fs13', 48)" class="fn">f</span><span class="pn">)</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs15', 49)" onmouseover="showTip(event, 'fs15', 49)" class="fn">b</span>
<span class="k">member</span> <span class="id">_</span><span class="pn">.</span><span class="fn">dfdb</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs8', 50)" onmouseover="showTip(event, 'fs8', 50)" class="fn">a</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs15', 51)" onmouseover="showTip(event, 'fs15', 51)" class="fn">b</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs13', 52)" onmouseover="showTip(event, 'fs13', 52)" class="fn">f</span><span class="pn">)</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs8', 53)" onmouseover="showTip(event, 'fs8', 53)" class="fn">a</span>
<span class="pn">}</span>
<span class="pn">(</span><span onmouseout="hideTip(event, 'fs8', 54)" onmouseover="showTip(event, 'fs8', 54)" class="fn">a</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs15', 55)" onmouseover="showTip(event, 'fs15', 55)" class="fn">b</span><span class="pn">)</span>
</code></pre>
<p><span class="math">\(f(a, b) = a^b\)</span>, with derivatives <span class="math">\(\frac{\partial f(a, b)}{\partial a} = b a^{b-1}\)</span>, <span class="math">\(\frac{\partial f(a, b)}{\partial b} = a^b \mathrm{log}(a) \;\)</span>. Note the use of the argument <code>f</code> in the derivative definitions that makes use of the pre-computed value of <span class="math">\(f(a, b) = a^b\)</span> that is available to the derivative implementation.</p>
<pre class="fssnip highlighted"><code lang="fsharp"><span class="k">type</span> <span onmouseout="hideTip(event, 'fs7', 56)" onmouseover="showTip(event, 'fs7', 56)" class="rt">Tensor</span> <span class="k">with</span>
<span class="k">member</span> <span onmouseout="hideTip(event, 'fs8', 57)" onmouseover="showTip(event, 'fs8', 57)" class="fn">a</span><span class="pn">.</span><span class="fn">pow</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs15', 58)" onmouseover="showTip(event, 'fs15', 58)" class="fn">b</span><span class="pn">)</span> <span class="o">=</span>
<span onmouseout="hideTip(event, 'fs7', 59)" onmouseover="showTip(event, 'fs7', 59)" class="rt">Tensor</span><span class="pn">.</span><span onmouseout="hideTip(event, 'fs9', 60)" onmouseover="showTip(event, 'fs9', 60)" class="id">Op</span>
<span class="pn">{</span> <span class="k">new</span> <span onmouseout="hideTip(event, 'fs16', 61)" onmouseover="showTip(event, 'fs16', 61)" class="rt">BinaryOpElementwise</span><span class="pn">(</span><span class="s">"pow"</span><span class="pn">)</span> <span class="k">with</span>
<span class="k">member</span> <span class="id">_</span><span class="pn">.</span><span class="fn">fRaw</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs11', 62)" onmouseover="showTip(event, 'fs11', 62)" class="fn">a</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs17', 63)" onmouseover="showTip(event, 'fs17', 63)" class="fn">b</span><span class="pn">)</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs11', 64)" onmouseover="showTip(event, 'fs11', 64)" class="fn">a</span><span class="pn">.</span><span onmouseout="hideTip(event, 'fs19', 65)" onmouseover="showTip(event, 'fs19', 65)" class="id">PowTT</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs17', 66)" onmouseover="showTip(event, 'fs17', 66)" class="fn">b</span><span class="pn">)</span>
<span class="k">member</span> <span class="id">_</span><span class="pn">.</span><span class="fn">dfda</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs8', 67)" onmouseover="showTip(event, 'fs8', 67)" class="fn">a</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs15', 68)" onmouseover="showTip(event, 'fs15', 68)" class="fn">b</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs13', 69)" onmouseover="showTip(event, 'fs13', 69)" class="fn">f</span><span class="pn">)</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs15', 70)" onmouseover="showTip(event, 'fs15', 70)" class="fn">b</span> <span class="o">*</span> <span onmouseout="hideTip(event, 'fs13', 71)" onmouseover="showTip(event, 'fs13', 71)" class="fn">f</span> <span class="o">/</span> <span onmouseout="hideTip(event, 'fs8', 72)" onmouseover="showTip(event, 'fs8', 72)" class="fn">a</span> <span class="c">// equivalent to b * a.pow(b-1)</span>
<span class="k">member</span> <span class="id">_</span><span class="pn">.</span><span class="fn">dfdb</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs8', 73)" onmouseover="showTip(event, 'fs8', 73)" class="fn">a</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs15', 74)" onmouseover="showTip(event, 'fs15', 74)" class="fn">b</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs13', 75)" onmouseover="showTip(event, 'fs13', 75)" class="fn">f</span><span class="pn">)</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs13', 76)" onmouseover="showTip(event, 'fs13', 76)" class="fn">f</span> <span class="o">*</span> <span onmouseout="hideTip(event, 'fs8', 77)" onmouseover="showTip(event, 'fs8', 77)" class="fn">a</span><span class="pn">.</span><span class="id">log</span><span class="pn">(</span><span class="pn">)</span> <span class="c">// equivalent to a.pow(b) * a.log()</span>
<span class="pn">}</span>
<span class="pn">(</span><span onmouseout="hideTip(event, 'fs8', 78)" onmouseover="showTip(event, 'fs8', 78)" class="fn">a</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs15', 79)" onmouseover="showTip(event, 'fs15', 79)" class="fn">b</span><span class="pn">)</span>
</code></pre>
<h2><a name="General-functions" class="anchor" href="#General-functions">General functions</a></h2>
<p>For more complicated functions, you can use the most general way of defining functions using the <a href="https://diffsharp.github.io/reference/diffsharp-unaryop.html">UnaryOp</a> or <a href="https://diffsharp.github.io/reference/diffsharp-binaryop.html">BinaryOp</a> types, which allow you to define the full forward and reverse mode differentiation rules. The documentation of these two types detail how they should be instantiated.</p>
<p>Let's see several examples.</p>
<p><span class="math">\(f(A) = A^{\intercal}\)</span>, with the forward derivative propagation rule <span class="math">\(\frac{\partial f(A)}{\partial X} = \frac{\partial A}{\partial X} \frac{\partial f(A)}{\partial A} = (\frac{\partial A}{\partial X})^{\intercal}\)</span> and the reverse derivative propagation rule <span class="math">\(\frac{\partial Y}{\partial A} = \frac{\partial Y}{\partial f(A)} \frac{\partial f(A)}{\partial A} = (\frac{\partial Y}{\partial f(A)})^{\intercal} \;\)</span>.</p>
<pre class="fssnip highlighted"><code lang="fsharp"><span class="k">type</span> <span onmouseout="hideTip(event, 'fs7', 80)" onmouseover="showTip(event, 'fs7', 80)" class="rt">Tensor</span> <span class="k">with</span>
<span class="k">member</span> <span onmouseout="hideTip(event, 'fs8', 81)" onmouseover="showTip(event, 'fs8', 81)" class="fn">a</span><span class="pn">.</span><span class="fn">transpose</span><span class="pn">(</span><span class="pn">)</span> <span class="o">=</span>
<span onmouseout="hideTip(event, 'fs7', 82)" onmouseover="showTip(event, 'fs7', 82)" class="rt">Tensor</span><span class="pn">.</span><span onmouseout="hideTip(event, 'fs9', 83)" onmouseover="showTip(event, 'fs9', 83)" class="id">Op</span>
<span class="pn">{</span> <span class="k">new</span> <span onmouseout="hideTip(event, 'fs20', 84)" onmouseover="showTip(event, 'fs20', 84)" class="rt">UnaryOp</span><span class="pn">(</span><span class="s">"transpose"</span><span class="pn">)</span> <span class="k">with</span>
<span class="k">member</span> <span class="id">_</span><span class="pn">.</span><span class="fn">fRaw</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs11', 85)" onmouseover="showTip(event, 'fs11', 85)" class="fn">a</span><span class="pn">)</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs11', 86)" onmouseover="showTip(event, 'fs11', 86)" class="fn">a</span><span class="pn">.</span><span onmouseout="hideTip(event, 'fs21', 87)" onmouseover="showTip(event, 'fs21', 87)" class="id">TransposeT2</span><span class="pn">(</span><span class="pn">)</span>
<span class="k">member</span> <span class="id">_</span><span class="pn">.</span><span class="fn">ad_dfda</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs8', 88)" onmouseover="showTip(event, 'fs8', 88)" class="fn">a</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs22', 89)" onmouseover="showTip(event, 'fs22', 89)" class="fn">ad</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs13', 90)" onmouseover="showTip(event, 'fs13', 90)" class="fn">f</span><span class="pn">)</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs22', 91)" onmouseover="showTip(event, 'fs22', 91)" class="fn">ad</span><span class="pn">.</span><span class="id">transpose</span><span class="pn">(</span><span class="pn">)</span>
<span class="k">member</span> <span class="id">_</span><span class="pn">.</span><span class="fn">fd_dfda</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs8', 92)" onmouseover="showTip(event, 'fs8', 92)" class="fn">a</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs13', 93)" onmouseover="showTip(event, 'fs13', 93)" class="fn">f</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs23', 94)" onmouseover="showTip(event, 'fs23', 94)" class="fn">fd</span><span class="pn">)</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs23', 95)" onmouseover="showTip(event, 'fs23', 95)" class="fn">fd</span><span class="pn">.</span><span class="id">transpose</span><span class="pn">(</span><span class="pn">)</span>
<span class="pn">}</span>
<span class="pn">(</span><span onmouseout="hideTip(event, 'fs8', 96)" onmouseover="showTip(event, 'fs8', 96)" class="fn">a</span><span class="pn">)</span>
</code></pre>
<p><span class="math">\(f(A, B) = AB\)</span>, with the forward derivative propagation rule <span class="math">\(\frac{\partial(A, B)}{\partial X} = \frac{\partial A}{\partial X} \frac{\partial f(A, B)}{\partial A} + \frac{\partial B}{\partial X} \frac{\partial f(A, B)}{\partial B} = \frac{\partial A}{\partial X} B + A \frac{\partial B}{\partial X}\)</span> and the reverse propagation rule <span class="math">\(\frac{\partial Y}{\partial A} = \frac{\partial Y}{\partial f(A, B)} \frac{\partial f(A, B)}{\partial A} = \frac{\partial Y}{\partial f(A, B)} B^{\intercal}\)</span>, <span class="math">\(\frac{\partial Y}{\partial B} = \frac{\partial Y}{\partial f(A, B)} \frac{\partial f(A, B)}{B} = A^{\intercal} \frac{\partial Y}{\partial f(A, B)} \;\)</span>.</p>
<pre class="fssnip highlighted"><code lang="fsharp"><span class="k">type</span> <span onmouseout="hideTip(event, 'fs7', 97)" onmouseover="showTip(event, 'fs7', 97)" class="rt">Tensor</span> <span class="k">with</span>
<span class="k">member</span> <span onmouseout="hideTip(event, 'fs8', 98)" onmouseover="showTip(event, 'fs8', 98)" class="fn">a</span><span class="pn">.</span><span class="fn">matmul</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs15', 99)" onmouseover="showTip(event, 'fs15', 99)" class="fn">b</span><span class="pn">)</span> <span class="o">=</span>
<span onmouseout="hideTip(event, 'fs7', 100)" onmouseover="showTip(event, 'fs7', 100)" class="rt">Tensor</span><span class="pn">.</span><span onmouseout="hideTip(event, 'fs9', 101)" onmouseover="showTip(event, 'fs9', 101)" class="id">Op</span>
<span class="pn">{</span> <span class="k">new</span> <span onmouseout="hideTip(event, 'fs24', 102)" onmouseover="showTip(event, 'fs24', 102)" class="rt">BinaryOp</span><span class="pn">(</span><span class="s">"matmul"</span><span class="pn">)</span> <span class="k">with</span>
<span class="k">member</span> <span class="id">_</span><span class="pn">.</span><span class="fn">fRaw</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs11', 103)" onmouseover="showTip(event, 'fs11', 103)" class="fn">a</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs17', 104)" onmouseover="showTip(event, 'fs17', 104)" class="fn">b</span><span class="pn">)</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs11', 105)" onmouseover="showTip(event, 'fs11', 105)" class="fn">a</span><span class="pn">.</span><span onmouseout="hideTip(event, 'fs25', 106)" onmouseover="showTip(event, 'fs25', 106)" class="id">MatMulTT</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs17', 107)" onmouseover="showTip(event, 'fs17', 107)" class="fn">b</span><span class="pn">)</span>
<span class="k">member</span> <span class="id">_</span><span class="pn">.</span><span class="fn">ad_dfda</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs8', 108)" onmouseover="showTip(event, 'fs8', 108)" class="fn">a</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs22', 109)" onmouseover="showTip(event, 'fs22', 109)" class="fn">ad</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs15', 110)" onmouseover="showTip(event, 'fs15', 110)" class="fn">b</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs13', 111)" onmouseover="showTip(event, 'fs13', 111)" class="fn">f</span><span class="pn">)</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs22', 112)" onmouseover="showTip(event, 'fs22', 112)" class="fn">ad</span><span class="pn">.</span><span class="id">matmul</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs15', 113)" onmouseover="showTip(event, 'fs15', 113)" class="fn">b</span><span class="pn">)</span>
<span class="k">member</span> <span class="id">_</span><span class="pn">.</span><span class="fn">bd_dfdb</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs8', 114)" onmouseover="showTip(event, 'fs8', 114)" class="fn">a</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs15', 115)" onmouseover="showTip(event, 'fs15', 115)" class="fn">b</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs26', 116)" onmouseover="showTip(event, 'fs26', 116)" class="fn">bd</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs13', 117)" onmouseover="showTip(event, 'fs13', 117)" class="fn">f</span><span class="pn">)</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs8', 118)" onmouseover="showTip(event, 'fs8', 118)" class="fn">a</span><span class="pn">.</span><span class="id">matmul</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs26', 119)" onmouseover="showTip(event, 'fs26', 119)" class="fn">bd</span><span class="pn">)</span>
<span class="k">member</span> <span class="id">_</span><span class="pn">.</span><span class="fn">fd_dfda</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs8', 120)" onmouseover="showTip(event, 'fs8', 120)" class="fn">a</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs15', 121)" onmouseover="showTip(event, 'fs15', 121)" class="fn">b</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs13', 122)" onmouseover="showTip(event, 'fs13', 122)" class="fn">f</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs23', 123)" onmouseover="showTip(event, 'fs23', 123)" class="fn">fd</span><span class="pn">)</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs23', 124)" onmouseover="showTip(event, 'fs23', 124)" class="fn">fd</span><span class="pn">.</span><span class="id">matmul</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs15', 125)" onmouseover="showTip(event, 'fs15', 125)" class="fn">b</span><span class="pn">.</span><span class="id">transpose</span><span class="pn">(</span><span class="pn">)</span><span class="pn">)</span>
<span class="k">member</span> <span class="id">_</span><span class="pn">.</span><span class="fn">fd_dfdb</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs8', 126)" onmouseover="showTip(event, 'fs8', 126)" class="fn">a</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs15', 127)" onmouseover="showTip(event, 'fs15', 127)" class="fn">b</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs13', 128)" onmouseover="showTip(event, 'fs13', 128)" class="fn">f</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs23', 129)" onmouseover="showTip(event, 'fs23', 129)" class="fn">fd</span><span class="pn">)</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs8', 130)" onmouseover="showTip(event, 'fs8', 130)" class="fn">a</span><span class="pn">.</span><span class="id">transpose</span><span class="pn">(</span><span class="pn">)</span><span class="pn">.</span><span class="id">matmul</span><span class="pn">(</span><span onmouseout="hideTip(event, 'fs23', 131)" onmouseover="showTip(event, 'fs23', 131)" class="fn">fd</span><span class="pn">)</span>
<span class="pn">}</span>
<span class="pn">(</span><span onmouseout="hideTip(event, 'fs8', 132)" onmouseover="showTip(event, 'fs8', 132)" class="fn">a</span><span class="pn">,</span><span onmouseout="hideTip(event, 'fs15', 133)" onmouseover="showTip(event, 'fs15', 133)" class="fn">b</span><span class="pn">)</span>
</code></pre>
<div class="fsdocs-tip" id="fs1">namespace DiffSharp</div>
<div class="fsdocs-tip" id="fs2">type dsharp =
static member abs: input: Tensor -> Tensor
static member acos: input: Tensor -> Tensor
static member add: a: Tensor * b: Tensor -> Tensor
static member arange: endVal: float * ?startVal: float * ?step: float * ?device: Device * ?dtype: Dtype * ?backend: Backend -> Tensor + 1 overload
static member arangeLike: input: Tensor * endVal: float * ?startVal: float * ?step: float * ?device: Device * ?dtype: Dtype * ?backend: Backend -> Tensor + 1 overload
static member argmax: input: Tensor -> int[] + 1 overload
static member argmin: input: Tensor -> int[] + 1 overload
static member asin: input: Tensor -> Tensor
static member atan: input: Tensor -> Tensor
static member backends: unit -> Backend list
...<br /><em><summary>
Tensor operations
</summary></em></div>
<div class="fsdocs-tip" id="fs3">static member DiffSharp.dsharp.config: unit -> DiffSharp.Device * DiffSharp.Dtype * DiffSharp.Backend * DiffSharp.Printer<br />static member DiffSharp.dsharp.config: configuration: (DiffSharp.Device * DiffSharp.Dtype * DiffSharp.Backend * DiffSharp.Printer) -> unit<br />static member DiffSharp.dsharp.config: ?device: DiffSharp.Device * ?dtype: DiffSharp.Dtype * ?backend: DiffSharp.Backend * ?printer: DiffSharp.Printer -> unit</div>
<div class="fsdocs-tip" id="fs4">Multiple items<br />module Backend
from DiffSharp<br /><em><summary>
Contains functions and settings related to backend specifications.
</summary></em><br /><br />--------------------<br />type Backend =
| Reference
| Torch
| Other of name: string * code: int
override ToString: unit -> string
member Name: string<br /><em><summary>
Represents a backend for DiffSharp tensors
</summary></em></div>
<div class="fsdocs-tip" id="fs5">union case DiffSharp.Backend.Reference: DiffSharp.Backend<br /><em><summary>
The reference backend
</summary></em></div>
<div class="fsdocs-tip" id="fs6">static member DiffSharp.dsharp.seed: ?seed: int -> unit</div>
<div class="fsdocs-tip" id="fs7">type Tensor =
private | TensorC of primalRaw: RawTensor
| TensorF of primal: Tensor * derivative: Tensor * nestingTag: uint32
| TensorR of primal: Tensor * derivative: Tensor ref * parentOp: TensorOp * fanout: uint32 ref * nestingTag: uint32
interface IConvertible
interface IComparable
override Equals: other: obj -> bool
override GetHashCode: unit -> int
member GetSlice: bounds: int[,] -> Tensor
override ToString: unit -> string
member abs: unit -> Tensor
member acos: unit -> Tensor
member add: b: Tensor -> Tensor + 1 overload
member addSlice: location: seq<int> * b: Tensor -> Tensor
...<br /><em><summary>
Represents a multi-dimensional data type containing elements of a single data type.
</summary><br /><example>
A tensor can be constructed from a list or sequence using <see cref="M:DiffSharp.dsharp.tensor(System.Object)" /><code>
let t = dsharp.tensor([[1.; -1.]; [1.; -1.]])
</code></example></em></div>
<div class="fsdocs-tip" id="fs8">val a: Tensor</div>
<div class="fsdocs-tip" id="fs9">static member Tensor.Op: ext: BinaryOp -> (Tensor * Tensor -> Tensor)<br />static member Tensor.Op: ext: UnaryOp -> (Tensor -> Tensor)</div>
<div class="fsdocs-tip" id="fs10">Multiple items<br />type UnaryOpElementwise =
inherit UnaryOp
new: name: string -> UnaryOpElementwise
override ad_dfda: a: Tensor * ad: Tensor * f: Tensor -> Tensor
abstract dfda: a: Tensor * f: Tensor -> Tensor
override fd_dfda: a: Tensor * f: Tensor * fd: Tensor -> Tensor<br /><em><summary>Defines a new op implementing an elementwise unary function and its derivatives. Instances of this class are used with the <see cref="M:DiffSharp.Tensor.Op(DiffSharp.UnaryOp)" /> method to define a new differentiable tensor function that supports forward, reverse, and nested differentiation.</summary><br /><remarks><para>This type is specialized to elementwise ops. It requires the user to specify only (1) the <see cref="T:DiffSharp.Backends.RawTensor" /> operation and (2) the derivative of the function with respect to its argument. The corresponding derivative propagation rules for the forward and reverse differentiation modes are automatically generated.</para><para>If you are implementing a complex op that is not elementwise, you can use the generic type <see cref="T:DiffSharp.UnaryOp" />, which allows you to define the full derivative propagation rules.</para></remarks><br /><example><code>
{ new UnaryOpElementwise("cos") with
member _.fRaw(a) = a.CosT()
member _.dfda(a,f) = -a.sin()
}
{ new UnaryOpElementwise("exp") with
member _.fRaw(a) = a.ExpT()
member _.dfda(a,f) = f
}
{ new UnaryOpElementwise("log") with
member _.fRaw(a) = a.LogT()
member _.dfda(a,f) = 1/a
}
</code></example></em><br /><br />--------------------<br />new: name: string -> UnaryOpElementwise</div>
<div class="fsdocs-tip" id="fs11">val a: Backends.RawTensor</div>
<div class="fsdocs-tip" id="fs12">abstract Backends.RawTensor.SinT: unit -> Backends.RawTensor</div>
<div class="fsdocs-tip" id="fs13">val f: Tensor</div>
<div class="fsdocs-tip" id="fs14">abstract Backends.RawTensor.LogT: unit -> Backends.RawTensor</div>
<div class="fsdocs-tip" id="fs15">val b: Tensor</div>
<div class="fsdocs-tip" id="fs16">Multiple items<br />type BinaryOpElementwise =
inherit BinaryOp
new: name: string -> BinaryOpElementwise
override ad_dfda: a: Tensor * ad: Tensor * b: Tensor * f: Tensor -> Tensor
override bd_dfdb: a: Tensor * b: Tensor * bd: Tensor * f: Tensor -> Tensor
abstract dfda: a: Tensor * b: Tensor * f: Tensor -> Tensor
abstract dfdb: a: Tensor * b: Tensor * f: Tensor -> Tensor
override fd_dfda: a: Tensor * b: Tensor * f: Tensor * fd: Tensor -> Tensor
override fd_dfdb: a: Tensor * b: Tensor * f: Tensor * fd: Tensor -> Tensor<br /><em><summary>Defines a new op implementing an elementwise binary function and its derivatives. Instances of this class are used with the <see cref="M:DiffSharp.Tensor.Op(DiffSharp.BinaryOp)" /> method to define a new differentiable tensor function that supports forward, reverse, and nested differentiation.</summary><br /><remarks>
This type is specialized to elementwise ops. It requires the user to specify only (1) the <see cref="T:DiffSharp.Backends.RawTensor" /> operation and (2) the derivative of the function with respect to each argument. The corresponding derivative propagation rules for the forward and reverse differentiation modes are automatically generated.
<para>If you are implementing a complex op that is not elementwise, you can use the generic type <see cref="T:DiffSharp.BinaryOp" />, which allows you to define the full derivative propagation rules.</para></remarks><br /><example><code>
{ new BinaryOpElementwise("pow") with
member _.fRaw(a,b) = a.PowTT(b)
member _.dfda(a,b,f) = b * f / a
member _.dfdb(a,b,f) = f * a.log()
}
{ new BinaryOpElementwise("mul") with
member _.fRaw(a,b) = a.MulTT(b)
member _.dfda(a,b,f) = b
member _.dfdb(a,b,f) = a
}
</code></example></em><br /><br />--------------------<br />new: name: string -> BinaryOpElementwise</div>
<div class="fsdocs-tip" id="fs17">val b: Backends.RawTensor</div>
<div class="fsdocs-tip" id="fs18">abstract Backends.RawTensor.MulTT: t2: Backends.RawTensor -> Backends.RawTensor</div>
<div class="fsdocs-tip" id="fs19">abstract Backends.RawTensor.PowTT: t2: Backends.RawTensor -> Backends.RawTensor</div>
<div class="fsdocs-tip" id="fs20">Multiple items<br />type UnaryOp =
new: name: string -> UnaryOp
abstract ad_dfda: a: Tensor * ad: Tensor * f: Tensor -> Tensor
abstract fRaw: a: RawTensor -> RawTensor
abstract fd_dfda: a: Tensor * f: Tensor * fd: Tensor -> Tensor
member name: string<br /><em><summary>Defines a new op implementing a unary function and its derivatives. Instances of this class are used with the <see cref="M:DiffSharp.Tensor.Op(DiffSharp.UnaryOp)" /> method to define a new differentiable tensor function that supports forward, reverse, and nested differentiation.</summary><br /><remarks><para>This type represents the most generic definition of a new op representing a unary function, allowing the specification of: (1) the <see cref="T:DiffSharp.Backends.RawTensor" /> operation, (2) the derivative propagation rule for the forward differentiation mode and (3) the derivative propagation rule for the reverse differentiation mode.</para><para>In general, if you are implementing a simple elementwise op, you should prefer using the <see cref="T:DiffSharp.UnaryOpElementwise" /> type, which is much simpler to use.</para></remarks><br /><example><code>
{ new UnaryOp("transpose") with
member _.fRaw(a) = a.TransposeT2()
member _.ad_dfda(a,ad,f) = ad.transpose()
member _.fd_dfda(a,f,fd) = fd.transpose()
}
</code></example></em><br /><br />--------------------<br />new: name: string -> UnaryOp</div>
<div class="fsdocs-tip" id="fs21">abstract Backends.RawTensor.TransposeT2: unit -> Backends.RawTensor</div>
<div class="fsdocs-tip" id="fs22">val ad: Tensor</div>
<div class="fsdocs-tip" id="fs23">val fd: Tensor</div>
<div class="fsdocs-tip" id="fs24">Multiple items<br />type BinaryOp =
new: name: string -> BinaryOp
abstract ad_dfda: a: Tensor * ad: Tensor * b: Tensor * f: Tensor -> Tensor
abstract bd_dfdb: a: Tensor * b: Tensor * bd: Tensor * f: Tensor -> Tensor
abstract fRaw: a: RawTensor * b: RawTensor -> RawTensor
abstract fd_dfda: a: Tensor * b: Tensor * f: Tensor * fd: Tensor -> Tensor
abstract fd_dfdb: a: Tensor * b: Tensor * f: Tensor * fd: Tensor -> Tensor
member name: string<br /><em><summary>Defines a new op implementing a binary function and its derivatives. Instances of this class are used with the <see cref="M:DiffSharp.Tensor.Op(DiffSharp.BinaryOp)" /> method to define a new differentiable tensor function that supports forward, reverse, and nested differentiation.</summary><br /><remarks><para>This type represents the most generic definition of a new op representing a binary function, allowing the specification of: (1) the <see cref="T:DiffSharp.Backends.RawTensor" /> operation, (2) the derivative propagation rule for the forward differentiation mode and (3) the derivative propagation rule for the reverse differentiation mode.</para><para>In general, if you are implementing a simple elementwise op, you should prefer using the <see cref="T:DiffSharp.BinaryOpElementwise" /> type, which is much simpler to use.</para></remarks><br /><example><code>
{ new BinaryOp("matmul") with
member _.fRaw(a,b) = a.MatMulTT(b)
member _.ad_dfda(a,ad,b,f) = ad.matmul(b)
member _.bd_dfdb(a,b,bd,f) = a.matmul(bd)
member _.fd_dfda(a,b,f,fd) = fd.matmul(b.transpose())
member _.fd_dfdb(a,b,f,fd) = a.transposeExt().matmul(fd)
}
</code></example></em><br /><br />--------------------<br />new: name: string -> BinaryOp</div>
<div class="fsdocs-tip" id="fs25">abstract Backends.RawTensor.MatMulTT: t2: Backends.RawTensor -> Backends.RawTensor</div>
<div class="fsdocs-tip" id="fs26">val bd: Tensor</div>
</div>
</div>
<div class="row">
<div class="span3"></div>
<div class="span9">
<hr>
<p style="height:50px; display: table-cell; vertical-align: bottom;">© Copyright 2021, DiffSharp Contributors.</p>
<br>
</div>
</div>
</div>
<!-- BEGIN SEARCH BOX: this adds support for the search box -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/JavaScript-autoComplete/1.0.4/auto-complete.css" />
<script type="text/javascript">var fsdocs_search_baseurl = 'https://diffsharp.github.io/'</script>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/lunr.js/2.3.8/lunr.min.js"></script>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/JavaScript-autoComplete/1.0.4/auto-complete.min.js"></script>
<script type="text/javascript" src="https://diffsharp.github.io/content/fsdocs-search.js"></script>
<!-- END SEARCH BOX: this adds support for the search box -->
</body>
</html>