You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -59,9 +59,11 @@ If you try to compile on a non-AVX2 system with `ON`, you will get something lik
59
59
../immintrin.h:14 error "This header is only meant to be used on x86 and x64 architecture"
60
60
```
61
61
62
-
### `float` Hessian approximation
62
+
### `float` Hessian approximation (performance)
63
63
Using `SYMX_HESS_STORAGE_FLOAT=float` lets SymX store the global Hessian used in Newton's Method in single point precision.
64
-
Importantly, all _operations_ are still performed in `double` (via casting). This option simply quantizes the global matrix values to `float`, halving memory traffic and significantly increasing performance.
64
+
Importantly, all _operations_ are still performed in `double` (via casting).
65
+
This option simply quantizes the global matrix values to `float`, halving memory traffic and significantly increasing performance.
66
+
The default remains `double` as very numerically stiff problems might require it, but the user is encouraged to assess whether `float` works for their application.
Copy file name to clipboardExpand all lines: _sources/symbols.md.txt
+6Lines changed: 6 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -59,6 +59,12 @@ These can be differentiated and their generated code will feature actual `if-els
59
59
Operations such as `min(), max()` use `branch` internally.
60
60
Note that the use of `branch` prevents emitting SIMD code.
61
61
62
+
**Important:** The condition passed to `branch` is a `Scalar` whose **sign** determines which branch executes.
63
+
When you write `a > b`, SymX internally stores `a - b` as the condition scalar.
64
+
Opposite for `a < b`.
65
+
The true branch executes when that condition scalar is **strictly positive** (`> 0`), exact zero will then fall to the false branch.
66
+
Due to the floating point nature of the comparison, the user might consider writing activation mechanisms with conditions that are `+1` and `-1` (instead of `0`) to more clearly indicate sign.
67
+
62
68
## `Vector` and `Matrix`
63
69
`Vector` and `Matrix` are simply a list of `Scalar`s that provide typical algebraic operator overloads.
64
70
Further, `Vector` provides `norm(), dot(), cross3()` and such, while `Matrix` provides `det(), inv(), trace()` etc.
Copy file name to clipboardExpand all lines: setup.html
+7-5Lines changed: 7 additions & 5 deletions
Original file line number
Diff line number
Diff line change
@@ -335,7 +335,7 @@ <h2>CMake options<a class="headerlink" href="#cmake-options" title="Link to this
335
335
<td><p>Output directory for generated files; defaults to <codeclass="docutils literal notranslate"><spanclass="pre"><build>/codegen</span></code></p></td>
<td><p>Hessian storage precision (<codeclass="docutils literal notranslate"><spanclass="pre">float</span></code> or <codeclass="docutils literal notranslate"><spanclass="pre">double</span></code>)</p></td>
340
340
</tr>
341
341
</tbody>
@@ -352,10 +352,12 @@ <h3>AVX2 support<a class="headerlink" href="#avx2-support" title="Link to this h
352
352
</pre></div>
353
353
</div>
354
354
</section>
355
-
<sectionid="float-hessian-approximation">
356
-
<h3><codeclass="docutils literal notranslate"><spanclass="pre">float</span></code> Hessian approximation<aclass="headerlink" href="#float-hessian-approximation" title="Link to this heading">¶</a></h3>
<h3><codeclass="docutils literal notranslate"><spanclass="pre">float</span></code> Hessian approximation (performance)<aclass="headerlink" href="#float-hessian-approximation-performance" title="Link to this heading">¶</a></h3>
357
357
<p>Using <codeclass="docutils literal notranslate"><spanclass="pre">SYMX_HESS_STORAGE_FLOAT=float</span></code> lets SymX store the global Hessian used in Newton’s Method in single point precision.
358
-
Importantly, all <em>operations</em> are still performed in <codeclass="docutils literal notranslate"><spanclass="pre">double</span></code> (via casting). This option simply quantizes the global matrix values to <codeclass="docutils literal notranslate"><spanclass="pre">float</span></code>, halving memory traffic and significantly increasing performance.</p>
358
+
Importantly, all <em>operations</em> are still performed in <codeclass="docutils literal notranslate"><spanclass="pre">double</span></code> (via casting).
359
+
This option simply quantizes the global matrix values to <codeclass="docutils literal notranslate"><spanclass="pre">float</span></code>, halving memory traffic and significantly increasing performance.
360
+
The default remains <codeclass="docutils literal notranslate"><spanclass="pre">double</span></code> as very numerically stiff problems might require it, but the user is encouraged to assess whether <codeclass="docutils literal notranslate"><spanclass="pre">float</span></code> works for their application.</p>
Copy file name to clipboardExpand all lines: symbols.html
+5Lines changed: 5 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -315,6 +315,11 @@ <h3>Branching<a class="headerlink" href="#branching" title="Link to this heading
315
315
<p>These can be differentiated and their generated code will feature actual <codeclass="docutils literal notranslate"><spanclass="pre">if-else</span></code> statements.
316
316
Operations such as <codeclass="docutils literal notranslate"><spanclass="pre">min(),</span><spanclass="pre">max()</span></code> use <codeclass="docutils literal notranslate"><spanclass="pre">branch</span></code> internally.
317
317
Note that the use of <codeclass="docutils literal notranslate"><spanclass="pre">branch</span></code> prevents emitting SIMD code.</p>
318
+
<p><strong>Important:</strong> The condition passed to <codeclass="docutils literal notranslate"><spanclass="pre">branch</span></code> is a <codeclass="docutils literal notranslate"><spanclass="pre">Scalar</span></code> whose <strong>sign</strong> determines which branch executes.
319
+
When you write <codeclass="docutils literal notranslate"><spanclass="pre">a</span><spanclass="pre">></span><spanclass="pre">b</span></code>, SymX internally stores <codeclass="docutils literal notranslate"><spanclass="pre">a</span><spanclass="pre">-</span><spanclass="pre">b</span></code> as the condition scalar.
320
+
Opposite for <codeclass="docutils literal notranslate"><spanclass="pre">a</span><spanclass="pre"><</span><spanclass="pre">b</span></code>.
321
+
The true branch executes when that condition scalar is <strong>strictly positive</strong> (<codeclass="docutils literal notranslate"><spanclass="pre">></span><spanclass="pre">0</span></code>), exact zero will then fall to the false branch.
322
+
Due to the floating point nature of the comparison, the user might consider writing activation mechanisms with conditions that are <codeclass="docutils literal notranslate"><spanclass="pre">+1</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">-1</span></code> (instead of <codeclass="docutils literal notranslate"><spanclass="pre">0</span></code>) to more clearly indicate sign.</p>
0 commit comments