DiffSharp is a tensor library with advanced support for differentiable programming. It is designed for use in machine learning, probabilistic programming, optimization and other domains.

DiffSharp provides advanced automatic differentiation capabilities for tensor code, making it possible to use derivative-taking operations, including gradients, Hessians, Jacobians, directional derivatives, and matrix-free Hessian- and Jacobian-vector products, as higher-order function compositions. This goes far beyond the standard reverse-mode gradients of traditional tensor libraries such as TensorFlow and PyTorch, allowing the use of nested forward and reverse differentiation up to any level, meaning that you can compute higher-order derivatives efficiently or differentiate functions that are internally making use of differentiation. Please see API Overview for a list of available operations.

DiffSharp 1.0 is implemented in F# and uses PyTorch C++ tensors (without the derivative computation graph) as the default raw-tensor backend. It is tested on Linux and Windows. DiffSharp is developed by Atılım Güneş Baydin, Don Syme and other contributors, having started as a project supervised by Barak Pearlmutter and Jeffrey Siskind. Please join us!

**The library and documentation are undergoing development.**

The primary features of DiffSharp 1.0 are:

- A tensor programming model for F#.
- A reference backend for correctness testing.
PyTorch backend for CUDA support and highly optimized native tensor operations.

- Nested differentiation for tensors, supporting forward and reverse AD, or any combination thereof, up to any level.
- Matrix-free Jacobian- and Hessian-vector products.
- Common optimizers and model elements including convolutions.
- Probability distributions.

Below is a sample of using DiffSharp. You can access this sample as a script or a .NET Interactive Jupyter Notebook (open in ).

```
open DiffSharp
// A 1D tensor
let t1 = dsharp.tensor [ 0.0 .. 0.2 .. 1.0 ]
// A 2x2 tensor
let t2 = dsharp.tensor [ [ 0; 1 ]; [ 2; 2 ] ]
// Define a scalar-to-scalar function
let f (x: Tensor) = sin (sqrt x)
f (dsharp.tensor 1.2)
// Get its derivative
let df = dsharp.diff f
df (dsharp.tensor 1.2)
// Now define a vector-to-scalar function
let g (x: Tensor) = exp (x.[0] * x.[1]) + x.[2]
g (dsharp.tensor [ 0.0; 0.3; 0.1 ])
// Now compute the gradient of g
let gg = dsharp.grad g
gg (dsharp.tensor [ 0.0; 0.3; 0.1 ])
// Compute the hessian of g
let hg = dsharp.hessian g
hg (dsharp.tensor [ 0.0; 0.3; 0.1 ])
```

To learn more about DiffSharp, use the navigation links to the left.

If you are using DiffSharp, please raise any issues you might have on GitHub. We also have a Gitter chat room. If you would like to cite this library, please use the following information:

*Baydin, A.G., Pearlmutter, B.A., Radul, A.A. and Siskind, J.M., 2017. Automatic differentiation in machine learning: a survey. The Journal of Machine Learning Research, 18(1), pp.5595-5637.* (link)

namespace DiffSharp

val t1 : Tensor

type dsharp =

static member abs : input:Tensor -> Tensor

static member acos : input:Tensor -> Tensor

static member add : a:Tensor * b:Tensor -> Tensor

static member arange : endVal:int * ?startVal:int * ?step:int * ?dtype:Dtype * ?device:Device * ?backend:Backend -> Tensor

static member arange : endVal:float * ?startVal:float * ?step:float * ?dtype:Dtype * ?device:Device * ?backend:Backend -> Tensor

static member arangeLike : input:Tensor * endVal:int * ?startVal:int * ?step:int * ?dtype:Dtype * ?device:Device * ?backend:Backend -> Tensor

static member arangeLike : input:Tensor * endVal:float * ?startVal:float * ?step:float * ?dtype:Dtype * ?device:Device * ?backend:Backend -> Tensor

static member argmax : input:Tensor -> int []

static member argmin : input:Tensor -> int []

static member asin : input:Tensor -> Tensor

...

static member abs : input:Tensor -> Tensor

static member acos : input:Tensor -> Tensor

static member add : a:Tensor * b:Tensor -> Tensor

static member arange : endVal:int * ?startVal:int * ?step:int * ?dtype:Dtype * ?device:Device * ?backend:Backend -> Tensor

static member arange : endVal:float * ?startVal:float * ?step:float * ?dtype:Dtype * ?device:Device * ?backend:Backend -> Tensor

static member arangeLike : input:Tensor * endVal:int * ?startVal:int * ?step:int * ?dtype:Dtype * ?device:Device * ?backend:Backend -> Tensor

static member arangeLike : input:Tensor * endVal:float * ?startVal:float * ?step:float * ?dtype:Dtype * ?device:Device * ?backend:Backend -> Tensor

static member argmax : input:Tensor -> int []

static member argmin : input:Tensor -> int []

static member asin : input:Tensor -> Tensor

...

static member dsharp.tensor : value:obj * ?dtype:Dtype * ?device:Device * ?backend:Backend -> Tensor

val t2 : Tensor

val f : x:Tensor -> Tensor

val x : Tensor

type Tensor =

private | Tensor0 of primalRaw: RawTensor

| TensorF of primal: Tensor * derivative: Tensor * nestingTag: uint32

| TensorR of primal: Tensor * derivative: Tensor ref * parentOp: TensorOp * fanout: uint32 ref * nestingTag: uint32

interface IConvertible

interface IComparable

override Equals : other:obj -> bool

override GetHashCode : unit -> int

member private GetSlice : bounds:int [,] -> Tensor

override ToString : unit -> string

member abs : unit -> Tensor

member acos : unit -> Tensor

member add : b:scalar -> Tensor

member add : b:Tensor -> Tensor

...

private | Tensor0 of primalRaw: RawTensor

| TensorF of primal: Tensor * derivative: Tensor * nestingTag: uint32

| TensorR of primal: Tensor * derivative: Tensor ref * parentOp: TensorOp * fanout: uint32 ref * nestingTag: uint32

interface IConvertible

interface IComparable

override Equals : other:obj -> bool

override GetHashCode : unit -> int

member private GetSlice : bounds:int [,] -> Tensor

override ToString : unit -> string

member abs : unit -> Tensor

member acos : unit -> Tensor

member add : b:scalar -> Tensor

member add : b:Tensor -> Tensor

...

val sin : value:'T -> 'T (requires member Sin)

val sqrt : value:'T -> 'U (requires member Sqrt)

val df : (Tensor -> Tensor)

static member dsharp.diff : f:(Tensor -> Tensor) -> x:Tensor -> Tensor

val g : x:Tensor -> Tensor

val exp : value:'T -> 'T (requires member Exp)

val gg : (Tensor -> Tensor)

static member dsharp.grad : f:(Tensor -> Tensor) -> x:Tensor -> Tensor

val hg : (Tensor -> Tensor)

static member dsharp.hessian : f:(Tensor -> Tensor) -> x:Tensor -> Tensor

© Copyright 2020, DiffSharp Contributors.