Portability | GHC only |
---|---|
Stability | experimental |
Maintainer | ekmett@gmail.com |
- Gradients (Reverse Mode)
- Jacobians (Mixed Mode)
- Monadic Gradient/Jacobian (Reverse Mode)
- Functorial Gradient/Jacobian (Reverse Mode)
- Transposed Jacobians (Forward Mode)
- Hessian (Forward-On-Reverse)
- Hessian Tensors (Forward-On-Mixed)
- Hessian Vector Products (Forward-On-Reverse)
- Derivatives (Forward Mode)
- Derivatives (Tower)
- Directional Derivatives (Forward Mode)
- Directional Derivatives (Tower)
- Taylor Series (Tower)
- Maclaurin Series (Tower)
- Monadic Combinators (Forward Mode)
- Exposed Types
Mixed-Mode Automatic Differentiation.
Each combinator exported from this module chooses an appropriate AD mode.
- grad :: (Traversable f, Num a) => FU f a -> f a -> f a
- grad' :: (Traversable f, Num a) => FU f a -> f a -> (a, f a)
- gradWith :: (Traversable f, Num a) => (a -> a -> b) -> FU f a -> f a -> f b
- gradWith' :: (Traversable f, Num a) => (a -> a -> b) -> FU f a -> f a -> (a, f b)
- jacobian :: (Traversable f, Traversable g, Num a) => FF f g a -> f a -> g (f a)
- jacobian' :: (Traversable f, Traversable g, Num a) => FF f g a -> f a -> g (a, f a)
- jacobianWith :: (Traversable f, Traversable g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (f b)
- jacobianWith' :: (Traversable f, Traversable g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (a, f b)
- gradM :: (Traversable f, Monad m, Num a) => FF f m a -> f a -> m (f a)
- gradM' :: (Traversable f, Monad m, Num a) => FF f m a -> f a -> m (a, f a)
- gradWithM :: (Traversable f, Monad m, Num a) => (a -> a -> b) -> FF f m a -> f a -> m (f b)
- gradWithM' :: (Traversable f, Monad m, Num a) => (a -> a -> b) -> FF f m a -> f a -> m (a, f b)
- gradF :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> g (f a)
- gradF' :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> g (a, f a)
- gradWithF :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (f b)
- gradWithF' :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (a, f b)
- jacobianT :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> f (g a)
- jacobianWithT :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> FF f g a -> f a -> f (g b)
- hessian :: (Traversable f, Num a) => FU f a -> f a -> f (f a)
- hessianTensor :: (Traversable f, Traversable g, Num a) => FF f g a -> f a -> g (f (f a))
- hessianProduct :: (Traversable f, Num a) => FU f a -> f (a, a) -> f a
- hessianProduct' :: (Traversable f, Num a) => FU f a -> f (a, a) -> f (a, a)
- diff :: Num a => UU a -> a -> a
- diffF :: (Functor f, Num a) => UF f a -> a -> f a
- diff' :: Num a => UU a -> a -> (a, a)
- diffF' :: (Functor f, Num a) => UF f a -> a -> f (a, a)
- diffs :: Num a => UU a -> a -> [a]
- diffsF :: (Functor f, Num a) => UF f a -> a -> f [a]
- diffs0 :: Num a => UU a -> a -> [a]
- diffs0F :: (Functor f, Num a) => UF f a -> a -> f [a]
- du :: (Functor f, Num a) => FU f a -> f (a, a) -> a
- du' :: (Functor f, Num a) => FU f a -> f (a, a) -> (a, a)
- duF :: (Functor f, Functor g, Num a) => FF f g a -> f (a, a) -> g a
- duF' :: (Functor f, Functor g, Num a) => FF f g a -> f (a, a) -> g (a, a)
- dus :: (Functor f, Num a) => FU f a -> f [a] -> [a]
- dus0 :: (Functor f, Num a) => FU f a -> f [a] -> [a]
- dusF :: (Functor f, Functor g, Num a) => FF f g a -> f [a] -> g [a]
- dus0F :: (Functor f, Functor g, Num a) => FF f g a -> f [a] -> g [a]
- taylor :: Fractional a => UU a -> a -> a -> [a]
- taylor0 :: Fractional a => UU a -> a -> a -> [a]
- maclaurin :: Fractional a => UU a -> a -> [a]
- maclaurin0 :: Fractional a => UU a -> a -> [a]
- diffM :: (Monad m, Num a) => UF m a -> a -> m a
- diffM' :: (Monad m, Num a) => UF m a -> a -> m (a, a)
- type UU a = forall s. Mode s => AD s a -> AD s a
- type UF f a = forall s. Mode s => AD s a -> f (AD s a)
- type FU f a = forall s. Mode s => f (AD s a) -> AD s a
- type FF f g a = forall s. Mode s => f (AD s a) -> g (AD s a)
- newtype AD f a = AD {
- runAD :: f a
- class Lifted t => Mode t where
Gradients (Reverse Mode)
grad :: (Traversable f, Num a) => FU f a -> f a -> f aSource
grad' :: (Traversable f, Num a) => FU f a -> f a -> (a, f a)Source
gradWith :: (Traversable f, Num a) => (a -> a -> b) -> FU f a -> f a -> f bSource
function calculates the gradient of a non-scalar-to-scalar function grad
g ff
with reverse-mode AD in a single pass.
The gradient is combined element-wise with the argument using the function g
.
grad == gradWith (\_ dx -> dx) id == gradWith const
gradWith' :: (Traversable f, Num a) => (a -> a -> b) -> FU f a -> f a -> (a, f b)Source
Jacobians (Mixed Mode)
jacobian :: (Traversable f, Traversable g, Num a) => FF f g a -> f a -> g (f a)Source
Calculate the Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward and reverse mode AD based on the number of inputs and outputs.
If you need to support functions where the output is only a Functor
or Monad
, consider Numeric.AD.Reverse.jacobian
or Numeric.AD.Reverse.gradM
from Numeric.AD.Reverse.
jacobian' :: (Traversable f, Traversable g, Num a) => FF f g a -> f a -> g (a, f a)Source
Calculate both the answer and Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward- and reverse- mode AD based on the relative, number of inputs and outputs.
If you need to support functions where the output is only a Functor
or Monad
, consider Numeric.AD.Reverse.jacobian'
or Numeric.AD.Reverse.gradM'
from Numeric.AD.Reverse.
jacobianWith :: (Traversable f, Traversable g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (f b)Source
calculates the Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward and reverse mode AD based on the number of inputs and outputs.
jacobianWith
g f
The resulting Jacobian matrix is then recombined element-wise with the input using g
.
If you need to support functions where the output is only a Functor
or Monad
, consider Numeric.AD.Reverse.jacobianWith
or Numeric.AD.Reverse.gradWithM
from Numeric.AD.Reverse.
jacobianWith' :: (Traversable f, Traversable g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (a, f b)Source
calculates the answer and Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward and reverse mode AD based on the number of inputs and outputs.
jacobianWith'
g f
The resulting Jacobian matrix is then recombined element-wise with the input using g
.
If you need to support functions where the output is only a Functor
or Monad
, consider Numeric.AD.Reverse.jacobianWith'
or Numeric.AD.Reverse.gradWithM'
from Numeric.AD.Reverse.
Monadic Gradient/Jacobian (Reverse Mode)
gradWithM' :: (Traversable f, Monad m, Num a) => (a -> a -> b) -> FF f m a -> f a -> m (a, f b)Source
Functorial Gradient/Jacobian (Reverse Mode)
gradF :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> g (f a)Source
The gradF
function calculates the jacobian of a non-scalar-to-non-scalar function with reverse AD lazily in m
passes for m
outputs.
gradWithF :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (f b)Source
'gradWithF g f' calculates the Jacobian of a non-scalar-to-non-scalar function f
with reverse AD lazily in m
passes for m
outputs.
Instead of returning the Jacobian matrix, the elements of the matrix are combined with the input using the g
.
gradF == gradWithF (\_ dx -> dx) gradWithF const == (\f x -> const x <$> f x)
gradWithF' :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (a, f b)Source
gradWithF
g f' calculates both the result and the Jacobian of a nonscalar-to-nonscalar function f
, using m
invocations of reverse AD,
where m
is the output dimensionality. Applying fmap snd
to the result will recover the result of gradWithF
Instead of returning the Jacobian matrix, the elements of the matrix are combined with the input using the g
.
jacobian' == gradWithF' (\_ dx -> dx)
Transposed Jacobians (Forward Mode)
jacobianT :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> f (g a)Source
A fast, simple transposed Jacobian computed with forward-mode AD.
jacobianWithT :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> FF f g a -> f a -> f (g b)Source
A fast, simple transposed Jacobian computed with forward-mode AD.
Hessian (Forward-On-Reverse)
hessian :: (Traversable f, Num a) => FU f a -> f a -> f (f a)Source
Compute the hessian via the jacobian of the gradient. gradient is computed in reverse mode and then the jacobian is computed in forward mode.
Hessian Tensors (Forward-On-Mixed)
hessianTensor :: (Traversable f, Traversable g, Num a) => FF f g a -> f a -> g (f (f a))Source
Compute the order 3 Hessian tensor on a non-scalar-to-non-scalar function via the forward-mode Jacobian of the mixed-mode Jacobian of the function.
Hessian Vector Products (Forward-On-Reverse)
hessianProduct :: (Traversable f, Num a) => FU f a -> f (a, a) -> f aSource
computes the product of the hessian hessianProduct
f wvH
of a non-scalar-to-scalar function f
at w =
with a vector fst
$ wvv = snd $ wv
using "Pearlmutter's method" from http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.29.6143, which states:
H v = (d/dr) grad_w (w + r v) | r = 0
Or in other words, we take the directional derivative of the gradient.
hessianProduct' :: (Traversable f, Num a) => FU f a -> f (a, a) -> f (a, a)Source
computes both the gradient of a non-scalar-to-scalar hessianProduct'
f wvf
at w =
and the product of the hessian fst
$ wvH
at w
with a vector v = snd $ wv
using "Pearlmutter's method". The outputs are returned wrapped in the same functor.
H v = (d/dr) grad_w (w + r v) | r = 0
Or in other words, we take the directional derivative of the gradient.
Derivatives (Forward Mode)
diff' :: Num a => UU a -> a -> (a, a)Source
The d'UU
function calculates the result and first derivative of scalar-to-scalar function by Forward
AD
d' sin == sin &&& cos d' f = f &&& d f
Derivatives (Tower)
Directional Derivatives (Forward Mode)
Directional Derivatives (Tower)
Taylor Series (Tower)
taylor :: Fractional a => UU a -> a -> a -> [a]Source
taylor0 :: Fractional a => UU a -> a -> a -> [a]Source
Maclaurin Series (Tower)
maclaurin :: Fractional a => UU a -> a -> [a]Source
maclaurin0 :: Fractional a => UU a -> a -> [a]Source
Monadic Combinators (Forward Mode)
diffM :: (Monad m, Num a) => UF m a -> a -> m aSource
The dUM
function calculates the first derivative of scalar-to-scalar monadic function by Forward
AD
diffM' :: (Monad m, Num a) => UF m a -> a -> m (a, a)Source
The d'UM
function calculates the result and first derivative of a scalar-to-scalar monadic function by Forward
AD
Exposed Types
type UU a = forall s. Mode s => AD s a -> AD s aSource
A scalar-to-scalar automatically-differentiable function.
type UF f a = forall s. Mode s => AD s a -> f (AD s a)Source
A scalar-to-non-scalar automatically-differentiable function.
type FU f a = forall s. Mode s => f (AD s a) -> AD s aSource
A non-scalar-to-scalar automatically-differentiable function.
type FF f g a = forall s. Mode s => f (AD s a) -> g (AD s a)Source
A non-scalar-to-non-scalar automatically-differentiable function.
AD
serves as a common wrapper for different Mode
instances, exposing a traditional
numerical tower. Universal quantification is used to limit the actions in user code to
machinery that will return the same answers under all AD modes, allowing us to use modes
interchangeably as both the type level "brand" and dictionary, providing a common API.
Typeable1 f => Typeable1 (AD f) | |
Primal f => Primal (AD f) | |
Mode f => Mode (AD f) | |
Lifted f => Lifted (AD f) | |
Var (AD Reverse) | |
Iso (f a) (AD f a) | |
(Num a, Lifted f, Bounded a) => Bounded (AD f a) | |
(Num a, Lifted f, Enum a) => Enum (AD f a) | |
(Num a, Lifted f, Eq a) => Eq (AD f a) | |
(Lifted f, Floating a) => Floating (AD f a) | |
(Lifted f, Fractional a) => Fractional (AD f a) | |
(Typeable1 f, Typeable a, Data (f a), Data a) => Data (AD f a) | |
(Lifted f, Num a) => Num (AD f a) | |
(Num a, Lifted f, Ord a) => Ord (AD f a) | |
(Lifted f, Real a) => Real (AD f a) | |
(Lifted f, RealFloat a) => RealFloat (AD f a) | |
(Lifted f, RealFrac a) => RealFrac (AD f a) | |
(Lifted f, Show a) => Show (AD f a) | |
(Typeable1 f, Typeable a) => Typeable (AD f a) |
class Lifted t => Mode t whereSource
lift :: Num a => a -> t aSource
Embed a constant
(<+>) :: Num a => t a -> t a -> t aSource
Vector sum
(*^) :: Num a => a -> t a -> t aSource
Scalar-vector multiplication
(^*) :: Num a => t a -> a -> t aSource
Vector-scalar multiplication
(^/) :: Fractional a => t a -> a -> t aSource
Scalar division
'zero' = 'lift' 0