Copyright | (c) Huw Campbell 2016-2017 |
---|---|
License | BSD2 |
Stability | experimental |
Safe Haskell | None |
Language | Haskell98 |
This module defines what a Layer is in a Grenade neural network.
There are two classes of interest: UpdateLayer
and Layer
.
UpdateLayer
is required for all types which are used as a layer
in a network. Having no shape information, this class is agnotostic
to the input and output data of the layer.
An instance of Layer
on the other hand is required for usage in
a neural network, but also specifies the shapes of data that the
network can transform. Multiple instance of Layer
are permitted
for a single type, to transform different shapes. The Reshape
layer
for example can act as a flattening layer, and its inverse, projecting
a 1D shape up to 2 or 3 dimensions.
Instances of Layer
should be as strict as possible, and not emit
runtime errors.
- class UpdateLayer x => Layer x i o where
- class UpdateLayer x where
Documentation
class UpdateLayer x => Layer x i o where Source #
Class for a layer. All layers implement this, however, they don't need to implement it for all shapes, only ones which are appropriate.
The Wengert tape for this layer. Includes all that is required to generate the back propagated gradients efficiently. As a default, `S i` is fine.
runForwards :: x -> S i -> (Tape x i o, S o) Source #
Used in training and scoring. Take the input from the previous layer, and give the output from this layer.
runBackwards :: x -> Tape x i o -> S o -> (Gradient x, S i) Source #
Back propagate a step. Takes the current layer, the input that the layer gave from the input and the back propagated derivatives from the layer above.
Returns the gradient layer and the derivatives to push back further.
((~) Shape a b, SingI Shape a) => Layer Logit a b Source # | |
((~) Shape a b, SingI Shape a) => Layer Tanh a b Source # | |
(~) Shape a b => Layer Trivial a b Source # | |
KnownNat i => Layer Dropout (D1 i) (D1 i) Source # | |
KnownNat i => Layer Elu (D1 i) (D1 i) Source # | |
KnownNat i => Layer Relu (D1 i) (D1 i) Source # | |
KnownNat i => Layer Softmax (D1 i) (D1 i) Source # | |
(KnownNat a, KnownNat x, KnownNat y, (~) Nat a (* x y)) => Layer Reshape (D1 a) (D2 x y) Source # | |
(KnownNat a, KnownNat x, KnownNat y, KnownNat (* x z), KnownNat z, (~) Nat a (* (* x y) z)) => Layer Reshape (D1 a) (D3 x y z) Source # | |
(KnownNat a, KnownNat x, KnownNat y, (~) Nat a (* x y)) => Layer Reshape (D2 x y) (D1 a) Source # | |
(KnownNat i, KnownNat j) => Layer Elu (D2 i j) (D2 i j) Source # | |
(KnownNat i, KnownNat j) => Layer Relu (D2 i j) (D2 i j) Source # | |
(KnownNat y, KnownNat x, KnownNat z, (~) Nat z 1) => Layer Reshape (D2 x y) (D3 x y z) Source # | |
(KnownNat a, KnownNat x, KnownNat y, KnownNat (* x z), KnownNat z, (~) Nat a (* (* x y) z)) => Layer Reshape (D3 x y z) (D1 a) Source # | |
(KnownNat y, KnownNat x, KnownNat z, (~) Nat z 1) => Layer Reshape (D3 x y z) (D2 x y) Source # | |
(KnownNat i, KnownNat j, KnownNat k) => Layer Elu (D3 i j k) (D3 i j k) Source # | |
(KnownNat i, KnownNat j, KnownNat k) => Layer Relu (D3 i j k) (D3 i j k) Source # | |
(CreatableNetwork sublayers subshapes, (~) Shape i (Head Shape subshapes), (~) Shape o (Last Shape subshapes)) => Layer (Network sublayers subshapes) i o Source # | Ultimate composition. This allows a complete network to be treated as a layer in a larger network. |
(SingI Shape i, SingI Shape o, Layer x i o, Layer y i o) => Layer (Merge x y) i o Source # | Combine the outputs and the inputs, summing the output shape |
(KnownNat i, KnownNat o) => Layer (FullyConnected i o) (D1 i) (D1 o) Source # | |
(SingI Shape i, Layer x i (D1 m), Layer y i (D1 n), KnownNat o, KnownNat m, KnownNat n, (~) Nat o ((+) m n), (~) Nat n ((-) o m), (~) Bool ((<=?) m o) True) => Layer (Concat (D1 m) x (D1 n) y) i (D1 o) Source # | |
(SingI Shape i, Layer x i (D1 o), Layer y i (D1 o)) => Layer (Concat (D1 o) x (D1 o) y) i (D2 2 o) Source # | |
(SingI Shape i, Layer x i (D3 rows cols m), Layer y i (D3 rows cols n), KnownNat (* rows n), KnownNat (* rows m), KnownNat (* rows o), KnownNat o, KnownNat m, KnownNat n, (~) Nat ((+) (* rows m) (* rows n)) (* rows o), (~) Nat ((-) (* rows o) (* rows m)) (* rows n), (~) Bool ((<=?) (* rows m) (* rows o)) True) => Layer (Concat (D3 rows cols m) x (D3 rows cols n) y) i (D3 rows cols o) Source # | Concat 3D shapes, increasing the number of channels. |
(KnownNat cropLeft, KnownNat cropTop, KnownNat cropRight, KnownNat cropBottom, KnownNat inputRows, KnownNat inputColumns, KnownNat outputRows, KnownNat outputColumns, (~) Nat ((-) ((-) inputRows cropTop) cropBottom) outputRows, (~) Nat ((-) ((-) inputColumns cropLeft) cropRight) outputColumns) => Layer (Crop cropLeft cropTop cropRight cropBottom) (D2 inputRows inputColumns) (D2 outputRows outputColumns) Source # | A two dimentional image can be cropped. |
(KnownNat padLeft, KnownNat padTop, KnownNat padRight, KnownNat padBottom, KnownNat inputRows, KnownNat inputColumns, KnownNat outputRows, KnownNat outputColumns, (~) Nat ((+) ((+) inputRows padTop) padBottom) outputRows, (~) Nat ((+) ((+) inputColumns padLeft) padRight) outputColumns) => Layer (Pad padLeft padTop padRight padBottom) (D2 inputRows inputColumns) (D2 outputRows outputColumns) Source # | A two dimentional image can be padped. |
(KnownNat kernelRows, KnownNat kernelColumns, KnownNat strideRows, KnownNat strideColumns, KnownNat inputRows, KnownNat inputColumns, KnownNat outputRows, KnownNat outputColumns, (~) Nat (* ((-) outputRows 1) strideRows) ((-) inputRows kernelRows), (~) Nat (* ((-) outputColumns 1) strideColumns) ((-) inputColumns kernelColumns)) => Layer (Pooling kernelRows kernelColumns strideRows strideColumns) (D2 inputRows inputColumns) (D2 outputRows outputColumns) Source # | A two dimentional image can be pooled. |
(KnownNat cropLeft, KnownNat cropTop, KnownNat cropRight, KnownNat cropBottom, KnownNat inputRows, KnownNat inputColumns, KnownNat outputRows, KnownNat outputColumns, KnownNat channels, KnownNat (* inputRows channels), KnownNat (* outputRows channels), (~) Nat ((+) ((+) outputRows cropTop) cropBottom) inputRows, (~) Nat ((+) ((+) outputColumns cropLeft) cropRight) inputColumns) => Layer (Crop cropLeft cropTop cropRight cropBottom) (D3 inputRows inputColumns channels) (D3 outputRows outputColumns channels) Source # | A two dimentional image can be cropped. |
(KnownNat padLeft, KnownNat padTop, KnownNat padRight, KnownNat padBottom, KnownNat inputRows, KnownNat inputColumns, KnownNat outputRows, KnownNat outputColumns, KnownNat channels, KnownNat (* inputRows channels), KnownNat (* outputRows channels), (~) Nat ((+) ((+) inputRows padTop) padBottom) outputRows, (~) Nat ((+) ((+) inputColumns padLeft) padRight) outputColumns) => Layer (Pad padLeft padTop padRight padBottom) (D3 inputRows inputColumns channels) (D3 outputRows outputColumns channels) Source # | A two dimentional image can be padped. |
(KnownNat kernelRows, KnownNat kernelColumns, KnownNat strideRows, KnownNat strideColumns, KnownNat inputRows, KnownNat inputColumns, KnownNat outputRows, KnownNat outputColumns, KnownNat channels, KnownNat (* outputRows channels), (~) Nat (* ((-) outputRows 1) strideRows) ((-) inputRows kernelRows), (~) Nat (* ((-) outputColumns 1) strideColumns) ((-) inputColumns kernelColumns)) => Layer (Pooling kernelRows kernelColumns strideRows strideColumns) (D3 inputRows inputColumns channels) (D3 outputRows outputColumns channels) Source # | A three dimensional image can be pooled on each layer. |
(KnownNat kernelRows, KnownNat kernelCols, KnownNat strideRows, KnownNat strideCols, KnownNat inputRows, KnownNat inputCols, KnownNat outputRows, KnownNat outputCols, (~) Nat (* ((-) outputRows 1) strideRows) ((-) inputRows kernelRows), (~) Nat (* ((-) outputCols 1) strideCols) ((-) inputCols kernelCols), KnownNat (* (* kernelRows kernelCols) 1), KnownNat (* outputRows 1)) => Layer (Convolution 1 1 kernelRows kernelCols strideRows strideCols) (D2 inputRows inputCols) (D2 outputRows outputCols) Source # | A two dimensional image may have a convolution filter applied to it producing a two dimensional image if both channels and filters is 1. |
(KnownNat kernelRows, KnownNat kernelCols, KnownNat strideRows, KnownNat strideCols, KnownNat inputRows, KnownNat inputCols, KnownNat outputRows, KnownNat outputCols, (~) Nat (* ((-) inputRows 1) strideRows) ((-) outputRows kernelRows), (~) Nat (* ((-) inputCols 1) strideCols) ((-) outputCols kernelCols), KnownNat (* (* kernelRows kernelCols) 1), KnownNat (* outputRows 1)) => Layer (Deconvolution 1 1 kernelRows kernelCols strideRows strideCols) (D2 inputRows inputCols) (D2 outputRows outputCols) Source # | A two dimentional image may have a Deconvolution filter applied to it |
(KnownNat kernelRows, KnownNat kernelCols, KnownNat filters, KnownNat strideRows, KnownNat strideCols, KnownNat inputRows, KnownNat inputCols, KnownNat outputRows, KnownNat outputCols, (~) Nat (* ((-) outputRows 1) strideRows) ((-) inputRows kernelRows), (~) Nat (* ((-) outputCols 1) strideCols) ((-) inputCols kernelCols), KnownNat (* (* kernelRows kernelCols) 1), KnownNat (* outputRows filters)) => Layer (Convolution 1 filters kernelRows kernelCols strideRows strideCols) (D2 inputRows inputCols) (D3 outputRows outputCols filters) Source # | A two dimentional image may have a convolution filter applied to it |
(KnownNat kernelRows, KnownNat kernelCols, KnownNat filters, KnownNat strideRows, KnownNat strideCols, KnownNat inputRows, KnownNat inputCols, KnownNat outputRows, KnownNat outputCols, (~) Nat (* ((-) inputRows 1) strideRows) ((-) outputRows kernelRows), (~) Nat (* ((-) inputCols 1) strideCols) ((-) outputCols kernelCols), KnownNat (* (* kernelRows kernelCols) filters), KnownNat (* outputRows filters)) => Layer (Deconvolution 1 filters kernelRows kernelCols strideRows strideCols) (D2 inputRows inputCols) (D3 outputRows outputCols filters) Source # | A two dimentional image may have a Deconvolution filter applied to it |
(KnownNat kernelRows, KnownNat kernelCols, KnownNat strideRows, KnownNat strideCols, KnownNat inputRows, KnownNat inputCols, KnownNat outputRows, KnownNat outputCols, KnownNat channels, (~) Nat (* ((-) outputRows 1) strideRows) ((-) inputRows kernelRows), (~) Nat (* ((-) outputCols 1) strideCols) ((-) inputCols kernelCols), KnownNat (* (* kernelRows kernelCols) channels), KnownNat (* outputRows 1)) => Layer (Convolution channels 1 kernelRows kernelCols strideRows strideCols) (D3 inputRows inputCols channels) (D2 outputRows outputCols) Source # | A three dimensional image can produce a 2D image from a convolution with 1 filter |
(KnownNat kernelRows, KnownNat kernelCols, KnownNat strideRows, KnownNat strideCols, KnownNat inputRows, KnownNat inputCols, KnownNat outputRows, KnownNat outputCols, (~) Nat (* ((-) inputRows 1) strideRows) ((-) outputRows kernelRows), (~) Nat (* ((-) inputCols 1) strideCols) ((-) outputCols kernelCols), KnownNat (* (* kernelRows kernelCols) 1), KnownNat (* outputRows 1), KnownNat channels) => Layer (Deconvolution channels 1 kernelRows kernelCols strideRows strideCols) (D3 inputRows inputCols channels) (D2 outputRows outputCols) Source # | A two dimentional image may have a Deconvolution filter applied to it |
(KnownNat kernelRows, KnownNat kernelCols, KnownNat filters, KnownNat strideRows, KnownNat strideCols, KnownNat inputRows, KnownNat inputCols, KnownNat outputRows, KnownNat outputCols, KnownNat channels, (~) Nat (* ((-) outputRows 1) strideRows) ((-) inputRows kernelRows), (~) Nat (* ((-) outputCols 1) strideCols) ((-) inputCols kernelCols), KnownNat (* (* kernelRows kernelCols) channels), KnownNat (* outputRows filters)) => Layer (Convolution channels filters kernelRows kernelCols strideRows strideCols) (D3 inputRows inputCols channels) (D3 outputRows outputCols filters) Source # | A three dimensional image (or 2d with many channels) can have an appropriately sized convolution filter run across it. |
(KnownNat kernelRows, KnownNat kernelCols, KnownNat filters, KnownNat strideRows, KnownNat strideCols, KnownNat inputRows, KnownNat inputCols, KnownNat outputRows, KnownNat outputCols, KnownNat channels, (~) Nat (* ((-) inputRows 1) strideRows) ((-) outputRows kernelRows), (~) Nat (* ((-) inputCols 1) strideCols) ((-) outputCols kernelCols), KnownNat (* (* kernelRows kernelCols) filters), KnownNat (* outputRows filters)) => Layer (Deconvolution channels filters kernelRows kernelCols strideRows strideCols) (D3 inputRows inputCols channels) (D3 outputRows outputCols filters) Source # | A three dimensional image (or 2d with many channels) can have an appropriately sized Deconvolution filter run across it. |
class UpdateLayer x where Source #
Class for updating a layer. All layers implement this, as it describes how to create and update the layer.
The type for the gradient for this layer. Unit if there isn't a gradient to pass back.
runUpdate :: LearningParameters -> x -> Gradient x -> x Source #
Update a layer with its gradient and learning parameters
createRandom :: MonadRandom m => m x Source #
Create a random layer, many layers will use pure
runUpdates :: LearningParameters -> x -> [Gradient x] -> x Source #
Update a layer with many Gradients
UpdateLayer Dropout Source # | |
UpdateLayer Elu Source # | |
UpdateLayer Logit Source # | |
UpdateLayer Relu Source # | |
UpdateLayer Reshape Source # | |
UpdateLayer Softmax Source # | |
UpdateLayer Tanh Source # | |
UpdateLayer Trivial Source # | |
CreatableNetwork sublayers subshapes => UpdateLayer (Network sublayers subshapes) Source # | Ultimate composition. This allows a complete network to be treated as a layer in a larger network. |
(KnownNat i, KnownNat o) => UpdateLayer (FullyConnected i o) Source # | |
(UpdateLayer x, UpdateLayer y) => UpdateLayer (Merge x y) Source # | Run two layers in parallel, combining their outputs. This just kind of "smooshes" the weights together. |
(KnownNat i, KnownNat o, KnownNat ((+) i o)) => UpdateLayer (BasicRecurrent i o) Source # | |
(KnownNat i, KnownNat o) => UpdateLayer (LSTM i o) Source # | |
(UpdateLayer x, UpdateLayer y) => UpdateLayer (Concat m x n y) Source # | Run two layers in parallel, combining their outputs. |
UpdateLayer (Crop l t r b) Source # | |
UpdateLayer (Pad l t r b) Source # | |
UpdateLayer (Pooling kernelRows kernelColumns strideRows strideColumns) Source # | |
(KnownNat channels, KnownNat filters, KnownNat kernelRows, KnownNat kernelColumns, KnownNat strideRows, KnownNat strideColumns, KnownNat (* (* kernelRows kernelColumns) channels)) => UpdateLayer (Convolution channels filters kernelRows kernelColumns strideRows strideColumns) Source # | |
(KnownNat channels, KnownNat filters, KnownNat kernelRows, KnownNat kernelColumns, KnownNat strideRows, KnownNat strideColumns, KnownNat (* (* kernelRows kernelColumns) filters)) => UpdateLayer (Deconvolution channels filters kernelRows kernelColumns strideRows strideColumns) Source # | |