Streamly.Internal.Data.MutByteArray

MutByteArray

MutByteArray

data MutByteArray

A lifted mutable byte array type wrapping MutableByteArray# RealWorld. This is a low level array used to back high level unboxed arrays and serialized data.

Constructors

MutByteArray (MutableByteArray# RealWorld)

type MutableByteArray = MutByteArray

Deprecated: Please use MutByteArray instead

getMutableByteArray# :: MutByteArray -> MutableByteArray# RealWorld

Pinning

data PinnedState

Constructors

Pinned

Unpinned

Instances

Show PinnedState

Defined in Streamly.Internal.Data.MutByteArray.Type

Methods

showsPrec :: Int -> PinnedState -> ShowS

show :: PinnedState -> String

showList :: [PinnedState] -> ShowS

Eq PinnedState

Defined in Streamly.Internal.Data.MutByteArray.Type

Methods

(==) :: PinnedState -> PinnedState -> Bool

(/=) :: PinnedState -> PinnedState -> Bool

isPinned :: MutByteArray -> Bool

Return True if the array is allocated in pinned memory.

pin :: MutByteArray -> IO MutByteArray

Return a copy of the array in pinned memory if unpinned, else return the original array.

unpin :: MutByteArray -> IO MutByteArray

Return a copy of the array in unpinned memory if pinned, else return the original array.

Allocation

empty :: MutByteArray

newBytesAs :: PinnedState -> Int -> IO MutByteArray

new :: Int -> IO MutByteArray

pinnedNew :: Int -> IO MutByteArray

pinnedNewAlignedBytes :: Int -> Int -> IO MutByteArray

Access

sizeOfMutableByteArray :: MutByteArray -> IO Int

Return the size of the array in bytes.

putSliceUnsafe :: MonadIO m => MutByteArray -> Int -> MutByteArray -> Int -> Int -> m ()

Put a sub range of a source array into a subrange of a destination array. This is not safe as it does not check the bounds of neither the src array nor the destination array.

cloneSliceUnsafeAs :: MonadIO m => PinnedState -> Int -> Int -> MutByteArray -> m MutByteArray

Unsafe as it does not check whether the start offset and length supplied are valid inside the array.

cloneSliceUnsafe :: MonadIO m => Int -> Int -> MutByteArray -> m MutByteArray

cloneSliceUnsafe offset len arr clones a slice of the supplied array starting at the given offset and equal to the given length.

pinnedCloneSliceUnsafe :: MonadIO m => Int -> Int -> MutByteArray -> m MutByteArray

pinnedCloneSliceUnsafe offset len arr

unsafePinnedAsPtr :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b

Use a MutByteArray as Ptr a. This is useful when we want to pass an array as a pointer to some operating system call or to a "safe" FFI call.

If the array is not pinned it is copied to pinned memory before passing it to the monadic action.

Performance Notes: Forces a copy if the array is not pinned. It is advised that the programmer keeps this in mind and creates a pinned array opportunistically before this operation occurs, to avoid the cost of a copy if possible.

Unsafe because of direct pointer operations. The user must ensure that they are writing within the legal bounds of the array.

Pre-release

unsafeAsPtr :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b

For use with unsafe FFI functions. Does not force pin the array memory.

Deprecated

asPtrUnsafe :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b

Deprecated: Please use unsafePinnedAsPtr instead.

nil :: MutByteArray

Deprecated: Please use empty instead

Unbox

Unbox type class

class Unbox a where

The Unbox type class provides operations for serialization (unboxing) and deserialization (boxing) of fixed-length, non-recursive Haskell data types to and from their byte stream representation.

Unbox uses fixed size encoding, therefore, size is independent of the value, it must be determined solely by the type. This restriction makes types with Unbox instances suitable for storing in arrays. Note that sum types may have multiple constructors of different sizes, the size of a sum type is computed as the maximum required by any constructor.

The peekAt operation reads as many bytes from the mutable byte array as the size of the data type and builds a Haskell data type from these bytes. pokeAt operation converts a Haskell data type to its binary representation which consists of size bytes and then stores these bytes into the mutable byte array. These operations do not check the bounds of the array, the user of the type class is expected to check the bounds before peeking or poking.

IMPORTANT: The serialized data's byte ordering remains the same as the host machine's byte order. Therefore, it can not be deserialized from host machines with a different byte ordering.

Instances can be derived via Generics, Template Haskell, or written manually. Note that the data type must be non-recursive. WARNING! Generic and Template Haskell deriving, both hang for recursive data types. Deriving via Generics is more convenient but Template Haskell should be preferred over Generics for the following reasons:

1. Instances derived via Template Haskell provide better and more reliable performance.
2. Generic deriving allows only 256 fields or constructor tags whereas template Haskell has no limit.

Here is an example, for deriving an instance of this type class using generics:

>>> import GHC.Generics (Generic)
>>> :{
data Object = Object
    { _int0 :: Int
    , _int1 :: Int
    } deriving Generic
:}

>>> import Streamly.Data.MutByteArray (Unbox(..))
>>> instance Unbox Object

To derive the instance via Template Haskell:

import Streamly.Data.MutByteArray (deriveUnbox)
$(deriveUnbox [d|instance Unbox Object|])

See deriveUnbox for more information on deriving using Template Haskell.

If you want to write the instance manually:

>>> :{
instance Unbox Object where
    sizeOf _ = 16
    peekAt i arr = do
       -- Check the array bounds
        x0 <- peekAt i arr
        x1 <- peekAt (i + 8) arr
        return $ Object x0 x1
    pokeAt i arr (Object x0 x1) = do
       -- Check the array bounds
        pokeAt i arr x0
        pokeAt (i + 8) arr x1
:}

Minimal complete definition

Nothing

Methods

sizeOf :: Proxy a -> Int

Get the size. Size cannot be zero, should be at least 1 byte.

default sizeOf :: SizeOfRep (Rep a) => Proxy a -> Int

peekAt :: Int -> MutByteArray -> IO a

peekAt byte-offset array reads an element of type a from the the given the byte offset in the array.

IMPORTANT: The implementation of this interface may not check the bounds of the array, the caller must not assume that.

default peekAt :: (Generic a, PeekRep (Rep a)) => Int -> MutByteArray -> IO a

peekByteIndex :: Int -> MutByteArray -> IO a

Deprecated: Use peekAt.

pokeAt :: Int -> MutByteArray -> a -> IO ()

pokeAt byte-offset array writes an element of type a to the the given the byte offset in the array.

IMPORTANT: The implementation of this interface may not check the bounds of the array, the caller must not assume that.

default pokeAt :: (Generic a, PokeRep (Rep a)) => Int -> MutByteArray -> a -> IO ()

pokeByteIndex :: Int -> MutByteArray -> a -> IO ()

Deprecated: Use pokeAt.

Instances

Unbox IntPtr

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy IntPtr -> Int

peekAt :: Int -> MutByteArray -> IO IntPtr

peekByteIndex :: Int -> MutByteArray -> IO IntPtr

pokeAt :: Int -> MutByteArray -> IntPtr -> IO ()

pokeByteIndex :: Int -> MutByteArray -> IntPtr -> IO ()

Unbox WordPtr

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy WordPtr -> Int

peekAt :: Int -> MutByteArray -> IO WordPtr

peekByteIndex :: Int -> MutByteArray -> IO WordPtr

pokeAt :: Int -> MutByteArray -> WordPtr -> IO ()

pokeByteIndex :: Int -> MutByteArray -> WordPtr -> IO ()

Unbox Fingerprint

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Fingerprint -> Int

peekAt :: Int -> MutByteArray -> IO Fingerprint

peekByteIndex :: Int -> MutByteArray -> IO Fingerprint

pokeAt :: Int -> MutByteArray -> Fingerprint -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Fingerprint -> IO ()

Unbox Int16

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Int16 -> Int

peekAt :: Int -> MutByteArray -> IO Int16

peekByteIndex :: Int -> MutByteArray -> IO Int16

pokeAt :: Int -> MutByteArray -> Int16 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Int16 -> IO ()

Unbox Int32

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Int32 -> Int

peekAt :: Int -> MutByteArray -> IO Int32

peekByteIndex :: Int -> MutByteArray -> IO Int32

pokeAt :: Int -> MutByteArray -> Int32 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Int32 -> IO ()

Unbox Int64

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Int64 -> Int

peekAt :: Int -> MutByteArray -> IO Int64

peekByteIndex :: Int -> MutByteArray -> IO Int64

pokeAt :: Int -> MutByteArray -> Int64 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Int64 -> IO ()

Unbox Int8

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Int8 -> Int

peekAt :: Int -> MutByteArray -> IO Int8

peekByteIndex :: Int -> MutByteArray -> IO Int8

pokeAt :: Int -> MutByteArray -> Int8 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Int8 -> IO ()

Unbox IoSubSystem

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy IoSubSystem -> Int

peekAt :: Int -> MutByteArray -> IO IoSubSystem

peekByteIndex :: Int -> MutByteArray -> IO IoSubSystem

pokeAt :: Int -> MutByteArray -> IoSubSystem -> IO ()

pokeByteIndex :: Int -> MutByteArray -> IoSubSystem -> IO ()

Unbox Word16

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Word16 -> Int

peekAt :: Int -> MutByteArray -> IO Word16

peekByteIndex :: Int -> MutByteArray -> IO Word16

pokeAt :: Int -> MutByteArray -> Word16 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Word16 -> IO ()

Unbox Word32

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Word32 -> Int

peekAt :: Int -> MutByteArray -> IO Word32

peekByteIndex :: Int -> MutByteArray -> IO Word32

pokeAt :: Int -> MutByteArray -> Word32 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Word32 -> IO ()

Unbox Word64

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Word64 -> Int

peekAt :: Int -> MutByteArray -> IO Word64

peekByteIndex :: Int -> MutByteArray -> IO Word64

pokeAt :: Int -> MutByteArray -> Word64 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Word64 -> IO ()

Unbox Word8

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Word8 -> Int

peekAt :: Int -> MutByteArray -> IO Word8

peekByteIndex :: Int -> MutByteArray -> IO Word8

pokeAt :: Int -> MutByteArray -> Word8 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Word8 -> IO ()

Unbox MicroSecond64

Defined in Streamly.Internal.Data.Time.Units

Methods

sizeOf :: Proxy MicroSecond64 -> Int

peekAt :: Int -> MutByteArray -> IO MicroSecond64

peekByteIndex :: Int -> MutByteArray -> IO MicroSecond64

pokeAt :: Int -> MutByteArray -> MicroSecond64 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> MicroSecond64 -> IO ()

Unbox MilliSecond64

Defined in Streamly.Internal.Data.Time.Units

Methods

sizeOf :: Proxy MilliSecond64 -> Int

peekAt :: Int -> MutByteArray -> IO MilliSecond64

peekByteIndex :: Int -> MutByteArray -> IO MilliSecond64

pokeAt :: Int -> MutByteArray -> MilliSecond64 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> MilliSecond64 -> IO ()

Unbox NanoSecond64

Defined in Streamly.Internal.Data.Time.Units

Methods

sizeOf :: Proxy NanoSecond64 -> Int

peekAt :: Int -> MutByteArray -> IO NanoSecond64

peekByteIndex :: Int -> MutByteArray -> IO NanoSecond64

pokeAt :: Int -> MutByteArray -> NanoSecond64 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> NanoSecond64 -> IO ()

Unbox ()

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy () -> Int

peekAt :: Int -> MutByteArray -> IO ()

peekByteIndex :: Int -> MutByteArray -> IO ()

pokeAt :: Int -> MutByteArray -> () -> IO ()

pokeByteIndex :: Int -> MutByteArray -> () -> IO ()

Unbox Bool

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Bool -> Int

peekAt :: Int -> MutByteArray -> IO Bool

peekByteIndex :: Int -> MutByteArray -> IO Bool

pokeAt :: Int -> MutByteArray -> Bool -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Bool -> IO ()

Unbox Char

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Char -> Int

peekAt :: Int -> MutByteArray -> IO Char

peekByteIndex :: Int -> MutByteArray -> IO Char

pokeAt :: Int -> MutByteArray -> Char -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Char -> IO ()

Unbox Double

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Double -> Int

peekAt :: Int -> MutByteArray -> IO Double

peekByteIndex :: Int -> MutByteArray -> IO Double

pokeAt :: Int -> MutByteArray -> Double -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Double -> IO ()

Unbox Float

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Float -> Int

peekAt :: Int -> MutByteArray -> IO Float

peekByteIndex :: Int -> MutByteArray -> IO Float

pokeAt :: Int -> MutByteArray -> Float -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Float -> IO ()

Unbox Int

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Int -> Int

peekAt :: Int -> MutByteArray -> IO Int

peekByteIndex :: Int -> MutByteArray -> IO Int

pokeAt :: Int -> MutByteArray -> Int -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Int -> IO ()

Unbox Word

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Word -> Int

peekAt :: Int -> MutByteArray -> IO Word

peekByteIndex :: Int -> MutByteArray -> IO Word

pokeAt :: Int -> MutByteArray -> Word -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Word -> IO ()

Unbox a => Unbox (Complex a)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy (Complex a) -> Int

peekAt :: Int -> MutByteArray -> IO (Complex a)

peekByteIndex :: Int -> MutByteArray -> IO (Complex a)

pokeAt :: Int -> MutByteArray -> Complex a -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Complex a -> IO ()

Unbox a => Unbox (Identity a)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy (Identity a) -> Int

peekAt :: Int -> MutByteArray -> IO (Identity a)

peekByteIndex :: Int -> MutByteArray -> IO (Identity a)

pokeAt :: Int -> MutByteArray -> Identity a -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Identity a -> IO ()

Unbox a => Unbox (Down a)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy (Down a) -> Int

peekAt :: Int -> MutByteArray -> IO (Down a)

peekByteIndex :: Int -> MutByteArray -> IO (Down a)

pokeAt :: Int -> MutByteArray -> Down a -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Down a -> IO ()

Unbox (FunPtr a)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy (FunPtr a) -> Int

peekAt :: Int -> MutByteArray -> IO (FunPtr a)

peekByteIndex :: Int -> MutByteArray -> IO (FunPtr a)

pokeAt :: Int -> MutByteArray -> FunPtr a -> IO ()

pokeByteIndex :: Int -> MutByteArray -> FunPtr a -> IO ()

Unbox (Ptr a)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy (Ptr a) -> Int

peekAt :: Int -> MutByteArray -> IO (Ptr a)

peekByteIndex :: Int -> MutByteArray -> IO (Ptr a)

pokeAt :: Int -> MutByteArray -> Ptr a -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Ptr a -> IO ()

Unbox a => Unbox (Ratio a)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy (Ratio a) -> Int

peekAt :: Int -> MutByteArray -> IO (Ratio a)

peekByteIndex :: Int -> MutByteArray -> IO (Ratio a)

pokeAt :: Int -> MutByteArray -> Ratio a -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Ratio a -> IO ()

Unbox (StablePtr a)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy (StablePtr a) -> Int

peekAt :: Int -> MutByteArray -> IO (StablePtr a)

peekByteIndex :: Int -> MutByteArray -> IO (StablePtr a)

pokeAt :: Int -> MutByteArray -> StablePtr a -> IO ()

pokeByteIndex :: Int -> MutByteArray -> StablePtr a -> IO ()

Unbox a => Unbox (Const a b)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy (Const a b) -> Int

peekAt :: Int -> MutByteArray -> IO (Const a b)

peekByteIndex :: Int -> MutByteArray -> IO (Const a b)

pokeAt :: Int -> MutByteArray -> Const a b -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Const a b -> IO ()

Peek and poke utilities

data BoundedPtr

A location inside a mutable byte array with the bound of the array. Is it cheaper to just get the bound using the size of the array whenever needed?

Constructors

BoundedPtr MutByteArray Int Int

newtype Peeker a

Chains peek functions that pass the current position to the next function

Constructors

Peeker (Builder BoundedPtr IO a)

Instances

Applicative Peeker

Defined in Streamly.Internal.Data.Unbox

Methods

pure :: a -> Peeker a

(<*>) :: Peeker (a -> b) -> Peeker a -> Peeker b

liftA2 :: (a -> b -> c) -> Peeker a -> Peeker b -> Peeker c

(*>) :: Peeker a -> Peeker b -> Peeker b

(<*) :: Peeker a -> Peeker b -> Peeker a

Functor Peeker

Defined in Streamly.Internal.Data.Unbox

Methods

fmap :: (a -> b) -> Peeker a -> Peeker b

(<$) :: a -> Peeker b -> Peeker a

Monad Peeker

Defined in Streamly.Internal.Data.Unbox

Methods

(>>=) :: Peeker a -> (a -> Peeker b) -> Peeker b

(>>) :: Peeker a -> Peeker b -> Peeker b

return :: a -> Peeker a

read :: Unbox a => Peeker a

readUnsafe :: Unbox a => Peeker a

skipByte :: Peeker ()

runPeeker :: Peeker a -> BoundedPtr -> IO a

pokeBoundedPtrUnsafe :: forall a. Unbox a => a -> BoundedPtr -> IO BoundedPtr

pokeBoundedPtr :: forall a. Unbox a => a -> BoundedPtr -> IO BoundedPtr

Generic Deriving

class PeekRep (f :: Type -> Type) where

Methods

peekRep :: Peeker (f x)

Instances

PeekRep (U1 :: Type -> Type)

Defined in Streamly.Internal.Data.Unbox

Methods

peekRep :: Peeker (U1 x)

PeekRep (V1 :: Type -> Type)

Defined in Streamly.Internal.Data.Unbox

Methods

peekRep :: Peeker (V1 x)

(PeekRep f, PeekRep g) => PeekRep (f :*: g)

Defined in Streamly.Internal.Data.Unbox

Methods

peekRep :: Peeker ((f :*: g) x)

(MaxArity256 (SumArity (f :+: g)), KnownNat (SumArity (f :+: g)), PeekRepSum 0 (f :+: g)) => PeekRep (f :+: g)

Defined in Streamly.Internal.Data.Unbox

Methods

peekRep :: Peeker ((f :+: g) x)

Unbox a => PeekRep (K1 i a :: Type -> Type)

Defined in Streamly.Internal.Data.Unbox

Methods

peekRep :: Peeker (K1 i a x)

PeekRep f => PeekRep (M1 i c f)

Defined in Streamly.Internal.Data.Unbox

Methods

peekRep :: Peeker (M1 i c f x)

class PokeRep (f :: Type -> Type) where

Methods

pokeRep :: f a -> BoundedPtr -> IO BoundedPtr

Instances

PokeRep (U1 :: Type -> Type)

Defined in Streamly.Internal.Data.Unbox

Methods

pokeRep :: U1 a -> BoundedPtr -> IO BoundedPtr

PokeRep (V1 :: Type -> Type)

Defined in Streamly.Internal.Data.Unbox

Methods

pokeRep :: V1 a -> BoundedPtr -> IO BoundedPtr

(PokeRep f, PokeRep g) => PokeRep (f :*: g)

Defined in Streamly.Internal.Data.Unbox

Methods

pokeRep :: (f :*: g) a -> BoundedPtr -> IO BoundedPtr

(MaxArity256 (SumArity (f :+: g)), PokeRepSum 0 (f :+: g)) => PokeRep (f :+: g)

Defined in Streamly.Internal.Data.Unbox

Methods

pokeRep :: (f :+: g) a -> BoundedPtr -> IO BoundedPtr

Unbox a => PokeRep (K1 i a :: Type -> Type)

Defined in Streamly.Internal.Data.Unbox

Methods

pokeRep :: K1 i a a0 -> BoundedPtr -> IO BoundedPtr

PokeRep f => PokeRep (M1 i c f)

Defined in Streamly.Internal.Data.Unbox

Methods

pokeRep :: M1 i c f a -> BoundedPtr -> IO BoundedPtr

class SizeOfRep (f :: Type -> Type) where

Implementation of sizeOf that works on the generic representation of an ADT.

Methods

sizeOfRep :: f x -> Int

Instances

SizeOfRep (U1 :: Type -> Type)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOfRep :: U1 x -> Int

SizeOfRep (V1 :: Type -> Type)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOfRep :: V1 x -> Int

(SizeOfRep f, SizeOfRep g) => SizeOfRep (f :*: g)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOfRep :: (f :*: g) x -> Int

(MaxArity256 (SumArity (f :+: g)), SizeOfRepSum f, SizeOfRepSum g) => SizeOfRep (f :+: g)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOfRep :: (f :+: g) x -> Int

Unbox a => SizeOfRep (K1 i a :: Type -> Type)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOfRep :: K1 i a x -> Int

SizeOfRep f => SizeOfRep (M1 i c f)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOfRep :: M1 i c f x -> Int

genericSizeOf :: forall a. SizeOfRep (Rep a) => Proxy a -> Int

genericPeekByteIndex :: (Generic a, PeekRep (Rep a)) => MutByteArray -> Int -> IO a

genericPokeByteIndex :: (Generic a, PokeRep (Rep a)) => MutByteArray -> Int -> a -> IO ()

deriveUnbox :: Q [Dec] -> Q [Dec]

Given an Unbox instance declaration splice without the methods (e.g. [d|instance Unbox a => Unbox (Maybe a)|]), generate an instance declaration including all the type class method implementations.

Usage:

$(deriveUnbox [d|instance Unbox a => Unbox (Maybe a)|])

data DataCon

Simplified info about a Con. Omits deriving, strictness, and kind info. This is much nicer than consuming Con directly, because it unifies all the constructors into one.

Constructors

DataCon

Fields

• dcName :: Name
• dcTvs :: [Name]
• dcCxt :: Cxt
• dcFields :: [(Maybe Name, Type)]

Instances

Show DataCon

Defined in Streamly.Internal.Data.Unbox.TH

Methods

showsPrec :: Int -> DataCon -> ShowS

show :: DataCon -> String

showList :: [DataCon] -> ShowS

Eq DataCon

Defined in Streamly.Internal.Data.Unbox.TH

Methods

(==) :: DataCon -> DataCon -> Bool

(/=) :: DataCon -> DataCon -> Bool

Ord DataCon

Defined in Streamly.Internal.Data.Unbox.TH

Methods

compare :: DataCon -> DataCon -> Ordering

(<) :: DataCon -> DataCon -> Bool

(<=) :: DataCon -> DataCon -> Bool

(>) :: DataCon -> DataCon -> Bool

(>=) :: DataCon -> DataCon -> Bool

max :: DataCon -> DataCon -> DataCon

min :: DataCon -> DataCon -> DataCon

data DataType

Simplified info about a DataD. Omits deriving, strictness, kind info, and whether it's data or newtype.

Constructors

DataType

Fields

• dtName :: Name
• dtTvs :: [Name]
• dtCxt :: Cxt
• dtCons :: [DataCon]

Instances

Show DataType

Defined in Streamly.Internal.Data.Unbox.TH

Methods

showsPrec :: Int -> DataType -> ShowS

show :: DataType -> String

showList :: [DataType] -> ShowS

Eq DataType

Defined in Streamly.Internal.Data.Unbox.TH

Methods

(==) :: DataType -> DataType -> Bool

(/=) :: DataType -> DataType -> Bool

Ord DataType

Defined in Streamly.Internal.Data.Unbox.TH

Methods

compare :: DataType -> DataType -> Ordering

(<) :: DataType -> DataType -> Bool

(<=) :: DataType -> DataType -> Bool

(>) :: DataType -> DataType -> Bool

(>=) :: DataType -> DataType -> Bool

max :: DataType -> DataType -> DataType

min :: DataType -> DataType -> DataType

reifyDataType :: Name -> Q DataType

Reify the given data or newtype declaration, and yields its DataType representation.

Serialize

class Serialize a where

The Serialize type class provides operations for serialization and deserialization of general Haskell data types to and from their byte stream representation.

Unlike Unbox, Serialize uses variable length encoding, therefore, it can serialize recursive and variable length data types like lists, or variable length sum types where the length of the value may vary depending on a particular constructor. For variable length data types the length is encoded along with the data.

The deserializeAt operation reads bytes from the mutable byte array and builds a Haskell data type from these bytes, the number of bytes it reads depends on the type and the encoded value it is reading. serializeAt operation converts a Haskell data type to its binary representation which must consist of as many bytes as added by the addSizeTo operation for that value and then stores these bytes into the mutable byte array. The programmer is expected to use the addSizeTo operation and allocate an array of sufficient length before calling serializeAt.

IMPORTANT: The serialized data's byte ordering remains the same as the host machine's byte order. Therefore, it can not be deserialized from host machines with a different byte ordering.

Instances can be derived via Template Haskell, or written manually.

Here is an example, for deriving an instance of this type class using template Haskell:

>>> :{
data Object = Object
    { _obj1 :: [Int]
    , _obj2 :: Int
    }
:}

import Streamly.Data.MutByteArray (deriveSerialize)
$(deriveSerialize [d|instance Serialize Object|])

See deriveSerialize and deriveSerializeWith for more information on deriving using Template Haskell.

Here is an example of a manual instance.

>>> import Streamly.Data.MutByteArray (Serialize(..))

>>> :{
instance Serialize Object where
    addSizeTo acc obj = addSizeTo (addSizeTo acc (_obj1 obj)) (_obj2 obj)
    deserializeAt i arr len = do
         -- Check the array bounds before reading
        (i1, x0) <- deserializeAt i arr len
        (i2, x1) <- deserializeAt i1 arr len
        pure (i2, Object x0 x1)
    serializeAt i arr (Object x0 x1) = do
        i1 <- serializeAt i arr x0
        i2 <- serializeAt i1 arr x1
        pure i2
:}

Methods

addSizeTo :: Int -> a -> Int

addSizeTo accum value returns accum incremented by the size of the serialized representation of value in bytes. Size cannot be zero. It should be at least 1 byte.

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, a)

deserializeAt byte-offset array arrayLen deserializes a value from the given byte-offset in the array. Returns a tuple consisting of the next byte-offset and the deserialized value.

The arrayLen passed is the entire length of the input buffer. It is to be used to check if we would overflow the input buffer when deserializing.

Throws an exception if the operation would exceed the supplied arrayLen.

serializeAt :: Int -> MutByteArray -> a -> IO Int

serializeAt byte-offset array value writes the serialized representation of the value in the array at the given byte-offset. Returns the next byte-offset.

This is an unsafe operation, the programmer must ensure that the array has enough space available to serialize the value as determined by the addSizeTo operation.

Instances

Serialize Int16

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> Int16 -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Int16)

serializeAt :: Int -> MutByteArray -> Int16 -> IO Int

Serialize Int32

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> Int32 -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Int32)

serializeAt :: Int -> MutByteArray -> Int32 -> IO Int

Serialize Int64

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> Int64 -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Int64)

serializeAt :: Int -> MutByteArray -> Int64 -> IO Int

Serialize Int8

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> Int8 -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Int8)

serializeAt :: Int -> MutByteArray -> Int8 -> IO Int

Serialize Word16

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> Word16 -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Word16)

serializeAt :: Int -> MutByteArray -> Word16 -> IO Int

Serialize Word32

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> Word32 -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Word32)

serializeAt :: Int -> MutByteArray -> Word32 -> IO Int

Serialize Word64

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> Word64 -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Word64)

serializeAt :: Int -> MutByteArray -> Word64 -> IO Int

Serialize Word8

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> Word8 -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Word8)

serializeAt :: Int -> MutByteArray -> Word8 -> IO Int

Serialize Integer

Defined in Streamly.Internal.Data.MutByteArray

Methods

addSizeTo :: Int -> Integer -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Integer)

serializeAt :: Int -> MutByteArray -> Integer -> IO Int

Serialize ()

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> () -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, ())

serializeAt :: Int -> MutByteArray -> () -> IO Int

Serialize Bool

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> Bool -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Bool)

serializeAt :: Int -> MutByteArray -> Bool -> IO Int

Serialize Char

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> Char -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Char)

serializeAt :: Int -> MutByteArray -> Char -> IO Int

Serialize Double

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> Double -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Double)

serializeAt :: Int -> MutByteArray -> Double -> IO Int

Serialize Float

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> Float -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Float)

serializeAt :: Int -> MutByteArray -> Float -> IO Int

Serialize Int

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> Int -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Int)

serializeAt :: Int -> MutByteArray -> Int -> IO Int

Serialize Word

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> Word -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Word)

serializeAt :: Int -> MutByteArray -> Word -> IO Int

Serialize (FunPtr a)

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> FunPtr a -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, FunPtr a)

serializeAt :: Int -> MutByteArray -> FunPtr a -> IO Int

Serialize (Ptr a)

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> Ptr a -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Ptr a)

serializeAt :: Int -> MutByteArray -> Ptr a -> IO Int

Serialize (StablePtr a)

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> StablePtr a -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, StablePtr a)

serializeAt :: Int -> MutByteArray -> StablePtr a -> IO Int

Serialize (Array a)

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> Array a -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Array a)

serializeAt :: Int -> MutByteArray -> Array a -> IO Int

Serialize a => Serialize (Maybe a)

Defined in Streamly.Internal.Data.MutByteArray

Methods

addSizeTo :: Int -> Maybe a -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Maybe a)

serializeAt :: Int -> MutByteArray -> Maybe a -> IO Int

Serialize a => Serialize [a]

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> [a] -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, [a])

serializeAt :: Int -> MutByteArray -> [a] -> IO Int

(Serialize a, Serialize b) => Serialize (Either a b)

Defined in Streamly.Internal.Data.MutByteArray

Methods

addSizeTo :: Int -> Either a b -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Either a b)

serializeAt :: Int -> MutByteArray -> Either a b -> IO Int

Serialize (Proxy a)

Defined in Streamly.Internal.Data.MutByteArray

Methods

addSizeTo :: Int -> Proxy a -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Proxy a)

serializeAt :: Int -> MutByteArray -> Proxy a -> IO Int

(Serialize a, Serialize b) => Serialize (a, b)

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> (a, b) -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, (a, b))

serializeAt :: Int -> MutByteArray -> (a, b) -> IO Int

Serialize TH

deriveSerialize :: Q [Dec] -> Q [Dec]

Given an Serialize instance declaration splice without the methods (e.g. [d|instance Serialize a => Serialize (Maybe a)|]), generate an instance declaration including all the type class method implementations.

>>> deriveSerialize = deriveSerializeWith id

Usage:

$(deriveSerialize
      [d|instance Serialize a => Serialize (Maybe a)|])

deriveSerializeWith :: (SerializeConfig -> SerializeConfig) -> Q [Dec] -> Q [Dec]

deriveSerializeWith config-modifier instance-dec generates a template Haskell splice consisting of a declaration of a Serialize instance. instance-dec is a template Haskell declaration splice consisting of a standard Haskell instance declaration without the type class methods (e.g. [d|instance Serialize a => Serialize (Maybe a)|]).

The type class methods for the given instance are generated according to the supplied config-modifier parameter. See SerializeConfig for default configuration settings.

Usage:

$(deriveSerializeWith
      ( inlineSerializeAt (Just NoInline)
      . inlineDeserializeAt (Just NoInline)
      )
      [d|instance Serialize a => Serialize (Maybe a)|])

Config

data SerializeConfig

Configuration to control how the Serialize instance is generated. The configuration is opaque and is modified by composing config modifier functions, for example:

>>> (inlineSerializeAt (Just NoInline)) . (inlineSerializeAt (Just Inlinable))

The default configuration settings are:

• inlineAddSizeTo Nothing
• inlineSerializeAt (Just Inline)
• inlineDeserializeAt (Just Inline)

The following experimental options are also available:

• encodeConstrNames False
• encodeRecordFields False

Constructors

SerializeConfig

Fields

• cfgInlineSize :: Maybe Inline
• cfgInlineSerialize :: Maybe Inline
• cfgInlineDeserialize :: Maybe Inline
• cfgConstructorTagAsString :: Bool
• cfgRecordSyntaxWithHeader :: Bool

serializeConfig :: SerializeConfig

inlineAddSizeTo :: Maybe Inline -> SerializeConfig -> SerializeConfig

How should we inline the addSizeTo function? The default is Nothing which means left to the compiler. Forcing inline on addSizeTo function actually worsens some benchmarks and improves none.

inlineSerializeAt :: Maybe Inline -> SerializeConfig -> SerializeConfig

How should we inline the serialize function? The default 'Just Inline'. However, aggressive inlining can bloat the code and increase in compilation times when there are big functions and too many nesting levels so you can change it accordingly. A Nothing value leaves the decision to the compiler.

inlineDeserializeAt :: Maybe Inline -> SerializeConfig -> SerializeConfig

How should we inline the deserialize function? See guidelines in inlineSerializeAt.

encodeConstrNames :: Bool -> SerializeConfig -> SerializeConfig

Experimental

In sum types, use Latin-1 encoded original constructor names rather than binary values to identify constructors. This option is not applicable to product types.

This option enables the following behavior:

• Reordering: Order of the fields can be changed without affecting serialization.
• Addition: If a field is added in the new version, the old version of the data type can still be deserialized by the new version. The new value would never occur in the old one.
• Deletion: If a field is deleted in the new version, deserialization of the old version will result in an error. TBD: We can possibly designate a catch-all case to handle this scenario.

Note that if you change a type, change the semantics of a type, or delete a field and add a new field with the same name, deserialization of old data may result in silent unexpected behavior.

This option has to be the same on both encoding and decoding side.

The default is False.

encodeRecordFields :: Bool -> SerializeConfig -> SerializeConfig

Experimental

In explicit record types, use Latin-1 encoded record field names rather than binary values to identify the record fields. Note that this option is not applicable to sum types. Also, it does not work on a product type which is not a record, because there are no field names to begin with.

This option enables the following behavior:

• Reordering: Order of the fields can be changed without affecting serialization.
• Addition: If a Maybe type field is added in the new version, the old version of the data type can still be deserialized by the new version, the field value in the older version is assumed to be Nothing. If any other type of field is added, deserialization of the older version results in an error but only when that field is actually accessed in the deserialized record.
• Deletion: If a field is deleted in the new version and it is encountered in a previously serialized version then the field is discarded.

This option has to be the same on both encoding and decoding side.

There is a constant performance overhead proportional to the total length of the record field names and the number of record fields.

The default is False.

Other Utilities

data TypeOfType

Constructors

UnitType Name

TheType SimpleDataCon

MultiType [SimpleDataCon]

Instances

Eq TypeOfType

Defined in Streamly.Internal.Data.Serialize.TH.Bottom

Methods

(==) :: TypeOfType -> TypeOfType -> Bool

(/=) :: TypeOfType -> TypeOfType -> Bool

typeOfType :: Type -> [DataCon] -> TypeOfType

data SimpleDataCon

Constructors

SimpleDataCon Name [Field]

Instances

Eq SimpleDataCon

Defined in Streamly.Internal.Data.Serialize.TH.Bottom

Methods

(==) :: SimpleDataCon -> SimpleDataCon -> Bool

(/=) :: SimpleDataCon -> SimpleDataCon -> Bool

simplifyDataCon :: DataCon -> SimpleDataCon

type Field = (Maybe Name, Type)

mkFieldName :: Int -> Name

isUnitType :: [DataCon] -> Bool

isRecordSyntax :: SimpleDataCon -> Bool

c2w :: Char -> Word8

wListToString :: [Word8] -> String

xorCmp :: [Word8] -> Name -> Name -> Q Exp

serializeW8List :: Name -> Name -> [Word8] -> Q Exp

litIntegral :: Integral a => a -> Q Exp

litProxy :: Unbox a => Proxy a -> Q Exp

matchConstructor :: Name -> Int -> Q Exp -> Q Match

openConstructor :: Name -> Int -> Q Pat

makeI :: Int -> Name

makeN :: Int -> Name

makeA :: Int -> Name

int_w8 :: Int -> Word8

int_w32 :: Int -> Word32

w32_int :: Word32 -> Int

w8_int :: Word8 -> Int

_acc :: Name

_arr :: Name

_endOffset :: Name

_initialOffset :: Name

_x :: Name

_tag :: Name

_val :: Name

errorUnsupported :: String -> a

errorUnimplemented :: a

Common

mkDeserializeExprOne :: Name -> SimpleDataCon -> Q Exp

mkSerializeExprFields :: Name -> [Field] -> Q Exp

RecHeader

mkRecSerializeExpr :: Name -> SimpleDataCon -> Q Exp

mkRecDeserializeExpr :: Name -> Name -> Name -> SimpleDataCon -> Q Exp

mkRecSizeOfExpr :: SimpleDataCon -> Q Exp

conUpdateFuncDec :: Name -> [Field] -> Q [Dec]

mkDeserializeKeysDec :: Name -> Name -> SimpleDataCon -> Q [Dec]

mkRecSerializeExpr :: Name -> SimpleDataCon -> Q Exp

mkRecDeserializeExpr :: Name -> Name -> Name -> SimpleDataCon -> Q Exp

mkRecSizeOfExpr :: SimpleDataCon -> Q Exp

conUpdateFuncDec :: Name -> [Field] -> Q [Dec]

mkDeserializeKeysDec :: Name -> Name -> SimpleDataCon -> Q [Dec]

Orphan instances

Serialize Integer

Methods

addSizeTo :: Int -> Integer -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Integer)

serializeAt :: Int -> MutByteArray -> Integer -> IO Int

Serialize a => Serialize (Maybe a)

Methods

addSizeTo :: Int -> Maybe a -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Maybe a)

serializeAt :: Int -> MutByteArray -> Maybe a -> IO Int

(Serialize a, Serialize b) => Serialize (Either a b)

Methods

addSizeTo :: Int -> Either a b -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Either a b)

serializeAt :: Int -> MutByteArray -> Either a b -> IO Int

Serialize (Proxy a)

Methods

addSizeTo :: Int -> Proxy a -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Proxy a)

serializeAt :: Int -> MutByteArray -> Proxy a -> IO Int