Streamly.Data.MutArray

This module provides a mutable version of Streamly.Data.Array. The contents of a mutable array can be modified in-place. For general documentation, please refer to the original module.

Please refer to Streamly.Internal.Data.MutArray for functions that have not yet been released.

For mutable arrays that work on boxed types, not requiring the Unbox constraint, please refer to Streamly.Data.MutArray.Generic.

Setup

To execute the code examples provided in this module in ghci, please run the following commands first.

>>> :m
>>> import qualified Streamly.Data.Fold as Fold
>>> import qualified Streamly.Data.MutArray as MutArray
>>> import qualified Streamly.Data.Stream as Stream

For APIs that have not been released yet.

>>> import Streamly.Internal.Data.MutArray as MutArray

Mutable Array Type

data MutArray a

An unboxed mutable array. An array is created with a given length and capacity. Length is the number of valid elements in the array. Capacity is the maximum number of elements that the array can be expanded to without having to reallocate the memory.

The elements in the array can be mutated in-place without changing the reference (constructor). However, the length of the array cannot be mutated in-place. A new array reference is generated when the length changes. When the length is increased (upto the maximum reserved capacity of the array), the array is not reallocated and the new reference uses the same underlying memory as the old one.

Several routines in this module allow the programmer to control the capacity of the array. The programmer can control the trade-off between memory usage and performance impact due to reallocations when growing or shrinking the array.

Construction

emptyOf :: (MonadIO m, Unbox a) => Int -> m (MutArray a)

Allocates an unpinned array of zero length but growable to the specified capacity without reallocation.

pinnedEmptyOf :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)

Allocates a pinned array of zero length but growable to the specified capacity without reallocation.

fromListN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)

Create a MutArray from the first N elements of a list. The array is allocated to size N, if the list terminates before N elements then the array may hold less than N elements.

fromList :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)

Create a MutArray from a list. The list must be of finite size.

createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)

createOf n folds a maximum of n elements from the input stream to an MutArray.

>>> createOf = MutArray.createOfWith MutArray.new
>>> createOf n = Fold.take n (MutArray.unsafeCreateOf n)
>>> createOf n = MutArray.appendN n (MutArray.emptyOf n)

create :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)

Fold the whole input to a single array.

Same as createWith using an initial array size of arrayChunkBytes bytes rounded up to the element size.

Caution! Do not use this on infinite streams.

Pinning & Unpinning

pin :: MutArray a -> IO (MutArray a)

Return a copy of the array in pinned memory if unpinned, else return the original array.

unpin :: MutArray a -> IO (MutArray a)

Return a copy of the array in unpinned memory if pinned, else return the original array.

isPinned :: MutArray a -> Bool

Return True if the array is allocated in pinned memory.

Appending elements

snoc :: forall m a. (MonadIO m, Unbox a) => MutArray a -> a -> m (MutArray a)

The array is mutated to append an additional element to it. If there is no reserved space available in the array then it is reallocated to double the original size.

This is useful to reduce allocations when appending unknown number of elements.

Note that the returned array may be a mutated version of the original array.

>>> snoc = MutArray.snocWith (* 2)

Performs O(n * log n) copies to grow, but is liberal with memory allocation.

Appending streams

appendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)

Append n elements to an existing array. Any free space left in the array after appending n elements is lost.

>>> appendN n initial = Fold.take n (MutArray.unsafeAppendN n initial)

append :: forall m a. (MonadIO m, Unbox a) => m (MutArray a) -> Fold m a (MutArray a)

append action mutates the array generated by action to append the input stream. If there is no reserved space available in the array it is reallocated to double the size.

Note that the returned array may be a mutated version of original array.

>>> append = MutArray.appendWith (* 2)

Inplace mutation

putIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m ()

O(1) Write the given element at the given index in the array. Performs in-place mutation of the array.

>>> putIndex ix arr val = MutArray.modifyIndex ix arr (const (val, ()))
>>> f = MutArray.putIndices
>>> putIndex ix arr val = Stream.fold (f arr) (Stream.fromPure (ix, val))

putIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m ()

Write the given element to the given index of the array. Does not check if the index is out of bounds of the array.

Pre-release

modifyIndex :: forall m a b. (MonadIO m, Unbox a) => Int -> MutArray a -> (a -> (a, b)) -> m b

Modify a given index of an array using a modifier function.

Pre-release

modifyIndexUnsafe :: forall m a b. (MonadIO m, Unbox a) => Int -> MutArray a -> (a -> (a, b)) -> m b

Modify a given index of an array using a modifier function.

Unsafe because it does not check the bounds of the array.

Pre-release

modify :: forall m a. (MonadIO m, Unbox a) => MutArray a -> (a -> a) -> m ()

Modify each element of an array using the supplied modifier function.

This is an in-place equivalent of an immutable map operation.

Pre-release

Random access

getIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (Maybe a)

O(1) Lookup the element at the given index. Index starts from 0.

getIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a

Return the element at the specified index without checking the bounds.

Unsafe because it does not check the bounds of the array.

Conversion

toList :: forall m a. (MonadIO m, Unbox a) => MutArray a -> m [a]

Convert a MutArray into a list.

Streams

read :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Stream m a

Convert a MutArray into a stream.

>>> read = Stream.unfold MutArray.reader

readRev :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Stream m a

Convert a MutArray into a stream in reverse order.

>>> readRev = Stream.unfold MutArray.readerRev

Unfolds

reader :: forall m a. (MonadIO m, Unbox a) => Unfold m (MutArray a) a

Unfold an array into a stream.

readerRev :: forall m a. (MonadIO m, Unbox a) => Unfold m (MutArray a) a

Unfold an array into a stream in reverse order.

Casting

cast :: forall a b. Unbox b => MutArray a -> Maybe (MutArray b)

Cast an array having elements of type a into an array having elements of type b. The length of the array should be a multiple of the size of the target element otherwise Nothing is returned.

asBytes :: MutArray a -> MutArray Word8

Cast an MutArray a into an MutArray Word8.

Size

length :: forall a. Unbox a => MutArray a -> Int

O(1) Get the length of the array i.e. the number of elements in the array.

Note that byteLength is less expensive than this operation, as length involves a costly division operation.

Re-exports

class Unbox a where

The Unbox type class provides operations for serialization (unboxing) and deserialization (boxing) of fixed-length, non-recursive Haskell data types to and from their byte stream representation.

Unbox uses fixed size encoding, therefore, size is independent of the value, it must be determined solely by the type. This restriction makes types with Unbox instances suitable for storing in arrays. Note that sum types may have multiple constructors of different sizes, the size of a sum type is computed as the maximum required by any constructor.

The peekAt operation reads as many bytes from the mutable byte array as the size of the data type and builds a Haskell data type from these bytes. pokeAt operation converts a Haskell data type to its binary representation which consists of size bytes and then stores these bytes into the mutable byte array. These operations do not check the bounds of the array, the user of the type class is expected to check the bounds before peeking or poking.

IMPORTANT: The serialized data's byte ordering remains the same as the host machine's byte order. Therefore, it can not be deserialized from host machines with a different byte ordering.

Instances can be derived via Generics, Template Haskell, or written manually. Note that the data type must be non-recursive. WARNING! Generic and Template Haskell deriving, both hang for recursive data types. Deriving via Generics is more convenient but Template Haskell should be preferred over Generics for the following reasons:

1. Instances derived via Template Haskell provide better and more reliable performance.
2. Generic deriving allows only 256 fields or constructor tags whereas template Haskell has no limit.

Here is an example, for deriving an instance of this type class using generics:

>>> import GHC.Generics (Generic)
>>> :{
data Object = Object
    { _int0 :: Int
    , _int1 :: Int
    } deriving Generic
:}

>>> import Streamly.Data.MutByteArray (Unbox(..))
>>> instance Unbox Object

To derive the instance via Template Haskell:

import Streamly.Data.MutByteArray (deriveUnbox)
$(deriveUnbox [d|instance Unbox Object|])

See deriveUnbox for more information on deriving using Template Haskell.

If you want to write the instance manually:

>>> :{
instance Unbox Object where
    sizeOf _ = 16
    peekAt i arr = do
       -- Check the array bounds
        x0 <- peekAt i arr
        x1 <- peekAt (i + 8) arr
        return $ Object x0 x1
    pokeAt i arr (Object x0 x1) = do
       -- Check the array bounds
        pokeAt i arr x0
        pokeAt (i + 8) arr x1
:}

Minimal complete definition

Nothing

Methods

sizeOf :: Proxy a -> Int

Get the size. Size cannot be zero, should be at least 1 byte.

default sizeOf :: SizeOfRep (Rep a) => Proxy a -> Int

peekAt :: Int -> MutByteArray -> IO a

peekAt byte-offset array reads an element of type a from the the given the byte offset in the array.

IMPORTANT: The implementation of this interface may not check the bounds of the array, the caller must not assume that.

default peekAt :: (Generic a, PeekRep (Rep a)) => Int -> MutByteArray -> IO a

peekByteIndex :: Int -> MutByteArray -> IO a

Deprecated: Use peekAt.

pokeAt :: Int -> MutByteArray -> a -> IO ()

pokeAt byte-offset array writes an element of type a to the the given the byte offset in the array.

IMPORTANT: The implementation of this interface may not check the bounds of the array, the caller must not assume that.

default pokeAt :: (Generic a, PokeRep (Rep a)) => Int -> MutByteArray -> a -> IO ()

pokeByteIndex :: Int -> MutByteArray -> a -> IO ()

Deprecated: Use pokeAt.

Instances

Unbox IntPtr

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy IntPtr -> Int

peekAt :: Int -> MutByteArray -> IO IntPtr

peekByteIndex :: Int -> MutByteArray -> IO IntPtr

pokeAt :: Int -> MutByteArray -> IntPtr -> IO ()

pokeByteIndex :: Int -> MutByteArray -> IntPtr -> IO ()

Unbox WordPtr

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy WordPtr -> Int

peekAt :: Int -> MutByteArray -> IO WordPtr

peekByteIndex :: Int -> MutByteArray -> IO WordPtr

pokeAt :: Int -> MutByteArray -> WordPtr -> IO ()

pokeByteIndex :: Int -> MutByteArray -> WordPtr -> IO ()

Unbox Fingerprint

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Fingerprint -> Int

peekAt :: Int -> MutByteArray -> IO Fingerprint

peekByteIndex :: Int -> MutByteArray -> IO Fingerprint

pokeAt :: Int -> MutByteArray -> Fingerprint -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Fingerprint -> IO ()

Unbox Int16

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Int16 -> Int

peekAt :: Int -> MutByteArray -> IO Int16

peekByteIndex :: Int -> MutByteArray -> IO Int16

pokeAt :: Int -> MutByteArray -> Int16 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Int16 -> IO ()

Unbox Int32

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Int32 -> Int

peekAt :: Int -> MutByteArray -> IO Int32

peekByteIndex :: Int -> MutByteArray -> IO Int32

pokeAt :: Int -> MutByteArray -> Int32 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Int32 -> IO ()

Unbox Int64

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Int64 -> Int

peekAt :: Int -> MutByteArray -> IO Int64

peekByteIndex :: Int -> MutByteArray -> IO Int64

pokeAt :: Int -> MutByteArray -> Int64 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Int64 -> IO ()

Unbox Int8

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Int8 -> Int

peekAt :: Int -> MutByteArray -> IO Int8

peekByteIndex :: Int -> MutByteArray -> IO Int8

pokeAt :: Int -> MutByteArray -> Int8 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Int8 -> IO ()

Unbox IoSubSystem

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy IoSubSystem -> Int

peekAt :: Int -> MutByteArray -> IO IoSubSystem

peekByteIndex :: Int -> MutByteArray -> IO IoSubSystem

pokeAt :: Int -> MutByteArray -> IoSubSystem -> IO ()

pokeByteIndex :: Int -> MutByteArray -> IoSubSystem -> IO ()

Unbox Word16

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Word16 -> Int

peekAt :: Int -> MutByteArray -> IO Word16

peekByteIndex :: Int -> MutByteArray -> IO Word16

pokeAt :: Int -> MutByteArray -> Word16 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Word16 -> IO ()

Unbox Word32

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Word32 -> Int

peekAt :: Int -> MutByteArray -> IO Word32

peekByteIndex :: Int -> MutByteArray -> IO Word32

pokeAt :: Int -> MutByteArray -> Word32 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Word32 -> IO ()

Unbox Word64

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Word64 -> Int

peekAt :: Int -> MutByteArray -> IO Word64

peekByteIndex :: Int -> MutByteArray -> IO Word64

pokeAt :: Int -> MutByteArray -> Word64 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Word64 -> IO ()

Unbox Word8

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Word8 -> Int

peekAt :: Int -> MutByteArray -> IO Word8

peekByteIndex :: Int -> MutByteArray -> IO Word8

pokeAt :: Int -> MutByteArray -> Word8 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Word8 -> IO ()

Unbox MicroSecond64

Defined in Streamly.Internal.Data.Time.Units

Methods

sizeOf :: Proxy MicroSecond64 -> Int

peekAt :: Int -> MutByteArray -> IO MicroSecond64

peekByteIndex :: Int -> MutByteArray -> IO MicroSecond64

pokeAt :: Int -> MutByteArray -> MicroSecond64 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> MicroSecond64 -> IO ()

Unbox MilliSecond64

Defined in Streamly.Internal.Data.Time.Units

Methods

sizeOf :: Proxy MilliSecond64 -> Int

peekAt :: Int -> MutByteArray -> IO MilliSecond64

peekByteIndex :: Int -> MutByteArray -> IO MilliSecond64

pokeAt :: Int -> MutByteArray -> MilliSecond64 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> MilliSecond64 -> IO ()

Unbox NanoSecond64

Defined in Streamly.Internal.Data.Time.Units

Methods

sizeOf :: Proxy NanoSecond64 -> Int

peekAt :: Int -> MutByteArray -> IO NanoSecond64

peekByteIndex :: Int -> MutByteArray -> IO NanoSecond64

pokeAt :: Int -> MutByteArray -> NanoSecond64 -> IO ()

pokeByteIndex :: Int -> MutByteArray -> NanoSecond64 -> IO ()

Unbox ()

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy () -> Int

peekAt :: Int -> MutByteArray -> IO ()

peekByteIndex :: Int -> MutByteArray -> IO ()

pokeAt :: Int -> MutByteArray -> () -> IO ()

pokeByteIndex :: Int -> MutByteArray -> () -> IO ()

Unbox Bool

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Bool -> Int

peekAt :: Int -> MutByteArray -> IO Bool

peekByteIndex :: Int -> MutByteArray -> IO Bool

pokeAt :: Int -> MutByteArray -> Bool -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Bool -> IO ()

Unbox Char

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Char -> Int

peekAt :: Int -> MutByteArray -> IO Char

peekByteIndex :: Int -> MutByteArray -> IO Char

pokeAt :: Int -> MutByteArray -> Char -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Char -> IO ()

Unbox Double

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Double -> Int

peekAt :: Int -> MutByteArray -> IO Double

peekByteIndex :: Int -> MutByteArray -> IO Double

pokeAt :: Int -> MutByteArray -> Double -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Double -> IO ()

Unbox Float

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Float -> Int

peekAt :: Int -> MutByteArray -> IO Float

peekByteIndex :: Int -> MutByteArray -> IO Float

pokeAt :: Int -> MutByteArray -> Float -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Float -> IO ()

Unbox Int

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Int -> Int

peekAt :: Int -> MutByteArray -> IO Int

peekByteIndex :: Int -> MutByteArray -> IO Int

pokeAt :: Int -> MutByteArray -> Int -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Int -> IO ()

Unbox Word

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy Word -> Int

peekAt :: Int -> MutByteArray -> IO Word

peekByteIndex :: Int -> MutByteArray -> IO Word

pokeAt :: Int -> MutByteArray -> Word -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Word -> IO ()

Unbox a => Unbox (Complex a)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy (Complex a) -> Int

peekAt :: Int -> MutByteArray -> IO (Complex a)

peekByteIndex :: Int -> MutByteArray -> IO (Complex a)

pokeAt :: Int -> MutByteArray -> Complex a -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Complex a -> IO ()

Unbox a => Unbox (Identity a)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy (Identity a) -> Int

peekAt :: Int -> MutByteArray -> IO (Identity a)

peekByteIndex :: Int -> MutByteArray -> IO (Identity a)

pokeAt :: Int -> MutByteArray -> Identity a -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Identity a -> IO ()

Unbox a => Unbox (Down a)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy (Down a) -> Int

peekAt :: Int -> MutByteArray -> IO (Down a)

peekByteIndex :: Int -> MutByteArray -> IO (Down a)

pokeAt :: Int -> MutByteArray -> Down a -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Down a -> IO ()

Unbox (FunPtr a)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy (FunPtr a) -> Int

peekAt :: Int -> MutByteArray -> IO (FunPtr a)

peekByteIndex :: Int -> MutByteArray -> IO (FunPtr a)

pokeAt :: Int -> MutByteArray -> FunPtr a -> IO ()

pokeByteIndex :: Int -> MutByteArray -> FunPtr a -> IO ()

Unbox (Ptr a)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy (Ptr a) -> Int

peekAt :: Int -> MutByteArray -> IO (Ptr a)

peekByteIndex :: Int -> MutByteArray -> IO (Ptr a)

pokeAt :: Int -> MutByteArray -> Ptr a -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Ptr a -> IO ()

Unbox a => Unbox (Ratio a)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy (Ratio a) -> Int

peekAt :: Int -> MutByteArray -> IO (Ratio a)

peekByteIndex :: Int -> MutByteArray -> IO (Ratio a)

pokeAt :: Int -> MutByteArray -> Ratio a -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Ratio a -> IO ()

Unbox (StablePtr a)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy (StablePtr a) -> Int

peekAt :: Int -> MutByteArray -> IO (StablePtr a)

peekByteIndex :: Int -> MutByteArray -> IO (StablePtr a)

pokeAt :: Int -> MutByteArray -> StablePtr a -> IO ()

pokeByteIndex :: Int -> MutByteArray -> StablePtr a -> IO ()

Unbox a => Unbox (Const a b)

Defined in Streamly.Internal.Data.Unbox

Methods

sizeOf :: Proxy (Const a b) -> Int

peekAt :: Int -> MutByteArray -> IO (Const a b)

peekByteIndex :: Int -> MutByteArray -> IO (Const a b)

pokeAt :: Int -> MutByteArray -> Const a b -> IO ()

pokeByteIndex :: Int -> MutByteArray -> Const a b -> IO ()

Deprecated

newPinned :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)

Deprecated: Please use pinnedNew instead.

new :: (MonadIO m, Unbox a) => Int -> m (MutArray a)

pinnedNew :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)

writeN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)

write :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)

writeAppendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)

writeAppend :: forall m a. (MonadIO m, Unbox a) => m (MutArray a) -> Fold m a (MutArray a)