streaming-0.1.4.0: an elementary streaming prelude and general stream type.

Safe HaskellNone
LanguageHaskell2010

Streaming

Contents

Synopsis

An iterable streaming monad transformer

The Stream data type can be used to represent any effectful succession of steps arising in some monad. The form of the steps is specified by the first ("functor") parameter in Stream f m r. The monad of the underlying effects is expressed by the second parameter.

This module exports combinators that pertain to that general case. Some of these are quite abstract and pervade any use of the library, e.g.

  maps ::    (forall x . f x -> g x)     -> Stream f m r -> Stream g m r  
  mapped ::  (forall x . f x -> m (g x)) -> Stream f m r -> Stream g m r  
  hoist ::   (forall x . m x -> n x)     -> Stream f m r -> Stream f n r -- from the MFunctor instance
  concats :: Stream (Stream f m) m r -> Stream f m r          

(assuming here and thoughout that m or n satisfies a Monad constraint, and f or g a Functor constraint.)

Others are surprisingly determinate in content:

  chunksOf ::     Int -> Stream f m r -> Stream (Stream f m) m r
  splitsAt ::     Int -> Stream f m r -> Stream f m (Stream f m r)
  zipsWith ::     (forall x y. f x -> g y -> h (x, y)) -> Stream f m r -> Stream g m r -> Stream h m r
  intercalates :: Stream f m () -> Stream (Stream f m) m r -> Stream f m r
  groups:         Stream (Sum f g) m r -> Stream (Sum (Stream f m) (Stream g m)) m r

One way to see that any streaming library needs some such general type is that it is required to represent the segmentation of a stream, and to express the equivalents of Prelude/Data.List combinators that involve 'lists of lists' and the like. See for example this post on the correct expression of a streaming 'lines' function.

The module Streaming.Prelude exports combinators relating to

Stream (Of a) m r

where Of a r = !a :> r is a left-strict pair.

This expresses the concept of a Producer or Source or Generator and easily inter-operates with types with such names in e.g. conduit, iostreams and pipes.

data Stream f m r Source

Instances

(MonadBase b m, Functor f) => MonadBase b (Stream f m) Source 
Functor f => MFunctor (Stream f) Source 
Functor f => MMonad (Stream f) Source 
Functor f => MonadTrans (Stream f) Source 
(Functor f, Monad m) => Monad (Stream f m) Source 
(Functor f, Monad m) => Functor (Stream f m) Source 
(Functor f, Monad m) => Applicative (Stream f m) Source 
(MonadThrow m, Functor f) => MonadThrow (Stream f m) Source 
(MonadCatch m, Functor f) => MonadCatch (Stream f m) Source 
(MonadIO m, Functor f) => MonadIO (Stream f m) Source 
(MonadResource m, Functor f) => MonadResource (Stream f m) Source 
(Eq r, Eq (m (Stream f m r)), Eq (f (Stream f m r))) => Eq (Stream f m r) Source 
(Typeable (* -> *) f, Typeable (* -> *) m, Data r, Data (m (Stream f m r)), Data (f (Stream f m r))) => Data (Stream f m r) Source 
(Show r, Show (m (Stream f m r)), Show (f (Stream f m r))) => Show (Stream f m r) Source 

Constructing a Stream on a given functor

unfold :: (Monad m, Functor f) => (s -> m (Either r (f s))) -> s -> Stream f m r Source

Build a Stream by unfolding steps starting from a seed. See also the specialized unfoldr in the prelude.

unfold inspect = id -- modulo the quotient we work with
unfold Pipes.next :: Monad m => Producer a m r -> Stream ((,) a) m r
unfold (curry (:>) . Pipes.next) :: Monad m => Producer a m r -> Stream (Of a) m r

yields :: (Monad m, Functor f) => f r -> Stream f m r Source

Lift for items in the base functor. Makes a singleton or one-layer succession. It is named by similarity to lift:

lift :: (Monad m, Functor f)     => m r -> Stream f m r
yields ::  (Monad m, Functor f) => f r -> Stream f m r

replicates :: (Monad m, Functor f) => Int -> f () -> Stream f m () Source

Repeat a functorial layer, command or instruct several times.

repeats :: (Monad m, Functor f) => f () -> Stream f m r Source

Repeat a functorial layer, command or instruction forever.

repeatsM :: (Monad m, Functor f) => m (f ()) -> Stream f m r Source

effect :: (Monad m, Functor f) => m (Stream f m r) -> Stream f m r Source

wrap :: (Monad m, Functor f) => f (Stream f m r) -> Stream f m r Source

streamBuild :: (forall b. (f b -> b) -> (m b -> b) -> (r -> b) -> b) -> Stream f m r Source

Reflect a church-encoded stream; cp. GHC.Exts.build

destroy a b c (streamBuild psi)  = 

Transforming streams

decompose :: (Monad m, Functor f) => Stream (Compose m f) m r -> Stream f m r Source

Resort a succession of layers of the form m (f x). Though mapsM is best understood as:

mapsM phi = decompose . maps (Compose . phi)

we could as well define decompose by mapsM:

decompose = mapsM getCompose

maps :: (Monad m, Functor f) => (forall x. f x -> g x) -> Stream f m r -> Stream g m r Source

Map layers of one functor to another with a transformation. Compare hoist, which has a similar effect on the monadic parameter.

maps id = id
maps f . maps g = maps (f . g)

mapsM :: (Monad m, Functor f) => (forall x. f x -> m (g x)) -> Stream f m r -> Stream g m r Source

Map layers of one functor to another with a transformation involving the base monad maps is more fundamental than mapsM, which is best understood as a convenience for effecting this frequent composition:

mapsM phi = decompose . maps (Compose . phi)

The streaming prelude exports the same function under the better name mapped, which overlaps with the lens libraries.

mapped :: (Monad m, Functor f) => (forall x. f x -> m (g x)) -> Stream f m r -> Stream g m r Source

Map layers of one functor to another with a transformation involving the base monad maps is more fundamental than mapped, which is best understood as a convenience for effecting this frequent composition:

mapped = mapsM 
mapsM phi = decompose . maps (Compose . phi)  

mapped obeys these rules:

mapped return       = id
mapped f . mapped g = mapped (f <=< g)
map f . mapped g    = mapped (liftM f . g)
mapped f . map g    = mapped (f . g)

distribute :: (Monad m, Functor f, MonadTrans t, MFunctor t, Monad (t (Stream f m))) => Stream f (t m) r -> t (Stream f m) r Source

Make it possible to 'run' the underlying transformed monad.

groups :: (Monad m, Functor f, Functor g) => Stream (Sum f g) m r -> Stream (Sum (Stream f m) (Stream g m)) m r Source

Group layers in an alternating stream into adjoining sub-streams of one type or another.

Inspecting a stream

inspect :: (Functor f, Monad m) => Stream f m r -> m (Either r (f (Stream f m r))) Source

Inspect the first stage of a freely layered sequence. Compare Pipes.next and the replica Streaming.Prelude.next. This is the uncons for the general unfold.

unfold inspect = id
Streaming.Prelude.unfoldr StreamingPrelude.next = id

Zipping and unzipping streams

zipsWith :: (Monad m, Functor h) => (forall x y. f x -> g y -> h (x, y)) -> Stream f m r -> Stream g m r -> Stream h m r Source

zips :: (Monad m, Functor f, Functor g) => Stream f m r -> Stream g m r -> Stream (Compose f g) m r Source

unzips :: (Monad m, Functor f, Functor g) => Stream (Compose f g) m r -> Stream f (Stream g m) r Source

interleaves :: (Monad m, Applicative h) => Stream h m r -> Stream h m r -> Stream h m r Source

Interleave functor layers, with the effects of the first preceding the effects of the second.

interleaves = zipsWith (liftA2 (,))
>>> let paste = \a b -> interleaves (Q.lines a) (maps (Q.cons' '\t') (Q.lines b))
>>> Q.stdout $ Q.unlines $ paste "hello\nworld\n" "goodbye\nworld\n"
hello	goodbye
world	world

separate :: (Monad m, Functor f, Functor g) => Stream (Sum f g) m r -> Stream f (Stream g m) r Source

Given a stream on a sum of functors, make it a stream on the left functor, with the streaming on the other functor as the governing monad. This is useful for acting on one or the other functor with a fold.

>>> let odd_even = S.maps (S.distinguish even) $ S.each [1..10::Int]
>>> :t separate odd_even
separate odd_even
  :: Monad m => Stream (Of Int) (Stream (Of Int) m) ()

Now, for example, it is convenient to fold on the left and right values separately:

>>> toList $ toList $ separate odd_even
[2,4,6,8,10] :> ([1,3,5,7,9] :> ())

We can achieve the above effect more simply in the case of Stream (Of a) m r by using duplicate

>>> S.toList . S.filter even $ S.toList . S.filter odd $ S.duplicate $ each [1..10::Int]
[2,4,6,8,10] :> ([1,3,5,7,9] :> ())

But separate and unseparate are functor-general.

unseparate :: (Monad m, Functor f, Functor g) => Stream f (Stream g m) r -> Stream (Sum f g) m r Source

Eliminating a Stream

iterTM :: (Functor f, Monad m, MonadTrans t, Monad (t m)) => (f (t m a) -> t m a) -> Stream f m a -> t m a Source

Specialized fold following the usage of Control.Monad.Trans.Free

iterTM alg = streamFold return (join . lift)

iterT :: (Functor f, Monad m) => (f (m a) -> m a) -> Stream f m a -> m a Source

Specialized fold following the usage of Control.Monad.Trans.Free

iterT alg = streamFold return join alg 

destroy :: (Functor f, Monad m) => Stream f m r -> (f b -> b) -> (m b -> b) -> (r -> b) -> b Source

Map a stream directly to its church encoding; compare Data.List.foldr

streamFold :: (Functor f, Monad m) => (r -> b) -> (m b -> b) -> (f b -> b) -> Stream f m r -> b Source

streamFold reorders the arguments of destroy to be more akin to foldr It is more convenient to query in ghci to figure out what kind of 'algebra' you need to write.

>>> :t streamFold return join
(Monad m, Functor f) => 
     (f (m a) -> m a) -> Stream f m a -> m a        -- iterT
>>> :t streamFold return (join . lift)
(Monad m, Monad (t m), Functor f, MonadTrans t) =>
     (f (t m a) -> t m a) -> Stream f m a -> t m a  -- iterTM
>>> :t streamFold return effect
(Monad m, Functor f, Functor g) =>
     (f (Stream g m r) -> Stream g m r) -> Stream f m r -> Stream g m r
>>> :t \f -> streamFold return effect (wrap . f)
(Monad m, Functor f, Functor g) =>
     (f (Stream g m a) -> g (Stream g m a))
     -> Stream f m a -> Stream g m a                 -- maps
>>> :t \f -> streamFold return effect (effect . liftM wrap . f)
(Monad m, Functor f, Functor g) =>
     (f (Stream g m a) -> m (g (Stream g m a)))
     -> Stream f m a -> Stream g m a                 -- mapped

So for example, when we realize that

>>> :t streamFold return Q.mwrap
(Monad m, Functor f) =>
   (f (Q.ByteString m a) -> Q.ByteString m a)
   -> Stream f m a -> Q.ByteString m a

it is easy to see how to write fromChunks:

>>> streamFold return Q.mwrap (\(a:>b) -> Q.chunk a >>  b)
Monad m => Stream (Of B.ByteString) m a -> Q.ByteString m a -- fromChunks

mapsM_ :: (Functor f, Monad m) => (forall x. f x -> m x) -> Stream f m r -> m r Source

Map each layer to an effect, and run them all.

run :: Monad m => Stream m m r -> m r Source

Run the effects in a stream that merely layers effects.

Splitting and joining Streams

splitsAt :: (Monad m, Functor f) => Int -> Stream f m r -> Stream f m (Stream f m r) Source

Split a succession of layers after some number, returning a streaming or effectful pair.

>>> rest <- S.print $ S.splitAt 1 $ each [1..3]
1
>>> S.print rest
2
3
splitAt 0 = return
splitAt n >=> splitAt m = splitAt (m+n)

Thus, e.g.

>>> rest <- S.print $ splitsAt 2 >=> splitsAt 2 $ each [1..5]
1
2
3
4
>>> S.print rest
5

takes :: (Monad m, Functor f) => Int -> Stream f m r -> Stream f m () Source

chunksOf :: (Monad m, Functor f) => Int -> Stream f m r -> Stream (Stream f m) m r Source

Break a stream into substreams each with n functorial layers.

>>> S.print $ mapped S.sum $ chunksOf 2 $ each [1,1,1,1,1]
2
2
1

concats :: (Monad m, Functor f) => Stream (Stream f m) m r -> Stream f m r Source

Dissolves the segmentation into layers of Stream f m layers.

intercalates :: (Monad m, Monad (t m), MonadTrans t) => t m x -> Stream (t m) m r -> t m r Source

Interpolate a layer at each segment. This specializes to e.g.

intercalates :: (Monad m, Functor f) => Stream f m () -> Stream (Stream f m) m r -> Stream f m r

Base functor for streams of individual items

data Of a b Source

A left-strict pair; the base functor for streams of individual elements.

Constructors

!a :> b infixr 5 

Instances

Monoid a => Monad (Of a) Source 
Functor (Of a) Source 
Monoid a => Applicative (Of a) Source 
Foldable (Of a) Source 
Traversable (Of a) Source 
Eq a => Eq1 (Of a) Source 
Ord a => Ord1 (Of a) Source 
Read a => Read1 (Of a) Source 
Show a => Show1 (Of a) Source 
(Eq a, Eq b) => Eq (Of a b) Source 
(Data a, Data b) => Data (Of a b) Source 
(Ord a, Ord b) => Ord (Of a b) Source 
(Read a, Read b) => Read (Of a b) Source 
(Show a, Show b) => Show (Of a b) Source 
(Monoid a, Monoid b) => Monoid (Of a b) Source 

lazily :: Of a b -> (a, b) Source

Note that lazily, strictly, fst', and mapOf are all so-called natural transformations on the primitive Of a functor If we write

 type f ~~> g = forall x . f x -> g x

then we can restate some types as follows:

 mapOf            :: (a -> b) -> Of a ~~> Of b   -- bifunctor lmap
 lazily           ::             Of a ~~> (,) a
 Identity . fst'  ::             Of a ~~> Identity a

Manipulation of a Stream f m r by mapping often turns on recognizing natural transformations of f, thus maps is far more general the the map of the present module, which can be defined thus:

 S.map :: (a -> b) -> Stream (Of a) m r -> Stream (Of b) m r
 S.map f = maps (mapOf f)

This rests on recognizing that mapOf is a natural transformation; note though that it results in such a transformation as well:

 S.map :: (a -> b) -> Stream (Of a) m ~> Stream (Of b) m   

strictly :: (a, b) -> Of a b Source

ResourceT help

bracketStream :: (Functor f, MonadResource m) => IO a -> (a -> IO ()) -> (a -> Stream f m b) -> Stream f m b Source

re-exports

class MFunctor t where

A functor in the category of monads, using hoist as the analog of fmap:

hoist (f . g) = hoist f . hoist g

hoist id = id

Methods

hoist :: Monad m => (forall a. m a -> n a) -> t m b -> t n b

Lift a monad morphism from m to n into a monad morphism from (t m) to (t n)

class (MFunctor t, MonadTrans t) => MMonad t where

A monad in the category of monads, using lift from MonadTrans as the analog of return and embed as the analog of (=<<):

embed lift = id

embed f (lift m) = f m

embed g (embed f t) = embed (\m -> embed g (f m)) t

Methods

embed :: Monad n => (forall a. m a -> t n a) -> t m b -> t n b

Embed a newly created MMonad layer within an existing layer

embed is analogous to (=<<)

class MonadTrans t where

The class of monad transformers. Instances should satisfy the following laws, which state that lift is a monad transformation:

Methods

lift :: Monad m => m a -> t m a

Lift a computation from the argument monad to the constructed monad.

class Monad m => MonadIO m where

Monads in which IO computations may be embedded. Any monad built by applying a sequence of monad transformers to the IO monad will be an instance of this class.

Instances should satisfy the following laws, which state that liftIO is a transformer of monads:

Methods

liftIO :: IO a -> m a

Lift a computation from the IO monad.

Instances

MonadIO IO 
MonadIO m => MonadIO (ListT m) 
MonadIO m => MonadIO (ResourceT m) 
MonadIO m => MonadIO (MaybeT m) 
MonadIO m => MonadIO (IdentityT m) 
MonadIO m => MonadIO (ContT r m) 
MonadIO m => MonadIO (ReaderT r m) 
MonadIO m => MonadIO (StateT s m) 
MonadIO m => MonadIO (StateT s m) 
MonadIO m => MonadIO (ExceptT e m) 
(Error e, MonadIO m) => MonadIO (ErrorT e m) 
(Monoid w, MonadIO m) => MonadIO (WriterT w m) 
(Monoid w, MonadIO m) => MonadIO (WriterT w m) 
(MonadIO m, Functor f) => MonadIO (Stream f m) 
(Monoid w, MonadIO m) => MonadIO (RWST r w s m) 
(Monoid w, MonadIO m) => MonadIO (RWST r w s m) 

newtype Compose f g a :: (* -> *) -> (* -> *) -> * -> * infixr 9

Right-to-left composition of functors. The composition of applicative functors is always applicative, but the composition of monads is not always a monad.

Constructors

Compose infixr 9 

Fields

getCompose :: f (g a)
 

Instances

Functor f => MFunctor (Compose f) 
(Functor f, Functor g) => Functor (Compose f g) 
(Applicative f, Applicative g) => Applicative (Compose f g) 
(Foldable f, Foldable g) => Foldable (Compose f g) 
(Traversable f, Traversable g) => Traversable (Compose f g) 
(Alternative f, Applicative g) => Alternative (Compose f g) 
(Functor f, Eq1 f, Eq1 g) => Eq1 (Compose f g) 
(Functor f, Ord1 f, Ord1 g) => Ord1 (Compose f g) 
(Functor f, Read1 f, Read1 g) => Read1 (Compose f g) 
(Functor f, Show1 f, Show1 g) => Show1 (Compose f g) 
(Functor f, Eq1 f, Eq1 g, Eq a) => Eq (Compose f g a) 
(Functor f, Ord1 f, Ord1 g, Ord a) => Ord (Compose f g a) 
(Functor f, Read1 f, Read1 g, Read a) => Read (Compose f g a) 
(Functor f, Show1 f, Show1 g, Show a) => Show (Compose f g a) 

class Monad m => MonadThrow m where

A class for monads in which exceptions may be thrown.

Instances should obey the following law:

throwM e >> x = throwM e

In other words, throwing an exception short-circuits the rest of the monadic computation.

Methods

throwM :: Exception e => e -> m a

Throw an exception. Note that this throws when this action is run in the monad m, not when it is applied. It is a generalization of Control.Exception's throwIO.

Should satisfy the law:

throwM e >> f = throwM e

Instances

MonadThrow [] 
MonadThrow IO 
MonadThrow STM 
MonadThrow Maybe 
(~) * e SomeException => MonadThrow (Either e) 
MonadThrow m => MonadThrow (ListT m) 
MonadThrow m => MonadThrow (ResourceT m) 
MonadThrow m => MonadThrow (MaybeT m)

Throws exceptions into the base monad.

MonadThrow m => MonadThrow (IdentityT m) 
MonadThrow m => MonadThrow (ContT r m) 
MonadThrow m => MonadThrow (ReaderT r m) 
MonadThrow m => MonadThrow (StateT s m) 
MonadThrow m => MonadThrow (StateT s m) 
MonadThrow m => MonadThrow (ExceptT e m)

Throws exceptions into the base monad.

(Error e, MonadThrow m) => MonadThrow (ErrorT e m)

Throws exceptions into the base monad.

(MonadThrow m, Monoid w) => MonadThrow (WriterT w m) 
(MonadThrow m, Monoid w) => MonadThrow (WriterT w m) 
(MonadThrow m, Functor f) => MonadThrow (Stream f m) 
(MonadThrow m, Monoid w) => MonadThrow (RWST r w s m) 
(MonadThrow m, Monoid w) => MonadThrow (RWST r w s m) 

class (MonadThrow m, MonadIO m, Applicative m, MonadBase IO m) => MonadResource m where

A Monad which allows for safe resource allocation. In theory, any monad transformer stack included a ResourceT can be an instance of MonadResource.

Note: runResourceT has a requirement for a MonadBaseControl IO m monad, which allows control operations to be lifted. A MonadResource does not have this requirement. This means that transformers such as ContT can be an instance of MonadResource. However, the ContT wrapper will need to be unwrapped before calling runResourceT.

Since 0.3.0

Methods

liftResourceT :: ResourceT IO a -> m a

Lift a ResourceT IO action into the current Monad.

Since 0.4.0

class (Applicative b, Applicative m, Monad b, Monad m) => MonadBase b m | m -> b where

Methods

liftBase :: b α -> m α

Lift a computation from the base monad

Instances

MonadBase [] [] 
MonadBase IO IO 
MonadBase Identity Identity 
MonadBase STM STM 
MonadBase Maybe Maybe 
MonadBase b m => MonadBase b (ResourceT m) 
MonadBase b m => MonadBase b (MaybeT m) 
MonadBase b m => MonadBase b (ListT m) 
MonadBase b m => MonadBase b (IdentityT m) 
(Monoid w, MonadBase b m) => MonadBase b (WriterT w m) 
(Monoid w, MonadBase b m) => MonadBase b (WriterT w m) 
MonadBase b m => MonadBase b (StateT s m) 
MonadBase b m => MonadBase b (StateT s m) 
MonadBase b m => MonadBase b (ReaderT r m) 
MonadBase b m => MonadBase b (ExceptT e m) 
(Error e, MonadBase b m) => MonadBase b (ErrorT e m) 
MonadBase b m => MonadBase b (ContT r m) 
(MonadBase b m, Functor f) => MonadBase b (Stream f m) 
(Monoid w, MonadBase b m) => MonadBase b (RWST r w s m) 
(Monoid w, MonadBase b m) => MonadBase b (RWST r w s m) 
MonadBase ((->) r) ((->) r) 
MonadBase (Either e) (Either e) 
MonadBase (ST s) (ST s) 
MonadBase (ST s) (ST s) 

data ResourceT m a :: (* -> *) -> * -> *

The Resource transformer. This transformer keeps track of all registered actions, and calls them upon exit (via runResourceT). Actions may be registered via register, or resources may be allocated atomically via allocate. allocate corresponds closely to bracket.

Releasing may be performed before exit via the release function. This is a highly recommended optimization, as it will ensure that scarce resources are freed early. Note that calling release will deregister the action, so that a release action will only ever be called once.

Since 0.3.0

runResourceT :: MonadBaseControl IO m => ResourceT m a -> m a

Unwrap a ResourceT transformer, and call all registered release actions.

Note that there is some reference counting involved due to resourceForkIO. If multiple threads are sharing the same collection of resources, only the last call to runResourceT will deallocate the resources.

Since 0.3.0

join :: Monad m => m (m a) -> m a

The join function is the conventional monad join operator. It is used to remove one level of monadic structure, projecting its bound argument into the outer level.

liftA2 :: Applicative f => (a -> b -> c) -> f a -> f b -> f c

Lift a binary function to actions.

liftA3 :: Applicative f => (a -> b -> c -> d) -> f a -> f b -> f c -> f d

Lift a ternary function to actions.

void :: Functor f => f a -> f ()

void value discards or ignores the result of evaluation, such as the return value of an IO action.

Examples

Replace the contents of a Maybe Int with unit:

>>> void Nothing
Nothing
>>> void (Just 3)
Just ()

Replace the contents of an Either Int Int with unit, resulting in an Either Int '()':

>>> void (Left 8675309)
Left 8675309
>>> void (Right 8675309)
Right ()

Replace every element of a list with unit:

>>> void [1,2,3]
[(),(),()]

Replace the second element of a pair with unit:

>>> void (1,2)
(1,())

Discard the result of an IO action:

>>> mapM print [1,2]
1
2
[(),()]
>>> void $ mapM print [1,2]
1
2