Frames.MapReduce

Frames-map-reduce provides helper functions for using map-reduce-folds with Frames. Please see those packages for more details.

Unpackers

unpackFilterRow :: (Record rs -> Bool) -> Unpack (Record rs) (Record rs)

Filter records using a function on the entire record.

unpackFilterOnField :: forall t rs. (KnownField t, ElemOf rs t) => (Snd t -> Bool) -> Unpack (Record rs) (Record rs)

Filter records based on a condition on only one field in the row. Will usually require a Type Application to indicate which field.

unpackGoodRows :: forall cs rs. cs ⊆ rs => Unpack (Rec (Maybe :. ElField) rs) (Record cs)

An unpack step which specifies a subset of columns, cs, (via a type-application) and then filters a Rec (Maybe :. Elfield) rs to only rows which have all good data in that subset.

Assigners

assignKeysAndData :: forall ks cs rs. (ks ⊆ rs, cs ⊆ rs) => Assign (Record ks) (Record rs) (Record cs)

Assign both keys and data cols. Uses type applications to specify them if they cannot be inferred. Keys usually can't. Data sometimes can.

assignKeys :: forall ks rs. ks ⊆ rs => Assign (Record ks) (Record rs) (Record rs)

Assign keys and leave all columns, including the keys, in the data passed to reduce.

splitOnKeys :: forall ks rs cs. (ks ⊆ rs, cs ~ RDeleteAll ks rs, cs ⊆ rs) => Assign (Record ks) (Record rs) (Record cs)

Assign keys and leave the rest of the columns, excluding the keys, in the data passed to reduce.

splitOnData :: forall cs rs ks. (cs ⊆ rs, ks ~ RDeleteAll cs rs, ks ⊆ rs) => Assign (Record ks) (Record rs) (Record cs)

Assign data and leave the rest of the columns, excluding the data, as the key.

Reduce and Re-Attach Key Cols

reduceAndAddKey

Arguments

:: RecVec (ks ++ cs)

=> (forall h. Foldable h => h x -> Record cs)

reduction step

-> Reduce (Record ks) x (FrameRec (ks ++ cs))

Reduce the data to a single row and then re-attach the key.

foldAndAddKey

Arguments

:: RecVec (ks ++ cs)

=> Fold x (Record cs)

reduction fold

-> Reduce (Record ks) x (FrameRec (ks ++ cs))

Reduce by folding the data to a single row and then re-attaching the key.

Re-Attach Key Cols

makeRecsWithKey

Arguments

:: (Functor g, Foldable g, RecVec (ks ++ as))

=> (y -> Record as)

map a result to a record

-> Reduce (Record ks) x (g y)

original reduce

-> Reduce (Record ks) x (FrameRec (ks ++ as))

Transform a reduce which produces a container of results, with a function from each result to a record, into a reduce which produces a FrameRec of the result records with the key re-attached.

makeRecsWithKeyM

Arguments

:: (Monad m, Functor g, Foldable g, RecVec (ks ++ as))

=> (y -> Record as)

map a result to a record

-> ReduceM m (Record ks) x (g y)

original reduce

-> ReduceM m (Record ks) x (FrameRec (ks ++ as))

Transform an effectful reduce which produces a container of results, with a function from each result to a record, into a reduce which produces a FrameRec of the result records with the key re-attached.

Re-Exports

module Control.MapReduce

module Control.MapReduce

Orphan instances

(KnownField t, Hashable (Snd t), Hashable (Record rs), rs ⊆ (t ': rs)) => Hashable (Record (t ': rs))

Methods

hashWithSalt :: Int -> Record (t ': rs) -> Int

hash :: Record (t ': rs) -> Int

Hashable (Record ([] :: [(Symbol, Type)]))

This is only here so we can use hash maps for the grouping step. This should properly be in Frames itself.

Methods

hashWithSalt :: Int -> Record [] -> Int

hash :: Record [] -> Int