Data.Csv.Streaming

This module allows for streaming decoding of CSV data. This is useful if you need to parse large amounts of input in constant space. The API also allows you to ignore type conversion errors on a per-record basis.

Usage example

A short usage example:

for_ (decode NoHeader "John,27\r\nJane,28\r\n") $ \ (name, age :: Int) ->
    putStrLn $ name ++ " is " ++ show age ++ " years old"

N.B. The Foldable instance, which is used above, skips records that failed to convert. If you don't want this behavior, work directly with the Cons and Nil constructors.

Stream representation

A stream of records is represented as a (lazy) list that may contain errors.

data Records a

A stream of parsed records. If type conversion failed for the record, the error is returned as Left errMsg.

Constructors

Cons (Either String a) (Records a)

A record or an error message, followed by more records.

Nil (Maybe String) ByteString

End of stream, potentially due to a parse error. If a parse error occured, the first field contains the error message. The second field contains any unconsumed input.

Instances

Functor Records

Methods

fmap :: (a -> b) -> Records a -> Records b

(<$) :: a -> Records b -> Records a

Foldable Records

Skips records that failed to convert.

Methods

fold :: Monoid m => Records m -> m

foldMap :: Monoid m => (a -> m) -> Records a -> m

foldr :: (a -> b -> b) -> b -> Records a -> b

foldr' :: (a -> b -> b) -> b -> Records a -> b

foldl :: (b -> a -> b) -> b -> Records a -> b

foldl' :: (b -> a -> b) -> b -> Records a -> b

foldr1 :: (a -> a -> a) -> Records a -> a

foldl1 :: (a -> a -> a) -> Records a -> a

toList :: Records a -> [a]

null :: Records a -> Bool

length :: Records a -> Int

elem :: Eq a => a -> Records a -> Bool

maximum :: Ord a => Records a -> a

minimum :: Ord a => Records a -> a

sum :: Num a => Records a -> a

product :: Num a => Records a -> a

Traversable Records

Methods

traverse :: Applicative f => (a -> f b) -> Records a -> f (Records b)

sequenceA :: Applicative f => Records (f a) -> f (Records a)

mapM :: Monad m => (a -> m b) -> Records a -> m (Records b)

sequence :: Monad m => Records (m a) -> m (Records a)

Eq a => Eq (Records a)

Methods

(==) :: Records a -> Records a -> Bool

(/=) :: Records a -> Records a -> Bool

Show a => Show (Records a)

Methods

showsPrec :: Int -> Records a -> ShowS

show :: Records a -> String

showList :: [Records a] -> ShowS

NFData a => NFData (Records a)

Methods

rnf :: Records a -> ()

Decoding records

Just like in the case of non-streaming decoding, there are two ways to convert CSV records to and from and user-defined data types: index-based conversion and name-based conversion.

Index-based record conversion

See documentation on index-based conversion in Data.Csv for more information.

data HasHeader

Is the CSV data preceded by a header?

Constructors

HasHeader

The CSV data is preceded by a header

NoHeader

The CSV data is not preceded by a header

decode

Arguments

:: FromRecord a

=> HasHeader

Data contains header that should be skipped

-> ByteString

CSV data

-> Records a

Efficiently deserialize CSV records in a streaming fashion. Equivalent to decodeWith defaultDecodeOptions.

decodeWith

Arguments

:: FromRecord a

=> DecodeOptions

Decoding options

-> HasHeader

Data contains header that should be skipped

-> ByteString

CSV data

-> Records a

Like decode, but lets you customize how the CSV data is parsed.

Name-based record conversion

See documentation on name-based conversion in Data.Csv for more information.

decodeByName

Arguments

:: FromNamedRecord a

=> ByteString

CSV data

-> Either String (Header, Records a)

Efficiently deserialize CSV in a streaming fashion. The data is assumed to be preceeded by a header. Returns Left errMsg if parsing the header fails. Equivalent to decodeByNameWith defaultDecodeOptions.

decodeByNameWith

Arguments

:: FromNamedRecord a

=> DecodeOptions

Decoding options

-> ByteString

CSV data

-> Either String (Header, Records a)

Like decodeByName, but lets you customize how the CSV data is parsed.