Text.Megaparsec.Stream

Megaparsec's input stream facilities.

You probably do not want to import this module directly because Text.Megaparsec re-exports it anyway.

Since: 6.0.0

Documentation

class (Ord (Token s), Ord (Tokens s)) => Stream s where

Type class for inputs that can be consumed by the library.

Note: before the version 9.0.0 the class included the methods from VisualStream and TraversableStream.

Minimal complete definition

tokensToChunk, chunkToTokens, chunkLength, take1_, takeN_, takeWhile_

Associated Types

type Token s :: Type

Type of token in the stream.

type Tokens s :: Type

Type of “chunk” of the stream.

Methods

tokenToChunk :: Proxy s -> Token s -> Tokens s

Lift a single token to chunk of the stream. The default implementation is:

tokenToChunk pxy = tokensToChunk pxy . pure

However for some types of stream there may be a more efficient way to lift.

tokensToChunk :: Proxy s -> [Token s] -> Tokens s

The first method that establishes isomorphism between list of tokens and chunk of the stream. Valid implementation should satisfy:

chunkToTokens pxy (tokensToChunk pxy ts) == ts

chunkToTokens :: Proxy s -> Tokens s -> [Token s]

The second method that establishes isomorphism between list of tokens and chunk of the stream. Valid implementation should satisfy:

tokensToChunk pxy (chunkToTokens pxy chunk) == chunk

chunkLength :: Proxy s -> Tokens s -> Int

Return length of a chunk of the stream.

chunkEmpty :: Proxy s -> Tokens s -> Bool

Check if a chunk of the stream is empty. The default implementation is in terms of the more general chunkLength:

chunkEmpty pxy ts = chunkLength pxy ts <= 0

However for many streams there may be a more efficient implementation.

take1_ :: s -> Maybe (Token s, s)

Extract a single token form the stream. Return Nothing if the stream is empty.

takeN_ :: Int -> s -> Maybe (Tokens s, s)

takeN_ n s should try to extract a chunk of length n, or if the stream is too short, the rest of the stream. Valid implementation should follow the rules:

• If the requested length n is 0 (or less), Nothing should never be returned, instead Just ("", s) should be returned, where "" stands for the empty chunk, and s is the original stream (second argument).
• If the requested length is greater than 0 and the stream is empty, Nothing should be returned indicating end of input.
• In other cases, take chunk of length n (or shorter if the stream is not long enough) from the input stream and return the chunk along with the rest of the stream.

takeWhile_ :: (Token s -> Bool) -> s -> (Tokens s, s)

Extract chunk of the stream taking tokens while the supplied predicate returns True. Return the chunk and the rest of the stream.

For many types of streams, the method allows for significant performance improvements, although it is not strictly necessary from conceptual point of view.

Instances

Stream ByteString

Defined in Text.Megaparsec.Stream

Associated Types

type Token ByteString

type Tokens ByteString

Methods

tokenToChunk :: Proxy ByteString -> Token ByteString -> Tokens ByteString

tokensToChunk :: Proxy ByteString -> [Token ByteString] -> Tokens ByteString

chunkToTokens :: Proxy ByteString -> Tokens ByteString -> [Token ByteString]

chunkLength :: Proxy ByteString -> Tokens ByteString -> Int

chunkEmpty :: Proxy ByteString -> Tokens ByteString -> Bool

take1_ :: ByteString -> Maybe (Token ByteString, ByteString)

takeN_ :: Int -> ByteString -> Maybe (Tokens ByteString, ByteString)

takeWhile_ :: (Token ByteString -> Bool) -> ByteString -> (Tokens ByteString, ByteString)

Stream ByteString

Defined in Text.Megaparsec.Stream

Associated Types

type Token ByteString

type Tokens ByteString

Methods

tokenToChunk :: Proxy ByteString -> Token ByteString -> Tokens ByteString

tokensToChunk :: Proxy ByteString -> [Token ByteString] -> Tokens ByteString

chunkToTokens :: Proxy ByteString -> Tokens ByteString -> [Token ByteString]

chunkLength :: Proxy ByteString -> Tokens ByteString -> Int

chunkEmpty :: Proxy ByteString -> Tokens ByteString -> Bool

take1_ :: ByteString -> Maybe (Token ByteString, ByteString)

takeN_ :: Int -> ByteString -> Maybe (Tokens ByteString, ByteString)

takeWhile_ :: (Token ByteString -> Bool) -> ByteString -> (Tokens ByteString, ByteString)

Stream Text

Defined in Text.Megaparsec.Stream

Associated Types

type Token Text

type Tokens Text

Methods

tokenToChunk :: Proxy Text -> Token Text -> Tokens Text

tokensToChunk :: Proxy Text -> [Token Text] -> Tokens Text

chunkToTokens :: Proxy Text -> Tokens Text -> [Token Text]

chunkLength :: Proxy Text -> Tokens Text -> Int

chunkEmpty :: Proxy Text -> Tokens Text -> Bool

take1_ :: Text -> Maybe (Token Text, Text)

takeN_ :: Int -> Text -> Maybe (Tokens Text, Text)

takeWhile_ :: (Token Text -> Bool) -> Text -> (Tokens Text, Text)

Stream Text

Defined in Text.Megaparsec.Stream

Associated Types

type Token Text

type Tokens Text

Methods

tokenToChunk :: Proxy Text -> Token Text -> Tokens Text

tokensToChunk :: Proxy Text -> [Token Text] -> Tokens Text

chunkToTokens :: Proxy Text -> Tokens Text -> [Token Text]

chunkLength :: Proxy Text -> Tokens Text -> Int

chunkEmpty :: Proxy Text -> Tokens Text -> Bool

take1_ :: Text -> Maybe (Token Text, Text)

takeN_ :: Int -> Text -> Maybe (Tokens Text, Text)

takeWhile_ :: (Token Text -> Bool) -> Text -> (Tokens Text, Text)

Ord a => Stream [a]

Since: 9.0.0

Defined in Text.Megaparsec.Stream

Associated Types

type Token [a]

type Tokens [a]

Methods

tokenToChunk :: Proxy [a] -> Token [a] -> Tokens [a]

tokensToChunk :: Proxy [a] -> [Token [a]] -> Tokens [a]

chunkToTokens :: Proxy [a] -> Tokens [a] -> [Token [a]]

chunkLength :: Proxy [a] -> Tokens [a] -> Int

chunkEmpty :: Proxy [a] -> Tokens [a] -> Bool

take1_ :: [a] -> Maybe (Token [a], [a])

takeN_ :: Int -> [a] -> Maybe (Tokens [a], [a])

takeWhile_ :: (Token [a] -> Bool) -> [a] -> (Tokens [a], [a])

Ord a => Stream (Seq a)

Since: 9.0.0

Defined in Text.Megaparsec.Stream

Associated Types

type Token (Seq a)

type Tokens (Seq a)

Methods

tokenToChunk :: Proxy (Seq a) -> Token (Seq a) -> Tokens (Seq a)

tokensToChunk :: Proxy (Seq a) -> [Token (Seq a)] -> Tokens (Seq a)

chunkToTokens :: Proxy (Seq a) -> Tokens (Seq a) -> [Token (Seq a)]

chunkLength :: Proxy (Seq a) -> Tokens (Seq a) -> Int

chunkEmpty :: Proxy (Seq a) -> Tokens (Seq a) -> Bool

take1_ :: Seq a -> Maybe (Token (Seq a), Seq a)

takeN_ :: Int -> Seq a -> Maybe (Tokens (Seq a), Seq a)

takeWhile_ :: (Token (Seq a) -> Bool) -> Seq a -> (Tokens (Seq a), Seq a)

class Stream s => VisualStream s where

Type class for inputs that can also be used for debugging.

Since: 9.0.0

Minimal complete definition

showTokens

Methods

showTokens :: Proxy s -> NonEmpty (Token s) -> String

Pretty-print non-empty stream of tokens. This function is also used to print single tokens (represented as singleton lists).

Since: 7.0.0

tokensLength :: Proxy s -> NonEmpty (Token s) -> Int

Return the number of characters that a non-empty stream of tokens spans. The default implementation is sufficient if every token spans exactly 1 character.

Since: 8.0.0

Instances

VisualStream String

Defined in Text.Megaparsec.Stream

Methods

showTokens :: Proxy String -> NonEmpty (Token String) -> String

tokensLength :: Proxy String -> NonEmpty (Token String) -> Int

VisualStream ByteString

Defined in Text.Megaparsec.Stream

Methods

showTokens :: Proxy ByteString -> NonEmpty (Token ByteString) -> String

tokensLength :: Proxy ByteString -> NonEmpty (Token ByteString) -> Int

VisualStream ByteString

Defined in Text.Megaparsec.Stream

Methods

showTokens :: Proxy ByteString -> NonEmpty (Token ByteString) -> String

tokensLength :: Proxy ByteString -> NonEmpty (Token ByteString) -> Int

VisualStream Text

Defined in Text.Megaparsec.Stream

Methods

showTokens :: Proxy Text -> NonEmpty (Token Text) -> String

tokensLength :: Proxy Text -> NonEmpty (Token Text) -> Int

VisualStream Text

Defined in Text.Megaparsec.Stream

Methods

showTokens :: Proxy Text -> NonEmpty (Token Text) -> String

tokensLength :: Proxy Text -> NonEmpty (Token Text) -> Int

class Stream s => TraversableStream s where

Type class for inputs that can also be used for error reporting.

Since: 9.0.0

Minimal complete definition

reachOffset | reachOffsetNoLine

Methods

reachOffset

Arguments

:: Int

Offset to reach

-> PosState s

Initial PosState to use

-> (Maybe String, PosState s)

See the description of the function

Given an offset o and initial PosState, adjust the state in such a way that it starts at the offset.

Return two values (in order):

• Maybe String representing the line on which the given offset o is located. It can be omitted (i.e. Nothing); in that case error reporting functions will not show offending lines. If returned, the line should satisfy a number of conditions that are described below.
• The updated PosState which can be in turn used to locate another offset o' given that o' >= o.

The String representing the offending line in input stream should satisfy the following:

• It should adequately represent location of token at the offset of interest, that is, character at sourceColumn of the returned SourcePos should correspond to the token at the offset o.
• It should not include the newline at the end.
• It should not be empty, if the line happens to be empty, it should be replaced with the string "<empty line>".
• Tab characters should be replaced by appropriate number of spaces, which is determined by the pstateTabWidth field of PosState.

Note: type signature of the function was changed in the version 9.0.0.

Since: 7.0.0

reachOffsetNoLine

Arguments

:: Int

Offset to reach

-> PosState s

Initial PosState to use

-> PosState s

Reached source position and updated state

A version of reachOffset that may be faster because it doesn't need to fetch the line at which the given offset in located.

The default implementation is this:

reachOffsetNoLine o pst =
  snd (reachOffset o pst)

Note: type signature of the function was changed in the version 8.0.0.

Since: 7.0.0

Instances

TraversableStream String

Defined in Text.Megaparsec.Stream

Methods

reachOffset :: Int -> PosState String -> (Maybe String, PosState String)

reachOffsetNoLine :: Int -> PosState String -> PosState String

TraversableStream ByteString

Defined in Text.Megaparsec.Stream

Methods

reachOffset :: Int -> PosState ByteString -> (Maybe String, PosState ByteString)

reachOffsetNoLine :: Int -> PosState ByteString -> PosState ByteString

TraversableStream ByteString

Defined in Text.Megaparsec.Stream

Methods

reachOffset :: Int -> PosState ByteString -> (Maybe String, PosState ByteString)

reachOffsetNoLine :: Int -> PosState ByteString -> PosState ByteString

TraversableStream Text

Defined in Text.Megaparsec.Stream

Methods

reachOffset :: Int -> PosState Text -> (Maybe String, PosState Text)

reachOffsetNoLine :: Int -> PosState Text -> PosState Text

TraversableStream Text

Defined in Text.Megaparsec.Stream

Methods

reachOffset :: Int -> PosState Text -> (Maybe String, PosState Text)

reachOffsetNoLine :: Int -> PosState Text -> PosState Text