Copyright | © 2015–2018 Megaparsec contributors © 2007 Paolo Martini © 1999–2001 Daan Leijen |
---|---|
License | FreeBSD |
Maintainer | Mark Karpov <markkarpov92@gmail.com> |
Stability | experimental |
Portability | portable |
Safe Haskell | None |
Language | Haskell2010 |
This module includes everything you need to get started writing a parser. If you are new to Megaparsec and don't know where to begin, take a look at the tutorials https://markkarpov.com/learn-haskell.html#megaparsec-tutorials.
In addition to the Text.Megaparsec module, which exports and re-exports
most everything that you may need, we advise to import
Text.Megaparsec.Char if you plan to work with a stream of Char
tokens
or Text.Megaparsec.Byte if you intend to parse binary data.
It is common to start working with the library by defining a type synonym like this:
type Parser = Parsec Void Text ^ ^ | | Custom error component Type of input
Then you can write type signatures like Parser Int
—for a parser that
returns an Int
for example.
Similarly (since it's known to cause confusion), you should use
ParseError
type parametrized like this:
ParseError Char Void ^ ^ | | Token type Custom error component (the same you used in Parser)
Token type for String
and Text
(strict and lazy) is Char
,
for ByteString
s it's Word8
.
Megaparsec uses some type-level machinery to provide flexibility without
compromising on type safety. Thus type signatures are sometimes necessary
to avoid ambiguous types. If you're seeing a error message that reads
like “Type variable e0
is ambiguous …”, you need to give an explicit
signature to your parser to resolve the ambiguity. It's a good idea to
provide type signatures for all top-level definitions.
Megaparsec is capable of a lot. Apart from this standard functionality you can parse permutation phrases with Text.Megaparsec.Perm, expressions with Text.Megaparsec.Expr, do lexing with Text.Megaparsec.Char.Lexer and Text.Megaparsec.Byte.Lexer. These modules should be imported explicitly along with the modules mentioned above.
- module Text.Megaparsec.Pos
- module Text.Megaparsec.Error
- module Text.Megaparsec.Stream
- module Control.Monad.Combinators
- data State s = State {
- stateInput :: s
- statePos :: NonEmpty SourcePos
- stateTokensProcessed :: !Int
- stateTabWidth :: Pos
- type Parsec e s = ParsecT e s Identity
- data ParsecT e s m a
- parse :: Parsec e s a -> String -> s -> Either (ParseError (Token s) e) a
- parseMaybe :: (Ord e, Stream s) => Parsec e s a -> s -> Maybe a
- parseTest :: (ShowErrorComponent e, Ord (Token s), ShowToken (Token s), Show a) => Parsec e s a -> s -> IO ()
- parseTest' :: (ShowErrorComponent e, ShowToken (Token s), LineToken (Token s), Show a, Stream s) => Parsec e s a -> s -> IO ()
- runParser :: Parsec e s a -> String -> s -> Either (ParseError (Token s) e) a
- runParser' :: Parsec e s a -> State s -> (State s, Either (ParseError (Token s) e) a)
- runParserT :: Monad m => ParsecT e s m a -> String -> s -> m (Either (ParseError (Token s) e) a)
- runParserT' :: Monad m => ParsecT e s m a -> State s -> m (State s, Either (ParseError (Token s) e) a)
- class (Stream s, Alternative m, MonadPlus m) => MonadParsec e s m | m -> e s where
- (<?>) :: MonadParsec e s m => m a -> String -> m a
- unexpected :: MonadParsec e s m => ErrorItem (Token s) -> m a
- customFailure :: MonadParsec e s m => e -> m a
- match :: MonadParsec e s m => m a -> m (Tokens s, a)
- region :: MonadParsec e s m => (ParseError (Token s) e -> ParseError (Token s) e) -> m a -> m a
- takeRest :: MonadParsec e s m => m (Tokens s)
- atEnd :: MonadParsec e s m => m Bool
- getInput :: MonadParsec e s m => m s
- setInput :: MonadParsec e s m => s -> m ()
- getPosition :: MonadParsec e s m => m SourcePos
- getNextTokenPosition :: forall e s m. MonadParsec e s m => m (Maybe SourcePos)
- setPosition :: MonadParsec e s m => SourcePos -> m ()
- pushPosition :: MonadParsec e s m => SourcePos -> m ()
- popPosition :: MonadParsec e s m => m ()
- getTokensProcessed :: MonadParsec e s m => m Int
- setTokensProcessed :: MonadParsec e s m => Int -> m ()
- getTabWidth :: MonadParsec e s m => m Pos
- setTabWidth :: MonadParsec e s m => Pos -> m ()
- setParserState :: MonadParsec e s m => State s -> m ()
- dbg :: forall e s m a. (Stream s, ShowToken (Token s), ShowErrorComponent e, Show a) => String -> ParsecT e s m a -> ParsecT e s m a
Re-exports
Note that we re-export monadic combinators from
Control.Monad.Combinators because these are more efficient than
Applicative
-based ones. Thus many
and some
may clash with the
functions from Control.Applicative. You need to hide the functions like
this:
import Control.Applicative hiding (many, some)
Also note that you can import Control.Monad.Combinators.NonEmpty if you
wish that combinators like some
return NonEmpty
lists. The module
lives in the parser-combinators
package (you need at least version
0.4.0).
This module is intended to be imported qualified:
import qualified Control.Monad.Combinators.NonEmpty as NE
module Text.Megaparsec.Pos
module Text.Megaparsec.Error
module Text.Megaparsec.Stream
module Control.Monad.Combinators
Data types
This is the Megaparsec's state parametrized over stream type s
.
State | |
|
is a parser with custom data component of error
ParsecT
e s m ae
, stream type s
, underlying monad m
and return type a
.
(Ord e, Stream s) => MonadParsec e s (ParsecT e s m) Source # | |
(Stream s, MonadState st m) => MonadState st (ParsecT e s m) Source # | |
(Stream s, MonadReader r m) => MonadReader r (ParsecT e s m) Source # | |
(Stream s, MonadError e' m) => MonadError e' (ParsecT e s m) Source # | |
MonadTrans (ParsecT e s) Source # | |
Stream s => Monad (ParsecT e s m) Source # |
|
Functor (ParsecT e s m) Source # | |
(Stream s, MonadFix m) => MonadFix (ParsecT e s m) Source # | Since: 6.0.0 |
Stream s => MonadFail (ParsecT e s m) Source # | |
Stream s => Applicative (ParsecT e s m) Source # |
|
(Stream s, MonadIO m) => MonadIO (ParsecT e s m) Source # | |
(Ord e, Stream s) => Alternative (ParsecT e s m) Source # |
|
(Ord e, Stream s) => MonadPlus (ParsecT e s m) Source # |
|
(Stream s, MonadCont m) => MonadCont (ParsecT e s m) Source # | |
((~) * a (Tokens s), IsString a, Eq a, Stream s, Ord e) => IsString (ParsecT e s m a) Source # | Since: 6.3.0 |
(Stream s, Semigroup a) => Semigroup (ParsecT e s m a) Source # | Since: 5.3.0 |
(Stream s, Monoid a) => Monoid (ParsecT e s m a) Source # | Since: 5.3.0 |
Running parser
:: Parsec e s a | Parser to run |
-> String | Name of source file |
-> s | Input for parser |
-> Either (ParseError (Token s) e) a |
runs parser parse
p file inputp
over Identity
(see runParserT
if you're using the ParsecT
monad transformer; parse
itself is just a
synonym for runParser
). It returns either a ParseError
(Left
) or a
value of type a
(Right
). parseErrorPretty
can be used to turn
ParseError
into the string representation of the error message. See
Text.Megaparsec.Error if you need to do more advanced error analysis.
main = case (parse numbers "" "11,2,43") of Left err -> putStr (parseErrorPretty err) Right xs -> print (sum xs) numbers = integer `sepBy` char ','
parseMaybe :: (Ord e, Stream s) => Parsec e s a -> s -> Maybe a Source #
runs the parser parseMaybe
p inputp
on input
and returns the
result inside Just
on success and Nothing
on failure. This function
also parses eof
, so if the parser doesn't consume all of its input, it
will fail.
The function is supposed to be useful for lightweight parsing, where error messages (and thus file name) are not important and entire input should be parsed. For example, it can be used when parsing of a single number according to a specification of its format is desired.
:: (ShowErrorComponent e, Ord (Token s), ShowToken (Token s), Show a) | |
=> Parsec e s a | Parser to run |
-> s | Input for parser |
-> IO () |
The expression
applies the parser parseTest
p inputp
against
input input
and prints the result to stdout. Useful for testing.
:: (ShowErrorComponent e, ShowToken (Token s), LineToken (Token s), Show a, Stream s) | |
=> Parsec e s a | Parser to run |
-> s | Input for parser |
-> IO () |
A version of parseTest
that also prints offending line in parse
errors.
Since: 6.0.0
:: Parsec e s a | Parser to run |
-> String | Name of source file |
-> s | Input for parser |
-> Either (ParseError (Token s) e) a |
runs parser runParser
p file inputp
on the input stream of
tokens input
, obtained from source file
. The file
is only used in
error messages and may be the empty string. Returns either a ParseError
(Left
) or a value of type a
(Right
).
parseFromFile p file = runParser p file <$> readFile file
:: Monad m | |
=> ParsecT e s m a | Parser to run |
-> String | Name of source file |
-> s | Input for parser |
-> m (Either (ParseError (Token s) e) a) |
runs parser runParserT
p file inputp
on the input list of tokens
input
, obtained from source file
. The file
is only used in error
messages and may be the empty string. Returns a computation in the
underlying monad m
that returns either a ParseError
(Left
) or a
value of type a
(Right
).
:: Monad m | |
=> ParsecT e s m a | Parser to run |
-> State s | Initial state |
-> m (State s, Either (ParseError (Token s) e) a) |
This function is similar to runParserT
, but like runParser'
it
accepts and returns parser state. This is thus the most general way to
run a parser.
Since: 4.2.0
Primitive combinators
class (Stream s, Alternative m, MonadPlus m) => MonadParsec e s m | m -> e s where Source #
Type class describing monads that implement the full set of primitive parsers.
Note carefully that the following primitives are “fast” and should be
taken advantage of as much as possible if your aim is a fast parser:
tokens
, takeWhileP
, takeWhile1P
, and takeP
.
failure, fancyFailure, label, try, lookAhead, notFollowedBy, withRecovery, observing, eof, token, tokens, takeWhileP, takeWhile1P, takeP, getParserState, updateParserState
:: Maybe (ErrorItem (Token s)) | Unexpected item (if any) |
-> Set (ErrorItem (Token s)) | Expected items |
-> m a |
The most general way to stop parsing and report a trivial
ParseError
.
Since: 6.0.0
:: Set (ErrorFancy e) | Fancy error components |
-> m a |
The most general way to stop parsing and report a fancy ParseError
.
To report a single custom parse error, see customFailure
.
Since: 6.0.0
label :: String -> m a -> m a Source #
The parser
behaves as parser label
name pp
, but whenever the
parser p
fails without consuming any input, it replaces names of
“expected” tokens with the name name
.
behaves just like parser hidden
pp
, but it doesn't show any
“expected” tokens in error message when p
fails.
The parser
behaves like parser try
pp
, except that it
backtracks the parser state when p
fails (either consuming input or
not).
This combinator is used whenever arbitrary look ahead is needed. Since
it pretends that it hasn't consumed any input when p
fails, the
(<|>
) combinator will try its second alternative even if the first
parser failed while consuming input.
For example, here is a parser that is supposed to parse the word “let” or the word “lexical”:
>>>
parseTest (string "let" <|> string "lexical") "lexical"
1:1: unexpected "lex" expecting "let"
What happens here? The first parser consumes “le” and fails (because it
doesn't see a “t”). The second parser, however, isn't tried, since the
first parser has already consumed some input! try
fixes this behavior
and allows backtracking to work:
>>>
parseTest (try (string "let") <|> string "lexical") "lexical"
"lexical"
try
also improves error messages in case of overlapping alternatives,
because Megaparsec's hint system can be used:
>>>
parseTest (try (string "let") <|> string "lexical") "le"
1:1: unexpected "le" expecting "let" or "lexical"
Please note that as of Megaparsec 4.4.0, string
backtracks
automatically (see tokens
), so it does not need try
. However, the
examples above demonstrate the idea behind try
so well that it was
decided to keep them. You still need to use try
when your
alternatives are complex, composite parsers.
lookAhead :: m a -> m a Source #
If p
in
succeeds (either consuming input or not)
the whole parser behaves like lookAhead
pp
succeeded without consuming anything
(parser state is not updated as well). If p
fails, lookAhead
has no
effect, i.e. it will fail consuming input if p
fails consuming input.
Combine with try
if this is undesirable.
notFollowedBy :: m a -> m () Source #
only succeeds when the parser notFollowedBy
pp
fails. This
parser never consumes any input and never modifies parser state. It
can be used to implement the “longest match” rule.
:: (ParseError (Token s) e -> m a) | How to recover from failure |
-> m a | Original parser |
-> m a | Parser that can recover from failures |
allows continue parsing even if parser withRecovery
r pp
fails. In this case r
is called with the actual ParseError
as its
argument. Typical usage is to return a value signifying failure to
parse this particular object and to consume some part of the input up
to the point where the next object starts.
Note that if r
fails, original error message is reported as if
without withRecovery
. In no way recovering parser r
can influence
error messages.
Since: 4.4.0
:: m a | The parser to run |
-> m (Either (ParseError (Token s) e) a) |
allows to “observe” failure of the observing
pp
parser, should
it happen, without actually ending parsing, but instead getting the
ParseError
in Left
. On success parsed value is returned in Right
as usual. Note that this primitive just allows you to observe parse
errors as they happen, it does not backtrack or change how the p
parser works in any way.
Since: 5.1.0
This parser only succeeds at the end of the input.
:: (Token s -> Either (Maybe (ErrorItem (Token s)), Set (ErrorItem (Token s))) a) | Matching function for the token to parse, it allows to construct arbitrary error message on failure as well; things in the tuple are: unexpected item (if any) and expected items |
-> Maybe (Token s) | Token to report when input stream is empty |
-> m a |
The parser
accepts a token token
test mrept
with result x
when the function test t
returns
. Right
xmrep
may provide
representation of the token to report in error messages when input
stream in empty.
This is the most primitive combinator for accepting tokens. For
example, the satisfy
parser is implemented as:
satisfy f = token testChar Nothing where testChar x = if f x then Right x else Left (pure (Tokens (x:|[])), Set.empty)
:: (Tokens s -> Tokens s -> Bool) | Predicate to check equality of chunks |
-> Tokens s | Chunk of input to match against |
-> m (Tokens s) |
The parser
parses a chunk of input and returns it.
Supplied predicate tokens
testtest
is used to check equality of given and parsed
chunks after a candidate chunk of correct length is fetched from the
stream.
This can be used for example to write string
:
string = tokens (==)
Note that beginning from Megaparsec 4.4.0, this is an auto-backtracking
primitive, which means that if it fails, it never consumes any input.
This is done to make its consumption model match how error messages for
this primitive are reported (which becomes an important thing as user
gets more control with primitives like withRecovery
):
>>>
parseTest (string "abc") "abd"
1:1: unexpected "abd" expecting "abc"
This means, in particular, that it's no longer necessary to use try
with tokens
-based parsers, such as string
and
string'
. This feature does not affect
performance in any way.
:: Maybe String | Name for a single token in the row |
-> (Token s -> Bool) | Predicate to use to test tokens |
-> m (Tokens s) | A chunk of matching tokens |
Parse zero or more tokens for which the supplied predicate holds.
Try to use this as much as possible because for many streams the
combinator is much faster than parsers built with many
and
satisfy
.
The following equations should clarify the behavior:
takeWhileP (Just "foo") f = many (satisfy f <?> "foo") takeWhileP Nothing f = many (satisfy f)
The combinator never fails, although it may parse an empty chunk.
Since: 6.0.0
:: Maybe String | Name for a single token in the row |
-> (Token s -> Bool) | Predicate to use to test tokens |
-> m (Tokens s) | A chunk of matching tokens |
Similar to takeWhileP
, but fails if it can't parse at least one
token. Note that the combinator either succeeds or fails without
consuming any input, so try
is not necessary with it.
Since: 6.0.0
:: Maybe String | Name for a single token in the row |
-> Int | How many tokens to extract |
-> m (Tokens s) | A chunk of matching tokens |
Extract the specified number of tokens from the input stream and return them packed as a chunk of stream. If there is not enough tokens in the stream, a parse error will be signaled. It's guaranteed that if the parser succeeds, the requested number of tokens will be returned.
The parser is roughly equivalent to:
takeP (Just "foo") n = count n (anyChar <?> "foo") takeP Nothing n = count n anyChar
Note that if the combinator fails due to insufficient number of tokens
in the input stream, it backtracks automatically. No try
is necessary
with takeP
.
Since: 6.0.0
getParserState :: m (State s) Source #
Return the full parser state as a State
record.
updateParserState :: (State s -> State s) -> m () Source #
applies the function updateParserState
ff
to the parser state.
MonadParsec e s m => MonadParsec e s (IdentityT * m) Source # | |
(Monoid w, MonadParsec e s m) => MonadParsec e s (WriterT w m) Source # | |
(Monoid w, MonadParsec e s m) => MonadParsec e s (WriterT w m) Source # | |
MonadParsec e s m => MonadParsec e s (StateT st m) Source # | |
MonadParsec e s m => MonadParsec e s (StateT st m) Source # | |
MonadParsec e s m => MonadParsec e s (ReaderT * r m) Source # | |
(Ord e, Stream s) => MonadParsec e s (ParsecT e s m) Source # | |
(Monoid w, MonadParsec e s m) => MonadParsec e s (RWST r w st m) Source # | Since: 5.2.0 |
(Monoid w, MonadParsec e s m) => MonadParsec e s (RWST r w st m) Source # | Since: 5.2.0 |
Derivatives of primitive combinators
(<?>) :: MonadParsec e s m => m a -> String -> m a infix 0 Source #
A synonym for label
in the form of an operator.
unexpected :: MonadParsec e s m => ErrorItem (Token s) -> m a Source #
The parser
fails with an error message telling
about unexpected item unexpected
itemitem
without consuming any input.
unexpected item = failure (pure item) Set.empty
customFailure :: MonadParsec e s m => e -> m a Source #
Report a custom parse error. For a more general version, see
fancyFailure
.
Since: 6.3.0
match :: MonadParsec e s m => m a -> m (Tokens s, a) Source #
Return both the result of a parse and a chunk of input that was
consumed during parsing. This relies on the change of the
stateTokensProcessed
value to evaluate how many tokens were consumed.
If you mess with it manually in the argument parser, prepare for
troubles.
Since: 5.3.0
:: MonadParsec e s m | |
=> (ParseError (Token s) e -> ParseError (Token s) e) | How to process |
-> m a | The “region” that the processing applies to |
-> m a |
Specify how to process ParseError
s that happen inside of this
wrapper. As a side effect of the current implementation changing
errorPos
with this combinator will also change the final statePos
in
the parser state (try to avoid that because statePos
will go out of
sync with factual position in the input stream, which is probably OK if
you finish parsing right after that, but be warned).
Since: 5.3.0
takeRest :: MonadParsec e s m => m (Tokens s) Source #
Consume the rest of the input and return it as a chunk. This parser never fails, but may return an empty chunk.
takeRest = takeWhileP Nothing (const True)
Since: 6.0.0
atEnd :: MonadParsec e s m => m Bool Source #
Return True
when end of input has been reached.
Since: 6.0.0
Parser state combinators
getInput :: MonadParsec e s m => m s Source #
Return the current input.
setInput :: MonadParsec e s m => s -> m () Source #
getPosition :: MonadParsec e s m => m SourcePos Source #
Return the current source position.
See also: setPosition
, pushPosition
, popPosition
, and SourcePos
.
getNextTokenPosition :: forall e s m. MonadParsec e s m => m (Maybe SourcePos) Source #
Get the position where the next token in the stream begins. If the
stream is empty, return Nothing
.
Since: 5.3.0
setPosition :: MonadParsec e s m => SourcePos -> m () Source #
sets the current source position to setPosition
pospos
.
See also: getPosition
, pushPosition
, popPosition
, and SourcePos
.
pushPosition :: MonadParsec e s m => SourcePos -> m () Source #
Push a position into stack of positions and continue parsing working with this position. Useful for working with include files and the like.
See also: getPosition
, setPosition
, popPosition
, and SourcePos
.
Since: 5.0.0
popPosition :: MonadParsec e s m => m () Source #
Pop a position from the stack of positions unless it only contains one
element (in that case the stack of positions remains the same). This is
how to return to previous source file after pushPosition
.
See also: getPosition
, setPosition
, pushPosition
, and SourcePos
.
Since: 5.0.0
getTokensProcessed :: MonadParsec e s m => m Int Source #
Get the number of tokens processed so far.
Since: 6.0.0
setTokensProcessed :: MonadParsec e s m => Int -> m () Source #
Set the number of tokens processed so far.
Since: 6.0.0
getTabWidth :: MonadParsec e s m => m Pos Source #
Return the tab width. The default tab width is equal to
defaultTabWidth
. You can set a different tab width with the help of
setTabWidth
.
setTabWidth :: MonadParsec e s m => Pos -> m () Source #
Set tab width. If the argument of the function is not a positive
number, defaultTabWidth
will be used.
setParserState :: MonadParsec e s m => State s -> m () Source #
sets the parser state to setParserState
stst
.
Debugging
:: (Stream s, ShowToken (Token s), ShowErrorComponent e, Show a) | |
=> String | Debugging label |
-> ParsecT e s m a | Parser to debug |
-> ParsecT e s m a | Parser that prints debugging messages |
parser works exactly like dbg
label pp
, but when it's evaluated
it also prints information useful for debugging. The label
is only used
to refer to this parser in the debugging output. This combinator uses the
trace
function from Debug.Trace under the hood.
Typical usage is to wrap every sub-parser in misbehaving parser with
dbg
assigning meaningful labels. Then give it a shot and go through the
print-out. As of current version, this combinator prints all available
information except for hints, which are probably only interesting to
the maintainer of Megaparsec itself and may be quite verbose to output in
general. Let me know if you would like to be able to see hints in the
debugging output.
The output itself is pretty self-explanatory, although the following abbreviations should be clarified (they are derived from the low-level source code):
COK
—“consumed OK”. The parser consumed input and succeeded.CERR
—“consumed error”. The parser consumed input and failed.EOK
—“empty OK”. The parser succeeded without consuming input.EERR
—“empty error”. The parser failed without consuming input.
Finally, it's not possible to lift this function into some monad
transformers without introducing surprising behavior (e.g. unexpected
state backtracking) or adding otherwise redundant constraints (e.g.
Show
instance for state), so this helper is only available for
ParsecT
monad, not MonadParsec
in general.
Since: 5.1.0