Lima.Converter

Terms

• format - specific encoding of some information. See Format.
• document - Text in a specific format, e.g., Haskell (.hs) file.
• document block - consecutive lines of a document.
• Token - a representation of a document block as a Haskell type.
• Tokens - a list of Tokens.
• parser - a function that reads a document line by line and converts it to Tokens. Example: hsToTokens.
• printer - a function that converts Tokens to a document. Example: hsFromTokens.

• tag - a marker that affects how Tokens are parsed.

  • Each parser recognizes tags of a specific form.

  • Tags can be represented as a wrapper and a name.

    E.g., in '% LIMA_DISABLE some text', a TeX tag, the wrapper is '% ' and the name is 'LIMA_DISABLE some text'.

  • Parsers recognize the tag names that start with tag names specified in a Config.

    E.g., in the example above, a parser will recognize the _disable tag and will become disabled.

  • When a parser is disabled, it copies lines verbatim into a Disabled Token and doesn't recognize any tags until it finds an _enable tag.

Assumptions

The following assumptions must hold for outputs of parsers and inputs of printers:

• Tokens are in the same order as the corresponding blocks of document.

• Lines inside Tokens are reversed compared to the document. Example:

  • Literate Haskell document:

    line 1
    line 2
    
    % line 3
    
    % line 4
    

  • Corresponding Tokens:

    [
      Text {manyLines = ["line2","line 1"]},
      Comment {someLines = "line 4" :| ["", "line 3"]}
    ]
    

• There are no leading or trailing empty lines inside of Tokens.

There are several forms of Haskell code blocks in Literate Haskell recognized by GHC.

1. Code between '\begin{code}' and '\end{code}' tags.

   begin{code}
      a = 42
   end{code}
   begin{code}
      b = a
   end{code}
   

   • The line starting with '\begin{code}' cannot have other non-space characters after '\begin{code}'.
   • The indentation of all expressions in code blocks must be the same.

2. Code lines starting with '> '.

   begin{mycode}
   
   >    a = 42
   
   end{mycode}
   begin{mycode}
   
   >    b = a
   
   end{mycode}
   

   • There must be at least a single empty line before and after each Haskell code block.
   • Any text may surround Haskell code blocks.
   • The indentation of all expressions in code blocks must be the same.

This library supports only the second form as this form is more versatile.

Moreover, this form does not require writing Markdown tags like '```haskell'.

Such tags will automatically be printed when converting Literate Haskell to Markdown.

Config

type family Mode a where ...

Calculates the mode for data.

Equations

Mode User = Maybe String

Mode Internal = Text

type User = 'User

Marks data supplied by a user.

type Internal = 'Internal

Marks data for internal usage.

data Config (a :: Mode')

Configuration of tag names.

The default values of Config User are all Nothings.

Inside the library functions, Config User is converted to Config Internal.

The below examples show the names from Config Internal.

>>> pp (def :: Config User)
Config {
  _disable = "LIMA_DISABLE",
  _enable = "LIMA_ENABLE",
  _indent = "LIMA_INDENT",
  _dedent = "LIMA_DEDENT",
  _mdHaskellCodeStart = "```haskell",
  _mdHaskellCodeEnd = "```",
  _texHaskellCodeStart = "\\begin{mycode}",
  _texHaskellCodeEnd = "\\end{mycode}",
  _texSingleLineCommentStart = "SINGLE_LINE ",
  _lhsSingleLineCommentStart = "SINGLE_LINE "
}

It's possible to override these names.

>>> pp ((def :: Config User) & disable ?~ "off" & enable ?~ "on" & indent ?~ "indent" & dedent ?~ "dedent")
Config {
  _disable = "off",
  _enable = "on",
  _indent = "indent",
  _dedent = "dedent",
  _mdHaskellCodeStart = "```haskell",
  _mdHaskellCodeEnd = "```",
  _texHaskellCodeStart = "\\begin{mycode}",
  _texHaskellCodeEnd = "\\end{mycode}",
  _texSingleLineCommentStart = "SINGLE_LINE ",
  _lhsSingleLineCommentStart = "SINGLE_LINE "
}

Constructors

Config

Fields

• _disable :: Mode a

  Make parser ignore tags and just copy the following lines verbatim.

  Set indentation to 0.

• _enable :: Mode a

  Stop parser from ignoring tags.

• _indent :: Mode a

  Set code indentation to a given Int.

• _dedent :: Mode a

  Set code indentation to 0.

• _mdHaskellCodeStart :: Mode a

  Mark the start of a Haskell code block in Markdown.

• _mdHaskellCodeEnd :: Mode a

  Mark the end of a Haskell code block in Markdown.

• _texHaskellCodeStart :: Mode a

  Mark the start of a Haskell code block in TeX.

• _texHaskellCodeEnd :: Mode a

  Mark the end of a Haskell code block in TeX.

• _texSingleLineCommentStart :: Mode a

  Mark start of a comment that must be single-line in TeX.

• _lhsSingleLineCommentStart :: Mode a

  Mark start of a comment that must be single-line in Literate Haskell.

Instances

Generic (Config a)

Defined in Lima.Converter

Associated Types

type Rep (Config a) :: Type -> Type

Methods

from :: Config a -> Rep (Config a) x

to :: Rep (Config a) x -> Config a

Show (Config Internal)

Defined in Lima.Converter

Methods

showsPrec :: Int -> Config Internal -> ShowS

show :: Config Internal -> String

showList :: [Config Internal] -> ShowS

Show (Config User)

Defined in Lima.Converter

Methods

showsPrec :: Int -> Config User -> ShowS

show :: Config User -> String

showList :: [Config User] -> ShowS

Default (Config Internal)

Defined in Lima.Converter

Methods

def :: Config Internal

Default (Config User)

Defined in Lima.Converter

Methods

def :: Config User

Eq (Config User)

Defined in Lima.Converter

Methods

(==) :: Config User -> Config User -> Bool

(/=) :: Config User -> Config User -> Bool

PrettyPrint (Config User)

Defined in Lima.Converter

Methods

pp :: Config User -> Pretty String

type Rep (Config a)

Defined in Lima.Converter

type Rep (Config a) = D1 ('MetaData "Config" "Lima.Converter" "lima-0.3.0.0-KKExkMMAGbi7eI7DsY4MIe" 'False) (C1 ('MetaCons "Config" 'PrefixI 'True) (((S1 ('MetaSel ('Just "_disable") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 (Mode a)) :*: S1 ('MetaSel ('Just "_enable") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 (Mode a))) :*: (S1 ('MetaSel ('Just "_indent") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 (Mode a)) :*: (S1 ('MetaSel ('Just "_dedent") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 (Mode a)) :*: S1 ('MetaSel ('Just "_mdHaskellCodeStart") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 (Mode a))))) :*: ((S1 ('MetaSel ('Just "_mdHaskellCodeEnd") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 (Mode a)) :*: S1 ('MetaSel ('Just "_texHaskellCodeStart") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 (Mode a))) :*: (S1 ('MetaSel ('Just "_texHaskellCodeEnd") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 (Mode a)) :*: (S1 ('MetaSel ('Just "_texSingleLineCommentStart") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 (Mode a)) :*: S1 ('MetaSel ('Just "_lhsSingleLineCommentStart") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 (Mode a)))))))

def :: Default a => a

The default value for this type.

toConfigInternal :: Config User -> Config Internal

Convert a user Config to an internal Config with user-supplied values.

It's important to do this conversion at a single entrypoint.

Otherwise, repeated conversions will accumulate changes such as appended spaces.

Lenses

disable :: forall a. Lens' (Config a) (Mode a)

enable :: forall a. Lens' (Config a) (Mode a)

indent :: forall a. Lens' (Config a) (Mode a)

dedent :: forall a. Lens' (Config a) (Mode a)

mdHaskellCodeStart :: forall a. Lens' (Config a) (Mode a)

mdHaskellCodeEnd :: forall a. Lens' (Config a) (Mode a)

texHaskellCodeStart :: forall a. Lens' (Config a) (Mode a)

texHaskellCodeEnd :: forall a. Lens' (Config a) (Mode a)

texSingleLineCommentStart :: forall a. Lens' (Config a) (Mode a)

lhsSingleLineCommentStart :: forall a. Lens' (Config a) (Mode a)

microlens

(&) :: a -> (a -> b) -> b infixl 1

& is a reverse application operator. This provides notational convenience. Its precedence is one higher than that of the forward application operator $, which allows & to be nested in $.

>>> 5 & (+1) & show
"6"

Since: base-4.8.0.0

(?~) :: ASetter s t a (Maybe b) -> b -> s -> t infixr 4

(?~) is a version of (.~) that wraps the value into Just before setting.

l ?~ b = l .~ Just b

It can be useful in combination with at:

>>> Map.empty & at 3 ?~ x
fromList [(3,x)]

Format

data Format

A format of a document.

Constructors

Hs

Haskell

Lhs

Literate Haskell

Md

Markdown

TeX

TeX

Instances

Eq Format

Defined in Lima.Converter

Methods

(==) :: Format -> Format -> Bool

(/=) :: Format -> Format -> Bool

convertTo :: Format -> Format -> Config User -> Text -> Text

Compose a function that converts a document in one Format to a document in another Format.

showFormatExtension :: Format -> String

Show a Format as a file extension.

>>> showFormatExtension Lhs
"lhs"

showFormatName :: Format -> String

Show a Format as a full name.

>>> showFormatName Lhs
"Literate Haskell"

Tokens

data Token

Internal representation of a document.

A printer processes Tokens one by one.

A Token can have:

• Action - how this Token affects the subsequent Tokens.
• Target - a type of Tokens that are affected by this Token.
• Range - the nearest Token until which this Token affects the subsequent Tokens.

Constructors

Indent

• Action: set indentation to n.
• Target: HaskellCode.
• Range: Indent, Dedent, or Disabled.

Fields

• n :: Int

Dedent

• Action: set indentation to 0.
• Target: HaskellCode.
• Range: Indent, Dedent, or Disabled.

Disabled

A block that should be invisible when rendered outside of .hs.

• Action: set indentation to 0.
• Target: HaskellCode.
• Range: Indent, Dedent, or Disabled.

Fields

• manyLines :: [Text]

HaskellCode

Lines copied verbatim while a parser was in a Haskell code block.

Fields

• manyLines :: [Text]

Text

Lines copied verbatim while a parser was in a text block.

Fields

• someLines :: NonEmpty Text

Comment

Lines copied verbatim while a parser was in a comment block.

Fields

• someLines :: NonEmpty Text

CommentSingleLine

A line of a comment that must be kept on a single-line.

E.g., {- FOURMOLU_DISABLE -} from a .hs.

Fields

• someLine :: Text

Instances

Data Token

Defined in Lima.Converter

Methods

gfoldl :: (forall d b. Data d => c (d -> b) -> d -> c b) -> (forall g. g -> c g) -> Token -> c Token

gunfold :: (forall b r. Data b => c (b -> r) -> c r) -> (forall r. r -> c r) -> Constr -> c Token

toConstr :: Token -> Constr

dataTypeOf :: Token -> DataType

dataCast1 :: Typeable t => (forall d. Data d => c (t d)) -> Maybe (c Token)

dataCast2 :: Typeable t => (forall d e. (Data d, Data e) => c (t d e)) -> Maybe (c Token)

gmapT :: (forall b. Data b => b -> b) -> Token -> Token

gmapQl :: (r -> r' -> r) -> r -> (forall d. Data d => d -> r') -> Token -> r

gmapQr :: forall r r'. (r' -> r -> r) -> r -> (forall d. Data d => d -> r') -> Token -> r

gmapQ :: (forall d. Data d => d -> u) -> Token -> [u]

gmapQi :: Int -> (forall d. Data d => d -> u) -> Token -> u

gmapM :: Monad m => (forall d. Data d => d -> m d) -> Token -> m Token

gmapMp :: MonadPlus m => (forall d. Data d => d -> m d) -> Token -> m Token

gmapMo :: MonadPlus m => (forall d. Data d => d -> m d) -> Token -> m Token

Show Token

Defined in Lima.Converter

Methods

showsPrec :: Int -> Token -> ShowS

show :: Token -> String

showList :: [Token] -> ShowS

Eq Token

Defined in Lima.Converter

Methods

(==) :: Token -> Token -> Bool

(/=) :: Token -> Token -> Bool

PrettyPrint Tokens

Defined in Lima.Converter

Methods

pp :: Tokens -> Pretty String

type Tokens = [Token]

A list of Tokens.

selectFromTokens :: Config User -> Format -> Tokens -> Text

Select a printer function based on a given format.

selectToTokens :: Config User -> Format -> Text -> Tokens

Select a parser function based on a given format.

mergeTokens :: Tokens -> Tokens

Merge specific consecutive Tokens.

>>> pp exampleNonTexTokens'
[
  Indent {n = 3},
  Disabled {manyLines = ["-- What's the answer?"]},
  Indent {n = 1},
  Indent {n = 2},
  Text {someLines = "- Intermediate results" :| []},
  HaskellCode {manyLines = ["   b = a 4","   a = const 3"]},
  Dedent,
  HaskellCode {manyLines = ["answer = b * 14"]},
  Comment {someLines = "Hello from comments," :| []},
  Comment {someLines = "world!" :| []},
  CommentSingleLine {someLine = "Comment on a single line."},
  Text {someLines = "Hello from text," :| []},
  Text {someLines = "world!" :| []}
]

>>> pp $ mergeTokens exampleNonTexTokens'
[
  Indent {n = 3},
  Disabled {manyLines = ["-- What's the answer?"]},
  Indent {n = 1},
  Indent {n = 2},
  Text {someLines = "- Intermediate results" :| []},
  HaskellCode {manyLines = ["   b = a 4","   a = const 3"]},
  Dedent,
  HaskellCode {manyLines = ["answer = b * 14"]},
  Comment {someLines = "world!" :| ["","Hello from comments,"]},
  CommentSingleLine {someLine = "Comment on a single line."},
  Text {someLines = "world!" :| ["","Hello from text,"]}
]

stripTokens :: Tokens -> Tokens

Strip empty lines and leading spaces in Tokens.

• Remove empty lines in Tokens.
• Shift lines in HaskellCode to the left by the minimal number of leading spaces in nonempty lines.

>>> pp exampleNonTexTokens'
[
  Indent {n = 3},
  Disabled {manyLines = ["-- What's the answer?"]},
  Indent {n = 1},
  Indent {n = 2},
  Text {someLines = "- Intermediate results" :| []},
  HaskellCode {manyLines = ["   b = a 4","   a = const 3"]},
  Dedent,
  HaskellCode {manyLines = ["answer = b * 14"]},
  Comment {someLines = "Hello from comments," :| []},
  Comment {someLines = "world!" :| []},
  CommentSingleLine {someLine = "Comment on a single line."},
  Text {someLines = "Hello from text," :| []},
  Text {someLines = "world!" :| []}
]

>>> pp $ stripTokens exampleNonTexTokens'
[
  Indent {n = 3},
  Disabled {manyLines = ["-- What's the answer?"]},
  Indent {n = 1},
  Indent {n = 2},
  Text {someLines = "- Intermediate results" :| []},
  HaskellCode {manyLines = ["b = a 4","a = const 3"]},
  Dedent,
  HaskellCode {manyLines = ["answer = b * 14"]},
  Comment {someLines = "Hello from comments," :| []},
  Comment {someLines = "world!" :| []},
  CommentSingleLine {someLine = "Comment on a single line."},
  Text {someLines = "Hello from text," :| []},
  Text {someLines = "world!" :| []}
]

normalizeTokens :: Tokens -> Tokens

mergeTokens and stripTokens.

>>> pp $ normalizeTokens exampleNonTexTokens
[
  Indent {n = 3},
  Disabled {manyLines = ["-- What's the answer?"]},
  Indent {n = 1},
  Indent {n = 2},
  Text {someLines = "- Intermediate results" :| []},
  HaskellCode {manyLines = ["b = a 4","a = const 3"]},
  Dedent,
  HaskellCode {manyLines = ["answer = b * 14"]},
  Comment {someLines = "world!" :| ["","Hello from comments,"]},
  CommentSingleLine {someLine = "Comment on a single line."},
  Text {someLines = "world!" :| ["","Hello from text,"]}
]

Printers

hsFromTokens :: Config User -> Tokens -> Text

Convert Tokens to Haskell code.

Rules

• Certain assumptions must hold for inputs.

• These are the relations between Tokens and document blocks when the default Config values are used.

  • Indent ~ '{- LIMA_INDENT N -}' where N is an Int.
  • Dedent ~ '{- LIMA_DEDENT -}'.

  • Disabled ~ '{- LIMA_DISABLE -}' and '{- LIMA_ENABLE -}' and lines between them.

    {- LIMA_DISABLE -}
    
    disabled
    
    {- LIMA_ENABLE -}
    

  • Text ~ a multiline comment starting with '{-\n' and ending with '\n-}'.

    {-
    line 1
    -}
    

    • Consecutive Texts are merged into a single Text.
    • There must be at list one nonempty line inside this comment.

  • CommentSingleLine ~ a multiline comment on a single line.

    {- line -}
    

  • Comment ~ a multiline comment starting with '{- TEXT', where TEXT is nonempty text, and ending with \n-}

    {- line 1
    line 2
    -}
    

    • Consecutive Comments are merged into a single Comment.
  • HaskellCode ~ other lines.

Example

>>> pp $ hsFromTokens def exampleNonTexTokens
{- LIMA_INDENT 3 -}

{- LIMA_DISABLE -}

-- What's the answer?

{- LIMA_ENABLE -}

{- LIMA_INDENT 1 -}

{- LIMA_INDENT 2 -}

{-
- Intermediate results
-}

a = const 3
b = a 4

{- LIMA_DEDENT -}

answer = b * 14

{- Hello from comments,

world!
-}

{- Comment on a single line. -}

{-
Hello from text,

world!
-}

hsFromTokens' :: Config User -> Tokens -> [Text]

Convert Tokens to Haskell code.

Each Token becomes a Text in a list.

These Texts are concatenated in hsFromTokens.

lhsFromTokens :: Config User -> Tokens -> Text

Convert Tokens to Literate Haskell code.

Rules

• Certain assumptions must hold for inputs.

• These are the relations between document blocks and tokens when the default Config values are used.

  • Indent ~ '% LIMA_INDENT N' (N is an Int).
  • Dedent ~ '% LIMA_DEDENT'.

  • Disabled ~ Lines between and including '% LIMA_DISABLE' and '% LIMA_ENABLE'.

    • There must be at least one nonempty line between these tags.

  • CommentSingleLine ~ a line starting with '% SINGLE_LINE '.

    % SINGLE_LINE line
    

  • Comment ~ consecutive lines, either empty or starting with '% '.

    % Hello,
    % world!
    
    % Hello,
    % user!
    

    • At least one line must have nonempty text after '% '

  • HaskellCode ~ consecutive lines starting with '> '.

    > a4 = 4
    > a2 = 2
    

    • Inside a Token, code is shifted to the left. See normalizeTokens.
    • During printing, code is indented according to previous Tokens.
  • Text ~ other lines.

Example

>>> pp $ lhsFromTokens def exampleNonTexTokens
% LIMA_INDENT 3

% LIMA_DISABLE

% -- What's the answer?

% LIMA_ENABLE

% LIMA_INDENT 1

% LIMA_INDENT 2

- Intermediate results

>   a = const 3
>   b = a 4

% LIMA_DEDENT

> answer = b * 14

% Hello from comments,

% world!

% SINGLE_LINE Comment on a single line.

Hello from text,

world!

lhsFromTokens' :: Config User -> Tokens -> [Text]

Convert Tokens to Literate Haskell code.

Each Token becomes a Text in a list.

These Texts are concatenated in lhsFromTokens.

mdFromTokens :: Config User -> Tokens -> Text

Convert Tokens to Markdown code.

Rules

• Certain assumptions must hold for inputs.

• These are the relations between document blocks and tokens when the default Config values are used.

  • Indent ~ '<!-- LIMA_INDENT N -->', where N is an Int.
  • Dedent ~ '<!-- LIMA_DEDENT -->'.

  • Disabled ~ a multiline comment starting with '<!-- LIMA_DISABLE\n' and ending with '\nLIMA_ENABLE -->'.

    <!-- LIMA_DISABLE
    a4 = 4
    a2 = 2
    LIMA_ENABLE -->
    

  • CommentSingleLine ~ a line starting with '<!-- ' and ending with ' -->'.

    line --
    

  • Comment ~ a multiline comment starting with '<!-- {text}', where {text} is nonempty text.

    <!-- line 1
    line 2
    -->
    

    • Consecutive Comments are merged into a single Comment.

  • HaskellCode ~ possibly indented block starting with '```haskell' and ending with '```'.

      ```haskell
        a4 = 2
      ```
    

  • Text ~ other lines.

Example

>>> pp $ mdFromTokens def exampleNonTexTokens
   <!-- LIMA_INDENT 3 -->

<!-- LIMA_DISABLE

-- What's the answer?

LIMA_ENABLE -->

 <!-- LIMA_INDENT 1 -->

  <!-- LIMA_INDENT 2 -->

- Intermediate results

  ```haskell
  a = const 3
  b = a 4
  ```

<!-- LIMA_DEDENT -->

```haskell
answer = b * 14
```

<!-- Hello from comments,

world!
-->

<!-- Comment on a single line. -->

Hello from text,

world!

mdFromTokens' :: Config User -> Tokens -> [Text]

Convert Tokens to Haskell code.

Each Token becomes a Text in a list.

These Texts are concatenated in mdFromTokens.

texFromTokens :: Config User -> Tokens -> Text

Convert Tokens to TeX code.

Rules

• Certain assumptions must hold for inputs.

• These are the relations between tokens and document blocks when the default Config values are used.

  • Indent ~ '% LIMA_INDENT N' (N is an Int).
  • Dedent ~ '% LIMA_DEDENT'.
  • Disabled ~ '% LIMA_DISABLE' and '% LIMA_ENABLE' and lines between them.

  • CommentSingleLine ~ a line starting with '% SINGLE_LINE '.

    % SINGLE_LINE line
    

  • Comment ~ consecutive lines, either empty or starting with '% '.

    % Hello,
    % world!
    
    % Hello,
    % user!
    

    • At least one line must have nonempty text after '% '.

  • HaskellCode ~ lines between possibly indented tags '\begin{code}' and '\end{code}'.

    • Inside a Token, code will be shifted to the left. See normalizeTokens.
    • When printing the Tokens, code will be indented according to previous Tokens.
  • Text ~ other lines.

Example

>>> pp $ texFromTokens def exampleTexTokens
% LIMA_INDENT 3

% LIMA_DISABLE

% -- What's the answer?

% LIMA_ENABLE

% LIMA_INDENT 1

% LIMA_INDENT 0

Intermediate results

\begin{mycode}
a = const 3
b = a 4
\end{mycode}

% LIMA_DEDENT

\begin{mycode}
answer = b * 14
\end{mycode}

% Hello from comments,

% world!

% SINGLE_LINE Comment on a single line.

texFromTokens' :: Config User -> Tokens -> [Text]

Convert Tokens to TeX code.

Each Token becomes a Text in a list.

These Texts are concatenated in texFromTokens.

Parsers

hsToTokens :: Config User -> Text -> Tokens

Convert Tokens to Haskell code.

Inverse of hsFromTokens.

>>> (hsToTokens def $ hsFromTokens def exampleNonTexTokens) == exampleNonTexTokens
True

lhsToTokens :: Config User -> Text -> Tokens

Convert Tokens to Markdown code.

Inverse of lhsFromTokens.

>>> (lhsToTokens def $ lhsFromTokens def exampleNonTexTokens) == exampleNonTexTokens
True

mdToTokens :: Config User -> Text -> Tokens

Convert Tokens to Markdown code.

Inverse of mdFromTokens.

>>> (mdToTokens def $ mdFromTokens def exampleNonTexTokens) == exampleNonTexTokens
True

texToTokens :: Config User -> Text -> Tokens

Convert Tokens to TeX code.

Inverse of texFromTokens.

>>> (texToTokens def $ texFromTokens def exampleTexTokens) == exampleTexTokens
True

Helpers

mkFromTokens :: (Config User -> Tokens -> [Text]) -> Config User -> Tokens -> Text

Compose a function from Tokens to a Text.

mkToTokens :: (State -> [(Int, Text)] -> [Token] -> [Token]) -> Text -> Tokens

Compose a function from a Text to Tokens.

parseLineToToken :: Config Internal -> Format -> Token -> Text -> Int -> Tokens

Parse a single line to a token.

• Merge comments

errorExpectedToken :: Int -> Token -> Token -> a

Show error with line number for a token.

errorNotEnoughTokens :: Format -> a

pp :: PrettyPrint a => a -> Pretty String

A printing function

It's not meant to be used outside of this library.

Examples

exampleNonTexTokens' :: Tokens

Example non-TeX Tokens. See exampleTexTokens.

When printed to a TeX document, these Tokens can't be correctly parsed. This is because they don't have necessary tags surrounding Haskell code blocks.

>>> pp $ exampleNonTexTokens'
[
  Indent {n = 3},
  Disabled {manyLines = ["-- What's the answer?"]},
  Indent {n = 1},
  Indent {n = 2},
  Text {someLines = "- Intermediate results" :| []},
  HaskellCode {manyLines = ["   b = a 4","   a = const 3"]},
  Dedent,
  HaskellCode {manyLines = ["answer = b * 14"]},
  Comment {someLines = "Hello from comments," :| []},
  Comment {someLines = "world!" :| []},
  CommentSingleLine {someLine = "Comment on a single line."},
  Text {someLines = "Hello from text," :| []},
  Text {someLines = "world!" :| []}
]

exampleNonTexTokens :: Tokens

Normalized exampleNonTexTokens'.

>>> pp $ exampleNonTexTokens
[
  Indent {n = 3},
  Disabled {manyLines = ["-- What's the answer?"]},
  Indent {n = 1},
  Indent {n = 2},
  Text {someLines = "- Intermediate results" :| []},
  HaskellCode {manyLines = ["b = a 4","a = const 3"]},
  Dedent,
  HaskellCode {manyLines = ["answer = b * 14"]},
  Comment {someLines = "world!" :| ["","Hello from comments,"]},
  CommentSingleLine {someLine = "Comment on a single line."},
  Text {someLines = "world!" :| ["","Hello from text,"]}
]

exampleTexTokens :: Tokens

same as exampleNonTexTokens, but with TeX-specific tags that make Haskell code blocks correctly parsable.

>>> pp $ exampleTexTokens
[
  Indent {n = 3},
  Disabled {manyLines = ["-- What's the answer?"]},
  Indent {n = 1},
  Indent {n = 0},
  Text {someLines = "\\begin{mycode}" :| ["","Intermediate results"]},
  HaskellCode {manyLines = ["b = a 4","a = const 3"]},
  Text {someLines = "\\end{mycode}" :| []},
  Dedent,
  Text {someLines = "\\begin{mycode}" :| []},
  HaskellCode {manyLines = ["answer = b * 14"]},
  Text {someLines = "\\end{mycode}" :| []},
  Comment {someLines = "world!" :| ["","Hello from comments,"]},
  CommentSingleLine {someLine = "Comment on a single line."}
]