# co-log-json

`co-log-json` allows writing structured logs in your application.

The library allows adding additional machine-readable context to the log
messages. I.e. each log message is a JSON object with a predefined structure
where additional user context can be added. Such logs can be easily indexed
by external logs systems such as [Graylog](https://www.graylog.org) or 
[Elastic search](https://www.elastic.co). The message logging is done using
a special context, that keeps the information about additional attributes.
Such contexts form a nested scope, and all messages in the score inherit provided
context:

```
   scope [ user = "Joe"] 
     |
     | ==>  {"message":"user arrived", "data": { "user": "Joe"}
     |
     +------ scope [ tag = "human" ]
              |
              | ==> {"message": "user entered the door", "data": { "user": "Joe", "tag": "human"}
```


In other words, this library takes two choices:

  1. We write full context with each message.
  This approach allows using external tools (like elastic-search) that can index message
  without any additional work and information.
  Any message can be lost without affecting the ability to decode other messages. An alternative
  approach could be emitting a list of events (possibly as a bytecode), such an approach
  is taken by the tracing libraries. It requires more works during reading and indexing
  logs, and in the case in some logs are logs may render later ones unusable or lose some info.

  2. We keep a state context so we don't need to attach context messages to each one.
  An alternative approach is the extraction of the structure information from the message itself.
  This approach is taken in some structured logging libraries, it can provide better error
  messages. However, it requires to think about the message context all the time (and developer 
  user may not even know all interesting context where the message will be used).


# Using a library

To to use a library you'll need to add `co-log-json` to the dependency list.

```
library.cabal

library
  build-depends:
    base,
    co-log-json ^>= 0.0
```


## Setting the library


```haskell
import Colog.Json                       -- Core API for structured logging
import Colog.Json.Action (logToHandle)  -- Actions to store the message
import System.IO (stderr)

main :: IO ()
main = do
  -- First, we need to set up log storing function:
  --   To emit logs we need to create a context.
  --    Context takes the action to store log and it's possible to
  --    attach additional information to it.

  let context = mkLogger (logToHandle stderr)
  --                          ^
  --                          |
  --                          +------  `LogAction IO Message` 
  --                                    (see discussion "Why IO below")
  --
  -- Once 'context' is created it can be passed to the other parts
  -- of the code or used to emit logs.

  logDebug context "Hi, there!"
  --         ^         ^
  --         |         |
  --         |         +----- `LogStr` type - is an efficient text builder.
  --         |                 
  --         |
  --         +-- log context

  -- Will output:
  --
  -- ```
  -- {"severity":"DEBUG", "thread": 1, "message": "Hi, there!"}
  -- ```
  --                ^               ^                ^
  --                |               |                +------------ Message itself
  --                |               +----------------------------- Id of the thread that emitted the message
  --                +----------------- Severity level: Debug, Info, Notice, Error, Critical, Alert, Emergency
  -- 
  -- to the stderr.

  -- There are other helper functions list `logInfo`, `logNotice`, etc.
```

Now let's discuss `LogStr` type. It exists to efficiently generate message without
extra allocations that can be avoided. It's just a [Text.Builder](http://hackage.haskell.org/package/text-1.2.4.0/docs/Data-Text-Lazy-Builder.html) though internal
representation may evolve in the future. `LogStr` allows message concatenation without allocating
intermediate structures, so it allows efficient log building. For example line:

```haskell
  logDebug context $ "a" <> "b" <> "c"
```

will write all text string "a", "b", "c" directly to the buffer, and can do more optimizations.
So it's quite safe to use the library even with a large number of logs.

The 'LogStr' type implement [IsString](http://hackage.haskell.org/package/base-4.14.0.0/docs/Data-String.html#t:IsString) instance it means that you can just write string literals
and they will be treated as 'LogStr'. To concatenate two 'LogStr' you can use [`<>`](https://hackage.haskell.org/package/base-4.14.0.0/docs/Data-Semigroup.html#t:Semigroup) operation.

In addition you can convert any string lines that has [StringConv a T.Text](https://hackage.haskell.org/package/string-conv-0.1.2/docs/Data-String-Conv.html), i.e. can be converted
to LogStr using `ls :: StringConv a T.Text => a -> LogStr`.
Efficiency and safety of this conversion depends on the concrete instance implementation and is out
of control of the package.

```haskell
   logDebug context $ "foo is " <> ls ("ы":: ByteString)
```

In case your data structure does not implement `StringConv a` it's possible to use not efficient but
very general `showLS :: Show a => a -> LogStr` method that will work for any type that has Show instance.

## Adding context.

But just writing JSON messages is not very interesting, we want to control the context of the message.
In order to add user data to the context we can use `addContext :: PushContext -> LoggerEnv -> LoggerEnv`
method.

```haskell

   let context1 = addContext (sl "user_id" (1::Int)) context
   --    ^                    ^     ^         ^        ^
   --    |                    |     |         |        +--- old context
   --    |                    |     |         +------------ user data 
   --    |                    |     +---------------------- key for the user data
   --    |                    +---------------------------- function that generates an entry
   --    +------------------------------------------------- new context that has data attached

   logDebug context1 "Hi again!"

   -- Will emit to the stderr:
   --
   -- ```
   -- {"severity":"DEBUG", "thread": 1, "message": "Hi, there!", "data": {"user_id":1}}
   -- ```
   --                                                              ^           ^    ^
   --                                                              |           |    +-- value
   --                                                              |           +------- key
   --                                                              +--- all user data is kept under 'data' key
```

Sidenote: 'sl' function looks redundant, but it's needed in order to provide a future
compatibility for a case when internal structure of user data is changing. Historically
it came from [katip](https://hackage.haskell.org/package/katip) library, there are more functions that allows context creation but

Function `sl :: ToJSON a => T.Text -> a -> PushContext` prepares update to the context. 
It encodes user data to JSON. This is why we had to add type annotation to `1` otherwise GHC had
no means to infer the type).

It's important that the library does not perform any compaction of key-values, i.e.:

```haskell
   logDebug (addContext (sl "user_id" 2) context1) "Who am I?"
   -- Will emit:
   -- ```
   -- {"severity":"DEBUG", "thread": 1, "message": "Hi, there!", "data": {"user_id":1, "user_id": 2}}
   -- ```
```

This is done for purpose, but may be a subject to change in the future major versions.

`addContext` is not the only function that allows to modify context, there is the other one
`addNamespace :: Text -> LoggerEnv -> LoggerEnv`, this function adds an entry about the 
current namespace. Namespace is a nested list of strings that tells the logical part of 
the codebase the log belongs to. It's kept separate fom the data, and it's possible to
use in library filtering (using  [cfilter](https://kowainik.github.io/posts/2018-09-25-co-log#cfilter)) or filtering in external system.


The described functionality is enough to perform logging. However, it may worth discussing 
interoperability with the rest of co-log ecosystem and ergonomics.

## co-log interoperability

The co-log ecosystem works with "LogAction m a" and in the package, we use "LogEnv" it means
that we are losing most of the benefits of the library and can't use a lot of utility
functions. To improve the situation it's possible to convert LogEnv into
`LogAction m (Severity,LogStr)` using function `unLogger`. Then we will have log action
that will emit a message with the current context, but the context will no longer be modifiable.

# Ergonomics

The package provides no ergonomic tools by default and it's important that `addContext` is a pure
function. On the one han,d it introduces a log of boilerplate code but on the other.

It allows to modify context even outside of the effectful computation for example with servant you may have:

```haskell
-- | Catalogue of items 
data CatalogueApi route = CatalogueApi
  { _the_catalogue :: route :- Capture "catalogue_id" CatalogueId :> ToServantAPI Bar
  , ...
  } 

-- Item
data ItemApi route = ItemApi
  { _items :: route :- "items" :> Get '[JSON] (Vector Item)
  , ...
  }

handleCatalogue :: LoggerEnv -> ToServant CatalogueApi AsServer
handleCatalogue ctx' = CatalogueApi
  { _the_catalogue = \catalogue_id ->
     handleItem (addContext "catalogue_id" catalogue_id) ctx) catalogue_id
  , ...
  }
  where 
    ctx = addNamespace "catalogue" ctx'

handleItem :: LoggerEnv -> CatalogueId -> ToServant ItemApi AsServer
handleItem ctx' catalogue_id = ItemApi ...
  where ctx = addNamespace "item" ctx'
```

So we can build computation that we will execute later while modifying the context.

On the other hand, we can always wrap this pure function into an effect system 
of choice be it either handle pattern or readert or MTL or effect system.
This library does not target any particular solution and  want to introduce 
neither additional dependencies nor additional restrictions on the user.