Haxl.Core

Everything needed to define data sources and to invoke the engine.

The monad and operations

newtype GenHaxl u w a

The Haxl monad, which does several things:

• It is a reader monad for Env, which contains the current state of the scheduler, including unfetched requests and the run queue of computations.
• It is a writer monad for WriteTree. These can be used to do arbitrary "logs" from any Haxl computation. These are better than doing arbitrary IO from a Haxl computation as these writes also get memoized if the Haxl computation associated with them is memoized. Now if this memoized computation is run again, you'll get the writes twice.

Constructors

GenHaxl

Fields

• unHaxl :: Env u w -> IO (Result u w a)

Instances

Monad (GenHaxl u w)

Defined in Haxl.Core.Monad

Methods

(>>=) :: GenHaxl u w a -> (a -> GenHaxl u w b) -> GenHaxl u w b

(>>) :: GenHaxl u w a -> GenHaxl u w b -> GenHaxl u w b

return :: a -> GenHaxl u w a

fail :: String -> GenHaxl u w a

Functor (GenHaxl u w)

Defined in Haxl.Core.Monad

Methods

fmap :: (a -> b) -> GenHaxl u w a -> GenHaxl u w b

(<$) :: a -> GenHaxl u w b -> GenHaxl u w a

Applicative (GenHaxl u w)

Defined in Haxl.Core.Monad

Methods

pure :: a -> GenHaxl u w a

(<*>) :: GenHaxl u w (a -> b) -> GenHaxl u w a -> GenHaxl u w b

liftA2 :: (a -> b -> c) -> GenHaxl u w a -> GenHaxl u w b -> GenHaxl u w c

(*>) :: GenHaxl u w a -> GenHaxl u w b -> GenHaxl u w b

(<*) :: GenHaxl u w a -> GenHaxl u w b -> GenHaxl u w a

MonadThrow (GenHaxl u w)

Since: 0.3.1.0

Defined in Haxl.Core.Monad

Methods

throwM :: Exception e => e -> GenHaxl u w a

MonadCatch (GenHaxl u w)

Since: 0.3.1.0

Defined in Haxl.Core.Monad

Methods

catch :: Exception e => GenHaxl u w a -> (e -> GenHaxl u w a) -> GenHaxl u w a

Fractional a => Fractional (GenHaxl u w a)

Defined in Haxl.Prelude

Methods

(/) :: GenHaxl u w a -> GenHaxl u w a -> GenHaxl u w a

recip :: GenHaxl u w a -> GenHaxl u w a

fromRational :: Rational -> GenHaxl u w a

Num a => Num (GenHaxl u w a)

Defined in Haxl.Prelude

Methods

(+) :: GenHaxl u w a -> GenHaxl u w a -> GenHaxl u w a

(-) :: GenHaxl u w a -> GenHaxl u w a -> GenHaxl u w a

(*) :: GenHaxl u w a -> GenHaxl u w a -> GenHaxl u w a

negate :: GenHaxl u w a -> GenHaxl u w a

abs :: GenHaxl u w a -> GenHaxl u w a

signum :: GenHaxl u w a -> GenHaxl u w a

fromInteger :: Integer -> GenHaxl u w a

IsString a => IsString (GenHaxl u w a)

Defined in Haxl.Core.Monad

Methods

fromString :: String -> GenHaxl u w a

Semigroup a => Semigroup (GenHaxl u w a)

Defined in Haxl.Core.Monad

Methods

(<>) :: GenHaxl u w a -> GenHaxl u w a -> GenHaxl u w a

sconcat :: NonEmpty (GenHaxl u w a) -> GenHaxl u w a

stimes :: Integral b => b -> GenHaxl u w a -> GenHaxl u w a

Monoid a => Monoid (GenHaxl u w a)

Defined in Haxl.Core.Monad

Methods

mempty :: GenHaxl u w a

mappend :: GenHaxl u w a -> GenHaxl u w a -> GenHaxl u w a

mconcat :: [GenHaxl u w a] -> GenHaxl u w a

u1 ~ u2 => IfThenElse (GenHaxl u1 w Bool) (GenHaxl u2 w a)

Defined in Haxl.Prelude

Methods

ifThenElse :: GenHaxl u1 w Bool -> GenHaxl u2 w a -> GenHaxl u2 w a -> GenHaxl u2 w a

runHaxl :: forall u w a. Env u w -> GenHaxl u w a -> IO a

Runs a Haxl computation in the given Env.

Note: to make multiple concurrent calls to runHaxl, each one must have a separate Env. A single Env must not be shared between multiple concurrent calls to runHaxl, otherwise deadlocks or worse will likely ensue.

However, multiple Envs may share a single StateStore, and thereby use the same set of datasources.

runHaxlWithWrites :: forall u w a. Env u w -> GenHaxl u w a -> IO (a, [w])

Env

data Env u w

Constructors

Env

Fields

• dataCache :: !(DataCache (DataCacheItem u w))

  cached data fetches

• memoCache :: !(DataCache (DataCacheItem u w))

  memoized computations

• memoKey :: !CallId

  current running memo key

• flags :: !Flags
• userEnv :: u

  user-supplied data, retrievable with env

• statsRef :: !(IORef Stats)

  statistics, collected according to the report level in flags.

• statsBatchIdRef :: !(IORef Int)

  keeps track of a Unique ID for each batch dispatched with stats enabled, for aggregating after.

• callIdRef :: !(IORef CallId)

  keeps track of a Unique ID for each fetch/memo.

• profCurrent :: ProfileCurrent

  current profiling label, see withLabel

• profRef :: !(IORef Profile)

  profiling data, collected according to the report level in flags.

• states :: StateStore

  Data sources and other components can store their state in here. Items in this store must be instances of StateKey.

• reqStoreRef :: !(IORef (RequestStore u))

  The set of requests that we have not submitted to data sources yet. Owned by the scheduler.

• runQueueRef :: !(IORef (JobList u w))

  runnable computations. Things get added to here when we wake up a computation that was waiting for something. When the list is empty, either we're finished, or we're waiting for some data fetch to return.

• submittedReqsRef :: !(IORef ReqCountMap)

  all outgone fetches which haven't yet returned. Entries are removed from this map as the fetches finish. This field is useful for tracking outgone fetches to detect downstream failures.

• completions :: !(TVar [CompleteReq u w])

  Requests that have completed. Modified by data sources (via putResult) and the scheduler. Waiting for this list to become non-empty is how the scheduler blocks waiting for data fetches to return.

• writeLogsRef :: !(IORef (WriteTree w))

  A log of all writes done as part of this haxl computation. Any haxl computation that needs to be memoized runs in its own environment so that we can get a hold of those writes and put them in the IVar associated with the compuatation.

• writeLogsRefNoMemo :: !(IORef (WriteTree w))

  This is just a specialized version of writeLogsRef, where we put logs that user doesn't want memoized. This is a better alternative to doing arbitrary IO from a (memoized) Haxl computation.

type Caches u w = (DataCache (DataCacheItem u w), DataCache (DataCacheItem u w))

caches :: Env u w -> Caches u w

Operations in the monad

env :: (Env u w -> a) -> GenHaxl u w a

Extracts data from the Env.

withEnv :: Env u w -> GenHaxl u w a -> GenHaxl u w a

Returns a version of the Haxl computation which always uses the provided Env, ignoring the one specified by runHaxl.

withLabel :: ProfileLabel -> GenHaxl u w a -> GenHaxl u w a

Label a computation so profiling data is attributed to the label.

Building the Env

initEnvWithData :: StateStore -> u -> Caches u w -> IO (Env u w)

Initialize an environment with a StateStore, an input map, a preexisting DataCache, and a seed for the random number generator.

initEnv :: StateStore -> u -> IO (Env u w)

Initializes an environment with StateStore and an input map.

emptyEnv :: u -> IO (Env u w)

A new, empty environment.

sanitizeEnv :: Env u w -> IO (Env u w)

If you're using the env from a failed Haxl computation in a second Haxl computation, it is recommended to sanitize the Env to remove all empty IVars - especially if it's possible the first Haxl computation could've been interrupted via an async exception. This is because if the Haxl computation was interrupted by an exception, it's possible that there are entries in the cache which are still blocked, while the results from outgone fetches have been discarded.

Building the StateStore

data StateStore

The StateStore maps a StateKey to the State for that type.

Instances

Semigroup StateStore

Defined in Haxl.Core.StateStore

Methods

(<>) :: StateStore -> StateStore -> StateStore

sconcat :: NonEmpty StateStore -> StateStore

stimes :: Integral b => b -> StateStore -> StateStore

Monoid StateStore

Defined in Haxl.Core.StateStore

Methods

mempty :: StateStore

mappend :: StateStore -> StateStore -> StateStore

mconcat :: [StateStore] -> StateStore

stateGet :: forall r. StateKey r => StateStore -> Maybe (State r)

Retrieves a State from the StateStore container.

stateSet :: forall f. StateKey f => State f -> StateStore -> StateStore

Inserts a State in the StateStore container.

stateEmpty :: StateStore

A StateStore with no entries.

Writes inside the monad

tellWrite :: w -> GenHaxl u w ()

tellWriteNoMemo :: w -> GenHaxl u w ()

Exceptions

throw :: Exception e => e -> GenHaxl u w a

Throw an exception in the Haxl monad

catch :: Exception e => GenHaxl u w a -> (e -> GenHaxl u w a) -> GenHaxl u w a

Catch an exception in the Haxl monad

catchIf :: Exception e => (e -> Bool) -> GenHaxl u w a -> (e -> GenHaxl u w a) -> GenHaxl u w a

Catch exceptions that satisfy a predicate

try :: Exception e => GenHaxl u w a -> GenHaxl u w (Either e a)

Returns Left e if the computation throws an exception e, or Right a if it returns a result a.

tryToHaxlException :: GenHaxl u w a -> GenHaxl u w (Either HaxlException a)

Like try, but lifts all exceptions into the HaxlException hierarchy. Uses unsafeToHaxlException internally. Typically this is used at the top level of a Haxl computation, to ensure that all exceptions are caught.

Data fetching and caching

dataFetch :: (DataSource u r, Request r a) => r a -> GenHaxl u w a

Performs actual fetching of data for a Request from a DataSource.

uncachedRequest :: forall a u w (r :: * -> *). (DataSource u r, Request r a) => r a -> GenHaxl u w a

A data request that is not cached. This is not what you want for normal read requests, because then multiple identical requests may return different results, and this invalidates some of the properties that we expect Haxl computations to respect: that data fetches can be arbitrarily reordered, and identical requests can be commoned up, for example.

uncachedRequest is useful for performing writes, provided those are done in a safe way - that is, not mixed with reads that might conflict in the same Haxl computation.

if we are recording or running a test, we fallback to using dataFetch This allows us to store the request in the cache when recording, which allows a transparent run afterwards. Without this, the test would try to call the datasource during testing and that would be an exception.

cacheRequest :: Request req a => req a -> Either SomeException a -> GenHaxl u w ()

Inserts a request/result pair into the cache. Throws an exception if the request has already been issued, either via dataFetch or cacheRequest.

This can be used to pre-populate the cache when running tests, to avoid going to the actual data source and ensure that results are deterministic.

dupableCacheRequest :: Request req a => req a -> Either SomeException a -> GenHaxl u w ()

Similar to cacheRequest but doesn't throw an exception if the key already exists in the cache. If this function is called twice to cache the same Haxl request, the first value will be discarded and overwritten with the second value. Useful e.g. for unit tests

cacheResult :: Request r a => r a -> IO a -> GenHaxl u w a

Transparently provides caching. Useful for datasources that can return immediately, but also caches values. Exceptions thrown by the IO operation (except for asynchronous exceptions) are propagated into the Haxl monad and can be caught by catch and try.

cacheResultWithShow :: (Eq (r a), Hashable (r a), Typeable (r a)) => ShowReq r a -> r a -> IO a -> GenHaxl u w a

Transparently provides caching in the same way as cacheResult, but uses the given functions to show requests and their results.

cachedComputation :: forall req u w a. (Eq (req a), Hashable (req a), Typeable (req a)) => req a -> GenHaxl u w a -> GenHaxl u w a

cachedComputation memoizes a Haxl computation. The key is a request.

Note: These cached computations will not be included in the output of dumpCacheAsHaskell.

preCacheComputation :: forall req u w a. (Eq (req a), Hashable (req a), Typeable (req a)) => req a -> GenHaxl u w a -> GenHaxl u w a

Like cachedComputation, but fails if the cache is already populated.

Memoization can be (ab)used to "mock" a cached computation, by pre-populating the cache with an alternative implementation. In that case we don't want the operation to populate the cache to silently succeed if the cache is already populated.

dumpCacheAsHaskell :: GenHaxl u w String

Dump the contents of the cache as Haskell code that, when compiled and run, will recreate the same cache contents. For example, the generated code looks something like this:

loadCache :: GenHaxl u w ()
loadCache = do
  cacheRequest (ListWombats 3) (Right ([1,2,3]))
  cacheRequest (CountAardvarks "abcabc") (Right (2))

Memoization

newMemo :: GenHaxl u w (MemoVar u w a)

Create a new MemoVar for storing a memoized computation. The created MemoVar is initially empty, not tied to any specific computation. Running this memo (with runMemo) without preparing it first (with prepareMemo) will result in an exception.

newMemoWith :: GenHaxl u w a -> GenHaxl u w (MemoVar u w a)

Convenience function, combines newMemo and prepareMemo.

prepareMemo :: MemoVar u w a -> GenHaxl u w a -> GenHaxl u w ()

Store a computation within a supplied MemoVar. Any memo stored within the MemoVar already (regardless of completion) will be discarded, in favor of the supplied computation. A MemoVar must be prepared before it is run.

runMemo :: MemoVar u w a -> GenHaxl u w a

Continue the memoized computation within a given MemoVar. Notes:

1. If the memo contains a complete result, return that result.
2. If the memo contains an in-progress computation, continue it as far as possible for this round.
3. If the memo is empty (it was not prepared), throw an error.

For example, to memoize the computation one given by:

one :: Haxl Int
one = return 1

use:

do
  oneMemo <- newMemoWith one
  let memoizedOne = runMemo aMemo one
  oneResult <- memoizedOne

To memoize mutually dependent computations such as in:

h :: Haxl Int
h = do
  a <- f
  b <- g
  return (a + b)
 where
  f = return 42
  g = succ <$> f

without needing to reorder them, use:

h :: Haxl Int
h = do
  fMemoRef <- newMemo
  gMemoRef <- newMemo

  let f = runMemo fMemoRef
      g = runMemo gMemoRef

  prepareMemo fMemoRef $ return 42
  prepareMemo gMemoRef $ succ <$> f

  a <- f
  b <- g
  return (a + b)

memo :: (Typeable a, Typeable k, Hashable k, Eq k) => k -> GenHaxl u w a -> GenHaxl u w a

Memoize a computation using an arbitrary key. The result will be calculated once; the second and subsequent time it will be returned immediately. It is the caller's responsibility to ensure that for every two calls memo key haxl, if they have the same key then they compute the same result.

memoUnique :: (Typeable a, Typeable k, Hashable k, Eq k) => MemoFingerprintKey a -> Text -> k -> GenHaxl u w a -> GenHaxl u w a

Memoize a computation using its location and a Fingerprint. This ensures uniqueness across computations.

memoize :: GenHaxl u w a -> GenHaxl u w (GenHaxl u w a)

Transform a Haxl computation into a memoized version of itself.

Given a Haxl computation, memoize creates a version which stores its result in a MemoVar (which memoize creates), and returns the stored result on subsequent invocations. This permits the creation of local memos, whose lifetimes are scoped to the current function, rather than the entire request.

memoize1 :: (Eq a, Hashable a) => (a -> GenHaxl u w b) -> GenHaxl u w (a -> GenHaxl u w b)

Transform a 1-argument function returning a Haxl computation into a memoized version of itself.

Given a function f of type a -> GenHaxl u w b, memoize1 creates a version which memoizes the results of f in a table keyed by its argument, and returns stored results on subsequent invocations with the same argument.

e.g.:

allFriends :: [Int] -> GenHaxl u w [Int]
allFriends ids = do
  memoizedFriendsOf <- memoize1 friendsOf
  concat <$> mapM memoizeFriendsOf ids

The above implementation will not invoke the underlying friendsOf repeatedly for duplicate values in ids.

memoize2 :: (Eq a, Hashable a, Eq b, Hashable b) => (a -> b -> GenHaxl u w c) -> GenHaxl u w (a -> b -> GenHaxl u w c)

Transform a 2-argument function returning a Haxl computation, into a memoized version of itself.

The 2-ary version of memoize1, see its documentation for details.

memoFingerprint :: Typeable a => MemoFingerprintKey a -> GenHaxl u w a -> GenHaxl u w a

data MemoFingerprintKey a where

A memo key derived from a 128-bit MD5 hash. Do not use this directly, it is for use by automatically-generated memoization.

Constructors

MemoFingerprintKey :: !Word64 -> !Word64 -> Addr# -> Addr# -> MemoFingerprintKey a

Instances

Eq (MemoFingerprintKey a)

Defined in Haxl.Core.Memo

Methods

(==) :: MemoFingerprintKey a -> MemoFingerprintKey a -> Bool

(/=) :: MemoFingerprintKey a -> MemoFingerprintKey a -> Bool

Hashable (MemoFingerprintKey a)

Defined in Haxl.Core.Memo

Methods

hashWithSalt :: Int -> MemoFingerprintKey a -> Int

hash :: MemoFingerprintKey a -> Int

Conditionals

pAnd :: GenHaxl u w Bool -> GenHaxl u w Bool -> GenHaxl u w Bool infixr 5

Parallel version of '(.&&)'. Both arguments are evaluated in parallel, and if either returns False then the other is not evaluated any further.

WARNING: exceptions may be unpredictable when using pAnd. If one argument returns False before the other completes, then pAnd returns False immediately, ignoring a possible exception that the other argument may have produced if it had been allowed to complete.

pOr :: GenHaxl u w Bool -> GenHaxl u w Bool -> GenHaxl u w Bool infixr 4

Parallel version of '(.||)'. Both arguments are evaluated in parallel, and if either returns True then the other is not evaluated any further.

WARNING: exceptions may be unpredictable when using pOr. If one argument returns True before the other completes, then pOr returns True immediately, ignoring a possible exception that the other argument may have produced if it had been allowed to complete.

unsafeChooseFirst :: GenHaxl u w a -> GenHaxl u w b -> GenHaxl u w (Either a b)

This function takes two haxl computations as input, and returns the output of whichever computation finished first. This is clearly non-deterministic in its output and exception behavior, be careful when using it.

Statistics

newtype Stats

Stats that we collect along the way.

Constructors

Stats [FetchStats]

Instances

Show Stats

Defined in Haxl.Core.Stats

Methods

showsPrec :: Int -> Stats -> ShowS

show :: Stats -> String

showList :: [Stats] -> ShowS

Semigroup Stats

Defined in Haxl.Core.Stats

Methods

(<>) :: Stats -> Stats -> Stats

sconcat :: NonEmpty Stats -> Stats

stimes :: Integral b => b -> Stats -> Stats

Monoid Stats

Defined in Haxl.Core.Stats

Methods

mempty :: Stats

mappend :: Stats -> Stats -> Stats

mconcat :: [Stats] -> Stats

ToJSON Stats

Defined in Haxl.Core.Stats

Methods

toJSON :: Stats -> Value

toEncoding :: Stats -> Encoding

toJSONList :: [Stats] -> Value

toEncodingList :: [Stats] -> Encoding

data FetchStats

Maps data source name to the number of requests made in that round. The map only contains entries for sources that made requests in that round.

Constructors

FetchStats

Timing stats for a (batched) data fetch

Fields

• fetchDataSource :: Text
• fetchBatchSize :: !Int
• fetchStart :: !Timestamp
• fetchDuration :: !Microseconds
• fetchSpace :: !Int64
• fetchFailures :: !Int
• fetchIgnoredFailures :: !Int
• fetchBatchId :: !Int
• fetchIds :: [CallId]

FetchCall

The stack trace of a call to dataFetch. These are collected only when profiling and reportLevel is 5 or greater.

Fields

• fetchReq :: String
• fetchStack :: [String]
• fetchStatId :: !CallId

MemoCall

Fields

• memoStatId :: !CallId
• memoSpace :: !Int64

FetchWait

Fields

• fetchWaitReqs :: HashMap Text Int

  What DataSources had requests that were being waited for

• fetchWaitStart :: !Timestamp
• fetchWaitDuration :: !Microseconds

Instances

Eq FetchStats

Defined in Haxl.Core.Stats

Methods

(==) :: FetchStats -> FetchStats -> Bool

(/=) :: FetchStats -> FetchStats -> Bool

Show FetchStats

Defined in Haxl.Core.Stats

Methods

showsPrec :: Int -> FetchStats -> ShowS

show :: FetchStats -> String

showList :: [FetchStats] -> ShowS

ToJSON FetchStats

Defined in Haxl.Core.Stats

Methods

toJSON :: FetchStats -> Value

toEncoding :: FetchStats -> Encoding

toJSONList :: [FetchStats] -> Value

toEncodingList :: [FetchStats] -> Encoding

type CallId = Int

type Microseconds = Int64

type Timestamp = Microseconds

emptyStats :: Stats

numFetches :: Stats -> Int

ppStats :: Stats -> String

Pretty-print Stats.

ppFetchStats :: FetchStats -> String

Pretty-print RoundStats.

aggregateFetchBatches :: ([FetchStats] -> a) -> Stats -> [a]

Aggregate stats merging FetchStats from the same dispatched batch into one.

data Profile

Constructors

Profile

Fields

• profile :: HashMap ProfileKey ProfileData

  Data per key (essentially per call stack)

• profileTree :: HashMap (ProfileLabel, ProfileKey) ProfileKey

  (label, parent) -> current. The exception is the root which will have (MAIN, 0) -> 0

• profileNextKey :: ProfileKey

  Provides a unique key per callstack

data ProfileMemo

Constructors

ProfileMemo

Fields

• profileMemoId :: !CallId
• profileMemoWasCached :: !Bool

Instances

Eq ProfileMemo

Defined in Haxl.Core.Stats

Methods

(==) :: ProfileMemo -> ProfileMemo -> Bool

(/=) :: ProfileMemo -> ProfileMemo -> Bool

Show ProfileMemo

Defined in Haxl.Core.Stats

Methods

showsPrec :: Int -> ProfileMemo -> ShowS

show :: ProfileMemo -> String

showList :: [ProfileMemo] -> ShowS

data ProfileFetch

Constructors

ProfileFetch

Fields

• profileFetchFetchId :: !CallId
• profileFetchMemoId :: !CallId
• profileFetchWasCached :: !Bool

Instances

Eq ProfileFetch

Defined in Haxl.Core.Stats

Methods

(==) :: ProfileFetch -> ProfileFetch -> Bool

(/=) :: ProfileFetch -> ProfileFetch -> Bool

Show ProfileFetch

Defined in Haxl.Core.Stats

Methods

showsPrec :: Int -> ProfileFetch -> ShowS

show :: ProfileFetch -> String

showList :: [ProfileFetch] -> ShowS

emptyProfile :: Profile

type ProfileLabel = Text

type ProfileKey = Int64

data ProfileData

Constructors

ProfileData

Fields

• profileAllocs :: !AllocCount

  allocations made by this label

• profileFetches :: [ProfileFetch]

  fetches made in this label

• profileLabelHits :: !LabelHitCount

  number of hits at this label

• profileMemos :: [ProfileMemo]

  memo and a boolean representing if it was cached at the time

• profileTime :: !Microseconds

  amount of time spent in computation at this label

Instances

Show ProfileData

Defined in Haxl.Core.Stats

Methods

showsPrec :: Int -> ProfileData -> ShowS

show :: ProfileData -> String

showList :: [ProfileData] -> ShowS

emptyProfileData :: ProfileData

type AllocCount = Int64

type LabelHitCount = Int64

Tracing flags

data Flags

Flags that control the operation of the engine.

Constructors

Flags

Fields

• trace :: !Int

  Tracing level (0 = quiet, 3 = very verbose).

• report :: !Int

  Report level: * 0 = quiet * 1 = outgone fetches, for debugging eg: timeouts * 2 = data fetch stats & errors * 3 = (same as 2, this used to enable errors) * 4 = profiling * 5 = log stack traces of dataFetch calls

• caching :: !Int

  Non-zero if caching is enabled. If caching is disabled, then we still do batching and de-duplication, but do not cache results.

• recording :: !Int

  Non-zero if recording is enabled. This allows tests to record cache calls for datasources by making uncachedRequest behave like dataFetch

defaultFlags :: Flags

ifTrace :: Monad m => Flags -> Int -> m a -> m ()

Runs an action if the tracing level is above the given threshold.

ifReport :: Monad m => Flags -> Int -> m a -> m ()

Runs an action if the report level is above the given threshold.

ifProfiling :: Monad m => Flags -> m a -> m ()

Building data sources

class (DataSourceName req, StateKey req, ShowP req) => DataSource u req where

The class of data sources, parameterised over the request type for that data source. Every data source must implement this class.

A data source keeps track of its state by creating an instance of StateKey to map the request type to its state. In this case, the type of the state should probably be a reference type of some kind, such as IORef.

For a complete example data source, see Examples.

Minimal complete definition

fetch

Methods

fetch

Arguments

:: State req

Current state.

-> Flags

Tracing flags.

-> u

User environment.

-> PerformFetch req

Fetch the data; see PerformFetch.

Issues a list of fetches to this DataSource. The BlockedFetch objects contain both the request and the ResultVars into which to put the results.

schedulerHint :: u -> SchedulerHint req

classifyFailure :: u -> req a -> SomeException -> FailureClassification

Instances

(Typeable tag, ShowP (ConcurrentIOReq tag), ConcurrentIO tag) => DataSource u (ConcurrentIOReq tag)

Defined in Haxl.DataSource.ConcurrentIO

Methods

fetch :: State (ConcurrentIOReq tag) -> Flags -> u -> PerformFetch (ConcurrentIOReq tag)

schedulerHint :: u -> SchedulerHint (ConcurrentIOReq tag)

classifyFailure :: u -> ConcurrentIOReq tag a -> SomeException -> FailureClassification

class ShowP f where

A class of type constructors for which we can show all parameterizations.

Methods

showp :: f a -> String

class DataSourceName (req :: * -> *) where

Methods

dataSourceName :: Proxy req -> Text

The name of this DataSource, used in tracing and stats. Must take a dummy request.

Instances

Typeable tag => DataSourceName (ConcurrentIOReq tag)

Defined in Haxl.DataSource.ConcurrentIO

Methods

dataSourceName :: Proxy (ConcurrentIOReq tag) -> Text

type Request req a = (Eq (req a), Hashable (req a), Typeable (req a), Show (req a), Show a)

A convenience only: package up Eq, Hashable, Typeable, and Show for requests into a single constraint.

data BlockedFetch r

A BlockedFetch is a pair of

• The request to fetch (with result type a)
• A ResultVar to store either the result or an error

We often want to collect together multiple requests, but they return different types, and the type system wouldn't let us put them together in a list because all the elements of the list must have the same type. So we wrap up these types inside the BlockedFetch type, so that they all look the same and we can put them in a list.

When we unpack the BlockedFetch and get the request and the ResultVar out, the type system knows that the result type of the request matches the type parameter of the ResultVar, so it will let us take the result of the request and store it in the ResultVar.

Constructors

BlockedFetch (r a) (ResultVar a)

data PerformFetch req

A data source can fetch data in one of four ways.

Constructors

SyncFetch ([BlockedFetch req] -> IO ())

Fully synchronous, returns only when all the data is fetched. See syncFetch for an example.

AsyncFetch ([BlockedFetch req] -> IO () -> IO ())

Asynchronous; performs an arbitrary IO action while the data is being fetched, but only returns when all the data is fetched. See asyncFetch for an example.

BackgroundFetch ([BlockedFetch req] -> IO ())

Fetches the data in the background, calling putResult at any time in the future. This is the best kind of fetch, because it provides the most concurrency.

class Typeable f => StateKey (f :: * -> *) where

StateKey maps one type to another type. A type that is an instance of StateKey can store and retrieve information from a StateStore.

Minimal complete definition

Nothing

Associated Types

data State f

Methods

getStateType :: Proxy f -> TypeRep

We default this to typeOf1, but if f is itself a complex type that is already applied to some paramaters, we want to be able to use the same state by using typeOf2, etc

Instances

Typeable tag => StateKey (ConcurrentIOReq tag)

Defined in Haxl.DataSource.ConcurrentIO

Associated Types

data State (ConcurrentIOReq tag) :: Type

Methods

getStateType :: Proxy (ConcurrentIOReq tag) -> TypeRep

data SchedulerHint (req :: * -> *)

Hints to the scheduler about this data source

Constructors

TryToBatch

Hold data-source requests while we execute as much as we can, so that we can hopefully collect more requests to batch.

SubmitImmediately

Submit a request via fetch as soon as we have one, don't try to batch multiple requests. This is really only useful if the data source returns BackgroundFetch, otherwise requests to this data source will be performed synchronously, one at a time.

data FailureClassification

Hints to the stats module about how to deal with these failures

Constructors

StandardFailure

IgnoredForStatsFailure

Result variables

newtype ResultVar a

A sink for the result of a data fetch in BlockedFetch

Constructors

ResultVar (Either SomeException a -> Bool -> IO ())

mkResultVar :: (Either SomeException a -> Bool -> IO ()) -> ResultVar a

putFailure :: Exception e => ResultVar a -> e -> IO ()

putResult :: ResultVar a -> Either SomeException a -> IO ()

putSuccess :: ResultVar a -> a -> IO ()

putResultFromChildThread :: ResultVar a -> Either SomeException a -> IO ()

Like putResult, but used to get correct accounting when work is being done in child threads. This is particularly important for data sources that are using BackgroundFetch, The allocation performed in the child thread up to this point will be propagated back to the thread that called runHaxl.

Note: if you're doing multiple putResult calls in the same thread ensure that only the last one is putResultFromChildThread. If you make multiple putResultFromChildThread calls, the allocation will be counted multiple times.

If you are reusing a thread for multiple fetches, you should call System.Mem.setAllocationCounter 0 after putResultFromChildThread, so that allocation is not counted multiple times.

Default fetch implementations

asyncFetch

Arguments

:: ((service -> IO ()) -> IO ())

Wrapper to perform an action in the context of a service.

-> (service -> IO ())

Dispatch all the pending requests and wait for the results

-> (forall a. service -> request a -> IO (IO (Either SomeException a)))

Submits an individual request to the service.

-> State request

Currently unused.

-> Flags

Currently unused.

-> u

Currently unused.

-> PerformFetch request

asyncFetchWithDispatch

Arguments

:: ((service -> IO ()) -> IO ())

Wrapper to perform an action in the context of a service.

-> (service -> IO ())

Dispatch all the pending requests

-> (service -> IO ())

Wait for the results

-> (forall a. service -> request a -> IO (IO (Either SomeException a)))

Enqueue an individual request to the service.

-> State request

Currently unused.

-> Flags

Currently unused.

-> u

Currently unused.

-> PerformFetch request

Common implementation templates for fetch of DataSource.

Example usage:

fetch = syncFetch MyDS.withService MyDS.retrieve
  $ \service request -> case request of
    This x -> MyDS.fetchThis service x
    That y -> MyDS.fetchThat service y

asyncFetchAcquireRelease

Arguments

:: IO service

Resource acquisition for this datasource

-> (service -> IO ())

Resource release

-> (service -> IO ())

Dispatch all the pending requests and wait for the results

-> (service -> IO ())

Wait for the results

-> (forall a. service -> request a -> IO (IO (Either SomeException a)))

Submits an individual request to the service.

-> State request

Currently unused.

-> Flags

Currently unused.

-> u

Currently unused.

-> PerformFetch request

A version of asyncFetch (actually asyncFetchWithDispatch) that handles exceptions correctly. You should use this instead of asyncFetch or asyncFetchWithDispatch. The danger with asyncFetch is that if an exception is thrown by withService, the inner action won't be executed, and we'll drop some data-fetches in the same round.

asyncFetchAcquireRelease behaves like the following:

asyncFetchAcquireRelease acquire release dispatch wait enqueue =
  AsyncFetch $ \requests inner ->
    bracket acquire release $ \service -> do
      getResults <- mapM (submitFetch service enqueue) requests
      dispatch service
      inner
      wait service
      sequence_ getResults

except that inner is run even if acquire, enqueue, or dispatch throws, unless an async exception is received.

backgroundFetchSeq

Arguments

:: (forall a. request a -> IO (Either SomeException a))

Run one request, will be run in a background thread

-> State request

Currently unused.

-> Flags

Currently unused.

-> u

Currently unused.

-> PerformFetch request

backgroundFetchPar

Arguments

:: (forall a. request a -> IO (Either SomeException a))

Run one request, will be run in a background thread

-> State request

Currently unused.

-> Flags

Currently unused.

-> u

Currently unused.

-> PerformFetch request

backgroundFetchAcquireRelease

Arguments

:: IO service

Resource acquisition for this datasource

-> (service -> IO ())

Resource release

-> (service -> Int -> StablePtr PrimMVar -> IO ())

Dispatch all the pending requests and when ready trigger the given mvar

-> (service -> IO ())

Process all requests

-> (forall a. service -> request a -> IO (IO (Either SomeException a)))

Submits an individual request to the service.

-> State request

Currently unused.

-> Flags

Currently unused.

-> u

Currently unused.

-> PerformFetch request

A version of backgroundFetchAcquireReleaseMVar where the dispatch function is given a 'StablePtr PrimMVar' which is more useful for C based APIs.

backgroundFetchAcquireReleaseMVar

Arguments

:: IO service

Resource acquisition for this datasource

-> (service -> IO ())

Resource release

-> (service -> Int -> MVar () -> IO ())

Dispatch all the pending requests and when ready trigger the given mvar

-> (service -> IO ())

Process all requests

-> (forall a. service -> request a -> IO (IO (Either SomeException a)))

Submits an individual request to the service.

-> State request

Currently unused.

-> Flags

Currently unused.

-> u

Currently unused.

-> PerformFetch request

stubFetch :: Exception e => (forall a. r a -> e) -> State r -> Flags -> u -> PerformFetch r

syncFetch

Arguments

:: ((service -> IO ()) -> IO ())

Wrapper to perform an action in the context of a service.

-> (service -> IO ())

Dispatch all the pending requests and wait for the results

-> (forall a. service -> request a -> IO (IO (Either SomeException a)))

Submits an individual request to the service.

-> State request

Currently unused.

-> Flags

Currently unused.

-> u

Currently unused.

-> PerformFetch request

Utilities

except :: Exception e => e -> Either SomeException a

setError :: Exception e => (forall a. r a -> e) -> BlockedFetch r -> IO ()

Function for easily setting a fetch to a particular exception

getMapFromRCMap :: ReqCountMap -> Map Text (Map TypeRep Int)

Exceptions

module Haxl.Core.Exception

Recording the function callgraph

module Haxl.Core.CallGraph