| Safe Haskell | None |
|---|---|
| Language | Haskell2010 |
Database.Postgres.Temp
Description
This module provides functions for creating a temporary postgres instance.
By default it will create a temporary data directory and
a temporary directory for a UNIX domain socket for postgres to listen on in addition to
listening on 127.0.0.1 and ::1.
Here is an example using the expection safe with function:
with$ \db ->bracket(connectPostgreSQL(toConnectionStringdb))close$ \conn ->execute_conn "CREATE TABLE foo (id int)"
To extend or override the defaults use withConfig (or startConfig).
tmp-postgres ultimately calls initdb (optionally), postgres and
createdb (optionally).
All of the command line, environment variables and configuration files that are generated by default for the respective executables can be extended or overriden.
In general tmp-postgres is useful if you want a clean temporary
postgres and do not want to worry about clashing with an existing
postgres instance (or needing to ensure postgres is already running).
Here are some different use cases for tmp-postgres and their respective
configurations:
- The default
withandstartfunctions can be used to make a sandboxed temporary database for testing. - By disabling
initdbone could run a temporary isolated postgres on a base backup to test a migration. - By using the
stopPostgresandwithRestartfunctions one can test backup strategies.
WARNING!
Ubuntu's PostgreSQL installation does not put initdb on the PATH. We need to add it manually.
The necessary binaries are in the /usr/lib/postgresql/VERSION/bin/ directory, and should be added to the PATH
echo "export PATH=$PATH:/usr/lib/postgresql/VERSION/bin/" >> /home/ubuntu/.bashrc
Synopsis
- with :: (DB -> IO a) -> IO (Either StartError a)
- withConfig :: Config -> (DB -> IO a) -> IO (Either StartError a)
- withRestart :: DB -> (DB -> IO a) -> IO (Either StartError a)
- defaultConfig :: Config
- defaultConfig_9_3_10 :: Config
- verboseConfig :: Config
- autoExplainConfig :: Int -> Config
- optionsToDefaultConfig :: Options -> Config
- data DB
- toConnectionString :: DB -> ByteString
- toConnectionOptions :: DB -> Options
- toDataDirectory :: DB -> FilePath
- toTemporaryDirectory :: DB -> FilePath
- toPostgresqlConfigFile :: DB -> String
- makeDataDirectoryPermanent :: DB -> DB
- prettyPrintDB :: DB -> String
- start :: IO (Either StartError DB)
- startConfig :: Config -> IO (Either StartError DB)
- stop :: DB -> IO ()
- restart :: DB -> IO (Either StartError DB)
- stopPostgres :: DB -> IO ExitCode
- withDbCache :: (Cache -> IO a) -> IO a
- withDbCacheConfig :: CacheConfig -> (Cache -> IO a) -> IO a
- data CacheConfig = CacheConfig {}
- defaultCacheConfig :: CacheConfig
- data Cache
- cacheConfig :: Cache -> Config
- setupInitDbCache :: CacheConfig -> IO Cache
- cleanupInitDbCache :: Cache -> IO ()
- withSnapshot :: DB -> (Snapshot -> IO a) -> IO (Either StartError a)
- data Snapshot
- snapshotConfig :: Snapshot -> Config
- takeSnapshot :: DB -> IO (Either StartError Snapshot)
- cleanupSnapshot :: Snapshot -> IO ()
- cacheAction :: FilePath -> (DB -> IO ()) -> Config -> IO (Either StartError Config)
- data StartError
- = StartPostgresFailed ExitCode
- | InitDbFailed { }
- | CreateDbFailed { }
- | PlanFailed String [String]
- | CompleteProcessConfigFailed String [String]
- | ConnectionTimedOut
- | DeleteDbError SqlError
- | EmptyDataDirectory
- | CopyCachedInitDbFailed String ExitCode
- | FailedToFindDataDirectory String
- | SnapshotCopyFailed String ExitCode
- data Config = Config {
- logger :: Last Logger
- initDbConfig :: Accum ProcessConfig
- copyConfig :: Last (Maybe CopyDirectoryCommand)
- createDbConfig :: Accum ProcessConfig
- postgresConfig :: ProcessConfig
- connectionOptions :: Options
- postgresConfigFile :: [(String, String)]
- connectionTimeout :: Last Int
- socketDirectory :: DirectoryType
- dataDirectory :: DirectoryType
- port :: Last (Maybe Int)
- temporaryDirectory :: Last FilePath
- initDbCache :: Last (Maybe (Bool, FilePath))
- prettyPrintConfig :: Config -> String
- data ProcessConfig = ProcessConfig {}
- data EnvironmentVariables = EnvironmentVariables {}
- data CommandLineArgs = CommandLineArgs {}
- data DirectoryType
- data CompleteDirectoryType
- data Accum a
- type Logger = Event -> IO ()
- data Event
Start and Stop postgres
Exception safe interface
Arguments
| :: (DB -> IO a) |
|
| -> IO (Either StartError a) |
Arguments
| :: Config | The |
| -> (DB -> IO a) |
|
| -> IO (Either StartError a) |
Exception safe database create with options. See startConfig for more
details. Calls stop even in the face of exceptions.
Since: 1.21.0.0
withRestart :: DB -> (DB -> IO a) -> IO (Either StartError a) Source #
Exception safe version of restart.
Since: 1.12.0.0
Configuration
Defaults
defaultConfig :: Config Source #
The default configuration. This will create a database called "postgres"
via initdb (it's default behavior).
It will create a temporary directory for the data and a temporary directory
for a unix socket and listen on 127.0.0.1 and ::1 on a random port.
Additionally it will use the following "postgresql.conf"
which is optimized for performance.
shared_buffers = 12MB fsync = off synchronous_commit = off full_page_writes = off log_min_messages = PANIC log_min_error_statement = PANIC log_statement = none client_min_messages = ERROR commit_delay = 100000 wal_level = minimal archive_mode = off max_wal_senders = 0
defaultConfig also passes the --no-sync flag to initdb.
If you would like to customize this behavior you can start with the
defaultConfig and overwrite fields or combine a defaultConfig with another Config
using <> (mappend).
Alternatively you can eschew defaultConfig altogether, however
your postgres might start and run faster if you use
defaultConfig.
The defaultConfig redirects all output to /dev/null. See
verboseConfig for a version that logs more output.
To append additional lines to "postgresql.conf" file create a
custom Config like the following.
custom = defaultConfig <> mempty
{ postgresConfigFile =
[ ("wal_level, "replica")
, ("archive_mode", on")
, ("max_wal_senders", "2")
, ("fsync", "on")
, ("synchronous_commit", "on")
]
}
As an alternative to using defaultConfig one could create a
config from connections parameters using optionsToDefaultConfig.
Since: 1.21.0.0
defaultConfig_9_3_10 :: Config Source #
Default configuration for PostgreSQL versions 9.3 and greater but less than 10.
If you get an error that "--no-sync" is an invalid parameter then you should use this config.
Since: 1.21.1.0
verboseConfig :: Config Source #
This is similar to defaultConfig but it logs as much as possible..
Since: 1.21.0.0
A config which loads and configures auto_explain. Useful for
understanding slow queries plans.
Since: 1.34.1.0
Custom Config builder helpers
Main resource handle
Handle for holding temporary resources, the postgres process handle
and postgres connection information. The DB also includes the
final plan used to start initdb, createdb and
postgres.
Since: 1.12.0.0
DB accessors
toConnectionString :: DB -> ByteString Source #
Convert a DB to a connection string. Alternatively one can access the
Options using toConnectionOptions.
Since: 1.12.0.0
toConnectionOptions :: DB -> Options Source #
toDataDirectory :: DB -> FilePath Source #
Access the data directory. This was either generated or
specified explicitly when creating the Config
Since: 1.12.0.0
toTemporaryDirectory :: DB -> FilePath Source #
Get the directory that is used to create other temporary directories
Since: 1.12.0.0
toPostgresqlConfigFile :: DB -> String Source #
Get the final postgresql.conf
Since: 1.25.0.0
DB modifiers
makeDataDirectoryPermanent :: DB -> DB Source #
Make the data directory permanent. Useful for debugging.
If you are using with or withConfig this function will
not modify the DB that is passed for cleanup. You will
need to setup your own bracket like
bracket (fmapmakeDataDirectoryPermanentstart) (either memptystop)
Since: 1.24.0.0
DB debugging
Separate start and stop interface.
start :: IO (Either StartError DB) Source #
Default start behavior. Equivalent to calling startConfig with the
defaultConfig.
Since: 1.21.0.0
Arguments
| :: Config |
|
| -> IO (Either StartError DB) |
Create zero or more temporary resources and use them to make a Config.
The passed in config is inspected and a generated config is created. The final config is built by
generated <> extra
Based on the value of socketDirectory a "postgresql.conf" is created with:
listen_addresses = '127.0.0.1, ::1' unix_socket_directories = 'SOCKET_DIRECTORY'
Additionally the generated Config also:
- Sets a
connectionTimeoutof one minute. - Redirects output to
/dev/null.
All of these values can be overrided by the extra config.
The returned DB requires cleanup. startConfig should be
used with a bracket and stop, e.g.
withConfig::Config-> (DB-> IO a) -> IO (EitherStartErrora)withConfigplan f =bracket(startConfigplan) (either memptystop) $ either (pure . Left) (fmap Right . f)
or just use withConfig. If you are calling startConfig you
probably want withConfig anyway.
Since: 1.15.0.0
Stop the postgres process and cleanup any temporary resources that
might have been created.
Since: 1.12.0.0
restart :: DB -> IO (Either StartError DB) Source #
Restart the postgres from DB using the prior Config. This
will also start an instance previously stoppped with stopPostgres.
Since: 1.12.0.0
stopPostgres :: DB -> IO ExitCode Source #
Only stop the postgres process but leave any temporary resources.
Useful for testing backup strategies when used in conjunction with
restart or withRestart.
Since: 1.12.0.0
Faster Startup
with and related functions are fast by themselves but
by utilizing various forms of caching we can make them
much faster.
The slowest part of starting a new postgres cluster is the initdb
call which initializes the database files. However for a given initdb version and configuration parameters the output
is the same.
To take advantage of this idempotent behavior we can cache the output of
initdb and copy the outputted database cluster files instead of recreating
them. This leads to a 4x improvement in startup time.
See withDbCache and related functions for more details.
Additionally one can take snapshots of a database cluster and start
new postgres instances using the snapshot as an initial database
cluster.
This is useful if one has tests that require a time consuming migration process. By taking a snapshot after the migration we can start new isolated clusters from the point in time after the migration but before any test data has tainted the database.
See withSnapshot for details.
withDbCache :: (Cache -> IO a) -> IO a Source #
Equivalent to withDbCacheConfig with the CacheConfig
defaultCacheConfig makes.
Here is an example using caching:
withDbCache $ \cache -> do withConfig (cacheConfig cache) $ \db -> ... withConfig (cacheConfig cache) $ \db -> ...
Since: 1.25.0.0
Arguments
| :: CacheConfig | Configuration |
| -> (Cache -> IO a) | action for which caching is enabled |
| -> IO a |
Enable initdb data directory caching. This can lead to a 4x speedup.
Exception safe version of setupInitDbCache. Equivalent to
withDbCacheConfig= bracket (setupInitDbCacheconfig)cleanupInitDbCache
Since: 1.25.0.0
initdb cache configuration.
data CacheConfig Source #
Configuration for the initdb data directory cache.
Since: 1.25.0.0
Constructors
| CacheConfig | |
Fields
| |
defaultCacheConfig :: CacheConfig Source #
defaultCacheConfig attempts to determine if the cp on the path
supports "copy on write" flags and if it does, sets cacheUseCopyOnWrite
to True.
It sets cacheDirectoryType to Permanent ~/.tmp-postgres and
cacheTemporaryDirectory to /tmp (but this is not used when
Permanent is set).
Since: 1.25.0.0
initdb cache handle.
A handle to cache temporary resources and configuration.
Since: 1.25.0.0
Instances
| Generic Cache Source # | |
| NFData Cache Source # | |
Defined in Database.Postgres.Temp.Internal | |
| type Rep Cache Source # | |
Defined in Database.Postgres.Temp.Internal type Rep Cache = D1 (MetaData "Cache" "Database.Postgres.Temp.Internal" "tmp-postgres-1.34.1.0-DKDjvRpjblb7GUUaSbcxl6" False) (C1 (MetaCons "Cache" PrefixI True) (S1 (MetaSel (Just "cacheResourcesCow") NoSourceUnpackedness NoSourceStrictness DecidedLazy) (Rec0 Bool) :*: S1 (MetaSel (Just "cacheResourcesDirectory") NoSourceUnpackedness NoSourceStrictness DecidedLazy) (Rec0 CompleteDirectoryType))) | |
Separate start and stop interface.
setupInitDbCache :: CacheConfig -> IO Cache Source #
Setup the initdb cache folder.
Since: 1.25.0.0
cleanupInitDbCache :: Cache -> IO () Source #
Cleanup the cache directory if it was Temporary.
Since: 1.25.0.0
Data Directory Snapshots
Exception safe interface
withSnapshot :: DB -> (Snapshot -> IO a) -> IO (Either StartError a) Source #
Exception safe method for taking a file system level copy of the database cluster.
Snapshots are useful if you would like to start every test from a migrated database and the migration process is more time consuming then copying the additional data.
Here is an example with caching and snapshots:
withDbCache $ \cache -> withConfig (cacheConfig cache) $ \db ->
migrate db
withSnapshot Temporary db $ \snapshot -> do
withConfig (snapshotConfig db) $ \migratedDb -> ...
withConfig (snapshotConfig db) $ \migratedDb -> ...
withConfig (snapshotConfig db) $ \migratedDb -> ...
The Snapshots are ephemeral. If you would like the Snapshots to persistent
consider using cacheAction instead.
Since: 1.29.0.0
Snapshot handle
A type to track a possibly temporary snapshot directory
Since: 1.20.0.0
Instances
| Generic Snapshot Source # | |
| NFData Snapshot Source # | |
Defined in Database.Postgres.Temp.Internal | |
| type Rep Snapshot Source # | |
Defined in Database.Postgres.Temp.Internal type Rep Snapshot = D1 (MetaData "Snapshot" "Database.Postgres.Temp.Internal" "tmp-postgres-1.34.1.0-DKDjvRpjblb7GUUaSbcxl6" True) (C1 (MetaCons "Snapshot" PrefixI True) (S1 (MetaSel (Just "unSnapshot") NoSourceUnpackedness NoSourceStrictness DecidedLazy) (Rec0 CompleteDirectoryType))) | |
snapshotConfig :: Snapshot -> Config Source #
Convert a snapshot into a Config that includes a copyConfig for copying the
snapshot directory to a temporary directory.
Since: 1.20.0.0
Separate start and stop interface.
Arguments
| :: DB | The handle. The |
| -> IO (Either StartError Snapshot) |
Shutdown the database and copy the directory to a folder.
Since: 1.29.0.0
cleanupSnapshot :: Snapshot -> IO () Source #
Cleanup any temporary resources used for the snapshot.
Since: 1.20.0.0
Conditional caching of DB actions
Arguments
| :: FilePath | Location of the data directory cache. |
| -> (DB -> IO ()) |
|
| -> Config |
|
| -> IO (Either StartError Config) |
Check to see if a cached data directory exists.
If the file path does not exist the initial config is used to start a postgres
instance. After which the action is applied, the data directory is cached
and postgres is shutdown.
cacheAction mappends a config to copy the cached data directory
on startup onto the initial config and returns it. In other words:
initialConfig <> configFromCachePath
cacheAction can be used to create a snapshot of migrated database and not
remigrate as long as the migration does not change. See withSnapshot for
a ephemeral version of taking snapshots.
You can nest calls to cacheAction and safe to call it from several threads.
However cacheAction uses locks internal to prevent multiple threads from
stomping on each other.
If one makes a nested call and accidently uses the same cache directory in both calls the calls will deadlock. If this occurs on the same thread RTS will throw an exception. However do not rely on this and just be careful to not reuse the same cache path when nesting calls.
There is no good reuse the cache path when nesting so one is unlikely to run into this.
Since: 1.34.0.0
Errors
data StartError Source #
A list of failures that can occur when starting. This is not and exhaustive list but covers the errors that the system catches for the user.
Since: 1.29.0.0
Constructors
| StartPostgresFailed ExitCode |
|
| InitDbFailed |
|
Fields | |
| CreateDbFailed |
|
Fields | |
| PlanFailed String [String] | The |
| CompleteProcessConfigFailed String [String] | The |
| ConnectionTimedOut | Timed out waiting for |
| DeleteDbError SqlError | |
| EmptyDataDirectory | This will happen if a |
| CopyCachedInitDbFailed String ExitCode | This is called if copying a folder cache fails. |
| FailedToFindDataDirectory String | Failed to find a data directory when trying to get
a cached |
| SnapshotCopyFailed String ExitCode | We tried to copy a data directory to a snapshot folder and it failed |
Instances
| Eq StartError Source # | |
Defined in Database.Postgres.Temp.Internal.Core | |
| Show StartError Source # | |
Defined in Database.Postgres.Temp.Internal.Core Methods showsPrec :: Int -> StartError -> ShowS # show :: StartError -> String # showList :: [StartError] -> ShowS # | |
| Exception StartError Source # | |
Defined in Database.Postgres.Temp.Internal.Core Methods toException :: StartError -> SomeException # fromException :: SomeException -> Maybe StartError # displayException :: StartError -> String # | |
Configuration Types
Config
The high level options for overriding default behavior.
Since: 1.22.0.0
Constructors
| Config | |
Fields
| |
Instances
ProcessConfig
data ProcessConfig Source #
Process configuration
Since: 1.12.0.0
Constructors
| ProcessConfig | |
Fields
| |
Instances
EnvironmentVariables
data EnvironmentVariables Source #
The environment variables can be declared to inherit from the running process or they can be specifically added.
Since: 1.12.0.0
Instances
CommandLineArgs
data CommandLineArgs Source #
A type to help combine command line Args.
Since: 1.12.0.0
Constructors
| CommandLineArgs | |
Fields | |
Instances
DirectoryType
data DirectoryType Source #
Used to specify a Temporary folder that is automatically
cleaned up or a Permanent folder which is not
automatically cleaned up.
Since: 1.12.0.0
Constructors
| Permanent FilePath | A permanent file that should not be generated. |
| Temporary | A temporary file that needs to generated. |
Instances
CompleteDirectoryType
data CompleteDirectoryType Source #
A type to track whether a file is temporary and needs to be cleaned up.
Since: 1.12.0.0
Constructors
| CPermanent FilePath | |
| CTemporary FilePath |
Instances
Accum
Accum is a monoid.
It's <> behavior is analogous to 1 and 0 with *. Think of DontCare
as 1 and Zlich as 0.
The behavior of Merge is like Justs.
Since: 1.17.0.0
Logger
Internal events passed to the logger .
Internal events for debugging
Since: 1.12.0.0
Constructors
| StartPlan String | The first event. This useful for debugging
what is actual passed to the |
| StartPostgres | The second event. Postgres is about to get called |
| WaitForDB | The third event. Postgres started. We are now about to setup a reconnect loop (racing with a process checker) |
| TryToConnect | The fourth event and (possibly all subsequent events).
We are looping trying to successfully connect to the |