| Safe Haskell | None |
|---|---|
| Language | Haskell2010 |
Pipes.Aws.S3.Upload
Contents
Description
pipes utilities for uploading data to AWS S3 objects.
- toS3 :: forall m a. (MonadIO m, MonadCatch m) => ChunkSize -> Bucket -> Object -> Producer ByteString m a -> m a
- toS3' :: forall m a. (MonadIO m, MonadCatch m) => Configuration -> S3Configuration NormalQuery -> ChunkSize -> Bucket -> Object -> Producer ByteString m a -> m a
- toS3WithManager :: forall m a. (MonadIO m, MonadCatch m) => Manager -> Configuration -> S3Configuration NormalQuery -> ChunkSize -> Bucket -> Object -> Producer ByteString m a -> m a
- type ChunkSize = Int
- defaultChunkSize :: ChunkSize
- data EmptyS3UploadError = EmptyS3UploadError Bucket Object
- data FailedUploadError = FailedUploadError {}
- newtype UploadId = UploadId Text
Documentation
These internally use the S3 multi-part upload interface to achieve streaming upload behavior.
In the case of failure one of two exceptions will be thrown,
EmptyS3UploadError: In the event that theProducerfails to produce any content to uploadFailedUploadError: In any other case.
The FailedUploadError exception carries the UploadId of the failed
upload as well as the inner exception. Note that while the library makes
an attempt to clean up the parts of the partial upload, there may still
be remnants due to limitations in the aws library.
toS3 :: forall m a. (MonadIO m, MonadCatch m) => ChunkSize -> Bucket -> Object -> Producer ByteString m a -> m a Source #
Upload content to an S3 object.
May throw a EmptyS3UploadError if the producer fails to provide any content.
Arguments
| :: (MonadIO m, MonadCatch m) | |
| => Configuration | e.g. from |
| -> S3Configuration NormalQuery | e.g. |
| -> ChunkSize | |
| -> Bucket | |
| -> Object | |
| -> Producer ByteString m a | |
| -> m a |
Upload content to an S3 object, explicitly specifying an
Configuration, which provides credentials and logging configuration.
May throw a EmptyS3UploadError if the producer fails to provide any content.
Arguments
| :: (MonadIO m, MonadCatch m) | |
| => Manager | |
| -> Configuration | e.g. from |
| -> S3Configuration NormalQuery | e.g. |
| -> ChunkSize | |
| -> Bucket | |
| -> Object | |
| -> Producer ByteString m a | |
| -> m a |
Download an object from S3 explicitly specifying an http-client Manager
and aws Configuration (which provides credentials and logging
configuration).
This can be more efficient when submitting many small requests as it allows
re-use of the Manager across requests. Note that the Manager provided
must support TLS; such a manager can be created with
import qualified Aws.Core as Aws
import qualified Network.HTTP.Client as HTTP.Client
import qualified Network.HTTP.Client.TLS as HTTP.Client.TLS
putObject :: MonadSafe m => Bucket -> Object -> Producer BS.ByteString m () -> m ()
putObject bucket object prod = do
cfg <- liftIO baseConfiguration
mgr <- liftIO $ newManager tlsManagerSettings
toS3WithManager mgr cfg defServiceConfig defaultChunkSize bucket object prod
May throw a EmptyS3UploadError if the producer fails to provide any content.
Chunk size
To maintain healthy streaming uploads are performed in a chunked manner. This is the size of the upload chunk size in bytes. Due to S3 interface restrictions this must be at least five megabytes.
defaultChunkSize :: ChunkSize Source #
A reasonable chunk size of 10 megabytes.
Error handling
data EmptyS3UploadError Source #
Thrown when an upload with no data is attempted.
Constructors
| EmptyS3UploadError Bucket Object |
Instances
data FailedUploadError Source #
Thrown when an error occurs during an upload.
Constructors
| FailedUploadError | |
Instances