Streamly.Internal.FileSystem.File

Read and write streams and arrays to and from files specified by their paths in the file system. Unlike the handle based APIs which can have a read/write session consisting of multiple reads and writes to the handle, these APIs are one shot read or write APIs. These APIs open the file handle, perform the requested operation and close the handle. Thease are safer compared to the handle based APIs as there is no possibility of a file descriptor leakage.

import qualified Streamly.Internal.FileSystem.File as File

Streaming IO

Stream data to or from a file or device sequentially. When reading, the stream is lazy and generated on-demand as the consumer consumes it. Read IO requests to the IO device are performed in chunks limited to a maximum size of 32KiB, this is referred to as defaultChunkSize in the documentation. One IO request may or may not read the full chunk. If the whole stream is not consumed, it is possible that we may read slightly more from the IO device than what the consumer needed. Unless specified otherwise in the API, writes are collected into chunks of defaultChunkSize before they are written to the IO device.

File IO Using Handle

withFile :: (MonadIO m, MonadCatch m) => FilePath -> IOMode -> (Handle -> Stream m a) -> Stream m a

withFile name mode act opens a file using openFile and passes the resulting handle to the computation act. The handle will be closed on exit from withFile, whether by normal termination or by raising an exception. If closing the handle raises an exception, then this exception will be raised by withFile rather than any exception raised by act.

Pre-release

Streams

read :: (MonadIO m, MonadCatch m) => FilePath -> Stream m Word8

Generate a stream of bytes from a file specified by path. The stream ends when EOF is encountered. File is locked using multiple reader and single writer locking mode.

Pre-release

readChunksWith :: (MonadIO m, MonadCatch m) => Int -> FilePath -> Stream m (Array Word8)

readChunksWith size file reads a stream of arrays from file file. The maximum size of a single array is specified by size. The actual size read may be less than or equal to size.

Pre-release

readChunks :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Array Word8)

readChunks file reads a stream of arrays from file file. The maximum size of a single array is limited to defaultChunkSize. The actual size read may be less than defaultChunkSize.

readChunks = readChunksWith defaultChunkSize

Pre-release

Unfolds

readerWith :: (MonadIO m, MonadCatch m) => Unfold m (Int, FilePath) Word8

Unfolds the tuple (bufsize, filepath) into a byte stream, read requests to the IO device are performed using buffers of bufsize.

Pre-release

reader :: (MonadIO m, MonadCatch m) => Unfold m FilePath Word8

Unfolds a file path into a byte stream. IO requests to the device are performed in sizes of defaultChunkSize.

Pre-release

chunkReaderWith :: (MonadIO m, MonadCatch m) => Unfold m (Int, FilePath) (Array Word8)

Unfold the tuple (bufsize, filepath) into a stream of Word8 arrays. Read requests to the IO device are performed using a buffer of size bufsize. The size of an array in the resulting stream is always less than or equal to bufsize.

Pre-release

chunkReaderFromToWith :: (MonadIO m, MonadCatch m) => Unfold m (Int, Int, Int, FilePath) (Array Word8)

Unfold the tuple (from, to, bufsize, filepath) into a stream of Word8 arrays. Read requests to the IO device are performed using a buffer of size bufsize starting from absolute offset of from till the absolute position of to. The size of an array in the resulting stream is always less than or equal to bufsize.

Pre-release

chunkReader :: (MonadIO m, MonadCatch m) => Unfold m FilePath (Array Word8)

Unfolds a FilePath into a stream of Word8 arrays. Requests to the IO device are performed using a buffer of size defaultChunkSize. The size of arrays in the resulting stream are therefore less than or equal to defaultChunkSize.

Pre-release

Write To File

putChunk :: FilePath -> Array a -> IO ()

Write an array to a file. Overwrites the file if it exists.

Pre-release

Folds

write :: (MonadIO m, MonadCatch m) => FilePath -> Fold m Word8 ()

Write a byte stream to a file. Accumulates the input in chunks of up to defaultChunkSize before writing to the IO device.

Pre-release

writeWith :: (MonadIO m, MonadCatch m) => Int -> FilePath -> Fold m Word8 ()

writeWith chunkSize handle writes the input stream to handle. Bytes in the input stream are collected into a buffer until we have a chunk of size chunkSize and then written to the IO device.

Pre-release

writeChunks :: (MonadIO m, MonadCatch m) => FilePath -> Fold m (Array a) ()

Write a stream of chunks to a handle. Each chunk in the stream is written to the device as a separate IO request.

Pre-release

Writing Streams

fromBytes :: (MonadIO m, MonadCatch m) => FilePath -> Stream m Word8 -> m ()

Write a byte stream to a file. Combines the bytes in chunks of size up to defaultChunkSize before writing. If the file exists it is truncated to zero size before writing. If the file does not exist it is created. File is locked using single writer locking mode.

Pre-release

fromBytesWith :: (MonadIO m, MonadCatch m) => Int -> FilePath -> Stream m Word8 -> m ()

Like write but provides control over the write buffer. Output will be written to the IO device as soon as we collect the specified number of input elements.

Pre-release

fromChunks :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Array a) -> m ()

Write a stream of arrays to a file. Overwrites the file if it exists.

Pre-release

Append To File

writeAppend :: (MonadIO m, MonadCatch m) => FilePath -> Stream m Word8 -> m ()

Append a byte stream to a file. Combines the bytes in chunks of size up to defaultChunkSize before writing. If the file exists then the new data is appended to the file. If the file does not exist it is created. File is locked using single writer locking mode.

Pre-release

writeAppendWith :: (MonadIO m, MonadCatch m) => Int -> FilePath -> Stream m Word8 -> m ()

Like append but provides control over the write buffer. Output will be written to the IO device as soon as we collect the specified number of input elements.

Pre-release

writeAppendArray :: FilePath -> Array a -> IO ()

append an array to a file.

Pre-release

writeAppendChunks :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Array a) -> m ()

Append a stream of arrays to a file.

Pre-release

Deprecated

readWithBufferOf :: (MonadIO m, MonadCatch m) => Unfold m (Int, FilePath) Word8

Deprecated: Please use readerWith instead

readChunksWithBufferOf :: (MonadIO m, MonadCatch m) => Unfold m (Int, FilePath) (Array Word8)

Deprecated: Please use chunkReaderWith instead

readChunksFromToWith :: (MonadIO m, MonadCatch m) => Unfold m (Int, Int, Int, FilePath) (Array Word8)

Deprecated: Please use chunkReaderFromToWith instead

toBytes :: (MonadIO m, MonadCatch m) => FilePath -> Stream m Word8

Deprecated: Please use read instead

toChunks :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Array Word8)

Deprecated: Please use readChunks instead

toChunksWithBufferOf :: (MonadIO m, MonadCatch m) => Int -> FilePath -> Stream m (Array Word8)

Deprecated: Please use readChunksWith instead

writeWithBufferOf :: (MonadIO m, MonadCatch m) => Int -> FilePath -> Fold m Word8 ()

Deprecated: Please use writeWith instead

fromBytesWithBufferOf :: (MonadIO m, MonadCatch m) => Int -> FilePath -> Stream m Word8 -> m ()

Deprecated: Please use fromBytesWith instead