Data.ByteString.Internal

A module containing semi-public ByteString internals. This exposes the ByteString representation and low level construction functions. As such all the functions in this module are unsafe. The API is also not stable.

Where possible application should instead use the functions from the normal public interface modules, such as Data.ByteString.Unsafe. Packages that extend the ByteString system at a low level will need to use this module.

The ByteString type and representation

data ByteString

A space-efficient representation of a Word8 vector, supporting many efficient operations.

A ByteString contains 8-bit bytes, or by using the operations from Data.ByteString.Char8 it can be interpreted as containing 8-bit characters.

Constructors

BS !(ForeignPtr Word8) !Int

Since: 0.11.0.0

Bundled Patterns

pattern PS :: ForeignPtr Word8 -> Int -> Int -> ByteString

PS foreignPtr offset length represents a ByteString with data backed by a given foreignPtr, starting at a given offset in bytes and of a specified length.

This pattern is used to emulate the legacy ByteString data constructor, so that pre-existing code generally doesn't need to change to benefit from the simplified BS constructor and can continue to function unchanged.

Note: Matching with this constructor will always be given a 0 offset, as the base will be manipulated by plusForeignPtr instead.

Instances

Data ByteString

Defined in Data.ByteString.Internal.Type

Methods

gfoldl :: (forall d b. Data d => c (d -> b) -> d -> c b) -> (forall g. g -> c g) -> ByteString -> c ByteString

gunfold :: (forall b r. Data b => c (b -> r) -> c r) -> (forall r. r -> c r) -> Constr -> c ByteString

toConstr :: ByteString -> Constr

dataTypeOf :: ByteString -> DataType

dataCast1 :: Typeable t => (forall d. Data d => c (t d)) -> Maybe (c ByteString)

dataCast2 :: Typeable t => (forall d e. (Data d, Data e) => c (t d e)) -> Maybe (c ByteString)

gmapT :: (forall b. Data b => b -> b) -> ByteString -> ByteString

gmapQl :: (r -> r' -> r) -> r -> (forall d. Data d => d -> r') -> ByteString -> r

gmapQr :: forall r r'. (r' -> r -> r) -> r -> (forall d. Data d => d -> r') -> ByteString -> r

gmapQ :: (forall d. Data d => d -> u) -> ByteString -> [u]

gmapQi :: Int -> (forall d. Data d => d -> u) -> ByteString -> u

gmapM :: Monad m => (forall d. Data d => d -> m d) -> ByteString -> m ByteString

gmapMp :: MonadPlus m => (forall d. Data d => d -> m d) -> ByteString -> m ByteString

gmapMo :: MonadPlus m => (forall d. Data d => d -> m d) -> ByteString -> m ByteString

IsString ByteString

Beware: fromString truncates multi-byte characters to octets. e.g. "枯朶に烏のとまりけり秋の暮" becomes �6k�nh~�Q��n�

Defined in Data.ByteString.Internal.Type

Methods

fromString :: String -> ByteString

Monoid ByteString

Defined in Data.ByteString.Internal.Type

Methods

mempty :: ByteString

mappend :: ByteString -> ByteString -> ByteString

mconcat :: [ByteString] -> ByteString

Semigroup ByteString

Defined in Data.ByteString.Internal.Type

Methods

(<>) :: ByteString -> ByteString -> ByteString

sconcat :: NonEmpty ByteString -> ByteString

stimes :: Integral b => b -> ByteString -> ByteString

IsList ByteString

Since: 0.10.12.0

Defined in Data.ByteString.Internal.Type

Associated Types

type Item ByteString

Methods

fromList :: [Item ByteString] -> ByteString

fromListN :: Int -> [Item ByteString] -> ByteString

toList :: ByteString -> [Item ByteString]

Read ByteString

Defined in Data.ByteString.Internal.Type

Methods

readsPrec :: Int -> ReadS ByteString

readList :: ReadS [ByteString]

readPrec :: ReadPrec ByteString

readListPrec :: ReadPrec [ByteString]

Show ByteString

Defined in Data.ByteString.Internal.Type

Methods

showsPrec :: Int -> ByteString -> ShowS

show :: ByteString -> String

showList :: [ByteString] -> ShowS

NFData ByteString

Defined in Data.ByteString.Internal.Type

Methods

rnf :: ByteString -> ()

Eq ByteString

Defined in Data.ByteString.Internal.Type

Methods

(==) :: ByteString -> ByteString -> Bool

(/=) :: ByteString -> ByteString -> Bool

Ord ByteString

Defined in Data.ByteString.Internal.Type

Methods

compare :: ByteString -> ByteString -> Ordering

(<) :: ByteString -> ByteString -> Bool

(<=) :: ByteString -> ByteString -> Bool

(>) :: ByteString -> ByteString -> Bool

(>=) :: ByteString -> ByteString -> Bool

max :: ByteString -> ByteString -> ByteString

min :: ByteString -> ByteString -> ByteString

Lift ByteString

Since: 0.11.2.0

Defined in Data.ByteString.Internal.Type

Methods

lift :: Quote m => ByteString -> m Exp

liftTyped :: forall (m :: Type -> Type). Quote m => ByteString -> Code m ByteString

type Item ByteString

Defined in Data.ByteString.Internal.Type

type Item ByteString = Word8

type StrictByteString = ByteString

Type synonym for the strict flavour of ByteString.

Since: 0.11.2.0

Internal indexing

findIndexOrLength :: (Word8 -> Bool) -> ByteString -> Int

findIndexOrLength is a variant of findIndex, that returns the length of the string if no element is found, rather than Nothing.

Conversion with lists: packing and unpacking

packBytes :: [Word8] -> ByteString

packUptoLenBytes :: Int -> [Word8] -> (ByteString, [Word8])

unsafePackLenBytes :: Int -> [Word8] -> ByteString

packChars :: [Char] -> ByteString

packUptoLenChars :: Int -> [Char] -> (ByteString, [Char])

unsafePackLenChars :: Int -> [Char] -> ByteString

unpackBytes :: ByteString -> [Word8]

unpackAppendBytesLazy :: ByteString -> [Word8] -> [Word8]

unpackAppendBytesStrict :: ByteString -> [Word8] -> [Word8]

unpackChars :: ByteString -> [Char]

unpackAppendCharsLazy :: ByteString -> [Char] -> [Char]

unpackAppendCharsStrict :: ByteString -> [Char] -> [Char]

unsafePackAddress :: Addr# -> IO ByteString

O(n) Pack a null-terminated sequence of bytes, pointed to by an Addr# (an arbitrary machine address assumed to point outside the garbage-collected heap) into a ByteString. A much faster way to create an Addr# is with an unboxed string literal, than to pack a boxed string. A unboxed string literal is compiled to a static char [] by GHC. Establishing the length of the string requires a call to strlen(3), so the Addr# must point to a null-terminated buffer (as is the case with "string"# literals in GHC). Use unsafePackAddressLen if you know the length of the string statically.

An example:

literalFS = unsafePackAddress "literal"#

This function is unsafe. If you modify the buffer pointed to by the original Addr# this modification will be reflected in the resulting ByteString, breaking referential transparency.

Note this also won't work if your Addr# has embedded '\0' characters in the string, as strlen will return too short a length.

unsafePackLenAddress :: Int -> Addr# -> IO ByteString

See unsafePackAddress. This function is similar, but takes an additional length argument rather then computing it with strlen. Therefore embedding '\0' characters is possible.

Since: 0.11.2.0

unsafePackLiteral :: Addr# -> ByteString

See unsafePackAddress. This function has similar behavior. Prefer this function when the address in known to be an Addr# literal. In that context, there is no need for the sequencing guarantees that IO provides. On GHC 9.0 and up, this function uses the FinalPtr data constructor for ForeignPtrContents.

Since: 0.11.1.0

unsafePackLenLiteral :: Int -> Addr# -> ByteString

See unsafePackLiteral. This function is similar, but takes an additional length argument rather then computing it with strlen. Therefore embedding '\0' characters is possible.

Since: 0.11.2.0

Low level imperative construction

empty :: ByteString

O(1) The empty ByteString

create :: Int -> (Ptr Word8 -> IO ()) -> IO ByteString

Create ByteString of size l and use action f to fill its contents.

createUptoN :: Int -> (Ptr Word8 -> IO Int) -> IO ByteString

Given a maximum size l and an action f that fills the ByteString starting at the given Ptr and returns the actual utilized length, createUptoN' l f returns the filled ByteString.

createUptoN' :: Int -> (Ptr Word8 -> IO (Int, a)) -> IO (ByteString, a)

Like createUptoN, but also returns an additional value created by the action.

Since: 0.10.12.0

createAndTrim :: Int -> (Ptr Word8 -> IO Int) -> IO ByteString

Given the maximum size needed and a function to make the contents of a ByteString, createAndTrim makes the ByteString. The generating function is required to return the actual final size (<= the maximum size), and the resulting byte array is reallocated to this size.

createAndTrim is the main mechanism for creating custom, efficient ByteString functions, using Haskell or C functions to fill the space.

createAndTrim' :: Int -> (Ptr Word8 -> IO (Int, Int, a)) -> IO (ByteString, a)

unsafeCreate :: Int -> (Ptr Word8 -> IO ()) -> ByteString

A way of creating ByteStrings outside the IO monad. The Int argument gives the final size of the ByteString.

unsafeCreateUptoN :: Int -> (Ptr Word8 -> IO Int) -> ByteString

Like unsafeCreate but instead of giving the final size of the ByteString, it is just an upper bound. The inner action returns the actual size. Unlike createAndTrim the ByteString is not reallocated if the final size is less than the estimated size.

unsafeCreateUptoN' :: Int -> (Ptr Word8 -> IO (Int, a)) -> (ByteString, a)

Since: 0.10.12.0

mallocByteString :: Int -> IO (ForeignPtr a)

Wrapper of mallocForeignPtrBytes with faster implementation for GHC

Conversion to and from ForeignPtrs

mkDeferredByteString :: ForeignPtr Word8 -> Int -> IO ByteString

Variant of fromForeignPtr0 that calls deferForeignPtrAvailability

Since: 0.11.5.0

fromForeignPtr

Arguments

:: ForeignPtr Word8

-> Int

Offset

-> Int

Length

-> ByteString

O(1) Build a ByteString from a ForeignPtr.

If you do not need the offset parameter then you should be using unsafePackCStringLen or unsafePackCStringFinalizer instead.

toForeignPtr

Arguments

:: ByteString

-> (ForeignPtr Word8, Int, Int)

(ptr, offset, length)

O(1) Deconstruct a ForeignPtr from a ByteString

fromForeignPtr0

Arguments

:: ForeignPtr Word8

-> Int

Length

-> ByteString

Since: 0.11.0.0

toForeignPtr0

Arguments

:: ByteString

-> (ForeignPtr Word8, Int)

(ptr, length)

O(1) Deconstruct a ForeignPtr from a ByteString

Since: 0.11.0.0

Utilities

nullForeignPtr :: ForeignPtr Word8

The 0 pointer. Used to indicate the empty Bytestring.

deferForeignPtrAvailability :: ForeignPtr a -> IO (ForeignPtr a)

Most operations on a ByteString need to read from the buffer given by its ForeignPtr Word8 field. But since most operations on ByteString are (nominally) pure, their implementations cannot see the IO state thread that was used to initialize the contents of that buffer. This means that under some circumstances, these buffer-reads may be executed before the writes used to initialize the buffer are executed, with unpredictable results.

deferForeignPtrAvailability exists to help solve this problem. At runtime, a call deferForeignPtrAvailability x is equivalent to pure $! x, but the former is more opaque to the simplifier, so that reads from the pointer in its result cannot be executed until the deferForeignPtrAvailability x call is complete.

The opaque bits evaporate during CorePrep, so using deferForeignPtrAvailability incurs no direct overhead.

Since: 0.11.5.0

data SizeOverflowException

The type of exception raised by overflowError and on failure by overflow-checked arithmetic operations.

Instances

Exception SizeOverflowException

Defined in Data.ByteString.Internal.Type

Methods

toException :: SizeOverflowException -> SomeException

fromException :: SomeException -> Maybe SizeOverflowException

displayException :: SizeOverflowException -> String

Show SizeOverflowException

Defined in Data.ByteString.Internal.Type

Methods

showsPrec :: Int -> SizeOverflowException -> ShowS

show :: SizeOverflowException -> String

showList :: [SizeOverflowException] -> ShowS

overflowError :: String -> a

Raises a SizeOverflowException, with a message using the given function name.

checkedAdd :: String -> Int -> Int -> Int

Add two non-negative numbers. Calls overflowError on overflow.

checkedMultiply :: String -> Int -> Int -> Int

Multiplies two non-negative numbers. Calls overflowError on overflow.

Standard C Functions

c_strlen :: CString -> IO CSize

c_free_finalizer :: FunPtr (Ptr Word8 -> IO ())

memchr :: Ptr Word8 -> Word8 -> CSize -> IO (Ptr Word8)

memcmp :: Ptr Word8 -> Ptr Word8 -> Int -> IO CInt

memcpy :: Ptr Word8 -> Ptr Word8 -> Int -> IO ()

Deprecated: Use Foreign.Marshal.Utils.copyBytes instead

deprecated since bytestring-0.11.5.0

memset :: Ptr Word8 -> Word8 -> CSize -> IO (Ptr Word8)

Deprecated: Use Foreign.Marshal.Utils.fillBytes instead

deprecated since bytestring-0.11.5.0

cbits functions

c_reverse :: Ptr Word8 -> Ptr Word8 -> CSize -> IO ()

c_intersperse :: Ptr Word8 -> Ptr Word8 -> CSize -> Word8 -> IO ()

c_maximum :: Ptr Word8 -> CSize -> IO Word8

c_minimum :: Ptr Word8 -> CSize -> IO Word8

c_count :: Ptr Word8 -> CSize -> Word8 -> IO CSize

c_sort :: Ptr Word8 -> CSize -> IO ()

Chars

w2c :: Word8 -> Char

Conversion between Word8 and Char. Should compile to a no-op.

c2w :: Char -> Word8

Unsafe conversion between Char and Word8. This is a no-op and silently truncates to 8 bits Chars > '255'. It is provided as convenience for ByteString construction.

isSpaceWord8 :: Word8 -> Bool

Selects words corresponding to white-space characters in the Latin-1 range

isSpaceChar8 :: Char -> Bool

Selects white-space characters in the Latin-1 range

Deprecated and unmentionable

accursedUnutterablePerformIO :: IO a -> a

This "function" has a superficial similarity to unsafePerformIO but it is in fact a malevolent agent of chaos. It unpicks the seams of reality (and the IO monad) so that the normal rules no longer apply. It lulls you into thinking it is reasonable, but when you are not looking it stabs you in the back and aliases all of your mutable buffers. The carcass of many a seasoned Haskell programmer lie strewn at its feet.

Witness the trail of destruction:

• https://github.com/haskell/bytestring/commit/71c4b438c675aa360c79d79acc9a491e7bbc26e7
• https://github.com/haskell/bytestring/commit/210c656390ae617d9ee3b8bcff5c88dd17cef8da
• https://github.com/haskell/aeson/commit/720b857e2e0acf2edc4f5512f2b217a89449a89d
• https://ghc.haskell.org/trac/ghc/ticket/3486
• https://ghc.haskell.org/trac/ghc/ticket/3487
• https://ghc.haskell.org/trac/ghc/ticket/7270
• https://gitlab.haskell.org/ghc/ghc/-/issues/22204

Do not talk about "safe"! You do not know what is safe!

Yield not to its blasphemous call! Flee traveller! Flee or you will be corrupted and devoured!

Exported compatibility shim

plusForeignPtr :: ForeignPtr a -> Int -> ForeignPtr b

Advances the given address by the given offset in bytes.

The new ForeignPtr shares the finalizer of the original, equivalent from a finalization standpoint to just creating another reference to the original. That is, the finalizer will not be called before the new ForeignPtr is unreachable, nor will it be called an additional time due to this call, and the finalizer will be called with the same address that it would have had this call not happened, *not* the new address.

Since: base-4.10.0.0

unsafeWithForeignPtr :: ForeignPtr a -> (Ptr a -> IO b) -> IO b

This is similar to withForeignPtr but comes with an important caveat: the user must guarantee that the continuation does not diverge (e.g. loop or throw an exception). In exchange for this loss of generality, this function offers the ability of GHC to optimise more aggressively.

Specifically, applications of the form: unsafeWithForeignPtr fptr (forever something)

See GHC issue #17760 for more information about the unsoundness behavior that this function can result in.