hw-json-simd: SIMD-based JSON semi-indexer

[ bsd3, data, library, program ] [ Propose Tags ] [ Report a vulnerability ]

Please see the README on GitHub at https://github.com/haskell-works/hw-json-simd#readme

[Skip to Readme]

Modules

[Index] [Quick Jump]

HaskellWorks
- Data
  - Json
    - Simd

Flags

Automatic Flags

Name	Description	Default
avx2	Enable avx2 instruction set	Disabled
bmi2	Enable bmi2 instruction set	Disabled
sse42	Enable SSE 4.2 optimisations.	Enabled

Use -f <flag> to enable a flag, or -f -<flag> to disable that flag. More info

Downloads

hw-json-simd-0.1.1.2.tar.gz [browse] (Cabal source package)
Package description (revised from the package)

Note: This package has metadata revisions in the cabal description newer than included in the tarball. To unpack the package including the revisions, use 'cabal get'.

Maintainer's Corner

Package maintainers

newhoggy, haskellworks

For package maintainers and hackage trustees

edit package information

Candidates

0.1.0.0, 0.1.0.1

Versions [RSS]	0.1.0.0, 0.1.0.1, 0.1.0.2, 0.1.0.3, 0.1.0.4, 0.1.1.0, 0.1.1.1, 0.1.1.2
Change log	ChangeLog.md
Dependencies	base (>=4.11 && <5), bytestring (>=0.10.6 && <0.13), hw-json-simd, hw-prim (>=0.6.2.35 && <0.7), lens (>=4 && <6), optparse-applicative (>=0.14 && <0.19), vector (>=0.12 && <0.14) [details]
Tested with	ghc ==9.4.2, ghc ==9.2.4, ghc ==9.0.2, ghc ==8.10.7, ghc ==8.8.4, ghc ==8.6.5
License	BSD-3-Clause
Copyright	2018-2021 John Ky
Author	John Ky
Maintainer	newhoggy@gmail.com
Revised	Revision 1 made by haskellworks at 2024-01-09T10:56:23Z
Category	Data
Home page	https://github.com/haskell-works/hw-json-simd#readme
Bug tracker	https://github.com/haskell-works/hw-json-simd/issues
Source repo	head: git clone https://github.com/haskell-works/hw-json-simd
Uploaded	by haskellworks at 2022-12-19T23:27:24Z
Distributions	LTSHaskell:0.1.1.2, Stackage:0.1.1.2
Reverse Dependencies	3 direct, 6 indirect [details]
Executables	hw-json-simd
Downloads	3563 total (17 in the last 30 days)
Rating	(no votes yet) [estimated by Bayesian average]
Your Rating	λ λ λ
Status	Docs uploaded by user Build status unknown [no reports yet]

Readme for hw-json-simd-0.1.1.2

[back to package description]

hw-json-simd

This library/tool will generate semi-indexes on JSON files as per the paper: Semi-Indexing Semi-Structured Data in Tiny Space.

The command line tool

For a given JSON file, hw-json-simd will generate two semi-index files, which both together can be loaded into a single in-memory semi-index.

The semi-index files can be generated using two methods, which will be called standard and simple, which correspond to sections 4 and 5 of the Semi-Indexing Semi-Structured Data in Tiny Space paper respectively.

Navigation of the JSON text using the standard index is suppored by the hw-json project for more information. There is currently no support for navigation of the JSON text using the simple index.

Using on the command line

cat test.json | pv -t -e -b -a | time hw-json-simd create-index \
  -i /dev/stdin
  --output-ib-file test.json.ib.idx
  --output-bp-file test.json.bp.idx
  --method standard

cat test.json | pv -t -e -b -a | time hw-json-simd create-index \
  -i /dev/stdin
  --output-ib-file test.json.ib.idx
  --output-bp-file test.json.bp.idx
  --method simple

Installation

From hackage

cabal new-install hw-json-simd