tiktoken: Haskell implementation of tiktoken
This packages only implements tokenization. In other words,
given an existing encoding (cl100k_base
) you can tokenize
an input.
[Skip to Readme]
Downloads
- tiktoken-1.0.3.tar.gz [browse] (Cabal source package)
- Package description (as included in the package)
Maintainer's Corner
For package maintainers and hackage trustees
Candidates
Versions [RSS] | 1.0.0, 1.0.1, 1.0.2, 1.0.3 |
---|---|
Change log | CHANGELOG.md |
Dependencies | base (>=4.15.0.0 && <5), base64 (>=1.0 && <1.1), bytestring (>=0.11.3.0), containers (>=0.5.0.0), deepseq (>=1.4.0.0), filepath, megaparsec (<9.7), pcre-light (>=0.2), raw-strings-qq, text, unordered-containers [details] |
License | BSD-3-Clause |
Author | Gabriella Gonzalez |
Maintainer | GenuineGabriella@gmail.com |
Uploaded | by GabrielGonzalez at 2024-09-02T21:19:08Z |
Distributions | |
Downloads | 108 total (6 in the last 30 days) |
Rating | (no votes yet) [estimated by Bayesian average] |
Your Rating | |
Status | Docs available [build log] Last success reported on 2024-09-02 [all 1 reports] |