archiver: Archive supplied URLs in WebCite & Internet Archive
archiver
is a daemon which will process a specified text file,
each line of which is a URL, and will (randomly) one by one request that
the URLs be archived or spidered by http://www.webcitation.org,
http://www.archive.org, and http://www.wikiwix.com for future reference.
(One may optionally specify an arbitrary sh
command like wget
to download URLs locally.)
Because the interface is a simple text file, this can be combined with other scripts; for example, a script using Sqlite to extract visited URLs from Firefox, or a program extracting URLs from Pandoc documents. (See http://www.gwern.net/Archiving%20URLs.)
For explanation of the derivation of the code in Network.URL.Archiver
,
see http://www.gwern.net/haskell/Wikipedia%20Archive%20Bot.
Downloads
- archiver-0.6.2.1.tar.gz [browse] (Cabal source package)
- Package description (as included in the package)
Maintainer's Corner
For package maintainers and hackage trustees
Candidates
- No Candidates
Versions [RSS] | 0.1, 0.2, 0.3, 0.3.1, 0.4, 0.5, 0.5.1, 0.6.0, 0.6.1, 0.6.2, 0.6.2.1 |
---|---|
Dependencies | base (>=4 && <5), bytestring, containers, curl, HTTP, network, process, random [details] |
Tested with | ghc ==6.12.1 |
License | BSD-3-Clause |
Author | Gwern |
Maintainer | Gwern <gwern@gwern.net> |
Category | Documentation, Network |
Source repo | head: git clone git://github.com/gwern/archiver-bot.git |
Uploaded | by GwernBranwen at 2014-01-03T19:44:15Z |
Distributions | |
Reverse Dependencies | 1 direct, 0 indirect [details] |
Executables | archiver |
Downloads | 9265 total (33 in the last 30 days) |
Rating | (no votes yet) [estimated by Bayesian average] |
Your Rating | |
Status | Docs available [build log] Successful builds reported [all 1 reports] |