simplexmq: SimpleXMQ message broker

This is a package candidate release! Here you can preview how this package release will appear once published to the main package index (which can be accomplished via the 'maintain' link below). Please note that once a package has been published to the main package index it cannot be undone! Please consult the package uploading documentation for more information.

[maintain] [Publish]

This package includes server, client and agent for SMP protocols:

See terminal chat prototype built with SimpleXMQ broker.

[Skip to Readme]

Properties

Versions	0.3.1, 0.3.2, 0.4.0, 0.4.0, 0.4.1, 0.5.0, 0.5.1, 0.5.2, 1.0.0, 1.0.2, 1.1.0
Change log	CHANGELOG.md
Dependencies	ansi-terminal (>=0.10 && <0.12), asn1-encoding (>=0.9 && <0.10), asn1-types (>=0.3 && <0.4), async (>=2.2 && <2.3), attoparsec (>=0.13 && <0.14), base (>=4.7 && <5), base64-bytestring (>=1.0 && <1.3), bytestring (>=0.10 && <0.11), composition (>=1.0 && <1.1), constraints (>=0.12 && <0.14), containers (>=0.6 && <0.7), cryptonite (>=0.27 && <0.30), cryptostore (>=0.2 && <0.3), direct-sqlite (>=2.3 && <2.4), directory (>=1.3 && <1.4), file-embed (>=0.0.14 && <0.0.15), filepath (>=1.4 && <1.5), generic-random (>=1.3 && <1.5), ini (>=0.4 && <0.5), iso8601-time (>=0.1 && <0.2), memory (>=0.15 && <0.16), mtl (>=2.2 && <2.3), network (>=3.1 && <3.2), network-transport (>=0.5 && <0.6), optparse-applicative (>=0.15 && <0.17), QuickCheck (>=2.14 && <2.15), random (>=1.1 && <1.3), simple-logger (>=0.1 && <0.2), simplexmq, sqlite-simple (>=0.4 && <0.5), stm (>=2.5 && <2.6), template-haskell (>=2.16 && <2.17), text (>=1.2 && <1.3), time (>=1.9 && <1.10), transformers (>=0.5 && <0.6), unliftio (>=0.2 && <0.3), unliftio-core (>=0.2 && <0.3), websockets (>=0.12 && <0.13), x509 (>=1.7 && <1.8) [details]
License	AGPL-3.0-only
Copyright	2020 simplex.chat
Author	simplex.chat
Maintainer	chat@simplex.chat
Category	Chat, Network, Web, System, Cryptography
Home page	https://github.com/simplex-chat/simplexmq#readme
Uploaded	by epoberezkin at 2021-09-11T16:12:22Z

Modules

Simplex
- Messaging
  - Simplex.Messaging.Agent
    - Simplex.Messaging.Agent.Client
    - Env
      - Simplex.Messaging.Agent.Env.SQLite
    - Simplex.Messaging.Agent.Protocol
    - Simplex.Messaging.Agent.RetryInterval
    - Simplex.Messaging.Agent.Store
      - Simplex.Messaging.Agent.Store.SQLite
        Simplex.Messaging.Agent.Store.SQLite.Migrations
  - Simplex.Messaging.Client
  - Simplex.Messaging.Crypto
  - Simplex.Messaging.Parsers
  - Simplex.Messaging.Protocol
  - Simplex.Messaging.Server
    - Env
      - Simplex.Messaging.Server.Env.STM
    - Simplex.Messaging.Server.MsgStore
      - Simplex.Messaging.Server.MsgStore.STM
    - Simplex.Messaging.Server.QueueStore
      - Simplex.Messaging.Server.QueueStore.STM
    - Simplex.Messaging.Server.StoreLog
  - Simplex.Messaging.Transport
    - Simplex.Messaging.Transport.WebSockets
  - Simplex.Messaging.Util

Downloads

simplexmq-0.4.0.tar.gz [browse] (Cabal source package)
Package description (as included in the package)

Maintainer's Corner

Package maintainers

epoberezkin, EfimPoberezkin

For package maintainers and hackage trustees

edit package information

Readme for simplexmq-0.4.0

[back to package description]

SimpleXMQ

Message broker for unidirectional (simplex) queues

SimpleXMQ is a message broker for managing message queues and sending messages over public network. It consists of SMP server, SMP client library and SMP agent that implement SMP protocol for client-server communication and SMP agent protocol to manage duplex connections via simplex queues on multiple SMP servers.

SMP protocol is inspired by Redis serialization protocol, but it is much simpler - it currently has only 8 client commands and 6 server responses.

SimpleXMQ is implemented in Haskell - it benefits from robust software transactional memory (STM) and concurrency primitives that Haskell provides.

SimpleXMQ roadmap

Streams - high performance message queues. See Streams RFC for details.
"Small" connection groups, when each message will be sent by the SMP agent to multiple connections with a single client command. See Groups RFC for details.
SMP agents cluster to share connections and message management by multiple agents (for example, it would enable multi-device use for simplex-chat).
SMP queue redundancy and rotation in SMP agent duplex connections.
"Large" groups design and implementation.

Components

SMP server

SMP server can be run on any Linux distribution without any dependencies. It uses in-memory persistence with an optional append-only log of created queues that allows to re-start the server without losing the connections. This log is compacted on every server restart, permanently removing suspended and removed queues.

To enable the queue logging, uncomment enable: on option in smp-server.ini configuration file that is created the first time the server is started.

On the first start the server generates an RSA key pair for encrypted transport handshake and outputs hash of the public key every time it runs - this hash should be used as part of the server address: <hostname>:5223#<key hash>.

SMP server implements SMP protocol.

SMP client library

SMP client is a Haskell library to connect to SMP servers that allows to:

execute commands with a functional API.
receive messages and other notifications via STM queue.
automatically send keep-alive commands.

SMP agent

SMP agent library can be used to run SMP agent as part of another application and to communicate with the agent via STM queues, without serializing and parsing commands and responses.

Haskell type ACommand represents SMP agent protocol to communicate via STM queues.

See simplex-chat terminal UI for the example of integrating SMP agent into another application.

SMP agent executable can be used to run a standalone SMP agent process that implements plaintext SMP agent protocol via TCP port 5224, so it can be used via telnet. It can be deployed in private networks to share access to the connections between multiple applications and services.

Using SMP server and SMP agent

You can either run your own SMP server locally or deploy using Linode StackScript, or try local SMP agent with the deployed demo server:

smp1.simplex.im:5223#pLdiGvm0jD1CMblnov6Edd/391OrYsShw+RgdfR0ChA=

It's the easiest to try SMP agent via a prototype simplex-chat terminal UI.

Deploy SMP server on Linode

You can get Linode free credits to deploy SMP server.

Deployment on Linode is performed via StackScripts, which serve as recipes for Linode instances, also called Linodes. To deploy SMP server on Linode:

Create a Linode account or login with an already existing one.
Open SMP server StackScript and click "Deploy New Linode".
You can optionally configure the following parameters:
- SMP Server store log flag for queue persistence on server restart (recommended).
- Linode API token for attaching server info as tags to Linode (server address, public key hash, version) and adding A record to your 2nd level domain (Note: 2nd level e.g. example.com domain should be created in your account prior to deployment). The API token access scope should be read/write access to "linodes" (to update linode tags - you need them), and "domains" (to add A record for the 3rd level domain, e.g. smp).
- Domain name to use instead of Linode ip address, e.g. smp.example.com.
Choose the region and plan according to your requirements (for regular use Shared CPU Nanode should be sufficient).
Provide ssh key to be able to connect to your Linode via ssh. This step is required if you haven't provided a Linode API token, because you will need to login to your Linode and get a public key hash either from the welcome message or from the file /etc/opt/simplex/pub_key_hash on your Linode after SMP server starts.
Deploy your Linode. After it starts wait for SMP server to start and for tags to appear (if a Linode API token was provided). It may take up to 5 minutes depending on the connection speed on the Linode. Connecting Linode IP address to provided domain name may take some additional time.
Get hostname and hash either from Linode tags (click on a tag and copy it's value from the browser search panel) or via ssh. Linode has a good guide about ssh.
Great, your own SMP server is ready! Use address#hash as SMP server address in the client.

Please submit an issue if any problems occur.

Deploy SMP server on DigitalOcean

You can deploy SMP server using SimpleX Server 1-click app from DigitalOcean marketplace:

Create a DigitalOcean account or login with an already existing one.
Click 'Create SimpleX server Droplet' button.
Choose the region and plan according to your requirements (cheapest Regular plan should be sufficient).
Provide ssh key and confirm Droplet creation.
SSH to created Droplet (ssh root@<droplet_ip_address>) to get SMP server public key hash - either from the welcome message or from /etc/opt/simplex/pub_key_hash. DigitalOcean has a good guide on how to login to Droplet via ssh.
Great, your own SMP server is ready! Use ip_address#hash as SMP server address in the client.