# P2P Multiplex Connection
MConnection is a multiplex connection that supports multiple independent streams
with distinct quality of service guarantees atop a single TCP connection.
Each stream is known as a
Channel and each
Channel has a globally unique byte id.
Channel also has a relative priority that determines the quality of service
Channel compared to other
The byte id and the relative priorities of each
Channel are configured upon
initialization of the connection.
MConnection supports three packet types:
# Ping and Pong
The ping and pong messages consist of writing a single byte to the connection; 0x1 and 0x2, respectively.
When we haven't received any messages on an
MConnection in time
pingTimeout, we send a ping message.
When a ping is received on the
MConnection, a pong is sent in response only if there are no other messages
to send and the peer has not sent us too many pings (TODO).
If a pong or message is not received in sufficient time after a ping, the peer is disconnected from.
Messages in channels are chopped into smaller
msgPackets for multiplexing.
msgPacket is serialized using Proto3 (opens new window).
Bytes of a sequential set of packets are appended together
until a packet with
EOF=1 is received, then the complete serialized message
is returned for processing by the
onReceive function of the corresponding channel.
Messages are sent from a single
sendRoutine, which loops over a select statement and results in the sending
of a ping, a pong, or a batch of data messages. The batch of data messages may include messages from multiple channels.
Message bytes are queued for sending in their respective channel, with each channel holding one unsent message at a time.
Messages are chosen for a batch one at a time from the channel with the lowest ratio of recently sent bytes to channel priority.
# Sending Messages
There are two methods for sending messages:
Send(chID, msg) is a blocking call that waits until
msg is successfully queued
for the channel with the given id byte
chID. The message
msg is serialized
WriteBinary() reflection routine.
TrySend(chID, msg) is a nonblocking call that queues the message msg in the channel
with the given id byte chID if the queue is not full; otherwise it returns false immediately.
TrySend() are also exposed for each
Each peer has one
MConnection instance, and includes other information such as whether the connection
was outbound, whether the connection should be recreated if it closes, various identity information about the node,
and other higher level thread-safe data used by the reactors.
Switch handles peer connections and exposes an API to receive incoming messages
Reactor is responsible for handling incoming messages of one
Channels. So while sending outgoing messages is typically performed on the peer,
incoming messages are received on the reactor.