The library provides a mechanism to: * handle abstraction for multiple instances of an EVS library in one application * Deliver messages * Deliver configuration changes * join one or more groups * leave one or more groups * send messages to one or more groups * send messages to currently joined groups
The EVS library implements a messaging model known as Extended Virtual Synchrony. This model allows one sender to transmit to many receivers using standard UDP/IP. UDP/IP is unreliable and unordered, so the EVS library applies ordering and reliability to messages. Hardware multicast is used to avoid duplicated packets with two or more receivers. Erroneous messages are corrected automatically by the library.
Certain guarantees are provided by the EVS library. These guarantees are related to message delivery and configuration change delivery.
All applications receive a copy of transmitted messages even if there are errors on the transmission media. This allows optimiziations when every processor must receive a copy of the message for replication.
All messages are ordered according to agreed ordering. This mechanism allows the avoidance of race conditions. Consider a lock service implemented over several processors. Two requests occur at the same time on two seperate processors. The requests are ordered for every processor in the same order and delivered to the processors. Then all processors will get request A before request B and can reject request B. Any type of creation or deletion of a shared data structure can benefit from this mechanism.
Self delivery ensures that messages that are sent by a processor are also delivered back to that processor. This allows the processor sending the message to execute logic when the message is self delivered according to agreed ordering and the virtual synchrony rules. It also permits all logic to be placed in one message handler instead of two seperate places.
Virtual Synchrony allows the current configuration to be used to make decisions in partitions and merges. Since the configuration is sent in the stream of messages to the application, the application can alter its behavior based upon the configuration changes.
The corosync executive uses a ring protocol and membership protocol to send messages according to the semantics required by extended virtual synchrony. The ring protocol creates a virtual ring of processors. A token is rotated around the ring of processors. When the token is possessed by a processor, that processor may multicast messages to other processors in the system.
The token is called the ORF token (for ordering, reliability, flow control). The ORF token orders all messages by increasing a sequence number every time a message is multicasted. In this way, an ordering is placed on all messages that all processors agree to. The token also contains a retransmission list. If a token is received by a processor that has not yet received a message it should have, a message sequence number is added to the retransmission list. A processor that has a copy of the message then retransmits the message. The ORF token provides configuration-wide flow control by tracking the number of messages sent and limiting the number of messages that may be sent by one processor on each posession of the token.
The membership protocol is responsible for ring formation and detecting when a processor within a ring has failed. If the token fails to make a rotation within a timeout period known as the token rotation timeout, the membership protocol will form a new ring. If a new processor starts, it will also form a new ring. Two or more configurations may be used to form a new ring, allowing many partitions to merge together into one new configuration.
80% of CPU utilization occurs because of encryption and authentication. The corosync can be built without encryption and authentication for those with no security requirements and low CPU utilization requirements. Even without encryption or authentication, under heavy load, processor utilization can reach 25% on 1.5 GHZ CPU processors.
The current corosync executive supports 16 processors, however, support for more processors is possible by changing defines in the corosync executive. This is untested, however.