Consensus Channel Diagram aka ValidatorLoop()

I have to admit, this one did not turn out the way I wanted it to. The validator loop is the beating heart of the factomd node, responsible for keeping the consensus mechanism going along. It receives messages from the network, executes them, updates the state, updates the VMs, runs the VMs, saves to the database, sends and receives data to so many areas.

At first I set out to make this as detailed as the other diagrams but the complexity and interaction of messages made that an impossible task, particularly as I'm not fluent in that aspect myself yet. That's why I ended up making a simplified diagram that shows what I intended these diagrams to be: the flow of messages along channels in the system. I'm making liberal use of clouds to indicate polymorphic areas and am limiting accuracy to the consumers of channels rather than listing all the producers.

Q: So you're saying this is too complex to be useful?
A: Yes and no. It's more for orientation when you come across a message in the code. You can look at the channel it arrived from or the channel it goes to and see what's going to happen next.

Q: Which messages are those, exactly? What's an IMsg?
A: IMsg is the interface for internal messages. They're located in factomd/common/messages, or you can look at the constants for a list of message types.

Some of the clouds have their own chart:
* Networking
* EntrySync
* Interrupts
* MMR

Legend:
Clouds are external areas, or areas not detailed in this diagram.
Pink outlines mean the behavior is polymorphic and what happens depends on the message code.
Dot-Outlined areas are the entities responsible for this behavior.
Green boxes are Golang channels. The title bar consists of entity.channelname, type, capacity as they are defined with default values.
Brown boxes are Golang maps, which are used mainly as holding areas for messages from queues in this diagram.
Top Level Containers are specific functions responsible for reading or sending on channels. I included the function name (minus parameters passed) in the title bar, so you can search the codebase to find out what they do in detail
Containers inside Top Level Containers are either subfunctions or just "steps" that I thought were worth noting.

Arrows going to and from channels denote a message being added or read from a channel.
Olive arrows mean there's a loop involved.

And a one time thing, the funny magnifying glass arrow means a closer look into this area.

First up is the simple chart:
ValidatorLoopSimple.png

Original on draw.io


Sidenote #1:
This is one of the central routing functions for messages so this code is very complex.
* It tries to send out all valid messages but only the ones not flagged as local are actually sent
* Commits and Reveals are also added to the state.Holding map, in addition to everything else
* "Is Leader" is actually a conditional with 8 conditions which boils down to "is leader and is all caught up and stuff"
* if leader messages are received before the processlist is initiated properly, they are added to state.XReview

And here's the more complex chart:
Consensus.png

Original on draw.io


If you have any ideas on how to improve these, please let me know.
 
Last edited:
Top