Miscellaneous Channel Diagrams

[Who]

In my search for all the Golang channels, I stumbled across several that were their own independent ecosystem and would just add clutter if included in the bigger diagrams. They're less interesting in terms of relevancy to the protocol but still an important function inside the code.

This thread is going to be a collection for these.

Legend for Channel Diagrams:
Clouds are external areas, or areas not detailed in this diagram.
Dot-Outlined areas are the entities responsible for this behavior.
Green boxes are Golang channels. The title bar consists of entity.channelname, type, capacity as they are defined with default values.
Top Level Containers are specific functions responsible for reading or sending on channels. I included the function name (minus parameters passed) in the title bar, so you can search the codebase to find out what they do in detail
Containers inside Top Level Containers are either subfunctions or just "steps" that I thought were worth noting.
Lists inside Top Level Containers are the ones with dashed separators and [*] in the title. They're a list of valid message types.

Arrows going to and from channels denote a message being added or read from a channel. I tried to label them as best as I could with the actual message type sent along the path, where asterisks denote multiple types.
Olive arrows mean there's a loop involved.



First up: ShutdownChan

This channel is what happens when someone hits Ctrl+C on their keyboard. The arrows are mostly unlabeled in this one because the underlying code triggers on any input. The code just sends "0" to these channels. Another oddity is that apart from State, all the channels are available in local scope only.

Q: I don't care how this works, I just want to use it. What do I need to know?
A: Pass a function to AddInterruptHandler() and it will be executed when the system is shut down via console.

Diagram:


Direct Link
View original on draw.io

 

[Who]

InputChan

This one is about as small as it gets. SimControl runs pretty much "outside" of the node itself and just sends various signals inside. There are a lot of different options.

Q: I don't care how this works, what can I do with it?
A: Type "h" in the factomd console for a list of functions and what they do

Q: How do I add my own code to run when I type "abc"?
A: Add it to this 1,300 line switch statement

Diagram:


Direct link
View original on draw.io

 

[Who]

Missing Message Requests (MMR)

This is a helper system that is responsible for sending out requests for missing messages inside a ProcessList. It ties together a few loose ends and seems to be more or less a buffer that limits the number of requests made.

Q: What does this do?
A: It's a listener that reacts to events from the rest of the system. The system tells it "I need this data" (asks) and "I have this data now" (adds) and the MMR sends out requests for the ones it still needs until you get to a new block.

Q: Where are the inputs coming from?
A:
1. Every time the node moves to a new height, it sends a message over "state.dbheights".
2. Whenever the VM Process() detects that it has a missing message somewhere, it sends a request to "state.ask".
3. Whenever a message is added to a ProcessList, it sends a notification to "state.adds". (This may be the reply to a pending one)

Those three steps are what fills and depletes the pending requests. The ticker periodically sends all the pending requests to the network. (Every 12 seconds on mainnet)

Q: What's the difference between Sync Entries and MMR?
A: Sync Entries is for the data contained in entries. MMR is for consensus messages in the process lists.

Legend (new):
Brown boxes are Golang maps, which are used mainly as holding areas for messages from queues in this diagram.


Diagram:


Direct Link
Original on draw.io

 

[Who]

SyncEntries aka "Second Pass"

This is the system that ensures every node has all of the entry data. It consists of two goroutines, one (GoSyncEntries) that goes through the database block by block to add existing entries to the replay system and detect which ones are missing, and the other (MakeMissingEntryRequests) that requests those missing entries from the network and saves them in the database when they arrive.

Q: Does this sync entry commits and hashes, or entry data?
A: Just the data itself. The commits and hashes are in the directoryblocks (more specifically the entryblocks)

Q: Who fulfills the requests for missing entries?
A: It picks a random peer from the list. If the request is still in the system after 5 seconds, it sends another request to a random peer.

Q: What's the difference between Sync Entries and MMR?
A: Sync Entries is for the data contained in entries. MMR is for consensus messages in the process lists.

Diagram:


Direct Link
Original on draw.io

 

[Paul B.]

Question about the MMR part: how does a node knows it is missing a message?

 

[Who]

Question about the MMR part: how does a node knows it is missing a message?

The protocol goes through a lot of effort to ensure that the messages in the processlist are in the same order for everyone, which is implemented by giving every message/ack a specific "height" in the processlist. Since it's a p2p network, messages can arrive in any order or get lost, so sometimes there are gaps between two heights. Those are the missing messages. The code will also request messages for maximum height seen + 1.

 

[Who]

This turned out to be somewhat smaller than I expected because, once again, most of the logic is contained via the messages.

Q: What does an Election do?
A: Replace unresponsive fed servers with audit servers

Q: How does an election get started?
A: Every Minute, factomd starts a "Fault()" timer. After the specified time (two minutes on mainnet) is up, it checks whether or not all servers have sent an EOM/DBSig message for that Minute. If messages are missing, it starts an election to replace them.

Q: I see Elections.Input but there's also an Elections.Output in the code. Why isn't that on this diagram?
A: Elections.Output is just a reference to State.inMsgQueue but all the messages using it interact with State.inMsgQueue directly.


Diagram:


Direct Link
Original on draw.io