One of the key components of the Vocdoni ecosystem is the Vochain, a full blockchain optimized for digital voting. We have already introduced it on the technical overview v1 and also in the Vochain explorer article.

Differently from other current layer 2 voting solutions, in order to achieve decentralization, scalability and security, our design is based on a distributed state machine that holds the state of current ongoing voting processes in addition to previous, already finished ones.

After one year of development on this scope, the Vochain is already deployed and being used by some early-access participants. With this article we will explain what a distributed state machine means, the technology we use as basis for our  blockchain (Tendermint), some internal details of the Vochain and the very promising results of a benchmark we recently ran.

Distributed state machines

A Blockchain is mainly a distributed state machine. By state, we mean a set of bytes ordered in some specific way. For instance, a state could be the complete list of tweets that all users of a system have published.

In this case we could have state like this (State 1):

{
   "State":{
      "users":[
         "user1",
         "user2",
         "user3"
      ],
      "tweets":{
         "user1":[
            "hello world",
            "I am hungry"
         ],
         "user2":[
            "my name is John"
         ],
         "user3":[
            "today is sunday",
            "I am happy"            
         ]
      }
   }
}

The state transition (from one state to the next) is called a transaction. So a transaction is executed in order to migrate from one state to another.

The following JSON message could be an example transaction:

{ "tx1": {"type":"addTweet", "user":"user1", "data":"I like Satoshi"} } 

And the new state (State 2) will be the following:

...
      "tweets":{
         "user1":[
            "hello world",
            "I am hungry",
            "I like Satoshi"
         ],
...
}

The rules for accepting and executing transactions on the current state are defined within the blockchain logic. If users are identified by a public key, for instance, the logic might require a signature proof in order to demonstrate that user1 is really the one sending the transaction.

Simplifying the case of Bitcoin, for example, the state is the number of tokens (Bitcoins) that each address has. A transaction is a state transition from one list of address:amount to another. A transaction is only valid if the sender demonstrates their ownership of the address moving the tokens by using its privateKey.

Fine, so this does not seem that complicated, right? Well, to be honest, state machines have existed at least since Alan Turing wrote the theory behind his famous machine. The great contribution of the Satoshi paper, which unveiled Bitcoin, is a way to make a state machine distributed and avoid trust between the participants.

The state of such a machine must be deterministic in order to allow all the nodes of the network to calculate exactly the same output after a transaction is executed, and also execute the transactions in the same order. If a single bit becomes different between two or more nodes, the chain is forked and there is a consensus failure. This is why consensus protocols such as Proof-of-Work or Proof-of-Stake and the figure of miners (those nodes selecting, ordering and committing transactions) are so important.

As an anecdote, on the first proof of concept of the Vocdoni blockchain, we were using a Golang Map to save some state data (serialized on a KV store). As maps in Go do not have a deterministic order, we came up against consensus failure the moment we added a second validator to the blockchain.

To fix this problem, most of the Blockchains nowadays implement Merkle Trees for storing the state, because they: 1) Are deterministic 2) Can easily be summarized on a Root Hash 3) Allow for the creation of merkle proofs, a mechanism to demonstrate to a third party that a data set is included inside a merkle tree without providing the whole data structure (just the root and the siblings are needed).

All nodes of the network (miners and full nodes) must be able to fetch the transactions and execute them in order to calculate the same set of transition states and thus the same block hash.

In Ethereum the transactions are executed by the EVM (Ethereum Virtual Machine) which is a Turing complete virtual machine that every node includes.

Tendermint

Tendermint is an amazing project that aims to provide a framework for implementing distributed state machines. Tendermint handles the Networking and Consensus layers while requiring developers to implement only the Application layer.

In order to let users build the application layer, Tendermint offers three main steps (actually more, but let's focus on these three):

  1. checkTX: logic to decide if a transaction should be accepted in the mempool or not (thus broadcasted to other nodes)
  2. deliverTX: logic to execute the transaction and modify the state
  3. commit: operations for saving the state on a persistent storage and marking it as valid

Then the Cosmos SDK stack is a huge set of predefined modules and libraries that can be easily used for building this application logic. In Vocdoni we decided to go low-level so we are not using anything from the Cosmos SDK.

Tendermint integration for Vocdoni

Tendermint exposes a RPC endpoint which uses the ABCI protocol. The standard way to use this protocol is to program a standalone piece of software that will act as an application layer and will communicate with the Tendermint core through a network or unix socket.

[ Tendermint Core ] <--- ABCI/RPC ---> [ Application ]

This is great because it really simplifies the programming of the application and is very flexible. You can use any programming language that is able to speak ABCI. But in Vocdoni we decided not to follow this approach. Firstly because it requires us to run two different programs (daemons) on the host system, secondly because the RPC connection can become a bottleneck and it might be unstable, third because we like to have control over everything we deal with.

So we decided to use Tendermint as a GoLang library instead, and build everything on a single program. To communicate with tendermint we directly use this library in order to avoid any RPC connection. This was hard at first (there was no documentation) but now we are happy with this choice.

Here the code that starts Tendermint:
https://github.com/vocdoni/go-dvote/-/blob/master/vochain/start.go

And here is our main Application layer:
https://github.com/vocdoni/go-dvote/-/blob/master/vochain/app.go

The Vochain: Vocdoni's state machine

So now we are ready to discuss our voting blockchain, codename Vochain, which it is designed to be a very fast, purpose-specific, layer 2 sidechain.

The state of the blockchain is composed by three merkle trees, currently tendermint IAVL trees:

  1. Application: stores all the metadata related to the miners, oracles, block details, height, etc.
  2. Processes: stores all the election processes
  3. Votes: stores all the votes sent to an election process

The main function of the Vochain is to count and store votes. Votes can only be added if they belong to an active election process. The way to create an election process is via a special transaction named newProcessTx. This transaction requires the signature of a valid Oracle which is constantly monitoring the Ethereum Smart Contract.

This is how a newProcessTx looks in our GoLang Vochain implementation.

type NewProcessTx struct {
	EntityID string  // ethereum address or the organization
	MkRoot string  // census merkle root
	MkURI string  // census ipfs URI
	NumberOfBlocks int64 // duration
	ProcessID string // unique identifier for the election
	ProcessType string // type of election process
	StartBlock int64  // start block on Vochain
	Signature string // oracle signature
}

This transaction will modify the Process Tree data structure of the state.

Once the process is included in the Vochain and the current block is between StartBlock and StartBlock+NumberOfBlocks, the elegible voters (those that can prove they belong in the census merkle root) can send VoteTx transactions.

type VoteTx struct {
	EncryptionKeyIndexes []int // if private election, the encryption keys used
	Nonce string // nonce for entropy
	Nullifier string // deterministic unique identifier for the vote
	ProcessID string // unique identifier for the election
	Proof string // census proof (merkle-proof or zk-snark)
	VotePackage string // the vote package (mainly an array of integers)
	Signature string // if non anonymous, the signature of the voter
}

This transaction will modify the Vote Tree data structure of the state.

Note that for sending transactions on the Vochain there is no need to hold any kind of token or spend any kind of Gas. The sender must either be a valid Oracle or a valid Voter for an existing election process.

More information regarding the Vochain implementation can be found on the documentation page.

A very good tool to visualize the Vochain state is the explorer we recently published.

Benchmarking

One of Vocdoni's main focuses is the scalability of voting. Most of our implementation decisions (such as avoiding the tendermint RPC connection) were made in order to maximize the number of transactions per second the Vochain is able to handle.

Currently we have successfully achieved around 300 TPS (18 000 votes per minute) in our development environment:

  • 6 validators (2 VCPU and 4GB of RAM, around $7 VPS)
  • 3 gateway full nodes (4 VCPU and 16GB of RAM, around $18 VPS)

The voting benchmark was performed by the vochaintest tool from another VPS spreading the vote transactions among the three gateways via the websockets API connection.

go run cmd/vochaintest/vochaintest.go --operation=vtest --electionSize=200000 --oracleKey=0x7BA... --parallelCons=12 --doubleVote=False --gwHost=ws://127.0.0.1:9443/dvote  --gwExtra=ws://gateway2,ws://gateway3

An election of 200.000 voters was handled in around 11 minutes. The following chart shows the votes per minute and the block time and size.

More graphics and details regarding this benchmark can be found on this grafana snapshot

Benchmark Conclusions

We are very happy with these results. Taking into account that the miners are low-resource machines, we think that just updating the hardware, the TPS achieved could increase greatly. There are still some to-do optimizations planned (such as mempool checkTx parallelization) that could increase the bandwidth of the blockchain even further.

Our mid-term objective is to reach 500 TPS. Taking into account that the Vochain sidechains can also scale horizontally (by using some kind of sharding) for a single election process, we may soon be ready to handle nation-size elections: two sharded parallel Vochains able to process 500 TPS would be able to handle 86 000 000 votes in 24 hours.