Layered structure of the blockchain architecture

This section covers the layered architecture of blockchain. In this section, we will be diverging to Ethereum and Hyperledger Fabric as well. While discussing the Hyperledger Fabric infrastructure, we will drill into the OBP's infrastructure as well.

The following diagram displays the layered architecture of blockchain:

Blockchain layered architecture

Hardware and infrastructure layer

The content of the blockchain is hosted on a server that resides in a data center on this beautiful planet. While browsing the web or using any applications, clients request content or data from application servers, commonly referred to as client-server architecture. However, today, clients can connect with peer clients as well and share data among each other. Such a massive collection of computers sharing data with each other is termed a P2P network. Blockchain is a P2P network of computers that computes transactions, validates them, and stores them in an ordered form in a shared ledger. This results in a distributed database that records all the data, transactions, and other relevant information. Each computer in a P2P network is called a node. Nodes are responsible for validating transactions, organizing them into blocks, broadcasting them to the blockchain network, and so on. Upon reaching consensus, nodes commit the block to the blockchain network and update their local ledger copy. This layer comprises of virtualization (creation of virtual resources such as storage, network, servers etc.). Significantly, nodes are the core of this layer. When a device gets connected to a blockchain network, it is termed and considered as a node. On a blockchain network, these nodes are decentralized and distributed.

Ethereum - Infrastructure layer

Let's look at the nodes in Ethereum. Anyone and everyone can run an Ethereum node on their machine. This will enable their machine to participate on the Ethereum blockchain network. Nodes can run a client (compatible client), such as Geth, Parity, or Pantheon, to connect to Ethereum blockchain. Geth is written in Go, Parity is written in Rust, and Pantheon is written in Java. A node (node-running client), can be either a light node (client) or a full node (client). Light nodes store the cache, while full nodes (client) store the dataset, which grows linearly with time. Light nodes (clients) get high assurance from the Ethereum blockchain network about the state of the Ethereum, and they can participate to verify the execution of transactions. On the other hand, any node that participates in the full enforcement of consensus and downloads the entire blockchain to the node’s local storage is known as a full node (client). Full nodes verify signatures, format the data of the transactions and blocks, check double spending, and so on. They essentially validate the transactions and use a gossip protocol to relay this information to other nodes, called peers.

These Ethereum nodes (clients) run the Ethereum Virtual Machine (EVM). EVM is a Turing complete software; a stake-based virtual machine that enables untrusted code to be executed by a global P2P network of computers. EVM handles the internal state and computation. Ethereum blockchain is a Turing complete blockchain where developers can also develop programs (smart contracts) for the blockchain. EVMs are like JVMs, and they run on each node on the blockchain. EVMs are like transaction engines, which are responsible for changing the blockchain's world state. EVMs run as sandboxes and offer an execution environment for the smart contracts.

Hyperledger Fabric – Infrastructure Layer

This blockchain network, which is based on Hyperledger Framework, is comprised of peer nodes, and these nodes host ledgers and chaincode (also known as smart contracts). Essentially, peers host instances of the ledger and chaincode, which keeps an eye on single points of failure. As peer nodes are responsible for hosting the ledger and chaincode, applications and administrators need to interact with these peer nodes. A node in Hyperledger Fabric can host more than one ledger. In some cases, a peer node can only host a ledger, and not the chaincode (it is rare, but possible). Most nodes have at least one chaincode installed to update or query the node’s ledger. A node can host multiple chaincode and multiple ledgers too, which are powered by channels. You will learn more about channels and the Hyperledger Fabric architecture in Chapter 3, Delving into Hyperledger Fabric.

To access the chaincode or ledger, applications and administrators (via admin applications) will always connect with peers via Fabric software development kit (SDK) APIs. These APIs allow applications to execute transactions on the blockchain network and receive events related to the confirmation of the process. There are two types of transaction—query and update transactions. For query transaction, consensus is not required, as the peer will return the result immediately from its local copy of the ledger. However, for update transactions, no individual peer can update the ledger because other peers need to agree before updating the ledger. This process of reaching an agreement to update the ledger is termed consensus. You can read more about the ledger-update transaction process in Chapter 3, Delving into Hyperledger Fabric.

A specific set of applications and peers can communicate via channels, since a channel is a partition – a pathway of communication – between the specific application and peer(s). Hyperledger Fabric is for enterprises and it caters to private-permissioned (consortium) and private-permissionless use cases. Various like-minded organizations form a consortium to build a blockchain business network. Hence, peers are owned by various organizations. These organizations offer resources for the setup, maintenance, and operations of the blockchain network. One of the resources is nodes (peers), and the business network can continue to exist as long as one organization with one peer remains alive on the blockchain business network.

The administrators of that organization assign nodes to the blockchain business network. Each organization has a certificate authority, which assigns a digital certificate to these nodes. This digital certificate (X.509) is the digital identity of these peers. This digital identity helps in identifying the owning organization of the peer when the peer tries to connect to a channel on the blockchain business network. A channel has policies, which determine the rights and privileges of the peer. This mapping of the peer’s role in an organization and the peer’s identity to an organization is provided the Membership Service Provider (MSP). Anything that interacts with the blockchain business network, such as peers, applications, admins, orderers, and so on, must have an identity and an associated MSP to enable their integration with it.

Orderers nodes ensure the consistency of the ledger across the blockchain business network. Let’s take a quick glance at the transaction flow of a blockchain network based on Hyperledger Fabric. This entire process is mediated by orderers (orderers' nodes), where all the peers reach a consensus on the content of the transaction, and also the order of the transactions.

Transaction Flow: Transactions in a Hyperledger Fabric-based blockchain network happen in a multi-phase process. Please visit the Transaction flow section of Chapter 3, Delving into Hyperledger Fabric, for more details. Here is the glimpse of the transaction flow:

1. First phase (endorsing phase): The application initiates a ledger-update transaction. This transaction request is handled by endorsing peers (nodes as endorsers). These nodes endorse the proposed ledger update and send the endorsement to the application. However, no commits are performed to the ledger.
2. Second phase (ordering phase): Proposal response from endorsed transaction, from various applications are received by the orderers' nodes. These nodes order the transactions into blocks.
3. Third phase (distribution phase): Finally, ordered blocks are distributed to all the peers in the blockchain business network. These peers will validate the transaction and, upon successful validation, commit the transaction to the local copy of the ledger.

Orderers' nodes are the mediators of the entire transaction process. This transaction process is known as a consensus, as all the peers in the blockchain business network have agreed about the transactions and the data of the transactions. The ordering service leverages a message-oriented architecture, Ordering service can be implemented in one of the following three ways:

• Solo
• Kafka
• Raft

The following table highlights the features of various order service implementations:

Features	Solo	Kafka	Raft
Number of nodes	Single ordering node	Multiple ordering nodes from one organization.	Multiple ordering nodes from different organizations.
Fault-tolerant	Not fault-tolerant	Crash fault-tolerant (CFT), which uses a ledger and follows node configuration. Uses ZooKeeper ensemble for management.	CFT based on Raft protocol.
Implementation	Development and testing (not for production)	Production grade, however has management overhead.	Production grade and easier to implement than Kafka.
Distributed order service		Kafka clusters are practically run by one organization (maybe a founder organization). Hence, all the ordering actually goes to one single organization. Hence it's partially decentralized.	Allows distributed ordering service as various organizations can contribute their nodes to form a distributed order service. Hence, it is fully decentralized.

From a physical presence perspective, nodes can reside in one of the following locations:

• Cloud tenanted or owned by one of the organizations
• Data center or on-premise owned by one of the organizations
• On a local machine

Essentially, identity of the peer associates its affiliation with an organization and determines that it is owned by that organization. These nodes are the basis and core of the blockchain network. They are of different types and perform various functions, such as endorsement, ordering, committing, and hosting chaincode, and ensure the consistency of the ledger. So far, we've discussed infrastructure from an Ethereum and Hyperledger Fabric perspective. If you want to check the infrastructure offerings for a specific vendor, you can visit the Oracle's Baas – OBP section of this chapter.

Data layer

Blockchain is a decentralized, massively replicated database (distributed ledger), where transactions are arranged in blocks, and placed in a P2P network. The current state of all accounts is stored in such a database. A network (public or private) is comprised of many nodes and without a common consensus, data cannot be altered. The data structure of a blockchain can be represented as a linked-list of blocks, where transactions are ordered. The blockchain's data structure includes two primary components—pointers and a linked list. The pointers are the variables, which refer to the location of another variable, and linked list is a list of chained blocks, where each block has data and pointers to the previous block. A Merkle tree is a binary tree of hashes. Each block contains a hash of the Merkle root with information such as the hash of the previous block, timestamp, nonce, the block version number, and the current difficulty target. A Merkle tree offers security, integrity, and irrefutability for the blockchain technology. Merkle trees, along with cryptography and consensus algorithms, are the basis of the blockchain technology. For example, Ethereum blockchain uses a Patricia tree database to store information. Patricia tree (Trie) is a Merkle tree, which is like a key-value store. Just like Merkle tree, a Patricia tree has a root hash. This root hash can be used to refer to the entire tree. Hence, you cannot modify the content of the tree without modifying the root hash. Each block contains a list of transactions that happened since the last block, and after applying those transactions, the root hash of the Patricia tree represents the new state (state tree).

The genesis block (the first block) does not contain the pointer, as it's the first in the chain. The following diagram shows the connected list of blocks in a blockchain:

Blockchain structure

Depending on the type of blockchain, data is stored in blocks. For example, Hyperledger Fabric's blocks will include channel information, while a Bitcoin blockchain will have data about the sender, receiver, and the amount. We've used the term hash a few times already. A hash is a unique digest of the data. A cryptographic hash algorithm (such as the SHA 256 algorithm) can generate a fixed length hash value of the data. These hashes help in identifying blocks easily and also help to detect any changes that are made to the blocks. Each block has a hash of the previous block; hence, blockchain is essentially a chain of hashes. Any new node connected to the blockchain will receive a copy of the blockchain network. Only upon consensus are blocks added to the local blockchain.

Transactions are digitally signed on the blockchain to ensure the security and integrity of the data stored on it. They secure information about the block, transactions, transacting parties, and so via a digital signature, which uses asymmetric cryptography. Transactions are signed using a private key, and anyone in possession of the public key can verify the signer. The digital signature checks for tampering. Digital signatures guarantee integrity as the data that is encrypted is also signed. Hence, any tampering will invalidate the signature. As the data is already encrypted, it cannot be detected. Even if it is detected, it cannot be tampered with. A digital signature secures the sender's (owner) identity as well. Private keys are linked to owners (users); hence, signatures are legally associated with the owner and cannot be repudiated. In this section, we talked about transactions in detail. We will walk through the transaction flow for Ethereum-based blockchain platforms in the next section.

Network layer

The network layer, also known as the P2P layer, is the one that is responsible for internode communication. It takes care of discovery, transactions, and block propagation. This layer can also be termed as propagation layer. This P2P layer ensures that nodes can discover each other and can communicate, propagate and synchronize with each other to maintain valid current state of the blockchain network. Visit the following Transaction flow subsection, in this chapter to experience the P2P layer in terms of transaction broadcast, transaction proposals, transaction validation and transaction commit. This layer also takes care of the world state propagation. A P2P network is a computer network where computers (nodes) are distributed and share the network’s workload to reach the end goal. Nodes perform transactions on the blockchain. There are two kinds of nodes—full node and light node. Full nodes ensure the verification and validation of transactions, mining, and the enforcement of consensus rules. They are responsible for maintaining trust in the network. Light nodes only keep the header of the blockchain (keys) and can send transactions.

Transaction flow

The transaction flow described here highlights the interaction between nodes in a P2P network layer. The following diagram displays the transaction flow for the Ethereum blockchain:

Blockchain transaction flow

The following bullet points highlight the transaction flow of Ethereum blockchain, as shown in the preceding diagram:

• Transaction Initiated: A light node or a full node that has an Ethereum client digitally signs and initiates the transaction
• Local Node Verification: Once the local Ethereum node receives the transaction, the local Ethereum node will perform the following checks:
  • Check the digital signature's consistency with the sender's address and transaction's content
  • Check if the sender has enough gas to fuel the transaction being processed
  • Check if the transaction will result in a smart contract's functions to fail

• Broadcast to Blockchain Network: Transactions are broadcasted to the blockchain P2P network, where the full nodes will perform the following checks:
  • Repeat the aforementioned validation checks
  • Full nodes communicate with each other
  • Miner nodes will put the transaction in a pending block and start the consensus (for example, PoW) so that it can try to solve the puzzle
• Validated, mined, and added to the block: Following steps ensure the block validation and addition to blockchain :
  
  • Once the transaction is validated and mined, it's added to the valid block
  • The miner will solve the puzzle and find the valid block, which will be added to the blockchain
• Valid Block added to the Blockchain: The valid block is appended to the blockchain and broadcasted to nodes:
  • The miner node will append the valid block to the blockchain.
  • The miner will broadcast the valid block to nodes.
  • Upon appending the block to the blockchain, the miner will broadcast this to blockchain peer nodes, and each node will again validate the block to ensure the consistency of the current (new) block with the previous block. Once validation is successful, the node will broadcast to its nodes and so on.
• Transaction Completed: The initiating node will sync the local copy with the blockchain and execute the transaction:
  • Finally, the block reaches the node that initiated the transaction
  • The local node will sync its local copy with the blockchain and will execute the transactions in the block
  • Transaction is marked as completed

Consensus layer

The consensus protocol is the core to the existence of blockchain platforms. As the saying goes, behind every blockchain, there is a consensus algorithm. The consensus layer is the most critical and crucial layer for any blockchain (Ethereum, Hyperledger, or any other). Consensus is responsible for validating the blocks, ordering the blocks, and ensuring everyone agrees on it. The following are the key points regarding the consensus layer:

• Consensus protocols (algorithms) create an irrefutable set of agreements between nodes across the distributed P2P network.
• Consensus keeps all the nodes synchronized. Consensus ensures that all the nodes agree to the truth.
• Consensus ensures that power remains distributed and decentralized. No single entity can control the entire blockchain network.
• Consensus guarantees that a single chain is followed and that it holds the truth.
• Consensus is the rules that nodes follow to ensure that transactions are validated within the boundaries of those rules and that blocks follow those rules.
• Consensus results in unanimous acceptance of truth among the participating nodes.
• For a blockchain with cryptocurrency (for example, Ethereum), consensus also rewards the nodes for validating the transactions and maintaining the blockchain network.
• By design, consensus protocols cannot be replicated as replication or imitating them is costly and time-consuming.
• Reliability in a P2P network is achieved by a consensus protocol.

Consensus methods vary for different types of blockchain. For example, consensus, when followed by a permissionless blockchain network such as Ethereum, Bitcoin, and so on, is known as a probabilistic consensus. Such a consensus guarantees consistency of the ledger, though there is a possibility that various participants have different views of the blocks. This means that they remain vulnerable to ledger forks (also known as divergent ledgers). Permissioned blockchains such as Hyperledger Fabric follow deterministic algorithms. Such blockchain networks have specific nodes called ordering nodes; blocks validated by these ordering nodes are considered as final and true. Hence, there is no probability of a fork.

The following table outlines a quick comparison of some of the consensus algorithms mentioned in this book:

Facts	PoW	PoS	PBFT
Type of Blockchain	Permissionless	Permissionless and Permissioned	Permissioned
Finality of Transaction	Probabilistic	Probabilistic	Deterministic
Needs Token	Yes	Yes	No
Example Usage	Bitcoin, Ethereum	Ethereum	Hyperledger Fabric

Visit the Structure of the blockchain section for a detailed analysis of the various types of consensus algorithms.

Application layer

The application layer is comprised of smart contracts, chaincode, and dApps. Application layer can be further divided into two sub-layers –application layer and execution layer. Application layer has the applications that are used by end users to interact with the blockchain network. It comprises of scripts, APIs, user interfaces, frameworks. For these applications, blockchain network is the back-end system and they often connect with blockchain network via APIs. Execution layer is the sublayer which constitutes of smart contracts, underlying rules and chaincode. This sublayer has the actual code that gets executed and rules that are executed. A transaction propagates from application layer to execution layer, however the transaction is validated and executed at the semantic layer (smart contracts and rules). Applications sends instructions to execution layer (chaincode; in case of Hyperledger fabric), which performs the execution of transactions and ensure the deterministic nature of the blockchain (such as permissioned blockchain like hyperledger fabric).

Smart contracts to be executed on the Ethereum runtime engine are written in Solidity. It needs a compiler to syntactically prove the code. Since it is compiled, the bytecode is smaller and runs faster on EVM. Code executed on EVM is fully isolated and does not have any interaction with the network or filesystem.

Smart contract: A code with business logic is identified by a unique address, and it resides on the EVM. A smart contract contains functions that are executed when a transaction is performed against those functions. Depending on the logic of the smart contract, a transaction can result in a change of state in the contract. Developers can use any language such as Solidity or Python, to write a smart contract, and can use a specific complier to compile the code into bytecode and then deploy those bytecodes to blockchain. Once deployed, a unique address is assigned to the smart contract. Any user on the blockchain can execute a transaction against that smart contract. Refer to the following transaction flow for the steps of a transaction on an Ethereum blockchain. Smart contracts are written in a high language such as Solidity and deployed to EVM for execution. However, there are codes that link the smart contract to the outside world; for example, inter-blockchain, logic execution, and so on. These are called oracles and dApps.

Oracles: Smart contracts operate on values and trigger contract state change, but only when the defined logic is met. An oracle is an agent whose task is to securely provide these values to a smart contract. Oracles are like data feeds from third-party services, which supply values to smart contracts.

Chaincode (Hyperledger Fabric): Smart contracts are the transaction logic that controls the life cycle of business objects, which are contained in the world state. Smart contracts are then packaged together into chaincode, which is then deployed to the blockchain business network. In Hyperledger, smart contracts govern the transactions, while chaincode governs the packaging and deployment of smart contracts. A chaincode can contain many smart contracts. For example, in an insurance chaincode, there can be smart contracts for claims, liability, processing, and so on. Chaincode defines the ledger’s data schema, initiates it, performs updates to ledgers (consensus-based), and responds to queries for ledger data. Chaincode also emits events, which allows other applications to subscribe to chaincode events and perform subsequent downstream functions or processes.

In Hyperledger Fabric, there is no VM like EVM (Ethereum). Chaincode is deployed on network nodes, and smart contracts run on a peer node owned by an organization, mostly written in standard languages such as Java, Node.js, and Go. Chaincode runs on a secure Docker container that's available to each blockchain instance. These containers are independent of other nodes in the network; however, these chaincodes are orchestrated by the peer nodes and act as proxies, allowing access to client applications via REST APIs or SDK.

Chaincodes are initiated for channels. An administrator can define an endorsement policy for a chaincode for a given channel. This ensures that all the smart contracts, which are packaged in the chaincode, are available for that channel. Because of this, a chaincode might follow different endorsement policies on different channels based on the endorsement policy that's been configured for that channel. Smart contracts can communicate with other smart contracts on the same channel or other channels.

dApps: dApps is a distributed application that runs on top of a distributed technology like Blockchain, such as Ethereum, Bitcoin, or Hyperledger Fabric. It's a decentralized application that leverages smart contracts or chaincode. dApps can be considered a web application that interacts with the smart contract or chaincode; however, the dApps are not controlled by a single entity or an organization. Once deployed, they belong to the blockchain network. dApps are user-friendly applications, which business users can use to transact onto a blockchain network. Smart contracts allow you to connect to blockchains, whereas dApps allows you to connect to a smart contract or chaincode. For example, if you go to LinkedIn, the web page calls APIs, which gather data from a database. However, in dApps and the smart contract world, dApps are API-based web applications that connect with smart contracts, which in turn execute transactions on the ledger. A few examples of dApps are financial applications such as invoice factoring, KYC, and so on.