Although blockchain has a variety of definitions, a blockchain can best be described as a data structure of blocks that are chained together to form a collection of records, called a ledger, with cryptography being a key ingredient in the process. A blockchain doesn't have a storage mechanism; instead, it has a set of protocols that govern the way in which information is forged. So, a blockchain can be stored in flat files or in a database.

Blockchain technology gained popularity due to the fact that its integrity can't easily be compromised. A compromised blockchain can be recognized for what it is, and rejected quite easily by anyone in a network. This integrity is achieved by cryptography, which is what binds the blocks together; we'll study this idea of cryptography in Chapter 2, A Bit of Cryptography.

Blockchain's promise of providing such robust integrity is what eventually paved the way for the idea of sharing chains of data in untrusted peer-to-peer (P2P) networks. Validation of the blocks in a blockchain is what makes sure that a blockchain has a valid global state that can be accepted by everyone. Due to a blockchain's ability to share information in an open P2P network without any central authority governing it, the technology can have many different applications; however, the technology could not simply just be deployed to these applications immediately without any troubleshooting. Although blockchain technology, from the beginning, had a huge role to play in the decentralization of applications, it still faced several challenges with regards to its application in trustless environments. One of the biggest challenges was keeping a blockchain consistent across all the participants of a P2P network. This was solved by creating a consensus algorithm, which agrees on how the blocks should be appended to grow the chain in a trustless environment.

The term blockchain actually entails a number of concepts, including P2P network management, consensus mechanism, and more, all contributing to the creation of a decentralized application.

As we've just discussed, despite blockchain being fascinating due to its cryptography-based security, decentralized nature, and nearly immutable data storage mechanism, it's very important to understand its limitations.

Blockchain's ideal implementation is with atomic events or transactions, where minimal information about an event is stored as a transaction; these transactions can be clubbed together in a single block and added to a blockchain. Although a blockchain network is good at handling a global state, it would not add much value when it comes to storing data in bulk, as there would be scalability issues. It's very important to understand when best to apply blockchain technology to develop an application. We will explore when not to use blockchain in Chapter 11, When Shouldn't We Use Blockchain?.

You'll remember that one of the very first things we did in this chapter was note there are several definitions of the word blockchain. Before we move on, let's have a look at several definitions of the word:

A blockchain is a read- and append-only storage methodology. This means that blocks can only be created and read in the blockchain ledger. Blocks in a blockchain cannot be updated or deleted; blocks can only be appended to the end of a blockchain. There is no access control in a public blockchain as it is open for both read and write operations.

On the other hand, relational databases follow the create, read, update, and delete (CRUD) operation model. Unlike the case with a blockchain, each database has an administrator when it is created, and they will assign access control to the other users. Relational databases are mostly maintained by a single entity who is in control of all the application data, whereas blockchain technology was designed for decentralized applications.

Figure 1.1 should help you to visualize the difference between a centralized database architecture and a blockchain architecture:

We know that blockchain technology now has its main application where the tracking of transactions or events in a decentralized network is concerned; currently, the greatest scope to be found for this use case is in the area of finance. However, as a matter of fact, blockchain technology didn't appear as we know it and use it today until quite recently. The first ever idea of maintaining a chain of blocks to construct a tamper-proof timestamp of digital documents came about in 1991. But the concept wasn't popularized until an author with the pseudonym of Satoshi Nakamoto – whose identity is still not known today – showcased blockchains and their true use in decentralized networks in 2008, by publishing a paper titled Bitcoin: A Peer-to-Peer Electronic Cash System (https://bitcoin.org/bitcoin.pdf).

Later, in 2009, a reference implementation of blockchain technology was created by Satoshi Nakamoto. This was called Bitcoin. This was the first – and still remains the most popular – implementation of a blockchain-based electronic cash system. Satoshi took inspiration from several previous inventions, such as b-money and Hashcash, to create a decentralized digital currency.

Bitcoin was released in 2009 when Satoshi made the first transaction, which was inserted into the first block of the Bitcoin blockchain. This block is called the genesis block, and is the proof of the entire blockchain's legitimacy. Satoshi stopped contributing to the Bitcoin project in 2011 and is now allowing the open community to contribute to the project. Since then, it has grown from being a fairly simple digital currency to a resilient protocol that has become the reference implementation for every blockchain application. Although Bitcoin's market value is volatile, it's nearing the market value of $200 billion US dollars as of late 2018, which is almost half of the entire cryptocurrency market.

Bitcoin was the first decentralized digital currency to solve the problem of the double-spending attack. Maintaining an open distributed blockchain with a consensus for validating the block was the main thing that made Bitcoin a practically implementable decentralized currency. Bitcoin made use of a consensus algorithm called the proof-of-work (PoW) algorithm to prove that a node has actually worked to create a new block of the blockchain. This concept was also applied in an implementation called Hashcash, which was proposed to limit email spam by forcing the spammer to do some work before sending each email. This system prevented spammers from sending email in bulk, as a computation task was required before each email was sent. All the receiver needed to do was verify the work done by the sender. Similarly, the PoW consensus algorithm implemented in Bitcoin prevents any node from flooding the blockchain with its own created blocks, thus preventing any single entity from dominating the blockchain.

Bitcoin was what really gave birth to blockchain technology, and since then, several new applications of the distributed blockchain database have emerged, most notably during early 2014. This era is widely known as the era of blockchain 2.0. Namecoin was one of the first concepts to emerge. It expanded the scope of Bitcoin's blockchain and was introduced as a distributed naming system based on Bitcoin. However, unlike Bitcoin, it was able to store data as key-value pairs in the public blockchain. This concept influenced a number of blockchain 2.0 applications that would go on to gain popularity.

One of the most popular enhancements that came with blockchain 2.0 was the introduction of smart contracts. Several blockchain platforms were developed that allowed the user to write higher-level scripts and not worry about the actual blockchain implementation. One of the platforms that was most successful in doing this was Ethereum, which was proposed in late 2013 by Vitalik Buterin. The intention of Ethereum was to make the best use of the technology that Bitcoin used. Bitcoin's initial implementation was intended for the flow of digital currency. Now, although a scripting language was implemented for other applications, such as asset transfer, it was very primitive and only had a few use cases. Vitalik Buterin, the co-founder of Ethereum, suggested that Bitcoin needed a scripting language for decentralized application development in order to increase its scope. Failing to gain agreement, Vitalik proposed the development of a new platform with a more general scripting language. This platform, Ethereum, gained popularity due to the implementation of smart contracts in its environment. Smart contracts are high-level scripts that run on the Ethereum Virtual Machine (EVM). A number of decentralized applications have been developed using the Ethereum platform.

The Ethereum platform inspired developers to come up with a framework that would use Bitcoin's core protocol and build a platform to develop decentralized applications. This was a revolutionary approach, as the technology would be used to modify any application where having a third party would be both expensive and redundant.

In fact, there are plenty of motives that drive developers to integrate blockchain technology in their existing applications. In the next section, we'll explain this in more depth.

Several blockchain platforms have been created to build scalable decentralized applications; some of the most popular ones are listed here:

• Corda: This is a distributed ledger platform designed to record, manage, and automate legal agreements between business partners. It was designed by R3 in collaboration with the world's biggest financial institutions, which makes Corda suitable for financial enterprises shifting toward distributed ledger technology.
• Hyperledger: This is an open source effort to advance cross-industry blockchain technologies. It's hosted by the Linux Foundation and achieves collaboration between various industries and organizations. IBM and Intel, for instance, are active contributors to Hyperledger projects. There are a number of Hyperledger projects, all aimed at solving different enterprise-level problems using blockchain.
• Multichain: This is a simple and powerful private blockchain framework that is compatible with Bitcoin. It has support for complete asset cycle management. Due to its support for access control, it's an ideal framework for developing permissioned blockchain applications.
• NEO: Formerly known as Antshares, this application is often called "the Ethereum of China." It uses blockchain technology and digital identity to digitize assets.

Every new innovation is the result of an attempt to solve a problem. Blockchain technology is no exception. It's quite evident after learning about the evolution of blockchain technology that it arose because of a need to address the inevitability of uncertainty in the existing economy.

Uncertainty could never be eliminated, but only lowered: there have always been institutions that have acted as third-party lawmakers to lower uncertainty, or lack of trust, whenever there was a need for an agreement between parties. A typical example would be buying an item on eBay. You would always need as much certainty as possible about the trade. One party expects fair goods, and the other expects agreed money. Now, though the buyer and seller have no reason to trust each other, they complete their trade as they trust the third party, which is eBay, who assures them both of a legitimate trade. Again, there was a need to trust these "medium" institutions. Trusting an institution requires a lot of research and knowledge. Blockchain promised to overcome these issues by implementing applications in a decentralized and secure way, assuring some level of certainty. This was one of the main reasons behind the widespread adoption of blockchain in a trustless society.

We know that blockchain is an ideal technology for implementation in trustless environments. However, the blockchain alone is not responsible for the success of the complete implementation. It's assisted by several other protocols that make it the robust and resilient technology it is. Blockchain can be implemented in trustless networks mainly due to the decentralization of computation in dense P2P networks and the maintenance of a secure and publicly distributed ledger that gives complete transparency over the entire blockchain. The P2P protocol makes sure that every node holds the latest state of the blockchain.

The need for decentralization is the key motivation behind the blockchain technology, and decentralization is achieved by distributing the computation tasks to all the nodes of the blockchain network. Decentralization solves several problems of traditional systems; the single point of failure is one such problem. For example, in a centralized system such as a bank, the user would always communicate with the same third-party bank to fetch their account details. Although this transaction may be possible almost every time, 100% uptime is not guaranteed, as this server is centralized and has just a few backup servers for load balancing. There could well be a situation where all the servers could be flooded with requests, resulting in crashes and server shutdown. This downtime is something that's inevitable, even in perfectly architected servers. If the same scenario was faced in a decentralized network, it wouldn't be an issue, because all the transaction data would be distributed across all the nodes, meaning that each node can act as a backup node in case of failure, maintaining the integrity of the data (another key benefit of blockchain-based solutions). This is something that's achieved by maintaining a distributed ledger of blockchain data. Blockchain immutability, which is a key factor in trusting the integrity of the blockchain, ensures the integrity of the ledger, which is publicly accessible to all nodes.

A blockchain, put simply, is a chain of blocks that are secured to each other via cryptography. Cryptography hash pointers are used as references to link each block in the blockchain of the public ledger. Although this sounds very secure since no intruder would be able to break the blockchain and insert their own versions of the blocks, it's not completely secure. Since the ledger is entirely transparent and public, any node could insert their block to reproduce the entire blockchain and create their own version of it. Eventually, they could later propagate the block to every node in the network and prove that their blockchain is the legitimate one. This shows that immutability cannot be achieved just by connecting all the blocks and forming a secure ledger. Achieving immutability needs to be assisted by some kind of decentralized economic mechanism that gives a fair chance to each and every node in the network to vote on block creation, and also makes it harder to reconstruct blocks once they are appended.

Satoshi Nakamoto's proposed solution to this problem is the only reason that Bitcoin is implementable in a decentralized environment. The PoW consensus algorithm used in Bitcoin was the first – and is still the best known – solution. It promises a high degree of immutability to the public ledger and secures it, even in a trustless network. In terms of cryptocurrency, the nodes that perform PoW are called mining nodes. As the name suggests, mining is the act of forging new blocks to be appended to the blockchain. The amount of work that it takes to mine ensures that the blockchain is immutable and that tampering with any past transactions is nearly impossible.

This is because of the fact that any node that wants to tamper with past data should be able to reconstruct all the blocks by providing PoW and competing with all the other mining nodes. This is nearly impossible unless the node in question owns the majority of the computing power of the network, in which case, the attacker would stand a chance of beating all the nodes. This is why Bitcoin's consensus algorithm is widely used in public blockchain applications to achieve higher immutability of records.

However, immutability is not the only characteristic exhibited by blockchain technology. Due to the decentralized nature of the blockchain, every single transaction in the blockchain is replicated across all the nodes of a network. The replication of information provides greater robustness to the blockchain. Replicated transactions must be validated by every node to achieve consensus. This ensures that the transactions are publicly visible and all the blockchain data is transparent to the network. The transparency provided by the blockchain can be a boon for some use cases and a curse for others. This is why variants of blockchain were created, as described in a later section of this chapter.

All these characteristics of the blockchain make it a perfect public ledger, or an effective instance of distributed ledger technology (DLT). Bitcoin's blockchain, along with its consensus mechanism, is the most resilient DLT to date.

Ever since the invention of networking, there has been debate about the centralization and decentralization of computing architecture. We have seen interest between these two computing architecture models fluctuate over time. Mainframe architectures were used in enterprises to house great amounts of computing power, memory, and storage. They were largely centralized, and terminals without much computing power were used to connect to these machines in order to perform required operations. Then, personal computers were introduced for household usage, with enough computation power, memory, and storage to perform basic operations. This gave rise to the client-server architecture, in which clients communicate with the server to perform computation. The server usually performs heavy computing in a distributed system and syncs the result with the client.

The cloud computing architecture provides easy access to the server from any computing device, since the architecture itself is globally accessible. However, a cloud computing architecture is centralized, and its hardware resources are distributed and not transparent to the client. There is still a lack of trust between cloud vendors and end users. This is the reason why we are witnessing a transition from other computing models to decentralization. DLT is the key to achieving this milestone and ushering in the age of decentralization.

A distributed ledger is, at its core, a replicated and shared digital database that is spread across geographical regions. A P2P network and a consensus algorithm are required to ensure an effective distributed ledger. Blockchain technology is one of the techniques used to achieve a distributed ledger, but it is not the only data structure of DLT:

The growing number of use cases for blockchain, as well as an increasing awareness of its limitations, has given rise to a variety of successful blockchain implementations. In this section, we'll be trying to grasp the essence of each one of them.

The blockchain concept has been widely used and adapted due to its transparency and how every node participates in contributing to the growth of the blockchain. The early blockchain model, the product of Bitcoin, is completely open and permissionless and is popularly known as a public blockchain. Public blockchains are popular due to the impartial way in which the nodes are treated.

Public blockchains work seamlessly in trustless networks due to the immutable nature of the records. Bitcoin, Ethereum, and several other projects that have inherited PoW-like consensus algorithms ensure that recorded transactions are non-editable.

Public blockchains are ideal for cryptocurrency projects where recorded transactions should not be modified. However, public blockchains often face scalability issues at some point if the necessary changes aren't implemented. One of the most visible issues of Bitcoin is its mining approach (PoW), which is very expensive in terms of the electricity required for miners to solve the puzzle. The average time for block creation is 10 minutes. Therefore, the difficulty level of mining has been adjusted to maintain this time. This has resulted in a very expensive PoW environment due to competition among miners. We'll not be able to predict the future of Bitcoin or any other public blockchain due to these complex attributes, and only the natural evolution of the technology can decide its fate.

Due to these pros and cons, the permissionless or public blockchain is well suited for transparent applications, where the blockchain should inherently secure the system since the network is trustless.

Private blockchains were introduced mostly to widen the scope of blockchain technology. The permissioned blockchain, as the name suggests, uses the opposite approach to that of the public blockchain. Private blockchains came about mainly to solve some of the issues we saw in public blockchains and to make blockchain technology scalable.

Permissioned blockchains introduce access control to provide specific access to the participants in a network. Each permissioned blockchain will have an administrator who assigns roles for the participants in the network. Permissioned blockchains ensure that bad actors are not a part of the validation or block-creation processes and thus eliminate any potential attacks on the blockchain. A network involving a permissioned blockchain is mostly a trusted network.

Private blockchains are suitable for organizations where a ledger only needs to be shared internally. Permissioned blockchains are often mutable or not strictly immutable, and their transactions can be modified with some effort; this is in stark contrast to public blockchains, where this is nearly impossible. Permissioned blockchains are still decentralized ledgers, but they will have some nodes with limited capability within the organization, whereas nodes in public blockchains are treated impartially.

The consortium blockchain is a hybrid blockchain that is semi-decentralized. It combines the best features of both permissionless and permissioned blockchains. Instead of assigning most tasks to a single organization, a consortium blockchain assigns the same tasks to nodes maintained by multiple organizations. Instead of having a single validator node, there can be multiple nodes. Although a consortium blockchain is permissioned, it's more decentralized than a private blockchain.

Now that we have a fair understanding of blockchain, we're going to give an overview of blocks, which are responsible for building a blockchain.

If we consider a blockchain as a data structure, then the blocks are aggregated sets of data that are used for formation of the blockchain. Blockchain formation is similar to linked list formation, where each node has a reference to the next node in the sequence. In the case of blockchain, each block has a reference to the previous node, thus forming a link all the way to the initial block (known as the genesis block) of the chain. As we mentioned earlier, a blockchain can be stored in either flat file or database format. Bitcoin uses LevelDB to store metadata about all the blocks that are downloaded to the disk.

Just like a linked list node, each block has a pointer, which is the identifier of the block. These are just hashed values of the block's header data. More detail about hashing will be covered in Chapter 2, A Bit of Cryptography. We can consider the hash as a unique identifier of a fixed size that represents each block; no two blocks will have the same identifier. Since all the blocks are linked together by this hash value, each block will have the identifier of the previous block. The previous block is referred to as the parent block, and each block can have only one parent.

Each block could also be referenced by the height of the blockchain. This height is nothing but the distance of the block, or the block count, from the genesis block. Height is an important attribute of the blockchain, as it's easier to refer to a block with a plain number rather than a lengthy hash value. The block hash is not a hash value of the entire block, but rather the hash value of the block header, which consists only of metadata. In Bitcoin, a SHA256 hashing algorithm is used to hash the block header and to create a unique identifier for the block.

Although all blockchains consist of linking blocks together to form an immutable ledger, different block structures can be adopted depending on the application. Permissioned and permissionless blockchains, for instance, have slight variations in their block structure. We'

We'll be using Bitcoin's permissionless block structure as a reference to try and identify its characteristics:

A block consists of the components mentioned in Figure 1.3. Block Header and Transactions are the most important parts of the block, as they are responsible for the hash value, which is the identity of the block. The Block Size is the size of the entire block. The Block Header contains all the metadata of the block, and the Transaction Counter has the count of transactions. Finally, all the Transactions are stored in the block.

As mentioned before, a blockchain starts with an initial block called the genesis block. If the chain is traversed backward from any given block, it will end up at the genesis block, proving that the entire chain is legitimate and valid. The genesis block is often statically coded in a public or permissionless blockchain, but it's created by the first participant in the case of the permissioned blockchain.

The block header, as stated before, consists of the metadata of the block. This holds the information that's needed to link the blocks in the blockchain:

Each block header will have the components outlined in Figure 1.4. These are the minimum fields required in a permissionless blockchain, such as Bitcoin, to efficiently create a block that can be appended to an immutable blockchain. The Previous Block Hash field is a reference to the last block created. The Merkle Root is the value of the Merkle hash tree; it summarizes all the transactions in the block. Timestamp, Difficulty Target, and Nonce are used by the PoW consensus algorithm to solve the hash puzzle. We'll be revisiting these concepts in more depth throughout the book.

As we know, blocks are linked in a blockchain using references, just like in a linked list, but here the blocks are linked by referencing the hash value (identifier) of the previous block. Each full node in a blockchain network will maintain a complete blockchain and append a new block whenever it has one to append. Due to the decentralized nature of the blockchain, each node will verify the block before linking it to the local blockchain record.

The computed hash value of each block is the combination of the hash of the previous block and its own block data. This results in a dependency between neighboring blocks and nearly unbreakable links:

Satoshi explained how the concept of timestamping should be used. All the items are hashed and the block is timestamped, meaning that the subsequent block will include this timestamp, creating an ordered chain of blocks.

Each node in the blockchain network follows a simple process for appending new blocks to its existing local blockchain. Whenever a node receives a block from the network, it checks for the previous block hash. If the hash value matches with the hash value of the last block on the node's local blockchain, then the node accepts this block and appends it to the current blockchain. As long as this is the longest known blockchain, the blocks would be considered valid by all the peers in the network in a PoW-based blockchain.

Gordon Moore, the co-founder of Fairchild Semiconductor and Intel, observed that the number of components per electrical integrated circuit would grow by at least a factor of two for every year. Back in 1965, he also projected that this rate of growth would continue for at least another decade. Over the years, he revised the forecast to doubling every two years. This observation was geared toward the number of transistors in a dense integrated circuit and has been used in the semiconductor industry to set targets for research and development. But it isn't only limited to the chip-manufacturing field; it has also been used to make observations about technological and social change, as well as productivity and economic growth.

Moore's law has been adapted and applied to approximate the rate of change in network capacity, pixels in images, storage device size, and much more. Blockchain is a technology of the future that might have to overcome multiple limitations in order to achieve healthy long-term development. Moore's law would help in deciding the complexity required for any blockchain application so that the application doesn't have to struggle with future scalability issues.

Since every node in the network maintains the complete blockchain ledger, blockchain data keeps increasing in size as time goes on. This raises some concerns regarding scalability, as each node needs to maintain the blockchain locally (such is the nature of the distributed network). Satoshi Nakamoto had mentioned that the growth of the block header size would be around 4.2 MB per year, and Moore's law would guarantee growth of at least 1.2 GB RAM (in 2008) per year, which should not pose any problems for block storage even if they are maintained in node memory.

Public blockchains, such as Bitcoin, have to deal with the hash rate of the hardware for their consensus algorithms. Bitcoin-mining hardware has been able to keep up with Moore's law, providing the required hash rate in accordance with the growing difficulty rate. However, the future of Bitcoin mining relies on Moore's law and the hardware being able to keep up with the difficulty without causing much loss to the miners:

This chapter has introduced us to everything we need to fuel the study of blockchain in the coming chapters. Having got to know the background of blockchain technology, including its purposes and how it works, we should now fully understand the potential of this technology and how it can solve problems in some of our current systems.

Now that we have a fair understanding of the technology, in the next chapter, Chapter 2, A Bit of Cryptography, we will walk through the concepts surrounding the backbone of blockchain technology. In that chapter, we'll be exploring the building blocks for an understanding of blockchain.