As we navigate through the technological revolution of the 21st century, few buzzwords have captured the public imagination as much as “blockchain.”
Blockchains are a component of the ingenious invention of Bitcoin, the brainchild of an individual or group known as Satoshi Nakamoto, and due to Bitcoins success, the underlying technology of blockchain has been heralded as the panacea for everything from financial transactions to supply chain management, from voting systems to healthcare records.
We’ve seen thousands of attempts to scale blockchains as we journey from the realms of theory to practical implementation; it’s becoming increasingly evident that blockchain technology has its fair share of hiccups. Despite its many touted advantages, blockchains, in their current form, can be slow, clunky, and have significant scalability issues.
Blockchains hit the wall very quickly.
Blockchains have some limitations that can make them seem slow and cumbersome, particularly when compared to traditional, centralised databases.
Scalability
One of the main limitations of blockchain is scalability. Blockchains like Bitcoin and Ethereum have a limited capacity to process transactions. This is because each block has a maximum size, and blocks are only added to the chain at certain time intervals. Bitcoin, for instance, can only handle approximately seven transactions per second. This is significantly slower than traditional payment systems like Visa, which can handle tens of thousands of transactions per second.
Consensus mechanisms
The process of achieving consensus on a blockchain can also be slow. This is particularly true for blockchains that use Proof of Work (PoW) like Bitcoin. PoW requires miners to solve complex mathematical problems to add a block to the blockchain, which is a process that takes about 10 minutes on the Bitcoin network. This slows down transaction processing and can lead to delays should there be thousands of transactions all competing for block space at any one time.
Data storage
Each node in a blockchain network stores a copy of the entire blockchain. As the blockchain grows, the amount of data each node must store increases, which can lead to significant storage requirements. This can be problematic for devices with limited storage capacity.
Network propagation
When a new block is mined, it needs to be propagated to all the full nodes in the network. Depending on the network’s size and the nodes’ geographical distribution, this can take some time.
Immutability
While the immutability of blockchains is one of their key benefits, it can also be a drawback. Once a transaction is added to the blockchain, it cannot be changed or removed. This can make it difficult to correct mistakes or fraudulent transactions.
Energy consumption
Finally, the PoW consensus mechanism used by many blockchains is energy-intensive, which has led to concerns about the environmental impact of blockchain technology.
Reducing your on-chain footprint.
To get around blockchain’s limitations, you could try to increase the block size and increase the speed of blocks, but this only migrates the problem; it doesn’t solve it. If Bitcoin were to require more resources to run, it would encourage it to centralise, and this can quickly become a race to the bottom.
Instead of trying to improve Bitcoin through brute force, the other option is to look at ways of using block space more efficiently. The fewer times you need to touch the chain and the smaller your transactions are, the more throughput be processed by the chain.
One way to wrap up a lot of transaction data into a small on-chain proof is through the use of Merkle trees.
What is a Merkle tree?
Merkle trees were invented by Ralph Merkle, one of the forefathers of modern cryptography. Though he patented the Merkle tree in 1982, the patent on them has long expired, and we’ve seen it rolled out in several applications. Merkle trees are used widely in Git, BitTorrent, ZFS, the Certificate Transparency framework, and pretty much every cryptocurrency.
Merkle trees are a cryptographic data accumulator that allows you to prove something is true or verified with only a hash, which is a much smaller on-chain footprint than storing all that data. Think of a Merkle tree as an independent branch where a whole bunch of transactions can take place, but instead of adding all the transaction history, it’s represented with a single consolidated proof.
One use for Merkle trees in the Bitcoin ecosystem has been through the proposed UtreeXO node, which is an alternative Bitcoin node that can be used to verify the blockchain. While SPV (Simple Payment Verification) transactions are also validated using Merkle trees.
The adoption of Merkle trees in Bitcoin was first intended to be more efficient with the data footprint of transactions, but we’re now seeing Merkle trees used to house arbitrary data in the Bitcoin blockchain but retain a smaller footprint.
What is a Sparse Merkle tree?
A sparse Merkle tree is similar to a standard Merkle tree in that you can prove data exists in its index. However, it also has the ability to prove that something is not, or is no longer part of, a Merkle tree is where the Sparse Merkle tree comes into play. A Sparse Merkle Tree (SMT) is a data structure which allows for non-inclusion proofs.
In an SMT, the contained data is indexed, and each data point is placed at the leaf that corresponds to that data point’s index.
In an SMT, the location of a piece of data (which leaf of the tree) and the data itself are bound to each other. This means that for a given piece of data, there is only one location within the tree at which that data could be placed. If that location is empty, the data is not present in the entire tree.
SMTs acquire this ability by the way the contents of a leaf are hashed, and a Merkle Tree is created in which the leaf’s position corresponds to the hash. This requires a Merkle Tree of 256 levels and 2^256 leaves. Generating such a large tree is efficient because the vast majority of the leaves are empty.
The Benefits of Sparse Merkle Trees
- Efficiency: SMTs are highly efficient when dealing with large or dynamic data sets. They only store actual data points, treating all others as default values, thereby saving substantial storage space.
- Proofs of Non-Inclusion: SMTs can efficiently provide proofs of non-inclusion, confirming that a certain data point does not exist in the tree. This is useful in systems where confirming the absence of data is as important as confirming its presence.
- Constant Height: Regardless of the number of actual entries, SMTs maintain a constant height because they consider the entire space of possible entries. This makes operations like adding and removing entries more predictable.
The Shortcomings of Sparse Merkle Trees
- Complexity: SMTs are more complex to implement than standard Merkle Trees due to their sparse nature and the need to handle default values.
- Default Value Collision: There’s a chance of collision with real data if the default value isn’t chosen carefully. If a real data point’s hash matches the default value, it could lead to incorrect proofs.
- Large Proofs: While SMTs can handle large datasets efficiently, proofs for data points can be larger than in standard Merkle trees, especially when dealing with proofs of non-inclusion.
Where are sparse Merkle trees used in Bitcoin?
A Sparse (meaning ‘thinly scattered’) Merkle tree is a data structure in which it can be proven that specific data doesn’t exist within a Merkle tree. An SMT is an authenticated key-value store, meaning that the key, or location, of a leaf and the content of the leaf are bound to each other. This allows the foundation for the issuance, tracking, association and verification of arbitrary data inside a hash stored on a Bitcoin block.
These Merkle trees can then be used to store secondary information on the chain, like the creation of smart contracts or issue secondary assets using Bitcoin as the anchor.
The Taproot Assets protocol makes use of SMT in combination with Taproot, taptweak and Sum Merkle trees to allow for the issuance of Bitcoin-native assets without the need for an additional blockchain.
Sparse Merkle trees and Merkle sum trees are combined into sparse Merkle sum trees. The root of this tree is added to a taproot tapscript, and together a taproot address is created.
Instead of its own blockchain, Taproot Assets issuers store sparse Merkle sum trees off-chain and issue proofs to asset holders out of band. The owners of such assets can independently verify that their account is included in the tree, is filled with the appropriate amount, and the corresponding taproot transaction exists and is confirmed on the Bitcoin blockchain.
Do your own research.
If you want to learn more about Merkle trees, use this article as a jumping-off point and don’t trust what we say as the final say. Take the time to research, check out their official resources below or review other articles and videos tackling the topic.
Are you a Bitcoin and Lightning fan?
Have you been using Lightning to make micro-payments? Stream sats or engage with apps? Which app is your favourite? Do you run a Lightning node? Have you tried all the forms of Lightning payments? Which one do you prefer?
Let us know in the comments down below.
