Segregated Witness's soft fork and a vote of no confidence

Hey,
The Core's impending release of Segregated Witness represents a turning point for the Bitcoin industry. While the blocksize limitation has continued to hold back the currency for a year, until now there has never been a concrete deadline for people opposed to the Core leadership's roadmap. Segregated Witness changes that, as it represents a fundamental shift to how Bitcoin will operate moving forward.

The promise of the proposed "soft fork" to Segregated Witness is that existing nodes will not be kicked off the network and will continue to run. While that sounds great in promise, in reality every programmer writes "assert" statements on a regular basis to create explicit exceptions when something goes wrong. One of the worst possible events that can happen to software is that it starts to behave differently and do bad things because nobody is ever notified that something has gone wrong.

By activating through a soft fork, the deployment of Segregated Witness will break a large proportion of existing software. Furthermore, the additional complexity will reduce developer productivity for years to come. In this article, I will explain why Segregated Witness has significant disadvantages, and why the soft fork approach to rollout will have disproportionate consequences to regular users. I will encourage others to take a stand against the Core's "soft-fork" decision by escalating the soft fork to a hard fork, which will take the decision out of miners' hands and to place it into those of users and investors. I will point out that the window of opportunity to do so has been reduced to weeks or a few months at best.

At a high level, Segregated Witness is an addition to the Bitcoin protocol that separates data into two formats: the existing blocks and a "witness" segment. Transactions in the existing blocks serve as pointers to the detailed data in the witness segments. For example, a transaction with many recipients may take up less space in the existing blocks because the signatures in the existing blocks don't describe everything that the transaction represents.

Proponents of Segregated Witness point out a number of advantages to the scheme. For one, it solves the "transaction malleability" issue, which makes it difficult to rely on a transaction hash to track unconfirmed transactions. Supporters also believe that more data can be fit onto the blockchain with Segregated Witness, because although the proposal does not eliminate the 1MB transaction limit, it reduces the size of the data present in the original 1MB blocks. The Core's official statement suggests that future development will be easier because of the way that reserved operator statements are handled in the Bitcoin scripting language.

Unfortunately, however, I believe that the drawbacks to Segregated Witness outnumber its benefits. I'll list the technical drawbacks first, and then focus on the deployment method, which is its largest drawback.

First, it's important to examine the validity of increasing the number of transactions in a block. While Segregated Witness does indeed reduce space utilization in the existing 1MB blocks, all of the data describing the transactions still needs to be received and included in the blockchain. In Segregated Witness, that witness data is received separately, so a 1MB block can contain 100KB of additional transaction witness data, for a total of 1.1MB of data. It's not as simple as that, though: the transactions actually use more bandwidth than they would if the blocksize were simply increased.

View this as two Excel spreadsheets - when two spreadsheets with different data about a list of cars exist, the license plate number needs to be present on both the accident report and on the inspection report. If one created a combined "accident/inspection report," then the license plate only needs to be listed once with both pieces of data below it. In Segregated Witness, this duplication of the transaction identifier results in extra data.

Plus, as I pointed out in March, Segregated Witness only increases transaction space if most Bitcoin transactions are complicated and people use it. Simple transactions that send one input to one output do not significantly increase block capacity with Segregated Witness. People also have to use Segregated Witness for anyone to benefit, and soft-fork adoption is likely to happen slowly or not at all.

Second, Segregated Witness breaks much of the existing software deployed across the cryptocurrency industry. Block explorers are likely to be one of the most noticeable losses - as soon as the first Segregated Witness transaction is mined, many explorers would go offline. Having created block explorers for 200 coins myself, I assert that handling non-standard transactions is an extremely complex task. Coin clients don't keep track of balances for all addresses, so the only way to do that is to rewrite your software to understand the new data format, and then reindex the entire blockchain.

In our case, it took over three months before we were able to get a successful run of our block explorer code where there were no math errors or missing transactions. Because the transactions are summed since the beginning of time, bugs don't appear until days of indexing have occurred, and since it isn't apparent where the sum is wrong, debugging consists largely of guessing. The only way to confirm that the new code works is to delete the database and reindex again - and that assumes that one knows that the totals are right. We have 1.1TB of block explorer data for coins whose blockchains total about twice the size of bitcoin's, so that means constant reindexing of 500GB of linked and summarized data. Many analysis services would fail (or worse, return erroneous data) because developers can't afford the time to upgrade them, leading to increased centralization of many critical Bitcoin functions.

But it's not just block explorers that would suffer. Exchanges, which often have custom implementations of the Bitcoin client, would need to modify production code that has in some cases been working for years with no issues. With huge amounts of money at stake, well-managed companies don't just change code in production without extensive testing. The cost to do this upgrade will be enormous, and that's why I believe that uptake of Segregated Witness would be far less than the Core projects. The companies that make the largest transactions (which offer the greatest benefit to Segregated Witness) are the most likely to judge that the cost of higher transaction fees is less than the cost of paying someone to write new code plus accepting the risk of a catastrophic loss.

Third, even if all services were to be upgraded, the community would still face a burden of increased complexity that would hinder development. Instead of understanding and managing one single and simpler blockchain, developers would need to work with both the main blockchain and the witness data.

Cryptocurrency is such a complex field that the ability of developers to get started is a serious concern. The system is approaching such complexity that no one person can understand every part of it, and adding Segregated Witness to the mix increases training time and will discourage a greater proportion of potential developers. Bitcoin already has a shortage of competent and (especially) of ethical developers, and that pool becomes even smaller as the system grows more complex.

Fourth, even if every single developer sees the additional complexity as a small challenge that can be easily overcome, once Segregated Witness is deployed, it is deployed forever. That means that every change to every client will henceforth need to be tested against both the existing blockchain code and also against the witness data. Is it really worth slowing down development, permanently, for improvements that are relatively minor? The Core leadership estimates that there are over 3300 lines in additional tests required to maintain the stability of Segregated Witness. All of these tests need to be run and updated with every change, and there will be issues found that require additional tests that do not currently exist.

Given that one of the two main selling points of Segregated Witness in the official Core statement is to increase the number of transactions contained in blocks, the few lines that need to be changed to provide for 32MB blocks pale in comparison to the 4824 lines of implementation and test code required for Segregated Witness.

Fifth, it is true that Segregated Witness solves some problems that have been with Bitcoin since the beginning. For example, the "transaction malleability" problem is eliminated. However, the fixes that Segregated Witness provides can be accomplished in simpler ways. Bitcoin Unlimited pioneered Xtreme Thinblocks many months ago, a change which significantly reduces the odds that block processing will be delayed by requiring the receipt of a lot of data. Reducing block validation times can be accomplished with moral_agent's "alternating blocks" arrangement, which is a better way of addressing that problem.

Segregated Witness also introduces fixes to problems that don't really need fixing. Thousands of cryptocurrencies have functioned just fine with malleable transactions for seven years. When handled correctly, malleability will never result in a loss of money. The Core suggests that fixing transaction malleability makes smart contracts easier on Bitcoin, which is also true - but there is already a coin that does smart contracts far better than Bitcoin. Core leadership should be focusing on what Bitcoin does best - sending and receiving money.

Another reason to address transaction malleability now is to allow the Lightning Network to be deployed on Bitcoin - but the deployment of the Lightning Network will never happen. theymos, intentionally or not, ensured that the Lightning Network is dead on arrival when he split the Bitcoin community in August 2015. The stalemate and animosity he created on the blocksize debate will surely extend to the Lightning Network as well.

The Core has been adding features since version 0.12 without taking the time to analyze whether anyone is actually using them. The slow uptake of that version and subsequent releases implies that people do not see a compelling reason to upgrade. Rather than hosting ridiculous weekly development meetings where the blocksize problem is never discussed, the Core should be performing cost-benefit analyses. The central question for any feature should be as follows: will this be used by enough people that the lost time in supporting it outweighs the delay in finding a fix for the blocksize problem? It isn't clear that the problems Segregated Witness is intended to fix have been analyzed to see if users even care about fixing them.

Finally, the most important reason to be concerned about Segregated Witness is also the most catastrophic effect of it: that the Core is considering activating the change as a soft fork. A soft fork is the coward's way of pushing changes upon people who may not want them. In a soft fork, miners will be making an economic decision which does not affect them. They won't be paying developers to fix all the broken software, they won't have to worry about the risk of losing money from changing production code that has run fine for years, they don't care whether anyone will even use the features that Segregated Witness provides, and they won't even bother to notify people who don't check forums every day and therefore don't know a change is coming. They do not have the right to make that decision; the people who have to pay for the consequences do.

There is an alternate path forward: a hard fork, which allows the people who have to pay for the changes to make the decision of whether the changes are worth it. That is why I believe that the community should escalate Segregated Witness to a hard fork. Exchanges should be contacted and at least one of them will agree to list the existing "simple" blockchain. The only large challenge is getting someone who has enough time to make the small change to Bitcoin Core or Bitcoin Unlimited to fork the network at the activation threshold, and who will do so without running out of time through endless discussion or jeopardizing the fork by including other things. The change can be a simple commandline argument that uses the Segregated Witness code itself to reject blocks with witness data. The btcfork reddit subreddit is an example of what is not needed here; the solution is obvious enough that no discussion is needed.

One of the clear benefits of taking a stand on Segregated Witness, rather than on the blocksize itself, is because this issue allows people to cast an vote of no confidence independently of their stances on the highly contested blocksize issue. A fork here also prevents having to frame a later trivial change as a no-confidence fork. The rejection of the Segregated Witness fork, or even a prolonged battle that leads to fork acceptance, would send a clear message to the Core that the feature creep they have been introducing to Bitcoin since version 0.12 is not wanted or needed. It would refocus all attention to achieving a permanent and immediate resolution to the one single problem that is holding back the adoption of cryptocurrency: the limited blocksize of Bitcoin. It would teach the Core leadership that software doesn't get better if you just add lots of features that people don't use. The message would be the following: miners and developers exist to serve the software's users. Get on board with what the community has chosen, or get out.

In conclusion, Segregated Witness has some benefits. A lot of people worked hard on it. It probably has a place somewhere in the cryptocurrency industry. The Segregated Witness hard fork may result in a majority adopting it. If a majority of users stick with the existing code, then Core developers who support Segregated Witness will still be able to use it on the less valuable fork, just as a minority of Ethereum users decided to go their separate way and both communities are largely fine with that decision.

A soft fork places the decisionmaking authority in the hands of miners who will be unaffected by the economic consequences of that decision. Users should make that decision instead. If no action is taken before a soft fork, then everyone is stuck with the changes forever, and even a later successful fork to an unlimited blocksize will then have to deal with all the Segregated Witness transactions. If anyone is ever going to make a statement that Bitcoin exists for its users, not its developers or miners, it is time to put Segregated Witness to the test of a hard fork.....

Thanks,
@steves