Episode 39: How the Bitcoin Improvement Process Works

Aaron van Wirdum Live from Miami and Utrecht, this is the Van Wirdum Sjorsnado.

Sjors Provoost Hello.

Aaron van Wirdum Hey Sjors, how are you doing?

Sjors Provoost Not bad. I think it's warmer here than where you are, but...

Aaron van Wirdum It's pretty warm here. Yeah, when this podcast is being released, it will be day one of the conference. Bitcoin 2021 in Miami. We're expecting like 12,000 people, so that's pretty significant. It's the largest cryptocurrency conference yet to date. Are you impressed?

Sjors Provoost Yeah, it sounds big. I mean, I kind of like bear markets more because I think a subset of the people that go to that conference will stick around for the bear market.

Aaron van Wirdum Real men are made during the bear market, Sjors.

Sjors Provoost [unclear], yep.

Aaron van Wirdum All right. Today, we are discussing the Bitcoin improvement proposal process, the BIP process.

Sjors Provoost That's right.

Aaron van Wirdum So yeah, the BIP process, everyone has heard of it. Although, probably not actually, like you said, lots of new people in the space, lots of people have no idea what the BIP process even is. So I guess we're going to explain it to them as well. So let's start, Sjors. What is a Bitcoin improvement proposal, a.k.a. a BIP?

Sjors Provoost Well, it's just a document with a number that describes, that proposes an improvement to either Bitcoin, like the protocol, or, you know, how nodes communicate. Or it can be more of a standard, like a wallet format. Or, you know, the mnemonics, the 39, or the 24-word backup phrases. That's a BIP. Or it can be a meta thing, you know like BIP 1 and BIP 2 which describe the BIP process are also BIPs.

Aaron van Wirdum Right, so, yeah, in a way, this is a little bit of a meta episode as well because we have discussed a number of BIPs, and um… Now we're going to discuss what a BIP actually is, and how the process works. So yeah, you just explained what a BIP is: it's it's a it's a piece of documentation for something in Bitcoin that requires some level of coordination or that could benefit from some level of coordination, do you think that's that's a fair summary?

Sjors Provoost Yeah I guess yeah.

Aaron van Wirdum Yeah so if…

Sjors Provoost Yeah I mean one of the rules is that you should not write a BIP for something that only you use yourself.

Aaron van Wirdum Right, exactly, if you're building something patented for example, or just something that's just for your wallet, if your wallet developer, whatever it is, you're building and it's of no use to anyone else, and that doesn't need to be a BIP. But if you're building something that could benefit others, or that's useful for us to know, then you might want to draft a BIP, so other developers or maybe not even just developers, but I guess mostly developers, they'll know, you know, it offers a standard. So an example you mentioned is like a seed format for backups. So if you have one seed format, then other wallets can use that seed format as well. So people can easier switch wallets if they want, for example, or use some sort of standard backup device that's sort of compatible with all these things. Or there could be actual protocol changes. How do Bitcoin nodes communicate with each other? How fast? Let's say someone wants to make a protocol upgrade that precludes miners from mining for the first nine minutes. That [unclear] need a BIP.

Sjors Provoost No, that wouldn't need a BIP.

Aaron van Wirdum That wouldn't?

Sjors Provoost That [unclear] need a BIP.

Aaron van Wirdum Yeah, that would need a BIP. Exactly. That's what I said. Like any protocol change. This was something someone actually suggested on the mailing list the other day, which is not going to happen. But yeah, some protocol change. Soft forks, hard forks, peer-to-peer layer type of stuff. Anything that benefits others would, well, at least benefit from a BIP. So yeah, the BIP process. Do you know when it was launched? When it was started?

Sjors Provoost I believe Amir Taaki launched it in late 2011 because the first BIP is called BIP 1 and it's dated September 19th, 2011.

Aaron van Wirdum Exactly, yeah. Amir Taaki was an early Bitcoin developer and he... So he was getting… he wanted to be the Bitcoin development process. He wanted to make sure that it was a bit more transparent. Because up until then, you had developers discussing the Bitcoin protocol and upgrades. But there wasn't really any sort of documentation of it. So he wanted the process to be more transparent. So he introduced the BIP process.

Sjors Provoost And I guess he was one of the first people to work on an alternative implementation, right? libbitcoin.

Aaron van Wirdum Exactly, yeah. So that was a good reason for him as well, for that reason, to actually have some sort of... documentation process.

Sjors Provoost Yeah, because for the most part, everything is in the Bitcoin Core source code if you want to know what the rules are. And that's, of course, not ideal. So more recent soft forks have been written as BIPs.

Aaron van Wirdum Yeah, it was modeled after the, I think it was called the Python enhancement process, I think. So, BEP… wait, PEPs? PEPs, I think, PEP. Yeah, so it was modeled after that. So at least the first iteration of the BIP process which is detailed in BIP 1 by Amir Taaki that was modeled I think almost directly from the PEP process, and repurposed for Bitcoin in a way.

Sjors Provoost Yeah, and one of the changes of course is that Python still has a supreme commander you know, a leader, a central authority. So I think there was something like an appeals process or like a final decision maker in the Bitcoin improvement proposal. That is not the case because there is nobody in charge. So that makes it a bit more complicated.

Aaron van Wirdum Right. So, yeah, Sjors, how does the BIP process work exactly? Let's say, you know, you have an idea. You want to... You want miners to stop mining for the first nine minutes after a block is mined? How do you go about this? How do you turn this into a BIP?

Sjors Provoost Well, the first step, as recommended by BIP 2, is to talk to people. And especially talk on the mailing list.

Aaron van Wirdum So before you go on, yeah, this is probably good to mention at this point. So BIP 1 was drafted by Amir Taaki. And then a couple years ago, the... At that time and still current BIP editor is Luke Dashjr. And he created BIP 2, which was based on BIP 1, but it was refined in a number of ways. So that's why you referred to BIP 2 just now.

Sjors Provoost Yeah. So if you want to make a proposal, you Google BIP 2. And then it says, hey, you may want to check whether other people think this is a sane idea at all and if it's useful to them. Talk to people or you can post it on the Bitcoin dev mailing list. You should also do a Google search to make sure that you're not proposing something that's been proposed a few hundred times before. But even BIP 2 says sometimes you can't find that on the internet that has been discussed before so you may just have to post it on the mailing list and somebody will point it out.

Aaron van Wirdum Yeah, the point being, it's a good idea to first actually discuss it with people before drafting a BIP because odds are your BIP has already been, you know, your idea has already been shut down a while ago or, you know, something existed or just a really dumb idea. And you're going to save yourself a lot of trouble if you first discuss it with some people who've been around for a while.

Sjors Provoost Yeah, it saves yourself time and it saves others' time. So, yeah, that's the first step.

Aaron van Wirdum So that's sort of step one, sort of the informal step one, right?

Sjors Provoost Yeah, and then pretty much it depends on what you're doing, right? But let's take something easy, I guess.

Aaron van Wirdum Yeah, like not letting miners mine for nine minutes.

Sjors Provoost Well, let's take something simpler, like maybe this backup phrase, right? You want a BIP 39, the 24 words, and you have a word list. Probably you want to write some sort of implementation for it.

Aaron van Wirdum Yeah, but how do we save energy with that?

Sjors Provoost Oh, you still want to go to the 90s? I didn't read that mailing list.

Aaron van Wirdum All right, go on. I'll drop that one. Go on with your own example then, and we'll just keep wasting energy.

Sjors Provoost That's a good idea. So basically, you want to write the specification out in a document, but along with that specification should be, at least for some of the types of BIPs, should be an implementation that actually does that. So that you can point to actual software that is actually doing something rather than hand-waving. If you look at a patent application, those are usually just very vague, very lawyerly. It doesn't have to be a real product.

Aaron van Wirdum Did you know there's a patent for an elevator to space?

Sjors Provoost I'm sure there is, and that's because you don't actually have to build it.

Aaron van Wirdum So you can get to the satellite very easily?

Sjors Provoost Yeah, I can see some problems with that.

Aaron van Wirdum There's also a patent for... Like a thing that will swing a woman around when she needs to give birth and then like a centrifuge and then the kid will just shoot out in a net.

Sjors Provoost What could possibly go wrong?

Aaron van Wirdum But yeah, I don't think these things actually exist. That was your point, right?

Sjors Provoost Yeah, that's my point because a lot of people will propose improvements to Bitcoin that cannot actually be built. And so it's nice to have at least some, you know… that thing, yet you should actually write some code. So in the case of Taproot, I think there was a specification, but there was also a working software implementation. And generally, as you're writing software, you find problems with your specification. And as you write the specifications, you may realize there's problems in your software. And other people will review it, and other people will presumably, you know, you might at some point send a draft version on the mailing list, and then you'll get a bunch of feedback from people. So that's the next step.

Aaron van Wirdum So yeah, hang on. So there needs to be some sort of example code or what's the correct definition?

Sjors Provoost Reference implementation is a technical term, but yeah, pretty much.

Aaron van Wirdum Okay. So that's, so first one, step one was sort of discussing it with people that know what they're talking about. Step two is drafting it, including some code.

Sjors Provoost Yeah, I would say so.

Aaron van Wirdum Yeah. Okay. Step three.

Sjors Provoost I don't know the steps are not actually numbered that way. But the idea would be that you probably want to send that specification at that point to the mailing list or to a group of people that you think will give you good feedback. Because then the next step is that people will read it and give you feedback and find problems with it. And then it's your job to do something with that feedback.

Aaron van Wirdum You ask the smart people if your idea makes any sense, then you draft it into a draft BIP. You write some sort of example code, and then you go back for feedback, right?

Sjors Provoost Yeah, pretty much.

Aaron van Wirdum That's the story so far. What's next?

Sjors Provoost And so once you've addressed the feedback and you think the proposal is good enough, now it's time to request a BIP number, which is just a matter of saying, hey, Luke, can I have a BIP number?

Aaron van Wirdum Yeah, well, it's a little bit more than that, right? Your BIP does actually need to satisfy a number of formal requirements that are just a matter of doing it, just a matter of writing it up, mostly.

Sjors Provoost Yeah, there's some tedious formatting rules that are explained.

Aaron van Wirdum Yeah, exactly.

Sjors Provoost But usually what people, I think, will do is they copy-paste an earlier BIP and just turn it into their own BIP.

Aaron van Wirdum Smart. So now we have a BIP number. Now what?

Sjors Provoost Well, I guess the process kind of continues because now that it has a BIP number, more people will pay attention to it. So you'll probably get a little bit more feedback. Other people may, you know… if it's something that needs to be implemented, then other people may try to implement it. Now, if it's a soft fork, there's a lot more to it, I would say, because that's a consensus change.

Aaron van Wirdum Does it make the BIP binding?

Sjors Provoost Well, nothing is binding because we're still dealing with this set of anarchists, essentially, right? The Bitcoin community. Ultimately, what you do in a BIP can end up in a piece of software. And that piece of software still needs to be installed by somebody. And they might just not like it. Which is one reason why you really need to seek out this consensus. Because if people think that there was no consensus over something, they might just not run it.

Aaron van Wirdum Yeah, I think that's an important thing to point out, that the BIP process and the overview of BIPs, it is informational. I mean, well, that's also a subcategory of a specific type of BIPs which we mentioned, but just the whole thing in itself is informational. It doesn't actually define Bitcoin or even anything in Bitcoin. It is a way of communicating concepts, but it… it doesn't mean that now, you know, Bitcoin in the end, is still defined by people running the software and by the software they're running. The BIP process…

Sjors Provoost Exactly.

Aaron van Wirdum The BIP process is… the BIPs are a tool. They’re not Bitcoin.

Sjors Provoost Yeah, and a lot of these rules that are specified in the BIP 2 are just to make sure that mailing list discussions don't take forever and that there's some predictability and what's going to happen when you when you do something. But ultimately, if you're not happy with the whole way BIPs work, you're completely free to run your own website and say, hey, I have a bunch of numbers and a bunch of documents, and I have very different rules. And then people might ignore that or not.

Aaron van Wirdum Yeah, that was sort of the next question I was going to ask, for which I, of course, know the answer, Sjors, but I still am pretending to be the journalist here. So Sjors, Sjors. What happens? So Luke Dashjr is currently the BIP editor. What happens if he becomes evil? Luke turns evil. What's the problem?

Sjors Provoost Well, first of all, the BIP process is such that being evil shouldn't be a big deal because it's a fairly bureaucratic thing. But there is some subjectivity involved always, you know. Some people suggested using a robot to do this, but that just invites spam and problems. So an evil person could, you know. do things to the bureaucratic process that frustrate the community. Well, then somebody [unclear] the thing and just says, okay, now this is the new BIP place and probably copies everything that already exists over.

Aaron van Wirdum Right.

Sjors Provoost I mean, that's not, that's not ideal, but I mean, this is the same with, it's, it's even easier than forking the Bitcoin soft software, right? Because it really just, you just clone the website and say, here's a new website, or here's a new GitHub repo.

Aaron van Wirdum It's hassle, but it doesn't actually do anything. Yeah, so you mentioned GitHub already. The BIP lists, the BIPs are currently hosted on GitHub, is that correct?

Sjors Provoost Yeah, and I mean, it's a Git repository, remember? So everybody can clone a Git repository on their own computer. So there could be thousands of backups. So if GitHub decides to go evil, that is really not a problem.

Aaron van Wirdum Yeah, they used to be hosted as wikis for a while, but I think... A long time ago, I think probably like eight years ago or something, they transitioned to GitHub. So they've been hosted there ever since.

Sjors Provoost Yeah, wikis are nice because you can do collaborative editing. But on the other hand, if everybody starts editing the BIP, then some people might not understand that something is already implemented. And they start changing the specification of Taproot right from under you. And then you get confusion because one node is implementing an earlier edit than the other. So it's probably good to have a benevolent dictator with very limited powers having one place to edit the specs.

Aaron van Wirdum Yeah, every BIP has one specific champion is what it's called, I think, right? Or maybe sometimes several. But yeah, someone who's basically in charge of that specific BIP and who's responsible for updating it or taking

Sjors Provoost [unclear]

Aaron van Wirdum feedback into account.

Sjors Provoost Right. Sometimes when, for example, you want to fix a typo in a BIP or, well, not a typo, but like add a picture, add a chart to it or some clarification to it. What the BIP maintainer will do, then the editor will tag the original authors on GitHub and say, hey, what do you think about these changes? And if all the original authors say, yep, that looks good, then it's merged. So the original authors are kind of in charge of the BIP, but everybody can help improve it.

Aaron van Wirdum Right.

Sjors Provoost Which is a little bit less anarchy than a wiki where everybody can just go in and edit it without anybody signing off on it.

Aaron van Wirdum Makes sense. All right, I think this is going to be my final question for you. Sjors, are you ready?

Sjors Provoost I think so.

Aaron van Wirdum So I think there are sort of different specifications of BIPs. But that's not quite what we mentioned, is it? Like hard fork, soft fork, consensus, like these kinds of categories?

Sjors Provoost I think you mean layers.

Aaron van Wirdum Oh, that's it. You know everything, Sjors.

Sjors Provoost Right. So the layer, so you could be talking about a consensus change, like a soft fork or a hard fork. You could be talking about the peer-to-peer networks or how nodes communicate with each other. Or you can be talking about the RPC, which is like a part of the node itself. And you can be talking about applications, I guess, like wallets and things like that.

Aaron van Wirdum Right, yeah, and that's all specified in the BIP.

Sjors Provoost Yeah, and I guess depending on the layer, there are some guidelines on what the standard of quality should be for what you're writing. So, for example, when you want to change anything about the peer-to-peer network, I think there needs to be at least one implementation, but also there needs to be... In order to make it final, which is one of the states that the BIP can be in, there needs to be at least a number of nodes out there in the wild actually using this that you can detect to gauge whether this thing is actually being used. With the software, it's a bit more obvious that it's being used. It activates in ways we've described in earlier episodes. But with the peer-to-peer networking... You know, not everybody has to upgrade at the same time. So nobody might be using a new feature or everybody might be.

Aaron van Wirdum Yeah. Did you ever draft a BIP?

Sjors Provoost No, I have helped review a couple of BIPs, but I have not written one myself. Too lazy.

Aaron van Wirdum Would you like to? I have a great example that you could use as a BIP if you want.

Sjors Provoost I think I know where you're getting to. No, I would rather not.

Aaron van Wirdum Imagine how much energy we can save, Sjors.

Sjors Provoost Yeah. I don't have the energy, though, so unfortunately.

Aaron van Wirdum All right, yeah. You're not putting in any proof of work.

Sjors Provoost And I think that was our pun of the week. Anything else?

Aaron van Wirdum No, I think that’s it from me, Sjors.

Sjors Provoost All righty then. Well, enjoy the heat and the conference.

Aaron van Wirdum Yeah, if you're listening and you see me at the conference, come say hi.

Sjors Provoost Yep. If you're listening and you see me at the conference, it's not me. So run away. And thank you for listening to the Van Wirdum Sjorsnado.

Aaron van Wirdum There you go.