Bitcoin, Explained 65: Bitcoin Core 24.0

Aaron van Wirdum Live from Utrecht, this is Bitcoin.

Sjors Provoost Explained.

Aaron van Wirdum Hey Sjors.

Sjors Provoost What's up?

Aaron van Wirdum I'm good.

Sjors Provoost How do you like the weather?

Aaron van Wirdum Yeah, well, it was too hot all summer. And then it was nice for about a week. And now we're back to too cold.

Sjors Provoost So this is going to be a great winter.

Aaron van Wirdum That's that's winter for you at least for now it's still a little bit light for example while we're recording it's still light and it's past six that's that's the really dreadful moment for me when the clock changes when you go back to the we revert…

Sjors Provoost Yeah we go to standard time although some people want to change it and make the summertime the default which I think is terrible but…

Aaron van Wirdum I would definitely prefer that just because I don't get up that early anyways. So I like to optimize my sun hours.

Sjors Provoost I mean, ultimately, you can just get up whenever you want to get up, right? Unless you have to be in a physical place.

Aaron van Wirdum But you're kind of reliant on the rest of society around you, right? Like stores close at a certain time and people get out of work at a certain time. Like might be your friends or you want to record a podcast at a certain time because you want to eat at a certain time. So I don't have that full flexibility that you suggest because of all these people around me okay Sjors, so I'm gonna shill the conference one more time conference is coming up okay you can get a 10 discount if you know how to spell Sjorsnado that's your discount code Sjorsnado on I think it's b.tc slash amsterdam I think that's the that's the website for the conference all right Sjors, we're gonna discuss The new Bitcoin Core release that's coming up. It's not here yet. It's Bitcoin Core

Sjors Provoost 24.0.

Aaron van Wirdum 24.0 that's the upcoming one real quick for maybe there are some new listeners what does it mean that there's a new Bitcoin Core release coming out?

Sjors Provoost Yeah so because Bitcoin Core does not have a roadmap or anything like that there is just a new release every six months or so and things that are ready go into that new release and things that are not ready may go into another release so there's nothing very special about these releases.

Aaron van Wirdum Right. And so it's time for number 24. It's just an update that happens twice a year. Generally, whatever is done is done.

Sjors Provoost Yes. And then there are some minor updates throughout the rest of the year that might be 24.1, 24.2, et cetera. And those typically are just bug fixes, but no major changes.

Aaron van Wirdum Right. And right now, there's a release candidate. At the time of recording this podcast, there's, I think, the first release candidate for Bitcoin Core 24. What does a release candidate mean?

Sjors Provoost Kind of what the word suggests. It's a candidate for the release. So if all goes well, then the actual release will be made identical to the last candidate. But the idea is that people who know what they're doing should be downloading that release candidate and or compiling it themselves and play around with it and make sure that nothing crashes, that things they use, especially if you run some sort of automated service, like you might be running an exchange or maybe you're running BTCPay or something like that. You want to make sure nothing breaks. In your setup because Bitcoin Core tries to not break things that use it but it happens occasionally.

Aaron van Wirdum Right so if there's no report of anything breaking then this release candidate will basically be the release right?

Sjors Provoost Yeah but I think there's already some some things that have been improved so there will definitely be a second release candidate.

Aaron van Wirdum Okay yeah and I think in previous Bitcoin Core releases there were on average like four or so.

Sjors Provoost Yeah I'd say that's typical.

Aaron van Wirdum Yeah.

Sjors Provoost So usually there's two weeks between them. So you release it and then maybe after a week or so people start fixing things and then after two weeks there's a new one.

Aaron van Wirdum Right. So realistically, well, technically the new Bitcoin Core 24 release could be released any day now, but practically speaking will probably be at least a couple more weeks, right?

Sjors Provoost Yeah, I'd say so. I think there is a release, attempted release schedule on the Bitcoin Core repository, which I think is for mid-October.

Aaron van Wirdum Right. Okay. So there's a bunch of new features as well as bug fixes and improvements, sorry, performance improvements in this new release. We're going to discuss a couple of these. This is kind of just our selection. If there were, if this podcast was made by two other people. By Ors and Sharon. They might make a different selection, but we, Sjors and Aaron, we made this selection.

Sjors Provoost Yeah, I mean, you can look at all the commits, so all the atomic changes that go into a release, and there could be a thousand of them, but most of them are very boring, or at least they are very boring unless you are really into details of how compilers work or into cleaning up code for the sake of cleaning it up. So there's a lot of changes that are just... not very interesting to discuss although they are important and they should be they are being you know they they always get reviewed because any of those changes could of course be a really scary thing yeah.

Aaron van Wirdum I'm just saying we we made a selection of some of the changes that we think are probably most interesting to you our dear listener

Sjors Provoost That's right.

Aaron van Wirdum Okay so let's start Sjors, I think that this one is the first one mentioned in the release notes as well it's about the peer-to-peer and network changes is that is that a typo it should just say peer-to-peer changes peer-to-peer network changes whatever

Sjors Provoost Peer-to-peer networking

Aaron van Wirdum Yeah let's move on. So it says that there's a change and you're gonna explain to me what this is that's the format of the podcast in case you forgot it says download headers from peers so the downloading from downloading of headers from peers has been reworked

Sjors Provoost That's right.

Aaron van Wirdum Somehow okay.

Sjors Provoost Yeah, this one might be worth a separate episode, but I think we can explain the gist of it now. So when your node first starts up, it is going to find peers, which we discussed in earlier episode, and then it's going to download headers from peers, which we've also discussed in earlier episodes. So it tries to get all the headers first, and then...

Aaron van Wirdum Hang on, so every block has a header, which is essentially the hash of the block itself, right?

Sjors Provoost Plus a little bit of extra info that... for example, contains the timestamp and the amount of proof of work. Well, the amount of proof of work follows in the hash, but yeah.

Aaron van Wirdum It's basically hash plus some extra info.

Sjors Provoost Yes.

Aaron van Wirdum So it's not the whole block. Okay. So first you start downloading not the whole block or not all of the blocks, but you start downloading the headers, right?

Sjors Provoost Yes, and the idea is that you first want to make sure that the chain that you're following has enough proof of work that it's even worth downloading the blocks for. Now, that's been improved many years ago. It used to be that I think it would just download headers and then some blocks and then some headers and then some blocks. And that has been changed many years ago to first download all the headers. And only then when you know for sure that you have enough proof of work because you can check that based on the headers, then you start downloading the blocks. This prevents you from wasting time chasing and basically chasing dead ends.

Aaron van Wirdum Okay, that was already the case. So even in Bitcoin Core 23, you would already start with downloading only the headers and only later the blocks. Okay, so what has changed?

Sjors Provoost So what has changed now is that we're going to download the headers twice. And that might take some explanation of why you want to do that. So there is... The potential problem that you're worried about with headers is that you could receive quite a lot of them. Because, you know, right now there have been about 700,000 blocks since the genesis, but a miner could manipulate the timestamps and they could create a fake blockchain with blocks that are every second. And that would be billions and billions of... of blocks.

Aaron van Wirdum Well I guess anyone can can only my only miner can do well anyone who can mine you you do need basically the ASIC hardware to do that

Sjors Provoost Well you'd have to do the math on what kind of hardware you would need to do it I mean in principle you don't need an ASIC to mine blocks right.

Aaron van Wirdum You need the proof of work is what I'm saying yeah you need the actual proof of work.

Sjors Provoost Yeah but I mean you know the the proof of work in the beginning was quite low so you can do that with the CPU but maybe if you want to you know do some damage maybe you need more power anyway uh so the the worry is that you get billion headers from a chain and it's not the real chain but it's a lots of low work nonsense and the problem with that is that not just do you have to download it because that's kind of hard to avoid but you also tend to store it on your disk because maybe you receive half a billion headers And those don't go anywhere, but you don't know. Maybe there's another half a billion headers on top of that that do take you to the real tip. So you can't really throw those headers away that easily. And so the attack would be that lots of headers are sent to you. And that attack has been known for a long time. And the solution to that so far has been checkpoints.

Aaron van Wirdum Okay, hang on. Let's summarize this. So first, the problem was solved. The first problem

Sjors Provoost Was just getting lots of spam blocks.

Aaron van Wirdum Yes. So that problem was solved by only checking the headers. But now, basically, a new problem is introduced. Namely, someone could just send you a boatload of fake headers, essentially.

Sjors Provoost Yeah.

Aaron van Wirdum And then while downloading all these headers, you also have to store it. This is resource intensive.

Sjors Provoost Yeah, so it's basically the general category of problem is called a resource exhaustion attack. So I'm trying to exhaust one of your resources.

Aaron van Wirdum And then you mentioned this was solved with...

Sjors Provoost Checkpoints. This has been discussed very early on. That's kind of what the checkpoints are for. Fortunately... Well, checkpoints have not been added since 2013 because they are, you know, not very nice way to do it.

Aaron van Wirdum Just to be very clear, what is a checkpoint?

Sjors Provoost So what a checkpoint does is it says this specific block with this hash must exist in the chain.

Aaron van Wirdum Right.

Sjors Provoost So it's not optional.

Aaron van Wirdum Right.

Sjors Provoost And that has always been done retroactively, like a long time after. So a way to abuse checkpoints is to say, well, somebody stole my coins. I'm going to now introduce a checkpoint that happened before somebody stole my coins for a new block that I've created that does not steal my coins. And then, you know, we start history again. That's the worst case.

Aaron van Wirdum That should be after someone.

Sjors Provoost So somebody steals my coins.

Aaron van Wirdum Yeah.

Sjors Provoost Let's say that happens today. Then I call up a miner and say, could you make a couple of new blocks that don't steal my coins? Here's the double spend version that I want to include in that block. They mine it for me. And then I release a new version of Bitcoin Core that says, here's a checkpoint. You must ignore the real big chain that has built on top of the hack and actually go for this other chain.

Aaron van Wirdum Yeah, so with the checkpoint, you can sort of overrule the longest chain rule.

Sjors Provoost That's right. And that has been done in many altcoins. So the concern is that we don't want that to happen in Bitcoin. So one way to prevent that, I think, has been from the beginning to introduce the checkpoints only much after they've already happened. So you already know they're part of the longest chain. Everybody can verify that. And the checkpoint is buried quite deeply. But it's still not a very pretty solution. And so people stopped doing it. So it'd be nice, but it'd be very nice.

Aaron van Wirdum Did you mention when the last checkpoint was?

Sjors Provoost I think it was 2013.

Aaron van Wirdum Right.

Sjors Provoost It's quite a long time ago.

Aaron van Wirdum It's a while, yeah.

Sjors Provoost Yeah. And so it'd be nice to get rid of those things entirely because, you know, they are confusing. And, you know, we probably don't want to add new ones. So in order to get rid of them, you still need to fix this resource exhaustion attack that we just talked about. So then how do you do that? Well, the trick is to download headers twice, as I said. So what you do is you download them once. And when you download them once, you don't save them. You just... Look at them, check if they're correct, and you throw them away. So this means it does not use any of your disk space. And then if you see enough work at the end, if you checked all the headers and you see the proof of work is enough, it's the longest chain, then you ask the peer, hey, can you send them again, please? And you download them again.

Aaron van Wirdum Right. Yeah. I think I mentioned yesterday when we went over this in our preparation, this is kind of how my attention span usually works.

Sjors Provoost Exactly. You first let me ramble on a bit and then you think, oh, this might actually be interesting. And then you say, what did you say again? Please repeat.

Aaron van Wirdum Can you repeat that for me?

Sjors Provoost Yeah.

Aaron van Wirdum Yeah, so this is how Bitcoin nodes now actually work.

Sjors Provoost That's right.

Aaron van Wirdum They first listen, they don't store anything, but then if it sounds actually interesting, if they see the proof of work, they just ask again and they get all the blocks again. They're like, wow, that's cool. They get all the headers again, I guess, or do they now also get the blocks?

Sjors Provoost Well, then once those headers have been downloaded the second time, things continue as always. You just start fetching blocks for the headers. Now there is, of course, a little gotcha there because, well, who says that the second time you're getting the same headers is the first time? You don't know that because you didn't store them. So what you do instead is you store one checksum, which is a one-bit checksum every 50,000 blocks or so. So every 50,000 headers, you store a zero or a one, depending on the contents of those headers. And you do that every 50,000 headers, that is very little information, but it turns out that it's actually quite difficult for anybody to fake that information. So it's… for… anybody to create fake headers that do match your checksum, even though it's a very small checksum.

Aaron van Wirdum Right, yeah. A checksum is essentially you add up all kinds of numbers and you get a very short number. Which doesn't in itself prove anything, but I guess if you do it a bunch of times, then...

Sjors Provoost If you added the same numbers, you're going to get the same checksum.

Aaron van Wirdum Right, exactly.

Sjors Provoost So if you have 50,000 headers, you add all those up together, then you get either a 1 or a 0. If you change any of the headers, well, you'll either get a 1 or a 0, but it might be a different one. Now that, of course, is a 50-50 chance. So that's pretty easy for a hacker to go after, but there are many blocks. So it turns out that for long enough... I might be wrong on the exact number. It might be less than 50,000. But if you have enough of those one-bit checks, basically, then it becomes quite hard for an attacker to create a fake chain that has enough proof of work, but that is different from the last one they sent you.

Aaron van Wirdum Yep. Okay, that makes sense.

Sjors Provoost Yep.

Aaron van Wirdum So the... There's still a small part of the problem left, which is you do still need to download it the first time.

Sjors Provoost Yes.

Aaron van Wirdum But it solves another part of the problem, essentially.

Sjors Provoost Yeah, so the downside is you're downloading headers twice, so that might be another 100 megabytes or so in the ideal case. However, compared to the size of the blockchain, that's not too bad. And compared to the worst case attack, it's definitely not bad.

Aaron van Wirdum And of course, even if an attacker would want to try this for some reason, it's also costing the attacker resources because he has to upload all the data, the same data you have to download. It doesn't really do much, this attack, right? It can't allow you to steal coins or anything like that.

Sjors Provoost Well, with this defense, no, because all they can do is waste your bandwidth. And there are many ways to waste your bandwidth, right? An attacker can just send you a gigabyte block and... or just complete gibberish in general. So that's not a new problem.

Aaron van Wirdum Okay, so this was included in Bitcoin Core 24 and will be...

Sjors Provoost Yeah, and the checkpoints are still there, but the idea would be to remove them eventually.

Aaron van Wirdum Okay, so that would be removed in a future release.

Sjors Provoost Yeah, and of course, you know, there has to be some additional discussion to make sure that that was really the last thing we needed, that the checkpoints are not also protecting against something else that we forgot about.

Aaron van Wirdum Right. Okay. Well, that's the peer-to-peer part of Bitcoin 24. Then the next point is Mempool uses full RBF now or can use full RBF?

Sjors Provoost Can use full RBF.

Aaron van Wirdum Right.

Sjors Provoost So it used to be that if you wanted, I think we've done an episode about replaced by fee RBF.

Aaron van Wirdum Yes.

Sjors Provoost So I think the listeners should listen to that.

Aaron van Wirdum Well, you can summarize it in two sentences.

Sjors Provoost Yeah. So basically the Bitcoin protocol itself, when you have one transaction and then you want to double spend it. There's nothing stopping you from doing that before it's in a block, right? Now, from an incentive point of view, miners are most likely to include the block with the highest fee, but they don't necessarily have to. And so there was a proposal by Peter Todd many, many years ago to say, well, normally the nodes, well, let's go one step back. So what miners do is ultimately up to miners. You have no control over that. And Bitcoin Core can change things in the software, but miners will do whatever they do because what the mempool does is not consensus. However, the nodes will relay transactions. And so you can change the nodes to say, well, I'm going to... broadcast some transactions and not going to rebroadcast other transactions. And so this new rule, opt-in replaced by fee, opt-in RBF, basically said that normally we only broadcast the first version of the transaction we see, regardless of the second one. It doesn't matter if the second one pays more fees. But if you put a flag in the transaction that says, I want to opt-in to replace by fee, then nodes will relay transactions that pay a higher fee only and a bunch of other constraints. And so this gives the recipient some assurance, not much, but some assurance that if this flag is present, this transaction could be replaced anytime. You really have to wait for it to confirm. If it doesn't have this flag, it might still disappear. It's just a little bit less likely.

Aaron van Wirdum Yeah, I want to clarify one thing because you mentioned double spending in this context. And you are, of course, right that this can be used to double spend until it's included in a block, as you said. But the main purpose or the main idea for using a flag, for example, is to increase the fee on your own transaction, right?

Sjors Provoost That's definitely, I don't know how long that idea has been around because the idea of being stuck in a mempool, that concept didn't even exist until 2015 or 2016. That was never a problem.

Aaron van Wirdum Yeah, but the flag was introduced around that time and definitely in the context of that. That was the debate.

Sjors Provoost Okay.

Aaron van Wirdum Yeah, I'm sure about that. So to summarize that, at least the idea for the flag was you send a transaction, but the mempool was full and therefore your transaction is not confirming. Now with the RBF flag, you can basically resend the same transaction with a higher fee and therefore... Because the flag is included, nodes will actually forward it to miners and it can be included in a block. Now, you don't actually need to include the flag anymore. If this setting is turned on?

Sjors Provoost Yeah, I think that if you turn the setting on, then the flag does no longer have to be in there. It'll relay it anyway. I'm guessing there are other rules, though, because I assume the fee has to go up. But I'm not sure about that.

Aaron van Wirdum So in my case if so far I was running a Bitcoin Core node let's say what was the previous one 23 of course so far I was running Bitcoin Core 23 and then my node would not forward a transaction even if it had a higher fee and it conflicted with the previous transaction it would not do that but now with Bitcoin 24 I can switch the setting and now it will actually forward that transaction?

Sjors Provoost Yeah, I think that's it. And it basically means that if you're relying on this opt-in RBF system to prevent double spends, basically to prevent double spends, you should now rely on that less so because there's going to be more nodes that will relay this thing regardless.

Aaron van Wirdum Yeah, well, I mean, it was almost... Well, I don't know about almost certainly, but there's definitely a much bigger chance that a conflicting transaction will make it to a miner now, right? Because I think there only needs to be a relatively small amount of nodes on the network that actually do it for a transaction to just find its way over the entire network.

Sjors Provoost But there was already ways to do it, right? There was a patch by Peter Todd, which was pretty small, and you could use that to modify your nodes. So there's already some nodes doing it. Now there's more, presumably.

Aaron van Wirdum Right. So does this mean that some merchants are going to be unhappy?

Sjors Provoost Well, they might be for that reason.

Aaron van Wirdum There are still some merchants that rely on zero-conf.

Sjors Provoost Yeah, there are definitely merchants that do that. But the flip side is that there's also merchants that use Lightning. And Lightning is generally bothered a lot by the existing RBF rules because they make it much more complicated to deal with penalty transactions. So on the long run, they may get a better Lightning experience out of this.

Aaron van Wirdum Right. Yeah I mean my personal opinion is that this should just be the general rule. Transactions should always be broadcast and forwarded, especially if they have a higher fee.

Sjors Provoost I mean, maybe. We've already had this discussion during that episode, I guess. But I would also say that if you just turn on the RBF flag by default, it shouldn't bother you either. But I think the bigger problem is that this RBF flag has very specific rules. It's not just that you set the flag. You also have to do a few other things. And those other things cause complications. So that could be a reason to say, you know what, let's forget about this opt-in RBF completely and relay everything that is reasonable. But there are trade-offs. One is just bandwidth. Because if my node relays everything that you send to it, I could send you one transaction and then increase it by one satoshi per byte or increase it by, don't even increase the fee, but just change the destinations a bit. And I could send you millions and millions of variations of the same transaction, basically wasting everybody's bandwidth. So it's not entirely without trade-offs.

Aaron van Wirdum Fair. All right, let's move on to the third point. So we mentioned change on the peer-to-peer network, the block header thing, and we just mentioned RBF. And then the third point is this is related to descriptor wallets and migration to descriptor wallets.

Sjors Provoost Yeah, I'm not sure how much we covered descriptor wallets so far, but the gist of it is that the Bitcoin Core wallet is quite old. It used to be just a bag of keys. And then, you know, given a private key, the wallet would have to pay attention to certain scripts. I think in one of the first episodes we explained what scripts are. But the simplest scripts is just that anybody with the public key can spend this coin. The second simplest script is anybody with the hash of the public key can spend the coin. Or anybody with the public key for which the hash is blah, blah, blah.

Aaron van Wirdum Yeah.

Sjors Provoost But then later on came SegWit, and that created another two variations of how you could spend the coins.

Aaron van Wirdum Right.

Sjors Provoost Because you had the address of bc1, and you had the address that looked like a P2SH address. With SegWit wrapped in it.

Aaron van Wirdum Right.

Sjors Provoost So the wallet just became a giant mess.

Aaron van Wirdum Also, you have multisig rates and stuff like that.

Sjors Provoost Well, yeah, that makes it even more complicated. But basically, the Bitcoin Core wallet became a bit unwieldy. And so one of the improvements that was introduced was to create a new way to store data and basically saying, well, here's keys, but more importantly, here's exactly what scripts you want to watch for. So you can specifically say, I only want to check for Taproot transactions for this private key and not also for legacy transactions for this private key. Now your wallet handles all this.

Aaron van Wirdum What's the practical benefits of that? Just less resources?

Sjors Provoost No, for the end user, this doesn't matter. It just means that their wallet software is better maintained because it's less of a headache for the wallet developers to deal with this. So it basically involves a giant rewrite, mostly done by Andrew Chow. And one of the challenges there is we have this old wallet and we want to move people over to the new wallet. And so the first step for that is this new RPC call called migratewallet, which does what it says it does.

Aaron van Wirdum It will migrate your wallet?

Sjors Provoost Yeah. And for the simplest cases, you know, if you created a wallet in the last couple of years and you didn't do anything super fancy, very complicated multisig setups, whatever, this will work. If you did do something complicated, then please test it. It'll make a backup, but maybe make your own backup too.

Aaron van Wirdum Right.

Sjors Provoost And we'd like to see bug reports basically, because maybe you have some super complicated wallet setup that does not migrate properly.

Aaron van Wirdum Right. So basically what this does is your wallet software will just go over all of the UTXOs and say, well, this is a pay-to-public key. This is a [unclear]…

Sjors Provoost No, it will not go over the UTX—

Aaron van Wirdum And it will throw it in the different buckets.

Sjors Provoost No, it will go over the keys inside the wallet. The transaction list you have is kind of the same. It'll look through what keys you have in the wallet and it will restructure those as descriptors.

Aaron van Wirdum It will throw these in the different buckets.

Sjors Provoost Yeah.

Aaron van Wirdum Okay, instead of one big bucket.

Sjors Provoost That's right.

Aaron van Wirdum Okay, I think that's clear enough then. And as mentioned, this is mostly to benefit just the developers, not so much users.

Sjors Provoost I mean, yeah, the users benefit indirectly, but yeah. I mean, descriptors have already been used to allow Taproot. So if you wanted to use Taproot, you either need to create a whole new wallet or you need to migrate to a descriptor wallet and then add Taproot to it. So there is that too.

Aaron van Wirdum Right. Okay, there's been, moving on to the fourth point, Miniscript support. Has Miniscript support been added to Bitcoin Core?

Sjors Provoost Yes, in very limited fashion.

Aaron van Wirdum Okay.

Sjors Provoost I think we have done a whole episode about Miniscript, but it basically lets you do very advanced scripting systems. Something like I want to spend with two keys, or I want two signatures, or after five years, I want one signature. Or if somebody has the pre-image of this SHA-256 hash, then they only need one signature plus whatever. Miniscript allows you to do fairly arbitrarily complicated things. And Bitcoin Core can now, if you have a piece of Miniscript, it can now watch that. You cannot spend from it yet.

Aaron van Wirdum Right. I mean, the other way to put it, the simple way to put it maybe is a Miniscript allows for smart contracting type of stuff, right?

Sjors Provoost Yeah, I guess. I mean, that might be overselling it a bit, but yeah. To the degree that Bitcoin can support smart contracts, the Bitcoin script language is very complicated. You don't want to write Bitcoin script by hand. You will make mistakes, like even Andrew Poelstra and people like that will make mistakes. So Miniscript is designed to get the most out of the existing Bitcoin script system in a safe way. I think that's the most fair way to put it.

Aaron van Wirdum Right. Sorry. And what does it mean that it's been added now? In what way has it been added?

Sjors Provoost It means you can create a wallet. So if you create a new wallet, you can put a piece of Miniscript inside of that wallet. And it will basically let you create addresses for that Miniscript. And you can send coins to those addresses. But you can't spend them.

Aaron van Wirdum I see. So you can't spend them.

Sjors Provoost Not yet. So you shouldn't do that.

Aaron van Wirdum Okay, yeah, don't do it yet then. And this sounds like you might need to run a separate wallet or a separate piece of software that you connect to your Bitcoin Core node.

Sjors Provoost No, well, in order to create the Miniscript itself, you will need separate software. There is a Miniscript compiler, but you can write Miniscript by hand too if you want to. So in principle, you don't need a second piece of software. Now, the idea is that a future version of Bitcoin Core will also be able to sign for it.

Aaron van Wirdum Sure, yeah.

Sjors Provoost But you shouldn't count on that. You shouldn't use it until it does.

Aaron van Wirdum Okay, well, this sounds like a pretty big step then, doesn't it? This sounds to me like it's a pretty big step.

Sjors Provoost I think it is pretty big, yeah. And the other constraint is that it only works for the first version of SegWit. So those are addresses with bc1q. So it does not work for Taproot yet. And because there's a lot of work that needs to be done, or at least there is work that needs to be done in order to have Miniscript work with Taproot at all. And then once it does that in general, so the specification of Miniscript can handle it, then the next challenge is to actually get that into Bitcoin Core. And to really use Taproot and Miniscript well, you probably want MuSig too. The signature aggregation stuff.

Aaron van Wirdum Yeah.

Sjors Provoost That also needs to be added to Bitcoin Core. So there is a ton of work still left to do. But this is one step.

Aaron van Wirdum Is there currently, so we're right now talking about Bitcoin Core, obviously. Is there currently other Bitcoin software that already allows you to create and use Miniscript?

Sjors Provoost No, as far as I know, there is basically a library called Rust Miniscript and a library written by sipa that does Miniscript and C++. Those libraries you can use, but they're not fully functional wallets. So I don't know if anybody's using Miniscript in a while. Maybe Blockstream is for their, what is it?

Aaron van Wirdum Liquid sidechain.

Sjors Provoost Liquid sidechain stuff. I would know.

Aaron van Wirdum Okay. Well, interesting. Like I said, that sounds like a pretty big step actually.

Sjors Provoost Yeah. And it would also, I think this is the longer term trend of hopefully turning Bitcoin Core from a, not a very good wallet. It was quite slow, quite weird to hopefully a very good wallet. And at least a very good wallet for power users, right? So maybe exchanges want to use it more. And my hope is that if this wallet becomes very powerful, then more people will help review the code and more people will help improve it. So you kind of get this self-amplifying effect. Once it's useful enough, more people will look at it, but it takes a very long time to become useful enough because it's not useful enough, so not enough people are working on it.

Aaron van Wirdum Got it.

Sjors Provoost It's really, I think, maybe four people.

Aaron van Wirdum All right, moving on to the next point.

Sjors Provoost Yep.

Aaron van Wirdum Okay, the next point on our list, this is another point about RBF.

Sjors Provoost Yeah, but this is just about opt-in RBF, that the wallet RPC, so the command line wallet, will now use it by default.

Aaron van Wirdum Oh, that's right. Okay, wait. So we're not talking about the GUI, right?

Sjors Provoost No, the GUI already switched to use RBF by default.

Aaron van Wirdum Right. And now the command line does as well?

Sjors Provoost Yeah.

Aaron van Wirdum So why didn't... Okay, go on. Maybe I should just let you finish.

Sjors Provoost So why didn't it?

Aaron van Wirdum Yeah, why didn't it yet?

Sjors Provoost So one of the reasons is that people are using the Bitcoin Core wallet in some automated systems. So if you're running a Bitcoin ATM, you may have some really old software that... does not know that RBF exists, maybe, because it's really old. And it might get confused if it's turned on by default. So for those kind of systems, you don't want to change things too quickly. But I think it's been five years now, so it might be a good time to switch to default.

Aaron van Wirdum [unclear] enough time.

Sjors Provoost Well, hopefully.

Aaron van Wirdum Okay, so would we expect more RBF transactions on the network now?

Sjors Provoost We'll have to see. It depends on when people start using it. My hope is no, because my hope is that anybody... who you know builds this kind of software already decided whether they want RBF on or off because it's just a default value you probably should already set that default to what you want it to be so that you're not surprised by this update so I hope to see no difference right because the worst case scenario would be that some major company that has RBF turned off for some reason now suddenly turns it on and starts producing lots of RBF transactions confusing everybody in their ecosystem.

Aaron van Wirdum Yeah, it wouldn't really harm them in any way, would it?

Sjors Provoost I don't know. Maybe people are using that service to send coins to some other service that does not like it when transactions are RBF.

Aaron van Wirdum Yeah, that does not accept RBF transactions.

Sjors Provoost Or gets confused by them. Yeah.

Aaron van Wirdum Yeah, these are out there. Okay, makes sense.

Sjors Provoost So that's why you want to be a little bit slow with these command line tools to change them.

Aaron van Wirdum Right okay I think we've covered five points now and we got we got seven in total so we got two more to go all right number six the change output amounts are randomized it says that's what it says.

Sjors Provoost It's very cool. Yeah, so basically this has to do, I mean, I don't know the precise change, but the general problem is that when you're looking at the blockchain, you can kind of guess which address is change and which address is not change if you have an assumption about how the wallet picks coins. So if the wallet is very efficient, it will look for the smallest coin possible to spend, and then it will create as little change as possible. But that means that generally when you see two outputs to a transaction, the biggest one is probably the amount that's being sent and the smallest one is probably the change.

Aaron van Wirdum Right. Well, is that necessarily true?

Sjors Provoost It's not necessarily true.

Aaron van Wirdum That's maybe true if there are several inputs, right? That logic only holds up if there are several inputs, I think.

Sjors Provoost Or if there's one input.

Aaron van Wirdum Well, if there's only...

Sjors Provoost Let's say you... Well, it depends on how much there's available, right? But you're looking at the change, so you don't know what was available. But if your wallet is looking at a bag of coins it might you know start with the smallest coin saying no it's not big enough and then keep looking and then find one coin that's big enough and then use that to spend it.

Aaron van Wirdum Right yeah.

Sjors Provoost And if you it doesn't really matter but if you know what software the person was using based on some other fingerprinting aspects You can start then. You know how that software works. You know how that software does its coin selection. You can use that against the user.

Aaron van Wirdum Okay, so do you know what algorithm Bitcoin Core was using up till now?

Sjors Provoost It's using a bunch of algorithms.

Aaron van Wirdum Oh, several. And do you know which ones or no?

Sjors Provoost No, you should ask Murch.

Aaron van Wirdum Okay.

Sjors Provoost He wrote a whole thesis on it and implemented a bunch of things.

Aaron van Wirdum But something changed.

Sjors Provoost I think originally the system was called the knapsack. And I think the knapsack basically meant you just grabbed coins randomly and then see if it was enough. And then if it wasn't, you try it again. And then there was a new thing called branch and bound, which was a bit more smarter way to find coins. And there's also the try to find a single coin that is exactly the right value strategy. There's a bunch of things that are happening. It's a bit complicated. But what this basically does is make it more difficult to tell which one is the input. Sorry, which one is the change and which one is the destination. And the way you do that is basically saying, well, rather than looking for the coin of the exact right size to spend, I'm going to look for a coin that is the amount that I need to spend plus two times the yeah I'm going to look for a coin that is at least three times I guess the amount that I'm trying to spend right.

Aaron van Wirdum So you want to spend one Bitcoin so you're going to look for a UTXO of three Bitcoin.

Sjors Provoost Yeah except you're not going to do that you're going to say I'm going to pick a number between the amount I want to spend and three times the amount I want to spend

Aaron van Wirdum Between one and three.

Sjors Provoost Yeah and I'm going to pick that number randomly So that means sometimes the biggest, sometimes you're going to pick a very high number. And so the change address will actually be the smaller, will be the bigger output. And sometimes I pick the lower random number and the change output will be the... biggest amount.

Aaron van Wirdum Okay so we're basically gonna keep using the algorithm that we were using or at least I don't know but that's…

Sjors Provoost Yeah I think that doesn't change.

Aaron van Wirdum Yeah however we're now gonna look for coins as if we're looking for one to three times more than what we are actually spending.

Sjors Provoost Exactly and that number is random.

Aaron van Wirdum Right okay that sounds interesting doesn't that mean that you sometimes end up using more UTXOs than you would actually require? In other words, wouldn't you make transactions bigger than they really need to be?

Sjors Provoost Yeah, I think that might mean that sometimes you are spending more coins than necessary, yeah.

Aaron van Wirdum Okay, so it's not optimizing for fees then? It's not optimizing for...

Sjors Provoost I don't think so, but there may be some caveats in that code. You'd have to read through it to see what exactly it's doing. Maybe there's some protection against spending too much on fees.

Aaron van Wirdum Right, maybe it only works if this is not... the case.

Sjors Provoost I mean if you're using the GUI you can also do your coin selection completely manually right there is a little menu where you can sure see what your inputs are and just select which ones you want to spend.

Aaron van Wirdum Yeah okay so there might be trade-offs that we're not completely clear on but the general idea is you're going to look for more coins than you actually need for privacy purposes that's ultimately the benefit right? Privacy.

Sjors Provoost I suppose one day we should do an episode about coin selection this is also something I haven't specialized in.

Aaron van Wirdum Yeah, we should maybe. That does sound like an interesting topic. We can go over all of this.

Sjors Provoost Cool.

Aaron van Wirdum Exciting. Last point, Sjors.

Sjors Provoost Yeah, this one isn't super interesting. It's a new RPC called sendall. And so this is a command line tool. And the name suggests that you're sending all your money. That's not actually true. You can use it to send all your money away to a different wallet, let's say if you're trying to migrate. But you can also say, take these coins, so these specific input coins, and send them all to a specific destination. And this is stuff that you could already do because there was one command line tool called and sendmany and sendtoaddress. So there basically were already calls to do this.

Aaron van Wirdum Yeah, also in the GUI, this is already possible, right?

Sjors Provoost Yeah, so the underlying motivation here is actually that there are different intentions that the user might have. They might want to send an entire coin to an exchange for privacy reasons, or they might want to send an exact amount to an exchange. And in retrospect, you don't really know what the user intended, so that creates problems when you try to bump the fee. But also the code that actually implements this is a bit of a mess because it has all these different features. So you can do all these different things for different use cases. That means your code is full of if, then, else, if, if, else, else, if. And so basically this is the first step to splitting that up, to saying, okay, if you want to do this specific thing, use the sendall method. If you want to do this other thing, use the send method. And that's the first step. At the command line, it's easier to tell users to start using two different commands for two different things. And then eventually, I guess, under the hood, there will be some changes to the GUI to also use the separation.

Aaron van Wirdum Okay. I mean, this sounds like something that basically all wallets already do.

Sjors Provoost I don't know.

Aaron van Wirdum Like I said,

Sjors Provoost It's not a new piece of functionality.

Aaron van Wirdum The insulting thing, though, is that other wallets, the universally accepted term for this is send max. Send max.

Sjors Provoost Yeah, but it's send max given the set of coins.

Aaron van Wirdum That's also what send max is, I think, or not?

Sjors Provoost I don't know. Yeah, could be.

Aaron van Wirdum That's how I use it. That's how I think about it. No, but I get your point. Okay, yeah, this was not the most exciting feature of the seven, but I'm happy we covered it, Sjors. Are you?

Sjors Provoost Last but not least. Yeah, and like I said, there's probably at least probably 500 or maybe 1,000 individual changes that could each be worth their own episode if you were truly interested in how compilers work and whatever. So,

Aaron van Wirdum There's that. Okay, well, we covered the highlights. At least we covered our highlights.

Sjors Provoost I think so too.

Aaron van Wirdum And I think that makes for another successful episode.

Sjors Provoost We'll see. Thank you for listening to Bitcoin

Aaron van Wirdum Explained.