Episode 92: Bitcoin Core 27.0

Aaron van Wirdum Live from Utrecht, this is Bitcoin Explained. Hey Sjors.

Sjors Provoost Hello.

Aaron van Wirdum First of all, I'd like to thank our sponsor Coinkite, the creator of the OPENDIME as well.

Sjors Provoost That's right.

Aaron van Wirdum Are we going to get sued if we only mention the OPENDIME, Sjors?

Sjors Provoost No.

Aaron van Wirdum Are we contractually obligated to mention the COLDCARD, or can we just riff off the OPENDIME this episode?

Sjors Provoost If you want to riff on the OPENDIME for the great memories it brought us.

Aaron van Wirdum The OPENDIME is pretty cool. I still have a few.

Sjors Provoost What is an OPENDIME?

Aaron van Wirdum It's like a USB stick, kind of small stick. You put it, you stick it in your computer, you put some Bitcoin on it, you hand it to someone, very cypherpunk technology, off chain, no one needs to know anything. And then that person can check that the Bitcoin are really on there and they can take the Bitcoin off only once. You have to sort of destroy the thing to get the coin off.

Sjors Provoost Yeah, so you can see whether it was spent or not. You actually have to break it.

Aaron van Wirdum Right. Very cool. So we're making another episode about another Bitcoin Core release. I feel like one every four episodes we make now is about a Bitcoin Core release.

Sjors Provoost Yeah, those are a bit of our lazy episodes.

Aaron van Wirdum Well, also, we don't make as much episodes. Or is Bitcoin Core making more Bitcoin Core releases?

Sjors Provoost No, I can, however, tell you that Bitcoin Core version 27.0 will be the best Bitcoin node ever.

Aaron van Wirdum Nice. So, Bitcoin Core 27. As we're recording this, it's in release phase. What is it? Release candidate 1?

Sjors Provoost Yes, exactly. So people who would like to test it, they can.

Aaron van Wirdum For any new listeners, Sjors, what does it mean that there's a new Bitcoin Core release?

Sjors Provoost It kind of means nothing because in Bitcoin Core, there is a new release roughly every six months, regardless of what is finished. So whatever is finished goes into the release. What is not finished does not go into the release. It is not... Yeah, that's special, I guess.

Aaron van Wirdum We should next time just ask Stein, our editor, to just cut this from the previous episode. You don't have to repeat.

Sjors Provoost To see that I'm answering the question inconsistently.

Aaron van Wirdum So yeah, every half year there's a big Core release. Whatever is ready by then goes into the release. Whatever is not, is not.

Sjors Provoost Yeah, though one thing that I believe was observed is that the release started drifting a bit. So, you know, maybe it was in January on one year. And then the next year it was in February, and the next year it was in March. So this release is a relatively short one to try and reset the schedule a bit.

Aaron van Wirdum I see. Okay, so when was the previous one? Do you know this?

Sjors Provoost I think it was in November, maybe. It should be every six months.

Aaron van Wirdum That's six months ago. Okay, so Release Candidate 1, what does that mean? Also for new listeners?

Sjors Provoost Yeah, so what happens is there's a couple of phases to the release. The first phase is the so-called... sort of the bug fix time. So there's a master branch where all the changes go that Bitcoin Core makes. And everything in the master branch should be bug free. But sometimes it's not. And the idea is that a few weeks before the branch off point, which I'll explain in a bit, people stop merging or maintainers stop merging things that are features and try to merge only somewhat more conservative bug fixes, typo changes, that sort of stuff to make sure that there are, you know, to reduce the chance of having any bugs on that master branch in the first place. And then what happens is this branch off, which is where a separate Git branch is created where all the changes, which only has bug fixes. So to understand what that means is there is still the master branch, and people are still developing on that, but anything that goes onto the master branch will not go into the release.

Aaron van Wirdum Wait, but I was asking about the release process, basically. We're in release candidate one.

Sjors Provoost We'll get there.

Aaron van Wirdum Oh, okay.

Sjors Provoost I'm just going a little bit back there.

Aaron van Wirdum Okay, go.

Sjors Provoost So, yeah, so the master branch, people can continue to have fun there. That's going to be for the next release. Meanwhile, there is this separate branch for the release, which has only maybe very minor bug fixes. And then to that branch are added a few things to change the version number. And then that is turned into a release candidate. And that release candidate is first, let's say people test it a little bit who are developers. And then once that's done, there is a binary release that people can download and test that.

Aaron van Wirdum So what do we have now? Where are we? Do we have a binary release?

Sjors Provoost Yes, there's a binary release for release candidate.

Aaron van Wirdum Okay, so how many release candidates are there going to be?

Sjors Provoost Well, so what happens then is for about two weeks, people can test it. And then if they don't find any bugs, then we essentially make the same binary and call it the final version.

Aaron van Wirdum Right.

Sjors Provoost It's not technically the same binary. We talked about hashes before. The version number is in the binary. So a different version number means a different hash. But other than the version number, it should be identical.

Aaron van Wirdum Okay, so the current release candidate could potentially be the next Bitcoin Core 27, but maybe bugs are found, then the bugs are fixed, and then there will be another release candidate, and then that could be the next Bitcoin Core 27.

Sjors Provoost Yeah, and so whenever a bug is fixed, it is first fixed on the master branch, so it also applies to future releases, and then it's backported, as it's called, to the release branch. And that means that the same fix that was applied to the master branch is then also applied to the release branch. That's extra work. So ideally that should not happen too often. Hence the need to not have bugs in the master branch.

Aaron van Wirdum Right. Anyways, as soon as there's no bugs in the release candidate, we have a new Bitcoin Core release, which in this case will be Bitcoin Core 27.

Sjors Provoost Exactly.

Aaron van Wirdum And we are going to discuss some of the... highlights of this new Bitcoin Core release. Why should people download it? Why should people care? Why, dear listener, do you want to use Bitcoin Core 27? Sjors will explain it all today.

Sjors Provoost Exactly. Well, first of all, there's something that's not in Bitcoin Core version 27.

Aaron van Wirdum Yeah, something was jacked out. libbitcoinconsensus, which has nothing to do with libbitcoin.

Sjors Provoost Yeah, although I think I should say it's not removed, it's deprecated, which means that this is your last warning. It's going to go away in 28.

Aaron van Wirdum Okay. Wait, so it's still... Okay, just explain what it is.

Sjors Provoost Yeah, so deprecation is always a warning, like this thing is going to go away the next time. So sometimes if you use a feature that's deprecated, it will tell you, hey, this feature is deprecated. Sometimes you even have to actively set a configuration to use a deprecated feature. So the first time you try to use it, it says no. And then you make a change to the configuration, says, okay, I'll do it, but it's going to go away next time. So, you know, be careful there.

Aaron van Wirdum You've been warned.

Sjors Provoost Yeah. So this thing goes away.

Aaron van Wirdum So what is this thing? And why is it going away?

Sjors Provoost So in the early days, there was a demand to have or a desire to have the core functionality of Bitcoin Core. So the part that checks the consensus rules to somehow split that off from the rest. Other people can make their own node software or do other things and don't care about all the whistles and bells that are in Bitcoin Core. So they don't want a GUI, they don't want...

Aaron van Wirdum So in other words, that would basically be the rules that define which blocks are valid, right? That's really the core consensus rules?

Sjors Provoost Well, yeah, the question is, what is exactly the core consensus rules? That's an open question still. But at the time, this library, this libbitcoinconsensus, was limited to, I believe, things that can check signatures. It wasn't even fully checking a block. It could just verify individual transactions and signatures.

Aaron van Wirdum Okay.

Sjors Provoost So it was a very limited subset of what we consider the validity.

Aaron van Wirdum Okay, but that's because it wasn't finished yet?

Sjors Provoost Yeah, it was a work in progress. It was the first attempt. But it wasn't finished, and it's quite difficult to maintain. Wow.

Aaron van Wirdum So do you know why it wasn't finished? What's hard to maintain about it?

Sjors Provoost Not really. It was just a pain in the ass. And now we have the kernel project, which tries it again and uses a different approach.

Aaron van Wirdum Right. I mean, from what I understand or what I remember, at least years ago when I heard about this, the Bitcoin Core... at least as it was then and probably still, or at least as Satoshi released, it was kind of spaghetti code. Spaghetti code where everything sort of refers to everything else and some of it was consensus and some of it was not. So it was very hard to sort of derive from that mess what actually was consensus. And that sort of was the big challenge to getting this done. So it sounds like this project was started. I think especially Jorge Timon was working on this. And it sounds like he kind of went away and probably people sort of stopped caring about this project. And now Bitcoin Core Dev said, let's just get rid of this thing. Let's start over. Let's start fresh.

Sjors Provoost Yeah, I think so, and I think it was also getting in the way of some other refactoring cleanup projects.

Aaron van Wirdum Right, and this Bitcoin kernel you just mentioned, have we talked about this on the podcast?

Sjors Provoost We've mentioned it a couple of times. The TLDR is it's an attempt, again, to take the essence of Bitcoin Core and make it available for other software. Now, there's still an open question of what should and should not be part of the kernel. For example, does the mempool belong to the kernel or just blocks? And we've seen that even network rules impact consensus. We've seen that very clearly with an episode we did about a btcd bug. Yeah, I think it was a btcd bug for using with LND, where a block was sent and it was rejected. But it wasn't rejected based on the consensus rules. It was rejected based on a network rule, saying that the block had too many, I don't know what it was.

Aaron van Wirdum Yeah.

Sjors Provoost We have a little episode about it.

Aaron van Wirdum Okay. Anyways, libbitcoinconsensus is deprecated.

Sjors Provoost Yes. So as a user, you will not notice this at all.

Aaron van Wirdum Right. Okay. Let's move to something that you might use as a user, might notice as a user. Actually, I don't think the next one is going to be noticeable by a user either.

Sjors Provoost No, it might be. It might be.

Aaron van Wirdum Okay. This is about the mempool.

Sjors Provoost Yeah, to be precise, it's about saving the mempool.

Aaron van Wirdum About saving the mempools.

Sjors Provoost Well, your mempool, yeah.

Aaron van Wirdum Saving the mempool. Saving my mempool. How are we going to save my mempool?

Sjors Provoost Yeah, so Bitcoin Core, when it's running, it receives new transactions. It holds onto those, usually in memory. And then at some point, you shut Bitcoin Core down.

Aaron van Wirdum This is before they're included in a block.

Sjors Provoost Yes, exactly. And then either if a block arrives to go out of the mempool because now they're in a block, or if you shut down Bitcoin Core but the transaction is still not in a block, it's going to remember it by saving it to disk. And then next time you start Bitcoin Core, it loads it from disk. Otherwise, you'd have to wait for somebody else to announce it again to you, which is not ideal.

Aaron van Wirdum Or it's in a block, right, by then.

Sjors Provoost Yeah.

Aaron van Wirdum I mean, there's no real... What's actually the point now that I ask this question? Like, why should I care as a regular user that my mempool is cleared?

Sjors Provoost Well, we did an episode about why the mempool is useful in general. You know, it helps with things like block relay. It makes it quicker for you to verify a new block because you already verified those transactions before. So then when you restart Bitcoin Core, it helps you to verify all those new blocks that you're going to catch up with with that mempool that you still had.

Aaron van Wirdum Right, okay. So yeah, I have a Bitcoin Core node and the mempool, when I shut down my computer, my entire mempool is just saved on disk and then I start the computer again and Bitcoin Core starts up and it remembers what was in the mempool.

Sjors Provoost Exactly.

Aaron van Wirdum Okay, so what's the problem?

Sjors Provoost The problem is that... There, you might have a virus scanner, and that virus scanner is looking for suspicious files. And if some joker creates a transaction and then puts one of this OP_RETURN spam inscriptions, whatever it is, in there, they could put something in that looks like a virus.

Aaron van Wirdum Whether it is a virus or not. Well,

Sjors Provoost that's a philosophical question. Is a photo of a virus a virus? I don't think so because it's not going to replicate or do any harm.

Aaron van Wirdum Okay.

Sjors Provoost Right? So it's more like a picture of a virus, I guess.

Aaron van Wirdum Okay.

Sjors Provoost It might be the exact same code. It's not that your node is suddenly going to run the virus and destroy itself. But the virus scanner doesn't understand that. So it sees something that looks like a virus and says, I'm going to put this in quarantine now, and then puts your whole mempool in quarantine. And now when you start the node again, it's not going to break. It's just going to say, oh, I guess you don't have a mempool yet. Let me make a new one.

Aaron van Wirdum Uh-huh.

Sjors Provoost That's not ideal because we just explained why we saved this thing in the first place.

Aaron van Wirdum Can't say I feel very stressed about this problem, but sure, go on.

Sjors Provoost No, this is why this problem has not been fixed for the past 10 years. We do the same thing for blocks that are stored on disk. Well, what we'll do, we'll explain in a bit, but we have the same problem for blocks. I think that actually did happen in like 10 years ago. Some joker put something in a block that looked like a virus.

Aaron van Wirdum It's better to solve the problem.

Sjors Provoost Yeah, because if your blocks are killed by the virus scanner, then your node doesn't work anymore.

Aaron van Wirdum Right.

Sjors Provoost So that's more important.

Aaron van Wirdum Yes. And so transactions, yeah, okay, so what's the solution here? What are we trying to do?

Sjors Provoost The solution here... is to basically fool the virus scanner by encrypting the file. Now, encrypting sounds very fancy. It's not the strongest encryption ever, but it is using XOR, basically.

Aaron van Wirdum And what's XOR?

Sjors Provoost XOR, well, I guess the easiest is to explain it. It's like a one-time pad. So this is one of the oldest forms of encryptions that exist. You take a text that you want to encrypt, like the word hello, and then you add to it a set of random letters. That's the easiest way to see it that you've agreed to before. So you and I want to communicate. We exchange a long piece of paper with lots of letters on it. And then we say whenever we write a letter to each other, we both start with that piece of paper and we add the letter on that piece of paper to every letter that we write. Or I add the letter and you subtract it. And that way...

Aaron van Wirdum Wait, so this sounds like the Caesar cipher. Do you know what this is?

Sjors Provoost No, it's much more secure.

Aaron van Wirdum Oh, it is more secure than that?

Sjors Provoost Yeah, I think the Caesar cipher was... Actually, I don't know. It might be similar.

Aaron van Wirdum The Caesar cipher, which I describe... in the Genesis book.

Sjors Provoost Oh, yes, you wrote a book.

Aaron van Wirdum It's basically you take the word hello and then you do like plus five and then for each letter you replace the letter in hello for like the fifth next letter in the alphabet.

Sjors Provoost Yeah, but if you do it just plus five for every letter, that's a bit easy to crack.

Aaron van Wirdum I mean, it is, but that's the Caesar cipher.

Sjors Provoost So this cipher means every... Every character gets, you add a different letter or a different number to every character. And that number is random every time. So you have a page with completely random numbers that you created with dice throwing, whatever it is. Numbers 0 to 26, right? Or 0 to 25. And when I'm writing you the message, I'm adding a different number to every letter of my message, and you have to keep track of where I am, and then you can subtract the same number. And then you burn the piece of paper that we used as a cipher. So that means anybody who intercepts the communication midway can no longer decipher it.

Aaron van Wirdum Anyways, so the mempool is encrypted.

Sjors Provoost Using roughly that technique, except a very insecure version of it, because the entire thing is only like eight characters long, and it just repeats the same eight characters, which is insecure. And also at the start of the file, the cipher is actually revealed, so it is totally insecure, because it's not the goal to be secure, it's the goal to fool the virus scanner.

Aaron van Wirdum Yep. Okay, that's it. Mempool upgrade.

Sjors Provoost Yeah, and so the little thing there is that means you can't downgrade. Because older versions of the node, if you wanted to go back to an old version, won't understand the mempool file. But they'll just start a new one. So it's not a big deal.

Aaron van Wirdum Okay. Next one. BIP 324. We made an episode on that. That's network encryption. And that's now on by default.

Sjors Provoost Yes.

Aaron van Wirdum So it was already an option in the previous Bitcoin Core release.

Sjors Provoost Yep.

Aaron van Wirdum Maybe even the one before that? No, probably the previous one, I think, right?

Sjors Provoost I think the previous one or the one before it, it was made default.

Aaron van Wirdum No, it was.

Sjors Provoost It was available. You could turn it on.

Aaron van Wirdum Yeah.

Sjors Provoost And now it's on by default.

Aaron van Wirdum Right. Okay. Quick reminder what it was again.

Sjors Provoost Episode 77.

Aaron van Wirdum No, sure.

Sjors Provoost Stein can just paste it. No, that's fine. The idea is that you and I have a connection, and instead of sending everything in plain text, we encrypt everything.

Aaron van Wirdum Yep. Okay, I mean, I guess, actually, yeah, that's good enough. If you do want to know more, we made an episode on it.

Sjors Provoost Which starts with the Diffie-Hellman key exchange, which you have described in your book.

Aaron van Wirdum Oh, thanks. You're plugging it as well.

Sjors Provoost And then you use a stream cipher, which is essentially a one-time pad where this piece of paper that we talked about, we know that given the key exchange we did, we know what that page has to look like. So we don't have to give each other a piece of paper with the cipher on it. We both construct the same cipher, but it's the same cryptographic idea. So that's pretty cool. It's on by default. This does not mean that all your connections are now encrypted by default. The devil is in the details here. When you connect to a new node, you will try this encryption only if you think the other side supports it. And we talked in a very, very early episode about DNS seeds, probably episode two.

Aaron van Wirdum Oh, wow.

Sjors Provoost Where, you know, the way nodes bootstrap is they get a list of other nodes and then they call those other nodes and say, hey, do you know about other nodes? And you keep a list of which nodes exist in the network. But you don't just keep track of which nodes exist and their IP addresses. You also check which features they support. And one of those features can be that they support this encrypted communication. So only if through the gossip circuit you heard that this node supports encryption, then you're going to try it. Otherwise, you're not. And so this will probably be a future change. If we know that this works, then we might just try an encrypted connection all the time and fall back to unencrypted if it doesn't work. So now this is, even though it's on by default, it's still a bit conservative.

Aaron van Wirdum Right. Okay, so so far we talked about something that was removed, something that's a very niche problem.

Sjors Provoost Not niche if you use Windows, but...

Aaron van Wirdum Well, no, I think you're... We're not there yet.

Sjors Provoost Okay.

Aaron van Wirdum I said so far we've talked about something that was removed, something that solves a weird virus thing.

Sjors Provoost Yeah, but that may not be a weird problem. If somebody does this, then it's going to affect everybody who runs a virus scanner, like not just a few people.

Aaron van Wirdum I'm not impressed, Sjors. Where are my soft forks? Where are my... Does this make number go up? I don't think so.

Sjors Provoost This is finally fixed Lightning.

Aaron van Wirdum Yes, please. Okay, moving on to the next one. Network adjusted time removed.

Sjors Provoost Speaking of unimportant problems. Yeah, so there is this rule that says if you see a new block, that it cannot be from the future. Well, it cannot be for more than two hours in the future. There's a little grace period there. Question is, what is two hours in the future? What if your clock is wrong? You might be wrong about that.

Aaron van Wirdum Wait, let me repeat that for those that don't know and didn't quite follow. Yeah, you find a new block or someone sends you a new block. The block has a time in it. And then one of the things you do to check if the block is valid is to check that it wasn't mined in the future or in the past, I guess, right?

Sjors Provoost No, the past is fine because you might be catching up on all blocks.

Aaron van Wirdum Okay, yeah. It's not mined in the future. And the future means two hours or more into the future.

Sjors Provoost Yeah. Now, there are limitations to the past, by the way, but that is relative to earlier blocks, right?

Aaron van Wirdum Right.

Sjors Provoost Blocks can't go back in time.

Aaron van Wirdum Right.

Sjors Provoost But, yeah, you cannot accept the block.

Aaron van Wirdum Right, now we're talking about blocks in the future.

Sjors Provoost Yeah.

Aaron van Wirdum They can't be mined in the future, which is two hours in the future, and now what you said is how do you define what is the future? Well, usually you use a clock, and I guess that's what you're getting at.

Sjors Provoost Yeah, and so I think it was Satoshi or maybe one of the earlier contributors who thought, well, what if your clock is wrong? You might be rejecting blocks even though they're valid because your clock is wrong, not the miner's clock. And the solution to that was to when you connect to other peers, you ask them, hey, what time is it? What time is it? What time is it? You know, not that long ago, that's a question you would ask people on the street.

Aaron van Wirdum Yeah.

Sjors Provoost It's amazing.

Aaron van Wirdum Do you have the time?

Sjors Provoost Yeah, exactly. Nowadays, well, we stopped doing that as humans, and Bitcoin Core stops doing that now too. Well, not entirely, but it doesn't do anything with it anymore. That's the change. Because the problem is, what if your peers are lying to you about the clock? That's not ideal. And also, you know, I think most computers nowadays get their clocks set by a network, you know, get the time from the network, from their internet provider or from some other source. And so my thinking is that they're generally either very accurate within the second or they are completely wrong because...

Aaron van Wirdum Yeah, it's 1900.

Sjors Provoost Yeah, I've had one of these cheap computers where the little battery is dead. So when you turn it on, the clock is back at the second zero, which is January 1st, 1970.

Aaron van Wirdum Is that it?

Sjors Provoost That's typically the year zero for a computer. Right.

Aaron van Wirdum Okay.

Sjors Provoost So yeah, so now I think it will still warn you. Like, hey, this block seems to be [unclear]… Yeah. No, I think it'll warn you saying, hey, your peers think that time is different than I think it is, but that's it. It's just going to work. It's going to use its own clock.

Aaron van Wirdum Do you think that in the future when religion is just displaced, we'll actually use 1970 as the year zero? You think future humans will think that way? Sure.

Sjors Provoost Do you think that when we invent something better than a QWERTY keyboard, we'll actually use it?

Aaron van Wirdum I'm still holding my breath. I'm not going to hold my breath for that one.

Sjors Provoost I don't think we're going to change this.

Aaron van Wirdum Okay. Because the QWERTY keyboard is not going to change?

Sjors Provoost If we don't change the QWERTY keyboard with obvious productivity benefits, why would we replace the calendar system, which doesn't matter?

Aaron van Wirdum Never?

Sjors Provoost I'm sure at some point people will be like, oh, it's the year 370,225,695. Yeah, that gets a little tedious. But maybe they'll just not count the thousands or something like that.

Aaron van Wirdum Or we change it to 1971. That's what Bitcoiners will agree with. Should we move on to the next point?

Sjors Provoost A rounding error in 300 million years.

Aaron van Wirdum Next point.

Sjors Provoost Another amazing problem.

Aaron van Wirdum External signing for Windows disabled. Oh, yeah. This is about hardware wallets.

Sjors Provoost Yes. So there is a feature in Bitcoin Core where you can connect a hardware wallet to your USB port, and then you have to install HWI.

Aaron van Wirdum Some people think that Bitcoin itself will be used as a clock, Sjors. What do you think of that?

Sjors Provoost That annoys me.

Aaron van Wirdum That's philosophical.

Sjors Provoost Because Bitcoin is not a clock. Bitcoin nodes use a clock. A trusted clock, even.

Aaron van Wirdum You measure the time in blocks nowadays.

Sjors Provoost No, you do not.

Aaron van Wirdum Why not?

Sjors Provoost Because your node checks the blocks to see that their times are valid.

Aaron van Wirdum Yeah, I guess that's what we were just talking about.

Sjors Provoost Yeah, exactly.

Aaron van Wirdum Okay, moving on to hardware wallets. I won't interrupt you anymore.

Sjors Provoost Yeah, so there is a tool called HWI. We have talked about that in the past. It's a separate thing you download. You tell Bitcoin Core where it is, and then you can use hardware wallets right from the GUI. As simple as, like, create a new wallet. It says, oh, would you like to use your COLDCARD if you connected it?

Aaron van Wirdum Good choice.

Sjors Provoost And it'll import the keys. You can send with it. It'll just work. It's pretty cool. However, Windows support has always been a bit iffy. It's been disabled a few times. It's been enabled a few times.

Aaron van Wirdum What's iffy about it? Why is it iffy?

Sjors Provoost Windows is just painful.

Aaron van Wirdum Hard to program for? Like, what's the problem?

Sjors Provoost Yeah, I think it would just... Either it was broken in Windows or the support for Windows broke something else that we needed.

Aaron van Wirdum So this is a Windows-only problem.

Sjors Provoost Yes, and we've removed support for it, I think, a year ago. It wasn't mentioned in the release notes. Nobody complained about it. Then it was added back, which I think was also not mentioned in the release notes. Nobody was excited about it. Now it's removed again, which is mentioned in the release notes, and it should be back again in the next version. I think nobody's going to care about it.

Aaron van Wirdum If you have a hardware wallet and you're using Windows and you have a Bitcoin Core node where you use the wallet with the hardware wallet,

Sjors Provoost then maybe you shouldn't upgrade.

Aaron van Wirdum And you're using HWI as a tool.

Sjors Provoost Because there's other tools, right? There's Sparrow Wallet and there's Spectre. Those will work around the problem. So it's only those very specific circumstances that it would cause a problem for you.

Aaron van Wirdum Okay. And you think that's basically no one is using it that way?

Sjors Provoost Well, based on the fact that nobody complained the last time it was removed.

Aaron van Wirdum Yeah, but most people aren't gonna, they don't even know where to find you.

Sjors Provoost But they know where to complain.

Aaron van Wirdum Do they? What's the service number for Bitcoin Core?

Sjors Provoost X.com?

Aaron van Wirdum That's true.

Sjors Provoost This would have gone viral if somebody was bothered by this. Maybe so, yeah. Clearly, clearly. Anyway, yeah. So yeah, if you do need this, yeah, then I guess don't upgrade to 27.0. Wait for 28.0. It'll be fixed again, probably.

Aaron van Wirdum Okay. The last part, we have one more, and that one is actually kind of interesting.

Sjors Provoost Yes.

Aaron van Wirdum I think. So this is about UTXO selection in the wallet. And there's a new algorithm, as I understand it, called CoinGrinder. Is it a new algorithm?

Sjors Provoost Yes.

Aaron van Wirdum Right. So this is about you want to send coins from your wallet and then your wallet has to decide which coins to use. And there's like a way to determine which UTXOs, which coins it's going to use.

Sjors Provoost Yeah.

Aaron van Wirdum So what's new about CoinGrinder?

Sjors Provoost Yeah. So a little bit of a background on coin selection. The initial coin selection it was built in was using something called the Knapsack algorithm, which I guess is what it sounds like. You have a bag and you want to put a bunch of stones in it from different sizes. And the question is, how do you optimally... What's the maximum number of stones you can fit into the bag or a problem like that? If you only put the big stones in, then you have lots of room left. If you only put the small stones in, then it's not efficient either.

Aaron van Wirdum The analogy here would be a lot of room left is a lot of change, right? But that's not really a problem.

Sjors Provoost Yeah, I don't know. Well, the problem is, in this case, if you use lots of very small coins, then you're paying lots of fees because every coin you put into a transaction increases the fees. And that general problem is the knapsack problem, if my computer science understanding is correctly. It's correct, and that's unsolvable. There's no perfect solution. You can just approximately solve it.

Aaron van Wirdum It's one of these hard problems, right?

Sjors Provoost Yes. So there were other algorithms. Branch and bound was one. I don't actually know exactly how it works, but it boils down to it. It tries to find a reasonable set of coins to put into the bag to spend with, but not necessarily the cheapest one.

Aaron van Wirdum Branch and bound try to optimize for not having any change, I think, right?

Sjors Provoost I think, yeah, that's as little waste as possible. So I think we do want to have change. Well, I think either it tries to find a coin that just spends it directly. That's nice if you don't need change at all. But otherwise, you actually, if you do have change, then you don't want to have the absolute minimum amount of change.

Aaron van Wirdum You don't? I mean, it's just one.

Sjors Provoost No, because of privacy.

Aaron van Wirdum Change is also one UTXO, right?

Sjors Provoost Yeah, but for privacy reasons, you don't want to reveal which side is change. And so it deliberately spends a bit more and creates higher change.

Aaron van Wirdum Oh, I remember that. Yeah, we mentioned that in the earlier episode.

Sjors Provoost I don't know if that is actually dependent on the algorithm used, whether you use Knapsack or Branch and Bound or this other one. It might just be something that we do anyway. And then there's a different algorithm to find coins based on that choice. I don't actually know that much about coin selection. Should ask Murch. He wrote a whole thesis on it. And he also designed this algorithm.

Aaron van Wirdum It's kind of a science of its own.

Sjors Provoost Yeah, there's all sorts of trade-offs, right? For privacy reasons, you want to maybe have a bit more change than you really need. But also, there's a time preference element here where the cheapest transaction right now, so using the fewest coins as possible, that's great now, but in the long run, it's going to create lots of very small coins in your wallet, and then eventually you have to pay. So normally the wallet does not try to find the very cheapest transaction. It tries to also act in your own long-term interest. However, and this is where the new algorithm CoinGrinder comes in, if fees are very high, which in this case is defined as more than 30 sats per byte, but you can change that, then it's going to find the cheapest way to spend it. So it's going to ignore the future. It's going to say, okay, right now fees are high. Hopefully they'll go down again in the future. I'm just going to do whatever's cheap because otherwise...

Aaron van Wirdum I can make this payment with using only one UTXO, so that's what I'm going to do because that's the cheapest way.

Sjors Provoost Yeah, because there were situations where fees were 600 sats per byte and the Bitcoin Core wallet was like, oh, let me spend 25 coins now.

Aaron van Wirdum Right.

Sjors Provoost And then you were paying, I don't know, hundreds of dollars in fees.

Aaron van Wirdum Right. Okay, so it does that if... fees are 30 vbytes per...

Sjors Provoost 30 sats per vbyte or more.

Aaron van Wirdum [unclear]. But you say you can adjust that manually?

Sjors Provoost Yeah, I believe there's a setting. It's specified in the [unclear].

Aaron van Wirdum It's not just an algorithmically changing thing, but for now...

Sjors Provoost No, I guess the problem is what is high depends on the exchange rate and also depends on the long-term fee rates. Because if fees will never go below 30, then you should really... not use this algorithm even at 30. It's really designed for spikes. But what is a spike?

Aaron van Wirdum Well, I mean, there's probably some way... Murch, if you're listening, we need an algorithm to algorithmically adjust your algorithm.

Sjors Provoost Yeah, well, one of the problems is that if you're not running your node all the time, you don't necessarily know what the fee rates are historically. Because if you only look at what's in blocks, you could be fooled by the miners into thinking fees are higher than they are. So it's a very tricky problem.

Aaron van Wirdum Yeah. Anyway, so that's in Bitcoin Core 27 now.

Sjors Provoost That's right.

Aaron van Wirdum So if fees go up, then this will sort of click on in the same way that everything in your home Sjors clicks on automatically.

Sjors Provoost Exactly.

Aaron van Wirdum For those that don't know that, Sjors doesn't like touching things, so he has everything automated.

Sjors Provoost And spend more time maintaining it.

Aaron van Wirdum It's like Doc from Back to the Future where feeding the dog is automated.

Sjors Provoost Exactly.

Aaron van Wirdum I think that's it for our episode, right?

Sjors Provoost Yeah, I think we should mention there is a testing guide. Hopefully we'll remember to put that in the show notes, which takes you by the hand and explains if you wanted to test this new version, ideally with fake coins, testnet coins, signet coins. It'll tell you how to do that. And that's nice because we prefer it when people find bugs before the release is final. So help is welcome.

Aaron van Wirdum Okay, one other quick question. I also saw, so there's also new releases, like 26.1. I saw, what's the point of that actually right now?

Sjors Provoost Yeah, so let's say, you know, you're really passionate about libbitcoinconsensus.

Aaron van Wirdum Uh-huh.

Sjors Provoost And, well, in this case, that doesn't matter because it's still in there, but let's say it's removed.

Aaron van Wirdum Well, or you're very passionate about using hardware wallets on Bitcoin Core with the thing on Windows.

Sjors Provoost Yes. Exactly. If that's your passion, you can use an older, instead of only using the old release, which probably has bugs in it, there is a 0.1, 0.2, etc. that only fixes bugs in that earlier release, but doesn't add any of the new features.

Aaron van Wirdum Okay, there's a new old release.

Sjors Provoost Yeah, so this is quite typical in software development. You have a major version that's released, and then there'll be backports for older versions. So something like Python, you might have version Python 3.7.9, but also 3.6.12 and 3.5.11, whatever it is. So this is indeed to not force you into using the latest and greatest version. but also not saying, oh, well, if you use the old version, you can have all the bugs in it. It's a bit subjective what goes into these backport releases. Of course, because usually the more relevant bugs.

Aaron van Wirdum Anyways, [unclear] get Bitcoin Core 27. Get up to date, get the newest, the shiny release soon to be found on bitcoincore.org

Sjors Provoost or wherever you find Bitcoin nodes. Thank you for listening to Bitcoin,

Aaron van Wirdum Explained.