Episode 97: Bitcoin Core 29

Aaron van Wirdum Live from Utrecht, this is Bitcoin

Sjors Provoost Explained.

Aaron van Wirdum Hey Sjors.

Sjors Provoost Hello.

Aaron van Wirdum Welcome back.

Sjors Provoost Welcome back. It's just been a few days.

Aaron van Wirdum Yeah, well, this episode is delayed by like two weeks. We were supposed to record this at MIT in Massachusetts at the Bitcoin Expo, which you did attend.

Sjors Provoost I did attend it and I did join some panels there. But we couldn't do the podcast.

Aaron van Wirdum Yeah, we were supposed to do the podcast on stage. Big spectacle.

Sjors Provoost But you got deported to El Salvador, and it was complicated to get you out there.

Aaron van Wirdum It was a long story, Sjors. I'm surprised you didn't get deported to El Salvador.

Sjors Provoost I'm happy.

Aaron van Wirdum You look like MS-13. I mean, that's what people usually say when I say, yeah, I record a podcast with Sjors, and they say, oh, that's the MS-13 guy? And I have to explain, no, no, he just looks like that. He's actually a Bitcoin Core developer.

Sjors Provoost All right.

Aaron van Wirdum Well, somehow you made it back.

Sjors Provoost I made it back.

Aaron van Wirdum So we're going to record an episode on Bitcoin Core 29.

Sjors Provoost That's right.

Aaron van Wirdum Which means it's been about six months since the previous Bitcoin Core release.

Sjors Provoost That's right.

Aaron van Wirdum And as usual, that means there's some new features which we're going to discuss.

Sjors Provoost Yep, that's right.

Aaron van Wirdum Let's just get into it. What's up? Yeah, we have a sponsor.

Sjors Provoost That's right.

Aaron van Wirdum We are being sponsored by our very generous contributors to the show, Coinkite, who produce the COLDCARD hardware wallet. Right now I'm holding the...

Sjors Provoost I think it's called the Q.

Aaron van Wirdum COLDCARD Q, which looks like a big calculator.

Sjors Provoost Or BlackBerry.

Aaron van Wirdum You can store your Bitcoin on it. Have you actually used it or is it just laying around here?

Sjors Provoost This one.

Aaron van Wirdum Yeah.

Sjors Provoost I'm not going to answer that question.

Aaron van Wirdum Smart. I'll take that as a yes.

Sjors Provoost I will check that it's still here.

Aaron van Wirdum All right. So there's a new Bitcoin Core release. And then what we're going to do is we're going to highlight some features that are in this Bitcoin Core release. There's a bunch of features, including bug fixes and all kinds of smaller stuff. But we highlighted some features that are at least kind of worth discussing for a couple of minutes.

Sjors Provoost I think they are. I mean, I had to convince you of some of them. But, yeah.

Aaron van Wirdum I mean, you know, none of them are major of course, but

Sjors Provoost I think some of them are major.

Aaron van Wirdum Okay, let's get into it. So the first one is called UPnP has dropped.

Sjors Provoost That's right. And by has dropped, I mean has been dropped.

Aaron van Wirdum I mean, it's, wait, oh yeah, it's not included anymore.

Sjors Provoost Yeah, it's removed. That's what that means.

Aaron van Wirdum Yeah, I was thinking dropping, like, you know, dropping a new album. Yeah,

Sjors Provoost exactly.

Aaron van Wirdum The Bitcoin Core developers have dropped UPnP. Okay, it's no longer in Bitcoin Core, actually.

Sjors Provoost Exactly.

Aaron van Wirdum That's what it means.

Sjors Provoost Yeah.

Aaron van Wirdum So what is UPnP?

Sjors Provoost Yeah, so UPnP is and was a way for you to make your Bitcoin node at home or wherever it is, reachable from the outside. Because typically, so when you're on a server, your node will find other peers to connect to, but it will also listen for other peers that want to connect to it. And if you're sitting on Amazon somewhere in a server park, that tends to just work. But if you're sitting at home, most routers or your home create a firewall and anybody that tries to connect to you will not succeed. You can connect to anybody you want. but nobody can connect to you as sort of a standard safety measure because they don't want, you know, random hackers to connect to random insecure Windows computers that you might be running.

Aaron van Wirdum Yeah, okay, so... Because I always kind of wondered about this. Well, I always sort of knew the answer, but this confirms it. Like in my case, I do run Bitcoin Core, but I just downloaded the software, run it on my computer. But in my case, that means that I'm not actually sharing blocks or transactions with other nodes because...

Sjors Provoost You are. However, you're not receiving connections. So if you make a connection to another node... you will send and receive blocks and transactions. So it's not about, you're not a leecher, basically, as they used to call that in Torrent jargon. You are sharing just as much. It doesn't really matter whether you are, whether who initiates the connection. It's just that. If nobody in the world is listening and everybody's trying to connect, then they have nowhere to connect to. So somebody needs to listen.

Aaron van Wirdum Right, that's it. In your case, I don't know if you run a Node with or without the firewall.

Sjors Provoost Yeah, so I run multiple nodes that you can reach.

Aaron van Wirdum Right, so then my node, because I never changed any settings, right? I just, well, I changed some settings, but not this setting. So I basically run it out of the box. So then I am connecting to you. So then I'm sharing blocks and transactions with you that I might also get from like another source that is doing the same thing. But I will not connect to another Aaron. because the other Aaron is also running it out of the box. So I have no way of connecting to him and he has no way of connecting to me. So then, yeah, I mean, isn't that a problem? Like, isn't it a problem that if there's just too few, if there's too many Aarons and too few Sjorses, that it just becomes a form of centralization because everyone is running, you know, it becomes basically like a hub and spoke network, I guess would be the technical term.

Sjors Provoost Yes, now we're nowhere near that point. because there are many, many thousands of nodes that are listening. So there's no shortage. However, we talked about in an earlier episode about ASMAP, which is like a map of the internet of which providers. And one of the things that Bitcoin Core wants to do in the future is to have more diverse connections. And I don't mean that in a DEI sense. I mean that in a sense of you should not have all your connections to Amazon. You should have one connection to Amazon, maybe one to Google, one to some ISP in Kenya, one to some ISP in Germany. So that if somebody is trying to mess with you, it's very difficult for them because they have to somehow pretend to be a lot of different internet providers in the world. And if we do that, and that means that the number of nodes that are listening... The nodes that are listening now that are sitting on Amazon and Google, exactly the kind of places where it's easy to set that up, they will get fewer connections. And all the nodes in obscure apartments and... random countries will get a lot more connections. And it could be, depending on the simulation, that that's too much, that you would start overloading the more obscure places.

Aaron van Wirdum Yeah, because there's only one guy in Kenya running a node.

Sjors Provoost Exactly.

Aaron van Wirdum So that everyone wants to connect to the one guy in Kenya node.

Sjors Provoost Yes, because it's not Amazon.

Aaron van Wirdum Yeah.

Sjors Provoost So in order to prevent that, it's probably necessary to increase the number of listening nodes at least somewhat. Now, so there are two kinds of nodes that listen by default. One is... Oh, sorry. No, let me rephrase that. There is another node that listens by default, which is a Tor node. Because if you set up Tor, you set it up usually to also use a hidden service. And it turns out that projects like Umbrel will do that automatically for you. And that's easier. But those are only on Tor, so it's not that useful if you're not on Tor.

Aaron van Wirdum Yeah, they would only connect to other Tor users.

Sjors Provoost And if you're running in a data center, then typically everything is just wide open, so everybody can connect to you. But there's also a tool that used to be included in very early versions of Bitcoin Core called UPnP. And what this would do is it would look for your router. So from the inside, it would look at the router and say, hey, router, please open this port for me, open this door, so that everybody who wants to talk to me can talk to me.

Aaron van Wirdum And that worked. UPnP is the setting you use if you want to be a listening node.

Sjors Provoost It was, yes. And so this was introduced quite a while ago, and that's why there are, from the beginning, there were quite a few listening nodes. I think it was on by default. So everybody who downloaded Bitcoin Core would... If possible, punch a hole in their router and start listening.

Aaron van Wirdum Right, okay, yeah.

Sjors Provoost Or in their firewall, I guess. Problem is that UPnP as a software package was extremely insecure. There were multiple bugs found in it. That would, say, allow somebody to take over your computer and steal your coins.

Aaron van Wirdum That sounds bad.

Sjors Provoost Yes, exactly. And so this was not removed, but it was turned off by default. So as a result, for many, many years, if you installed a new version of Bitcoin Core, it would simply not do any of this unless you turned it on yourself at your own risk.

Aaron van Wirdum Do you know when this change was made?

Sjors Provoost Many, many years ago. Probably 2012.

Aaron van Wirdum Okay, so if I had downloaded...

Sjors Provoost If you had downloaded Bitcoin in 2011 or something like that, you would be listening.

Aaron van Wirdum Right.

Sjors Provoost If you were still running a node from 2011, you would be listening and you would be hacked. But yeah, you could. So, you know, it's been...

Aaron van Wirdum In any case, this setting was called, I mean, it's still called UPnP

Sjors Provoost Yes.

Aaron van Wirdum And this was removed.

Sjors Provoost Yes. And in recent years, we've added a whole new system to it. I'll skip some intermediate steps. So basically, we added something called Port Control Protocol, PCP, which is a much more modern way of doing the same thing, of asking your router to please open it. But it is much simpler in terms of code. Therefore, we're a little less worried about any security vulnerabilities there. But still not turned on by default. So nothing really changes. You still, if you download Bitcoin Core and you have the GUI, there's a settings dialog and you can click on basically PCP now. Where it used to say UPnP, it now says PCP.

Aaron van Wirdum Okay, maybe you just said it, but is PCP new in this Bitcoin Core release?

Sjors Provoost Yes.

Aaron van Wirdum Okay, and that's why now UPnP has been removed.

Sjors Provoost Yes, exactly. It's essentially a replacement. And then perhaps we'll turn it on by default in the future. I mean, that's something that has to be considered somewhat carefully because it would impact a lot of different nodes.

Aaron van Wirdum Okay, but the functionality has not changed so far. It's just fewer bugs, presumably.

Sjors Provoost Now, the bugs that were originally found were fixed, but the problem is, of course, if you found multiple really bad bugs in a project, do you still want to use it? Maybe not, because maybe you didn't find all of them.

Aaron van Wirdum And did I understand correctly that... PCP, either PCP itself or the integration of PCP. Okay, first question. Is PCP a Bitcoin-specific project?

Sjors Provoost No.

Aaron van Wirdum Okay.

Sjors Provoost No, it's a general project, simply just like UPnP, which was not a Bitcoin -specific project. It was used by torrent clients, all that sort of stuff, to be able to open your ports automatically. Games tend to use this stuff like this.

Aaron van Wirdum Right. And then, did I understand that? Then the integration of PCP into Bitcoin Core was done by Wladimir van der Laan?

Sjors Provoost That's right, yes. And he actually re-implemented it. So he rewrote the code specifically in C++. So that's kind of nice.

Aaron van Wirdum Right. Yeah, that's cool. Nice to hear that Wladimir is back at it.

Sjors Provoost Yep.

Aaron van Wirdum In case there's listeners that don't know, Wladimir was the lead developer of Bitcoin Core for eight years?

Sjors Provoost Very long time.

Aaron van Wirdum Yeah, a whole while. Took a step back. So very nice to hear that he's back at it.

Sjors Provoost Well, not back at the annoying work of being a maintainer, but simply doing the...

Aaron van Wirdum Contributing.

Sjors Provoost Doing the interesting stuff. Yeah, exactly.

Aaron van Wirdum Okay, the next one has definitely the coolest name out of the list.

Sjors Provoost Ephemeral Dust.

Aaron van Wirdum There we go. Although, yeah, no, that's got to be the coolest one. Okay, Ephemeral Dust. What does it do? What is it? What are we talking about?

Sjors Provoost It's a new Finnish metal band started by Martti Malmi. No.

Aaron van Wirdum Embedded in Bitcoin Core?

Sjors Provoost Exactly.

Aaron van Wirdum Plays by default?

Sjors Provoost I guess embedded in Bitcoin Core would also be a nice metal band name but in any case basically the idea we've talked about child pays for parent kind of situations in the mempool back in our episode about version 28 which I think is just two episodes ago so for you it would have been 20 minutes ago if you're listening to the whole series in one go

Aaron van Wirdum Child pays for parent means you are receiving a transaction, but the fee is so low that it's just sort of hanging around in the mempool. kind of being unconfirmed in your wallet. And then the way you can solve that is to actually spend these coins to yourself with a higher fee in the new transaction. And then miners will sort of make the overall calculation that if they include both transactions, both the original transaction to you, as well as the new transaction you just made to yourself with the same coins, then they're both getting confirmed and your coins are finally... In your wallet.

Sjors Provoost Exactly. So you have to bring some of your own coins in order to pay the fees for the transaction that you want to receive. Now, in your example, it was still in the mempool. But imagine the fees are so low that they're not even in the mempool. Then you have to relay them together because nobody's mempool will accept it if the fee of the original is too low. And this is called package relay, which we covered in that episode. So you relay two packages, two transactions as a package. And then every node along the way can reason about it and say, hey, okay, this is fine. This should be interesting enough for a miner to include.

Aaron van Wirdum Yeah, so basically nodes would start to use... How far are we with this? I know we covered it in previous episodes.

Sjors Provoost Yeah, we covered most of this, but this might be useful to go to the basics. That's what a package is. And the only form of packages that Bitcoin Core supports is one parent, one child. And that has simply to do with the fact that it's complicated to reason about anything bigger than that.

Aaron van Wirdum Well, that was included in the previous Bitcoin Core release.

Sjors Provoost Yes, in 28.

Aaron van Wirdum Yeah, so, I mean, the idea then is basically that nodes will use a similar -ish... strategy as miners. Like miners already make this calculation and now nodes have been starting to make a similar calculation to decide if transactions show up in the mempool. Yes.

Sjors Provoost And so then the part where dust comes in, part of the ephemeral part, is that, well, there's other rules that don't allow you to relay stuff in the mempool. And one of them is if you're creating an output that is too small. It used to be like 543 bytes. sorry, sats, if you had an output that was less than 543 sats, nodes would say, this is dust, I'm not going to relay it. It's valid in the blockchain, but I'm not going to relay it. That was just basically a way to prevent somebody from creating lots of very small outputs. And the UTXO set that are just wasting everybody's memory, or whatever they're wasting. And so... If you have a one-child-one-parent construction, but the child is... Basically, the parent has a bunch of dust as the output, and the child is consuming the dust, and then paying their own fees. So they might be paying enough fees, but because there's dust involved, the nodes would still reject it, because they don't like dust. And the ultimate example of dust is a zero output.

Aaron van Wirdum Yeah, which is apparently possible.

Sjors Provoost Yes, yeah, definitely. There was, I think, a year ago, back when everybody was, you know, remember this ordinals inscriptions hype?

Aaron van Wirdum Mm-hmm.

Sjors Provoost So the ordinals inscriptions, they come up with this set of rules that are not consensus rules, so just their own rules, and it would count, like, you know, you are sat number one, sat number two, yadda yadda yadda. And it turns out, I think they didn't take account of the possibility to have a transaction that has zero sats as an input and zero sats as an output. Somebody made one as a joke, like just sent it directly to a miner because it's consensus valid, even though it's not standard. And so all their tools were off by one. Because the numbering system was broken, basically.

Aaron van Wirdum Okay, so you can, because I didn't, I don't know, I don't think I knew this. You can apparently, in Bitcoin, spend zero Bitcoin.

Sjors Provoost Yes.

Aaron van Wirdum To an address that then receives zero Bitcoin. That's like a valid transaction.

Sjors Provoost Yeah, and you're paying zero fees.

Aaron van Wirdum And zero fees, yeah. So as long as a miner includes that in a block, then everyone's going to say, yeah, makes sense. Zero, zero to zero.

Sjors Provoost Yes, and of course miners have no financial. good financial reason to do that unless they're being paid out of band so we don't want to encourage this but yeah you can fill a whole block with just zero fee transactions as far as I know.

Aaron van Wirdum Okay, what's…

Sjors Provoost Zero amount.

Aaron van Wirdum And I know we were getting there but what is the what's the benefit of making because there is apparently an actual benefit to doing this sometimes

Sjors Provoost so I think the easiest example would be a Lightning transaction or so we you and I have a Lightning channel And there's two ways we can close it. One is we are online together and we decide to close it cooperatively. We look at the fee market and we just make a transaction that closes the channel, pays enough fees, done.

Aaron van Wirdum Yeah.

Sjors Provoost But the other scenario is one of us is not offline or one of us is trying to cheat.

Aaron van Wirdum But one of us is offline.

Sjors Provoost Yeah, sorry, is offline, is not online. One of us is trying to cheat some scenario like that where we cannot cooperate. That means we have to broadcast the transaction that we signed in the past. So we previously got together when we were still buddies. We signed this transaction and we say, this closes the channel for both of us. However, we don't know when or if we're going to use this transaction. So we have no idea how much fees to put in this thing. So what Lightning nodes in the beginning had to do is just pick the worst case possible fee and that was kind of annoying because you're reserving a lot of money for a case that might never happen. So what you can do instead is you add zero fees to this transaction and you say this has to be paid by a child. So it's like bring your own fees. That way we don't have to decide in advance what the fees are going to be. And if this scenario occurs... you or I want to close the channel, we add one of our own inputs to it, and that pays for the fees.

Aaron van Wirdum Yeah, right. So basically, you and I establish a Lightning channel, and the closing transaction, in case one of us is unresponsive, in case Sjors ends up in the El Salvadoran prison, then I can still broadcast the Lightning transaction with the zero fee. And the way I get that included in an actual block is to take some of my own coins, spend these to myself, child pays for parents style. But in that transaction also includes that zero, zero to zero with zero fee transactions. So now that's part of one transaction.

Sjors Provoost And it's not just zero fees, it's also zero amount.

Aaron van Wirdum Yeah, exactly. Zero to zero.

Sjors Provoost Yeah, exactly. So the nice thing is you have a transaction that is creating an output that is zero, which is dust. and you have a transaction that spends the dust. And so this transaction that creates dust normally would not get broadcast because it is dust. But this thing called ephemeral dust is saying, well, as long as you're creating dust and immediately removing the dust, we're going to allow that. And so that's being relayed, and it's hopefully being mined as a pair. So in the end, there's no dust.

Aaron van Wirdum Okay, so what's new here is, I think we're talking here about the relay policy of Bitcoin Core now takes this into account. My Bitcoin Core node or your Bitcoin Core node or anyone else's Bitcoin Core node, as long as they upgrade it to Bitcoin Core 29, of course, they will actually look at these transactions in tandem and they will say, okay, that's a dust transaction, but because it's immediately being spent as well, it's fine. This is not going to create an extra UTXO in my database or anyone else's database. In other words, it's ephemeral.

Sjors Provoost Exactly.

Aaron van Wirdum Now we know why it has such a cool name.

Sjors Provoost Yeah, and my understanding is that there are some fancy future protocols people have in mind, like LN-Symmetry, etc., where you might have a series of transactions for whatever reason, like three in a row. And it's nice if all three of them are the exact same amount versus, you know, transaction one is a little bit lower than transaction two is a little bit lower than transaction three because they're all paying for their own fees. Makes it easier to reason about if the amounts stay identical. And so for those kind of constructs, ephemeral dust is also useful to have, but that's for the future.

Aaron van Wirdum Okay. And just to wrap this up.

Sjors Provoost And apparently Lightning devs are super happy with this ephemeral dust stuff.

Aaron van Wirdum Well, that's how I wanted to wrap it up, basically. The practical benefit of this is that so far, if you wanted to make a Lightning channel, then you kind of had to be on the safe side and include a high fee for the fallback transaction because you don't know how high fees are going to be in the future. So you're likely going to overpay on fees if you use this fallback mechanism. And now with this, you can just... make the exact right fee for depending on how much in the rush you are.

Sjors Provoost And it means you can also use more.

Aaron van Wirdum In other words, you can use lower fees for closing Lightning channels.

Sjors Provoost Yeah, and it means in general you can use more of your channel because if you're, you know, reserving 10% of your channel for a worst case fee, you can't actually spend that.

Aaron van Wirdum Right, yeah. I see, yeah. So it improves the Lightning network in... That way. Lower fees and therefore more actual liquidity on Lightning itself.

Sjors Provoost The little caveat there is that you do have to bring your own UTXO. So if you have a Lightning wallet that only has a channel, now things get a little bit more complicated because how are you actually going to do the child pay for parents if you don't have any children? But, you know, the rabbit hole does not end here, but it is a lot better this way.

Aaron van Wirdum You need to have at least one other UTXO or another channel in the same wallet.

Sjors Provoost Yeah, maybe.

Aaron van Wirdum I guess that could work, depending on how it's designed. Okay, that was Ephemeral Dust. Then we go to a slightly less cool name. Still pretty cool, but less cool. Block Reserved Weight.

Sjors Provoost Yes.

Aaron van Wirdum What is the update here? Block Reserved Weight.

Sjors Provoost Yeah, so basically a little background here is Bitcoin Core is able to make block templates that miners can use to create a block, basically. And Bitcoin Core tries to stuff as many transactions as it can into a block to give you the maximum fees. However, as a miner, you need to bring your own Coinbase transaction. That's just how the protocol works.

Aaron van Wirdum Yeah, that's the transaction that pays the coins, the new coins and the fees to the miners.

Sjors Provoost It does that. It might also have some commitments to various, like, what's it called? Like co-mining.

Aaron van Wirdum Do you mean like the OCEAN thing?

Sjors Provoost Not OCEAN thing, but usually there's all these other protocols that you're mining on, so merge mining.

Aaron van Wirdum Sure, yeah.

Sjors Provoost So there may be other stuff in the Coinbase that Bitcoin Core just doesn't do for you. So what that means is that Bitcoin Core does not know how big the Coinbase transaction is going to be. Therefore, it doesn't precisely know how much transactions it can actually put in your block. Because the Coinbase does count towards the limit, as does the header. You don't want to forget that. So it used to be that Bitcoin Core, I think, tried to reserve 4,000 bytes for this. Basically just like here, okay, I'll just not use 4,000 bytes and then you should be okay. Accidentally, that was increased to 8,000 bytes. So it wasn't a complete accident because the commit that did that, the pull request that did that had a comment saying, oh, I think we're doing it twice now. And then the reaction was, yeah, it's fine.

Aaron van Wirdum That's the attitude I like to hear from the Bitcoin Core developers that are building the future of money.

Sjors Provoost Well, it's better to be safe than sorry. But basically, you know, some miners really want to squeeze out every last bit because the downside is, you know, if we reserve 8,000 bytes by default, but you only need, I don't know, 500 bytes for your Coinbase. kind of, you know, you're throwing away some potential fees there.

Aaron van Wirdum Yeah, you're leaving, you have unnecessary headroom.

Sjors Provoost Yeah, you're leaving sats on the table and then what happens is miners think, hey, I'm leaving sats on the table. Let me edit the Bitcoin Core source code and change this stuff so it uses the exact, you know, limit. And then they make a mistake because Bitcoin Core stuff is hard to reason about and they produce a block with 4,000,001 weight units and now they've lost hundreds of thousands of dollars. while trying to save themselves two cents. So we want to discourage that. And one way to do that is by giving people what they want, mostly, so that they don't have to start doing this editing. So there's a new setting now that's called block reserved weight. It defaults to this 8,000 number that Bitcoin Core is already using, but you can set it lower if you know what you're doing.

Aaron van Wirdum Okay. So I think that was clear, but I'll repeat it anyways. So yeah, right now... In blocks, Bitcoin Core reserves 8,000 bytes for the Coinbase transaction.

Sjors Provoost 8,000 weight units, but yeah.

Aaron van Wirdum Sure. That's a standard number, and now that's become configurable.

Sjors Provoost That's right. Not less than 2,000. We don't think you should go below 2,000. If you do, then edit the source code, but don't blame us.

Aaron van Wirdum I mean, I have no idea what these numbers mean. I don't know how big is a Coinbase transaction usually.

Sjors Provoost That really depends on what you're doing. Like OCEAN pool is trying to pay all of their miners directly, so those are quite big. But they already know how to use settings like this. But, you know, a minimum Coinbase transaction might be a lot smaller. You also have to include the block header. So the block header is 80 bytes, which times 4 is [unclear] units. And then a transaction might be another 400. So maybe the minimum is 800, 900. But basically, while we were reviewing this pull request, people kept getting confused. They're like, well, if this confuses us, probably we should just pick a higher number than what we think it can be. Because we might be wrong.

Aaron van Wirdum Okay, but why make the minimum, maximum, wait, which one is it?

Sjors Provoost The minimum is 2,000.

Aaron van Wirdum Yeah, why not? Why?

Sjors Provoost Because if you set a number that's too low, you produce a block that's invalid.

Aaron van Wirdum That's not your problem, though, as a Bitcoin developer.

Sjors Provoost Who do you think they're going to get angry at?

Aaron van Wirdum Okay, so you're still giving some room for error there, or some safety, some seat belt.

Sjors Provoost Belt and suspenders, and if you really know what you're doing, you can edit the source code.

Aaron van Wirdum Okay, fair enough.

Sjors Provoost You really shouldn't for those five sats extra.

Aaron van Wirdum Right. Okay, so that's new. Next one, there's some mediation or solution. No, I'll just let you explain it. It has to do with a time warp attack.

Sjors Provoost Exactly. So as you know, we talked about the time warp attack in general many episodes ago. Now, there is a new proposal called the Great Consensus Cleanup that tries to fix the time warp attack once and for all. That is not deployed yet, but we do want to deploy some code in the miner that makes sure no accidents happen should this soft fork ever become active.

Aaron van Wirdum Okay, so there's now... I do want you to explain what the time warp attack was, or at least in brief. But if I understand this upgrade correctly, Bitcoin Core has included a solution to a potential future problem based on a future upgrade. So the solution is now there, but the problem not yet.

Sjors Provoost Basically, if you upgrade to V29 as a miner, but then you don't upgrade to whatever new version has this eventual soft fork, you will not accidentally violate the soft fork.

Aaron van Wirdum Right, okay. So yeah, I see how that makes sense. It sounded more silly when I said it, but yeah, of course, sometimes miners will be slow to upgrade, so actually it's a good idea to already.

Sjors Provoost Yeah, and generally with soft fork, it is important for miners to upgrade, but there are, ideally, you need 90% of miners to upgrade just to signal that they did so and safely enforce it so that there's no long reorgs.

Aaron van Wirdum That depends on... what settings you use, right?

Sjors Provoost If we ignore all the UASF stuff, the normal way that soft forks have been deployed so far, most of them, all of them, depending on who you ask, is 90% of the miners signal that they're ready to enforce it, which means that if a miner doesn't upgrade, their block gets orphaned, but there's not going to be a long chain split. Because if only half the miners enforce the soft fork, then you might get two roughly equally sized blockchains going in different ways and it becomes an absolute mess.

Aaron van Wirdum Strictly speaking, I think the 90% number has only been used once with Taproot. But that's another topic, because I think SegWit was 95%.

Sjors Provoost That's even safer.

Aaron van Wirdum Sure. And then, of course, there's... Well, let's not rehash that.

Sjors Provoost The point is, what happens with miners that don't upgrade?

Aaron van Wirdum I think during the block size, whatever the thing was called, that was even lower, like 75%. Anyways. That was not the Bitcoin Core thing. That was the Jeff Garzik thing.

Sjors Provoost The question is, in this way that some people would like to see soft forks happen, what happens to the miners that don't upgrade? Now, usually there's some, for example, one of the reasons for standardness transactions is that if there's a soft fork, the new transaction type that appears in this soft fork is non -standard for the older nodes. So an old miner will think a transaction is non -standard and won't include it in their blocks. So you cannot accidentally violate the rules of a soft fork when it comes to transactions, even if you as a miner don't upgrade.

Aaron van Wirdum Let's get away from a soft fork topic and let's go to the time warp topic. All right. So what is the time warp attack?

Sjors Provoost The time warp attack is basically shenanigans with the timestamp where you make it look by basically messing with the timestamps, you make it look like the block period was four weeks instead of two weeks or six weeks instead of two weeks. And then the difficulty goes down. And you do this because the timestamp of the first block, basically the difficulty adjustment looks at the last block of the period minus the first block of the period. Instead of what it should have done, the first block of the new period minus the first block of the old period. So because it doesn't overlap, you can mess with the timestamps, you decrease the difficulty very rapidly, and somebody can basically bring the difficulty back to one, which is basically a CPU miner. in 38 days, I believe, and then they can just mine all the remaining subsidy and create a complete mess.

Aaron van Wirdum This is a hypothetical form of attack that could be done. But then there's...

Sjors Provoost Well, it's not hypothetical. You can just do it. Yeah. It's a bad idea, probably.

Aaron van Wirdum I mean, it's not that easy either. You need a majority of miners, right?

Sjors Provoost Yes.

Aaron van Wirdum I mean, okay.

Sjors Provoost You need a majority of miners to... So the main concern would be... not a full time warp attack, but a limited time warp attack. For example, you can double the speed of blocks to have one block every five minutes. That means miners are getting paid two times the subsidy per day or per hour or per week. And everybody gets, you know, double the block size. I mean, that's, you know, at least something you can imagine actually happening. And [unclear] still be very bad.

Aaron van Wirdum Sure, I mean, I'm not... Yes. Agreed. And now there's a solution. Again, there is like a proposed solution for that that might be rolled out in the future. So what is the proposed solution that might be rolled out in the future?

Sjors Provoost So the proposed solution really boils down to putting some tie wrap around the last block of the period and the first block of the new period. Because right now, it's like they can just shift. You can shift them all the way back. But if you tie-wrap them together, you can no longer mess with the timestamp. And so the actual rule is the first block of the new period must be no older than the last block of the previous period, minus two hours.

Aaron van Wirdum Okay. So, yeah, we just had a block period of two weeks. The last block was mined at, you know, January 1st at noon. So that means that the first block of the next period the timestamp in that block can be at the earliest 10 a.m. on January 1st, right?

Sjors Provoost Yes.

Aaron van Wirdum That's the proposed rule. And that proposed rule is there to prevent this time warp attack.

Sjors Provoost Exactly. It prevents you from setting the timestamp [unclear] two weeks in the past.

Aaron van Wirdum Right. But now there's another problem.

Sjors Provoost Yes, the problem is...

Aaron van Wirdum Now there's a problem with the solution that needs to be fixed.

Sjors Provoost Exactly.

Aaron van Wirdum And again, the solution, this solution is actually now in Bitcoin Core for the future problem potentially.

Sjors Provoost Exactly.

Aaron van Wirdum And what is the solution?

Sjors Provoost Now, testnet4 already has...

Aaron van Wirdum What's the problem?

Sjors Provoost Testnet4, well, the problem can be seen on testnet4 because testnet4 does have the time warp fix deployed. However, instead of two hours, it uses 10 minutes as the rule. So if your block was mined at 12 o'clock... the last block in the period, the first block of the new period must be mined at no earlier, sorry, no earlier than 10 minutes before 12.

Aaron van Wirdum Yeah, 11.50 a.m.

Sjors Provoost I think, yeah, exactly. Now, the problem is that if you, let's say the last miner is a bit mean, the miner of the last block, they can put their timestamp two hours in the future, because Bitcoin Core allows up to two hours in the future. So now you are mining, and if you're just using your clock, you will just use the time of today, of right now. But we've just said that you have to be less than 10 minutes in the past from the last block. So now you've created an invalid block under the new time warp rules.

Aaron van Wirdum Okay, so indeed, you, Sjors, you are mining, and then because you're a criminal and a troll... You mine a block at noon on January 1st, but you put the timestamp in it of 2 p.m.

Sjors Provoost Exactly.

Aaron van Wirdum And now I'm also mining and I'm like a nice honest guy. So I now find the next block at 10 past 12, 10 past noon. You know, I'm just putting the honest.

Sjors Provoost Yep.

Aaron van Wirdum time in there, but then because you were trolling and put the timestamp in the future, now my block that is actually telling the truth with the timestamp, that one is now considered invalid.

Sjors Provoost Exactly.

Aaron van Wirdum So you successfully trolled me and robbed me of my money.

Sjors Provoost Exactly.

Aaron van Wirdum Well, you didn't get the money, but at least I didn't get it either.

Sjors Provoost I trolled you?

Aaron van Wirdum That in itself is a source of pleasure for you.

Sjors Provoost Yeah, well, I trolled you, I may have trolled the person after you because people may try to keep finding the first block and until actually two hours have passed, they will lose. So there's two solutions to this. One is that we just made the margin two hours so that you can't really exploit this anymore because you can't put the clock more than two hours in the future.

Aaron van Wirdum Right.

Sjors Provoost So you can't attack somebody who uses the real clock.

Aaron van Wirdum Right. So while on testnet, it's 10 minutes. So that's why the example that I just gave worked on the actual mainnet.

Sjors Provoost The difference is two hours.

Aaron van Wirdum It's two hours. So even if you try to troll me by putting the clock two hours in the future on your miner, then me, the honest guy with the real time in the block, that's still going to be fine because there's a margin of two hours.

Sjors Provoost Exactly.

Aaron van Wirdum So, okay, so...

Sjors Provoost But basically, you want to, yeah, but it's not, you know, it's nice to solve a problem in multiple ways, I guess, to make sure that it really works. And this solution also works in Testnet 4, so that's kind of nice. With this slightly more strict rule, the nice thing about Testnet 4 is it helps reveal bugs like this.

Aaron van Wirdum But what is the solution?

Sjors Provoost The solution is that Bitcoin Core will, when it creates a block template, it also provides some hints of what, you know... Remember, you have to make your own Coinbase, but Bitcoin Core will give you hints of what to put in that Coinbase. And one of the hints is the minimum time. So it will say, your timestamp must be at least this timestamp. So it will tell you your timestamp should be one hour and 15 minutes in the future. And as long as your mining software actually pays attention to that, they should be fine. Now we found some open source software that did not pay attention to that hint. So that even if Bitcoin Core fixes this, they're still... they still have a problem because they're not looking at the hint that Bitcoin Core gives them. But we don't know if mining pools have this bug because all the mining pool software is closed source. That's why you want to have sort of a double solution.

Aaron van Wirdum Hang on. Let me repeat the solution just to make sure I got that part right.

Sjors Provoost Yeah.

Aaron van Wirdum Okay. So it's noon on January 1st, and you Sjors are mining and trolling and putting the timestamp in there of 2 p.m. Now me, I'm also mining super honest guy, I want to put the real time in there. But then Bitcoin Core says, no, no, no. If you're being honest right now, then your block will be invalid. So you also have to troll along with Sjors.

Sjors Provoost Yeah.

Aaron van Wirdum And then I will put the block time also at like 2 p.m. or 2.1 or whatever.

Sjors Provoost And generally, Bitcoin Core will give you two values. It does that for every block. It says the minimum time. Like, do not go below this number. And it's a field called the current time. but now it's going to lie about the current time.

Aaron van Wirdum Okay, so because you were lying and cheating and stealing and being your usual self, now I go along in that behavior, but only for the first block of the period.

Sjors Provoost Exactly.

Aaron van Wirdum After that, because that's the time warp block, that's where that matters. And then whoever is mining after me, they'll start using the real time again.

Sjors Provoost Yes.

Aaron van Wirdum Right? Okay.

Sjors Provoost Bitcoin Core will suggest the real time again. Yeah.

Aaron van Wirdum When you say suggest...

Sjors Provoost It's only a suggestion. Bitcoin Core says you should put this stuff in the block, these transactions, and it says you should put this timestamp in the block with one field saying this is the absolute minimum. So maybe you don't want to use the timestamp that Bitcoin Core suggests for whatever reason, but it does say do not go below this number. Again, we don't know what miners are doing because it's all closed source. Now, on testnet 4, there's another weird little thing. that exposes this problem. So there is another rule called the MTP rule, which is that the timestamp of every block has to be higher than the median timestamp of the last 11 blocks. I don't know if you're familiar with that rule.

Aaron van Wirdum I mean, I've heard of it, yeah.

Sjors Provoost Yeah, so in practice, that tends to mean that... your timestamp must be at least no more than one hour in the past or something like that, right? Because it's six blocks per hour. So typically the median block from the last 11 would be the middle one, which is about an hour old. So typically your minimum time is about an hour in the past. So you don't even pay attention to this. You don't pay attention to minimum time. You just use the current time. But on Testnet 4, there's some crazy people that are CPU mining. as much as they can, and they're putting blocks two hours in the future to CPU mine for weird reasons. So because all these miners are putting their blocks two hours in the future, the actual median time past is one hour in the future, not one hour in the past. So if a miner does not use the field that Bitcoin Core gives them, the min time and the current time field, which are both lies, well, the current time field is a lie. Because the mining code will say, you must use one hour in the future as a timestamp, otherwise it's invalid. You seem confused.

Aaron van Wirdum I mean, you are getting very into the... Also, you're talking about Testnet now, right?

Sjors Provoost This is Testnet.

Aaron van Wirdum I don't care about Testnet.

Sjors Provoost Well, Testnet is how we found this problem, basically. Because these people are doing weird things on Testnet, being degens... Timestamps in testnet are actually in the future, which doesn't happen on mainnet. And therefore, the Bitcoin Core mining code always tells you to lie. It always says you should produce a block one hour in the future or it's going to be invalid. And especially around that time warp window, of course, it's even more clear. And then people started producing invalid blocks. And they were like, why am I producing invalid blocks? Well, it's because your software is not checking these values that Bitcoin Core is telling you to use. So that happened. But it was with some open source mining projects and maybe the real mining projects do actually check the hint. Because otherwise this could happen on mainnet.

Aaron van Wirdum Let's move to the next topic. I think, right? We're done with the...

Sjors Provoost Yeah, I think so.

Aaron van Wirdum Time warp potential future attack solution that's been implemented.

Sjors Provoost I think we'll do the other three topics a little quickly.

Aaron van Wirdum Yeah, that sounds good. I mean, okay, the first one is we can run through this in a... Let's see in 20 seconds.

Sjors Provoost Yeah, I think the mainstream media has already covered this one.

Aaron van Wirdum Right. Okay, Mempool, the full RBF is now the standard in Bitcoin Core. There are no other options.

Sjors Provoost Exactly.

Aaron van Wirdum So if you run a Bitcoin node and you receive a transaction and then you receive a conflicting transaction, so spending the same coins, but that has a higher fee. then now your node will say, that's perfectly fine by me, I'll just take this one and throw the other one out.

Sjors Provoost It's more complicated than just a higher fee, but you don't need to signal anymore. Because it used to be that we would only, we would take the first transaction seen, unless it's signaled this RBF flag, then we would say, okay, we'll look at the fees. But we also look at the fee rate has to be higher, the absolute fee has to be higher, a bunch of other rules that decide whether you replace it or not.

Aaron van Wirdum Okay, yeah, yeah. I mean, sure.

Sjors Provoost The marketeers always like to say, oh, we just replace everything with a better fee. It's like, eh, it's more complicated. But anyway, that's the default now. We've done multiple episodes on RBF and mempool full RBF.

Aaron van Wirdum Yeah, I think including the last one. So step by step. Bitcoin Core has been moving more towards a world where highest fee just sort of wins. Of course, needless to say, but I'll say it anyways, this is all before a transaction is included in a block. Which transaction is a miner going to include in a block? rather than looking at which transaction is received first, or like this flag that you mentioned, it will just say, okay, this one is paying a higher fee.

Sjors Provoost Unconfirmed transactions are unsafe. They were always unsafe. They are now marginally more unsafe. But that's sort of the direction. This was sort of slowly coming. There was mostly an argument about how soon.

Aaron van Wirdum Okay, we far exceeded our 20-second limit for that one, so let's move on to the next one, which is CMake.

Sjors Provoost Yes, so as a user, you won't notice anything, but this was a giant change under the hood in the way that you go from the source code to the actual binary. So source code stays more or less the same, binary stays more or less the same, at least from what you experience as a user. But the way you go from source to binary has been revamped by using this thing called CMake that everybody else already used. And yeah, this makes things much better.

Aaron van Wirdum So Sjors and other Bitcoin Core developers, they're writing their code on their keyboards. And it looks all like C++-y. But then your computer needs to translate it into ones and zeros. And now there's a new way for computers to translate.

Sjors Provoost Yes, we used to type make and then hit enter. And now we type cmake and then hit enter.

Aaron van Wirdum It's not what I was referring to, but also good to know for people that you know, well, that... Must be a drag, though. That's like an extra...

Sjors Provoost An extra character, yes.

Aaron van Wirdum An extra character you have to work with.

Sjors Provoost You can make shortcuts, so it doesn't matter.

Aaron van Wirdum Okay. Again, nothing that regular users like myself will ever notice, but developers...

Sjors Provoost You will see fewer grumpy developers. But yeah, that's about it.

Aaron van Wirdum You're always... Even you, Sjors? Even [unclear]?

Sjors Provoost I don't work on the build system. You will see more happy build system developers Let's put it that way.

Aaron van Wirdum Okay, last one then. We're almost there. Oh, UTXO to SQLite. That's also, there's a new feature in Bitcoin Core, which is you can dump the UTXO set into SQ... SQLite, yeah.

Sjors Provoost Or SQLite, depending on who you ask.

Aaron van Wirdum Exactly.

Sjors Provoost So it's been possible for a while to make a snapshot of the UTXO set just to dump it. That's used by AssumeUTXO as well to load the UTXO set, but you can do it at any height. And it's just a list of every coin that exists, but the format is very unusable. So there's now a little script that takes that unusable format and turns it into a SQLite file, which is much easier to edit. It's not a spreadsheet quite yet, but it's a lot closer for most people that know what to do with databases. So it's basically a research tool. If you want to study what coins exist, or maybe you want to make multiple snapshots over time and compare how the UTXO set changes over time, this will make your life easier.

Aaron van Wirdum Yeah, okay, so it's basically a new tool for data nerds, researchers that want to figure out how are coins moving over the blockchain or how are coins distributed across different addresses and how long does it take before UTXOs are spent compared to older UTXOs or whatever you may come up with and you want to research. Now there's an easier tool to get that data from. Bitcoin Core into whatever.

Sjors Provoost So anyway, go to BitcoinCore.org, download version 29, do the checks for the checksums, and have fun with it. So is that all we got?

Aaron van Wirdum Where can people find Bitcoin Core 29?

Sjors Provoost BitcoinCore.org.

Aaron van Wirdum Did you already mention that?

Sjors Provoost I did.

Aaron van Wirdum Good. Okay, then we're there.

Sjors Provoost All right. Thank you for listening to Bitcoin.

Aaron van Wirdum Explained.