Episode 79: The Witness Discount

Aaron van Wirdum Live from Utrecht, this is Bitcoin

Sjors Provoost Explained.

Aaron van Wirdum Hey Sjors.

Sjors Provoost Hello.

Aaron van Wirdum Sjors, today we're going to cover another Ordinals related topic. Not really, but I'm just giving you some context why I came up with this suggestion. The reason is over the past couple of weeks I got the same question multiple times. Once on a Twitter spaces and I think maybe once or twice also just in a private conversation. and the question is why is the witness discount there in the first place because that's what Ordinals are using no well that's what inscriptions are using I should say

Sjors Provoost they're taking advantage of the fact that there is a witness discount Yeah.

Aaron van Wirdum Right, so I got the question why is the discount even there and that's the question we're going to answer in this episode isn't that right

Sjors Provoost That sounds like a great question let's answer that.

Aaron van Wirdum But first everyone's favorite part of the episode Sjors stacks sats. Sjors stacks sats. Sjors stacks sats.

Sjors Provoost Well, we have a very recursive sat this this week from ad fake for 12,837 sats saying Sjors stacks sats with a bunch of emojis.

Aaron van Wirdum You're not saying it right, just you gotta sing it you gotta sing it No, it's your you're the one stacking sats you gotta sing it you gotta read it out loud singing.

Sjors Provoost I don't sing.

Aaron van Wirdum Come on Sjors, everyone wants to hear this.

Sjors Provoost I don't sing.

Aaron van Wirdum All right. Well, do you want to tell our listeners how they can help you stack sats?

Sjors Provoost Yeah, they can use any of the podcasting 2.0 apps that do this.

Aaron van Wirdum And it's called boostagrams, right?

Sjors Provoost Yeah.

Aaron van Wirdum Okay. People can Google that. That was Sjors Stacks Sats. Ordinals. No inscriptions. No witness discount. That's what we were.

Sjors Provoost That's right.

Aaron van Wirdum Okay. Yeah.

Sjors Provoost So...

Aaron van Wirdum also it's it's probably a good topic to cover an episode at this point because when this was so the time this was discussed by now you know it's been seven eight years Pieter Wuille to be exact proposed SegWit which included the discount in December 2015 and then we had a couple of years of drama and block size wars and scaling all of that and it was all discussed to death for a while but and SegWit activated in 2017 so even that is almost six years ago by now so there's probably a lot of people that weren't around by back then

Sjors Provoost That's right.

Aaron van Wirdum that know that there is a discount but just weren't there at the time and even for us that's what I was saying when it was asked to me I could sort of answer it but to really sort of answer this question properly you and I you more than I went back way into the archives and you watched some old presentations and tried to figure out what the reasoning was back then. Isn't that right?

Sjors Provoost That's right.

Aaron van Wirdum Okay, so before we get there, maybe we should first give a very brief recap of what SegWit is?

Sjors Provoost Yeah, I think so. We did a whole episode about it. SegWit has many different benefits, but the thing that we're focusing here on is that it was also a block size increase and that it created the distinction between Well, it basically splits the transaction into two parts. The main transaction, so to speak, and the witness, which generally is the signature or stuff like a signature, like a hash pre-image, things like that. And the thing that I guess people are asking about is that these witness data, these signatures... are discounted so if you're you know if you're making a transaction that's 300 bytes then you're you're paying more for the part of the transaction that is not the witness than you are for the part of the transaction that is the witness.

Aaron van Wirdum Yeah, maybe the other thing to mention about SegWit is I don't know if it was the main reason but at least one of the big reasons what it was that it also made it it fixed the malleability bug right? Well, we did an episode about this we don't…

Sjors Provoost Yeah, I mean that that is the exactly that was the main point of it it segregates the witness and this fixes malleability. Yeah.

Aaron van Wirdum I mean was it the main point I don't think it really matters at this point but there were definitely two main reasons right malleability for Lightning and then a block size limit increase that didn't require hard fork and that's because of the discount,

Sjors Provoost Right, but I don't think that the idea of segregating the witness had nothing to do with increasing the block size it's just that people realize that you could use it to increase the block size which is kind of a nice bycatch right and once you do realize that you can start reasoning why the discount is roughly what it is.

Aaron van Wirdum Okay, so I think that's what we're getting into I think the best way to get into it is just restate the question and then you can take it for that from there what do you think?

Sjors Provoost I can try that.

Aaron van Wirdum Sure. why is there a witness discount?

Sjors Provoost Well, it's how you increase the block size.

Aaron van Wirdum Aha, but that's not the only reason right? There's there's more to it. Well, there's more going on you just mentioned you don't did you just mention this you don't actually have to include a discount to increase the block size right?

Sjors Provoost Yeah. So, okay. So the question is, if you just wanted to increase the block size, how would you do that? Well, the original idea was to do a hard fork where you simply increase the block size. So there's just instead of buying five ice creams per block, now you can buy 10 ice creams per block.

Aaron van Wirdum You can make double the amount of transactions.

Sjors Provoost That's right.

Aaron van Wirdum Right. And hang on. So at this point, we're not even talking about... the discount yet right?

Sjors Provoost No, this is a very simple way to increase the block size is just a hard fork that was known but there were good reasons not to do a hard fork.

Aaron van Wirdum Oh, sorry yeah that you were still in the hard fork part. Yeah, okay, yeah. Let's get to SegWit then why was SegWit better other than it wasn't a hard fork or wasn't it

Sjors Provoost Yeah, so SegWit was so the idea of segregating a witness existed before the idea of the soft fork called SegWit right But the question is, but it turns out that you can use the soft fork SegWit to increase the block size. And the reason is because you are basically making transactions that to old nodes look like they don't have a signature. They're just giving away the money for free to anybody else who wants it. But new nodes will download essentially a separate block or a separate part of the block that is a witness block. And this could be as big as you want in theory. which means that you could...

Aaron van Wirdum Let's pause there for one second. So even by doing that, that would in itself be a block size increase in a way, right?

Sjors Provoost It could be a block size increase really depending on how you do it because you can add an additional constraint. So if you just say that you can have as much witness data as you want, then it would be an infinite block size increase essentially. But you could also say, well, you cannot add as much stuff as you want the total amount of data. So the total between the block and the witness still needs to be one megabyte, for example. So it could have been done without a block size increase.

Aaron van Wirdum Okay, I think this is a good point to explain what I meant by that. The important part is that old nodes will only accept up to one megabyte of transaction data.

Sjors Provoost Yes.

Aaron van Wirdum A block must be one megabyte. Now, if you're moving part of the transaction, and I think a witness is a big part of the transaction, right?

Sjors Provoost Certainly, yes.

Aaron van Wirdum It's around half or even more than half. It depends on the transaction, right?

Sjors Provoost Yeah, I think for a typical transaction, it was slightly less than half.

Aaron van Wirdum Yeah, maybe 40%. Anyways, it's roughly half. So in that case, if we're assuming there's only going to be regular transactions, which is obviously a bad assumption, but let's make that bad assumption, then already... you're almost doubling the amount of transactions that can be processed on the network just by separating the witness yes right if you're not adding any additional constraints because now the base part of the transaction can be up to one megabyte all the base parts of every transaction in the block and then all the witnesses which is almost half is in a separate part so already you've doubled it just by segregating the witness under a bad assumption and no additional constraints, right?

Sjors Provoost Yes.

Aaron van Wirdum Okay, so maybe a logical next question is, why wasn't this done, I guess? Why wasn't this just what segregated witnesses?

Sjors Provoost Well, the thing here is that the question is, how big should that extra data be? Should there be a limit on it? And if you make that a separate limit, so let's say you say the main stuff in the blockchain is one megabyte per block. And the witness, we're just going to pick a random number, maybe call it also one megabyte or two megabytes, whatever.

Aaron van Wirdum Yeah, let's just also say one megabyte, right? That way it's a doubling. That's what I was just...

Sjors Provoost Yeah, but if you do that in a naive way where you just say these are separate buckets, then you create a very difficult fee market because now in order to decide what goes into a block, the miner has to solve what's called a nonlinear equation, basically, where they have to decide how... Basically, you have to compete for space in the main block and you have to compete for space in the witness block and those are completely separate markets. And that is mathematically a very hard problem to solve.

Aaron van Wirdum Yeah, I guess the problem, and correct me if I'm wrong, the problem would be that a transaction only includes one fee, but now all of a sudden there's kind of two markets, one for the base transactions and one for the witness. And so now the miner has to make some sort of calculation per transaction, whether to include it or not. And depending on those buckets, and this gets very complicated very fast for miners to figure out what to include. That's sort of right, right?

Sjors Provoost Yeah, and it also creates potential for abuse. So let's say the main part of the block is the most scarce, and so fees are quite high for that. But once you've paid for that, the witness part is really, there's not much competition. So if you want to spam the witness part, that might be very cheap, and that creates all sorts of weird incentives.

Aaron van Wirdum Right. So this argument can be sums up as things get complicated.

Sjors Provoost You don't want to have your fee market based on more than one variable.

Aaron van Wirdum Right.

Sjors Provoost And actually, if we look at the modern world, that's not the case because we'll probably talk about this in earlier episodes. There's such a thing as transaction packages where you cannot look at the fee of one transaction. You have to look at the fee of a combination of transactions. So actually, the problem is already very hard.

Aaron van Wirdum And also what we mentioned in one of the previous episodes, like the SIGHASH limits should maybe be taken into account at this point.

Sjors Provoost That was already a factor that you had to take into account. So your fee calculations get more difficult if you take the SIGHASH into account. which generally we just kind of don't.

Aaron van Wirdum Nevertheless, even though it's still more complicated than one variable, it would be worse if there were even more variables.

Sjors Provoost Well, there was some discussion on the mailing list that said, well, going from one to two variables is very, very bad. Going from two to three is probably not as bad. But I guess the hope was to really nail it to one variable and somehow find a way to deal with the SIGHASH limit. these things so anyway it's nice to have one fee rate.

Aaron van Wirdum Okay, so this okay so go on. so it's nice to have one fee rate I mean…

Sjors Provoost Well, that means that there has to be one limit and right now as we discussed it the simple way would be to have two limits so how do you turn two limits into one limit that's essentially what was done and the solution to that is this thing called virtual bytes where you say we're not actually counting the real bytes anymore we're making this thing called the virtual byte and the simplest way to do that might be to say one byte in the main block counts as one virtual byte and one byte in the witness counts as one virtual byte so really one byte is one byte but we are going to limit the total number of bytes to one million so one megabyte So this means that you can have as many witness bytes as you want or as many main bytes as you want, but the total has to be 1 million. So now you have one limit.

Aaron van Wirdum Okay. Well, the example you just gave would not be a block size increase then, right?

Sjors Provoost Exactly. That would be no block size increase.

Aaron van Wirdum Right.

Sjors Provoost If you wanted to do your 2 megabyte example, then you would have to make the limit 2 megabytes. So you would say, but that's not possible, right? Naively. Because if you... Because you said the total limit has to be two, but you can't have the main block be more than one. So that's not possible, naively.

Aaron van Wirdum The reason it's not possible to make that very clear is it wouldn't be backwards compatible.

Sjors Provoost Exactly.

Aaron van Wirdum Old nodes would now see two megabyte blocks and reject them while new nodes would be fine with them. So that's the reason you're saying it's not possible.

Sjors Provoost So first we started with, we split the witness out from the main data. Then we're like, oh, that gives us a lot of extra space. But oh my God, now we have this super complicated fee market that nobody can reason about. Let's bring back the limit of one megabyte for the total amount. So now we don't have a block size increase anymore. What do we do? and this is where the witness discount comes in.

Aaron van Wirdum Well, or you're missing one step or we can increase it but then it wouldn't be backwards compatible that'd be hard fork, so not Yeah, so now that so this is where the discount comes in okay so how does the discount actually work maybe that's a good next question?

Sjors Provoost Yeah, so what the discount does it's more like a you could also see it as a tax on the old stuff. It's basically saying every part of the old chain is, sorry, every part of the main part of the transaction, which means all the outputs and all the sort of stuff in the middle of a transaction counts as four, and all the witness stuff counts as one. And then we say the total is, in this way of counting, is four million. And so what that means is if you only use old stuff, every byte counts as four. So if you only use, basically you don't use SegWit at all, your block gets to be one megabyte and then you hit this four weight limit limit.

Aaron van Wirdum Yeah, it's a little bit confusing here because again, when we say we're going to make the new limit four, wait, what did you say?

Sjors Provoost You can also do it the other way around. You can say, well, we'll count the main parts as one. So one megabyte of main stuff is one megabyte block in total. And we're going to count everything in the witness. We're going to count that as 0.25.

Aaron van Wirdum So one quarter.

Sjors Provoost That's maybe easier to think about.

Aaron van Wirdum Actually, yeah, I was just going to say that's an easier way to understand it. And it's kind of equivalent.

Sjors Provoost And this is the difference between weight units and virtual bytes. So this last thing we talked about are virtual bytes. So we say one virtual byte in the main part of the block or in the main part of the transaction is just one byte. but one byte in the witness part counts as 0.25 virtual bytes.

Aaron van Wirdum Right. So now there's a block being mined with, yeah, just one megabyte of all transaction styles. So now both new and old nodes will just see one megabyte. It's fine.

Sjors Provoost Yep.

Aaron van Wirdum And the alternative is, okay, so the other extreme would be there's going to be four megabytes of witness stuff, and then new nodes are going to discount this. 0.25 so now they're seeing one megabyte again so that's within their limit as we're talking about weight limits yeah and then all the nodes are just going to see nothing essentially they're going to see an empty block because everything is in the witness precisely so an empty block is also within the one megabyte limit so now both nodes are okay yeah exactly okay so basically what we've said now that so this sounds a lot this argument sounds a lot like It is kind of an accounting trick. We just want to make sure that we have a block size increase while we don't create different fee markets because that gets very complicated. Right.

Sjors Provoost So I would always or I would tend to separate it in three concerns that we need to deal with. One is that you want to have one market despite having this separate witness. The other is that we wanted to have a block size increase somehow. And the third is then the way you do that is by having this discount in the witness data. But you could kind of do these things separately. So we could have separated the witness, but kept the block size the same. And we could have increased the block size without SegWit, I guess. But if you wanted to increase the block size and have SegWit, then the only way to do that is with this witness discount. Now, the exact number is somewhat arbitrary. Well, we can defend that number, but that will be the next phase. Now we've explained why you really need to have a discount.

Aaron van Wirdum yeah okay so we can defend the number before we get to defending the number I think there is also an argument or at least there was an argument a more technical argument for giving a discount it would better align incentives of the Bitcoin system is that right?

Sjors Provoost Yeah, and there's there's kind of two ways to look at those incentives we're talking about the UTXO set essentially so this is the the set that you're kind of keeping either in memory or at least you you very often need to read read it which is the list of who owns which coin and by using this discount mechanism that list tends to not grow a lot And there's two ways to look at that. One is we're saying, I guess the more abstract way is to say the block size is increasing, but the only part that is increasing is the witness. And the witness does not go into the UTXO set ever. You check the witness, but you don't put it in the UTXO set. So by only increasing the witness size, you are increasing the block size without increasing the UTXO set. The other way to look at it. It's more from an individual transaction point of view, where before SegWit, it was much more expensive to spend an input than to create an output for a very basic transaction.

Aaron van Wirdum Wait, before? Sorry, can you repeat that for me?

Sjors Provoost Before SegWit, it was more expensive to spend an input than to create a new output. And the reason is because when you're spending an input, you need to put on the blockchain usually a public key and a signature. Because you're trying to prove that you have the hash. And so that's about 90 bytes, roughly. Whereas to create a new output is basically just 20 bytes. Because you only need to put the hash of a thing on there. Well, at least it's 20 bytes more than a few other things. You also have to put the amount on the chain, etc. That doesn't change. So SegWit changes this because of the discount. Changes this math.

Aaron van Wirdum What does it, though? Because you just said it's 90 versus 20. 90 for an input, 20 for an output.

Sjors Provoost Yeah, and so the input now gets a 4x discount.

Aaron van Wirdum Yes.

Sjors Provoost So that makes it, you know, it's still slightly more expensive.

Aaron van Wirdum It's only the witness, right? Yes. And there's more to the input, I would assume.

Sjors Provoost Yes, exactly. So it doesn't work exactly like this because the input also has to point to the previous transaction that takes up space.

Aaron van Wirdum Right.

Sjors Provoost And that doesn't get a discount.

Aaron van Wirdum Right.

Sjors Provoost But at least it makes it significantly less expensive to spend an input than it used to be.

Aaron van Wirdum Okay, so it's about equal now? Creating an input is about as expensive as creating an output?

Sjors Provoost I mean, it still depends on which kind of input and which kind of output. I think it's still slightly more expensive to spend it. Yeah, slightly more expensive to spend it. But at least the difference isn't as big anymore.

Aaron van Wirdum Okay, and what is the practical benefit of this?

Sjors Provoost Well, so yeah, so I read, I think it was the Stack Overflow or Reddit somewhere where somebody gave a nice example of like you have a wallet and has three coins in it. It has a 0.25, a 0.5 and a one BTC coin in it. And you want to spend 0.75. So anybody who's very good at math can see there's two ways to do that. You can combine your 0.25 and your 0.5 coin. That's 0.75. Or you can use your one BTC coin and you get... 0.25 and change those are your two options and in the old situation because it is so expensive to spend an input you would be better off taking your one BTC coin and then just creating the change address rather than spending those two coins

Aaron van Wirdum Right, so then you have four—no, then you have three UTXOs still.

Sjors Provoost Yes.

Aaron van Wirdum Plus the UTXO that's being spent to the ice cream seller.

Sjors Provoost Well, that's not your problem.

Aaron van Wirdum Yeah. Okay, so maybe this is part of my confusion then.

Sjors Provoost The incentive for you in the old days...

Aaron van Wirdum We're really talking about individual incentives here.

Sjors Provoost Yeah, so in the old days for you as the sender, it was better to just take your 1BTC coin and get some change back. because the change that you have to create is only 22 bytes, at least the part that, you know, you're actually, 20 bytes, sorry, the part that you're actually putting, that you have some control over. Whereas if you had to spend that old coin, you had to use these 90 bytes, so these 90 bytes, if you wanted to add, sorry, if you wanted to add those two coins. You have to spend at least one coin, right? So the difference is, do you spend an extra coin? and make change and don't have change or do you not do that and only have change and so that the difference is 90 versus 20 so the difference is roughly 70 bytes

Aaron van Wirdum Yeah, but hang on.

Sjors Provoost And with SegWit the difference is much smaller.

Aaron van Wirdum Sjors, I'm confused okay you're saying this is in my individual benefits but who cares about my individual benefit isn't isn't the goal you

Sjors Provoost do.

Aaron van Wirdum Yes, but why do Bitcoin developers because isn't the goal wasn't the goal to reduce the UTXO set?

Sjors Provoost Yeah, exactly. So it would be nice if the incentives are such that you prefer to spend your coins, I.e. you reduce the size of the UTXO set. It should be in your interest to do that.

Aaron van Wirdum Yes.

Sjors Provoost And it wasn't. Because in this example we just made, it was much cheaper for you to not spend the small change that you had in your wallet, but rather just fragment it further and further and further and further. Maybe in the long run that changes, of course, because eventually you'll have to spend it.

Aaron van Wirdum Would this only hold up if you can add up your UTXOs to make a payment in such a way that there's no change address?

Sjors Provoost Yeah, I think so.

Aaron van Wirdum Right. Okay. So in the cases where you can add up UTXOs to prevent getting a change address, in these cases, it's better to have the discount because you're incentivized to actually do that rather than to take one big chunk of coin. and get a change address and only have one input yeah okay I think that makes sense that sounds about right to me

Sjors Provoost Yeah, and of course it is always more complicated because you have to think about what's the future fee rate going to be so I've created all these change outputs okay but now you're stuck with them so eventually maybe you still want to consolidate them and if you expect fees to go up in the future it may still be in your interest to make sure you don't have too many small coins because it's going to make your life harder in the end yep.

Aaron van Wirdum okay so this is this is the argument for why it's better aligns incentives with the greater Bitcoin protocol essentially right yeah with the Bitcoin system was this the only argument in that direction or was there something else

Sjors Provoost Yeah, well, the other was basically, well, I guess that gets us to why the 4x would be good.

Aaron van Wirdum That was going to be my next question. Yeah, why 4x and why not 2x or 8x or 500x?

Sjors Provoost Yeah, so first of all, I think, I mean, the number was, I don't think it was calculated with any precision, with any kind of algorithm. It was sort of a guesstimate. And then it's useful to understand that software developers like powers of two. So they like the number 1, they like the number 2, they like the number 4, they like the number 8, 16, 32, etc. They don't like numbers like 3 and 5. So really, you know, why is it 4, why is it not 2, why is it not 8 should be your question. And then you, basically the analysis is kind of like this. The part that you're making more difficult is the signature validation. So you're adding four times as much signature stuff potentially because you're giving it a discount. And it turned out that recently it became about five to seven times easier to verify those signatures.

Aaron van Wirdum Well, when you say recently, you mean recently before SegWit was introduced.

Sjors Provoost That's right. Because we did another episode a long time ago called libsecp256k1, where we explained this library that essentially is purpose-built for Bitcoin that just validates signatures and a few other things. And that library was made about, yeah, like I said, it was about five or seven times faster. So knowing that you can verify five times more signatures per second means that it's kind of okay if there are five times more signatures, at least from the point of view of your CPU. So if you look at computer science in general, there's kind of different bottlenecks. One of your bottlenecks is your CPU, the other is your RAM, the other is your download, and the other is maybe your disk storage. And so here the idea was, yeah, we can probably reasonably safely increase this by four because it doesn't make it much worse.

Aaron van Wirdum And that was because the library was updated is what you were saying. The other thing I do remember, I don't know how big of a factor that was really for the developers, but I think it was Jonathan Toomim who was a Bitcoin, one of the forks, Bitcoin Classic. Bitcoin Classic developer I think and he did some testnet testing and I think he also found that four megabytes transmits broadcasts over the network reasonably fast enough, and it kind of slows down if it gets bigger than that.

Sjors Provoost Yeah, and so this gets to the analysis of all these different bottlenecks, right? So one bottleneck we talked about CPU, the other might be how fast blocks propagate, and that tends to be a function of bandwidth. But the good news there is that there were ways already in development and already partially deployed that make it easier to transmit fairly big blocks fairly quickly. I think we haven't done an episode on compact blocks yet, but maybe one day we will. But basically the idea is you anticipate what the next block is going to be, and then you only get the header and a few missing transactions. So then it doesn't matter that these blocks are very large. Because if block propagation gets too slow, then you get too many orphan blocks, etc.

Aaron van Wirdum Do we have compact blocks in Bitcoin now?

Sjors Provoost Yeah, we've had them for a long time.

Aaron van Wirdum Right. Right. Okay.

Sjors Provoost In fact, I think it was done around the same time because it was one of these ways to mitigate this risk of increasing the bandwidth. And then I think the other bottleneck, well, I think there is a bottleneck storage, but for that, there was already a solution called pruning. So that really leaves two. One is to keep up with the blockchain. That is not solved with SegWit, makes it four times worse, right? Or two to four times worse. You have to just, when you're keeping up, you just have to download more because you have to check everything.

Aaron van Wirdum I mean, but keeping up also means validating the signatures, right?

Sjors Provoost Yeah, exactly. So that became four times worse.

Aaron van Wirdum Okay, except that the library, the crypto library was updated too. Handle it faster.

Sjors Provoost So your CPU is fine with it, but you still have to download it.

Aaron van Wirdum Oh, yeah, yeah, sure.

Sjors Provoost So that's the problem. But the good news there was if you look at, say, SPV wallets or light clients or mobile wallets, typically what they would do is they would not actually verify all the things. And so the stuff they still have to download, because they're not doing complete validation anyway, the stuff they have to download does not go up at all. because they don't have to download any of the witness data anyway. So the idea was there, okay, their life is not getting any worse for the light wallets. Of course, light wallets have their downsides, but at least their life doesn't get any worse. Then another constraint is if you are a new node and you need to download the entire history of the Bitcoin blockchain, how does that change? And there, I think the easiest use case to look at, if you're trying to download the whole blockchain and have an archive, then that just got four times worse, right? But if you are a little bit more conservative on how many resources you have, you may want to be pruning. And if you're pruning anyway, then as recently people started to actually implement this, then you can actually just not download the witness data. And if you don't download the witness data for the first part of the blockchain, well, then your blocks are essentially just as big as they used to be. There is no block size increase for the history. There's only a block size increase for the present, but there's no block size increase for the history.

Aaron van Wirdum so I remember this argument from back in the days but I was also under the impression that this argument was kind of dropped and there's probably a couple of reasons for that but the first one I can think of is well you can do that anyways right with or without putting the signatures in the witness you can always just decide not to validate or download the signatures or at least

Sjors Provoost No, technically possible not, because the problem is in the old system the signatures are really stuck inside the block in a way that you cannot take it out safely because in order to see if a block is what it says it should be you're checking the header of a block and if the header of the block basically says what's in the block and if you're not downloading the signatures you can't check whether that's any of that is true so that's pretty unsafe

Aaron van Wirdum Right, okay so in that okay so witness segregated witness does actually make what do we call it assumevalid more plausible

Sjors Provoost yeah it makes it more practical to because assumevalid what it does so far we talked about that in an earlier episode assumevalid simply downloads all the signatures but just ignores them but with this change assumevalid could actually download all this not download all the signatures right the downside then is of course you do not have an archive of the blockchain so I guess the practical utility there for that was somewhat limited in recent history so nobody's actually implemented that The only use case of this is if you plan to prune it anyway. So if you plan to prune all the blocks after you've verified them, there really is no point in downloading the witness data and then deleting it.

Aaron van Wirdum Okay, that would be the difference. It would save on downloads.

Sjors Provoost Yeah, and so for those users, the contrast with the regular hard fork becomes very clear, I think. So if your goal is to just download it and then prune it, you don't have to download the witness data. So for you, there is no block size increase from your perspective. But if there had been a hard fork doubling the block size, you would have to download all of that stuff because just the amount of data would have doubled.

Aaron van Wirdum Right. Okay. Taking one small step back. So I think it's fair to say that the 4 megabyte limit was sort of picked to deal with worst case scenarios, right?

Sjors Provoost Yeah, you don't want it to be too high. And I guess, yeah, some of the examples you called might have been the case. And also it was picked to be pretty close to what they knew was improved because they knew the 5 to 7x thing in the CPU performance was there. And then it happens to work out quite nicely that it creates a little bit of a better balance between spending and changing coins. But whether that was really part of the calculation, I'm not so sure.

Aaron van Wirdum Right. Is there anything else you want to say?

Sjors Provoost And of course, the number four will actually give you a decent block size increase, right? So if you look at what happens if you... Just create the same type of transaction, so you're still doing, I don't know, two inputs, two outputs. That gives you, I think, about a 75% increase in the block size, effectively, if everybody used SegWit, so you get 1.75. Whereas if you'd only done a factor two discount, maybe you would only get 1.25 megabyte block. So you want to actually make a dent in the block size without it being too crazy. The number four makes a lot more sense than the number two. But the next number eight, it gets a little bit scary because it's not that easy to justify that increase.

Aaron van Wirdum Yeah. Another thing I remember from back in those days. So one of the very few, if not only sort of valid criticism on SegWit at that time, in my opinion anyways, was that the real block size increase that you gain from it is about double, like you just said, while the worst case block size increase that you get is four megabytes. So an attacker could sort of do worse than honest users.

Sjors Provoost Yes.

Aaron van Wirdum That was an argument against SegWit at the time, which I found was one of the very few, like I said, one of the few, if not only, valid arguments.

Sjors Provoost But then, as opposed to doing what? I mean, obviously, it's better to not increase the block size from that point of view. But what was the alternative that they would propose in that case?

Aaron van Wirdum A hard fork. These were the people that wanted a hard fork.

Sjors Provoost Yeah, so you could say that, but we just went through all these other bottlenecks, and basically the number 4, 4X worst case, was acceptable for most of these scenarios.

Aaron van Wirdum Yes, but it would still be nice, so to say.

Sjors Provoost Whereas a straight hard fork would actually make things worse for a lot of these scenarios. So for the scenario, for example, the prune node that we talked about, the prune node that for the first time catches up to the blockchain. If you do a 2x increase, then yes, 2x is the worst case, but 2x is also the best case, because that prune node now has to download twice as much stuff to verify the blocks. Whereas with SegWit, in the good case, they don't have to download more data. And in the worst case, they also don't have to download more data, because all that witness stuff is just... can be completely ignored.

Aaron van Wirdum Well, so let me rephrase the question see if we're talking about the same thing?

Sjors Provoost Yeah, you're comparing a straight hard fork a straight 2x hard fork to…

Aaron van Wirdum Well, let me be more specific so what for example Gavin Andresen back in these days I think fairly shortly after the SegWit presentation by Pieter Wuille someone asked him for his opinion or he weighed in on a mailing list or something like that and he said he liked SegWit but he would like to see it as a hard fork so then you still get the witness separate and the transactions it's basically all the same however it's done as a hard fork and then you can for example do four megabytes so now you can have four megabytes of both regular data or worst case so now the worst case and the best case is the same is that what you were also saying?

Sjors Provoost I'm not sure if the hard fork makes any difference for this economic side of it it might. I'm… also because you still need to have one metric otherwise you have a problem But because you don't have to be backwards compatible, yes, you can simply say that the total of the witness and the normal data has to be 2 megabytes. And this time you don't care what the old nodes think about that. But then if you wanted to talk about the worst case safety, which you are, I think, then you want to look at the 2x variant, right?

Aaron van Wirdum I mean, we just established that the network, at least back then, that people reasoned 4MB would be feasible worst case, right?

Sjors Provoost The criticism you were... formulating was that there was a good case and a bad case and the good case was two megabytes and the bad case was four megabytes by the by the abuser basically which

Aaron van Wirdum is how SegWit is designed now.

Sjors Provoost Yes, yes. So then what do you want to compare that with if you if you compare that to a normal 4x increase then you're basically saying let's just build the worst case, whereas if you can right if you're comparing it…

Aaron van Wirdum Well, that's the I mean even the worst case can be fine, but it's nice if the good case is the same as the worst case, right? An attacker doesn't have a benefit, so to say.

Sjors Provoost Exactly. So if the good case is the same as the worst case, either for 2MB or 4MB, then that's only interesting for the person who is downloading the full blockchain archive. Because in the other scenarios where somebody is downloading a pruned version, it's actually better to have SegWit because they can safely ignore three out of those four megabytes.

Aaron van Wirdum Let's pull this conversation to the present day because that's where I was sort of heading at.

Sjors Provoost Okay, so now you have these actual four megabyte blocks and people are saying, oh, if we hadn't had this discount, this wouldn't have happened.

Aaron van Wirdum Right, yes. So that sounds like a...

Sjors Provoost If we had done a 4x hard fork, then this would have happened, right? We simply would have...

Aaron van Wirdum Well, but then that all the space for it yeah but then the inscriptions wouldn't have a cost benefit against regular transactions and I think that's like that's for me at least the one valid or the one criticism of sort of the inscriptions haters that I agree with like it's it doesn't really feel right that inscriptions are cheaper byte for byte versus regular transactions it would be nice if that wasn't the case.

Sjors Provoost Well, that's not I'm not sure if that's true because we talked about one of the reasons why it's good to discount witness data which is that it does not go into the UTXO set

Aaron van Wirdum Right, okay so that's yeah this is exactly what I was gonna ask you I guess so yeah go on I shouldn't…

Sjors Provoost So it is so I think it is actually good to discount witness data regardless of whether you use a soft fork or hard fork for that reason because you want to incentivize people not shoving stuff into the UTXO set

Aaron van Wirdum Yeah, but hang on. If you're creating an inscription, you're also not using up UTXOs, right? It's just a wash. You have one, you still have one, basically.

Sjors Provoost Yeah, so inscriptions don't pollute the UTXO set at all.

Aaron van Wirdum I mean, except that each inscription does probably create one UTXO in the form of an ordinal.

Sjors Provoost Sure, but just one. They're not big UTXOs,

Aaron van Wirdum right?

Sjors Provoost If there was no discount for doing that, you might as well just create a giant UTXO, perhaps.

Aaron van Wirdum Okay, if there was no discount...

Sjors Provoost Then there's no reason to not say, I'm just going to make an Opera... So there's OP_RETURN, right? OP_RETURN creates giant UTXOs, but we say, well, they're unspendable, so we ignore those UTXOs. That was kind of the fix for getting an infinitely large UTXO set. Because let's say there was a world where... OP_RETURN ish things existed but they could not be removed so people would be putting their pictures on the blockchain right into the UTXO set

Aaron van Wirdum Yeah, I don't want that. But however but now…

Sjors Provoost Yeah, okay now people are doing that using OP_RETURN but…

Aaron van Wirdum No, they're not. Well, they you're not referring to…

Sjors Provoost Let's say there is no discount then they would probably be doing it using OP_RETURN Now, you know, this 80-byte limit on OP_RETURN is just in the mempool, so they would probably lobby for, you know, just increase those mempool limits or just send it straight to a miner. But there still wouldn't be anything bad. But that's only because they're using OP_RETURN and not using something else. Because what if they go full, say, Mike in space and say, hey, wait a minute, this UTXO set can still get pruned. What if we just create a not really OP_RETURN? We change it slightly so that it's really... The blockchain doesn't know that it's unspendable. And we still pay the same fee because whether you use OP_RETURN or put actual spam in the UTXO set, there's no price difference. So you could do op, not really return, and then put four megabytes in a transaction. And now you've got this four megabytes sitting in the UTXO set. So that's pretty bad. So in that sense, it's good that the way inscriptions do it is four times cheaper. because they're putting their spam they have a 4x incentive to put their spam in a place where it does not blow up the UTXO set.

Aaron van Wirdum Okay, let me rephrase the question then and okay what if we removed the OP_RETURN limit or the OP_RETURN limit was never there?

Sjors Provoost Well, it's not really there right so let's

Aaron van Wirdum It's not there. Oh, you mean because it's a policy.

Sjors Provoost It's just a policy rule. So anybody can run a node that has no OP_RETURN limit. And in fact, there are some pull requests that suggest like, hey, just drop the OP_RETURN limit.

Aaron van Wirdum Yeah, which I agree with, I think. But that's a side quest. Well, or maybe it's not. I guess it's actually my question. So do you think the motivations for SegWit and the discount... hold up for inscriptions versus if the OP_RETURN would have no limits like would it still be beneficial would we still want to incentivize people putting it in witness rather than OP_RETURN

Sjors Provoost No, I I would imagine that that doesn't make a difference I guess what I'm worried about is the op not really return scenario.

Aaron van Wirdum Well, okay yeah I mean that's Mike in space doing Mike in space that's a completely different story right? Maybe. If the OP_RETURN is just OP_RETURN and there is no limit, then you're saying then it doesn't matter whether it's in the witness or in the OP_RETURN.

Sjors Provoost Yeah, if the choice is between OP_RETURN and the witness and there was no discount, then it didn't matter.

Aaron van Wirdum Right, but...

Sjors Provoost Also not for the health of the blockchain.

Aaron van Wirdum Yeah, but there is a discount. So wouldn't it be better if... I'm getting lost in my own argument. No, I think I'm still following myself. Wouldn't it be fairer if there was no discount and now that was either the OP_RETURN or the witness? Instead of now definitely people are going to use the witness with a discount, which means that the inscriptions are cheaper than transactions. Are you still following me?

Sjors Provoost I think I'm still following me,

Aaron van Wirdum but I'm not sure anymore.

Sjors Provoost I don't think that's good. And now we're talking about just the case of I'm downloading the old blockchain. So I'm a prune node and I arrived to Bitcoin for the first time. Now I'm downloading the old blockchain. And in your no discount scenario, where somehow we increased the block size to four megabytes as a hard fork, I'm now downloading four megabyte blocks. Whereas in the current world, I can download one megabyte block.

Aaron van Wirdum With real transactions.

Sjors Provoost Yeah, because these OP_RETURN transactions, you do have to download them. You can't skip that part. With the Ordinals, I can skip it if I'm a new node catching up. So there is a benefit to shoving things in your witness.

Aaron van Wirdum Why couldn't you skip OP_RETURNs?

Sjors Provoost Because in order to check what is in a block, you need to have everything to check the hash, basically.

Aaron van Wirdum You already said that. You're right.

Sjors Provoost Otherwise, I think you could have a scenario where somebody would give you the header of a block and the header would have the proof of work. And you're like, yeah, this is a great block. It has all the proof of work. Then they give you the contents of the block and they say, yeah, trust me, this belongs to that header. And that doesn't work unless you completely restructure the block internally so that you can have like Merkle trees of parts of the block. But that's really what SegWit does. essentially puts some of the stuff on somewhere else.

Aaron van Wirdum Okay, so you support the witness discount even today with today's knowledge that people are using it to upload [unclear]

Sjors Provoost I support it in the sense that given that we increased the block size to four megabytes worst case or even if we had only increased it to two megabytes this is a better way to do it.

Aaron van Wirdum You still think that the incentives work out this is how the incentives are supposed to be used?

Sjors Provoost Yeah, I mean the alternative was either we stick to one megabyte so we have SegWit but it's a one megabyte limit then it doesn't matter you don't need a discount but if we do want to increase the block size then it's nice that that all that extra stuff goes to a safe place yeah so I still think that's the right call.

Aaron van Wirdum Okay, I'm gonna think about this and I hope at least

Sjors Provoost Of course we didn't know that.

Aaron van Wirdum at least three listeners could keep up with the last five minutes of rambling Sjors, is there anything else we need to discuss which we haven't yet about the discount, about the way this discounts, or did that wrap it up more or less?

Sjors Provoost No, I think we've got it all. There's some good videos that you can watch, I guess. But some of this is really a matter of reconstructing. I think if you look at the mailing list posts from that time, they did not go... into as much, I think it didn't go as much into as much depth as we just did.

Aaron van Wirdum Yeah, definitely.

Sjors Provoost It seems to me that some of these decisions were correct. For example, the idea of having a single metric. But if you read the SegWit BIP and especially the original version of the SegWit BIP, it doesn't even explain, I think it doesn't even explain this need to have a single market.

Aaron van Wirdum That was my impression as well. I mean, I wasn't in every conversation, especially not, you know, they were probably...

Sjors Provoost Now, it is said in the presentation that sipa gave, I think either in Hong Kong or the follow-up a few days later in San Francisco, where he does explicitly point out this problem of having one market. And there's some discussion about it on the mailing list. But I think nowadays maybe you'd want to have a bit more elaboration on that in a BIP. Because for all you know, somebody comes along with a similar idea and just implements it incorrectly, forgets about this fee market problem. Now we could have had SegWit with a double super complicated fee market and it would have been very difficult to fix it if we didn't have somebody smart enough to do it correctly in the first place.

Aaron van Wirdum Yeah, I was just gonna say my impression at the time was that four megabyte was picked because people wanted the block size increase and four megabytes seemed about right and over time as the debate was very heated sort of arguments really only started to crystal crystallize that was my impression but maybe you know As you mentioned, there was probably some thought even before I realized that.

Sjors Provoost Probably, but of course we don't know what was happening. We don't know everything that was happening before these presentations were given, right?

Aaron van Wirdum Right.

Sjors Provoost The heat around the block size wars was started in 2015 or so, whereas this proposal was at the end of 2015.

Aaron van Wirdum Yeah, but it continued and then people started attacking SegWit for all kinds of weird reasons from all directions.

Sjors Provoost Yeah, and I guess part of that might be because some of these things that we just talked about today were not as clearly available. You could find out if you watched the technical presentations. went all the way to the Q&A and went into the mailing list discussions, but there was no Bitcoin Explained podcast at the time.

Aaron van Wirdum True.

Sjors Provoost Summing up these reasons in a super clear way. And that is why we thank you for listening to Bitcoin

Aaron van Wirdum Explained.