Episode 48: Lightning Network Upgrades (Eltoo + SIGHASH_ANYPREVOUT) with Christian Decker

Sjors Provoost Live from Zurich, this is Bitcoin Explained. And with me today is not Aaron, again, it's Christian Decker.

Christian Decker Hey, thank you for having me.

Sjors Provoost Well, but nice, this small city, you're just here and you meet all these people. Yeah, so we're actually going to do sort of a retake of episode 35, in which we talked about SIGHASH ANYPREVOUT and eltoo. So if you want to prepare for this episode, you could listen to it, but you don't have to, because we're just going to explain it again.

Christian Decker It's an excellent episode, by the way, so go listen to it.

Sjors Provoost Okay, but I figured it's good if we have the expert and also the latest state of things. Yeah, before the episode, we were trying to figure out what we want to do first. If we want to talk about eltoo first, or if we want to talk about SIGHASH ANYPREVOUT, without even explaining what eltoo is. And my guess is the listener will be more motivated to understand SIGHASH ANYPREVOUT if they know what it's useful for. At least that tends to motivate me. So maybe, is that a good idea?

Christian Decker Yeah, absolutely. SIGHASH_ANYPREVOUT, is basically what SIGHASH NOINPUT became after a couple of iterations of improving it. And SIGHASH NOINPUT was actually proposed by Joseph and Taj in the original lightning paper.

Sjors Provoost I did not know that.

Christian Decker And all I really did was pick up the idea and... basically run with it. And I wrote the BIP proposal, which then AJ wrote up. And it turns out that it enables quite a few interesting use cases that we currently cannot do on Bitcoin itself. And a whole bunch of other improvements were later proposed as alternatives or building on top of SIGHASH NOINPUT. The basic idea of eltoo is basically that in Lightning we have this mechanism where we penalize misbehavior. And we do so quite heavily. So if I have a channel with you and then I basically go back, then I give you the opportunity of stealing the entirety of my funds.

Sjors Provoost Yeah.

Christian Decker And so that has a whole bunch of implications. So for example, it makes it really hard for me to create backups and... and restore, because restoring basically means that I would go back in our history of agreements and I would then inadvertently cheat.

Sjors Provoost Right, because restoring is indistinguishable from cheating, right?

Christian Decker Exactly.

Sjors Provoost So maybe a quick rehash of the Lightning channels, right? So what you do is you put coins into a shared address, so to speak, in a multisig address once, and then you also have a transaction that takes it out in the same distribution as how it went in. And then that transaction that takes it out, you're not putting that on the blockchain. You're holding on to that. And you're updating that between the two parties so that I can shift some money to you and you can shift some money to me. And every time we do that, we create a new transaction, which is a real Bitcoin transaction. But the fundamental problem that was very hard to solve is that, well, they're all real transactions. So how do you, the blockchain has no preference about which one it is. And I think Satoshi first tried to deal with that by having sequence numbers and just saying, well, the most recent sequence number is the one that's valid. But it turns out that miners can just ignore the sequence number. And also it requires you to be really quickly, I think, because if... So you have the mempool where somebody cheats, they put something in the mempool, and then it goes into a block, unless somebody else broadcasts a competing transaction that would go into the block. So if I were to cheat and you were to try and send the real one, but my transaction is confirmed in two seconds because I got lucky, then I just won. And you cannot undo what's already in a block. That would be very expensive.

Christian Decker Exactly. So, yeah, the original nSequence scheme already contains the basic idea, right? Quickly iterating over what a transaction looks like and renegotiating what the distribution of funds is at the end of the lifetime of our channels. and basically having later versions override what prior versions were doing. With nSequence, it was replacing the entire transaction. And that turned out to be hard, not just from the user perspective, but there also is no way for us to prove that a miner... ignored a later sequence number because there is no sort of global knowledge that a later sequence number was even available when the miner mined their blocks. And so it's really hard to prove anything when you work with nSequence numbers. So miners could get incentivized to actually renege on this social contract of always using the latest sequence numbers. By basically me approaching them and saying, telling them, hey, I give you like 10% of whatever I'm about to steal from this guy. If instead you confirm an older version, right?

Sjors Provoost Yeah.

Christian Decker And the way that we later solve this in Lightning Penalty is exactly by basically allowing the settlement to be sort of split in two parts, right? There is the initial settlement where we send one of our agreements where we don't know yet whether it's the latest one or not to the blockchain. We allow it to be confirmed. And then we basically have a period where we allow the counterparty to react if we misbehaved, to basically present proof of that misbehavior to the blockchain, which in our case acts a bit like a judge. And when we present proof of this misbehavior, then we are allowed by the blockchain, the judge, to actually grab the funds from our counterparty.

Sjors Provoost Yeah, which is, you know, I still think that's a pretty cool mechanism, at least to solve this particular problem. But it turns out there is a better way.

Christian Decker Yeah, so, well, it's a different way. I don't know if it's strictly better, but I like it a bit more because it's sort of, it feels more correct to me. And the idea here is instead of having basically you misbehave and I punish you, I can basically go and present proof that there is a later state. There is a later agreement that we had and I can basically... present that to the judge, and then instead of the old revoked one, we are now working with the newer one.

Sjors Provoost Right, so instead of there's no punishment there, you're just updating, you're just putting up a newer version.

Christian Decker Exactly, so my basically correction is actually rectifying the misbehavior or the disagreement that we had, and we actually... move forwards towards the latest uh agreement that we had

Sjors Provoost right and and you can skip any of the intermediate steps too right all you need to know is that it's more recent

Christian Decker exactly and the the skipping part here is is the key to this whole puzzle where the ANYPREVOUT and NOINPUT and all of these proposals now come in um and uh that that allow us to make all of this basically really efficient

Sjors Provoost Right, because in the original protocol, you have to, like this punishment transaction, you have to remember it for every previous state. Or not the whole transaction, but some sort of punishment secret for every previous state, even if you've done like a million transactions on your channel. You need to remember a million different ways that you can punish your opponent if they broadcast any of those earlier ones. And now with this magic that we'll explain in a bit, you don't. You just need to make sure you have the most recent one or a fairly recent one. in case of a backup like of course if it's not the most recent one it might not completely work to your advantage but it's okay if you lose some and it's okay if you broadcast one that's not super old so you don't get punished And you don't have to hold on to a whole bunch of state very carefully.

Christian Decker Exactly. So the issue is that if we want to prove to the judge that a specific agreement was wrong, we have to have a tailor-made basically proof for that being wrong. And that is where the entire sort of idea comes from of having to remember every single revoked state in eternity until the channel is closed. that is a major downside of lightning penalty because you suddenly have to be able to produce those tailor-made proofs that my counterparty misbehaved on a whim and I have to have them ready and I might even have to sort of outsource those if I myself cannot be present all the time. So if, for example, you've run a watchtower, then you would actually have to remember a linear amount of data for the channel lifetime. Whereas with eltoo, what we have is basically we have this wildcard where a later state that we agreed upon can be attached to any of the prior ones, sort of directly jumping to the conclusion of what our current state is. But at the same time, it also doesn't mean to be the last state. So we could, for example, initiate a close with eltoo, but keep updating the channel until a different timeout runs out, or we basically don't want to speak to each other again, then we can just stop making updates. And it all just flows naturally onto the blockchain and gets all resolved without any issues.

Sjors Provoost And the magic thing that kind of makes that possible is this proposed soft fork, I think, called SIGHASH_ANYPREVOUT. And if I understand it correctly, like, well, it's good to illustrate that normally with Bitcoin transactions, there's two things you're doing. You're saying I'm spending this previous transaction and you're saying, okay, here's the solution to the script of the previous transaction. So there's really two keys that you need to provide. And that's still true. But your signature, like normally your signature covers all those things. So if you've made a signature, you cannot then apply that to any other transaction. But with this proposal, the signature does not cover the link to the previous transaction. It just covers the script that you're doing. So you're kind of creating a signed Lego puzzle piece that you can put onto any other Lego puzzle piece. Well, any compatible Lego puzzle piece. Is that a good way to say it?

Christian Decker Exactly. So in effect, what we do in the classical signatures, in a classical transaction scheme in Bitcoin, is we have this double linking between... output being spent and the transaction spending it and that is indeed done by basically pointing to where the funds come from and then providing a script that matches exactly the conditions that were set up when creating those outputs. So usually that would be in order to be spent I need a signature from pubkey xyz and once you create a signature that is specific to the output that you are spending. And what SIGHASH ANYPREVOUT does is it basically removes that tight coupling between signature and the specific funds that you're spending. But since the script is still there, you now rely on the pattern of the script still being compatible with the script that is spending it. but you can replace, you can swap out the actual funds that are being referenced in that. And so what happens is exactly as you point out, is now that we have this sort of generic pin where we can attach other Lego bricks to, Lego bricks in this case being the transactions, I like that similarity. I will use that in future as well. And by doing so, we can actually have the very nice properties that eltoo gives you, namely that we can skip intermediate states.

Sjors Provoost Yeah. Right, you can skip them without having to have, yeah, basically the one transaction per thing that you can connect to.

Christian Decker Exactly.

Sjors Provoost So it might be fun to go into the nitty-gritty of what that script actually looks like. So if I understand it correctly, the script is saying I can spend anything that has a sequence number above... 25 or something like that. Is that right?

Christian Decker Yeah, so what we create basically with eltoo is a very long chain where each individual agreement is represented by what's called an update transaction. So every time we come to an agreement about what our next state should look like, for example, I'm sending you one Bitcoin, so your balance increases by one, mine decreases by one. We now attach a new update transaction to this very long chain of update transactions now if we were to basically just use that structure then we then we would have to replay all of these update transactions on chain when settling because from state zero we go to one to two to three to four and five so we wouldn't we wouldn't have earned anything, right?

Sjors Provoost No, that defeats the whole purpose of doing things off-chain.

Christian Decker Exactly. And so what we want to do is instead skip these intermediate transactions. And so each of the outputs that we create from those update transactions actually has a state number in it. So the first one is 0, 1, 2, 3, 4, 5. And what we do then is basically any update can be attached to any prior update. And so what we have in the transaction spending the output... we have a check that basically the sequence number that we are attaching to is smaller than our current state number. And that is basically the area so that I can't basically go back in time and apply something newer or older than what I'm supposed to. So you can't have a sequence like we're in state 100 and then I go to state 90 and since... all of those outputs look the same, I could theoretically take 90 and attach it to 100, thus going back 10 steps in our history.

Sjors Provoost Yeah, that'd be quite annoying, especially because then you could go back and forth.

Christian Decker Exactly. We could have this infinite chain of going back and forth and never actually make any progress and settle things. We would just be spending loads of coins in on-chain fees because, well, I do something and then you have a favorite state where you had more money so you are applying that one and then I go back to my state where I had the most money. And so we have to make sure that there is this constant progression forward, and that's exactly where the state numbers come in, because we then can make sure that we can only attach later states to newer states, ensuring that eventually we get to the last one that we agreed upon, and that will eventually settle as well.

Sjors Provoost Right, and the state number is just the, I think, the nSequence field that is part of a transaction, right? So, where is it?

Christian Decker It's a bit more complex.

Sjors Provoost Or is it specified in the output script?

Christian Decker So in the output script, we can put the number directly because that is actually covered by the signature that created that transaction. For the input where we say, okay, I am actually state 100. I can attach to anything before 100. That would have to be in the input script. The problem is the input script containing the signature itself cannot be covered by the signature. Otherwise, we'd have this circular reference where I am trying to create an input script containing a signature which has to sign the input script. So that doesn't really work. And so we use a little trick where we say, okay, what parts of the transactions are actually covered by the signature? so that we don't have to put it into the script itself. And it turns out the nLockTime is. So the nLockTime is part of the transaction where we have a number which is covered by the signature. So any person cannot really modify that. And we can, with OP_CHECKLOCKTIMEVERIFY, we can do comparisons between numbers. And so what we do is basically state 100 as nLockTime 100, which is far, far, far in the past. So it doesn't really have its effect as locking for a certain time.

Sjors Provoost Right, because it's like 1970.

Christian Decker Exactly. But what we can do is, executing CHECKLOCKTIMEVERIFY will pull that onto the stack, and we can perform a comparison between two numbers. without anybody being able to unilaterally change those numbers. And that's how we enforce the ordering of transactions in all of this.

Sjors Provoost Right, so one Lego block that represents the update state, I guess what it contains is, well, obviously the input doesn't have an amount, right? It just points to any previous version.

Christian Decker Exactly.

Sjors Provoost And then the output says, okay, this is the new amount.

Christian Decker Yeah.

Sjors Provoost And the minimum version.

Christian Decker There is a bit of trickery going on here, of course, because if we were to pay any fees in these updates, we suddenly would have to ever decreasing amounts that are available to the outputs. And so even being able to skip the intermediate steps, we would slowly leak capacity of our channels. And so we have another trick there, which is actually what Rusty came up with. namely that these update transactions are single input, single output transactions. And so we can use SIGHASH_SINGLE to just sign that part of the transaction.

Sjors Provoost Right, so SIGHASH_SINGLE, if I understand it correctly, the signature covers the input and, sorry, the first input and the first output, or the nth input and the nth output.

Christian Decker Exactly, the corresponding ones at the same height. Yeah. And it leaves the rest of the transaction basically modifiable. The rest of the...

Sjors Provoost Both the inputs and the outputs are modifiable. You can add more.

Christian Decker Exactly. So you can add more. Particularly you can add fees. The update transactions themselves don't actually pay any fees, but we can modify them such that they can pay fees. And this is quite nice in practice, right? Because the fee weather is hard to predict.

Sjors Provoost Oh, yes. One of the complications in Lightning is that you have to constantly agree on what fee to put on all these transactions that may not be broadcast for several months. And you have no idea what the fee weather is going to be in several months. So then you need to... basically agree on what you think the fee weather is going to be over its lifetime and i remember at least in the early days lightning channels would just spontaneously close because lightning nodes could not agree on what the fee weather would be in the future and so they would say well if we can't agree on what the future is going to bring then we should just close this channel right now

Christian Decker exactly

Sjors Provoost and that has been partially mitigated with all sorts of fancy things like anchoring outputs i think

Christian Decker Yeah, so the anchor output is basically the idea of bring your own fees, backport it to the LN penalty mechanism.

Sjors Provoost Why can't it use the same single, sorry, what was it, SIGHASH_SINGLE solution?

Christian Decker SIGHASH_SINGLE, yeah. The issue is that... That basically requires you to have single input, single output transactions. And so it might work for HTLC transactions, but not really. And in particular, it doesn't work for commitment transactions because while you have a single input... the commitment transaction represents the full state of the agreement. And so it might have two outputs, one for you and one for me, plus any HTLC that we attach to it. Whereas the update in eltoo is really just there to ratchet forward and make sure that we are making progress in time. And the actual state is then represented by a settlement transaction that is separate from the updates.

Sjors Provoost Okay, yeah. All right. So this sounds great. What needs to happen? This is a soft fork, right?

Christian Decker Yeah.

Sjors Provoost So it's a soft fork on top of Taproot now?

Christian Decker Exactly. So it has always been a soft fork, though when I wrote the first version, back then still called NOINPUT, it was basically, it was not integrated with Taproot because back then we didn't know how long it would take to activate Taproot. Turns out we were really quick with that. And so AJ Towns came along and basically took the idea of no input, renamed it anyprevout, and integrated it closely with Taproot. So there is now a couple of features that are really nice that weren't present in the original proposal. Namely, you now have two sighash flags, which can turn on or off different parts of the transactions being covered by a signature. And so there is a lot of flexibility that we get from this soft fork. And we already have a couple of proposals of systems that we could build with that.

Sjors Provoost Cool. Well, anything else you want to add to this brief explanation? Do you want to go into controversies or do you want to just stick to the explanation and we'll see how it goes?

Christian Decker Yeah, I think in my mind, it is a much cleaner way to build off-chain contracts in general because it is a very clean update mechanism where we can just say, okay, we had an agreement, now we have a new one, and we invalidate the old one by basically overriding it or providing the ability to override when it comes to it. Lightning penalties might be attractive because it sort of has strong punishments for misbehavior. And so that's strong incentives to be well behaved. However... everybody likes to penalize until they're the ones being penalized, right? And so we often focus so much more on punishing the bad guys rather than helping people who are really just, well, they're trying to restore from a backup, which is already quite traumatic. And then we make them risk losing their funds as well.

Sjors Provoost Yeah. Well, it sounds like it makes, in general, this change makes it a lot simpler. Like, also, you don't have double, like, normally when you have an update, I have a transaction and you have a transaction that are sort of each other's mirror image. And that goes away. Like, there's just one transaction. There's not a mirror of transactions. Yeah, you only need to store one piece of information, namely the latest state. And it's okay if you don't, yeah, exactly. So it sounds a lot simpler. The downside could be that this... Maybe this punishment is needed. Now, I do know, like, you know, I've worked on this forkmonitor.info website, and one of the things it tracks is penalty transactions on the Lightning Network. And they're very rare, and they're usually quite trivial amounts. So that, to me, suggests that the mechanism really has not ever been used. for serious retaliation?

Christian Decker Well, it might be because it's a good threat, right?

Sjors Provoost Yeah, so that's the question then. Is it because nobody wants to attack or because it's really... I mean, one, it could be very good protection indeed. It could also just be that we are now in a happy kumbaya state of things. You know, and maybe one day, you know, Bitcoin wants to assume that maybe one day we won't be in a kumbaya state and people will be trying to attack or at least, you know, either drain the resources of a node or otherwise misbehave. But my understanding is you could, in principle, still add a penalty mechanism on top of eltoo.

Christian Decker Yeah, so I totally agree. We definitely shouldn't base our security on the assumption that everybody's well-behaved, because then, I mean, what's the point? We do have a tendency to be very sort of defensive thinkers, and I think that's when it comes to money and somebody's livelihood, that's probably the best way to approach it. It's not totally free to misbehave in eltoo either. Like I mentioned before, you actually have to bring your own fees whenever you perform any on-chain action. It's up to you to decide how much fees to attach and what your sort of time horizon is, and that will inform how many fees you attach. But you are the one that has to attach fees. So every time you try to do something, in eltoo that might have consequences, like going back in the history of our channel, you will have to bring fees with you and you will spend those fees to get your state started on the blockchain. But if it's not the latest state, you are basically already giving your opponent the opportunity to react to it. And what you've just done is waste a couple of fees and waste a couple of bytes in the blockchain for which the fee paid, right? And so you should always be incentivized to only ever produce the last state. And while not as deadly as LN penalty, where you lose all of your funds, a fee can still hurt, especially when we have congested mempools.

Sjors Provoost Yeah, and unless you really know for sure that your opponent is offline, you probably are wasting that fee. If you're just arbitrarily trying this attack on everybody you know, it's kind of wasteful because you have to pay a fee every time you try to attack.

Christian Decker Exactly. And I mean, since we make retaliation really, really cheap by basically making watchtowers constant memory. instead of this ever-increasing set of transactions a watchtower has to remember, we make it more likely that you might have a watchtower. And maybe that threat alone is enough to make it not worthwhile trying to even attempt a cheat. So, yeah. It's basically, I always compare it to being decapitated outright, which is LN penalty, versus death by a thousand cuts, which is... smaller, less hurtful, but it will still be deadly if you do it often enough.

Sjors Provoost Yeah, that sounds about right. Okay. Okay, when is this going to launch?

Christian Decker Two weeks, of course.

Sjors Provoost Two weeks. All right, that sounds good. I think we can wrap it up with that. Do you want people to reach you or do you want people to leave you alone?

Christian Decker Definitely reach me. I'm S-N-Y-K-E on Twitter, Snyke. and cdecker almost everywhere else.

Sjors Provoost And you hang out on various mailing lists?

Christian Decker All of them. Though I, like many others, keep lagging on reading those mailing lists. But if something catches my eyes, I will definitely jump in there and reply.

Sjors Provoost Okay, because where would you say the development is happening now? Is it just updating the BIP or arguing about the BIP?

Christian Decker So I mostly spend my time on the Lightning specification. So that's the Lightning mailing list, the GitHub for the Lightning spec, and of course our various implementations. And the anyprevout discussion is mostly happening on the BIP itself. So BIP 118. And of course, if you have an idea and just want to bounce that idea off of people, the mailing list is always a good place to sort of learn about prior art or get feedback about how to structure it better.

Sjors Provoost This is the Bitcoin Dev mailing list.

Christian Decker The Bitcoin Dev mailing list, yeah.

Sjors Provoost All right. Well then, thanks for joining me. And thank you for listening to Bitcoin Explained.

Christian Decker Thank you.