Episode 47: Lightning Network Channel Payments with Rene Pickhardt

Sjors Provoost Live from Zurich, this is Bitcoin Explained, formerly known as the Van Wirdum Sjorsnado. And we have with us today, no Aaron, but we have Rene Pickhardt.

Rene Pickhardt Hello, nice meeting you.

Sjors Provoost Nice to meet you too.

Rene Pickhardt Thanks for having me.

Sjors Provoost You're welcome. So we were chatting yesterday about something you're working on.

Rene Pickhardt You mean optimally and reliable payment flows on the Lightning Network.

Sjors Provoost That is right, yes. Yeah, tell me about it. I guess that would be one way to get started. I mean, maybe we can do a little bit of background. We've talked about Lightning before on the podcast. We've talked a little bit even about routing. So how to get a payment from A to B with Joost Jager. And you're working at kind of a different approach of doing it, I would say.

Rene Pickhardt Yeah, you could say so. So in general, if you look at the Lightning Network, you have the... design principle that you don't know how much liquidity a different node has. So if you want to send a payment from A to B, what will happen is you select some route through the network, you send out an onion, And at some node, the Onion arrives and they might not be able to forward the Onion because they just don't have enough liquidity. But you don't know as a sender, you don't know this ahead of time.

Sjors Provoost Yeah, so maybe we should go a little bit back to the basics of what we just said. So every Lightning channel has a certain amount of Bitcoin in it. But the only thing that's publicly announced by Nodes is how much... there is in the channel in total, but it's like a little abacus, right? Where some of the beads are on one side of the channel and some of the coins are on the other side of the channel. And what is not communicated publicly is how much coin is on each side of the channel for two reasons. One, it's just too much noise to keep updating that. And the other is it's just bad for privacy because then somebody looking at the network could say, oh, I just saw 20 Satoshi move from that channel to that channel to that channel. So in order to prevent that, it's all hidden. But it means that when you're trying to send a payment, you don't really know if it's going to fit. Because for all the channels on the route that you picked, you don't know where these points are.

Rene Pickhardt Exactly.

Sjors Provoost Like what the available balance is. So you kind of have to guess. You have to say, well, I'm going to guess it's about 50-50. Half the coins on one side, half the coins on the other side. And so that's a problem.

Rene Pickhardt Yeah, that's certainly a strategy you could do. But what this means in general is that the Lightning Network as a payment system is unreliable. You can't be sure that if you try to make a payment that it will arrive for certain. There's a certain chance that the payment will fail. So I was for the last two years basically looking at these problems and studying the reliability of the Lightning Network or the reliability of the payment process. And if you look into the scientific literature, you actually find a lot of papers that... already observed that it's unreliable. They all talk about success probabilities, right? So what they do is they make payments of certain sizes and they realize the smaller a payment is, the more likely it is to go through, which intuitively is very... clear right because

Sjors Provoost and this is literature about lightning or yeah scientific

Rene Pickhardt literature where they study payment channel networks the lightning network in particular these things right

Sjors Provoost well it's good to know that there actually is scientific literature nowadays covering yeah

Rene Pickhardt there's actually a lot um if there wasn't the um covid pandemic i was actually planning to do a scientific workshop and invite people from various fields to come together so that we could actually form as a scientific community because there's actually a lot of research going on But right now it's basically going into different fields and people don't really talk to each other. I mean, they cite each other and probably they know each other. But from a pure academic perspective, there is not that much. community building happened yet. And I think we are at the point where this actually really needs to happen. But since the pandemic was there, I didn't organize the workshop. But it's coming, hopefully.

Sjors Provoost It's a slight delay. This is a multi-decade project.

Rene Pickhardt A slight delay, yes. So, yeah, but if you look into the literature, they all describe the phenomenon. And as I just said, a smaller payment has a higher chance to go through, which intuitively is very clear, right? Because you use this Abacus model, right, where you have a certain number of beads to be on one side. Well, let's say I can make a payment of, say, 100,000 satoshis, but it's very clear that I can also make the payment, or could have also made a payment of 90,000 satoshis. It's not clear that I could have made a payment of 110,000 satoshis, right? Because on that side of the abacus, there might have been only just 105,000 beads, right? So in that case, it doesn't work. So these things were well known in the literature, but... I was basically the first person who turned this around and said, you know what, if you want to select a path, why don't you start with the success probability of a payment and you try to find the path that has the highest success probability for you to deliver the payment. So you basically compute a probability for every channel in the network and assume how likely is it that I can send this amount in this direction.

Sjors Provoost Right. And this likeliness is just, oh, how would you do that?

Rene Pickhardt Oh, so it basically depends on the amount that you want to send and the capacity of the channel. I mean, there is obviously, if you're a mathematician, an infinite amount of ways to derive a probability. But we just use a very simple model of basically saying we assume every distribution has the same likelihood. So this would mean that if you want to... If you want to send one Satoshi, the likelihood...

Sjors Provoost I think the way you explained it yesterday, if I remember correctly, is you don't know how much is in there. Could be one Satoshi, could be 10 Satoshi, right? If the channel capacity is 10. So you're going to say it's equal odds. So it could be one Satoshi, could be 10 Satoshi. We're going to say these odds are all the same.

Rene Pickhardt Yes, it's like rolling a dice.

Sjors Provoost Exactly. And therefore, if you wanted to guess, you have a probability density function, I guess is what you described it.

Rene Pickhardt So what this means is the larger a channel is, the higher the chance is to get your payment through, right? So in that sense, larger channels have a higher chance to provide the liquidity that you need for a certain amount. Right. They become very attractive.

Sjors Provoost Okay. And then I guess you extrapolate this observation to… To multiple channels?

Rene Pickhardt Yes, so I do this for every channel in particular, right? And then I can basically solve the shortest path problem the same way how nodes currently do this. But currently nodes try to optimize for the total fees that they pay. But what I said is, hey, we could optimize for the likelihood, right? So you find the path that is actually the most likely path. So taking all those probabilities into consideration. So yeah, we published that in March this year. And I had the feeling, even though we did some simulations and provided some strong evidence that pathfinding is getting much better. people didn't really pick this up too heavily.

Sjors Provoost They didn't pay attention to your publication?

Rene Pickhardt Well, I shared it on the lightning dev mailing list and I got one reply which was basically a clarification question. But I didn't see implementations picking it up too heavily. And there was an open question in the paper and also in the mailing list post where I said there's actually a question to be answered, which is how do we actually want to split payments, right? Because if you look at this probabilistic view, what you see is if the… As we just discussed, if the amount gets smaller, the payment is more likely. But the other thing that you see is the more channels you involve into a payment, your likelihood also decreases and it decreases exponentially.

Sjors Provoost Your likelihood of success.

Rene Pickhardt Your likelihood of success, yes.

Sjors Provoost Well, and that's not because of the capacity. That's because of the reliability, right?

Rene Pickhardt Yeah, that's because of the reliability. That's because, I mean, every time you have a certain success probability for every channel, and in order for the entire payment to be successful, it has to be successful at every channel. And this just means that you multiply all these probabilities together.

Sjors Provoost Right. Okay, so we're still talking about the probability of it fitting, basically. So if you have decided to, you know, you can send a payment through... a single line of hops essentially yeah or you can split it up as a multi it's called multi-path um through multiple routes to the final destination you split the amount between them but the trade-off there is the more channels that are involved the more likely it is that one of those will not have the capacity that you thought it would have that is that is the sort of the easy form of failure

Rene Pickhardt so so so the observation is So if you send the payment across one line, you have a certain success probability for the entire payment to arrive. If you split it into, let's say, two paths, well, each of these paths probably has a higher success probability. But the probability for both of those paths to be likely might be lower than actually just sending this payment across one line. So there was this question of... How do I split and when do I split? So in this original paper, we already tackled this, but we assumed that we split the payment into equal amounts. And we could actually provide a formula in saying, look, if the payment has a certain size, you want to start splitting the payment. But we already knew from looking at the graph, it doesn't make sense to split the payment in equal size chunks. And if you, again, look at current implementations, they try to send the payment. And if they discover it's too large, if it fails, they start splitting it, but they just split it into two halves or they split it in more chunks, but equal size, right? So this is kind of like...

Sjors Provoost obviously non-optimal.

Rene Pickhardt That is something that I observed, yes.

Sjors Provoost Oh yeah, it could potentially be better anyway. So I guess the... Yeah, there's two questions that you want to answer, right? Should you split the payment at all? And if you split it, how should you split it? And it's not obvious that the naive solution of just try to do it in one go unless it fits and then otherwise splitting it equally, that's not necessarily the best solution.

Rene Pickhardt Yeah. Yeah, exactly.

Sjors Provoost So you proceeded with that in your research?

Rene Pickhardt Yeah, so I actually started looking into this question more thoroughly in January this year. And while publishing the other paper, I started talking more and more with Stefan Richter, who is also from Germany. And he was actually listening very carefully. And at some point in time, he was basically pointing out, Rene, you are reinventing the wheel. The problem that you're solving is a min-cost flow problem. Here's the literature. Look into this. Solutions are actually well known to this.

Sjors Provoost Whose problem?

Rene Pickhardt It's a min-cost flow problem.

Sjors Provoost Oh, a min-cost flow. Yeah, I thought that was a Russian, like a min-cost flow. No, sorry. Okay, minimum cost flow. So basically, you have a flow of funds from A to B.

Rene Pickhardt Yes.

Sjors Provoost And there's a certain cost associated with that.

Rene Pickhardt To transport the funds at every edge, yes.

Sjors Provoost Yeah, and this cost could be, I guess, what is that in Lightning? Satoshis, right?

Rene Pickhardt Yeah, so the cost could be, for example, the fees that you have to pay to use a channel. which is very natural because you also want to have a cheap payment. But you could also translate the probability into a cost and you could just optimize for the probability. So what we're basically saying is just in general, if you want to make a payment, it's actually a transportation problem. Many people talk, when they talk routing, they talk about pathfinding problems. But what we are saying, it's a transportation problem. We have a certain supply of Satoshis. That's me, the sender. If I want to send them to you, it's yours. You have a demand for them. You issued an invoice, right? So now there's the question, can I transport those Satoshis through the network, right? And this is an extremely well-studied problem from logistics for the last 50 years. There's many people in English language, it's called operations research, because I think it originated from military questions. And there are algorithms to solve this, right?

Sjors Provoost That's great news. We should clarify a little bit what kind of costs we're dealing with in Lightning. Because we talked about the, well, first of all, each channel will have a certain fee rate that's set by the node, how much you're charging. And they also have a base fee, but we'll get into that later.

Rene Pickhardt Sure.

Sjors Provoost Without wanting to dox your clothing. But there's also what we talked about, the risk of simple failure as in not enough capacity. And you'll simply, in the best case, when there's not enough capacity, you very quickly get a reply back saying, hey, there's not enough capacity. So you want to minimize the odds of that happening in general. But a very important thing is that there's another kind of failure, which is the my payment is stuck failure. And in Lightning that is like the horror scenario where you won't actually really know that your payment is stuck. because you're not getting a message back. But there can be two reasons you're not getting a message back. One is it did succeed, and the other side is waiting for all the parts to combine. So if you're splitting a payment up, as soon as the other side gets all the parts, then it says, okay, done. So you get a whole bunch of messages back. So that could be the reason why you're still waiting on a reply. The other is that it actually failed and then you have to wait multiple days and close the channel on chain and incur fees and all that type of horror. So

Rene Pickhardt a remote channel could have failed but a remote channel could just decide not to forward it because it acts maliciously. or it's misconfigured right I mean these are also sources of errors I mean that could also happen yes

Sjors Provoost yeah and so those are things you really want to avoid because they're like very high pain situations but in the model that you're talking about you could just assign some sort of price to them so you could say like I really never want this to happen so I'll put the maximum weight to that not happening and in that case what you want to optimize for is the fewest number of channels that your payment goes through. So that could be an argument to make it a single payment if that is possible. And then the other factors could also push it towards a single payment or a split number of payments. So

Rene Pickhardt what you're basically referring to is the question of how do you assign a cost to the... problem, right? If I want to solve a minimum cost flow, I have to know a cost function, right? And it's very reasonable to take the routing fees, as you mentioned. It's also very reasonable to take the success probabilities, but there might be other features that we include into the cost function. And our research does not claim to know the perfect cost function. We just claim to know here's a cost function that currently works well. And this is the kind of problem you want to solve, right? But there's a lot of research to be done in the future to actually design a better cost function. And depending on your cost function, you would split the payment differently because, I mean, this defines your optimization goal.

Sjors Provoost Exactly. And I think this will bring us to the hairy issue that is on your t-shirt. Because at least if you look at the fees, the way it works in Lightning right now, there is something called the base fee, which is a fixed amount that you pay for every payment that goes through your channel. And there is a fee rate, which is simply a percentage of the amount that's being sent. And notice it's different than what we're used to on the Bitcoin base chain. On the Bitcoin base chain, you're paying per byte that you're sending, independent of the value that you're sending. And there's no setup cost. It's basically per byte. Although in practice, of course, you can't make a transaction that's that small. But on Lightning, the payment is based on the amount. And the reason for that, I guess, one reason for that is because your channel, once all the coins are on one side of the channel, you then have to rebalance it somehow, send those coins back. to the other side or create a whole new channel which has a cost on chain. So you want there to be some incentive to not drain the channel or not to move all the coins to one side. You want to be compensated for that because that's where your actual cost is going to be. Whereas on Bitcoin, the cost is bandwidth for processing transactions. So it makes sense to charge it per byte rather than per value. So

Rene Pickhardt I understand in Bitcoin, it's very clear why you would charge per byte. I mean, that makes a lot of sense to me because the block size is the limiting factor here, right? So you want to basically pay for how much space you use inside the blockchain.

Sjors Provoost Yeah.

Rene Pickhardt Whereas in Lightning, it makes sense to somehow tie this to the amount because as you just said, the channel drains, right? then you can't route anymore. So that makes sense.

Sjors Provoost Yeah, so channel capacity is the bottleneck, or at least that's sort of assumed here.

Rene Pickhardt Yeah. So initially in the protocol, as you mentioned, there was also the base fee, which is just the fixed fee, which you also noted Bitcoin transactions do not have. So they are just linear in the size, whereas Lightning payments are not just linear in the amount, but also have this like fixed cost. And the interesting observation is, I just mentioned, there are well-known algorithms to solve those min-cost flow problems. But it turns out that the computational time that you need very much depends on the structure of the fee function or the cost function. So there are, in general, in mathematics, I mean, there are way more, but three ones that are being studied. There are non-linear functions, there are convex functions, and there are linear functions. And for linear functions, the problem is ridiculously easy to solve. For convex functions, we can do it. And for nonlinear functions, the problem is very complex to solve.

Sjors Provoost Yeah, but I think you mentioned that it's not just that it has to be linear, because a linear function could be just 0 plus A plus BX plus, well, yeah.

Rene Pickhardt AX plus B.

Sjors Provoost Yeah. I guess, yeah, not all exponential, not all like powers of two you can do, right? You can't do like x to the third.

Rene Pickhardt So we're jumping into a nitty point of mathematics that the term linear actually has two semantics in math. One semantics is if you come from a perspective of polynomials.

Sjors Provoost Yeah, I was thinking polynomials.

Rene Pickhardt Then a x plus b would be considered linear. But the way how I talk about linearity is, that in the same way as Schnorr signatures are linear, right? It doesn't matter if you do the addition, right? So if you have a function f of x, you do f of a plus b. It's the same as f of a plus f of b. And this doesn't work with the base fee because now you have to pay the base fee twice if you execute the function twice.

Sjors Provoost So in other words, these super fancy algorithms that exist out there in the literature are able to handle a polynomial where the only thing is x, basically. So this base fee becomes a problem because it's this offset. Basically, if you look at a y-axis and an x-axis... You're looking at a diagonal line that starts at zero. That's what you can handle. If the diagonal line starts at one or two, something, then you can no longer handle it. And, well, then, you know, you can try to sort of ignore that problem by, I don't know, either pretending that this base fee doesn't exist and then just find the best route and filter some of the base fees. Or you can suggest to people in the Lightning Network to get rid of the base fee. And I guess the third solution would be to find this genius mathematician that can actually just solve it with a base fee if people really, really care about the base fee. Yeah. And that is, I guess, some discussion about that.

Rene Pickhardt So that was a surprise for us because we didn't anticipate this discussion. We were very afraid to publish the results. Because in earlier days, there was always this criticism coming from the Bitcoin Cash community that the Lightning routing is like an unsolved problem. And if it would be solved, it would be NP-complete and all these things, right? Well, I mean, now it turns out that with the current goal that nodes have to optimize for just the fees, the problem actually is NP-complete, right? And we were very afraid to actually publish this because we thought we would get a lot of ridicule from this community, but that community didn't care at all. However, we asked before, why was the base fee actually introduced to the Lightning Network Protocol? And Rusty Russell, a Lightning developer who worked at the spec at that time, and I think who actually put it there, said there was no particular reason. At that time, there was just many discussions of how we want to do this. And that was basically a compromise among developers. But there's no security reason. There's no privacy reason. There's no like, it's not desperately needed, right? So we thought, well, that's good then, right? Because now we figure out that if we drop the base fee, and set it to zero, well, then lightning routing becomes actually a very feasible and easy to compute optimization problem. But after we published this, there were actually a lot of people from within the community who provided arguments, which up to this day, I do not fully understand. But there's discussion, yes.

Sjors Provoost Okay, well, it's good to know there's discussion. And I think... We've given a nice outline now of this whole proposal and some of the issues. And you're wearing a nice shirt that says zero base fee.

Rene Pickhardt I didn't buy this shirt. Some people from the community gave it to me. So it's actually really funny I want to mention this here. I went to a German speaking podcast together with Stefan and we talked about our paper. And as soon as the episode aired, one member of the German Bitcoin community posted a screenshot to Twitter and said, I just set zero base fee to all my channels. And Stefan and I, we talked to each other and we're like, we wrote this paper and we didn't do it on our own node. That's kind of embarrassing. And from that on, basically originated in Germany. A lot of people started setting their base fee to zero.

Sjors Provoost Okay.

Rene Pickhardt And now I think it's 30% of the network already who do this, which is really good because if I do my routing decisions, I actually use those channels and I like those channels more. So if they even charge a higher fee rate, that's okay for me because they do a good service for the network.

Sjors Provoost Making the math easier. Okay. Is there anything else you want to add for now?

Rene Pickhardt I want to say that... There is ongoing mainnet tests that have been happening. So we have been able to deliver 0.36 Bitcoin over the Lightning Network, which I think is a really large amount. And we tested this against any implementation that was out there on the same node, basically. And we haven't been able with those nodes out of the box to deliver the amount. So because, I mean, I was talking about an optimization problem. What we know is we know the best solution. What wasn't clear before is, is the best solution actually good enough, right? And it turns out it makes a huge difference to do this. So this is one thing. And the other thing that I want to mention is there is a whole different problem with respect to reliability, and we touched this briefly, which are these stuck payments. And they are really a big issue. But luckily, we're coming to a Schnorr and Taproot world where we can switch to PTLCs and create something like cancelable payments or redundant payments. And in that sense, we can actually start to handle those problems where... nodes are just not forwarding amounts and basically disturbing the payment process.

Sjors Provoost Right, because as a user, you don't know if the payment is actually going to fail or not, and you'd like to be able to explicitly cancel it, and then you can swap out that particular part of the payment with just a new payment, essentially.

Rene Pickhardt Well, you said a user, actually, the node should do this secretly for you. The user shouldn't interfere into this, but even the node doesn't know it.

Sjors Provoost Okay. Perfect, yeah. That's all?

Rene Pickhardt Yeah.

Sjors Provoost Okay. Well, thank you for coming all the way to Zurich for this podcast.

Rene Pickhardt Thank you too.

Sjors Provoost And do you want people to find you somewhere?

Rene Pickhardt They can find me on the mailing list on Twitter, on GitHub. I'm just out there.

Sjors Provoost Okay. Well then, dear listener, thank you for listening to Bitcoin Explained.