WEBVTT - Automatic transcript; recognition may contain errors. Reviewed for technical wording and speaker labels. Times match the published episode. 00:00:08.880 --> 00:00:11.580 Live from Utrecht, this is the Van Wirdum Sjorsnado. 00:00:11.820 --> 00:00:12.060 Hello. 00:00:12.600 --> 00:00:13.060 Hey Sjors. 00:00:13.220 --> 00:00:14.640 I think we've got somebody else here. 00:00:15.020 --> 00:00:19.440 We've got Joost Jager. Is that how Americans pronounce your name, Joost? 00:00:19.440 --> 00:00:23.580 That's indeed how Americans pronounce my name. But the Dutch people say Joost 00:00:23.580 --> 00:00:24.120 Jager. 00:00:24.280 --> 00:00:30.180 Joost Jager is in the building. Joost, please introduce yourself 00:00:30.180 --> 00:00:31.560 for our audience. 00:00:32.420 --> 00:00:37.800 Yeah, I describe myself as a Lightning developer. I've been working at Lightning 00:00:37.800 --> 00:00:43.800 Labs for a year or two and recently switched to Bottlepay. And I've got also my 00:00:43.800 --> 00:00:48.940 own projects on the side, mostly contributing to the LND repository. 00:00:50.100 --> 00:00:55.860 Right. And I think one of the topics that you've sort of... specialized 00:00:55.860 --> 00:00:59.600 yourself in or become an expert in is Lightning routing is that correct? 00:00:59.600 --> 00:01:02.060 Yeah that's correct I did a lot of work on that in the past. 00:01:02.060 --> 00:01:06.840 Yeah, yeah, so Sjors, we did an episode on Lightning in the past. 00:01:06.840 --> 00:01:09.580 We've done two episodes on Lightning in the past. 00:01:09.580 --> 00:01:13.900 We've done some stuff on Lightning in the past and it's all been very… 00:01:15.450 --> 00:01:17.170 Superficial? Is that a way to describe it? 00:01:17.210 --> 00:01:20.770 I think so. So we explained Lightning at a high level, so we can listen to that 00:01:20.770 --> 00:01:24.250 episode. And then there was another episode where we talked about RBF replaced by 00:01:24.250 --> 00:01:28.350 fee and how it relates to some problems on Lightning. And that was very detailed, 00:01:28.410 --> 00:01:29.310 but also very bad. 00:01:29.850 --> 00:01:34.610 Suffice to say, we're not the biggest Lightning experts in the world. So when it 00:01:34.610 --> 00:01:39.770 gets to sort of more in detail Lightning stuff, it gets pretty complicated for 00:01:39.770 --> 00:01:45.150 us, at least pretty fast. So I'm very happy to have a real Lightning expert here 00:01:45.150 --> 00:01:46.710 today Joost welcome, 00:01:46.710 --> 00:01:48.090 Thanks for having me. 00:01:48.090 --> 00:01:52.630 And we're hopefully going to so we're going we're going to get into Lightning 00:01:52.630 --> 00:01:58.310 routing more in more detail and explain what it is what the challenges 00:01:58.310 --> 00:02:03.610 are what you know what the direction of development is there 00:02:03.640 --> 00:02:08.600 and really get into it for this episode. All right Sjors, are you ready? 00:02:08.600 --> 00:02:09.780 I think I am. 00:02:10.780 --> 00:02:16.240 All right, Joost, first of all, let's start with a very basic question, just for 00:02:16.240 --> 00:02:22.040 maybe our new listeners, or just a very brief premise of what is Lightning and 00:02:22.040 --> 00:02:25.960 what is routing, what are we going to be talking about here, in a very general 00:02:25.960 --> 00:02:26.420 sense. 00:02:28.110 --> 00:02:30.310 The question, what is Lightning? All right. 00:02:30.550 --> 00:02:32.510 Yeah, just in a few sentences. 00:02:32.850 --> 00:02:38.470 Okay, I'd say it like this. Bitcoin is awesome, but it has several downsides. 00:02:38.970 --> 00:02:44.430 And Lightning is a technology built on top of Bitcoin to solve those. And then 00:02:44.430 --> 00:02:48.910 we're talking mainly about the fees that you pay to execute a transaction and the 00:02:48.910 --> 00:02:50.870 speed at which a transaction executes. 00:02:51.550 --> 00:02:57.230 Yeah, it's a payment layer on top of Bitcoin that allows for instant transactions 00:02:57.230 --> 00:03:00.990 that are also cheaper because not all of the transactions end up in the 00:03:00.990 --> 00:03:06.470 blockchain. So then the next question would be, what is routing? What are we 00:03:06.470 --> 00:03:10.430 going to talk about? Why is this a problem or a challenge or what is Lightning routing? 00:03:10.950 --> 00:03:16.790 Yeah, so with regular Bitcoin, you can pay to anyone who gives you an 00:03:16.790 --> 00:03:20.310 address without restrictions. It's a pretty direct 00:03:21.820 --> 00:03:26.680 process, whereas with Lightning your payment needs to travel through channels and 00:03:26.680 --> 00:03:30.960 there's only a limited number of channels so it isn't feasible for everyone to 00:03:30.960 --> 00:03:33.580 have a channel with all the other people on on the globe. 00:03:33.580 --> 00:03:37.040 And so maybe we can explain channels a little bit. So I could have a channel to 00:03:37.040 --> 00:03:41.460 Aaron which basically allows me to send coins to Aaron and Aaron could have a 00:03:41.460 --> 00:03:46.910 channel with you and then he could send coins to you But I can also send coins to 00:03:46.910 --> 00:03:49.860 you directly through Aaron. [unclear] 00:03:48.910 --> 00:03:50.430 So indirectly, yeah. 00:03:50.690 --> 00:03:54.690 Yeah, indirectly, exactly. By basically sending it to Aaron first, and then he 00:03:54.690 --> 00:03:57.950 forwards it to you in a bunch of ways that that's made safe. 00:03:58.210 --> 00:04:01.030 Yeah, well, that's indeed why you need routing, because there's a limited number 00:04:01.030 --> 00:04:04.870 of those channels. You need to find a path from yourself to the payment 00:04:04.870 --> 00:04:05.430 destination. 00:04:06.890 --> 00:04:07.370 Yeah. 00:04:07.730 --> 00:04:12.630 Yeah, the general concept is similar to the six degrees of separation idea. where 00:04:12.630 --> 00:04:17.570 you know everyone in the world through a couple of people only in the end, even 00:04:17.570 --> 00:04:22.810 though there's billions of people in the world. The theory is that within just a 00:04:22.810 --> 00:04:26.570 few people who know each other, you actually sort of know everyone. So that's the 00:04:26.570 --> 00:04:29.670 concept that's being used here. The challenge is... 00:04:31.300 --> 00:04:35.520 which channels in this case, which channels are you going to use to reach your 00:04:35.520 --> 00:04:39.280 endpoint? And somehow you need to figure it out. You need to figure out the 00:04:39.280 --> 00:04:42.240 routes to actually reach the person you want to pay, right? 00:04:42.420 --> 00:04:47.760 Indeed, it's like a roadmap. So you know where you want to go. There's only a 00:04:47.760 --> 00:04:51.540 limited number of roads that you can take. And you'd like to find the best 00:04:51.540 --> 00:04:57.430 possible route to your destination based on... similar to if a car either 00:04:57.430 --> 00:05:01.070 you want the fastest route or the shortest route I like different options and 00:05:01.070 --> 00:05:02.210 this is also at play in Lightning 00:05:02.210 --> 00:05:05.370 And this is why it's different from the rest of the internet because on the 00:05:05.370 --> 00:05:10.450 internet you just say I want to go to this Google website and all your computer 00:05:10.450 --> 00:05:13.930 has to do is send it to your router and then your router sends it to your 00:05:13.930 --> 00:05:17.330 internet provider and your internet provider sends it onwards just like the 00:05:17.330 --> 00:05:21.970 postal mail like you send it onwards and every hop knows where it where to go 00:05:21.970 --> 00:05:26.430 next, they figured it out. But with Lightning, you have to decide in advance 00:05:26.430 --> 00:05:29.190 exactly where it's going and which route it's going to take. 00:05:29.470 --> 00:05:33.210 Yeah, that's right. In Lightning, the sender decides what route is going to be 00:05:33.210 --> 00:05:38.970 taken exactly. And the reason for this is that Lightning has privacy properties 00:05:38.970 --> 00:05:43.850 that allow intermediate nodes or that allow the sender to keep himself hidden 00:05:43.850 --> 00:05:48.570 from intermediate nodes. So the intermediate hops, they don't see where... 00:05:50.030 --> 00:05:53.270 the way the payment is going. Sorry, I'm explaining this. 00:05:53.290 --> 00:05:53.910 Or where it came from. 00:05:54.010 --> 00:05:57.030 Yeah, well, also not where it came from, but they also don't know where it's 00:05:57.030 --> 00:06:00.910 going. And if every of those hops would find out the best path for themselves 00:06:00.910 --> 00:06:04.110 towards the destination, it wouldn't be hidden what the destination is. 00:06:04.290 --> 00:06:08.990 And to make an analogy to the real postal mail, like I could send Aaron a letter 00:06:08.990 --> 00:06:14.450 and inside the envelope is another envelope that has Joost’s address on it. 00:06:14.750 --> 00:06:17.910 And if you do that multiple steps, then, you know, when Aaron opens it, he only 00:06:17.910 --> 00:06:21.970 knows it's going to Joost and he might know it comes from me. but not where it's 00:06:21.970 --> 00:06:25.910 going ultimately. This is how the Onion Network works too. 00:06:26.110 --> 00:06:30.390 Yeah, nor did I know that you were the first step. Like that could have been five 00:06:30.390 --> 00:06:30.950 steps before. 00:06:30.970 --> 00:06:33.510 Yeah, there could have been many, many envelopes outside of my envelope. 00:06:33.730 --> 00:06:36.370 Right, yeah. And like I said, this is also how Tor works. 00:06:36.530 --> 00:06:39.290 And you also have to pay all the postage in advance. 00:06:40.230 --> 00:06:44.310 Yeah, that's a great analogy. And also the size of the envelope is always the 00:06:44.310 --> 00:06:48.500 same in Lightning, so you can't even see how many smaller envelopes are in… still inside 00:06:48.500 --> 00:06:51.760 So that's where it breaks down because with real envelopes you could put them on a scale. 00:06:51.760 --> 00:06:54.380 Yeah that's where it breaks down indeed. 00:06:54.380 --> 00:06:57.980 Yeah well actually yeah no yeah no there's no way to fix that. 00:06:57.980 --> 00:06:59.920 Let's not get lost in the analogy. 00:06:59.920 --> 00:07:03.980 I was thinking about how to how to put fake envelopes inside of it but you can't 00:07:03.980 --> 00:07:04.980 really without opening it. 00:07:04.980 --> 00:07:09.470 You want to get into it Sjors you want to get into the envelope game or? 00:07:09.470 --> 00:07:12.930 Well if you did the envelope game then the way you would keep the weight the same 00:07:12.930 --> 00:07:18.530 is I guess by replacing… opening it and putting a new envelope 00:07:18.530 --> 00:07:21.850 in the innermost envelope without reading it. 00:07:23.330 --> 00:07:28.490 Could you put pockets of helium in it to sort of counteract the weight balance? 00:07:28.930 --> 00:07:32.070 And then release the helium every time you ship it? I don't know. 00:07:32.420 --> 00:07:36.000 But that's indeed what's happening in Lightning. Like extra bytes are padded at 00:07:36.000 --> 00:07:38.900 the end of the packet to keep it of the same size all the time. 00:07:39.440 --> 00:07:41.340 Right, so you have no idea where you are in the route. 00:07:41.460 --> 00:07:43.200 Yeah, you don't know what hop you are. 00:07:43.520 --> 00:07:48.980 Okay, so that's why the sender needs to figure out the whole route. That's why 00:07:48.980 --> 00:07:52.480 we're getting into the whole envelope thing, just to point out that the sender 00:07:52.480 --> 00:07:57.050 needs to figure out the whole route. So in order for the sender to do that... The 00:07:57.050 --> 00:07:59.550 sender needs the map of the network, right? 00:07:59.750 --> 00:07:59.930 Yes. 00:08:00.630 --> 00:08:02.450 Does the sender need the entire map? 00:08:03.660 --> 00:08:07.260 It doesn't need the entire map. It needs enough routes to find its destination. 00:08:07.520 --> 00:08:11.300 But in practice, you will receive the entire map. 00:08:11.400 --> 00:08:14.460 Right now, nodes use the entire map to figure out… 00:08:14.660 --> 00:08:19.520 At least the visible part of it. Because there is also private channels, like say 00:08:19.520 --> 00:08:23.540 private roads. They are not on the map, but you need to get that information from 00:08:23.540 --> 00:08:25.780 a different source if you want to use those. 00:08:26.070 --> 00:08:29.690 Yeah, and private channels can't be used for routing anyways. That's why they're 00:08:29.690 --> 00:08:30.430 private, I guess. 00:08:30.470 --> 00:08:33.850 You can route over private channels, but you still need to know where they are. 00:08:34.170 --> 00:08:34.610 Okay. 00:08:34.710 --> 00:08:38.670 So you can have a routing node that has only private channels and is routing for 00:08:38.670 --> 00:08:40.190 users that know those. 00:08:40.370 --> 00:08:45.930 So I guess the postal analogy here would be to say, dear postman, like when 00:08:45.930 --> 00:08:50.530 you ask your friend, like, what's your address? You say, well, this is where I 00:08:50.530 --> 00:08:56.260 live, but I'm... But the postal service doesn't really know my street 00:08:56.260 --> 00:09:00.340 name. They only know the city I'm in. So here's what you need to tell the 00:09:00.340 --> 00:09:03.460 postman, like how to get from the city to my street or something like that. It's 00:09:03.460 --> 00:09:07.080 like a hidden instruction for the last [unclear]. Or maybe the analogy is like 00:09:07.080 --> 00:09:11.060 use the mailbox on the left, not the mailbox on the right or something like that. 00:09:11.980 --> 00:09:16.100 Maybe we should not do the analogy. In the Lightning Network, basically... 00:09:16.100 --> 00:09:21.520 You need to be on the road outside of Hill Valley in 1955. Exactly. 00:09:21.900 --> 00:09:23.780 This is a Back to the Future analogy. 00:09:24.020 --> 00:09:28.360 Exactly. So in the actual Lightning Network, it's simpler, right? It's just that 00:09:28.360 --> 00:09:33.500 some of the channels are not known. So if you're receiving, say, on a mobile 00:09:33.500 --> 00:09:39.440 wallet, then what you do is when you make an invoice, you say, this is my 00:09:39.440 --> 00:09:44.520 node. And by the way... I am connected to this publicly visible node through this 00:09:44.520 --> 00:09:48.080 channel. So you have to use that at the end of your route, but the rest of the 00:09:48.080 --> 00:09:49.100 route is up to you. 00:09:50.410 --> 00:09:55.390 We're getting sidetracked. Let's just assume for convenience sake, I'm going to 00:09:55.390 --> 00:09:59.850 assume for convenience sake that we're using public channels. So in that case, 00:10:00.010 --> 00:10:03.530 your node needs to know the entire map of the network of all the public channels. 00:10:03.770 --> 00:10:07.830 So how does your node, I'm running in the Lightning node, how does my node get 00:10:07.830 --> 00:10:11.810 all of this public channels? How does my node know where all of the public 00:10:11.810 --> 00:10:12.350 channels are? 00:10:12.570 --> 00:10:16.630 Yeah, so your node is connected to a few other nodes. And those few other nodes 00:10:16.630 --> 00:10:20.070 are also connected to a few other nodes. And through that, a whole network is 00:10:20.070 --> 00:10:24.610 created. And on that network, gossip is being passed around. So if there's one 00:10:24.610 --> 00:10:29.690 node who knows about the public channel, they gossiped this to their neighbors. 00:10:30.030 --> 00:10:31.890 And those neighbors, they... 00:10:31.890 --> 00:10:36.730 Gossip, just to be clear, is just sending messages. It's just sharing data. 00:10:36.850 --> 00:10:41.740 Yeah, just sending messages indeed. Yeah. So it slowly propagates throughout the 00:10:41.740 --> 00:10:45.220 network because they're all passing on the new information that they received 00:10:45.220 --> 00:10:45.800 from their peers. 00:10:46.000 --> 00:10:51.020 So every node basically shares with all of the nodes it connects to which 00:10:51.020 --> 00:10:55.260 channels it has, And then the next node does that as well. 00:10:56.460 --> 00:11:00.320 The information it got from the first node, and that's how all of the nodes end 00:11:00.320 --> 00:11:05.460 up with knowledge of all of the channels and allows all nodes to construct a map 00:11:05.460 --> 00:11:06.880 of the entire network. Is this correct? 00:11:07.140 --> 00:11:10.520 Yeah, they share the information that they have with their peers indeed. And in 00:11:10.520 --> 00:11:13.920 the end, everybody has the same copy, roughly the same copy of the network. 00:11:14.120 --> 00:11:19.740 So what is the information that's being shared exactly? What information 00:11:19.740 --> 00:11:23.600 about these channels is being... Is it just the information that the channels 00:11:23.600 --> 00:11:25.900 exist? Or is there more that's being shared? 00:11:26.300 --> 00:11:31.320 Yeah, so most important are the endpoints of the channel. So between which nodes 00:11:31.320 --> 00:11:36.560 the channel lives. Another important one is the size of the channel. So you can 00:11:36.560 --> 00:11:40.580 use this to decide whether this channel is big enough to carry the payment that 00:11:40.580 --> 00:11:43.760 you want to send. So if you have a large payment, you can skip all the small 00:11:43.760 --> 00:11:47.380 channels. You don't need to evaluate those because they won't be able to carry it anyway. 00:11:49.760 --> 00:11:52.420 There's also a pointer to the... 00:11:53.740 --> 00:11:59.400 the the output on chain so you can verify on the chain that this 00:11:59.400 --> 00:12:00.820 channel actually exists 00:12:00.820 --> 00:12:02.840 and and the size of it as well I guess 00:12:02.840 --> 00:12:07.120 yeah you can also observe the size from there. yeah yeah um so this is very 00:12:07.120 --> 00:12:09.340 important to prevent spam on the network. 00:12:09.340 --> 00:12:14.800 Yeah the size in this context of course mean number of satoshi or number of coins 00:12:14.800 --> 00:12:15.340 in the channel. 00:12:15.340 --> 00:12:20.660 yeah the capacity of the channel. yeah so the maximum size that it can relay um 00:12:20.660 --> 00:12:26.430 so that's there. And there's also the forwarding policy for that 00:12:26.430 --> 00:12:30.750 channel. So if nodes advertise a channel that can be used for routing other 00:12:30.750 --> 00:12:36.650 payments, those nodes, they charge a fee for that. And the fee is also advertised 00:12:36.650 --> 00:12:38.570 in this peer-to-peer network. 00:12:39.030 --> 00:12:44.290 Right. So if my node is mapping the entire network, it 00:12:44.290 --> 00:12:49.470 knows… where all which channels exist or at least which public channels exist it 00:12:49.470 --> 00:12:54.370 knows how many satoshis or how many coins are in each channels it knows how much 00:12:54.370 --> 00:12:58.230 it's going to cost me to route through that channel But it does not, for example, 00:12:58.470 --> 00:13:04.030 know for each channel where the balance is because the balance can be on either 00:13:04.030 --> 00:13:04.990 side of the channel, right? 00:13:05.190 --> 00:13:05.370 Yeah. 00:13:05.470 --> 00:13:09.470 So that's not communicated. That's not part of the map that my node has. Is that correct? 00:13:09.830 --> 00:13:10.870 That's right. Yes. 00:13:11.110 --> 00:13:15.430 So that sounds like that could be a challenge when we actually start routing 00:13:15.430 --> 00:13:19.730 because it actually does matter. how the balances are divided over the different 00:13:19.730 --> 00:13:20.330 channels, right? 00:13:20.510 --> 00:13:23.750 That's right. If you start routing and you've picked your route, you've picked 00:13:23.750 --> 00:13:27.110 your channels, it could turn out that one of those channels doesn't have the 00:13:27.110 --> 00:13:29.710 balance on the right side and then your payment fails. 00:13:29.990 --> 00:13:35.990 Right. So how does my node calculate the best route? How does my node deal with 00:13:35.990 --> 00:13:36.810 something like that? 00:13:36.950 --> 00:13:42.630 Yeah. So you can look at this in different ways. But one 00:13:42.630 --> 00:13:47.730 way a node can implement this is that it assumes channels to be in balance 00:13:47.730 --> 00:13:53.470 always. So let's say if there's a channel that has one BTC. we will accept that 00:13:53.470 --> 00:13:57.790 it might be a little bit unbalanced, but not too much. So for example, you could 00:13:57.790 --> 00:14:03.750 say a one BTC channel should at least be able to carry a payment of 0.2 BTC, 00:14:04.650 --> 00:14:08.590 for example, because it's not reasonable to expect from a one BTC channel that 00:14:08.590 --> 00:14:10.470 it's always able to carry one BTC. 00:14:12.590 --> 00:14:18.570 So if you assume that good routing nodes keep their channels balanced and that 00:14:18.570 --> 00:14:24.570 they let payments through those channels and it doesn't happen, then you can 00:14:24.570 --> 00:14:28.230 take note of that as a node and avoid this channel the next time. 00:14:28.550 --> 00:14:34.370 Right. Taking one small step back, so we're now getting into sort of 00:14:34.370 --> 00:14:38.510 pathfinding. I guess that's the technically correct term, right? 00:14:38.650 --> 00:14:38.850 Yes. 00:14:39.250 --> 00:14:43.290 And pathfinding, we already mentioned that this is something that a node itself 00:14:43.290 --> 00:14:49.010 figures out. The paying node essentially figures this out. This isn't like 00:14:49.010 --> 00:14:53.550 a... protocol thing right this is something that individual nodes can do however 00:14:53.550 --> 00:14:59.070 they please essentially and and is there a difference between different Lightning 00:14:59.070 --> 00:15:03.830 implementations how they do this or is this more or less synchronized as far as you know? 00:15:03.830 --> 00:15:09.560 Oh so the part that is synchronized is that uh it is optimized for cost. So 00:15:09.560 --> 00:15:14.080 there's cheap channels, there's expensive channels, and all of these 00:15:14.080 --> 00:15:17.840 implementations, they primarily look for the cheapest path to the destination. 00:15:18.160 --> 00:15:18.620 Right. 00:15:18.780 --> 00:15:22.360 But then between the node implementation, there's variation in other factors that 00:15:22.360 --> 00:15:23.940 play into this as well. 00:15:24.160 --> 00:15:24.460 Right. 00:15:24.860 --> 00:15:26.120 Yeah, so one of those factors. 00:15:26.120 --> 00:15:30.600 Yeah, let's get into these factors. That's an interesting point. Okay, so we want 00:15:30.600 --> 00:15:36.240 to make a payment. We've mapped out the entire network. We know 00:15:36.240 --> 00:15:40.640 which channels exist. We know what kind of fees they will charge if they forward 00:15:40.640 --> 00:15:46.300 the payment. Now, there are obviously, or I mean, I guess in most cases, there 00:15:46.300 --> 00:15:50.170 are going to be multiple potential paths yeah definitely in the same way that if 00:15:50.170 --> 00:15:53.250 you're you know you're traveling from one city to another there are probably 00:15:53.250 --> 00:15:59.070 multiple roads you could potentially take how do you decide on the actual 00:15:59.070 --> 00:16:02.430 route that you're going to take? How does a Lightning node decide this? 00:16:02.570 --> 00:16:07.590 Yes. So besides the cost factor, there is the time lock factor. So in Lightning, 00:16:07.870 --> 00:16:12.330 it's quite technical, but if you send a payment in Lightning, there is a maximum 00:16:12.330 --> 00:16:17.070 time that this payment can take to complete. And this is a pretty long time. It 00:16:17.070 --> 00:16:22.030 could be like five days or even two weeks in some cases. So this is a parameter 00:16:22.030 --> 00:16:22.470 normally… 00:16:23.130 --> 00:16:28.930 So it takes long because every hub on the network can potentially wait 00:16:28.930 --> 00:16:31.990 for a little while before forwarding. Is that the reason? 00:16:32.110 --> 00:16:37.450 So in the happy flow, it's near instant. But in the worst case, a node can sit on 00:16:37.450 --> 00:16:43.190 that payment and not do anything. And then you have to wait until that ultimate 00:16:43.190 --> 00:16:45.570 time lock that you attach to a payment. 00:16:46.850 --> 00:16:50.010 Until that expires, until you can sort of take the money back, right? 00:16:50.250 --> 00:16:55.210 Yeah, so you can take the money back. And usually you don't see or hear anything 00:16:55.210 --> 00:16:59.890 of this, but only in a rare case that a node indeed does this, you see your money 00:16:59.890 --> 00:17:05.530 locked up for a long time. And if you've got two routes with identical costs, 00:17:05.750 --> 00:17:10.470 while one node only requires you to lock up the money for a maximum of one day 00:17:10.470 --> 00:17:14.110 and the other seven days, we prefer the one-day option. 00:17:14.540 --> 00:17:18.700 Right, because in the worst case scenario, you'll get your money back faster. 00:17:19.000 --> 00:17:19.160 Yes. 00:17:19.830 --> 00:17:25.770 So you're making a calculation based on fees, based on time. And 00:17:25.770 --> 00:17:26.590 was there another factor? 00:17:26.690 --> 00:17:31.310 Yeah, there's another factor, which is reliability. And this ties into the thing 00:17:31.310 --> 00:17:35.930 that we just talked about, where we keep track of the performance of nodes. And 00:17:35.930 --> 00:17:40.210 if certain channels don't perform well, we try to avoid them. 00:17:41.320 --> 00:17:46.200 And by not performing well, you mean they previously? Either didn't have enough 00:17:46.200 --> 00:17:50.460 capacity, so they immediately said, hey, I'm sorry, I can't actually do this for 00:17:50.460 --> 00:17:52.720 you now. Or they timed out? 00:17:53.020 --> 00:17:55.780 Yeah, so there are multiple things that can go wrong. Like one of the most common 00:17:55.780 --> 00:18:00.400 things is that the balance is not sufficient in the channel, but it could also, 00:18:00.500 --> 00:18:05.200 for example, be that a node is offline. That can also happen. So there is 00:18:05.200 --> 00:18:07.920 balance, but the node is offline, so it still can't carry your payment. And 00:18:07.920 --> 00:18:13.680 there's also a variety of more rare exceptions that we also use to 00:18:13.680 --> 00:18:16.140 update those reputation scores of nodes. 00:18:16.550 --> 00:18:20.670 Right. And one of the problems with that reputation, as we talked about this 00:18:20.670 --> 00:18:26.170 before, so that's why I sound really smart, is that you don't know, you don't 00:18:26.170 --> 00:18:29.690 necessarily know which node on the route screwed up, right? 00:18:30.140 --> 00:18:34.540 Yeah, if a payment fails, you know that the payment fails. But in some cases, or 00:18:34.540 --> 00:18:39.320 maybe even in all cases, you don't know where it failed exactly. Is that right? 00:18:39.600 --> 00:18:43.920 You do know where it fails. Like a node sends you back an error message to the 00:18:43.920 --> 00:18:49.480 sender, and it contains its identity, a signature. So the node, it signs the 00:18:49.480 --> 00:18:54.110 error. So as a sender, you know where the error occurred, but it's still not 00:18:54.110 --> 00:18:57.810 completely clear whether this node is actually also at fault. So it's actually 00:18:57.810 --> 00:19:02.970 responsible for the error. So it could happen that node A sends a bad payment 00:19:02.970 --> 00:19:08.170 request to node B, and then node B replies with an error message. But it's not 00:19:08.170 --> 00:19:12.250 fair to penalize node B in that case, because it's node A that actually sent the 00:19:12.250 --> 00:19:15.910 wrong request. So it's a little bit more complicated than... 00:19:15.910 --> 00:19:21.310 So if you're neither A nor B, you're the one that made the payment, then you're 00:19:21.310 --> 00:19:22.610 not sure if it was A or B? 00:19:22.830 --> 00:19:26.270 Yeah, you're not sure whether it's A or B. So what the sender does in this case 00:19:26.270 --> 00:19:31.770 is they penalize the pair of nodes. So they apply a penalty to the link between A and B. 00:19:33.100 --> 00:19:33.540 Right. 00:19:33.800 --> 00:19:38.420 So this can create guilt by association if you're an unlucky node next to a bad node. 00:19:38.560 --> 00:19:42.540 Yeah, definitely. So you have to be very careful about which nodes you peer to 00:19:42.540 --> 00:19:45.240 because they are also influencing your own reputation. 00:19:47.050 --> 00:19:52.990 Okay, so I think you've now mentioned, so yeah, you're basically attaching 00:19:52.990 --> 00:19:58.830 a score to nodes. Am I saying this right? So you've got a map of the network. 00:19:58.970 --> 00:20:04.690 You know all of the channels. You know how much funds each channel has. You know 00:20:04.690 --> 00:20:10.290 how much fees they charge. And now you're also applying a score to all 00:20:10.290 --> 00:20:16.140 nodes on the network based on their performance. based on your request to 00:20:16.140 --> 00:20:18.200 route through them. Is this right? 00:20:18.400 --> 00:20:22.560 Yeah, well, not to nodes, but to node pairs. So we attach the score to [unclear] 00:20:22.560 --> 00:20:23.160 node pairs. 00:20:24.640 --> 00:20:30.620 They're always two pairs. So it doesn't get calculated into individual nodes in the end. 00:20:32.990 --> 00:20:37.230 Yeah, you're right, it does get… it does get… so, like, at the basic level, we 00:20:37.230 --> 00:20:43.230 assign penalties to pairs of nodes. But some nodes have a lot of peers. And 00:20:43.230 --> 00:20:47.910 to prevent us from trying out all the combinations of that node with all its 00:20:47.910 --> 00:20:52.810 peers, we also extrapolate the performance of a specific pair to the performance 00:20:52.810 --> 00:20:53.410 of the node. 00:20:53.730 --> 00:20:57.250 Right, yeah. So nodes have a specific score, essentially. 00:20:57.250 --> 00:20:58.190 Yes, they have a score. 00:20:58.550 --> 00:21:02.450 So I think there was another heuristic you talked about when we prepared this, 00:21:02.830 --> 00:21:07.750 which is, I guess, never change a winning team would be a way to describe it. As 00:21:07.750 --> 00:21:13.570 in, if you're looking for a route to pay and you found one that's been 00:21:13.570 --> 00:21:17.090 used before and it worked really well, then you prefer that one rather than try 00:21:17.090 --> 00:21:17.630 something new. 00:21:18.030 --> 00:21:23.960 Yeah, now you're getting to the point… How is this reliability factored into 00:21:23.960 --> 00:21:29.180 the pathfinding? So what's the weight of a reliable or unreliable node? And the 00:21:29.180 --> 00:21:34.760 way this works, at least in LND, is that previously successful routes, they are 00:21:34.760 --> 00:21:40.000 very much preferred until proven otherwise. So if a route is successful, it will 00:21:40.000 --> 00:21:44.560 keep that label until it's proven otherwise, whereas a failure slowly decays 00:21:44.560 --> 00:21:48.420 because if you would hold on to a failure forever, we would never give that node 00:21:48.420 --> 00:21:54.270 another chance. So there's two different ways of dealing with a success versus a failure. 00:21:55.680 --> 00:21:56.300 All right. 00:21:56.520 --> 00:22:01.280 So if I'm understanding correctly, I think the way it essentially works is that 00:22:01.280 --> 00:22:07.260 your node... has a map of network figures out what the cheapest 00:22:07.260 --> 00:22:12.300 route is and by cheap cheapest actually factors in these other things so it 00:22:12.300 --> 00:22:17.880 factors in the fees itself so that's sort of the most plain way of calculating 00:22:17.880 --> 00:22:23.580 the cost but then it factors in the time locks which you mentioned and the 00:22:23.580 --> 00:22:28.900 reliability as sort of cost factors and then based on all these three things 00:22:28.900 --> 00:22:34.490 fees time and reliability, it figures out what sort of the real cheapest 00:22:34.490 --> 00:22:35.790 route is. Is that right? 00:22:35.810 --> 00:22:40.450 Yeah, I would call that the best route. So best is a little bit unqualified, but 00:22:40.450 --> 00:22:42.710 it means a combination of those things that you mentioned, yes. 00:22:43.050 --> 00:22:48.890 Right. And just to reiterate this, nodes on the network, 00:22:49.010 --> 00:22:53.430 so my node makes this calculation, but Sjors’ node could make a different 00:22:53.430 --> 00:22:57.830 calculation based on the past routing that Sjors has done and based on the past 00:22:57.830 --> 00:23:02.290 routing that I have done, right? It's an individual calculation for all nodes on the network. 00:23:02.370 --> 00:23:06.430 It is an individual calculation, and it's based indeed on the experiences that a 00:23:06.430 --> 00:23:10.590 node has, but also on how you configure the node. So there's also room for the 00:23:10.590 --> 00:23:16.410 user to decide what they think is important, whether they value cost over 00:23:16.410 --> 00:23:18.010 reliability or the other way around. 00:23:19.800 --> 00:23:24.440 So I've talked about an analogy with restaurants, for example. So if you have 00:23:24.440 --> 00:23:28.400 been to a good restaurant, the question is, will you go to that every time again 00:23:28.400 --> 00:23:31.540 the rest of your life? Or do you sometimes try a new restaurant? And between 00:23:31.540 --> 00:23:35.760 people, there's a difference in how they find this balance. So some people want 00:23:35.760 --> 00:23:38.380 to try something new all the time and have disappointments, and others, they 00:23:38.380 --> 00:23:41.760 never try something new. And this is a parameter that you can also set on your 00:23:41.760 --> 00:23:42.280 Lightning Node. 00:23:43.470 --> 00:23:44.470 Right. Interesting. 00:23:46.420 --> 00:23:51.700 So I I've written about Lightning uh type of stuff in the past 00:23:51.700 --> 00:23:56.320 including routing at some point and I but I think it's been like probably two 00:23:56.320 --> 00:24:00.960 years since I wrote about this back then the idea was because there was a lot of 00:24:00.960 --> 00:24:06.720 um criticism I guess or a little concern or maybe concern trolling 00:24:06.720 --> 00:24:11.280 whatever you want to call it about how these solutions don't really scale because 00:24:11.280 --> 00:24:16.340 you know it does mean that every node needs to keep track of the entire network. 00:24:17.740 --> 00:24:23.540 And at that point, the sort of answer to that concern 00:24:23.540 --> 00:24:29.320 was, for now it works. For now this is scalable enough. For now all 00:24:29.320 --> 00:24:34.560 nodes can keep track of this. And we'll now just keep track of how the network 00:24:34.560 --> 00:24:37.940 develops because there's a lot of questions about that. Are we going to see a 00:24:37.940 --> 00:24:42.510 network with a few power players that do most of the routing? Or are we going to 00:24:42.510 --> 00:24:45.550 see a very decentralized network? But for now, we're just going to see how it 00:24:45.550 --> 00:24:51.350 develops. And then we'll make our next decisions based on that. Is that 00:24:51.350 --> 00:24:56.170 still sort of the status? Or are there other... Are there by now other ideas of 00:24:56.170 --> 00:24:59.030 how to evolve this over time, how to develop this over time? 00:24:59.210 --> 00:25:03.690 Yeah, so as far as I know, that's still more or less the status. They're just on 00:25:03.690 --> 00:25:08.270 the part of the mobile nodes. There has been some developments in those areas. 00:25:09.490 --> 00:25:14.780 Like a copy of the network is not that big for like a modern time computer; it's 00:25:14.780 --> 00:25:17.140 very easy to keep track of what’s… what’s… what’s… there now. 00:25:17.400 --> 00:25:18.240 For now, it's very doable. 00:25:18.500 --> 00:25:21.460 Yeah, only for mobile nodes it's different. So you see that these mobile wallets, 00:25:21.540 --> 00:25:27.100 they look for different solutions. So either they outsource the route finding 00:25:27.100 --> 00:25:32.580 completely to a server or partially. And in the case of 00:25:32.580 --> 00:25:38.480 partial outsourcing, they use trampoline nodes. So these are nodes on the network 00:25:38.480 --> 00:25:44.280 that have a copy of... all the channels that exist, and you can delegate part of 00:25:44.280 --> 00:25:45.720 your pathfinding to that node. 00:25:46.160 --> 00:25:52.120 So how this works, I think, but do correct me if I'm wrong, is that 00:25:52.120 --> 00:25:57.800 your mobile wallet, in this case, would find a route to a 00:25:57.800 --> 00:26:03.280 special node, a trampoline node, which is, I guess, relatively close or something 00:26:03.280 --> 00:26:05.190 like that. I'm speculating here. 00:26:05.430 --> 00:26:09.470 It doesn't matter where it is. It's just that you might, as a note, decide to 00:26:09.470 --> 00:26:14.180 only know the trampoline nodes. So as a mobile client, instead of downloading... 00:26:13.300 --> 00:26:16.340 you just know much fewer nodes, right? 00:26:16.260 --> 00:26:20.820 Yeah. So when you listen to the gossip, you might tell your peers, like, only 00:26:20.820 --> 00:26:22.500 send me gossip about trampoline nodes. 00:26:22.620 --> 00:26:22.960 Right. 00:26:23.140 --> 00:26:27.440 And then that's a much smaller network. And then what you do is when you're 00:26:27.440 --> 00:26:32.240 creating a route, you create a route to a trampoline node. And then you tell the 00:26:32.240 --> 00:26:35.960 trampoline node, essentially, that's sort of the inner envelope, like, go to this 00:26:35.960 --> 00:26:36.760 final destination. 00:26:38.170 --> 00:26:40.450 The trampoline node figures it out from there. 00:26:40.630 --> 00:26:44.910 Yeah, so you're compromising a bit on your privacy and on your ability to find 00:26:44.910 --> 00:26:50.240 the cheapest route. But the upshot is that you don't have to know the map of the 00:26:50.240 --> 00:26:55.340 entire network. And there's some intermediate solutions where you might 00:26:55.340 --> 00:27:00.500