WEBVTT - Automatic transcript; recognition may contain errors. Reviewed for technical wording and speaker labels. Times match the published episode. 00:00:20.660 --> 00:00:23.200 Live from Utrecht, this is Bitcoin 00:00:23.200 --> 00:00:24.480 Explained. 00:00:25.200 --> 00:00:26.240 Welcome back, Sjors. 00:00:26.800 --> 00:00:27.160 Thank you. 00:00:27.440 --> 00:00:30.580 You were on holiday. That's why we were out for a couple of weeks. Where did you 00:00:30.580 --> 00:00:31.080 actually go? 00:00:32.320 --> 00:00:33.420 I'm not going to say. 00:00:35.320 --> 00:00:37.200 Is that your OPSEC strategy? 00:00:37.440 --> 00:00:37.780 Exactly. 00:00:37.780 --> 00:00:39.600 Or just ashamed of where you went? 00:00:39.960 --> 00:00:40.240 Both. 00:00:42.080 --> 00:00:46.820 Okay, well, Bitcoin Amsterdam is coming up just a couple of months from now. You 00:00:46.820 --> 00:00:51.480 know what that means, right, Sjors? A conference in Amsterdam. It means 00:00:51.480 --> 00:00:53.440 It means having to go to Amsterdam. 00:00:53.440 --> 00:00:58.410 It means 420 blaze it. That's why we're gonna talk about hash today. 00:00:58.410 --> 00:01:00.340 Do you get it? 00:01:00.440 --> 00:01:01.260 Oh. 00:01:01.260 --> 00:01:05.220 That's that's the first pun. The first pun of the new season. I'm gonna consider 00:01:05.220 --> 00:01:09.660 this a new season and we're strong out of the gate with with fresh new puns for our listeners 00:01:09.660 --> 00:01:13.220 I did expect you were gonna make a a pun around that topic. yes. 00:01:13.920 --> 00:01:19.200 We are going to talk about hashes, hash functions to be specific. This is a topic 00:01:19.200 --> 00:01:24.360 that has come up here and there. And it's obviously a topic that everyone knows 00:01:24.360 --> 00:01:29.340 something about. And you, our dear listener, might now think where you're talking 00:01:29.340 --> 00:01:34.000 about hashes, we already know what hashes are. But do you really? Do you really 00:01:34.000 --> 00:01:38.740 know what a hash is? I didn't. I thought I did. But I learned something in 00:01:38.740 --> 00:01:43.180 preparation for this show. So you might as well. Sjors, I'm going to start with 00:01:43.180 --> 00:01:47.360 the first basic question. I think most people have a general idea of what a hash 00:01:47.360 --> 00:01:50.700 is, but still, Sjors, what is a hash? 00:01:51.680 --> 00:01:57.280 Well, it's basically a function that goes in one way. It garbles the input. 00:01:57.560 --> 00:02:02.260 So if you start with the word hello, then out comes some garbage. And if you 00:02:02.260 --> 00:02:04.660 start with the word hello too, out comes some different garbage. 00:02:05.910 --> 00:02:10.630 Yeah, well, you say garbage, but what you really mean is a number. 00:02:11.270 --> 00:02:11.750 yes. 00:02:11.750 --> 00:02:12.130 Data. 00:02:12.130 --> 00:02:12.910 yes. 00:02:12.910 --> 00:02:15.790 Essentially, zeros and ones. 00:02:15.790 --> 00:02:21.670 Yes but I mean everything is zeros and ones so that's why a number. yeah. 00:02:21.670 --> 00:02:25.210 Is that the best way to put it? A number comes out a number goes in a number 00:02:25.210 --> 00:02:31.190 comes out, and any data you can convert any data a picture or a song or as 00:02:31.190 --> 00:02:37.070 long as it's digital you can convert it into a number and then you can You can 00:02:37.070 --> 00:02:39.310 hash that number and you get a different number, right? 00:02:39.590 --> 00:02:40.190 Yeah, that's right. 00:02:40.750 --> 00:02:46.690 Okay, now there are some important properties for a hash function to 00:02:46.690 --> 00:02:47.550 be a hash function. 00:02:49.500 --> 00:02:55.400 You can't just say multiply everything by two. If you do that, you get a 00:02:55.400 --> 00:02:59.320 different number, but that's not a hash function. A hash function needs to do 00:02:59.320 --> 00:03:01.000 something specific. So what does it need to do? 00:03:01.120 --> 00:03:06.380 Yeah, it needs to make sure that if you know the hash, 00:03:07.000 --> 00:03:11.500 like you can't just change it basically or maybe a better way to say it is so if 00:03:11.500 --> 00:03:16.380 you have the word hello and you hash it it comes out a number and then you 00:03:16.380 --> 00:03:19.200 shouldn't just be able to say well I want to make hello too I'm just going to 00:03:19.200 --> 00:03:22.560 change the hash now, and then I know that it was hello too. 00:03:23.300 --> 00:03:25.760 Right so what you're saying is… 00:03:25.760 --> 00:03:29.140 So if in your example of multiplying by two let's say that the hash of the word 00:03:29.140 --> 00:03:35.050 hello is one and if the hash of the word hello2 is 2, well, it's a little 00:03:35.050 --> 00:03:35.850 bit too predictable. 00:03:36.470 --> 00:03:36.610 Right. 00:03:36.850 --> 00:03:39.550 I mean, that would be a weird example. Maybe a better example would be if the 00:03:39.550 --> 00:03:44.250 hash of the word hello, hello was 2. It would be very predictable because now if 00:03:44.250 --> 00:03:49.170 I can take the number 3 and I can guess, oh, then the original must be hello, hello, hello. 00:03:50.750 --> 00:03:56.550 Yeah, so essentially... The outcome needs to be completely unpredictable. 00:03:56.990 --> 00:04:02.910 So if you hash an entire book and then you 00:04:02.910 --> 00:04:08.550 hash the exact same book, but you change one period into a comma, the hash needs 00:04:08.550 --> 00:04:09.850 to be completely different, right? 00:04:10.050 --> 00:04:10.630 Yes, absolutely. 00:04:10.830 --> 00:04:16.490 And there needs to be no way to figure out which hash came from 00:04:16.490 --> 00:04:18.010 which book. 00:04:18.450 --> 00:04:20.510 Yeah, well, in particular, like... 00:04:20.510 --> 00:04:22.450 Unless you have the book, I should add. 00:04:22.670 --> 00:04:26.050 Yeah, I think one thing you want to prevent is the thing called collisions, 00:04:26.330 --> 00:04:31.910 right? So that means that if I take your book and now I change some comma in a 00:04:31.910 --> 00:04:36.350 very specific place that I would get the same hash, that's a problem. Because 00:04:36.350 --> 00:04:39.830 then I could show you two different books and it would have the same hash. And so 00:04:39.830 --> 00:04:43.630 now I've cheated because the whole point of the hash was to prove that, you know, 00:04:43.750 --> 00:04:46.450 I was actually referring to the book and not some other book. 00:04:47.180 --> 00:04:52.720 Right. So each piece of data should result in a different hash. 00:04:52.940 --> 00:04:53.280 Yes. 00:04:53.660 --> 00:04:59.470 Right. Okay. So that ties into my next question. And 00:04:59.470 --> 00:05:03.550 I'm not sure if we completely answered this question already, but so let's see if 00:05:03.550 --> 00:05:08.870 we did or didn't. What would it make? So some hash functions have been broken in 00:05:08.870 --> 00:05:12.570 the past. So what does it mean if a hash function is broken? 00:05:13.130 --> 00:05:17.650 Yeah, so then we can get into some numbers, I guess. So let's say the resulting 00:05:17.650 --> 00:05:22.970 hash is a number with like it's 64 bytes or 256 00:05:22.970 --> 00:05:28.910 bits, right? 256 bits, that's the hash. And that means there are two to 00:05:28.910 --> 00:05:32.360 the power of 256 possibilities. Right? 00:05:32.860 --> 00:05:37.240 Yeah, and bits are literally the zeros and ones that I mentioned earlier. 00:05:37.360 --> 00:05:42.360 Right, so 256 zeros and ones, that's the hash itself. So that means that if you 00:05:42.360 --> 00:05:47.780 have some piece of text, say the word hello, and that results in this hash, now 00:05:47.780 --> 00:05:52.200 the question would be, can you find another word that results in the same hash? 00:05:53.460 --> 00:05:59.140 For example, the word hello too. And if the hash works really well, then you just 00:05:59.140 --> 00:06:04.580 have to guess. You would try, you know, 2 to the power of 256 different words 00:06:04.580 --> 00:06:10.420 or different sentences until you get the same hash as hello. And that should 00:06:10.420 --> 00:06:14.220 take a very long time. Now, the hash is very broken if it doesn't take a very 00:06:14.220 --> 00:06:18.960 long time. If it turns out that you can just try maybe 100 different things and 00:06:18.960 --> 00:06:22.300 then you find a duplicate hash or a collision. 00:06:22.780 --> 00:06:27.920 Right. So as soon as any hash collision is found, 00:06:28.870 --> 00:06:34.870 which has happened with previous, or I'm not sure why, with older hash 00:06:34.870 --> 00:06:40.570 functions. So as soon as any collision is found, is at that point the hash 00:06:40.570 --> 00:06:42.190 function considered broken? 00:06:42.430 --> 00:06:47.630 I don't know what the exact criteria is, but you can try and measure it. So if, 00:06:47.730 --> 00:06:53.710 for example, it should take 2 to the power of 256 attempts, but in practice, you 00:06:53.710 --> 00:06:57.890 can do it once in 100 times, then obviously it's broken. But if it takes 2 to the 00:06:57.890 --> 00:07:03.710 power of 255 attempts, that's still a huge number. So there might still be some 00:07:03.710 --> 00:07:09.310 practical limit. But probably in the case of this, if you're able to find a 00:07:09.310 --> 00:07:12.150 single collision, that means the odds are already too high because it should 00:07:12.150 --> 00:07:15.430 really never happen with that big of a number. It's such an astronomical number 00:07:15.430 --> 00:07:18.910 that it should take you longer than the age of the universe to try all these 00:07:18.910 --> 00:07:21.690 combinations. So if you find something in less than the age of the universe, 00:07:21.950 --> 00:07:23.650 that's a really bad sign. 00:07:24.210 --> 00:07:29.710 And if you do find something, it was presumably in less time than the age of the 00:07:29.710 --> 00:07:30.130 universe. 00:07:30.390 --> 00:07:35.330 Yes, because you're alive. And so in historical examples, there have been hash 00:07:35.330 --> 00:07:39.210 functions that were able to be broken earlier. See if I can find the list or you 00:07:39.210 --> 00:07:44.880 might have it. Yes. So for example, there was MD5, I think. And 00:07:44.880 --> 00:07:50.480 MD5 was supposed to be 128-bit of hash. So that's 2 to the power of 128 attempts. 00:07:50.480 --> 00:07:53.680 That is a very long time. Not as long as 256. 00:07:54.780 --> 00:07:56.040 Or the age of the universe. 00:07:56.180 --> 00:08:01.780 But still, that's still age of the universe longer than that. But 256 is even longer. 00:08:02.060 --> 00:08:02.440 Right. 00:08:02.760 --> 00:08:07.640 Yeah, yeah. Because 128 bits is the security of a 12-word mnemonic. So if you 00:08:07.640 --> 00:08:10.880 have 12 words written down, the 12-word seed, that is 128 bits. 00:08:11.620 --> 00:08:12.220 Got it. 00:08:12.220 --> 00:08:17.500 24 one is 256 bits, right so it's it should be very very secure. However it turns 00:08:17.500 --> 00:08:22.360 out that somebody was able to find a collision with only two to the power 18 00:08:22.360 --> 00:08:23.380 attempts. 00:08:23.380 --> 00:08:24.960 Right I see. 00:08:24.960 --> 00:08:28.120 That is roughly… Should not try to do that off my head, but I think it's less 00:08:28.120 --> 00:08:28.540 than a million. 00:08:29.090 --> 00:08:33.070 Right. So in that case, the hash function is considered broken. 00:08:33.210 --> 00:08:36.570 It's completely broken because if you can, you know, brute force a million 00:08:36.570 --> 00:08:41.090 different words on a computer, like a million different passwords to guess what 00:08:41.090 --> 00:08:43.890 the hash means, that's nothing. You can do that in a very short time. 00:08:44.230 --> 00:08:48.710 Yeah. And you just use the word brute force. Is that the only way to break 00:08:48.710 --> 00:08:54.440 a hash function in this way, or in this is well… 00:08:54.440 --> 00:08:55.480 Or can you also… 00:08:55.480 --> 00:08:59.100 Brute force should not help you at all, right because that should take you the 00:08:59.100 --> 00:09:02.400 age of the universe. No but it's usually combination with some trick right, 00:09:02.400 --> 00:09:05.100 whatever that trick is and then brute force. 00:09:05.100 --> 00:09:09.740 Right yeah that was going to be my question indeed. So there's a more pointed way 00:09:09.740 --> 00:09:13.620 to try and break it. You're you're using some kind of strategy to break it then. 00:09:13.620 --> 00:09:17.000 Yeah but you're still probably brute force looking but you have a better idea 00:09:17.000 --> 00:09:17.520 where to look. 00:09:17.900 --> 00:09:23.800 Right, got it. Is this the only or at least the main failure 00:09:23.800 --> 00:09:26.880 mode or failure model of a hash function? 00:09:27.430 --> 00:09:31.410 Yeah, that's when the hash function itself would be considered a failure, right? 00:09:31.690 --> 00:09:36.010 But there are ways that you can use hashes that make you more prone to failure. 00:09:36.010 --> 00:09:40.350 So for example there are websites that store passwords. Well they store they 00:09:40.350 --> 00:09:43.230 don't store the password, that's really bad, but there are websites that store 00:09:43.230 --> 00:09:44.290 the hash of a password. 00:09:44.290 --> 00:09:45.070 Yeah right. 00:09:45.070 --> 00:09:48.710 And now, let's say that that hash is not very good or even if it is very good 00:09:48.710 --> 00:09:53.230 actually, somebody could make something called a rainbow table. So somebody would 00:09:53.230 --> 00:09:57.030 say, well, what if the [unclear] password is hello? What if the password is hello 00:09:57.030 --> 00:10:01.510 too? And they calculate the hash for each of these passwords. And then if you 00:10:01.510 --> 00:10:05.850 find a leaked database with all these passwords, you can very quickly guess which 00:10:05.850 --> 00:10:09.050 password belongs to which hash because you have this lookup table. 00:10:10.090 --> 00:10:14.770 Right. It's basically you're just entering passwords yourself, seeing what the 00:10:14.770 --> 00:10:16.930 hash is, and then remembering that. 00:10:18.350 --> 00:10:23.310 If someone pops up using that hash as a password, then you can link the two, right? 00:10:23.310 --> 00:10:27.950 Yeah, and that's not a mistake in the hash function. That's just the way that 00:10:27.950 --> 00:10:31.130 you're using it. And the solution to that is called a salt, which means that 00:10:31.130 --> 00:10:36.030 every time you store a hash, before you do that, you add something to it, to the 00:10:36.030 --> 00:10:40.310 original text, like just the word salt or something, or one, two, three, four, 00:10:40.330 --> 00:10:45.520 and then you hash it. And then that means that these tables are of no use. 00:10:45.960 --> 00:10:46.440 Right. 00:10:46.680 --> 00:10:52.000 Which means the attacker will have to guess a password and then do this 00:10:52.000 --> 00:10:54.300 calculation. It just makes it more work for the attacker. 00:10:54.800 --> 00:10:59.300 So this reminds me of something that I've been wondering about for a while. 00:11:00.780 --> 00:11:05.400 And somehow I think this is going to be, well, let's see. So I've always wondered 00:11:05.400 --> 00:11:08.920 about this and I've never thought about it for much longer than 10 seconds, but 00:11:08.920 --> 00:11:14.860 maybe you have the answer. So when I enter a password, like I'm creating 00:11:14.860 --> 00:11:19.120 a new account on a website and I'm entering my password there. And then the 00:11:19.120 --> 00:11:23.660 website tells me, no, no, that password is no good because you need to use a 00:11:23.660 --> 00:11:26.880 capital letter and you need to use the number and the special, whatever. 00:11:28.600 --> 00:11:33.700 If the website is only getting my hash, then how do they know I'm not using a 00:11:33.700 --> 00:11:36.280 capital letter or a number or any of that? 00:11:36.560 --> 00:11:40.660 Well, if you're really lucky, then your browser is actually doing these checks. 00:11:41.020 --> 00:11:46.200 But usually, unfortunately, sadly, you are sending the actual password to the 00:11:46.200 --> 00:11:50.580 server. And the server is looking at that password, comparing it. Well, when 00:11:50.580 --> 00:11:53.980 there's a new password, it's just looking at the password. And then it tells you 00:11:53.980 --> 00:11:54.640 what's wrong with it. 00:11:55.500 --> 00:12:00.600 So they're storing it in a hash ideally, but the first time you're creating your 00:12:00.600 --> 00:12:04.480 account, you're actually sending them your text password. 00:12:04.780 --> 00:12:05.980 Yeah, and every time you log in. 00:12:06.610 --> 00:12:11.170 That is sort of the easy answer to my question, and I was hoping that there was a 00:12:11.170 --> 00:12:16.850 more sophisticated solution. I was hoping they had a more sophisticated way of 00:12:16.850 --> 00:12:19.830 dealing with that, but basically you're just trusting them to not. 00:12:20.070 --> 00:12:24.290 I'm sure there are theoretically better ways to do this, but I don't think 00:12:24.290 --> 00:12:26.610 they're used. At least they're usually not used. 00:12:27.210 --> 00:12:31.630 Okay well this was a little bit of an off-topic question. Let's move on to the 00:12:31.630 --> 00:12:36.770 probably more most important question of the of this entire episode here it 00:12:36.770 --> 00:12:42.470 comes. So sure what actually… So we've now explained what a 00:12:42.470 --> 00:12:48.390 hash does, but how does a hash do it? How does a hash 00:12:48.390 --> 00:12:52.130 function actually do it? Like what's going on? What's the magic sauce? 00:12:52.130 --> 00:12:57.590 yes. Well, it really does, to me, just looks like magic. So what happens is every 00:12:57.590 --> 00:13:00.710 piece of text that you put into a hash function, you can just represent as a 00:13:00.710 --> 00:13:05.480 series of zeros and ones. Basically bits. And then what happens is the hash 00:13:05.480 --> 00:13:10.980 function will take little chunks of it. So we'll cut it into chunks, say, of 64 00:13:10.980 --> 00:13:15.680 bits. It will cut the whole text in 64-bit chunks. And then it will say, okay, 00:13:15.700 --> 00:13:19.040 for each chunk, I'm going to look at the first bit and I'm going to flip it 00:13:19.040 --> 00:13:22.720 around and move it to the third position. And then I'm going to take the second 00:13:22.720 --> 00:13:26.260 bit and I'm going to flip that around twice and move it to the first position. 00:13:26.460 --> 00:13:29.660 And I'm going to take the third bit and I'm going to... Not flip it around, but 00:13:29.660 --> 00:13:34.820 compare it to the first bit and say, I want the inverse of the two bits added up 00:13:34.820 --> 00:13:38.640 together or something like that. So you're basically just doing this random, 00:13:38.920 --> 00:13:44.040 well, not random, but specifically designed way of scrambling the original text. 00:13:44.040 --> 00:13:50.200 And you repeat that for every 64-bit chunk. And that results in this this hash. 00:13:50.200 --> 00:13:54.780 Right so this was in this is what I mentioned at the start of the episode this 00:13:54.780 --> 00:13:57.700 was for me kind of probably the most surprising thing but… 00:13:57.700 --> 00:14:00.320 It looks very much like sorcery or like you know whatever. 00:14:00.320 --> 00:14:05.120 Well but the thing is it really isn't. Like it's really just a design that was 00:14:05.120 --> 00:14:11.120 designed by someone with the specific intent of just shoveling 00:14:11.120 --> 00:14:13.840 bits around in a specific way. 00:14:14.260 --> 00:14:19.820 Yeah, but in terms of design, it's not like they have an absolute formula of how 00:14:19.820 --> 00:14:23.500 to make something random. It's just that if you do these operations, people have 00:14:23.500 --> 00:14:26.160 found that that is enough to make it really random. 00:14:27.250 --> 00:14:32.290 Right. Yeah. In my mind, that was some sort of elegant math trick or something. 00:14:32.450 --> 00:14:36.770 But when I was looking into it more closely, it really seemed like more 00:14:36.770 --> 00:14:42.750 mechanical. It is a designed algorithm by someone 00:14:42.750 --> 00:14:48.630 that does something deterministically, but then the results 00:14:48.630 --> 00:14:52.290 should be random as far as we can hope. 00:14:52.770 --> 00:14:56.110 Well, it's predictably random, right? It's pseudo-random, because for anything 00:14:56.110 --> 00:14:59.410 that goes in, you know exactly what's going to come out. So in that sense, it's 00:14:59.410 --> 00:15:04.670 not random, but it looks random if you plot it, for example. If you represent the 00:15:04.670 --> 00:15:09.910 output as a number, then on a chart, then the number might be 1, and then the 00:15:09.910 --> 00:15:14.250 number might be 256, and then the number might be 128, and that should look 00:15:14.250 --> 00:15:15.970 completely random. It should look like noise. 00:15:16.190 --> 00:15:21.210 Right, so that's the way to test it. You basically chart out the results and... 00:15:21.670 --> 00:15:24.570 Just by looking at the chart, it should look random enough. 00:15:24.770 --> 00:15:27.350 Exactly. I'm sure there's a more mathematical way to prove that. 00:15:27.490 --> 00:15:30.170 I guess so, yeah. But that's how you prove it, right? 00:15:31.500 --> 00:15:36.420 Okay, and so there have been a number of hash functions. 00:15:38.050 --> 00:15:40.450 So which one does Bitcoin use? 00:15:40.770 --> 00:15:44.950 Well, Bitcoin uses at least two or three. The most important one, I would say, is 00:15:44.950 --> 00:15:48.130 called SHA-256, which is part of the SHA-2 family. 00:15:49.300 --> 00:15:51.680 Yeah, what does SHA stand for, Sjors? 00:15:52.290 --> 00:15:54.550 If you had asked me that earlier, I would have told you. 00:15:54.550 --> 00:15:59.530 Well, I can tell you because I didn't know either, but it just stands for secure 00:15:59.530 --> 00:16:00.810 hash algorithm. 00:16:01.070 --> 00:16:02.070 Okay, that's impressive. 00:16:04.200 --> 00:16:08.400 The little nuggets of knowledge you get from this podcast. 00:16:08.580 --> 00:16:10.480 And that proves that it is secure. 00:16:10.620 --> 00:16:11.120 There you go. 00:16:11.280 --> 00:16:15.360 Well, we'll get to that there's such a thing as a non-secure hash, which is not 00:16:15.360 --> 00:16:20.600 always a bad thing. But yeah, the SHA-256 one is important. It basically means 00:16:20.600 --> 00:16:26.240 that whatever you throw into it, you'll get back 256 bits of data. 00:16:26.800 --> 00:16:32.000 And that happens to be this, well, the size of a transaction identifier, but also 00:16:32.000 --> 00:16:35.780 the size of a block hash because a block hash is the SHA-256 or actually the 00:16:35.780 --> 00:16:40.880 double SHA-256. So you just do it twice of a block, of the contents of a block. 00:16:40.880 --> 00:16:45.740 Okay before we get because that will be the next and last part of the episode I 00:16:45.740 --> 00:16:50.690 guess is how hashes are used in Bitcoin so we'll get there in one minute. Okay 00:16:50.690 --> 00:16:55.750 but very briefly before we get there can we… So there have been several hash 00:16:55.750 --> 00:17:01.530 functions since so when was the first hash function introduced 00:17:01.530 --> 00:17:02.050 invented? 00:17:02.050 --> 00:17:07.530 As far as I know in the 1970s but I haven't done super deep research, but it's 00:17:07.530 --> 00:17:12.290 not that old. So the I think the general idea was known or suspected that there 00:17:12.290 --> 00:17:17.330 are mathematical operations which this is that will take some input and will 00:17:17.330 --> 00:17:22.110 produce predictably garbage output and that you cannot go back from the output to 00:17:22.110 --> 00:17:23.550 the input so it's a one-way function. 00:17:23.550 --> 00:17:27.890 Hang on you just use the term one-way function. So one-way function is an 00:17:27.890 --> 00:17:28.730 umbrella term 00:17:29.970 --> 00:17:35.510 for mathematical tricks, functions that are easy to calculate in one way and 00:17:35.510 --> 00:17:36.890 harder in the opposite direction. 00:17:36.890 --> 00:17:40.910 I would say impossible in the other direction. So this is distinct from the 00:17:40.910 --> 00:17:45.650 elliptic curve cryptography where it's very hard to go one way. So it's very easy 00:17:45.650 --> 00:17:49.950 to go from a public key to a... Sorry, it's very easy to go from a private key to 00:17:49.950 --> 00:17:53.570 a public key. It's very hard to go from a public key to a private key, but 00:17:53.570 --> 00:17:54.850 mathematically it's possible. 00:17:55.030 --> 00:17:56.050 Wait, so wouldn't you... 00:17:56.050 --> 00:17:57.690 But a hash, it is not possible to go back. 00:17:58.560 --> 00:18:03.420 Right, but this is just how we apply terms. Maybe this is not the most 00:18:03.420 --> 00:18:07.820 interesting thing, but wouldn't you consider both one-way functions? Like 00:18:07.820 --> 00:18:11.880 wouldn't one-way function be the umbrella term for both of these things? 00:18:11.790 --> 00:18:14.640 It could be. I don't know if mathematicians look at it that way, so you'd have to 00:18:14.640 --> 00:18:18.740 ask them. I mean, they are different in the sense that one of them you can, in 00:18:18.740 --> 00:18:23.620 theory, go back from a mathematical point of view. It just goes both ways. And 00:18:23.620 --> 00:18:25.520 these hash functions, really, they don't go back. 00:18:27.060 --> 00:18:32.820 Yeah at least yeah okay got it. So a but anyway so a one-way function 00:18:32.820 --> 00:18:38.520 is an umbrella term at least that's how you use it for mathematical functions 00:18:38.520 --> 00:18:43.340 that can only go one way and then hash is a good example of that. 00:18:43.340 --> 00:18:46.820 Yeah and a very useful example of that, right because a simple one-way function 00:18:46.820 --> 00:18:52.690 could be take the take the input and then output zero always that's that's one way. 00:18:52.690 --> 00:18:54.030 Sure right. 00:18:54.030 --> 00:18:55.070 Not useful yeah. 00:18:55.740 --> 00:19:01.340 Okay, yeah. As far as I know, one-way functions have been sort of 00:19:01.340 --> 00:19:06.790 considered and discussed within the circles of cryptographers since the 00:19:06.790 --> 00:19:12.230 early 70s. It's also what inspired Diffie and Hellman to come up with public key 00:19:12.230 --> 00:19:17.990 cryptography. And then the first hash functions were like the late 70s. 00:19:18.190 --> 00:19:23.910 And that evolved from, I think the first one, was it called MDA? 00:19:24.350 --> 00:19:28.310 Was that, am I saying it right? MDA, yeah. I think that's, that's what I had in 00:19:28.310 --> 00:19:31.080 mind. Possibly the first one or the oldest one. 00:19:31.080 --> 00:19:34.640 It could be much older I think because that's the 1990s and I think there were 00:19:34.640 --> 00:19:36.180 some ones in the 1970s too. 00:19:36.180 --> 00:19:42.060 But okay possibly but then… So where I'm getting at really is so 00:19:42.060 --> 00:19:47.240 where does Bitcoin's hashing algorithm come from? 00:19:47.240 --> 00:19:52.940 Who designed it? How did it develop over the years, because it 00:19:52.940 --> 00:19:58.440 started with MDA I think somehow and then it evolved into SHA. 00:19:59.810 --> 00:20:03.310 Yeah, I mean, it may have started, like I said, with something even older. But 00:20:03.310 --> 00:20:07.590 generally what happens is somebody designs this hash function in the way we just 00:20:07.590 --> 00:20:11.530 described. And then it turns out there's a collision or it's too easy to find 00:20:11.530 --> 00:20:15.910 collisions. And so somebody tweaks, usually just tweaks it a little bit, like 00:20:15.910 --> 00:20:20.490 adds one little operation that flips another bit. You know, it takes the left bit 00:20:20.490 --> 00:20:24.790 and moves it to the right. Just a slight change and then hope, praise that it 00:20:24.790 --> 00:20:29.770 works better. That seems to be, I'm sure I'm not doing it justice here, but 00:20:29.770 --> 00:20:34.030 generally they're constantly based on each other. So the other one that's in 00:20:34.030 --> 00:20:34.710 Bitcoin is... 00:20:34.710 --> 00:20:39.970 I think MDA was designed or at least based on the design by 00:20:39.970 --> 00:20:44.350 Ronald Rivest, which is also the R in RSA. 00:20:45.570 --> 00:20:51.330 One of the three designers of the RSA algorithm. And then that 00:20:51.550 --> 00:20:57.280 design inspired SHA. And wasn't SHA... Designed by the NSA? 00:20:58.340 --> 00:21:00.020 Didn't they have a hand in the design? 00:21:00.300 --> 00:21:04.360 They always have a hand in all these things. I don't know those specific details 00:21:04.360 --> 00:21:06.340 who was involved with what. What I do know is that... 00:21:06.340 --> 00:21:10.180 No, it is. Because while we're discussing this, I have my phone in my hand and 00:21:10.180 --> 00:21:13.620 I'm scrolling through Wikipedia. I did this before the show, but I'm just making 00:21:13.620 --> 00:21:16.280 sure. Yeah, no, it was designed by the NSA. 00:21:16.800 --> 00:21:16.960 Okay. 00:21:17.040 --> 00:21:19.120 Is this something we should be worried about? 00:21:19.280 --> 00:21:21.260 This was the pre-Snowden NSA, right? 00:21:22.580 --> 00:21:27.440 I mean, it was the NSA that was going to war on encryption in the 90s. 00:21:28.200 --> 00:21:31.100 Well, you wouldn't go to war over something if you thought it didn't work. 00:21:33.090 --> 00:21:37.610 But that wasn't about hash functions, I think. That was mostly about public key encryption. 00:21:37.730 --> 00:21:38.250 Yeah, it was. 00:21:38.530 --> 00:21:42.750 But what I was trying to say is that the Europeans didn't trust the NSA, and so 00:21:42.750 --> 00:21:47.370 they came up with this RIPEMD, or I don't know how you would pronounce it, but it 00:21:47.370 --> 00:21:52.390 is basically a variation of SHA-2 or SHA, where they just... If you read the 00:21:52.390 --> 00:21:56.370 original paper, you can see, oh, yeah, we changed a few bit flips here and there, 00:21:56.390 --> 00:21:59.190 but we didn't want to deviate too much from the original algorithm because we 00:21:59.190 --> 00:22:04.270 know it works. But one of the problems is, how do you pick these little bits and 00:22:04.270 --> 00:22:09.950 things and operations? It looks very arbitrary. And that means that if 00:22:09.950 --> 00:22:14.210 you don't trust the person who made it, you might be worried that they build in 00:22:14.210 --> 00:22:18.030 some sort of sneaky backdoor, some vulnerability that they know, but you don't 00:22:18.030 --> 00:22:22.230 know. And then one way to defend against that is, Well, there's two things. One 00:22:22.230 --> 00:22:25.430 is that the NSA would say, well, this is how we, you know, how we designed it. 00:22:25.510 --> 00:22:30.190 And that's all arbitrary. But here's like a series of numbers of like one plus 00:22:30.190 --> 00:22:34.350 five plus 17 plus like 15 different, like a very elegant looking series of 00:22:34.350 --> 00:22:38.870 numbers and say, well, if you just add this to it and then it still works. And 00:22:38.870 --> 00:22:43.130 that kind of proves like, well, these numbers are so elegant that they probably 00:22:43.130 --> 00:22:48.220 were chosen randomly or something. That's one way. And the other thing is you 00:22:48.220 --> 00:22:52.410 just change it— that they were… Or that the end result could not have been 00:22:52.410 --> 00:22:56.670 tampered with. Because you're saying you're adding so much stuff to it that is 00:22:56.670 --> 00:22:59.930 very hard, that makes it more difficult to design a malicious system. But it's 00:22:59.930 --> 00:23:03.790 still a bit hand-wavy. And the other thing you would do is what the Europeans 00:23:03.790 --> 00:23:07.210 did, I guess. They just changed it a bit. Not too much, but a little bit, so that 00:23:07.210 --> 00:23:09.210 any backdoor that might have been in there would be broken. 00:23:09.570 --> 00:23:09.770 Right. 00:23:11.210 --> 00:23:11.570 Okay. 00:23:13.160 --> 00:23:18.220 Let's move on to... what this podcast is 00:23:18.220 --> 00:23:24.040 or at least promises to be about in in the title… How does 00:23:24.040 --> 00:23:25.140 hashes… 00:23:26.400 --> 00:23:32.680 A hash is really maybe the core fundamental building block for Bitcoin, wouldn't you say? 00:23:32.960 --> 00:23:34.560 Well, it's certainly quite useful. 00:23:35.420 --> 00:23:40.240 It's used here and there. So where are hashes used? How 00:23:40.240 --> 00:23:42.980 do hashes help Bitcoin? 00:23:43.180 --> 00:23:47.300 Well, for one thing, every block, if you take the entire contents of the block, 00:23:47.520 --> 00:23:52.190 that's hashed. And, you know, descriptively call it the block hash. 00:23:53.310 --> 00:23:57.370 And it turns out that this hash of the block, well, first of all, the block 00:23:57.370 --> 00:24:01.710 itself consists of transactions. And so every transaction also has a hash. So 00:24:01.710 --> 00:24:02.690 that's two use cases. 00:24:02.850 --> 00:24:03.770 Which is its transaction ID. 00:24:04.050 --> 00:24:09.550 Yeah. Now blocks link to each other. So one block links to the previous block. 00:24:09.730 --> 00:24:13.180 And the way it links to that block is using the hash of that block. Because the 00:24:13.180 --> 00:24:16.240 hash itself of a block is not in the blockchain. The block is in the blockchain. 00:24:16.600 --> 00:24:19.460 And then your computer just reads the block and then figures out, okay, this is 00:24:19.460 --> 00:24:22.760 the hash. And so the next block will mention that hash, and that's how you know 00:24:22.760 --> 00:24:26.660 that the next block actually builds on top of the first one. This, again, is why 00:24:26.660 --> 00:24:30.440 you really don't want collisions, right? Because if it was possible to create 00:24:30.440 --> 00:24:34.440 another block with very different transactions in it with the same hash, you 00:24:34.440 --> 00:24:35.040 would have a problem. 00:24:38.920 --> 00:24:42.320 Right. What would be the problem exactly? 00:24:42.680 --> 00:24:47.980 Well, let's say the first block creates 25 Bitcoin and sends it to you. 00:24:48.610 --> 00:24:52.150 And everybody accepts that block and builds on top of it. And that means you will 00:24:52.150 --> 00:24:56.270 have that Bitcoin forever, essentially, until you decide to move it. But now I 00:24:56.270 --> 00:25:02.030 come in and I create a new block where that 25 Bitcoin 00:25:02.030 --> 00:25:03.170 actually goes to me. 00:25:05.280 --> 00:25:10.680 I find a way to produce the same hash of the block, which is impossible. But if I 00:25:10.680 --> 00:25:15.000 were able to produce the same hash, then I could send everybody else my block and 00:25:15.000 --> 00:25:19.560 they would look at the hash and say, okay, that's definitely a valid hash of that 00:25:19.560 --> 00:25:23.920 second block in the blockchain. So all the blocks are still connected. So that 00:25:23.920 --> 00:25:26.700 means that existing users wouldn't be tricked in this case, right? Because they 00:25:26.700 --> 00:25:30.140 would already have the block and they have no reason to replace it. But any new 00:25:30.140 --> 00:25:33.560 person downloading the blockchain would be confused because they would just get 00:25:33.560 --> 00:25:38.720 the fake block. But it's not fake in any sense that the Bitcoin software cares 00:25:38.720 --> 00:25:42.780 about because it would have the same hash. So there would be two different blocks 00:25:42.780 --> 00:25:46.440 with the same hash with very different events. And that would be a complete 00:25:46.440 --> 00:25:46.880 disaster. 00:25:47.240 --> 00:25:53.210 Right. So hashes are used to link blocks together into a blockchain. And then 00:25:53.210 --> 00:25:57.410 the other thing about a block hash is that it also... 00:25:58.790 --> 00:26:01.910 determines whether the block itself is even valid in the first place. 00:26:02.170 --> 00:26:03.330 Well, that's the proof of work part. 00:26:03.590 --> 00:26:03.790 Right. 00:26:03.970 --> 00:26:09.210 So what happens is that this hash is just 256 bits. And so what you can do, and 00:26:09.210 --> 00:26:13.650 that's what proof of work essentially does, is it says the first several bits 00:26:13.650 --> 00:26:17.670 need to be zero, or roughly, like the whole hash needs to be lower than a certain 00:26:17.670 --> 00:26:21.690 number, but it doesn't really matter. You could see it as, okay, so we're going 00:26:21.690 --> 00:26:26.710 to require that the first 10 bits of the block are zero. 00:26:27.490 --> 00:26:31.110 And then you can, because the only way to figure out what a hash is is brute 00:26:31.110 --> 00:26:35.990 force, the only way to get 10 zeros is to just try two to the power of 10 00:26:35.990 --> 00:26:41.190 different things. And so the thing you would change in the block, you'd have to 00:26:41.190 --> 00:26:44.270 change something in the block to get a different hash. And so one of the things 00:26:44.270 --> 00:26:46.370