Aaron van Wirdum Live from Utrecht, this is the Van Wirdum Sjorsnado.
Transcript
Sjors Provoost That's right. Hello.
Aaron van Wirdum Sjors, welcome back.
Sjors Provoost Thank you.
Aaron van Wirdum You confirmed your status as Bitcoin Core Elite Developer this week.
Sjors Provoost Absolutely. Thanks to me, there has never been a more amazing way to send Bitcoin. This is absolutely groundbreaking.
Aaron van Wirdum It changed everything.
Sjors Provoost It changed nothing.
Aaron van Wirdum Okay, to be clear, this is not what the episode is going to be about. The episode is going to be about libsecp256k1 library.
Sjors Provoost That's right.
Aaron van Wirdum But I want to know about this game-changing technology that you just built and implemented. What is it? What did it do? How did it change the world?
Sjors Provoost So there are many ways to send Bitcoin, but one of them is you download the Bitcoin Core software and then it gives you a really nice graphical interface. You click a bunch of buttons and poof.
Aaron van Wirdum So far so good?
Sjors Provoost So far so good, but there's also a command line way to do it. And especially if you need to do anything advanced, like using it with a hardware wallet, you'll have to use the command line. And it turns out the command line doesn't do exactly what I want it to do. And so then you just make a pull request to change it.
Aaron van Wirdum What did you want to do exactly that it didn't do?
Sjors Provoost Well, in this case, originally, I wanted to add PSBT support to...
Aaron van Wirdum Partially signed Bitcoin transactions.
Sjors Provoost Exactly, which we discussed in the first episode. I wanted to add that to the standard way of sending. But the problem is this command line stuff is used by a lot of automated systems. So a lot of Bitcoin exchanges, they might be running Bitcoin Core, and they might be relying, or Bitcoin ATMs, they might be relying on these methods. Which means if you change them, you have to be very, very careful not to suddenly break a million Bitcoin ATMs. Any change to the existing method will run into that type of problem. You can usually add stuff but then it becomes super ugly because you just add another argument and another argument and so it just becomes tedious to use. So ultimately I decided to just create a new send method. So now there's like five different ways to send from the command line.
Aaron van Wirdum Right.
Sjors Provoost But there was a couple that were really designed from the beginning just to send coins to somebody else. And there was a couple designed for PSBT specifically. But those are very inconvenient to use if you just want to send something and click OK on a hardware device.
Aaron van Wirdum Right.
Sjors Provoost So this one is future proof. It should be able to send coins very quickly. But if you want to then expand Bitcoin Core to work with hardware wallets, it's like a one line change so that it also works for that. Just kind of why I built it. So it was a prerequisite of the one of the many, many, many prerequisites to having easy to use hardware wallet support.
Aaron van Wirdum Right. Yeah, you've done a bunch of work on hardware wallet integration for Bitcoin Core, haven't you?
Sjors Provoost A little bit, yeah. I think the bulk is still Andrew Chow. He's been completely rewritten the wallet. I like to test this stuff and review it. But yeah, I'm using the library, the HWI library that he wrote, which is basically just a collection of drivers from all the different hardware wallets. So all the different hardware wallets, they usually have some Python code, but that also supports all their shit coins. and this we basically just took the essential parts of the code With just Bitcoin support.
Aaron van Wirdum Right.
Sjors Provoost And just shoved it into one little project that you can download.
Aaron van Wirdum Right. So yeah, and this new, the Ultimate Send RPG, what was it called?
Sjors Provoost RPC? Yeah, the Ultimate Send RPC is the name. I think it's going to be marked experimental because there'll probably be plenty of bike shedding of what the exact interface should look like.
Aaron van Wirdum Right. But it's a small step towards hardware wallet integration.
Sjors Provoost It's one of a hundred small steps, yes.
Aaron van Wirdum Great. Thank you, Sjors.
Sjors Provoost Yeah, I was a bit surprised that it got merged as quickly as it did.
Aaron van Wirdum Oh, should we be concerned? Should we be worried?
Sjors Provoost No.
Aaron van Wirdum Okay. We're going to discuss libsecp256k1. Thank you. Why are we going to discuss it? We are going to discuss it because BIP340 support was merged into libsecp256k1 this week.
Sjors Provoost What was merged?
Aaron van Wirdum Shut up.
Sjors Provoost Schnorr was added.
Aaron van Wirdum Yes.
Sjors Provoost Okay. Yeah.
Aaron van Wirdum So Schnorr. Exactly. Thank you. For actually making it clear for our listener. Oh, I misunderstood your question. Yeah, Schnorr was added.
Sjors Provoost Yes.
Aaron van Wirdum So, libsecp256k1 is a library.
Sjors Provoost That's right.
Aaron van Wirdum And we're going to explain what this library actually is or why it exists or what it does. And the reason we're going to explain that is because I actually didn't know that much about it. It's one of these things for me that I heard about and I kind of know what it is, but I never really... got into it to any sort of serious extent. Okay, libraries, first of all. Let's start with libraries.
Sjors Provoost Let's talk about libraries.
Aaron van Wirdum There's a thing called software libraries.
Sjors Provoost Yes.
Aaron van Wirdum And I'll just let you explain what a software library is, first of all. So for any programmer that's listening, this is probably going to be very, very new-ish for you, but for people like me, this is actually kind of interesting.
Sjors Provoost The easiest way to describe a library is it's a reusable piece of software. So... For example, OpenSSL is a library we'll talk about. It is a piece of software that lets you do all sorts of cryptographic operations from creating random numbers to signing stuff with every curve under the sun. But it's not an actual program. It doesn't really do anything by itself. But other programs can use a library to do whatever they want without having to rewrite that stuff.
Aaron van Wirdum Or I assume you can take a part of the library, not necessarily the whole library, but get a specific...
Sjors Provoost You take the entire library, but you use a subset of it.
Aaron van Wirdum Exactly, yes. So Bitcoin was at some point in the past relying on OpenSSL?
Sjors Provoost Yeah, until actually very recently, a few months ago. But for less and less and less stuff. So in the beginning, OpenSSL was used for all the things. In particular, the reason it was needed is because Satoshi picked a cryptographic curve, the secp256k1 curve, because it was pretty. And OpenSSL had support for it. So he did not have to write all this cryptographic functionality, which, of course, you never want to do yourself. It's very dangerous to write your own cryptographic stuff. And this is also a reason why he didn't use Schnorr, because there was no library for it. There were other reasons, but this was a reason, a practical reason.
Aaron van Wirdum So just to be clear, when you say Bitcoin Core or Satoshi use this library, the OpenSSL library, how does a software program actually use a library?
Sjors Provoost You just Google on Stack Overview how to use OpenSSL, and then you just look at the examples.
Aaron van Wirdum Let me rephrase the question. Where is the library?
Sjors Provoost oh the library is included in the software package when you download it right so in this binary file contains some of the bitcoin core specific stuff and then a whole bunch of libraries and that's what makes it so big, or like 20 20 megabytes.
Aaron van Wirdum Right. So when you download Bitcoin Core the software, Bitcoin Core 20 is the newest one I guess.
Sjors Provoost Yep.
Aaron van Wirdum Then you actually download well in this case not OpenSSL anymore but you... For Bitcoin 19, you actually downloaded the whole OpenSSL library.
Sjors Provoost Yeah, that's correct.
Aaron van Wirdum And then it's hosted on your computer from that point on. You have the library on your own computer.
Sjors Provoost There's two ways to go about that. You can have a library sitting on your computer already. And then software can say, let me just see if I can find that library. And I'll use that. Then your download gets smaller. But the problem is that libraries change. And so you don't want to be surprised by what's on the computer, especially with cryptographic stuff.
Aaron van Wirdum Right.
Sjors Provoost And even if you include it in the download, you can be surprised by what happens to the library because somebody else is maintaining that library. If you're not paying attention to what that other person is doing, they might break something. Very bad.
Aaron van Wirdum Right. So in the case of let's stick to Bitcoin 19 and Bitcoin core 19.
Sjors Provoost Well, in this case, maybe take an older one because I think it was Bitcoin core 0 .8 or something.
Aaron van Wirdum Let's take Bitcoin core. I don't know where you're going with this, but let's take that one.
Sjors Provoost Yep.
Aaron van Wirdum So someone else is maintaining this library.
Sjors Provoost Yeah.
Aaron van Wirdum Bitcoin core developers are maintaining Bitcoin.
Sjors Provoost Yep.
Aaron van Wirdum They write something in the code. They use some part of the library. You download the library from the Bitcoin core code. a part of libraries used and then the Bitcoin Core developers may not have noticed some change that happened in the library and all of the sudden the stuff that they wanted Bitcoin Core to do isn't actually doing what they wanted Bitcoin Core to do because the library wasn't doing what they thought it would do because someone else was maintaining the library. Is that a correct summary?
Sjors Provoost Yeah, that's right. And to clarify what specifically happened here.
Aaron van Wirdum You picked Bitcoin Core 8 because there was a specific example you wanted to go to.
Sjors Provoost Yeah, I might be wrong about the number because Bitcoin Core 8 had a different problem. But sort of around that time, there was another bug in OpenSSL that I believe was unrelated to the problem that happened. But they basically had to upgrade OpenSSL because the old version was simply not safe. Unbeknownst to the core devs, there was another change in OpenSSL when they upgraded. And in particular, this was about when you see a signature, do you consider it valid or not? And the original version of OpenSSL was pretty relaxed. So it would accept signatures as valid, even if they did not meet the exact spec. Now, they wouldn't be signed by somebody else, so it wasn't about stealing funds. But you could be a little bit sloppy about maybe you add a byte to the signature or maybe not. So the notation could be a bit sloppy. And the new version was very picky. Now, if you use Bitcoin software to create a transaction, that was not a problem. Because any Bitcoin transaction was signed very strictly, according to the protocol. But if you're now validating these transactions, if you use old software and you would see a sloppy version that was made with some other piece of software, the old software would be fine. The new software would say it's invalid. So all of a sudden you have an accidental soft fork.
Aaron van Wirdum Right. And that's what actually happens.
Sjors Provoost Well, yes.
Aaron van Wirdum That's the BIP-66 one?
Sjors Provoost Right.
Aaron van Wirdum Is that what we're talking about here?
Sjors Provoost Correct. BIP-66 was introduced. Because people became aware of this problem. At least some of the developers became aware of this problem. So they knew there was an accidental software time bomb, basically, in the code. And so they proposed BIP66 as saying, oh, by the way, we should be more strict about what these signatures look like. Without saying, oh, by the way, there's a bug in OpenSSL, so we'd better do this now.
Aaron van Wirdum Oh, it was like a secret bug fix of a problem with OpenSSL? I don't think I knew that. Okay.
Sjors Provoost Well, yeah. I mean, OpenSSL essentially improved itself by becoming more strict, but that made it a consensus change.
Aaron van Wirdum Right.
Sjors Provoost Because what's consensus code? It's also whatever your libraries are doing.
Aaron van Wirdum Right. Yep.
Sjors Provoost So basically, OpenSSL introduced the soft fork, but without saying, oh, you know, there's no deployment date in the OpenSSL update. It just randomly happened.
Aaron van Wirdum Right. So that's a great example of why a dependency, because that's the official term, is a problem.
Sjors Provoost Exactly.
Aaron van Wirdum This is a good example of that. And there have been more problems with OpenSSL, I think.
Sjors Provoost I mean, OpenSSL is famous for vulnerabilities. And, you know, the main, I think, big reason behind that is that these libraries are used by everyone for decades, but they're only maintained by one guy in Germany who doesn't get funded.
Aaron van Wirdum Right.
Sjors Provoost Just like curl, I think, is another famous example of that. It's like a library that downloads files. curl, it's used everywhere. It's probably used in a space shuttle, but there's just one guy that maintains it and nobody's helping.
Aaron van Wirdum Right.
Sjors Provoost And it's not good when the entire internet relies on it. And in the case of OpenSSL, yeah, there have been plenty of bugs, and it's very easy to make mistakes with cryptographic code, you know.
Aaron van Wirdum Yeah.
Sjors Provoost You forget, and it's written in C, so you forget a semicolon. Whoops.
Aaron van Wirdum Right.
Sjors Provoost Now you're skipping a line. So one of the bugs that was called Heartbleed.
Aaron van Wirdum Yeah, that was fairly recent, a couple of years ago.
Sjors Provoost Yeah, a couple of years ago. I think it was a missing colon or like literally just a one character mistake.
Aaron van Wirdum Yeah. And it affected basically.
Sjors Provoost Yeah, it allowed you to log into any computer on the internet.
Aaron van Wirdum Affected everything.
Sjors Provoost Without a password.
Aaron van Wirdum Yeah. Right.
Sjors Provoost That's a sort of severity. And something like that in Bitcoin, of course, you know, could mean, oh, now we have a problem. Everybody can just steal all the money. So eventually, well, at the same time, Pieter Wuille was working on a library.
Aaron van Wirdum For our American and English listeners, that's Pieter Wuille, or however they want to pronounce it.
Sjors Provoost Yeah, or sipa.
Aaron van Wirdum Pieter Wuille, go on.
Sjors Provoost He was working on a library, so a piece of software, that was specifically designed to create and verify Bitcoin signatures. And his original motivation was just to do it faster than OpenSSL.
Aaron van Wirdum Okay. So it wasn't a security motivation. It was just a performance improvement motivation.
Sjors Provoost Exactly. He explains this in a podcast he did with Chaincode. So if you Google that, it might be in the show notes. Basically, he wanted to make it, I think, about four times faster. And he could try and modify the OpenSSL code itself. But apparently it's like such a nightmare to change any of that code. And also... The OpenSSL code is very generic. It has to support all different kinds of cryptography. So it's more difficult if you want to change anything, you have to be very abstract in all the things you do. It's just like when you write a law. You can't just say, John can't go to the supermarket. You have to say something like, well, anybody over the 20 centimeters in size cannot go to a supermarket. So it's very difficult to write these abstract documents. So he basically wrote it from scratch specifically for that curve. And it was added to Bitcoin Core, I think pretty early, but just to verify signatures and then later on also to create signatures. And that coincided with this security vulnerability. So it was a good, but I don't think it was a cause of it. It was sort of around the same time. It's like, okay, we've had this near miss, like we could have had a serious problem. Let's not use OpenSSL for that critical stuff anymore.
Aaron van Wirdum Yeah. And so then the goal was to get rid of that. Dependent. Now I forgot the word.
Sjors Provoost Dependency.
Aaron van Wirdum Exactly. And write a whole new cryptographic software library for Bitcoin.
Sjors Provoost Right. Just the curve.
Aaron van Wirdum Just an elliptic curve. Just the thing that's used for signatures. Yeah.
Sjors Provoost Because there's other cryptographic code in the Bitcoin core code base. For example, SHA-256 is in there and a few other curves. And I think those were originally also from OpenSSL. But those things are a little bit less scary. Like you can implement SHA-256 in a day. If you're bored in any programming language.
Aaron van Wirdum Does it still use libraries for that though? Or was that rewritten?
Sjors Provoost No, so SHA256, as far as I know, is... directly in the code so it's just it's just copy pasted from somewhere and then improved.
Aaron van Wirdum Right, got it. Okay, so libsec 256 am I saying that right?
Sjors Provoost libsecp256k1
Aaron van Wirdum Thank you. that was to get rid... So it was meant as a performance improvement then it was pivoted to actually be a new library for bitcoin or at least sort of bitcoin specific library to get rid of this dependency you mentioned this before but Isn't that also a risk? Like rolling your own crypto?
Sjors Provoost Absolutely. So the fact that this thing was, like this was reviewed by a lot of people, a lot of good cryptographers before adding it. And I think it was also compared against OpenSSL in terms of like using the same tests. But yeah, at some point, I mean, at some point you have to take that risk because the other one is like just waiting for OpenSSL to explode.
Aaron van Wirdum Plus it was Pieter Wuille there, so can't really go wrong with that.
Sjors Provoost Well, you know, you don't want to have proof of Wuille. But a lot of very smart people looked at it. Probably the same people who would also look at OpenSSL. So that's good. But you don't want to make a habit of this. And in fact, they do constantly make very small tweaks to that library to make it a little bit faster. But you want to be very careful with that.
Aaron van Wirdum Right. Okay, so that's the library. Bitcoin has its own library now. Is this used by any other programs?
Sjors Provoost But keep in mind, it's turtles all the way down, right? Because OpenSSL is also just written by people.
Aaron van Wirdum Sure.
Sjors Provoost So everything is an implementation at some point.
Aaron van Wirdum Sure, sure.
Sjors Provoost Okay. So your question?
Aaron van Wirdum I guess my first question would be, is this library used by anything other than Bitcoin?
Sjors Provoost Yes. So this library is, I just heard it on a podcast with Vitalik. It's also used by Ethereum. So, and a whole bunch of other cryptocurrencies. Basically any cryptocurrencies that uses the secp256k1 elliptic curve. which is just a nice mathematical object.
Aaron van Wirdum Right. Mostly cryptocurrencies, though. Only cryptocurrencies. It's pretty cryptocurrency-specific, at least.
Sjors Provoost Yeah, I'm not aware of any non-cryptocurrency project that uses it. It could, right? It's just a library that allows you to sign stuff, sign messages, and verify the signature on a message. So you could write an encrypted chat application that uses this curve if you wanted to. But I don't know. I guess the encrypted chat applications out there might have their own curve that they use. for their thing. I don't know what Signal uses but they could yeah.
Aaron van Wirdum Okay, so that's libsec. I keep having to pronounce this.
Sjors Provoost Yeah, we just splice it in the audio later.
Aaron van Wirdum I'll just call it libsec. Is there anything else that's called libsec that would confuse people?
Sjors Provoost libsecp.
Aaron van Wirdum libsecp. Okay, so libsecp. Is that everything we need to know about libsecp?
Sjors Provoost I think so.
Aaron van Wirdum Yeah. Okay, so BIP340 was merged, which is Schnorr.
Sjors Provoost Exactly.
Aaron van Wirdum This has been in development for a long time as well.
Sjors Provoost Yep.
Aaron van Wirdum I think for years, right?
Sjors Provoost Correct.
Aaron van Wirdum So this is also a new implementation of Schnorr. Is this the first time Schnorr has been included in any library? Because you just mentioned that it wasn't.
Sjors Provoost I don't know about any library, but at least at the time when Bitcoin was created, there was no library for Schnorr. Or at least it wasn't in OpenSSL, which is like a widely tested library. You wouldn't just want to randomly download, oh, look, somebody implemented Schnorr.
Aaron van Wirdum Right.
Sjors Provoost So what happened is, I think Satoshi was aware of Schnorr, but there was a patent on it. And there was no implementation. So it's kind of both of these things. Because I think the patent was actually expired in 2008.
Aaron van Wirdum I think it just lapsed or something, yeah.
Sjors Provoost But either way, you don't just want to write the stuff from scratch. And if you're trying to develop a world-changing thing, you don't want to then spend three years just implementing the cryptography, given how long it takes to really do this. But actually, Schnorr is simpler. And I think we may have explained this in an earlier episode.
Aaron van Wirdum You mean simpler than?
Sjors Provoost Than ECDSA.
Aaron van Wirdum Which is the elliptic curve algorithm that Bitcoin currently uses.
Sjors Provoost Right, and which the libsecp library implements. But the thing is, you have the same elliptic curve, but then in order to make a signature, you have to do slightly different... calculations with it. So that also means that the change for Schnorr is not as complicated as, say, the initial version of libsecp was. The initial version of libsecp had to implement the curve, all the operations you can do on a curve, like addition and multiplication, and then implement the signature algorithm of ECDSA. But in order to do Schnorr, you just need to do the signature algorithm for Schnorr. You don't have to do all the math, the basic foundational math. So it's not a huge change. It's not like adding a whole new curve to it. It would be much more difficult to add, say, a different elliptic curve or even a completely different kind of curve. than it is to change just from ECDSA to Schnorr. It's a different way of signing. And in fact, a simpler way of signing.
Aaron van Wirdum So this was implemented again by Pieter Wuille, I assume. Well, I know. Or, right?
Sjors Provoost The spec was written by him. I think he also wrote most of the implementation, but there's a lot of people on top of that.
Aaron van Wirdum Sure. And that was merged this week. So what does that mean exactly? Where does this get us?
Sjors Provoost Right. So what that means is there now is an updated version of this library. But nobody is using that library yet. And another change is that Bitcoin Core was changed, I think, a few days ago to include that new version of the library.
Aaron van Wirdum Right.
Sjors Provoost To include it, not to actually use it in any way.
Aaron van Wirdum So the first major release of Bitcoin Core, when you download that, you'll download the library that includes Schnorr.
Sjors Provoost Exactly. Because the usual process is stuff gets merged into the master branch on GitHub. And every six months or so, we say, okay, let's stop at this point and release whatever is in there. And so next time that'll include the Schnorr code. Yeah, it'll be in there. It might not do anything. It might have a few tests that try it. If you don't run the tests, you're not going to run it.
Aaron van Wirdum Yeah, the next Bitcoin Core release is not going to use Schnorr yet, is your prediction here. That's your bold prediction.
Sjors Provoost I would say it would be extremely reckless if it did. But there are sources that use it, right? A certain Bcash coin uses Schnorr, I believe.
Aaron van Wirdum Oh, yeah, I think so.
Sjors Provoost But the actual spec for Schnorr was changed a little bit. So I don't know if they're going to change along with it or not. Not a huge change.
Aaron van Wirdum So anyways, it's going to be included in the library. Next time you download it, you're downloading this. But it doesn't actually do anything probably or not anything too important.
Sjors Provoost Right.
Aaron van Wirdum But that would be a next step then. I want to excite our audience, Sjors. We're getting somewhere, right?
Sjors Provoost Yeah, we are.
Aaron van Wirdum That's the plan, right?
Sjors Provoost So the idea here, of course, is to have Schnorr as part of Taproot. So the entire Taproot thing, there are already pull requests that describe what it's supposed to do that are not completely finished, but pretty far along. So maybe they'll go in the next version, so not in the upcoming one, but in the next one. What I would imagine happens is that it gets added not to mainnet, probably not even to testnet, but to this new thing called Signet, which is a whole new... type of way to do testnet, which we can do another episode about. But basically, it'll go in some innocent way. So maybe there's tests for it, you know, tests for everything Taproot-related, and then anybody who knows how to compile code can just flip a switch and try it on their own machine. But it won't be on mainnet, or probably not even on testnet. And then maybe next version, you know, this stuff takes time. And then even though it's in there...
Aaron van Wirdum Sure, this is not exciting our audience.
Sjors Provoost Well, our audience...
Aaron van Wirdum Gotta pump it. Gotta pump this coin.
Sjors Provoost I'm pumping low time preference. This stuff takes a long, long time. But basically, you add all the code in it, then you make... So everything is in there, but you don't activate it yet. And then the next time... You decide on activation mechanisms, and even those mechanisms might take a while.
Aaron van Wirdum That's a whole debate on its own, which we did an episode about, right?
Sjors Provoost Yes.
Aaron van Wirdum If I'm not misremembering. Okay, so that's what a library is. That's what a libsecp256k1 library specifically. Now, you also know what Schnorr is. Well, actually, we didn't even get into what Schnorr is, but we get... Did we do that in a previous episode? Sure, go for it.
Sjors Provoost It's simpler.
Aaron van Wirdum What Schnorr actually...
Sjors Provoost So what happened is there was this patent on this very simple system called Schnorr by a person called Schnorr. And it was very nice. It was a good way to make electronic signatures, but it was a patent on it. So people came up with a way to convolute the design, make it more complicated, such that it would no longer fall under the patent. So when the lawyer said, okay, this looks obscure enough, so they were just adding numbers to it and subtracting things, just making it more complicated. And then it didn't violate the patent, and so they shipped it. But now we ended up with this horrible thing. That is basically proof of lawyer, convoluted mess. And now that the patent has expired, we just go right back to the original design, which is much better. And mainly it's better because you can add signatures much more easily. And adding signatures is very nice.
Aaron van Wirdum Yeah, you could perform math on it.
Sjors Provoost Yeah, you could perform math on the original one, but you'd be able to publish papers just on the ability to add two numbers.
Aaron van Wirdum Right. Yeah, so for the layman listener, you know, performing math on it just means you can do cool mathematical tricks, like add numbers to both the signature or both the public key and the private key, and then it still adds up and still works, or you can add signatures or like all that kind of cool stuff.
Sjors Provoost Yeah, which in the end translates to more privacy and less block space usage. So it's all good.
Aaron van Wirdum Nice. And that's included in the libsec. This is the last time I'm going to pronounce this. libsecp256k1 library. And that means we're getting ready. We're getting ready to pump … [unclear] the bull’s warming up. Is that right?
Sjors Provoost I'm sure it is.
Aaron van Wirdum Okay, then I think this was the episode, wasn't it?
Sjors Provoost Yep. Thank you for listening to the Van Wirdum Sjorsnado.
Aaron van Wirdum There you go.