Episode 30: Hardware Wallet Integration in Bitcoin Core and HWI

Aaron van Wirdum Before the show, a quick word from our sponsor. Live from Utrecht, this is the Van Wirdum Sjorsnado.

Sjors Provoost Hello.

Aaron van Wirdum Hey Sjors.

Sjors Provoost What's up?

Aaron van Wirdum Are you running the BIP 8 true independent client yet?

Sjors Provoost Negative. I did not even know there was one.

Aaron van Wirdum Yeah, one has been launched, started. I don't think it's actually a client yet. A project has started.

Sjors Provoost Okay, a project has started. It's not a binary or a code that you can compile.

Aaron van Wirdum No, not yet. But I did see you were reviewing the BIP 8 code a little bit.

Sjors Provoost Yes. I mean, it's… it's a, you know, pull request that has a bunch of changes in it. A lot of them are totally not controversial. So I'm just working on it from top to bottom.

Aaron van Wirdum Just code review.

Sjors Provoost Code review, and I've also started my own pull request where I've taken a couple of commits from that just to split it up. Makes everybody's review life… life easier. Right. If everybody's talking about LOT is true or LOT is false on a pull request, but there's a lot of nitty-gritty stuff that you really don't want to screw up just because you're talking about one little thing.

Aaron van Wirdum That's good.

Sjors Provoost So that's all good, and we'll see where that ends.

Aaron van Wirdum And I think you were working on something else as well, and that's actually what this episode is going to be about.

Sjors Provoost By the way, if… if there are independent clients, just be sure to listen to our earlier episodes about open source and deterministic builds and all the things that can go wrong there, and then do it right.

Aaron van Wirdum Other project, other thing, the other thing you're working on.

Sjors Provoost Oh, yeah, hardware wallet stuff.

Aaron van Wirdum Yeah, you're working on not just you, but... I think you're one of the core devs working on hardware wallet integration for Bitcoin Core. Is that correct?

Sjors Provoost Yeah, that's right. It's me. It's Andrew Chow, instagibbs, as he likes to call himself on GitHub.

Aaron van Wirdum Yeah, and you've made some good progress. That was a new merge this week. And we're… Basically going to discuss hardware integration in Bitcoin Core in a general sense in this episode, I think, right?

Sjors Provoost Yep.

Aaron van Wirdum So basically your top specialty, maybe.

Sjors Provoost Definitely one of my favorite topics.

Aaron van Wirdum You wanted to refer to some previous episodes where we discussed some of this stuff, I think.

Sjors Provoost I mean, we talked in the first episode about hardware wallets, sort of the security things you need to worry about when it comes to change, etc, so that might be a nice refresher. Apologies for the atrocious audio in the first episode. There was a literal tornado. I think in episode 7 we explained to Peter McCormack but also to other listeners what an xpub is, which is quite relevant when it comes to hardware wallets because an xpub represents a whole bunch of addresses. And hardware wallets like to communicate that. And then there was episode 21, which I don't know what it was about, but you should just listen to all of them.

Aaron van Wirdum You put it in a show notes so it has some relevancy.

Sjors Provoost That's right.

Aaron van Wirdum Okay, Sjors, where are we with hardware wallet integration for Bitcoin Core? Let's just start from the top.

Sjors Provoost Yeah, so there was a lot of things that needed to happen, and mainly... that which has been done a while ago is to rewrite the entire wallet that Bitcoin Core comes with because it was atrocious and that's mostly been done by Andrew Chow. I've done a lot of testing and reviewing for that.

Aaron van Wirdum And, what was the problem? What was so bad about it?

Sjors Provoost Well it was started by Satoshi and it just had pay to public key addresses in it and then pay to public key hash I guess was added and then those other things added and then SegWit was added and it just became an unmaintainable mess. So all that unmaintainable mess has been put in a box and that box has been abstracted in a way that you can create another box that has the same functionality, but the implementation can be rewritten from scratch. And so that's kind of what we did. We put the existing stuff in a box because we don't want to understand it anymore. And we created a new box to replace it with the same form. And that uses descriptors. And I guess we'll dedicate some other episode in explaining what output descriptors are. But they're kind of a nice, elegant way to describe how to derive your addresses from, for example, from the seed. So we say, okay, take the seed and then... Take the first account, and then here's an xpub, and then take these address derivations, and then I want you to make it a native SegWit address. You can express that in one string. And the reason that's nice, and that gets just to the hardware wallet part, is because a hardware wallet can then spit out that information. So a hardware wallet can tell you, okay, these are the addresses I have, and then you just import it. And when your hardware wallet is disconnected, you can… you know, exactly which… which addresses the hardware wallet has, and you only need to connect when you want to sign something.

Aaron van Wirdum So would the hardware wallet in that case spit out the xpub? Is that why you mentioned xpub just now?

Sjors Provoost Yeah, well, actually, currently, indeed, it spits out xpubs. Ideally, it would spit out these descriptors. So because a hardware wallet might not understand what Taproot is, or it might not understand even what SegWit it is. And if it just gives you an xpub, well, that xpub doesn't really tell you anything. It's just a way to derive addresses, but it doesn't tell you what kind of things it can do. So you really want some way for a hardware wallet to tell it what it's capable of. But for now, we've avoided having to think about that, using a project called HWI.

Aaron van Wirdum Yes, I've seen it a lot, HWI. What does it stand for?

Sjors Provoost Hardware Wallet Integration,

Aaron van Wirdum Nice.

Sjors Provoost Or interface, I guess…

Aaron van Wirdum Or which one is it, Sjors?

Sjors Provoost I don't care. I think it's interface.

Aaron van Wirdum Okay.

Sjors Provoost So that is a Python library. And again, mostly by Andrew Chow. What he did is, because all these hardware wallets, like the Trezor and the Ledger and the KeepKey and the COLDCARD, etc., they all have their own little Python libraries that you can find on GitHub because it's all open source. But they're all different. They all have different ways to display an address. They have different ways to get an xpub. They have different ways to sign a transaction. So what he did is he combined all these drivers. Stripped all the shit coins out of them because some of them would do Ethereum things and we don't want Ethereum code.

Aaron van Wirdum I don't.

Sjors Provoost And then basically put that all together in one little library and made it consistent. So as far as the user is concerned, you tell HWI Enumerate and it will give you the list of all the connected hardware wallets using all these different drivers to find them. And if you say, get descriptors, then it will give you a list of descriptors that describe exactly what keys, what addresses the hardware wallet has. And it does all the magic of fetching xpubs using whatever method is necessary.

Aaron van Wirdum So could I describe HWI as sort of like a meta protocol that communicates with all of the different hardware wallet protocols, so to say?

Sjors Provoost Yeah, I guess. Another way to say it is that HWI is a tool. But it has a way of communicating. You have a way of giving it commands, and that is a protocol, essentially. So somebody else could make a tool that behaves the same as HWI, but is just for their own hardware wallet. So a hardware wallet manufacturer doesn't have to let people use HWI. They could make their own driver that just behaves the same way as HWI. And then Bitcoin Core, as we'll discuss next, will know how to talk to it.

Aaron van Wirdum Right. And if someone brings a new hardware wallet to the market, which does its own thing again, then HWI will be updated potentially to be able to communicate with that as well?

Sjors Provoost Yeah. So depending on the enthusiasm of the maintainers of HWI, they might... Go out and find that driver and include it. If the enthusiasm is slightly less, then the manufacturer or a fan, a user, can go to, you know, can go to HWI, make a pull request and say, hey, here's the support for this new hardware wallet. And then recently there were some guidelines merged for that. So the hardware wallet has to be open source. At least the firmware that runs on it has to be open source, with the only exception being if there's a secure element in it that's under NDA, that's okay. And it has to be sort of maintainable, the software. But then, yeah, anybody can just make support, add support for any other hardware wallet to HWI. But again, it's not necessary. You could also, the hardware manufacturer could also make their own software that just speaks the same language as HWI.

Aaron van Wirdum Right. And HWI was already merged into Bitcoin Core?

Sjors Provoost So HWI is a separate program. That's in the Bitcoin… not in the Bitcoin Core repository, but there is a GitHub, there's a GitHub organization called Bitcoin Core, which has a bunch of repositories. And that includes HWI. So it's separate from Bitcoin Core because it's all this Python stuff and USB drivers. So I guess we don't want to put that in the Bitcoin source code because that's too scary with all the dependencies. But it is maintained by Bitcoin Core people. So it's a trust slightly less or... I don't know how to put it.

Aaron van Wirdum Okay, so HWI is... A separate program, essentially. And it needs to be installed separately if you want to use it?

Sjors Provoost Exactly. You can download it separately, install it separately, or just put it somewhere.

Aaron van Wirdum Right. It's not in the Bitcoin Core client that I'm running at home?

Sjors Provoost No.

Aaron van Wirdum Got it.

Sjors Provoost Because that would, you know, in order for that to happen, first of all, it will probably have to be written in C++ or in Rust or something like that. And all the USB drivers would have to be inspected with a tooth comb. And that'll take forever. So yeah, it's something you install separately. And there is some risk, obviously, with just running that software.

Aaron van Wirdum Right. But this is not new, is it?

Sjors Provoost That's right. HWI has been around for at least a year.

Aaron van Wirdum Right. So what was merged this week? Or am I getting ahead of the podcast?

Sjors Provoost That's exactly right.

Aaron van Wirdum Okay, so what was merged last week?

Sjors Provoost So before last week, if you wanted to use HWI with Bitcoin Core, you can do that. But you have to execute a bunch of commands manually. So for example, if you wanted to sign a transaction, you would tell Bitcoin Core, okay, I'd like to create a PSBT. So we explained PSBT in an earlier episode.

Aaron van Wirdum Yeah, partially signed Bitcoin transactions.

Sjors Provoost Exactly. So you tell Bitcoin Core, okay, give me a PSBT, a partially signed transaction. In this case, actually an unsigned transaction probably to the destination. And then you would copy paste the result and you'd say, okay, HWI, please sign this thing. And then HWI would do its thing. And then you get a result and you copy-paste that back into Bitcoin Core and say, Bitcoin Core, please process this thing and send this thing.

Aaron van Wirdum Ah, right. So you're running both programs, the Bitcoin Core program and the HWI program, and then you're manually basically sending data from copy-pasting data from one to the other…

Sjors Provoost And [unclear]…

Aaron van Wirdum Using your hardware wallet on HWI, getting some signature and then copy-pasting it to Bitcoin Core, and that's how it would work.

Sjors Provoost Exactly.

Aaron van Wirdum And you're doing this all in the command line, I assume?

Sjors Provoost Yes, that's right. And there was already some help with that because HWI also comes with a little graphical program where you can actually click buttons to make it do these things. But you'd still be copy-pasting back and forth at Bitcoin Core.

Aaron van Wirdum I love buttons.

Sjors Provoost Yeah, so do I. So what is new now is that Bitcoin Core now knows how to communicate with HWI. Still from the command line, but it still gets easier. Because now when you want to send coins, you just tell Bitcoin Core, send, and then the destination, and the amount. And it will know that it needs the hardware wallet. So it will say, hello, HWI, please do these things. And it will wait for the result and it will process the result and it will send the transaction. So you go from like four back and forth with copy pasting to just send and it works.

Aaron van Wirdum No more copy pasting.

Sjors Provoost That's right. So all you need to do is tell Bitcoin Core where HWI lives. So where on your computer you installed it. And then it will just call it for you.

Aaron van Wirdum Right, so if you know how to work the command line, this is a pretty big improvement, I guess.

Sjors Provoost This is significantly less tedious, yeah. I have used both the approaches, like with the manual back and forth, and with this, it's much easier.

Aaron van Wirdum But you do still need to use the command line.

Sjors Provoost Yeah, or in Bitcoin Core, you have this window, the debug console, which is essentially a little command line.

Aaron van Wirdum Right, right.

Sjors Provoost So I cannot do it in the user interface, but you can actually, yeah, you can open this window console and then you can just type send and it'll work.

Aaron van Wirdum Right. So this was merged into Bitcoin Core, which will be released in, I guess it will probably be Bitcoin Core 22.

Sjors Provoost Yeah, Bitcoin Core 22 should be out in, I don't know, fall.

Aaron van Wirdum Might it also come into a minor release? I guess that might depend on Taproot activation stuff.

Sjors Provoost This is not the kind of stuff that would be backported.

Aaron van Wirdum No? So it's definitely going to be a major release. Is that what you're saying? Yeah.

Sjors Provoost Because for things to backport, it has to be simple enough, or like a really critical bug fix. But usually the only things that are backported are bug fixes and soft forks, but not entire new features.

Aaron van Wirdum Got it. Okay, so this will be in the upcoming Bitcoin Core 22 release.

Sjors Provoost Yeah, or if you compile the master branch yourself.

Aaron van Wirdum Sure. The step after that, I would assume, would be GUI support?

Sjors Provoost GUI support? That's right. Yeah, because still, you know, having to type commands is kind of annoying. So I have a pull request open that builds on top of this, and it adds a couple of things. It adds something to the settings screen. So there is a settings screen in Bitcoin Core, and there you tell it where HWI lives. So you don't have to give that to the command line. You just copy-paste that file directory. And then… When you go to the menu to create a wallet, if you have this HWI path correctly and you have a hardware wallet inside your computer and it's unlocked, if you create a new wallet, it'll detect it. So it'll say, oh, would you like to use this Ledger X or whatever? And then you click next. And it will automatically pull the keys from the device, and your wallet will just look like any other wallet, but if you click on your address…

Aaron van Wirdum It will pull the public keys, to be clear, right?

Sjors Provoost Correct, yeah, public keys. It pulls the public keys from the device, and your wallet will work just as any other wallet. You can click on receive address, you'll see an address. And then the other feature is that if you click on a receive address, so in Bitcoin Core you click on get me a new address, then it kind of makes a little mini address book. And you can right click on that, I believe, and then click on verify. That's a new button. And the verify button, again, if the device is plugged in, will [unclear] show the address on the hardware wallet.

Aaron van Wirdum Right, yep.

Sjors Provoost So you can check that it's real.

Aaron van Wirdum Yep.

Sjors Provoost And then the… the third thing that's added is you can now just send Bitcoin as you would in Bitcoin Core. Just click on send and type the destinations [unclear]. And when you hit send, if the device is plugged in, it'll show up on the device and you click on approve. And it just sends it. Now, the downside is that if anything does not go exactly right as I just described it, it will probably crash.

Aaron van Wirdum Right.

Sjors Provoost So it's still work in progress.

Aaron van Wirdum Would it work with all of Bitcoin Core's current features? So that would be, for example, be RBF, I guess, and sent to many or whatever it's called, batching, these kinds of things.

Sjors Provoost Well, if in the graphical interface you can send a transaction, you can send it to like 10 different recipients if you want to, that'll work. I believe RBF... I have not implemented because internally that does a few other things. I don't know, you can try. But it will be trivial to add that as a follow-up.

Aaron van Wirdum Right.

Sjors Provoost But I try to keep these pull requests as minimum as possible. So these are like three features that I think are really at least good enough.

Aaron van Wirdum Sorry, are these things possible now? Are we just talking about GUI?

Sjors Provoost Yeah, the GUI pull request. Right now you can do all these things with the command line, right, but they'll work in the GUI. Which is very nice because, especially coin selection. So in the command line, typing send an address and an amount, that's not too bad.

Aaron van Wirdum Yeah, other good example, right? Coin selection.

Sjors Provoost But the coin selection is only practical in the GUI. So you can actually decide which coins to spend. And having coin selection with hardware wallets is very nice.

Aaron van Wirdum Yeah, I agree.

Sjors Provoost It does work, but it only works if it doesn't crash. So that just needs more review and requires me to fix things and maybe make it asynchronous so the screen doesn't block while the hardware wallet is doing something.

Aaron van Wirdum Yeah, this will probably not be ready before Bitcoin Core 22. I don't know.

Sjors Provoost I'm bullish.

Aaron van Wirdum You think it might be?

Sjors Provoost It depends on the review because I have a couple of things that I want to improve about it, but I tend to get more motivated when people start screaming at me in the reviews.

Aaron van Wirdum I can scream at you in the reviews. I won't say anything useful, but I can scream at you.

Sjors Provoost Well, you do have to actually test it, right? Don't just come in from Twitter and comment on it. But if you actually run the code and something goes wrong, then I'll try to help or fix the bug.

Aaron van Wirdum I'll consider that as well. I could consider that. What about multisig? Using, that's like one of the arguably safest…

Sjors Provoost Yeah, so everything I've described so far is single sig.

Aaron van Wirdum Yeah, so for example, if you'd want to use three hardware wallets and then two out of three stuff like that.

Sjors Provoost Well, eventually I'd like to see something similar to how Electrum, you know, just lets you make a pie chart.

Aaron van Wirdum Yes, exactly.

Sjors Provoost None of that is there yet, not even in the command line. It is possible. I think I once made an experimental pull request that did do multi-sig at least from the command line. But there's a lot of moving parts. So I'd like to get all the single-sig stuff merged. Then we'll move on to multi-sig. So Bitcoin Core multi-sig with hardware wallets will be a while. That said, well, there's another way to do it.

Aaron van Wirdum There's another way. That's great. Let's hear it.

Sjors Provoost This project is called Specter.

Aaron van Wirdum Specter wallets, yes.

Sjors Provoost Which is very cool. Specter is also a hardware wallet, but in this case, what we're talking about is the Specter desktop software. Another Python project.

Aaron van Wirdum There's a device, a Specter device.

Sjors Provoost Yeah, it's very cool. I should have brought it with me. But it's a do-it-yourself hardware wallet. So you can actually order the components from any manufacturer that sort of makes these Arduino-like or smartphone-like things. And you take a QR scanner and you solder it on it. This is by by Stepan Snigirev. He's been working on that. He's also been working, I believe, on support for actual secure elements so that you can buy your own secure element from like a manufacturer that makes a lot of these general purpose secure elements. So they might not even know that you're using it for Bitcoin purposes, which is very nice supply chain wise.

Aaron van Wirdum Right.

Sjors Provoost And you don't have to trust them. Yeah, it's very cool. Basically, you can do a lot of stuff with it.

Aaron van Wirdum That's the device. We were actually going to talk about the wallet, I guess, the software.

Sjors Provoost Because… because I guess in order for them to develop that device, they wanted to use that inside a multisig setup. And so they… they spent a lot of time building a multisig system. And that is called the Specter Desktop. What it does is, again, it's a program you download and it opens on your computer. And then it says, hey, where's Bitcoin Core? Or hopefully it finds it. And then if it finds Bitcoin Core, it says, okay, well, Bitcoin Core is synced. It's got all the blocks. Do you have any devices you'd like to configure? And then you say, okay, yeah, I have a Ledger wallet and I have a Trezor wallet and whatever. You add those devices one by one by just plugging it in. And under the hood, that uses HWI.

Aaron van Wirdum Specter uses HWI as well, right?

Sjors Provoost Yeah, it just comes bundled with it. So it knows how to detect them, it knows how to add them.

Aaron van Wirdum So you need to install HWI as well?

Sjors Provoost No, I think Specter just has…

Aaron van Wirdum Oh, it includes it.

Sjors Provoost Yeah.

Aaron van Wirdum Right, got it.

Sjors Provoost Yeah, so that's the trade-off, right? So in Bitcoin Core, we don't just want to add HWI, but in Specter, they do.

Aaron van Wirdum Got it.

Sjors Provoost And then basically you add devices and then you say, okay, I'd like to configure a wallet. And the terminology is a bit confusing. That's why I use the word device… device because the device is also a wallet.

Aaron van Wirdum But yep.

Sjors Provoost We create a wallet and you say, okay, I want to combine the Trezor and the Ledger and I want a two of two sure, or one of two. And then it says, okay, I'm going to create a wallet now. And it creates that wallet in Bitcoin Core. So that means that Specter does not have to maintain all the code necessary to create a whole wallet from scratch, because that's a lot of work. It lets Bitcoin Core do that. It also means it can let Bitcoin Core do all the coin selection magic.

Aaron van Wirdum And this uses the GUI of Bitcoin Core?

Sjors Provoost No, it creates its own GUI, which is basically just a little web server on your own computer, kind of like how a modem looks if you log in, log into the admin panel.

Aaron van Wirdum Right. So it uses Bitcoin Core as a backend, but you're actually looking at Specter, which also includes HWI.

Sjors Provoost Yeah, that's right. And I guess one scenario where this might be nice is if you run Bitcoin Core, not on your computer, but on a Raspberry Pi somewhere, which might not have a screen, or at least you don't want to use the screen. And you could imagine remote connecting to your Raspberry Pi with Bitcoin Core running on it. And you're not worried because your keys are not on Bitcoin Core. It's just the read-only stuff is on Bitcoin Core. So your Raspberry Pi in your living room with no security is not really a threat, but it is a full node. And yeah, so you can use it that way.

Aaron van Wirdum Yeah, so that's one way.

Sjors Provoost It looks pretty slick. It's very easy to use in my experience once you get it running.

Aaron van Wirdum So that's one way to use a hardware wallet in combination with Bitcoin Core already, including... Including multi-sig solutions.

Sjors Provoost Yeah, especially the latter. I guess, you know, soon TM, you'll be able to use Bitcoin Core with single-sig, and then I would say the fewer moving parts you have, the better. So nothing against Specter, but if you don't need it, then don't run additional code.

Aaron van Wirdum Yeah.

Sjors Provoost But for multi-sig, it's great. And to get that kind of UI, especially in Bitcoin Core, that's going to take a while.

Aaron van Wirdum Yeah, there are a couple of other solutions like this. You already mentioned Electrum. Which you could, of course, also run in combination with your personal Electrum server, in which case you're also using full node security.

Sjors Provoost Yeah, although I'd have to see, I don't know if Electrum personal server also has support for multisig, but in theory that could be added.

Aaron van Wirdum Yeah, there's something called a Lily wallet. I don't know if that uses full node security. I've never used it. It looks very slick. I've seen that.

Sjors Provoost Okay.

Aaron van Wirdum But that also allows for hardware wallets, multi-setup stuff. I think, but I could be wrong. I think that's SPV light-client type of wallet though.

Sjors Provoost I'm not, I don't know.

Aaron van Wirdum I think there's also one called Caravan.

Sjors Provoost Caravan, yep. I've heard of that.

Aaron van Wirdum I don't know much about Caravan. I know it does something similar.

Sjors Provoost Yeah, that's what I heard too. And then there is, I guess, Casa. I don't know how they do it. So yeah, there's all sorts of ways. I… I imagine in the future this is going to get better and better. Right, so now we have…

Aaron van Wirdum Well, it's getting better already, clearly.

Sjors Provoost Well, we talked about very simple multisig setups, but imagine a scenario where we have not just Taproot. Yeah, we have Taproot, and then we have Miniscript, which we talked about in an earlier episode, which allows you to make far more complicated setups. So now you can imagine instead of a two or three setting where you have two hardware wallets, one you keep in a vault somewhere, one you keep wherever you are. And then the third key, quote unquote, is not actually a single key. It's not a single device. It is some sort of bank, some sort of bank that helps you. And they internally use some extremely complicated setup, which you don't have to understand. From your point of view, it just looks like one signature. In reality, it's very complicated. And it can all be communicated through Miniscript. And then you do something like Specter. So you and your site can make sure everything works. And only when you need the bank, then you need to ask them for a signature. Sure. But other than that, it all just works from your end. Even though they have some super complicated rule set that for you, it's just a third party and you don't care. So I'm very bullish on the future, but this is going to take a while.

Aaron van Wirdum Well, Sjors, thanks for the update. This is where we are with hardware wallet integration in Bitcoin Core now and soon, TM.

Sjors Provoost That's right. Anything else?

Aaron van Wirdum I think that's it for me.

Sjors Provoost Alrighty. Thank you for listening to the Van Wirdum Sjorsnado.

Aaron van Wirdum There you go.