I've always felt that having your keys tied to specific hardware will just lock you out when the hardware breaks. Maybe I'm overstating the risk. It's a good idea to have multiple methods and test the backups.
The general consensus has been that you create a key pair per client computer that you use. If one is stolen (say your laptop), you login from your desktop and revoke the stolen key. If the hard drive fails, you login from another client.
I don’t see much difference between that and storing the key on a TPM. If you have one key and you lose access to that key, then you lose access to the server.
One key per device is exactly what we recommend too. Private keys should always be protected as much as possible within that device and should never leave that device.
Just paste all of your devices' public keys into your authorized_keys file and leave a comment at the end for what device it's for. in Userify, it literally goes right into your nodes' authorized_keys file almost verbatim. (disclaimer: I work at https://Userify.com)
And then, if you leave your token or laptop at the airport or whatever, just remove that key right from your phone and it'll take effect in seconds across all the nodes/instances (if you're using Userify) or you can just write a quick for-inline-sed loop to remove it from your authorized keys everywhere.
It's unlikely to be inside the TPM. The way pretty much all TPMs store data is to use a key inside the TPM to encrypt data stored outside the TPM, because the TPM is a repurposed smart card with barely any storage or capabilities outside of DRM. Bitlocker is an extreme example of this, but things like Fapi_CreateSeal()/Fapi_Unseal() also store the sealed item outside the TPM even though they appear to be using the TPM for storage. So what you do is the same as what Bitlocker does, use the TPM's storage root key (SRK) to seal some master key (in Bitlocker terms the VMK) and the master key seals the encryption key used, which is also sealed with some user-entered emergency-access password or whatever that still gives you access if the TPM dies.
OTOH for SSH use if you lose the key you just create a new one, it's not like you've lost the only copy of your Bitlocker key.
A passkey is just a thing that authenticates with FIDO2 (or is it WebAuthn?), I believe.
With a password, you open your password manager, copy the password in memory, paste it into the input field and trust that nobody could read it from your clipboard and that the program handling the password does it correctly. If your password leaks on the way, it's leaked.
With FIDO2, the server sends a challenge and asks your HSM (or TPM, not sure what the right word is) to sign it with your private key. So the server can verify that you own the private key, but if the challenge or the response leaks, it's just this one time. Next time it will be a new challenge.
Also for the average Joe, the result is that the "passkey" is the fingerprint or the face recognition and there is no password. It feels like they have only one password: the biometry/face recognition (or a master password, I guess?). So passkeys are superior to passwords in that sense.
Fun fact 1: some people hate passkeys because they don't want to be forced to rely on TooBigTech for them. Currently I use my Yubikeys as passkeys everywhere and it works well, so I do NOT depend on TooBigTech.
Fun fact 2: FIDO2 on current Yubikeys (and HSM in general, I think) tend to use classic cryptography which would be broken by quantum computers. A password used with symmetric encryption is not broken by quantum computers. So there may be a period of time where this becomes a tradeoff (you may have to decide whether the most likely attack is a quantum computer breaking your authentication or a malware stealing your password)?
> With a password, you open your password manager, copy the password in memory, paste it into the input field and trust that nobody could read it from your clipboard and that the program handling the password does it correctly. If your password leaks on the way, it's leaked.
I don't do that. My password manager simulates keystrokes 2 seconds after I hit the button. I switch to the other window and my password gets punched in without going through the clipboard. Specifically to avoid this attack.
> Currently I use my Yubikeys as passkeys
I have Yubikeys but for 2FA. So we're back to 1FA now but just "something you have" and no "something you know" ?
> A passkey is just a thing that authenticates with FIDO2 (or is it WebAuthn?), I believe.
Not quite. First of all, passkey is not a standardized term. But usually it refers to a key that can be used for authentication on its own, not as a 2FA along with a password.
A FIDO2 key can be a passkey, or not, depending on the service or configuration.
FIDO2 and WebAuthn added some fields necessary to make this work "securely", e.g. asking the key to verify the user separately (e.g. a PIN, which serves as a second factor), or asking the key whether it is device bound or roaming, so individual sites/enterprises can enforce their security policies
Cloud-based passkeys are okayish (1pass, bitwarden), as they are available on multiple devices.
However not all devices play well with it, e.g. iOS and Android don't ask 1pass for the passkey. I also couldn't make it ask NFC for my hardware Yubikey with passkeys, but maybe I just did something wrong.
Didn't Tailscale try to do something similar but found out quickly that TPMs 1) aren't as reliable as common wisdom makes them out to be, and 2) have gotchas when it comes to BIOS updates?
I can't find it now, but I believe someone from Tailscale commented on HN (or was it github?) on what they ran into and why the default was reverted so that things were not stored in the TPM.
EDIT: just saw the mention in the article about the BIOS updates.
This is a neat trick that people have been doing with Yubikeys for a long time, but from an operational security perspective, if you have a fleet rather than just a couple of hosts, the win is only marginal vs. short-lived keys, certificates, and a phishing-proof IdP.
Seems a little pointless, your keys can't be stolen but they can be instantly used by malware to persist across anything you have access to. The keys don't have any value in their own right, the access they provide does.
Or put them in a $2 FLOSS Gnuk token/smart card that you can carry with you and still have strong password protection and AES encrypted data at rest with KDF/DO:
I would love a world where I could put all my API keys in the TPM so malware couldn't gain persistent access to services after wiping my computer. This would be so easy if more providers used asymmetric keys, like through SSH or mTLS. Unfortunately, many don't, which means that stealing a single bearer token gives full access to services.
There's also the TPM speed issue. My computer takes ~500ms to sign with an ECC256 key with the TPM, which starts to become an issue when running scripts that use git operations in serial. This is a recurring problem that people tend to blame on export controls: https://stiankri.substack.com/p/tpm-performance
I'd consider storing (generating) them in AWS KMS. It's $1/key/month and you don't have to worry about hardware failures, etc. Each key must have a separate policy attached which controls who it can be used by and how. It is possible to create keys the root account cannot touch. If you have anything running on EC2, it's an extremely compelling option because you can authenticate directly via IMDSv2 tokens and IAM roles, avoiding the need for any kind of secret strings.
We created Keeta Agent [0] to do this on macOS more easily (also works with GPG, which is important for things that don't yet support SSH Signatures, like XCode).
Since it just uses PKCS#11, it also works with tpm_pkcs11. Source for the various bits that are bundled is here [1].
Here's an overview of how it works:
1. Application asks to sign with GPG Key "1ABD0F4F95D89E15C2F5364D2B523B4FDC488AC7"
2. GPG looks at its key database and sees GPG Key "1ABD...8AC7" is a smartcard, reaches out to Smartcard Daemon (SCD), launching if needed -- this launches gnupg-pkcs11-scd per configuration
3. gnupg-pkcs11-scd loads the SSH Agent PKCS#11 module into its shared memory and initializes it and asks it to List Objects
4. The SSH Agent PKCS#11 module connects to the SSH Agent socket provided by Keeta Agent and asks it to List Keys
5. Key list is converted from SSH Agent protocol to PKCS#11 response by SSH Agent PKCS#11 module
6. Key list is converted from PKCS#11 response to gnupg-scd response by gnugpg-pkcs11-scd
7. GPG Reads the response and if the key is found, asks the SCD (gnugpg-pkcs11-scd) to Sign a hash of the Material
8. gnupg-pkgcs11-scd asks the PKCS#11 module to sign using the specified object by its Object ID
9. PKCS#11 module sends a message to Secretive over the SSH Agent socket to sign the material using a specific key (identified by its Key ID) using the requested signing algorithm and raw signing (i.e., no hashing)
10. Response makes it back through all those same layers unmodified except for wrapping
I had a friend tell me once that his yubikey is more secure than my authenticator app on my phone because my phone has this giant attack surface that his yubikey doesn't. Yet the yubikey has an entire attack surface of the computer it is plugged into. Which is largely the same or worse than my phone's.
I'm wondering why that doesn't apply here. The TPM holds the key to the cipher that is protecting your private keys. Someone uses some kind of RCE or LPE to get privileged access to your system. Now it sits and waits for you to do something that requires access to your SSH keys. When you do that you are expecting whatever user prompts come up, the malware rides along this expectation and gets ahold of your private SSH keys and stores them or sends them off somewhere. I'm not even positive that they need high degree of privileges on your box, if they can manipulate your invocation of the ssh client, by modifying your PATH or adding an ssh wrapper to something already in your path, then this pattern will also work.
What am I gaining from using this method that I don't get from using a password on my ssh private key?
For Yubikey, this guide is worth looking at: https://github.com/drduh/yubikey-guide ("Community guide to using YubiKey for GnuPG and SSH - protect secrets with hardware crypto.")
Termius integrates TPM/Secure Enclave keys nicely on Windows Mac iOS etc as well, which is nice because then you can have a key on your phone (so if you lose your laptop you won’t get locked out of your servers)
Or, alternatively, don't. Stuff in a TPM isn't for "security" in the abstract, it's fundamentally for authentication. Organizations want to know that the device used for connection is the one they expect to be connecting. It's an extra layer on top of "Organizations want to know the employee account associated with the connection".
"Your SSH keys" aren't really part of that threat model. "You" know the device you're connecting from (or to, though generally it's the client that's the mobile/untrusted thing). It's... yours. Or under your control.
All the stuff in the article about how the TPM contents can't be extracted is true, but missing the point. Yes, you need your own (outer) credentials to extract access to the (inner) credentials, which is no more or less true than just using your own credentials in the first place via something boring like a passphrase. It's an extra layer of indirection without value if all the hardware is yours.
TPMs and secure enclaves only matter when there's a third party watching[1] who needs to know the transaction is legitimate.
[1] An employer, a bank, a cloud service provider, a mobile platform vendor, etc... This stuff has value! But not to you.
Keep a CA (constrained to your one identity) with a longish (90 day?) TTL on the TPM. Use it to sign a short lived (16h?) keys from your TPM, use that as your working key.
This could make real sense for ssh host keys, since they need to be used without presence and they're generally tied to the lifetime of the machine anyway.
I saw a write up where someone successfully got sshd to use a host key from a fido2 yubikey without touch, but I can't find it...
As far as "TPM vs HSM", it is soooo much simpler to make a key pair with a fido2 hardware key:
115 comments
I don’t see much difference between that and storing the key on a TPM. If you have one key and you lose access to that key, then you lose access to the server.
Point: you need a backup key anyway.
Just paste all of your devices' public keys into your authorized_keys file and leave a comment at the end for what device it's for. in Userify, it literally goes right into your nodes' authorized_keys file almost verbatim. (disclaimer: I work at https://Userify.com)
And then, if you leave your token or laptop at the airport or whatever, just remove that key right from your phone and it'll take effect in seconds across all the nodes/instances (if you're using Userify) or you can just write a quick for-inline-sed loop to remove it from your authorized keys everywhere.
OTOH for SSH use if you lose the key you just create a new one, it's not like you've lost the only copy of your Bitlocker key.
I don't think average Joe is going to understand these passkeys either.
With a password, you open your password manager, copy the password in memory, paste it into the input field and trust that nobody could read it from your clipboard and that the program handling the password does it correctly. If your password leaks on the way, it's leaked.
With FIDO2, the server sends a challenge and asks your HSM (or TPM, not sure what the right word is) to sign it with your private key. So the server can verify that you own the private key, but if the challenge or the response leaks, it's just this one time. Next time it will be a new challenge.
Also for the average Joe, the result is that the "passkey" is the fingerprint or the face recognition and there is no password. It feels like they have only one password: the biometry/face recognition (or a master password, I guess?). So passkeys are superior to passwords in that sense.
Fun fact 1: some people hate passkeys because they don't want to be forced to rely on TooBigTech for them. Currently I use my Yubikeys as passkeys everywhere and it works well, so I do NOT depend on TooBigTech.
Fun fact 2: FIDO2 on current Yubikeys (and HSM in general, I think) tend to use classic cryptography which would be broken by quantum computers. A password used with symmetric encryption is not broken by quantum computers. So there may be a period of time where this becomes a tradeoff (you may have to decide whether the most likely attack is a quantum computer breaking your authentication or a malware stealing your password)?
> With a password, you open your password manager, copy the password in memory, paste it into the input field and trust that nobody could read it from your clipboard and that the program handling the password does it correctly. If your password leaks on the way, it's leaked.
I don't do that. My password manager simulates keystrokes 2 seconds after I hit the button. I switch to the other window and my password gets punched in without going through the clipboard. Specifically to avoid this attack.
> Currently I use my Yubikeys as passkeys
I have Yubikeys but for 2FA. So we're back to 1FA now but just "something you have" and no "something you know" ?
> A passkey is just a thing that authenticates with FIDO2 (or is it WebAuthn?), I believe.
Not quite. First of all, passkey is not a standardized term. But usually it refers to a key that can be used for authentication on its own, not as a 2FA along with a password.
A FIDO2 key can be a passkey, or not, depending on the service or configuration.
FIDO2 and WebAuthn added some fields necessary to make this work "securely", e.g. asking the key to verify the user separately (e.g. a PIN, which serves as a second factor), or asking the key whether it is device bound or roaming, so individual sites/enterprises can enforce their security policies
However not all devices play well with it, e.g. iOS and Android don't ask 1pass for the passkey. I also couldn't make it ask NFC for my hardware Yubikey with passkeys, but maybe I just did something wrong.
> We don't want that in our shell history
This may be bash-only, but a space before the command excludes something from history too.
I can't find it now, but I believe someone from Tailscale commented on HN (or was it github?) on what they ran into and why the default was reverted so that things were not stored in the TPM.
EDIT: just saw the mention in the article about the BIOS updates.
https://github.com/ran-sama/stm32-gnuk-usb-smartcard
> A big warning is that a lot desktop motherboards (at least consumer oriented ones) wipe the TPM when you update the BIOS.
Well no thanks, that risk is much higher than what this is worth.
There's also the TPM speed issue. My computer takes ~500ms to sign with an ECC256 key with the TPM, which starts to become an issue when running scripts that use git operations in serial. This is a recurring problem that people tend to blame on export controls: https://stiankri.substack.com/p/tpm-performance
Since it just uses PKCS#11, it also works with tpm_pkcs11. Source for the various bits that are bundled is here [1].
Here's an overview of how it works:
1. Application asks to sign with GPG Key "1ABD0F4F95D89E15C2F5364D2B523B4FDC488AC7"
2. GPG looks at its key database and sees GPG Key "1ABD...8AC7" is a smartcard, reaches out to Smartcard Daemon (SCD), launching if needed -- this launches gnupg-pkcs11-scd per configuration
3. gnupg-pkcs11-scd loads the SSH Agent PKCS#11 module into its shared memory and initializes it and asks it to List Objects
4. The SSH Agent PKCS#11 module connects to the SSH Agent socket provided by Keeta Agent and asks it to List Keys
5. Key list is converted from SSH Agent protocol to PKCS#11 response by SSH Agent PKCS#11 module
6. Key list is converted from PKCS#11 response to gnupg-scd response by gnugpg-pkcs11-scd
7. GPG Reads the response and if the key is found, asks the SCD (gnugpg-pkcs11-scd) to Sign a hash of the Material
8. gnupg-pkgcs11-scd asks the PKCS#11 module to sign using the specified object by its Object ID
9. PKCS#11 module sends a message to Secretive over the SSH Agent socket to sign the material using a specific key (identified by its Key ID) using the requested signing algorithm and raw signing (i.e., no hashing)
10. Response makes it back through all those same layers unmodified except for wrapping
(illustrated at [2])
[0] https://github.com/KeetaNetwork/agent
[1] https://github.com/KeetaNetwork/agent/tree/main/Agent/gnupg/...
[2] https://rkeene.org/tmp/pkcs-sign.png
I'm wondering why that doesn't apply here. The TPM holds the key to the cipher that is protecting your private keys. Someone uses some kind of RCE or LPE to get privileged access to your system. Now it sits and waits for you to do something that requires access to your SSH keys. When you do that you are expecting whatever user prompts come up, the malware rides along this expectation and gets ahold of your private SSH keys and stores them or sends them off somewhere. I'm not even positive that they need high degree of privileges on your box, if they can manipulate your invocation of the ssh client, by modifying your PATH or adding an ssh wrapper to something already in your path, then this pattern will also work.
What am I gaining from using this method that I don't get from using a password on my ssh private key?
"Your SSH keys" aren't really part of that threat model. "You" know the device you're connecting from (or to, though generally it's the client that's the mobile/untrusted thing). It's... yours. Or under your control.
All the stuff in the article about how the TPM contents can't be extracted is true, but missing the point. Yes, you need your own (outer) credentials to extract access to the (inner) credentials, which is no more or less true than just using your own credentials in the first place via something boring like a passphrase. It's an extra layer of indirection without value if all the hardware is yours.
TPMs and secure enclaves only matter when there's a third party watching[1] who needs to know the transaction is legitimate.
[1] An employer, a bank, a cloud service provider, a mobile platform vendor, etc... This stuff has value! But not to you.
Keep a CA (constrained to your one identity) with a longish (90 day?) TTL on the TPM. Use it to sign a short lived (16h?) keys from your TPM, use that as your working key.
> A big warning is that a lot desktop motherboards (at least consumer oriented ones) wipe the TPM when you update the BIOS.
that's not good
I saw a write up where someone successfully got sshd to use a host key from a fido2 yubikey without touch, but I can't find it...
As far as "TPM vs HSM", it is soooo much simpler to make a key pair with a fido2 hardware key:
You can get them for <$30.