ssh keys not working
-
The other day I upgraded my nodebb vm from Ubuntu 14.04 to 16.04 to match what we're running here.
Today I made a fix for the erroneous error message for the youtube plugin and then I went to push to github:
$ git push --follow-tags origin master Permission denied (publickey). fatal: Could not read from remote repository.
Let's look at .
ssh
:~/.ssh$ ls -la ~ ... drwx------ 2 boomzilla boomzilla 4096 Jul 17 14:48 .ssh ... ~/.ssh$ ls -l total 16 -rw------- 1 boomzilla boomzilla 398 Jul 17 13:05 authorized_keys -rw------- 1 boomzilla boomzilla 1675 Jul 17 13:05 boomzilla_id_rsa -rw-r--r-- 1 boomzilla boomzilla 398 Jul 17 14:48 boomzilla_id_rsa.pub -rw-r--r-- 1 boomzilla boomzilla 2214 Jul 18 17:09 known_hosts
OK...that all looks good.
$ ssh-add -l The agent has no identities.
Huh? What am I missing? This was working prior to the upgrade. (INB4: Idiot).
-
Go into your repo and
git config core.sshCommand "ssh -i $HOME/.ssh/boomzilla_id_rsa"
-
@ben_lubar Nope. Also it doesn't recognize the key when I try to ssh into the server here (although it does work when I specify the key on the command line).
Hmmm...I wonder if this is in any way related to the fact that post-upgrade the VM starts out at a text console instead of starting KDE like it did before? Doesn't seem like it should.
-
I used
ssh-add
to explicitly add the key and now the key works in that shell.
-
@boomzilla
Your key files do not have the standard names. It looks for~/.ssh/id_rsa
. You may also be able to tell it to use this key by default by creating an appropriate~/.ssh/config
- seeman ssh_config
-
@PleegWat Hmmm....I copied them to
id_rsa
. But I didn't have to do that before. Did something change with ssh? I can't see any configuration changes.
-
@boomzilla said in ssh keys not working:
@PleegWat Hmmm....I copied them to
id_rsa
. But I didn't have to do that before. Did something change with ssh? I can't see any configuration changes.in theory you shouldn't have needed to change anything, as ssh should have offered all the public keys it found, but your system level ssh configuration may have changed to disable that functionality as many distributions consider that behavior to be insecure (for reasons i've never been able to get explained properly)
you can probably reenable the functionality if you fiddle with your system ssh_config settings
-
@boomzilla said in ssh keys not working:
Huh? What am I missing? This was working prior to the upgrade.
Does it work when you explicitly put the identities in)? If not, you're being caught by something more obscure like a strengthening of the minimum-required-security-config that is making it reject your key during loading. That sort of thing is really annoying.
The
-v
option to ssh itself can help a lot with debugging this sort of thing. (Also, I use https URLs with github, not ssh ones, but that's because it works better with my IDE.)
-
@boomzilla said in ssh keys not working:
Hmmm...I wonder if this is in any way related to the fact that post-upgrade the VM starts out at a text console instead of starting KDE like it did before?
You upgraded Ubuntu version and expected things to still work ?
-
@TimeBandit said in ssh keys not working:
You
upgradedused Ubuntuversionand expected things tostillwork ?FTFY
-
@accalia said in ssh keys not working:
you can probably reenable the functionality if you fiddle with your system ssh_config settings
I've never changed them before and an inspection of what I have on a different 14.04 vs the 16.04 showed no changes.
@dkf said in ssh keys not working:
Does it work when you explicitly put the identities in)
Yes.
@dkf said in ssh keys not working:
Also, I use https URLs with github, not ssh ones, but that's because it works better with my IDE.
It wasn't a git thing. I got the same thing using plain old
ssh
.OK, comparing the output of
ssh -v
...On a 14.04 machine:
debug1: Host 'what.thedailywtf.com' is known and matches the ED25519 host key. debug1: Found key in /home/xxx/.ssh/known_hosts:9 debug1: ssh_ed25519_verify: signature correct debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: SSH2_MSG_SERVICE_REQUEST sent debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey debug1: Next authentication method: publickey debug1: Offering RSA public key: boomzilla debug1: Server accepts key: pkalg ssh-rsa blen 279 debug1: Authentication succeeded (publickey).
The respective lines in my 16.04:
debug1: Host 'what.thedailywtf.com' is known and matches the ED25519 host key. debug1: Found key in /home/boomzilla/.ssh/known_hosts:7 debug1: rekey after 4294967296 blocks debug1: SSH2_MSG_NEWKEYS sen
Got rid of known_hosts to see if that would make a difference...
Warning: Permanently added 'what.thedailywtf.com,158.69.225.103' (ED25519) to the list of known hosts. debug1: rekey after 4294967296 blocks debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: rekey after 4294967296 blocks
Nope.
-
@boomzilla said in ssh keys not working:
@accalia said in ssh keys not working:
you can probably reenable the functionality if you fiddle with your system ssh_config settings
I've never changed them before and an inspection of what I have on a different 14.04 vs the 16.04 showed no changes.
@dkf said in ssh keys not working:
Does it work when you explicitly put the identities in)
Yes.
@dkf said in ssh keys not working:
Also, I use https URLs with github, not ssh ones, but that's because it works better with my IDE.
It wasn't a git thing. I got the same thing using plain old
ssh
.OK, comparing the output of
ssh -v
...On a 14.04 machine:
debug1: Host 'what.thedailywtf.com' is known and matches the ED25519 host key. debug1: Found key in /home/xxx/.ssh/known_hosts:9 debug1: ssh_ed25519_verify: signature correct debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: SSH2_MSG_SERVICE_REQUEST sent debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey debug1: Next authentication method: publickey debug1: Offering RSA public key: boomzilla debug1: Server accepts key: pkalg ssh-rsa blen 279 debug1: Authentication succeeded (publickey).
The respective lines in my 16.04:
debug1: Host 'what.thedailywtf.com' is known and matches the ED25519 host key. debug1: Found key in /home/boomzilla/.ssh/known_hosts:7 debug1: rekey after 4294967296 blocks debug1: SSH2_MSG_NEWKEYS sen
Got rid of known_hosts to see if that would make a difference...
Warning: Permanently added 'what.thedailywtf.com,158.69.225.103' (ED25519) to the list of known hosts. debug1: rekey after 4294967296 blocks debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: rekey after 4294967296 blocks
Nope.
huh.... that's weird as hell.......
-
@boomzilla said in ssh keys not working:
I used ssh-add to explicitly add the key and now the key works in that shell.
My (partial)
~/.bashrc
on most systems. Will load (and request passwords, if required, on the first shell since boot) all private keys that have a public key present in~/.ssh
CERTFILES= for public in ~/.ssh/*.pub; do private=${public%.*} if [ -e $private ]; then CERTFILES="${CERTFILES} $private" fi done KEYCHAIN=`which keychain` if [ ! "" = "$SSH_AGENT_PID" ]; then echo "Keychain: SSH_AGENT_PID is set, so running keychain to load keys." $KEYCHAIN $CERTFILES && source ~/.keychain/$HOSTNAME-sh else echo "Keychain: Found no SSH_AUTH_SOCK, so running keychain to start ssh-agent & load keys." $KEYCHAIN $CERTFILES && source ~/.keychain/$HOSTNAME-sh fi
And if you want those keys to work on a remote system without having that system actually hold the private keys,
~/.ssh/config
:host * ForwardAgent yes
That goes at the end. Have it set to no, and explicitly set to yes on specific hosts as required if you think it's too open.
-
@PJH So, I just confirmed (again) that my configs are the same. Pretty sure it's all default. These are the only lines that aren't comments:
Host * SendEnv LANG LC_* HashKnownHosts yes GSSAPIAuthentication yes GSSAPIDelegateCredentials no
Nothing in either
.bashrc
doing anything ssh related. I mean...I know how to make it work now, I'm just confused as to why it stopped working. I'm assuming something changed between versions:- OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014
- OpenSSH_7.2p2 Ubuntu-4ubuntu2.1, OpenSSL 1.0.2g 1 Mar 2016
-
@boomzilla said in ssh keys not working:
I'm assuming something changed between versions:
<looks>
Nope. both MAJOR version numbers look the same... Even the supporting library PATCH versions look similar..
More seriously, I suspect some CVE mitigation between them has resulted in some assumptions being either being rescinded or made invalid resulting in your particular setup being affected.
-
@PJH said in ssh keys not working:
Nope. both MAJOR version numbers look the same.
On openssl, yes, not on openssh (6 vs 7). Though I have no idea which one is responsible for determining which keys to use (I would guess ssh, but that's just an ass pull).
-
@boomzilla said in ssh keys not working:
I would guess ssh, but that's just an ass pull
My gut reaction too, given the name of where the keys and config are actually stored...