Load nfs-mounted ssh keys at login automatically

I use multiple ssh keys across multiple systems. Some systems need to have the same ssh key loaded.

My solution is to store the generic ones on my nfs mount, accessible only to my user, and to use a function in .bashrc:

load_ssh_key() {
   test -n "${1}" && SSHKEY="${1}" ;
   test -z "${SSHKEY}" && SSHKEY=/mnt/bgstack15/.ssh/bgstack15_devuan.key
   if test -e "${SSHKEY}" ;
      test -z "${SSH_AGENT_PID}" && eval $( ssh-agent ) | grep -viE 'Agent pid'
      ssh-add "${SSHKEY}" 2>&1 | grep -viE "Identity added:" 1>&2
      echo "Unable to get to private key!" 1>&2
   unset SSHKEY


Also, now, the function is generally available for invoking with a filename to load that ssh key. I realize ssh-add is pretty trivial, but I want the function to fail silently for when I’m off-network (when I won’t be doing any ssh work anyway).

Ssh use 7z with password and suppress password echo

If you want to use 7z with a password, over ssh, by default the password will be displayed!

Suppress it with ssh -t. I don’t know why. Here’s the relevant section of the man page for ssh.

     -t      Force pseudo-terminal allocation.  This can be used to execute arbitrary screen-based programs on a remote machine, which
             can be very useful, e.g. when implementing menu services.  Multiple -t options force tty allocation, even if ssh has no
             local tty.

Use su with ssh X-forwarding

This is a shameless ripoff of Howto use su with ssh x-forwarding [coderwall.com]

If you ssh to a server and want to do X forwarding, make sure the server allows it in /etc/ssh/sshd_config:

X11Forwarding yes

And then your ssh command should include -X, or make sure your ~/.ssh/config includes:

ForwardX11 yes


ssh -X servername

And once on the server, prepare your xauth file to be shared the other user before switching and merging it.

xauth extract /tmp/x $DISPLAY
chmod 0644 /tmp/x
su otherusername
xauth merge /tmp/x


Web searches

ssh x forwarding with su

Notes on reacting to ssh key used to connect to server

The answers to https://unix.stackexchange.com/questions/15575/can-i-find-out-which-ssh-key-was-used-to-access-an-account provide some interesting details I’ve never seen before.

You can specify a command to run in the ~/.ssh/authorized_keys file:

command="/usr/share/bgscripts/work/react-ssh.sh ; /bin/bash" ssh-rsa AAAAB3NgaC1yc2EAAAABJQAAANEAnYh0nq5dzOgIgfkh50Th68hZoX+zR[...output truncated...]

Inside my example react-ssh.sh file:

journalctl -n30 -u sshd.service 2>/dev/null | grep -E "sshd\[$( ps --noheaders -o ppid $( ps --noheaders -o ppid $( ps --noheaders -o ppid $$ ) ) | xargs )]: Accepted publickey for ${USER}" | tail -n1 | awk '{print $(NF-1),$NF}'

So when I log in with an ssh key, it will print it out for me:

[bgstack15@example1|/home/bgstack15]$ ssh example2
RSA SHA256:I3wuJRyf1dWCzeqdLl6mWfMl9wONJLk38/xUwLCiNgA
[bgstack15@example2 ~]$

Here is a script that could be called with a parameter to show the entire public key of the hash.

# to show the full public key of the provided hash
test -z "${SFK_AUTHORIZED_KEYS}" && SFK_AUTHORIZED_KEYS=~/.ssh/authorized_keys

SFK_HASHES="$( ssh-keygen -l -f "${SFK_AUTHORIZED_KEYS}" )"
for word in $@ ;
   sed -n -r -e "$( echo "${SFK_HASHES}" | grep -hn "${word}" | awk -F':' '{print $1}' )p" "${SFK_AUTHORIZED_KEYS}"

sssd disable known_hosts hashing

If you use Fedora GNU/Linux, and you want bash autocompletion for the hosts in your FreeIPA domain when using the OpenSSH client, you have to disable knownhosts hashing in the global knownhosts file.

To find the global knownhosts file, check /etc/ssh/ssh_config

# grep -i knownhosts /etc/ssh/ssh_config
ProxyCommand /usr/bin/sss_ssh_knownhostsproxy -p %p %h
GlobalKnownHostsFile /var/lib/sss/pubconf/known_hosts

The file displayed here is the default on Fedora: /var/lib/sss/pubconf/known_hosts.

Now, to disable the hashing inside that file, you have to configure sssd.

# cat /etc/sssd/conf.d/50_ssh_hash_known_hosts.conf 
ssh_hash_known_hosts = false

I don’t know at what version the dot-dee directory for sssd.conf was introduced specifically (other then when it is compiled with libini>=1.3.0), but mine had it as of Fedora 27 and sssd-1.16.0. But with this entry in the config, restart sssd and maybe delete the previous /var/lib/sss/pubconf/known_hosts file which will be re-generated as it needs it.




Fedora 27 ssh and default kerberos config

On my new Fedora 27 system which I joined to my FreeIPA domain, I encountered an error I hadn’t seen before.
In the past I could just say “ssh remotehost” and it would connect me with GSSAPI auth using my kerberos key– no password or ssh key needed! It was wonderful. However, I ran into this issue, as seen with ssh -v remotehost

debug1: Unspecified GSS failure.  Minor code may provide more information
Server host/remotehost@IPA.EXAMPLE.COM not found in Kerberos database

But I know for a fact it’s in the kerberos database!
I duckducked (new verb) the error message and found the culprit.
In file /etc/krb5.conf, this variable should be set to this value:

  dns_canonicalize_hostname = true

The default is true according to man krb5.conf. but for whatever reason, whether by joining the domain, or some default of some package in Fedora 27, it was set to false.

For the followers of my bgscripts package, just use this command:

sudo updateval -a /etc/krb5.conf -s '[libdefaults]' '^(\s*dns_canonicalize_hostname\s*=\s*).*' '  dns_canonicalize_hostname = true'



  1. https://superuser.com/questions/1166094/ssh-single-sign-on-with-kerberos/1166101#1166101

X forwarding for virt-manager to Windows

Story time! When I was working on my virtual environment, I rebooted my main desktop. So I was stuck using my Windows desktop for a minute, and I wanted to work on my virtual machines.

I decided to do some X forwarding, which for virt-manager on CentOS 7 needs some special steps.

On the server

The first thing is to install virt-manager. You also will need a piece of software named xauth, and some special fonts.

yum install -y virt-manager xauth dejavu-\*fonts

Reference: https://superuser.com/questions/119792/how-to-use-x11-forwarding-with-putty/119908#119908

On the client

On the Windows client, you should install an X server. I picked Xming. It also needs it fonts installed.

Run Xming.

Connect to server with PuTTY. You will need to configure PuTTY to allow X forwarding, and to use the right X server.

Screenshot of Putty configuration screen showing X11 forwarding options
Telling PuTTY to allow X11 forwarding to localhost:0


Unfortunately, my keyboard input to the virtual machine does not work when I have it configured with a spice display. But it works when I use a vnc display. See https://bugzilla.redhat.com/show_bug.cgi?id=1236412 for a closed bug report that has similar symptoms to this problem.

virt-manager settings of a virtual machine, showing where to change display to VNC server type.
Setting on virtual machine of VNC server or spice server.



  1. http://straightrunning.com/XmingNotes/#head-131
  2. https://superuser.com/questions/119792/how-to-use-x11-forwarding-with-putty/119908#119908
  3. https://sourceforge.net/projects/xming/files/Xming/
  4. https://sourceforge.net/projects/xming/files/Xming-fonts/
  5. https://robert.penz.name/354/how-to-fix-the-font-for-virt-manager-via-x-forwarding/
  6. https://bugzilla.redhat.com/show_bug.cgi?id=1236412
  7. PuTTY https://www.chiark.greenend.org.uk/~sgtatham/putty/

List outbound ssh sessions


sudo netstat -Watp | grep 'ESTABLISHED.*ssh' | awk '{print $5}' | sed 's/:ssh//;' | sort | uniq | while read line; do ps -ef | grep -o "ssh\s.*${line}"; done | sort | uniq | sed -r -e 's/ssh //g;' -e 's/-l (\w*) /\1@/;'


During other work, it came up that I was interested in seeing what outbound ssh sessions I was using. Now I don’t even know why it came up, because I was just writing a shell script to programmatically adjust my xfce settings using its xfconf-query API.

Walking through the command

sudo netstat -Watp | grep 'ESTABLISHED.*ssh' | awk '{print $5}' | sed 's/:ssh//;' | sort | uniq | while read line; do ps -ef | grep -o "ssh\s.*${line}"; done | sort | uniq | sed -r -e 's/ssh //g;' -e 's/-l (\w*) /\1@/;'

This whole statement lists the established ssh connections and then finds the running processes for those and tries to identify the usernames for them.
Step by step:
Everything before the while collects the list of established ssh connections.
sudo netstat -Watp | grep ‘ESTABLISHED.*ssh’ gets the list of ssh connections, and awk | sed | sort | uniq just gets the information we want from each row and removes duplicates.
The while read line; do :; done loop iterates over the list. So for each line in the list, search all running processes for that name on the same line as the expression ‘ssh.’
sort | uniq removes duplicates (apparently qemu+kvm in virt-manager uses a lot of separate ssh processes).
sed -r -e ‘s/ssh //g;’ -e ‘s/-l (\w*) /\1@/;’ trims extra characters and also converts compatible outputs into “username@hostname.”

Improvements to be made

This snippet as is only works if the ssh command issued matches exactly the description of the output of netstat. If dns reverse zones are not configured correctly, so that the netstat shows an IP address but the ssh command was a hostname, this snippet will not find it. I need to improve that, which will probably require a fancier script and not just a oneliner.



  1. https://serverfault.com/questions/431034/getting-list-of-opened-ssh-connections-by-name

sshd_config match negate address


Match Address *,!

Negating address in match statement in sshd_config

I was locking down my ssh server configuration on a host, so that it will not accept password auth from outside a certain IP address range.
I had to learn how to get the Match Address directive to work with a negation. To make it work, you need to insert a wildcard before you then state the exclusion.

Match Address *,!

And then I added the directives for this matched IP address range.

   AuthenticationMethods publickey
   PubkeyAuthentication yes
   PasswordAuthentication no
   X11Forwarding no



  1. https://serverfault.com/questions/408284/how-can-the-address-condition-in-a-match-conditional-block-in-sshd-config-be-neg

Man pages

  1. sshd_config
  2. ssh_config