SSH Tunnel Proxy Traffic and Bastion

Sometimes you need to test a protocol and only have SSH access through a bastion host. You can display X back for example firefox or you can route traffic through a SSH tunnel. Here is a couple examples:

1. Display back should be fairly common and I don’t need to elaborate much. Use -X and connect to the host with firefox.

$ ssh -X -F M-config ociserver1
Last login: Mon Jul  9 07:46:39 2018 from desk01
$ firefox 

URL works http://ebs.domain1.com:8000/OA_HTML/OA.jsp?OAFunc=OASIMPLEHOMEPAGE

2. SSH Tunnel

$ ssh -L8000:10.35.6.4:8000 -i oci-M opc@pub.lic.ip.address
Last login: Mon Jul  9 07:36:01 2018 from c-desktop

$ grep ebs /etc/hosts
127.0.0.1 ebs.domain1.com

URL works http://ebs.domain1.com:8000/OA_HTML/OA.jsp?OAFunc=OASIMPLEHOMEPAGE

Posted in SSH

Rsync Plus SSH Config

Sometimes you need to use settings from the ssh config file or in my case a custom config file.  Here is a quick note on how I did it.

Example without SSH config just using the key and user@publicIP

$ pwd
/home/rrossouw/.ssh

$ rsync -avz --exclude "env-vars" -e "ssh -i /media/sf_DATA/ssh-keys/oci-mgmt" /media/sf_DATA/src/terraform/* opc@pu.blic.ip:~/terraform/
sending incremental file list
devtest/lb_private.tf

sent 2,650 bytes  received 91 bytes  1,096.40 bytes/sec
total size is 1,343,000  speedup is 489.97

Example with SSH config

$ pwd
/home/rrossouw/.ssh

$ rsync -avz --exclude "env-vars" -e "ssh -F My-config" /media/sf_DATA/src/terraform/* jump01:~/terraform/
sending incremental file list

sent 2,607 bytes  received 32 bytes  1,759.33 bytes/sec
total size is 1,343,000  speedup is 508.90

VNC over SSH Bastion host

You may ask why and the answer is just sometimes you have to do stupid things.

bastion == jump host

Setup a tunnel

$ ssh -f -N -p 22 -L 55901:10.35.5.6:5901 -i customer-priv-key opc@<bastion public IP>

Run vnc server

$ vncserver 
New 'ociserver:1 (opc)' desktop is ociserver:1
Starting applications specified in /home/opc/.vnc/xstartup
Log file is /home/opc/.vnc/ociserver:1.log

Note above vncserver also has a custom startup to bypass the systemwide xinit whihc was spawning gnome desktop.


$ pwd
/home/opc/.vnc

$ more xstartup 
#!/bin/sh
# unset SESSION_MANAGER
# unset DBUS_SESSION_BUS_ADDRESS
#exec /etc/X11/xinit/xinitrc
#!/bin/sh
xrdb $HOME/.Xresources
xsetroot -solid grey
xterm -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &

Connect

$ vncviewer localhost:55901
TigerVNC Viewer 64-bit v1.7.0
Tue Feb 20 13:14:43 2018
 DecodeManager: Detected 1 CPU core(s)
 DecodeManager: Decoding data on main thread
 CConn:       connected to host localhost port 55901
 CConnection: Server supports RFB protocol version 3.8
 CConnection: Using RFB protocol version 3.8
 CConnection: Choosing security type VeNCrypt(19)

Tue Feb 20 13:14:44 2018
 CVeNCrypt:   Choosing security type TLSVnc (258)

Tue Feb 20 13:14:51 2018
 X11PixelBuffer: Using default colormap and visual, TrueColor, depth 24.
 CConn:       Using pixel format depth 24 (32bpp) little-endian rgb888
 CConn:       Using Tight encoding
 CConn:       Enabling continuous updates

Bash variable in an awk search pattern

I spent some time getting this working so documenting for reference.

Trying to grab an IP address from a ssh config file works fine using a static string.

$ awk '/^Host rdpclient1$/{x=1}x&&/HostName/{print $2;exit}' ~/.ssh/Prod-config
10.1.4.4

Using double quotes and a static variable or double quotes and a bash variable does not work.

$ awk "/^Host rdpclient1$/{x=1}x&&/HostName/{print $2;exit}" ~/.ssh/Prod-config
     HostName 10.1.4.4
$ awk "/^Host $host$/{x=1}x&&/HostName/{print $2;exit}" ~/.ssh/Prod-config
     HostName 10.1.4.4

Using double quotes plus a bash variable and escaping the print variable $2 works.

$ awk  "/^Host ${host}$/{x=1}x&&/HostName/{print \$2;exit}" ~/.ssh/Prod-config
10.1.4.4

And an in a little script that use ssh config settings for my RDP through a jumphost or bastion Linux server.

Host gw01
     HostName <public IP>
     User opc
     IdentityFile mysshkey
Host rdpclient1
     HostName 10.1.4.4
     ProxyJump gw01

$ cat rdesktop_jumphost.sh 
#!/bin/bash
host=$1
privateIP=$(awk  "/^Host ${host}$/{x=1}x&&/HostName/{print \$2;exit}" ~/.ssh/Prod-config)
jumphost=$(awk "/^Host ${host}$/{x=1}x&&/ProxyJump/{print \$2;exit}" ~/.ssh/Prod-config)
jumphostIP=$(awk "/^Host ${jumphost}$/{x=1}x&&/HostName/{print \$2;exit}" ~/.ssh/Prod-config)
jumpuser=$(awk "/^Host ${host}$/{x=1}x&&/User/{print \$2;exit}" ~/.ssh/Prod-config)
localRdpPort=33389

ssh -f -N -p 22 -L $localRdpPort:$privateIP:3389 -i mysshkey $jumpuser@$jumphostIP
tunnelpid=$(ps -ef | grep $localRdpPort | grep -v grep | awk '{print $2}')

rdesktop localhost:$localRdpPort

kill $tunnelpid

SSH JumpHost

Newer versions of ssh simplified the ProxyCommand directive a little in config files with ProxyJump directive.

A command line ProxyCommand may work like this for you:

$ ssh -i private_key -o "ProxyCommand ssh -W %h:%p -i private_key user@<jumphost IP address" user@<private IP address>

Config file entries like this:

$ more config 
Host gw01
     HostName <jumphost IP>
     User <username>
     IdentityFile /full/path/private_key
Host server1
     HostName 10.2.3.3
     ProxyJump gw01
     User <username>
     IdentityFile /full/path/private_key

Using ssh config you can simply ssh like this:

$ ssh server1

Or better if you have many projects using ssh with a custom config file:

$ ssh -F my-config server1

Note: I am not able to use ProxyJump as a command line one liner with the -J flag when I have private keys on both the JumpHost and Private Host. For example below does not specify the -i for the jump host and not sure it will accept it command line.

$ ssh -i key -J user@public-IP user@private-IP

Couple examples of scp using the config file and/or jumping:

rrosso@rrosso-VirtualBox:~/.ssh$ scp -F my-config -oProxyJump=gw01 /media/antergos-17.6-x86_64.iso  host01:/pool/

rrosso@rrosso-VirtualBox:~/.ssh$ scp -F my-config  /media/antergos-17.6-x86_64.iso  gw01:
Posted in SSH

RDP Through SSH Server

Sometimes it becomes necessary to access Windows hosts not exposed externally and you do have a SSH server that is exposed as a “jumphost”. Quick notes on my usage.

Create the tunnel to the jumpbox.

$ ssh -p 22 -L 13389:10.3.1.4:3389 -i my-ssh-key user@<public-IP>
Last login: Tue Sep 19 16:49:54 2017

Connect using RDP to the local host:port.

$ rdesktop localhost:13389
Autoselected keyboard map en-us
Failed to negotiate protocol, retrying with plain RDP.
WARNING: Remote desktop does not support colour depth 24; falling back to 16

Example script…

$ cat rdesktop_jumphost.sh 
#!/bin/bash
#
#: Script Name  : rdesktop_jumphost.sh
#: Version      : 0.1.3
#: Author       : Riaan Rossouw
#: Date Created : October 21, 2017
#: Date Updated : October 22, 2017
#: Description  : Use ssh config file to pull enough information to rdp to windows servers through a ssh jumphost
#: Examples     : rdesktop_jumphost.sh -F configfile -u user -g 1024x768

usage()
{
cat << EOF
usage: $0 options

This script use ssh config file to pull enough information to rdp to windows servers through a ssh jumphost

OPTIONS:
           -h show this message.
           -F ssh config file (required).
 	   -s servername (HostName in ssh config) (required).
           -u pass username to rdesktop.
	   -g desktop geometry (WxH)
EOF
}

while getopts "hF:s:u:g:" OPTION
 do
  case $OPTION in
   h) usage; exit 1;;
   F) configfile=$OPTARG;;
   s) HostName=$OPTARG;;
   u) username=$OPTARG;;
   g) geometry=$OPTARG;;
   \?) usage; exit 1;;
  esac
 done

NUMARGS=$#
if [ $NUMARGS -eq 0 ]; then
  usage
  exit 1
fi

PARAMS="-u $username"
PARAMS+=" -g $geometry"

localRdpPort=33389

privateIP=$(awk  "/^Host ${HostName}$/{x=1}x&&/HostName/{print \$2;exit}" ~/.ssh/$configfile)
jumphost=$(awk "/^Host ${HostName}$/{x=1}x&&/ProxyJump/{print \$2;exit}" ~/.ssh/$configfile)

if [ -z "$jumphost" ]
then
  rdesktop $PARAMS privateIP:3389
else
  jumphostIP=$(awk "/^Host ${jumphost}$/{x=1}x&&/HostName/{print \$2;exit}" ~/.ssh/$configfile)
  jumpuser=$(awk "/^Host ${jumphost}$/{x=1}x&&/User/{print \$2;exit}" ~/.ssh/$configfile)
  identityfile=$(awk "/^Host ${jumphost}$/{x=1}x&&/IdentityFile/{print \$2;exit}" ~/.ssh/$configfile)
  ssh -f -N -p 22 -L $localRdpPort:$privateIP:3389 -i $identityfile $jumpuser@$jumphostIP
  tunnelpid=$(ps -ef | grep $localRdpPort | grep -v grep | awk '{print $2}')
  rdesktop $PARAMS localhost:$localRdpPort
  kill $tunnelpid
fi

SSH password manager

I have recently started using a distro called BunsenLabs which is a Debian/Openbox flavor. I run Linux inside VirtualBox and so far I really like this distro.  I have previously written about using Linux SSH connection managers like PAC(Perl Auto Connect), GCM, Remmina etc.  I have mostly settled on PAC for most Linux installations but it has a couple irritations and seem to be getting pretty old.  My goals is mostly to keep track of sometimes hundreds of machine names/usernames/passwords.  Couple options I have played with is putty from the command line and sshpass.

For putty something like this could probably be built on:

$ putty -load host01 -l root -pw mypass

With sshpass something like this works. Assuming you have sshpass to install on your distro.

$ sshpass -p 'mypass' ssh -o StrictHostKeyChecking=no root@host01
Last login: Sun Feb 19 11:11:45 2017 from 10.140.6.123
[root@HOST01 ~]# 

Since I am using OpenBox here I added a custom OpenBox pipemenu by changing the existing SSH pipemenu a little bit. This works for me but I will probably change it a little bit in future to use a better config file with XML and/or encrypting the details.

Create a config folder and file to save the host details. For now config file is in SSH config format so the existing paramiko.config class can still read it. WARNING nothing about saving passwords like this is secure. You were warned!

$ cat .sshpassdb/config 
Host host01
  HostName host01.localdomain
  User root:mypass

# Test complete command line that we will try in Openbox menus
$ x-terminal-emulator -e sshpass -p 'mypass' ssh -o StrictHostKeyChecking=no root@host01

Now lets add an Openbox pipemenu.

Right click desktop -> Preferences -> Openbox -> GUI Menu Editor
Expand Openbox 3 and add a pipemenu. I called is SSH (sshpass) and pointed it to /home/myuser/scripts/bl-sshpass-pipemenu
I copied cp /usr/bin/bl-sshconfig-pipemenu /home/myuser/scripts/bl-sshpass-pipemenu
Edit new this custom python file now to populate the Openbox custom menu when opened

$ cat scripts/bl-sshpass-pipemenu

#!/usr/bin/env python
#    bl-sshpass-pipemenu - an Openbox pipemenu for Graphics applications
import os
import warnings
with warnings.catch_warnings():
    warnings.simplefilter("ignore")
    from paramiko.config import SSHConfig
import argparse
import sys

ap = argparse.ArgumentParser(description="""Openbox pipemenu to handle secure shell installation and configuration.
The install menu item is only shown when '/usr/sbin/sshd' is not executable. """)
opts = ap.parse_args(sys.argv[1:])

cfgdir = os.getenv("HOME")+"/.sshpassdb"
cfgfile = cfgdir+"/config"

try:
    config_file = file(cfgfile)
except IOError:
    if not os.path.exists(cfgdir):
        os.makedirs(cfgdir, 0700)
    f = open(cfgfile, 'w')
    o = '# SSH config file, \'man ssh_config\' for more details.\n\n'
    o += '#Host example\n'
    o += '#    hostname example.com\n'
    o += '#    user joebloggs\n'
    f.write(o)
    f.close()
    os.chmod(cfgfile, 0600)
    config_file = file(cfgfile)
    config = SSHConfig()
    config.parse(config_file)
    hosts = config._config
else:
    config = SSHConfig()
    config.parse(config_file)
    hosts = config._config

print '<openbox_pipe_menu>\n'

need_separator = False

if len(hosts) >= 2:
    for h in hosts[1:]:
        if 'host' in h and 'hostname' in h['config']:
            conf = h['config']
            user = ''
            if 'user' in conf:
                user = conf['user'].split(':')[0]
                passw = ' -p ' + conf['user'].split(':')[1] + ' '
            port = ['', '']
            if 'port' in conf:
                port[0] = '-p ' + conf['port'] + ' '
                port[1] = ':' + conf['port']
            if need_separator:
                print '<separator/>\n'
                need_separator = False
            print '<menu id="ssh-'+h['host'][0]+'" label="'+h['host'][0]+'">'
            print '    <item label="Start terminal session">'
            print '        <action name="Execute">'
            print '            <command>'
            print '                x-terminal-emulator -e sshpass ' + passw + 'ssh -o StrictHostKeyChecking=no ' + user + '@' + conf['hostname']
            print '            </command>'
            print '        </action>'
            print '    </item>\n'
            print '</menu>\n'
    print '<separator/>\n'

if need_separator:
    print '<separator/>\n'
    need_separator = False

print '</openbox_pipe_menu>'

Test with Right Click on desktop -> Network -> SSH (sshpass) and select a host -> Start terminal session.
Add hosts to config file.

PAC Manager Login Issue

I had auto username / password login issues and suspect it was because of a long /etc/issue warning during logon.

This worked for me:

Edit Connection -> SSH Options -> Advanced Options -> Add
Right Click Option to List and Select LogLevel
Give it QUIET as a Value and Save and Close

Also see post on selection issue:

PAC Manager Double Click Selection

Posted in SSH

Solaris SFTP Containment Multiple Nodes

Previous post explaining SFTP containment: http://blog.ls-al.com/sftp-containment-solaris-10/

That solution does not work in a clustered environment. Since then I did also play with loop back (LOFS in Solaris) mounts to a NFS folder. That also works but it had issues being in the vfstab at boot time.

Below is my final solution:
– Since i am trying to avoid number of mounts I also used autofs in this case.
– Create a NFS share INTERFACES so we can share across multiple nodes.
– In order to not add more mounts I did this with autofs. If that does not work on bootup we can can just make a permanent /etc/vfstab mount.
– In our case the application use the following logical path so we need a soft link to our containment area. Soft link svcaccxfr -> /opt/interfaces/svcaccxfr/ in application tree.

Make direct automount
# grep direct /etc/auto_master
/- auto_direct -ro
# cat /etc/auto_direct
/opt/interfaces -rw,vers=3 10.2.13.35:/export/INTERFACES

# svcadm refresh autofs
# svcadm restart autofs

Ensure match in sshd correct folder
# tail -10 /etc/ssh/sshd_config
Match User svcxfr
ChrootDirectory /opt/interfaces/svcxfr
AllowTCPForwarding no
X11Forwarding no
ForceCommand internal-sftp -u 017 -l info

Folders and permissions
# cd /opt
# ls -l | grep interfaces
drwxr-xr-x 3 root root 3 Dec 21 14:12 interfaces
# cd interfaces/
# ls -l | grep svcxfr
drwxr-xr-x 3 root root 3 Dec 21 14:13 svcxfr
# ls -l | grep svcxfr/uploads
# cd svcxfr/
# ls -l | grep uploads
drwxrwxr-x 2 ebsppe_a ebsppe 3 Dec 21 14:50 uploads

Check soft link
# cd /apps/ebs11i/appltop/xxnp/11.5.0/interfaces
# ls -l | grep interfaces
lrwxrwxrwx 1 root root 26 Dec 21 14:14 svcxfr -> /opt/interfaces/svcxfr/

Test client
$ sftp svcxfr@server1
Password:
Connected to server1.
sftp> dir
uploads
sftp> cd uploads
sftp> put zfsrest_test1.py
Uploading zfsrest_test1.py to /uploads/zfsrest_test1.py
zfsrest_test1.py 100% 1934 1.9KB/s 00:00
sftp> exit

Can check sftp issues here.

For example sftp containment does not work if root does not own top levels.
# tail -f /var/log/authlog
Dec 21 14:49:48 server1 sshd[12790]: [ID 800047 auth.info] Accepted keyboard-interactive for svcxfr from 192.168.38.104 port 39788 ssh2
Dec 21 14:49:49 server1 sshd[12790]: [ID 800047 auth.info] subsystem request for sftp
Dec 21 14:50:04 server1 sshd[12790]: [ID 800047 auth.info] Received disconnect from 192.168.38.104: 11: disconnected by user

Unable to negotiate ssh-dss

openssh version deprecated DSA keys by default. To work around it:

Update in ~/.ssh/config:

Host your-host
    HostkeyAlgorithms +ssh-dss

or

ssh -oHostKeyAlgorithms=+ssh-dss user@host
Posted in SSH