Shell
Usage
Please read #2FA for initial setup. Then simply use SSH to login to this machine:
~# ssh username@shell.stud.informatik.uni-goettingen.de ####### shell.stud.informatik.uni-goettingen.de - login vm: shell5.cip.loc ... Verification code: Password: Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-87-generic x86_64) username@shell5:~$
Verify Key Fingerprint
The first time you connect to any ssh-reachable resource via an unstrusted network you need to verify the fingerprint of the used key and only then accept it. For our shellX-machines the set of valid public key fingerprints is:
ECDSA 07:84:c9:e1:59:4f:03:75:69:b1:e4:d0:b4:1f:9a:cd ED25519 93:11:29:c4:a2:03:e1:2d:b1:82:05:74:dd:a5:3b:9a RSA de:db:6e:72:52:de:30:73:db:bb:6e:79:df:f9:2c:0d
Target audience
These machines are meant to be used by students. But of course they can be used by all staff members!
First time users: the only pre-condition is to logon one single time using one of the (physical) pool computers in our building - this will make you a "known user" to our systems. Only then you can (and must) walk through #2FA.
Windows
For Windows: use PuTTY (simple) or Cygwin (more complex and powerful) or any other SSH-implementation.
Load Balancing
In fact this term is misleading on this specific installation as it simple does "round-robin", at least for now. The important point is that you'll get connected to any currently available login machine. This will be the "next" machine one after another and probably not the same one as one session before. If you landed on an overcrowded system simply disconnect/reconnect to use another machine.
Timeout
- The session Timeout is set to 36 hours -- this is the HAproxy related Timeout regarding the TCP connection
- Kerberos/OpenAFS have separate/shorter timeouts, usually 10 hours. Please check with klist. You need to run kinit && aklog when you're approaching timeout
Self defense of these servers
To fight brute force attacks we tried several mechanisms including fail2ban, knockd and rate limiting. All of them worked technically correctly. But all of them could not reduce the attacks to an acceptable low level. One consequence is the introduction of #2FA as a requirement.
Blacklists
Finally — as of 08.August 2016 — there are several external(!) lists involved. These lists are queried twice a day and a simple "iptables ... -j DROP" evaluates an aggregated list. It contains more than 40000 single addresses and 900 networks. The attempts to login as root dropped from >10000 (peaks were >80000) per day to a few dozen.
- "http://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1" # Project Honey Pot Directory of Dictionary Attacker IPs
- "https://check.torproject.org/cgi-bin/TorBulkExitList.py?ip=1.1.1.1" # TOR Exit Nodes
- "https://www.maxmind.com/en/proxy-detection-sample-list" # MaxMind GeoIP Anonymous Proxies
- "http://danger.rulez.sk/projects/bruteforceblocker/blist.php" # BruteForceBlocker IP List
- "https://www.spamhaus.org/drop/drop.lasso" # Spamhaus Don't Route Or Peer List (DROP)
- "http://cinsscore.com/list/ci-badguys.txt" # C.I. Army Malicious IP List
- "https://www.openbl.org/lists/base.txt" # OpenBL.org 30 day List
- "https://lists.blocklist.de/lists/all.txt" # blocklist.de attackers
- "https://www.stopforumspam.com/downloads/toxic_ip_cidr.txt" # StopForumSpam
- "http://blocklist.greensnow.co/greensnow.txt" # GreenSnow
Please give feedback if you feel this list to be too restrictive.
2FA
Two Factor Authentication -- required, not optional!
Concept
We use the well known google-authenticator to add a second factor as a requirement for (ssh-) logins. First you will get prompted for a "Verification code:". Then you'll get a second prompt asking for your normal password.
The "Verification Code" changes every minute, this approach is called TOTP = Time-based One Time Password.
(Do not try to use "Counter based OTP". It might work first, but it will do so only for a short while! We are using copies of the secret file. State updates required by the incremental counter strategy are not written back. Authentication will fail after reaching the windows size.)
The order of both inputs is relevant: if an attacker manages to crack the first element (being the TOTP) he has a benefit for some minutes only. If we would ask for the Password first then the benefit of cracking the first element gives advantages probably for a very long time.
You need to have a compatible generator - usually implemented as a small application.
Please note that often this approach is associated with a specific implementation: the Google Authenticator. This is misleading as there are other 100% compatible implementations. See also RFC 6238.
Initialization
Before you can use this technology the first time you need to prepare your personal secret credentials. You do this by using a simple command line tool and answering some questions.
Of course you can not do this on these shellX-machines as you can not login successfully - this is the classic chicken-and-egg problem. Use one of the physical pool computers for this!
(Your personal/private computer is probably not an option as you need access to your OpenAFS $HOME. If you do have access you might need to install the software by apt-get install libpam-google-authenticator which contains the required binary.)
The following instructions are copy-n-pastable as the commands are relative to anyones $HOME-folder.
~$ google-authenticator Do you want authentication tokens to be time-based (y/n) y ... # For full output see Shell/2fa-example
Due to some unusual behavior of OpenAFS regarding access rights (they work only on directories, not on files) we need to move that file into another, dedicated subdirectory. This man page explains the access rights mechanism and how to manipulate access-control-lists:
~$ man fs_setacl
First you need to create that directory. A special user with the name ifi-login needs to have read access to the files in the directory .ifi-login inside of your $HOME. To be able to reach into that directory he needs to "walk through" your home folder. The third line is required to make this possible by granting "l"="list" access rights to your $HOME:
~$ mkdir .ifi-login ~$ fs sa -dir .ifi-login -acl ifi-login read ~$ fs sa -dir . -acl ifi-login l
Be aware that we are working with "dotfiles": both .ifi-login & .google_authenticator begin with a "." and are usually hidden from users eye. To see them use ls -a.
As usual access rights are inherited. For this reason there are more rights granted than required. You might remove them now by commands like
~$ fs sa -dir .ifi-login -acl mta none ~$ fs sa -dir .ifi-login -acl spamassassin none ~$ fs sa -dir .ifi-login -acl web-home none
You can always check the current settings. At the end it may look like this:
~$ fs la .ifi-login Access list for .ifi-login is Normal rights: system:administrators rlidwka username rlidwka username.system rl ifi-login rl # this is the important one (in this context)
WARNING: do not remove rights if you are not absolutely sure they are not needed. It is very easy to remove too many rights, leaving you with a directory that is not usable anymore!
Now move the created credential file into that new destination:
~$ mv .google_authenticator .ifi-login/
Please remember to repeat this step if you modify/recreate your configuration!
Generators
The system time is used equivalent to a shared secret! Make sure your clock is set correctly or all generated codes will fail. |
For all generators you need the secret created above. You can use any tool you want to look into the file .ifi-login/.google_authenticator. A one-liner which outputs only the "secret" is this:
~$ head -n1 .ifi-login/.google_authenticator P2ZOMKQLEIC6SKCL
- Android
- Play Store: "Google Authenticator".
- F-Droid: https://f-droid.org/app/com.google.android.apps.authenticator2
- Linux
- install oathtool to get some compatible command line utilities. Then this works:
~$ oathtool --totp -b $(head -n1 .ifi-login/.google_authenticator) 123456
- Ubuntu Touch
- Authenticator
- Windows:
- WinAuth: https://github.com/winauth/winauth -- direct download as of 06.2016: https://winauth.com/downloads/3.x/WinAuth-3.5.1.zip
This is an installation-free application, no setup and no administrative access needed.
- WinAuth: https://github.com/winauth/winauth -- direct download as of 06.2016: https://winauth.com/downloads/3.x/WinAuth-3.5.1.zip
- Windows Phone:
- Token2: free application from Microsoft Store. The QR-Code seems to be incompatible, so you need to type in your secret manually. Nevertheless: it works.
- OS agnostic
- Chromium Browser: GAuth application
- https://5apps.com/gbraad/gauth -- direct use web-application (think twice!) & application for Chrome and Firefox
Tips 'n' Tricks
Connecting to a specific machine
If you are trying to connect to the same machine as last time (for example to execute Long Running Processes) you need to connect from a semi-local source address which include:
- 134.76.0.0/16 - official Gönet address space
- 10.0.0.0/8 - official Gönet wide routed RFC1918 address space
- 172.16.0.0/12 - Institute local address space
In this case circumventing the Round-Robin mechanism is possible by connecting to a specific port 42000+n with n={1..6}. The example connects to machine number 4:
~$ ssh -p 42004 username@shell.stud.informatik.uni-goettingen.de ####### ####### shell.stud.informatik.uni-goettingen.de - login vm: shell4.cip.loc
Duplicate your Generators
It is absolutely fine to have a well configured generator on every single device you own. Remember: without the second factor you can not login. That's the goal of the whole shebang after all.
Write down your Emergency codes
Remember the console output during creation of the secret? "Your emergency scratch codes are:...". Write them down (or print them) and put that piece of paper into your pocket...
Credential problems
- the actual password is not stored anywhere in Institute's systems
- students: StudIT - https://wiki.student.uni-goettingen.de/support/account/passwort
- staff: Gwdg - https://info.gwdg.de/faq/index.php?action=artikel&cat=52&id=215&artlang=de - Institute's Admins will help as we have some administrative access. While we can not tell you your password we can reset it
- problems with the Verification Code: simply start again with #Initialization and overwrite ~/.ifi-login/.google_authenticator. You need to re-configure all of your #Generators of course
Todo
- Testing! -- the current state is considered "BETA"
- ssh-ed25519
See also
- Remote Access
- Long Running Processes -- leave processes running
- 2FA/Multiple -- use multiple identities