Shell

Beta Test! Please give feedback via feedback(ät)informatik.uni-goettingen.de For problem reports: please include always information about your system - the exact date+time, your IP address, your user id, what you wanted to accomplish, what you did and what happened instead. To remove this warning I need some positive feedback first...

Usage

Please read #2FA for initial setup. Then simply use SSH to login to this machine:

~# ssh username@shell.stud.informatik.uni-goettingen.de 
#######  shell.stud.informatik.uni-goettingen.de - login vm: shell5.cip.loc
...
Verification code: 
Password: 
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-87-generic x86_64)
username@shell5:~$ 

For Windows: use PuTTY (simple) or Cygwin (more complex and powerful) or any other SSH-implementation.

Target audience

These machines are meant to be used by students. But of course they can be used by any staff members!

For first time users: the only requirement is to logon one single time using one of the (physical) pool computers in our building - this will make you a "known user" to our systems. Additionally you need to walk through #2FA.

Load Balancing

In fact this term is misleading on this specific installation as it simple does "round-robin", at least for now. The important point is that you'll get connected to any currently available login machine. This will be the "next" machine one after another and probably not the same one as one session before. If you landed on an overcrowded system simply disconnect/reconnect to use another machine.

Timeout

• The session Timeout is set to 36 hours -- this is the HAproxy related Timeout regarding the TCP connection
• Kerberos/OpenAFS have separate/shorter timeouts, usually 10 hours. Please check with klist. You need to run kinit && aklog when you're approaching timeout

Self defense of these servers

To fight brute force attacks we tried several mechanisms including fail2ban, knockd and rate limiting. All of them worked technically correctly. But all of them could not reduce the attacks to an acceptable low level. One consequence is the introduction of #2FA as a requirement.

Blacklists

Finally — as of 08.August 2016 — there are several external(!) lists involved. These lists are queried twice a day and a simple "iptables ... -j DROP" evaluates an aggregated list. Currently it contains approx. 38000 single addresses and 900 networks. The attempts to login as root dropped from >10000 (peaks were >80000) per day to basically zero!

• "http://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1" # Project Honey Pot Directory of Dictionary Attacker IPs
• "https://check.torproject.org/cgi-bin/TorBulkExitList.py?ip=1.1.1.1" # TOR Exit Nodes
• "https://www.maxmind.com/en/proxy-detection-sample-list" # MaxMind GeoIP Anonymous Proxies
• "http://danger.rulez.sk/projects/bruteforceblocker/blist.php" # BruteForceBlocker IP List
• "https://www.spamhaus.org/drop/drop.lasso" # Spamhaus Don't Route Or Peer List (DROP)
• "http://cinsscore.com/list/ci-badguys.txt" # C.I. Army Malicious IP List
• "https://www.openbl.org/lists/base.txt" # OpenBL.org 30 day List
• "https://lists.blocklist.de/lists/all.txt" # blocklist.de attackers
• "https://www.stopforumspam.com/downloads/toxic_ip_cidr.txt" # StopForumSpam
• "http://blocklist.greensnow.co/greensnow.txt" # GreenSnow

Please give feedback if you feel this list to be too restrictive.

2FA

Two Factor Authentication -- required, not optional

Concept

We use the well known google-authenticator to add a second factor as a requirement for (ssh-) logins. First you will get prompted for a "Verification code:". Then you'll get a second prompt asking for your normal "Password:".

The "Verification Code" changes every minute, this approach is called TOTP = Time-based One Time Password.

(Do not try to use "Counter based OTP". It might work first, but it will do so only for a short while! We are using copies of the secret file. State updates required by the incremental counter strategy are not written back. Authentication will fail after reaching the windows size.)

The order of both inputs is relevant: if an attacker manages to crack the first element (being the TOTP) he has a benefit for some minutes only. If we would ask for the Password first then the benefit of cracking the first element gives advantages probably for a very long time.

You need to have a compatible generator - usually implemented as a small application.

Please note that often this approach is associated with a specific implementation: the Google Authenticator. This is misleading as there are other 100% compatible implementations. See also RFC 6238.

Initialization

Before you can use this technology the first time you need to prepare your personal secret credentials. You do this by using a simple command line tool and answering some questions.

Of course you can not do this on these shellX-machines as you can not login successfully - this is the classic chicken-and-egg problem. Use one of the physical pool computers for this!

(Your personal/private computer is probably not an option as you need access to your OpenAFS $HOME. If you do have access you might need to install the package apt-get install libpam-google-authenticator which contains the required binary.)

The following instructions are copy-n-pastable as the commands are relative to anyones $HOME-folder.

~$ google-authenticator      
Do you want authentication tokens to be time-based (y/n) y
... # For full output see Shell/2fa-example

Due to some unusual behavior of OpenAFS regarding access rights (they work only on directories, not on files) we need to move that file into another, dedicated subdirectory. This man page explains the access rights mechanism and how to manipulate access-control-lists:

~$ man fs_setacl

First you need to create that directory. A special user with the name ifi-login needs to have read access to the files in the directory .ifi-login inside of your $HOME. To be able to reach into that directory he needs to "walk through" your home folder. The third line is required to make this possible by granting "l"="list" access rights to your $HOME:

~$ mkdir .ifi-login     
~$ fs sa -dir .ifi-login -acl ifi-login read 
~$ fs sa -dir . -acl ifi-login l

Be aware that we are working with "dotfiles": both .ifi-login & .google_authenticator begin with a "." and are usually hidden from users eye. To see them use ls -a.

As usual access rights are inherited. For this reason there are more rights granted than required. You might remove them now by commands like

~$ fs sa -dir .ifi-login -acl mta none
~$ fs sa -dir .ifi-login -acl spamassassin none
~$ fs sa -dir .ifi-login -acl web-home none

You can always check the current settings. At the end it may look like this:

~$ fs la .ifi-login
Access list for .ifi-login is
Normal rights:
 system:administrators rlidwka
 username rlidwka
 username.system rl
 ifi-login rl    # this is the important one (in this context) 

WARNING: do not remove rights if you are not absolutely sure they are not needed. It is very easy to remove too many rights, leaving you with a directory that is not usable anymore!

Now move the created credential file into that new destination:

~$ mv .google_authenticator .ifi-login/

Please remember to repeat this step if you modify/recreate your configuration!

Generators

The system time is used equivalent to a shared secret! Make sure your clock is set correctly or all generated codes will fail.

For all generators you need the secret created above. You can use any tool you want to look into the file .ifi-login/.google_authenticator. A one-liner which outputs only the "secret" is this:

~$ head -n1 .ifi-login/.google_authenticator 
P2ZOMKQLEIC6SKCL

• Android
  • Play Store: "Google Authenticator".
  • F-Droid: https://f-droid.org/app/com.google.android.apps.authenticator2

• Linux
  • install oathtool to get some compatible command line utilities. Then this works:

~$ oathtool --totp -b $(head -n1 .ifi-login/.google_authenticator)
123456

• Ubuntu Touch
  • Authenticator

• Windows:
  • WinAuth: https://github.com/winauth/winauth -- direct download as of 06.2016: https://winauth.com/downloads/3.x/WinAuth-3.5.1.zip
    This is an installation-free application, no setup and no administrative access needed.

• OS agnostic
  • Chromium Browser: GAuth application
  • https://5apps.com/gbraad/gauth -- direct use web-application (think twice!) & application for Chrome and Firefox

Tips 'n' Tricks

Connect to a specific machine

If your are using a semi-local source address from inside Gönet or inside the Institute = 134.76.0.0/16 + 10.0.0.0/8 + 172.16.0.0/12 circumventing the Round-Robin mechanism is possible: connect to a specific port 42000+n with n={1..6}

For machine number 4:

~$ ssh -p 42004 username@shell.stud.informatik.uni-goettingen.de
#######
#######  shell.stud.informatik.uni-goettingen.de - login vm: shell4.cip.loc

Duplicate your Generators

It is absolutely fine to have a well configured generator on every single device you own. Remember: without the second factor you can not login. That's the goal of the whole shebang after all.

Write down your Emergency codes

Remember the console output during creation of the secret? "Your emergency scratch codes are:...". Write them down (or print them) and put that piece of paper into your pocket...

Credential problems

• the actual password is not stored anywhere in Institute's systems
  • students: StudIT - https://wiki.student.uni-goettingen.de/support/account/passwort
  • staff: Gwdg - https://info.gwdg.de/faq/index.php?action=artikel&cat=52&id=215&artlang=de - Institute's Admins will help as we have some administrative access. While we can not tell you your password we can reset it

• problems with the Verification Code: simply start again with #Initialization and overwrite ~/.ifi-login/.google_authenticator. You need to re-configure all of your #Generators of course

Todo

• Testing! -- the current state is considered "BETA"

See also

• Remote Access
• Long Running Processes -- leave processes running
• 2FA/Multiple -- use multiple identities

Links

• https://tools.ietf.org/html/rfc6238
• https://github.com/google/google-authenticator