High Performance Networking with the SSH Protocol

1. High Performance Networking with the SSH Protocol Chris Rapier rapier@psc.eduVancouver Joint Techs July 19, 2005

2. Defining the Problem • Transferring data over insecure networks requires cryptographically secure authentication • Recent history has highlighted the need for this. RCP and FTP are no longer viable options • It should be fast, easy to use, and cheap to install/maintain. Pick two. Sometimes only one. Pittsburgh Supercomputing Center

3. First Solution: GridFTP • Pros • Cryptographically secure authentication • High transfer rates for bulk data • Cons • Difficult to install and maintain • Key distribution requires additional infrastructure • Often out of reach of smaller organizations • Limited distribution Pittsburgh Supercomputing Center

4. Next Solution: Kerberos • Pros • Relatively widely used protocol • Familiar interfaces (ftp, web, etc) • Fast transfer rates • Cons • Maintenance and configuration beyond some organizations • Insecure data transfers Pittsburgh Supercomputing Center

5. Another Solution: SCP/SSH • Pros • Strong cryptographic security of authentication and data • Easy to install and maintain • Ubiquitous • Cons • Astoundingly slow • Somewhat less secure than kerberos or globus (no signed certs, expiring tokens) Pittsburgh Supercomputing Center

6. Using SCP/SSH Anyway • In spite of faults SCP/SHH ends up being the default for most people. • This can be a *bad* thing • Security depends on universal compliance. • People will ‘drift’ to using the easiest solution. • SSH is the easiest *security* solution but the speed is frustrating. Some people will, eventually, resort to insecure but fast methods ‘just this once’ • Improve SSH performance and the security environment as a whole is enhanced and we’ll have fast, cheap, secure - pick three. • So why is SSH so slow? Pittsburgh Supercomputing Center

7. The Real Problem With SSH • It is *NOT* the encryption process! • If it was: • Faster computers would give faster throughput. Which doesn’t happen. • Transfer rates would be constant in local and wide area network. Which they aren’t. • In fact transfer rates seem dependent on RTT, the farther away the slower the transfer. • Any time rates are strongly linked to RTT it implies a receive buffer problem Pittsburgh Supercomputing Center

8. SSH is RWIN Limited • Analysis of the code reveals • SSH Protocol V2 is multiplexed • Multiple channels over one TCP connection • Must implement a flow control mechanism per channel • Essentially the same as the TCP receive window • This application level RWIN is effectively set to 64KB. So real connection RWIN is MIN(TCPrwin, SSHrwin) • Thus TPUTmax = 64KB/RTT Pittsburgh Supercomputing Center

9. Solving the Problem • Use getsockopt() to get TCPrwin and dynamically set SSHrwin • Performed several times throughout transfer to handle autotuning kernels • Results in 10x to 50x faster throughput depending on cipher used on well tuned system. Pittsburgh Supercomputing Center

10. HPN-SSH v. SSH Pittsburgh Supercomputing Center

11. Advantages • Speed is comparable to GridFTP and Kerberized FTP • No need for separate key infrastructure or realm administration • Provides authentication and data security • Can be used with other applications such as rsync, svn, SFTP, ssh port forwarding & more Pittsburgh Supercomputing Center

12. What’s involved? • Get the source code from www.openssh.org • Get the patch from www.psc.edu tar -zxf openssh-3.9p1.tgzcd openssh-3.9p1patch < openssh-3.9p1-hpn.diffconfigure make install or make install-nokeys • Time elapsed: < 5 minutes Pittsburgh Supercomputing Center

13. So what does that get you? • Speed increase in direction of HPN • No need to have HPN-SSH on both sides • Can set TCPrwin on the command line • To maximum buffer size allowed by system configuration • None cipher re-enabled • Using mid stream cipher switching authentication is still secure. Pittsburgh Supercomputing Center

14. New SSH Tool - PMVPN • The Poor Man’s VPN • Use ld_preload to load a custom library that captures all network open calls. • If it matches a rule then tunnel the connection through SSH • Requires ssh keys to be installed on remote host • Transparent. Secure. Most things ‘just work’ • Caveat: FTP has to run in passive mode. Pittsburgh Supercomputing Center

15. That’s a Neat Trick • Use PMVPN to secure ‘coffeehouse connections’ • Designate all outgoing connections to use ssh tunnels back to ‘home’ machine. • Every packet on wireless network is now encrypted without WEP or WAP • Always use the same SMTP server • Connections to SMTP are intercepted and tunneled back to your network. • Use as a secure proxy. • Web browsing can be effectively anonymized. Pittsburgh Supercomputing Center

16. It’s Not Perfect • FTP must be in passive mode • Even then it fails if using a redirect • Establishes a new connection each time • So browsing the web means a new SSH handshake for each new http connection • Other applications may or may not work as expected - we’re still testing • EVEN SO • Its easy to use, works reasonably well, and provides security here and now *without* additional infrastructure. Pittsburgh Supercomputing Center

17. Big Picture Ideas • Impact of hpn-ssh work will obscure packet contents • Multiple apps using port 22, no way to determine which is doing what, payload inspection methods can be easily circumvented. • Overall security of network environments will improve. • People will *invariably* circumvent security if its inconvenient - especially in academic environments. • Make the lowest acceptable level of security equivalent to insecure methods in terms of ease of use and performance. • Remember, most people will *not* strictly follow security guidelines unless they are pressured to or its easy. So make it easy. Pittsburgh Supercomputing Center

18. Availability • Source:http://www.openssh.org/ • Patch:http://www.psc.edu/networking/projects/hpn-ssh/ • Other:FreeBSD ports security/hpn-ssh/ Pittsburgh Supercomputing Center

19. Thanks! • Questions? • Business cards with URL available Pittsburgh Supercomputing Center