IPsec and other communication security measures by Richard Whalen

1. IPsec and other communication security measures by Richard Whalen

2. What is IPsec? • IPsec is an architecture and collection of protocols used to provide secure communication • Components of IPsec • ESP – Encapsulating Security Protection • AH – Authentication Header • IPComp – IP Compression • Security Policies and Associations • IKE/KINK– IPsec Key Exchange/Kerberized Internet Negotiation of Keys • Transport or Tunnel mode

3. Functions of IPsec • Authenticity of data • AH validates individual packets • IKE validates system • ESP may validate packets • Validity/Integrity of data • AH or ESP show data hasn’t been modified and isn’t being replayed • Data Privacy • ESP prevents third parties from observing the data

4. Why use IPsec? • Can be used to secure many applications with a single management policy and no modifications to applications • Confidentiality • Data integrity and source authentication • Can protect all communication below the IP level. • Setup and maintain one security scheme between systems instead of many • Invisible to users

5. Components of IPsec. • ESP - Encrypts all data inside the IP packet so that no intermediate system that does not know the algorithm and key can know the contents of the packet • AH - Verifies the authenticity of the immutable portions of the IP header and the data in the packet • IPComp - Compression must be done before encryption as encryption can hinder effective data compression

6. ESP in a packet • ESP protects the portion of the packet after the ESP header from observation or modification from the source to the destination.

7. AH in a packet • AH protects entire packet (except checksum & TTL) from alteration from the source to the destination and proves that the source generated the packet

8. IPsec algorithms • Encryption • 3DES-CBC • AES • Null • Hashed Message Authentication Codes (HMAC) • SHA-1, SHA-2 • MD5 • None

9. IPsec implementation choices • Integrated architecture • Capabilities provided as easily as regular IP • Greatest flexibility • Additional time spent at high IPL/with spinlock held • “Bump in the stack” • A retro-fit option for when it’s difficult to integrate • Potential performance problems with sending the data to process mode and back • “Bump in the wire” • External device provides VPN capability

10. Diagram of IPsec

11. Security Policies and Associations • Security Policies define what types of communication IPsec applies to • IP addresses, protocols, ports, encryption, authenticity, compression • Security Associations define how the Security Policies are implemented • Algorithms and keys • Must be maintained in a secure manner in order to guarantee security

12. Functions of IKE • Identify and authenticate remote system • Identification by IP address or FQDN • Authentication by pre-shared key or certificate chain • Negotiate algorithms and keys, and install security associations in a secure manner. • IKEv2 corrects a number of security problems with IKE • The current specification for configuration language requires that SPs and methods to determine SAs are processed by a single program.

13. IKE vs KINK • IKE • X509 certificate for authentication • Diffie-Hellman key exchange • Know & implement a security policy for every peer it will operate with • KINK • Mutually authenticate with the Authentication Server of the Key Distribution Center (KDC) in Kerberos • KDC controls distribution of keying material for encryption

14. Transport and Tunnel Modes • Transport is host to host – only the payload is encrypted/authenticated. (IP headers are in the clear). • Tunnel is network to network – entire packet is encrypted/authenticated then encapsulated in another IP packet. Used to create a VPN.

15. IPsec packet procssing steps (xmit) • IP packet is ready to transmit • SP is found (may be done at higher level such as TCP/UDP) • Attempt to find SA if SP says “use” or “require” • If “require” and no SA drop packet • Process packet with SA • Transmit packet with IPsec information.

16. IPsec processing steps (receive) • IP packet is received with IPsec header(s) present • Find SA by index in the header; drop packet if not found and notify IKE • Process packet by header type with SA • Hand packet off to next header processing

17. IKE processing • Initial contact ? • Send information proving identity of system to remote IKE agent • Negotiate key management program encryption algorithm and key • Receive request for a new SA • Negotiate algorithm, key and SA id • Install new SA

18. Public Key Encryption • Source encrypts data with its private key and destination public key • Destination decrypts with its private key and source public key • Computationally expensive • The longer an encryption scheme is used the less secure it becomes.

19. How certificates establish identity • Secure communication does not prove identity. • Certificates are signed with the private key of an implicitly trusted third party. • Certificates include information about how long they are valid.

20. How IPsec might work with IP clusters • Through an IPsec tunnel • In transport mode: • When a member is not present in the cluster it has a security policy of USE AH • When the member establishes IKE communication it adds a security policy of REQUIRE ESP • When IKE communication is lost all other members revert to USE AH for the member that left the cluster

21. How DNSSEC works • Resolver works with local caching name server over a secure communication path • Name server uses additional information in the packet to verify that the answer comes from a trusted source and has not been altered.

22. How SSH works • Destination returns its public key • Source verifies public key, and encrypts some data using public key encryption. • Destination decrypts and re-encrypts and sends data back to verify that it is who it says it is. • User authentication.

23. How TLS (FTPS/HTTPS) works • Signed Certificate is presented • If Certificate is signed by a known authority then this “proves” that the system is the desired system. • Public key encryption to establish session key and algorithm.

24. Choosing a Security Mechanism • IPsec • A well defined set of hosts that require secure communication over a variety of ports. • TLS (SSL – HTTPS/FTPS) • A potentially large number of hosts that you want secure communication for specific communication. • SSH (SFTP) • Telnet and file transfer replacement. • DNSSEC • Verification that DNS answers have not been tampered with.

25. Why an Intrusion Prevention System? • Firewalls don’t react at the application level. IPS allows applications to determine when an attack is occurring, based on factors unique to the application. • Allows applications (and in the future, the TCPware or MultiNet kernels as well) to react to and curtail attacks, such as dictionary attacks on SSH or FTP, or other DOS-type attacks. These attacks can consume large amounts of system resources as well as network bandwidth. • Unusual traffic patterns could also be a factor.

26. What is VAM? • VMS Authentication Module • Provides an interface to standard and non-standard VMS authentication methods: • LDAP (including Microsoft Active Directory) • RSA Security SecurID • RADIUS • Local UAF • Two ways to interface: • Via an API incorporated into a user application. • Via VMS LGI$ callout hooks into the standard VMS login mechanism (SecurID and LDAP only). • Supported on OpenVMS VAX V7.3, OpenVMS AXP V6.2 and higher, and OpenVMS I64 V8.2 and higher, on MultiNet, TCPware and HP TCP/IP Services.

27. Thank You *********** Contact Us www.process.com info@process.com Phone: (800) 722-7770 or 1(508) 879-6994