TechTalk Denial of Service and Availability Risk

1. TechTalkDenial of Service and Availability Risk By: Piotr T. Zbiegiel

2. Introduction • We will discuss the nature of denial of service. • We’ll examine the risk factors that virtualization brings to the table in regards to denial of service. • Finally we will discuss possible attacks that a malicious user could attempt to disrupt virtualized and cloud systems.

3. What is Denial of Service? • Occurs when a resource is unavailable for its intended purpose. • The server or service crashes or stops responding • The network path between users and service is blocked or severed. • Can occur accidentally or due to malicious intent (i.e. DOS attack)

4. Examples of DoS Attacks • Attacker sends massive amounts of traffic to a service overwhelming it. (Flooding) • Attacker spoofs DNS packets from the target and gets the DNS servers to respond to the target with large amounts of data. (DNS amplification) • DoScan happen indirectly: A massive SSH brute-force attack cripples the authentication service.

5. Evolving DoS (DDoS) • Targets got bigger so the attacks had to get bigger. • The attackers needed one of two things: • A big system with lots of bandwidth. • Lots of little systems that can work together to focus an attack. • Systems infected with malware can join a botnet. • Botnets can be directed to attack a given target, this is known as a Distributed Denial of Service (DDoS).

6. We Are Legion • Sometimes users decide to join a DDoS voluntarily. • Low Orbit Ion Cannon (LOIC) is a network stress testing and DoS tool. • Project Chanology and Operation Payback were Anonymous operations that utilized LOIC to attack Church of Scientology websites and companies that opposed WikiLeaks, respectively. • Participants were asked to download and install LOIC and targets were coordinated. (Documentary appears to be available on YouTube, maybe)

7. Denial of Service Risk • Three components help determine the risk of a denial of service: • Trust • Surface Area • Resource Depth Trust Resource Depth Surface Area

8. DoS Potential in the Cloud • Successful DoS is not just whether a service is up or down, degraded service can also be a win. • The shared resources of the cloud that make it attractive can also contribute to the risk. • Customers that are sharing a resource are competing for the underlying physical resources. • VMs that take more than their fair-share can impact the performance of their neighboring VMs. • Bad guys will consider how to bring about such performance hits.

9. Abstraction and Overcommitment • The abstraction of cloud VMs can cause complacency on the part of the customer. • Cloud providers typically oversubscribe their resources in various ways to provide more economical solutions to customers. • Providers can oversubscribe all aspects of VMs: • CPU • Memory • Disk • The customer must keep this in mind and make sure they are receiving the expected performance characteristics. (Test this.) • Ask the provider to explain details about the underlying infrastructure.

10. A Word About: Penetration Testing VMs • Cloud providers may have various restrictions on penetration testing within their service. • These may include: • Complete ban on penetration testing or vulnerability scanning. • Penetration testing of your VMs allowed but not the cloud provider infrastructure. • Limitations on types of tests (e.g. no DoS testing). • Requirement to notify the provider before testing. • Penetration and vulnerability testing are an important component of a comprehensive security program so understand limitations before choosing a CSP.

11. How are they even going to know? • Scanning and attempted exploits are a constant occurrence on the Internet. • While a provider may detect blatant scans (e.g. all 65k TCP ports) , scans performed at a smaller scale may blend into the noise. • In the same way you can distribute DoS you can distribute port scanning. • Bad guys do it today using botnets and most likely cloud systems as well. • Don’t run afoul of your service agreement but it may be worth finding out what, exactly, constitutes penetration testing.

12. Oversubscription Risk: CPU • One of the most overcommitted resources in virtualized systems. • But then again the principles of time slicing CPU for multiple processes are well known. • Also keep in mind the fact that many systems in the datacenter may be idle much of the time. • It is no wonder that virtualizing servers in the datacenter leads to less hardware

13. Oversubscription Risk: Memory • Memory overcommitment is a little more dangerous than CPU. • If a VM host runs out of memory there may be serious performance degradation as the system attempts to swap memory pages to disk. Some hypervisors still employ a few strategies: • Having VMs share identical pages of memory. If one VM changes that page the hypervisor makes a copy. • Ballooning involves having a driver on the VM request memory from the guest operating system. As the system allocates memory to the balloon within the VM the memory is freed for reallocation by the hypervisor.

14. Oversubscription Risk: Disk Storage Hypervisors can use many strategies to “stretch” disk storage: • New VM disks can be “thin provisioned” • Similar VMs can utilize shared and differential disks. • Deduplication can occur on the storage backend to make sure the system isn’t storing many copies of the same file.

15. Oversubscription Risk: IOPS • IOPS = Input/Output Operations Per Second • It is a measure of disk performance. • Even though a hypervisor may combine many disks to provide storage capacity and performance, that performance must still be divided up amongst all running VMs. • Amazon AWS now offers provisioned IOPS EBS volumes so users can guarantee a minimum service level for high-speed data access.

16. Cloud DoS Attacks: Exhausting Resources! So what can bad guys do? • Spin up many VMs and direct them to read and write virtual disks. (IOPS) • Create many VMs that are thin provisioned then start filling their disks potentially exhausting physical disk. (Disk) • Pull the same thing with memory. (Memory) • Flood the cloud network with traffic from VMs. • Coordinated bursts of CPU utilization or calls for hardware interrupts to the hypervisor. Overcommitment has the riskof contention for resources in the underlying systems which impacts all VMs running on a given hypervisor.

17. Cloud DoS Attacks: Account Lockout • Account lockouts due to failed login attempts may be used to prevent brute-force attacks. • But the attacker can use this to purposely cause accounts to get locked out in the system preventing legitimate users from accessing the system. • This can be mitigated by blocking the source IP of the failed logins but this now opens the possibility of distributed brute-force attacks using a botnet. We just can’t win!

18. Cloud DoS Attacks: API Lockout Other attacks that may be possible: • Flooding the API with requests. • Flooding the API with high-cost requests. Was able to test “high-cost” requests with Eucalyptus on Magellan: • Requesting large numbers of VMs at a time was dangerous to the health of the cloud system. • There would actually be contention for boot images and other resources as massive numbers of VMs would attempt to start. • Eucalyptus did not handle failed VM boot attempts gracefully.

19. Internet-wide Stealth Scan from a Botnet • This talk was given at the Large Installation System Administration (LISA) conference in 2012 • Researchers were able to identify a botnet scan of the entire IPv4 address space. • Usually identifying botnet scans is fairly difficult with all the other background noise on the internet but the researchers present several compelling pieces of evidence that this was a massive botnet performing a highly organized scan. • The slide deck doesn’t do the talk justice as all the animated elements that show scanning activity over time are not included in the PDF. If you have any interest in this topic I highly recommend watching the video of the talk itself. https://www.usenix.org/conference/lisa12/analysis-internet-wide-stealth-scan-botnet

20. Conclusion • This lecture provided an overview of denial of service attacks and their evolution. • Virtualized systems have additional risk factors, like over subscription of resources, that increase the potential types of denial of service attacks. • We finished the lecture discussing specific attacks that malicious individuals can carry out against a cloud system. • And don’t miss that botnet scan presentation from the previous slide.