Memory Resource Management in VMware ESX Server Carl A. Waldspurger

1. Memory Resource Management in VMware ESX ServerCarl A. Waldspurger woodsmen@daum.net 윤대석

2. Introduction • Virtualization • 하나의 물리적인 hardware box를 가상적인 몇 개의 system으로 logical partitioning을 제공 • Why? • Hardware가 system에서 수행되고 있는 App.에 비해 성능이 뛰어남 • System resources의 비효율적 사용

3. VMware ESX server • Virtual Machine • Physical machines’illusion • Virtual Machine Monitor • 가상하드웨어 생성, 물리적 하드웨어와 맵핑 및 자원 스케쥴링 • 하드웨어 자원에 대한 논리적인 서버 파티셔닝 제공 • 가상머신의 독립적인 자원 이용성 제공

4. Memory Virtualization Virtual Machine Kernel code & data.. Real Page Table stack MMU data PFN code Virtual Physical Memory Physical Memory • The Guest OS needs to install its own page tables • the MMU wouldn't be able to translate virtual addresses for the VM

5. Memory Virtualization Virtual Machine Real Page Table MMU Shadow PageTable (RDONLY) Kernel code & data.. stack PFN PFN data Trap VMM code Virtual Machine Monitor Virtual Physical Memory Physical Memory Shadow Page Table

6. Reclamation Mechanism • Over-commitment of memory • ∑ ( the physical memory size allocated for each VM ) > the total amount of actual memory • Page Replacement Issues • Need to reclaim pages from one or more VMs • Move some VM physical pages to a swap area

7. Reclaiming memory Virtual machine 1 Virtual machine 2 Virtual memory Virtual memory Phy. mem Phy. mem Total : 100M Total : 100M Swap out( 40M ) Machine Memory 60M 40M  60M • Problem • VMM has no idea what those pages are being used for • Can lead to double paging Meta level Page replacement policy

8. Reclamation Mechanism Ballooning

9. Sharing memory LINUX LINUX Virtual memory Virtual memory Phy. mem Phy. mem Machine Memory Match!!! PFN Scan memory • If, • Same guest OS, same Application, same data • Scan physical memory and look for identical pages • Keep only one copy of the identical page • Use COW when it’s modify

10. Page Sharing Content-Based Page Sharing

11. Performance

12. Shares • Global policy • Adjust memory allocations to improve some aggregate, system-wide performance metric • Local policy • Provide Quality-of-service guarantees • Critical for server consolidation • Different resource allocation based on • Importance • Ownership • Administration domains • The amount of money paid to a server provider

13. Shares • Share-based allocation • Shares : relative resource right • Dynamic min-funding revocation algorithm • For proportional-share allocation of space-shared resources • Shares-per-page = price • Revocation reallocates memory away from clients paying alower price to those willing to pay a higher price. • Problems • Do not incorporate any information about working sets • Idle clients with many shares can hoard memory unproductively.

14. Reclaiming idle memory • Idle memory tax • Charge a client more for an idle page • Tax rate (τ): the maximum fraction of idle pages that may bereclaimed from a client (0 ≤ τ < 1) • τ = 0: pure share-based isolation • τ ≈ 1: all of a client’s idle memory are reclaimed • ESX server default tax rate: 75% • Adjusted shares-per-page ratio ρ • f: active page ratio • idle page cost k = 1/(1- τ)

15. Measuring idle memory • Statistical sampling • A small number n of VM’s physical pages are selected randomly • For each sampled page, invalidate any cached mappings associated with its PPN. • The next guest access to a sampled page reestablish thesemappings, increasing a touched page count t. • At the end of sampling period, f is estimated as f = t/n • Estimates are smoothed across multiple sampling periods.