1 / 40

Lecture 14 Cache Manager

Lecture 14 Cache Manager. xlanchen@05/27/2005. Cache Manager. Key Features of the Windows 2000 Cache Manager Cache Structure Cache Size Cache Data Structures Cache Operation Cache Support Routines. Cache Manager. A set of kernel-mode functions and system threads

lanai
Download Presentation

Lecture 14 Cache Manager

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 14 Cache Manager xlanchen@05/27/2005

  2. Cache Manager • Key Features of the Windows 2000 Cache Manager • Cache Structure • Cache Size • Cache Data Structures • Cache Operation • Cache Support Routines Understanding the Inside of Windows2000

  3. Cache Manager • A set of kernel-mode functions and system threads • Provide data caching for all 2K FS drivers • Both local and network • Cooperate with the memory manager • Cache • the purpose of a cache is to keep a subset of frequently accessed data in physical memory as a way to improve I/O performance Understanding the Inside of Windows2000

  4. Key Features of 2K Cache Manager • Supports all file system types (both local and network) • Uses the memory manager to control what parts of what files are in physical memory • Caches data on a virtual block basis • Supports "hints" passed by applications at file open time (such as random versus sequential access, temporary file creation, and so on) • Supports recoverable file systems Understanding the Inside of Windows2000

  5. Single, Centralized System Cache • Some operating systems rely on each individual file system to cache data • 2K offers a centralized caching facility • caches all externally stored data • local hard disks, floppy disks, network file servers, or CD-ROMs • user data streams or file system metadata Understanding the Inside of Windows2000

  6. The memory manager • Cache held in virtual (no phys) memory • Cache manager can’t tell what’s in phys mem • Files mapped by Section Objects • Cc copies data to/from Section Object (ref I/O system) • MM pages data in and flushes it out • Each section logically n * (256-byte) views • Maintained until all handles for file x closed Understanding the Inside of Windows2000

  7. The memory manager Cache manager Virtual memory Memory manager Physical pages Understanding the Inside of Windows2000

  8. Cache coherency • Must ensure that any process accessing cached data will get the most recent version of that data • Example: • Process 1, open/read/write • Process 2, map into its address space Understanding the Inside of Windows2000

  9. Coherent caching scheme Understanding the Inside of Windows2000

  10. Virtual Block Caching • Logical blocks • Keeps track of which blocks of a disk partition are in the cache • Novell NetWare, OpenVMS, OS/2, and older UNIX systems • Virtualblock caching • Keeps track of which parts of which files are in the cache Understanding the Inside of Windows2000

  11. Virtual block caching • Benefits • Opens up the possibility of doing intelligent read-ahead • Allows the I/O system to bypass going to the file system for requests for data that is already in the cache (fast I/O) Understanding the Inside of Windows2000

  12. Stream-Based Caching • Streamcaching vs. file caching • A stream is a sequence of bytes within a file • Some file systems allow a file to contain more than one stream • NTFS • Stream caching: caching each stream independently Understanding the Inside of Windows2000

  13. Recoverable File System Support • Recoverable file systems • Can reconstruct the disk volume structure after a system failure • A log file is maintained • NTFS Understanding the Inside of Windows2000

  14. Recoverable File System Support • The cache manager and the file system must work together • The file system writes a log file record documenting the volume update it intends to make. • The file system calls the cache manager to flush the log file record to disk. • The file system writes the volume update to the cache; that is, it modifies its cached metadata. • The cache manager flushes the altered metadata to disk, updating the volume structure. (Actually, log file records are batched before being flushed to disk, as are volume modifications.) Understanding the Inside of Windows2000

  15. Cache structure • Cache manager caches data on a virtual basis • A region of system virtual address spaces is divided into views • View: 256KB slots • See System cache address space Understanding the Inside of Windows2000

  16. System cache address space Understanding the Inside of Windows2000

  17. Round-robin Understanding the Inside of Windows2000

  18. Cache Size • how Windows 2000 computes the size of the system cache ? • Cache Virtual Size ? • Cache Physical Size ? Understanding the Inside of Windows2000

  19. Cache Virtual Size • Default size : 64 MB • If physical memory is more than 16 MB : 128 MB + (64 MB - 16 MB) / 4 MB * PMEM = 896 MB • Size and Location of System Data Cache Understanding the Inside of Windows2000

  20. Cache Physical Size • The Windows 2000 Task Manager doesn't report the size of the system cache Understanding the Inside of Windows2000

  21. Some Confused concepts • system cache • system cache working set • system working set • Some System Variables • Memory: System Cache Resident Bytes • MmSystemCachePage • Memory: Cache Bytes • MmSystemCacheWs.WorkingSetSize • Memory: Cache Bytes Peak • MmSystemCacheWs.Peak • Memory: Cache Faults/Sec • MmSystemCacheWs.PageFaultCount A part Understanding the Inside of Windows2000

  22. Cache data structure • to keep track of cached files • A virtual address control block • For each 256-KB slot in the system cache • A private cache map • For each separately opened cached file • A single shared cache map structure • For mapped views of the cached file Understanding the Inside of Windows2000

  23. Systemwide Cache Data Structures • system cache and virtual address control blocks (VACBs) VACB structure 256KB Understanding the Inside of Windows2000

  24. Per-File Cache Data Structures File Handle Understanding the Inside of Windows2000

  25. If read file VACB index arrays • Is the file in the cache? • If so, which VACB, if any, refers to the requested location? Understanding the Inside of Windows2000

  26. Cache Operation • Write-Back Caching and Lazy Writing • Intelligent Read-Ahead Understanding the Inside of Windows2000

  27. Write-Back Caching and Lazy Writing • Data written to files is first stored in memory in cache pages and then written to disk later • Flush timing • Cache manager Memory manager • Demand for physical memory exceeds supply • System thread—the lazy writer • Created once per second • 1/8 of the dirty pages in the system cache or more to be written to disk Understanding the Inside of Windows2000

  28. Dirty Page Threshold • The number of pages that the system cache keeps in memory before waking up the lazy writer • Calculating the Dirty Page Threshold System Memory Size Dirty Page ThresholdSmall Physical pages/8Medium Physical pages/4Large Sum of the above two Understanding the Inside of Windows2000

  29. Disabling Lazy Writing for a File • Win32 CreateFile function • Specifies FILE_ATTRIBUTE_TEMPORARY to create a temporary file • The lazy writer won't write dirty pages to the disk unless there is a severe shortage of physical memory or the file is closed • To improve system performance Understanding the Inside of Windows2000

  30. Write Through Caching • Win32 CreateFile function • Specifies FILE_FLAG_WRITE_THROUGH flag • Or, Win32 FlushFileBuffers function Understanding the Inside of Windows2000

  31. Flushing Mapped Files Understanding the Inside of Windows2000

  32. Understanding the Inside of Windows2000

  33. Understanding the Inside of Windows2000

  34. Understanding the Inside of Windows2000

  35. Understanding the Inside of Windows2000

  36. Cache support routines Understanding the Inside of Windows2000

  37. Kernel-Mode Functions for Copying to and from the Cache CcCopyRead CcFastCopyRead CcCopyWrite CcFastCopyWrite Copying to and from the Cache Understanding the Inside of Windows2000

  38. Functions for Finding Metadata Locations CcMapData CcPinRead CcPreparePinWrite CcPinMappedData CcSetDirtyPinnedData CcUnpinData Mapping and Pinning Understanding the Inside of Windows2000

  39. Functions That Create the DMA Interface CcMdlRead CcMdlReadComplete CcMdlWrite CcMdlWriteComplete DMA MDL: Memory descriptor list Understanding the Inside of Windows2000

  40. Write Throttling • Write throttling: cache manager’s lazy writer • Upgrade system performance for large write operation • Useful for network redirectors transmitting data over slow communication line • Example: writes a large amount of data to a remote file system over a 9600-baud line Understanding the Inside of Windows2000

More Related