1 / 65

Windows File Systems and Improving Performance

Windows File Systems and Improving Performance. APCUG 1/5/2008 Greg Hayes Raxco Software ghayes@raxco.com. Introduction. Company Background Raxco founded 1978 Largest provider of OpenVMS system management software Spun off Axent Technologies (now part of Symantec) in 1996

orea
Download Presentation

Windows File Systems and Improving Performance

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Windows File Systems and Improving Performance APCUG 1/5/2008 Greg Hayes Raxco Software ghayes@raxco.com

  2. Introduction • Company Background • Raxco founded 1978 • Largest provider of OpenVMS system management software • Spun off Axent Technologies (now part of Symantec) in 1996 • First Windows product released in 1998

  3. Introduction • Company Background • Microsoft Gold Certified Partner • Microsoft ISV • Member of Microsoft Development Network • Software used by some of the largest companies and government organizations in the world

  4. Introduction • Speaker background • System manager for 10+ years • Raxco customer for 10 years • Started at Raxco in 1996 • Manager, Technical Solutions • Microsoft MVP 2003-2007 - Windows File Systems • Online communities • Speaking/Education

  5. Topic Background • Who remembers their first PC? • PC hardware performance continues to rise • CPU clock speeds > 2Ghz • 2GB RAM • 500GB+ capacity hard drives • Limiting factor is still disk I/O • Anything that speeds up access to disk improves performance

  6. Windows File Systems • FAT16 • FAT32 • exFAT (new) • NTFS

  7. Windows File Systems • FAT16/32 The File Allocation Table is created when you format a FAT drive and contains information about all of the files found on the drive and where they logically reside. There are actually 2 FATs per partition.

  8. Windows File Systems – FAT16 • Benefits: • Faster performance on smaller partitions - less than 4GB • Best Performance for Pagefile • Drawbacks: • Little or no security • High rate of file system corruption • Limits on partition size (FAT16 – 4GB)

  9. Windows File Systems – FAT16 • FAT16 • 4GB FAT16 partition has a cluster size of 64k. • Wastes space as even the smallest files take up a full cluster. • Maximum file size is 2GB • Files per volume 65,536

  10. Windows File Systems – FAT32 • FAT32 • Maximum file size is 4GB • Allows smaller cluster sizes to be used – resulting in less space “waste” • Maximum partition size is 32GB (as formatted by XP/Vista/2K3) • Up to 2TB can be written/read (XP/Vista/2K3) • Files per volume ~4,177,920

  11. Windows File Systems – exFAT • exFAT • “Extended FAT” • Designed for flash media – exchange media with PCs. • Free space bitmap for faster allocation • Maximum theoretical file size is 16EB • More than 1000 files per folder

  12. Windows File Systems - NTFS • Journaling/Transaction Based File System • “Self Describing” File System • Metadata - data that describes data

  13. Windows File Systems - NTFS • Benefits: • Better security • Better performance with large partitions – greater than 4GB • Better resistance to file system corruption • Drawbacks: • Can’t natively boot to MSDOS and get access to partition

  14. Windows File Systems - NTFS • NTFS • Maximum file size is approx 16TB • Maximum partition size is 2TB – 256TB (depends on cluster size used) • Files per volume 4,294,967,295

  15. Windows File Systems - NTFS • NTFS • Disk Quotas (Win2K and newer) • File/Folder compression built into the file system • Encryption built into the file system (Win2K and newer) • Volume Shadow Copy (VSS – WinXP and newer) • txNTFS

  16. NTFS Metadata • $MFT - Master File Table (0) • The $MFT contains at least 1 record for each file that exists on the partition. • $MFTMirr – Master File Table Mirror (1) • Duplicate of the 1st 4 records of the $MFT • Used for redundancy/recoverability

  17. NTFS Metadata • $LogFil – NTFS Transaction Log (2) • Updates to the file system first get posted to the transaction log • Gives NTFS its “self-healing” abilities • $Volume – Volume Information (3) • Volume Label/Version

  18. NTFS Metadata • $AttrDef – Attribute Definitions (4) • Names, Numbers, Descriptions • $Root – (5) • Root folder on the drive. (I.e. C:\)

  19. NTFS Metadata • $Bitmap – Cluster Bitmap (6) • Indicates whether a cluster is used or empty • Uses this to quickly find free space • $Boot – Partition Boot Sector (7) • Location of the partition boot sector • Bootstrap loader code if bootable drive

  20. NTFS Metadata • $BadClus – Bad Cluster File (8) • Clusters the file system has reported as bad • Updated by CHKDSK • $Secure – Security Descriptors (9) • $Upcase – Upcase Table (10) • Translation of lowercase characters to their equivalent Unicode upper case characters

  21. NTFS Metadata • $Extend – Optional Extensions (11) • User Change Journal • Quotas

  22. Anatomy of a $MFT Record • Fixed size records – 1K • Attribute Records • $FILE_NAME • 8.3 file name and parent directory • Long File Name • $DATA • Extent information for file. Includes Logical Cluster Number (LCN) and Run Length • $DATA record contains information on file size

  23. Anatomy of a $MFT Record • DiskEdit

  24. MFT Reserved Zone • Created when partition is formatted • $MFT will “grow into” as needed • File system will keep free if possible • Defragmenters can not use free space inside of MFT Reserved Zone (NT4/Win2K)

  25. MFT Reserved Zone • NT4 • Fixed Size • 12.5% by default • Can be increased up to 50% via registry key

  26. MFT Reserved Zone • Windows 2003/2000/XP/Vista/2008 • Dynamically created every time partition is mounted • First record of $MFT to first non-free cluster - up to a default max of 12.5%

  27. Logical vs Physical Clusters • Logical Clusters • File system level • Every partition starts at logical cluster 0 • No idea of hard drive technology in use • IDE • SCSI • RAIDx • # platters or read/write heads

  28. Logical vs Physical Clusters • Physical Clusters • Hard drive level • Hard drive controller translates logical-to-physical and positions heads

  29. Cluster Size and Performance • Smaller clusters • less wasted space • Worse performance – especially large files • Larger clusters • more wasted space • Better performance – especially large files

  30. Conversion from FATx to NTFS • NT4/Win2K • Results in 512byte clusters • Not “the best” for performance - especially with video/image applications • WinXP/Vista • Will try to convert using 4k clusters. • Best for general file system performance.

  31. OEM Vendors and Cluster Size • IBM/Dell/HP • Provide Software to install/configure Windows Server • Formats system drive with 512byte cluster • Absolute worse for system drive performance • Only way to convert is 3rd party tools and have server unavailable for extended period (HP SmartStart, Dell OpenManage, IBM ServerGuide)

  32. Biggest Cause of Poor File System Performance • Fragmentation!

  33. Fragmentation Causes • What causes fragmentation? • Occurs when files are created, extended or deleted • Happens regardless of how much free space is available (After XP/SP2 installation – 944 files/2943 fragments) • More than one Logical I/O request has to be made to the hard drive controller to access a file

  34. Fragmentation Impacts • What does fragmentation do to my system? • Slows down access to files • Extra CPU/Memory/Disk resource usage • Some applications may not run • Slow system boot/shutdown • Audio/Video record/playback drops frames or “skips”

  35. Measuring Impact of Fragmentation • Measuring the performance loss in reading a fragmented file

  36. What Can I Do About Fragmentation? • You can’t stop fragmentation from happening (you can only slow it down)! • What you CAN do is to defragment

  37. Defragmenting - Results • What does defragmenting do? • Locates logical pieces of a file and brings them together • Faster to access file and takes less resources • Improves read performance • Consolidates free space into larger pieces • New files get created in 1 piece • Improves write performance

  38. Measuring Impact of Fragmentation • Measuring the performance difference in reading a contiguous file

  39. Defragmenting - Issues to Consider • Safety, Safety, Safety! • No loss of data • No corruption of data

  40. Defragmenting - Issues to Consider • Free Space • How much is enough? • Where is free space located? • Inside MFT Reserved Zone • Outside of MFT Reserved Zone • Consolidation of free space

  41. Safety - Microsoft’s Defrag APIs • Provided as part of the operating system • Defragmenters do not actually move files • Integrated with caching system and Memory Manager • Performs all I/O synchronization • Allows even files in use to be defragmented

  42. Safety - Microsoft’s Defrag APIs • Restrictions • Move Granularity • Free Space in MFT Reserved Zone • NTFS Cluster Size • Special OS files • Pagefile • Hibernate file • NTFS Metadata

  43. Microsoft’s Defrag APIs • “Quirks” • Compressed files (NT4/Win2K) • File System Cache (NT4/Win2K) • Encrypted Files (Win2K) • NTFRS/DFS (Win2K) • VSS (Vista,Win2K3)

  44. Free Space Consolidation • Allows new files to be created contiguously • Maintains file system performance longer • Requires less frequent defrag passes

  45. Free Space Consolidation • Defragmenting files improves read performance • Free space consolidation improves write performance

  46. Single Pass Defragmentation • As my dad always told me… • If you are going to do a job – do it right the first time • After defrag has completed, you don’t have to “wonder” if you need to run again • Fragmentation issue is solved – done!

  47. File Placement Strategy • When you play chess, do you have a strategy or do you just start moving pieces around? • A good file placement strategy accomplishes several things: • Slows down re-fragmentation • Speeds up future defrag passes • Works with the operating system – not against

  48. Windows Prefetch • What is Prefetching? • Windows monitors system boots and application launches • Uses information gathered to perform driver load optimization (speeds up boot process) • Uses information gathered to speed up application launches

  49. Windows Prefetch • Windows stores this information in the \Windows\Prefetch folder • Layout.ini • Xxxx.pf • Every 3 days, Windows will automatically perform a “partial” defrag of the files indicated in layout.ini

More Related