410 likes | 604 Views
Performance SeNTry: Development Status Report. Demand Technology Software A DataCore Software company 1020 Eighth Avenue South, Suite 6, Naples, FL 34102 phone: (941) 261-8945 fax: (941) 261-5456 e-mail:support@demandtech.com http://www.demandtech.com. Agenda.
E N D
Performance SeNTry: Development Status Report Demand Technology Software A DataCore Software company 1020 Eighth Avenue South, Suite 6, Naples, FL 34102 phone: (941) 261-8945 fax: (941) 261-5456 e-mail:support@demandtech.com http://www.demandtech.com
Agenda • Performance SeNTry version 2.4.2 highlights • NTSMF Collector • SeNTry Administration • Version 3 roadmap • Alerts and Traps • File System Collector • Improved Systems Console for real-time notification, management, and control
Performance SeNTry version 2.4.2Topics • Windows 2000 ‘features’ • NTSMF features • SeNTry Administration features • Event log use • New Enhancements – SQL Server 2000 • New Enhancements - Exchange 2000 support • New collection service architecture – • Threaded collector • Improved Exception handling
Performance SeNTry version 2.4.2Windows 2000 ‘features’ • System level objects now in visible performance libraries • System object ‘missing’ some NT4 counters: %Total Processor Time, % Total User Time, % Total Privileged Time, % Total DPC Time, % Total Interrupt Time, Total Interrupts/sec • Delay in some objects reporting data at system start going throughRegQueryValueEx() • New values in Performance Registry keys • Timeouts • DisablePerformanceCounters
Performance SeNTry version 2.4.2New fields in Registry: HKLM\…\Services\PerfDisk\Performance
Performance SeNTry version 2.4.2 Additional NTSMF features • Install option to run under a user account -install -account <account name> -password <password> • Program defaults with filters active - NT4 and Win2K have specific sets • Memory size field in header fixed for Win 2000 systems with 4GB or more of physical memory • No fix for Windows NT4 systems with 4GB or more of physical memory since the GlobalMemoryStatus api call returns only a 32 bit number
SeNTry Administration enhancements • New default DCS for Win2K and Performance Gallery Gold templates • SQL Server 2000 object support • MS Exchange 2000 objects support • Tool to compact the Performance SeNTry Database ('Tools' menu item in the 'Network Browser' window) • More filters • More options
Process New Process Filter on Process Working Set Size Always write named processes – availability reporting Threads Add 'Ready Threads' counter in the 'Process' object No longer need to write Thread objects into .smf Terminal Server suppress Idle User record write User and Session records (automatic) only when activity occurs SeNTry Administration enhancementsCollection Filters - Process
SeNTry Administration enhancementsParameters Administration, Collection Cycle • On Error Command capability for launching commands when errors are encountered • Same rules as Cycle End commands • ‘poor man’s’ alert – something as simple as ‘Net Send user-name message’
SeNTry Administration enhancementsParameters Administration, Scheduling • Flexible DCS chaining specifying Next DCS in chain (e.g., prime shift.DCS chains to nighttimeDCS, which chains back) • Collection Period to specify the period that the collector is active and reporting data • Provide basic scheduling options without requiring a 3rd party package
Performance SeNTry version 2.4.2Parameters Administration, File Contents • Support for summarization program • Collect Inventory data from Registry keys
Performance SeNTry version 2.4.2Parameters Administration, File Contents • New option – “Write denominator counters” • Used by .smf Summarization (DMSumSMF.exe) uses in calculations to compute correct average values of compound Counters • e.g., Copy Read Hits % • One more option to manage the volume of data that can be collected effectively • “Base” appended to denominator counter name strings for identification in .smf file
Performance SeNTry version 2.4.2Parameters Administration, File Contents • Summarization utility (based on Counter Type) • Requires Type 6 records! • Designed to run at Cycle End, but can be run standalone • Creates sum.smf files • Identical in format to detail records • aged like any other .smf file in Current/Previous/Archive folders • Calculates the arithmetic mean for most Counters • Calculates interval peak (min, max) where appropriate • Some counters simply save the last observed value (e.g. elapsed time)
SeNTry Administration enhancementsParameters Administration, File Contents • ‘Include Registry Key Values’ to get registry key value information into configuration records in the SMF data file. • Used to collect hardware and software inventory, versioning, etc. • No restrictions: inventory as many values as you need • Must specify the full Registry key name and field name • Use in conjunction with regedit Copy Key Name function • Key Not Found earns a Warning message, but no other side effect • Invalid spec earns a Warning message, but no other side effect
SeNTry Administration enhancementsParameters Administration, File Management • PKZIP compression option at Cycle End • Another way to manage data volume
Performance SeNTry version 2.4.2Event Log messages • Informational, Warning and Error messages are placed in the Application Event log • Also in <system>.ntsmf.log (plain text) • Date, time User, System, source filled in by event facility • Type, category, event id, description defined by developers • I and W will not
Performance SeNTry version 2.4.2Event Log messages - Warning
Performance SeNTry version 2.4.2Event Log messages – ntsmf.log example 10/24/01-00:00:01 - Event ID: 210, Category: Discovery, Severity: Warning The Performance Library, "C:\Program Files\Exchsrvr\bin\exprfdll.dll", for service "ExIPC" (ExIPC), failed to load. The system error was "The specified module could not be found (126)". Performance data for this service will not be available. The Event Viewer (source Perflib) may have more details
Performance SeNTry version 2.4.2Event Log messages – unhandled exception 10/13/01-00:00:02 - Event ID: 96, Category: N/A, Severity: Error Unhandled exception encountered 10/13/01-00:00:02 - Ended Performance SeNTry service 10/13/01-00:00:02 - Event ID: 104, Category: Initialization, Severity: Info The Performance SeNTry service stopped with error code, -1073741728. Version=2.3.6
Performance SeNTry version 2.4.2New Enhancements – SQL Server 2000 • Lots of new SQL Server 2000 records • A single machine can be run multiple instances of SQL Server • No instance names for separate SQL Server processes, as delivered by MS • MSSQL$db-name:object-name • e.g. MSSQL$SUP:Buffer Manager,… MSSQL$SUP1:Buffer Manager,…) • Anticipate object selection and reporting problems • We write a MSSQL:Buffer Manager Object instance where the db-name token becomes an Instance name 2.2.2,0,5,2001,4,17,20,36,45,859,0,MSSQL:Buffer Manager,1,18,22,1,SQL Server DB,Buffer cache hit ratio,…
Performance SeNTry version 2.4.2New Enhancements - Exchange 2000 support • New objects visible when optional components installed (e.g. Instant Messaging) • Cases where a perflib loads another .dll • exprfdll.dll loads exchmem.dll • Objects have many counters: • MSExchange Web Mail … 75 counters • MSExchangeIMAP4 … 96 counters • MSExchangeIS ……… 107 counters (actual) • Database …………….. 127 Counters (many of ‘no data’ type)
Performance SeNTry version 2.4.2New Architecture - Threaded collector • Separate thread for each performance library found in the Registry • 3rd party Perflib DLLs perform the actual performance data gathering • Windows 2000, XP expose the ‘system’ perflibs, including perfos.dll, perfdisk.dll, perfproc.dll, perfnet.dll • Improved error handling, especially in 3rd party code • Additional collection service threads manage dispatching, file writes, other tasks
NT Performance monitoring API • Perflib Dlls are loaded into the dmperfss process address space when opened • 3rd party Perflib Dlls might need to synchronize with a separate monitored address space
NT Performance monitoring API • Open and Collect Timeout values exceeded trigger DisablePerformanceCounters setting, which must be disabled manually! • HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Perflib ExtCounterTestLevel can be used to verify integrity of 3rd party Perflib dlls.
Performance SeNTry version 2.4.2New Architecture - Exception handling • Try to catch exceptions that occur in a thread, i.e., inside code in a Perflib DLL • The failing thread and perflib (and the performance Objects it supplies) are deactivated, while other Objects continue to be collected • Problem perflib is automatically retried every new cycle or on demand (Notify Command) • No manual intervention required • Unless the Object is on the critical list: • System, Memory, Cache, Process, Thread, Processor • Critical missing means log and stop
Performance SeNTry version 2.4.2New Architecture - Exception handling • On exception: • system error text message • the loaded modules with addresses • and the call stack • Helps to identify ‘rogue Perflib dlls’ • Very compact and accurate
Performance SeNTry version 2.4.2Log message and module list 10/26/01-10:20:05 - Event ID: 1300, Category: PerfData, Severity: WarningCould not collect performance data for object, "Database ==> Instances (L28698, G3370), counter, Pages Converted/sec (L28700, G3372)".The system error was "Win32 exception, 0xC0000005, encountered at location 0x0043FBB1Read attempt at location 0x00000020Module List:(0x00400000-0x00480000) D:\NTSMFv24224\DMPerfss.exe(0x01000000-0x01017000) F:\WINNT\System32\BEPerfDll.dll(0x01140000-0x0114E000) D:\Program Files\Exchsrvr\bin\davexpc.dll(0x01630000-0x0163B000) D:\Program Files\Exchsrvr\bin\eseperf.dll(0x02000000-0x02010000) F:\WINNT\system32\dssdata.dll(0x02C40000-0x02C46000) D:\Program Files\Exchsrvr\bin\srsperf.dll …. (lots more of these)
Performance SeNTry version 2.4.2Stack Trace Stack Trace (ThreadId 4540): DMPerfss.exe: PSSPerfLocalCounter::RealizeCache + 111 bytes E:\DmPerfss 2.4.2\Collector\PSSPerfLocalCounter.cpp, Line 372 + 15 bytes DMPerfss.exe: PSSPerfObjectCache::RealizeCache + 821 bytes E:\DmPerfss 2.4.2\Collector\PSSPerfObjectCache.cpp, Line 247 + 11 bytes DMPerfss.exe: PSSPerfObjectCache::OnCollectionComplete + 12 bytes E:\DmPerfss 2.4.2\Collector\PSSPerfObjectCache.cpp, Line 408 + 9 bytes DMPerfss.exe: PSSPerfDataSink::Collect + 597 bytes E:\DmPerfss 2.4.2\Collector\PSSPerfDataSink.cpp, Line 395 + 14 bytes DMPerfss.exe: PSSPerfDataSink::Collect + 15 bytes E:\DmPerfss 2.4.2\Collector\PSSPerfDataSink.cpp, Line 463 + 15 bytes DMPerfss.exe: PSSPerfObject::_Collect + 75 bytes E:\DmPerfss 2.4.2\Collector\PSSPerfObject.cpp, Line 1688 + 26 bytes DMPerfss.exe: PSSPerfObject::Collect + 79 bytes E:\DmPerfss 2.4.2\Collector\PSSPerfObject.cpp, Line 1748 + 20 bytes
Performance SeNTry version 2.4.2Stack Trace DMPerfss.exe: PSSCollectionCycle::Start + 1483 bytes E:\DmPerfss 2.4.2\Collector\PSSCollectionCycle.cpp, Line 1500 + 45 bytes DMPerfss.exe: PSSPerformanceSeNTry::StartCollection + 248 bytes E:\DmPerfss 2.4.2\Collector\PSSPerformanceSeNTry.cpp, Line 1571 + 11 bytes DMPerfss.exe: PSSPerformanceSeNTry::Run + 109 bytes E:\DmPerfss 2.4.2\Collector\PSSPerformanceSeNTry.cpp, Line 1254 + 22 bytes"
Performance SeNTry version 2.4.2New Architecture - Exception handling • New dmperfss.exe ships with debug symbols included for extended error diagnostics report • Size of the image file is now approximately 5.8 MB • The larger image file installs by default • A dmperfss.exe without symbols is also on the CD • E.g., a floppy disk distribution/install • dmperfss.exe with symbols is preferred, especially if new applications or new versions are discovered
Performance SeNTry Version 3 Roadmap • Real-time Alerts • Event log & SNMP support • File System Collector • Revamped Systems Console
Performance SeNTry Version 3Alerts • Alert is written into the Application event log • Triggered by system condition (e.g. cpu busy threshold) • Frequency of alerts is also thresholded; won’t fill up the Event Log unnecessarily • Trigger, Release mechanism • Alerts are cleared when condition is absent • (based on the ‘lower threshold’)
Performance SeNTry Version 3SNMP - Traps • Like Alerts, defined by same rules • SNMP extension agent notified of the ‘alert’ by dmperfss.exe • SNMP extension agent generates a Trap message • Trap is sent to management entities by SNMP facility • Management entities would be OpenView, Tivoli, CA-Unicenter, others
Performance SeNTry Version 3File System Collector • At prototype stage now… • Probably another thread in the collector • Designed to run infrequently, say once/day (don’t know how long it takes yet) • Report files that are above threshold (e.g. file size > 2% Logical disk size) • Report folders that are above threshold when files in them are totaled (don’t also report parents, just siblings)
Performance SeNTry Version 3File System Collector • Summary records – size of files remaining • NTSMF header record format • Pseudo object id • Records are instances where the UNC path is the instance name • Probably optioned - allow include in .SMF file or write to a file of its own
Performance SeNTry Version 3System Console • Processes real-time alerts • Can examine objects and counters related to the alerts • Can change the alert definition, collection interval, etc., as needed for the display • Display results graphically • Capture the data and graphs for later review
Performance SeNTry Version 3Exchange 2000 references • http://www.exchange-mail.org/whitepapers.html • http://www.microsoft.com/technet/treeview/default.asp?url=/technet/prodtechnol/exchange/reskit/ex00res/resguide/c29monit.asp (Microsoft's online version of the Exchange Resource Kit Documentation) • http://www.netiq.com/Downloads/Library/White_Papers/NetIQ_Monitoring_and_Managing_Exchange.pdf
Performance SeNTryreach us at: • Demand Technology Software A DataCore Software company • Address: 1020 Eighth Avenue South, Suite 6, Naples, FL 34102 • Phone: (941) 261-8945 fax: (941) 261-5456 • E-mail: support@demandtech.com • Web site: http://www.demandtech.com
Performance SeNTry Version 3 Roadmap Questions?