1 Graduate Statistics Student, 2 Undergraduate Computer Science Student,

1. Denotes median of distribution Attribute Value Reset point H 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 H 168 1 2 3 4 5 6 7 8 9 10 11 12 13 Timeslot A Nonparametric CUSUM Algorithm for Timeslot Sequences with Applications to Network Surveillance Qi Zhang1, Carlos J. Rendon2, Daniel R. Jeske3 Veronica Montes de Oca1, and Mazda Marvasti4 • 1Graduate Statistics Student, 2Undergraduate Computer Science Student, • 3Professor and Director of Statistical Consulting Collaboratory • 4Chief Technology Officer, Integrien Corporation Variability in the Mean and Std. Dev. of the # of Live Sessions on a Network Server Illustrative Timeslot Distributions for # of Live Sessions on a Network Server Implementation Monte Carlo Simulations for H Introduction Flow Chart AliveTM is the major software product of Integrien Corporation that monitors, visually presents and reports the health of a business information technology system. The Statistical Consulting Collaboratory at the University of California, Riverside was contacted to develop a nonparametric statistical change-point detection procedure that would be applied to most types of univariate data. Our work extended the conventional CUSUM procedure to a nonparametric timeslot stationary context and is being implemented into the next release of AliveTM. Historical Data Off-line Processing ScreenAlerts from Historical Data Run CUSUM on Historical Data Construct Timeslot Distributions Determine H for Screening For each simulated sample path, compute . H is the 100(1-g)th percentile of the EDF of these values, where g is the nominal false alarm level. Generalized CUSUM for Live Sessions Construct Screened Timeslot Distributions Determine H for New Data Data Real-time Processing Data from a real client was available. Data within each hour timeslot were assumed to be i.i.d. Empirical distributions for each timeslot are estimated from a rolling window of 12 weeks of historical data. Monitor for Alerts CUSUM on New Data Performance Evaluation CUSUM Screening of Historical Data Assume the data windows causing alarms by the CUSUM procedure are anomalous. A slope test is used to find the start and end point of the data window. Start Point When the CUSUM statistic alerts, begin a backward sequence of fitted lines using windows of v points. Predicted start point is the rightmost point of the first window for which the hypothesis is not rejected on the basis of a t-test. End Point At the time the CUSUM statistic alerts, begin a forward sequence of fitted lines using windows containing the previous v points. Predicted end point is the time at which the CUSUM is the largest value within the first window for which the hypothesis is not rejected on the basis of a t-test. Study Based on Real Client Data 12 weeks of historical data and 9 new monitoring weeks (cycles). True alarms were determined by subject matter expert. Conventional CUSUM Procedure Let Xn denote the measurement of a univariate process at the nth time point and assume that with µ and σ2 known. If Xn shifts upward or downward more than K units from the mean, we say that there is a serious change. The CUSUM statistics are expressed as where K is generally called the reference value. If or are above some predetermined threshold H, we conclude that there is a change in the mean. The threshold H is determined to control the average run length (ARL) between false alarms, and is usually obtained from Monte Carlo Simulations. Conclusion: The procedure performs well with respect to 0 false negatives per cycle indicating alarms will be adequately detected. Study Based on Simulated Data Inject an event that shifts the timeslot distributions by 100X% during the second half of the week. Report the average number of samples between the starting point of an injected event and the point at which the CUSUM signals. Average is based on 1,000 sample path simulations for each cycle. Signal an alarm here First forward-window after the alarm where the slope is no longer positive H First backward-window before the alarm where the slope is not positive t Predicted Start Time Predicted End Time Level of change that is “serious.” CUSUM with Resetting target Reset CUSUM statistics after each alarm to eliminate the effect of previous alarm. Alarm end is determined via slope test. Real example from Integrien data Nonparametric CUSUM Procedure Conclusion: If the shift is small, the average number of samples until detection will be large. If the shift is large, the average number of samples until detection will small, therefore an alarm will be signaled immediately. For non-Gaussian measurements, use the 100ath and 100(1-a)th percentile, and , for each timeslot instead of m + K and m – K. The generalized CUSUM becomes where tn = timeslot associated with the current hour  {1, 2, …168} • S+ • - S- • S+ • - S- Special Thanks To: The Staff of Integrien Corporation, Pengyue James Lin (CTO, College of Humanities, Arts and Social Sciences at UCR), Dr. Huaying Karen Xu (Associate Director of Statistical Consulting Collaboratory at UCR), Prof. Keh-Shin Lii (Dept of Statistics at UCR), Graduate Students of the Spring 2006 offering of STAT 293.