350 likes | 486 Views
CSI518 – Group 1. Evaluating Web Software Reliability. By Zumrut Akcam, Kim Gero, Allen Chestoski, Javian Li & Rohan Warkad . Agile Development. What is Agile ? Focus on business value Collaboration within team; meetings Communication is key Sprints, prototypes, feedback
E N D
CSI518 – Group 1 Evaluating Web Software Reliability By Zumrut Akcam, Kim Gero, Allen Chestoski, Javian Li & Rohan Warkad
What is Agile? • Focus on business value • Collaboration within team; meetings • Communication is key • Sprints, prototypes, feedback • Changing requirements The Agile Development Process
How does Agile work? • How did our class use Agile? • Three Sprints • “Stand up” meetings at beginning of each class • Retrospective at the end of each sprint The Agile Development Process
What is reliability for Web applications? • The reliability for Web applications can be defined as the probability of failure-free Web operation completions.[2] • Failure is “the event of a system deviating from its specified behavior like obtaining or delivering information”.[1] Definition of Reliability
Failures are caused from the following sources: • Host, network or browser failures: computer systems, network or software failures, etc. • Source content failures: missing, inaccessible files, JavaScript errors, etc. • User errors: improper usage, mistyped URLs.[2] Failure Sources
Attempt to extend previous work on testing the reliability of websites. • Gain experience doing a research project Project Goal
Read relevant research papers • Identify factors that may effect reliability analysis • Gather status logs Sprint 1 Goals
Web Workload Characteristics • Byte Count • User Count • Session Count • Error Count Factors That May EffectReliability Analysis
Reliability analysis via status logs • Variety of reliability requirements • Commercial and non-commercial • We will try to record the technologies the websites employ (Apache, DNN, ISS, PHP, ColdFusion, etc..) System to Analyze Reliability On
Collect log files for calculation • Automate processes to extra data (user, session, byte, and error counts) • Convert them into excel format • Log Parser Sprint 2 Goals
DNN Logs (10 Websites) • PHP Logs Sprint 2 Progress
.NET version of Drupal • An open source platform for building websites and web applications based on Microsoft .Net technology. • Leading open source ASP.NET web content management • Has been downloaded over 6 million times • ~100 employees • 5th Version • Founded 2006 What is DotNetNuke (DNN)
Logs from 10 Websites • Window Server (Same Server) • SQL Server 2008 • ~1000 unique visitors per day • Logs contain • User count • Limited Error count Our DNN Logs
The DNN Logs do not contain • Session count • Byte count Major Problem
Generate our own DNN logs Alternative
Technologies Used • Windows XP Professional • Microsoft Internet Information Services (IIS) • Microsoft SQL Server 2008 • DotNetNuke (DNN) Server Side
Clients • Web-Crawlers • DotNetNuke Client API • Inducing Errors • Logs contain: • Client IP’s • Byte Counts (Uploaded & Downloaded) • Time-Taken • Status Code Generating Logs
Server log data consisted of 23 consecutive days of data. • Page Not Found (Error 404) is the most common type of error in our logs, with 46% of total recorded errors. • Accessing forbidden data (Error 403) follows with 41%. • 72 unique IPs, 33 thousand hits total, and each hit associated with about 5kb. Workload Measurement Facts
S U C C E S S E R R O R Error and Success Ratios *HTTP Status Codes defined By W3C-World Wide Web Consortium.
The Nelson Model • R: Reliability • f : Total number of failures • n: Number of workload units • r: Failure rate • Mean Time Between Failures (MTBF) • MTBF = n/f Identify a metric to analyze reliability
By Nelson Model, the site software reliability is R = 0.966, or that 96.6% of access to website is successful. • This model also shows that MTBF=29.6 hits or the site averages one error for every 29.6 hits. • From the number of errors chart, we can see that Server errors are very few among the other errors which shows what the reliability of the DNN server is. Conclusions
[1] J.Tian, S.Rudraraju, Z.Li, “Evaluating Web Software Reliability Based on Workload and Failure Data Extracted from Server Logs”,2004. Conclusions
[1] Jeff Tian, SunitaRudraraju, and Zhao Li. 2004. Evaluating Web Software Reliability Based on Workload and Failure Data Extracted from Server Logs. IEEE Trans. Softw. Eng. 30, 11 (November 2004), 754-769. DOI=10.1109/TSE.2004.87 http://dx.doi.org/10.1109/TSE.2004.87 • [2] Toan Huynh and James Miller. 2009. Another viewpoint on "evaluating web software reliability based on workload and failure data extracted from server logs". Empirical Softw. Engg. 14, 4 (August 2009), 371-396. DOI=10.1007/s10664-008-9084-6 http://dx.doi.org/10.1007/s10664-008-9084-6 • [3] G. Albeanu, A. Averian, I. Duda, “Web Software Reliability Engineering”,2009. Resources