1 / 43

Trustworthy Systems Through Quantitative Software Engineering

Trustworthy Systems Through Quantitative Software Engineering. Larry Bernstein Stevens Institute of Technology Castle Point, Hoboken, NJ 07030 USA. Trustworthy Software is:. Safe: Does no harm Reliable: No crash or hang. Secure: No Hacking Possible. What is a Requirement?.

gilon
Download Presentation

Trustworthy Systems Through Quantitative Software Engineering

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Trustworthy Systems Through Quantitative Software Engineering Larry Bernstein Stevens Institute of Technology Castle Point, Hoboken, NJ 07030 USA

  2. Trustworthy Software is: • Safe: Does no harm • Reliable: No crash or hang. • Secure: No Hacking Possible

  3. What is a Requirement? • A property that must be exhibited by a system to solve some problem. • Requirements may be • Functional providing product capabilities • Non-Functional constraining the implementation

  4. System Performance Resulting from Robust Requirements vs. Discrete Specifications Discrete Specifications Ideal Volume Robust Requirements Dynamic Range

  5. Relative Project Costs 4x 2x 1.5x 1.25x Relative Cost Range x 0.8x 0.67x 0.5x Concept of Operation Requirements Specifications Product Design Specifications Detailed Design Specifications Accepted Software 0.8x Feasibility Plans and Requirements Product Design Detailed Design Development and Test Phases and Milestones

  6. Top Ten Software Risk Items

  7. Highlights of Quantitative Approach • Lambda Protocol • Overlaps with Systems Engineering • Industrial Strength Requirements for Software Intensive Systems-of-Systems

  8. QSE Lambda Protocol • Prospectus • Measurable Operational Value • Prototyping or Modeling • sQFD • Schedule, Staffing, Quality Estimates • ICED-T • Trade-off Analysis

  9. Universal Software Engineering Equation Reliability (t) = ℮-k t when the error rate is constant and where k is a normalizing constant for a software shop and • = Complexity/ effectiveness x staffing

  10. Conditions That Cause Unreliability • Poor Algorithms • Missing Deadlines • Roundoff Error Build Up • Memory Leaks • Broken Pointers • Register Misuse

  11. QSE Characteristics • Solving the right problem the right way • Certified against requirements. • Certified against problem • Bounded execution domain

  12. Case Study: SchedulerPro Prospectus User friendly, efficient interface for students to create and modify class schedules. Features: • Visual schedule creation and editing • Schedule suggestion • Schedule comparison view • Monitor closed-out sections

  13. SchedulerPro Prototype Screen

  14. SchedulerPro Prototype Screen

  15. SchedulerPro Notification Emails

  16. Measurable Operational Value SchedulerPro MOV Course scheduling and registration time reduced by an average of 20 minutes per student per semester.

  17. SchedulerPro Functional Goals Schedule Classes and Personal Time  Searching  Course Placement  Course Detail Viewing  Course Removal  Scheduling Personal Blocks  Notification (optional)  Course Suggestions (optional)

  18. Functional Requirements • Search available classes by: • Same professor • Similar time • Same or equivalent class but different sections • Register and track registrations • Color classes and arbitrary time-blocks by user choice

  19. SchedulerPro Nonfunctional Requirements • Integrate with “Web for Students’ and existing authentication systems and avoid incompatibilities • Allow schedules to be saved/accessed from a server or local file • Provide a scaled time-accurate visual representation of the schedule

  20. More Non-functional requirements • Make schedules available even if the application is down, provided an internet connection is available • Perform some functions without a live connection to the ‘Web for Students’ registrar web site • Make compatible with all popular browsers • Display section states and print schedules without loss of detail

  21. sQFD

  22. SchedulerPro Product Reliability • Two hours of unavailability allows for daily backups, service, and reboots of the system • Connections to server are minimized, reducing overall activity on the server

  23. Parnas reliability checklist Failures in communication, secondary storage, memory, or any hardware that may interrupt a transaction: • The SQL Server DBMS will not commit incomplete transactions. User will be notified of the error, and will have to redo the transaction. Operator error: • Important operations are confirmed before they are completed to avoid large accidental errors.

  24. SchedulerPro Estimate of Reliability R(t) = 1 - F(t) F(t) = P(T ≤ t) • During load testing, we discovered the test server can support 1500 user queries a minute. • P(failures/query) = 55/1500 = 0.036 • Thus, F(t) = 3.6%, which means the software is 96.4% reliable

  25. SchedulerPro Reliability Estimate 1/ λ = MTTF = εE/kC k = scaling constant = 1 C is complexity = 2.78 E is the development effort = 36.4 ε is the expansion factor = 1.5 λ = 0.05 t is the continuous execution time for the software R(t) = 95.12%

  26. Complexity Chart - Client • Project Type: online transaction • Problem Domain: 2 • Architecture Complexity: 3 • Logic Design – Data: 2 • Logic Design – Code: 3 • Total Score: 10 • Complexity = (10/18) * 5 = 2.78

  27. Complexity Chart - Server • Project Type: online transaction • Problem Domain: 1 • Architecture Complexity: 2 • Logic Design – Data: 2 • Logic Design – Code: 2 • Total Score: 7 • Complexity = (7/18) * 5 = 1.94

  28. Complexity Chart - Overall • Project Type: client/server • Problem Domain: 2 • Architecture Complexity: 3 • Logic Design – Data: 2 • Logic Design – Code: 3 • Total Score: 10 • Complexity = (10/18) * 5 = 2.78

  29. Jan. Function Point Est.

  30. April Function Points Est.

  31. History of Function Points *Using COCOMO Model

  32. Web Points Effort = A * P cdi (Size)P1 Because it is a small web project, assume: A=2.1, B=2.0, P1=1.00, and P2=.5 Language Expansion Factor = 25 From this, we estimate 2.43 KLOC Effort = A* KLOC = 2.1 * 2.43 = 5.103 staff-months Duration =B * EffortP2 = 2.0 * 5.103.5 = 4.518 calendar months

  33. ICED-T

  34. Creeping Featurism • Endemic to the Software Industry • Occurs on more than 70% of all applications of over 1000 function points • From a 60 project sample • Average creep was 35% • Maximum observed was 200% • Creeping requirements change about 1% per month • For a 3 year project, 1/3 of the delivered requirements would have been added after requirements were initially defined • Rate of Requirements change is higher than for other forms of engineering (electrical, mechanical, civil)

  35. Root Causes of Creeping Requirements • Uncertainty in resolving true user needs • For multi-year projects, changes in normal business environment • Failure to adopt methodologies that limit the risk associated with creeping requirements • Primitive fundamental technologies for exploring and modeling requirements • Failure to use technology to measure the impact of creeping requirements • Engineering trade-off analysis is impossible

  36. Requirements Management • Establish and maintain a business case to support funding • Strategic linkages to business and technology organizations –AVOID SHELFWARE • Continuous customer agreement on requirements • Requirements agreement used as a basis for estimating, planning, implementing and tracking • FORMAL COMMITMENT PROCESS

  37. SchedulerPro Installation Plan Installation 1.Third Party Software Required Scheduler Pro requires the following products to be already installed on the target machine. Please consult the documentation of each product for installation instructions specific to each. - Windows 2000, XP, or 2003 Server - Microsoft IIS, version 5.0 or higher - Microsoft .NET, version 1.1 - Microsoft SQL Server 2000 - Message Queuing Service (Windows component) - ASP.NET State Service

  38. SchedulerPro Installation Plan 2. Installing the Scheduler Pro application To install Scheduler Pro, please follow these steps: I. Setting up the web site 1. Create a virtual directory for the site in IIS 2. Copy the contents of the “site” folder to the directory on your hard drive represented by the IIS virtual directory II. Setting up the database 1. Using the SQL Server Enterprise Manager tool, attach the database located under the “sqldb” directory. 2. Create a new user account for accessing this database, and give it read/write access for the database. III. Installing the Suggest, Notified, and Data Loader Windows Services Open the directory titled “services”. Run each of these files: - InstallSuggestService.exe - InstallNotifierService.exe - InstallDataLoaderService.exe Each installer will properly install the service. The Data Loader installer will also ask you for the location and name of the course data file it will load.

  39. SchedulerPro Installation Plan 3. Configuring Your Installation Open the directory where you installed the site files (step 2A above). Open the file “web.config”, and find the properties labeled “DBLocation”, “DBUser”, “DBPassword”, and “SecretAuthenticateKey”. These are set to default values that you should modify. Make sure that the location, username, and password match those you set up in SQL Server in step 2B above. Set SecretAuthenticateKey to the random string you are using in your authentication web page. Technical Information Scheduler Pro uses HTTP and XMLTHTTP protocols: - HTTP: As a typical client/server application, an HTTP request is sent to server. Server processes the request and returns HTTP response. - XMLHTTP: client opens XMLHTTP connection to the server. Server executes query on the database and returns needed data. Refer to Development view and process view from the “4+1” architecture views for further technical details.

  40. People Software Trustworthiness depends on people: I propose that customers insist that software products identify a Software Architect and Software Project Manager in their contracts

  41. Software Architect: • Affirms that the software product solves the customer’s problem • Affirms that the software product is suitably reliable, easy-to-use, extendible, not harmful and robust. That it is trustworthy. • Affirms that the requirements are valid.

  42. Software Project Manager: • Affirms that the software was successfully tested against the requirements. • Affirms and identifies the good software engineering processes were used in the software development and integration. • Affirms that the project is within budget, on-time and performs satisfactorily.

More Related