1 / 13

20 th Annual Test & Evaluation Conference & Exhibition

Learn how early involvement and spiral development can resolve issues and improve testing outcomes. Gain insight on requirements management, database collection, scripting, and test result reporting.

heise
Download Presentation

20 th Annual Test & Evaluation Conference & Exhibition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Joint Fires Systems Engineering – NSWC Dahlgren, VA 20th Annual Test & Evaluation Conference & Exhibition “First Steps Have The Biggest Impacts” 4 March 04 Anthony J. Devino

  2. The Focus “In Acquisition, one can make a decision and not realize the real impact of that decision until two years later.” (Bo Heikes, 1999) “Plans are nothing, Planning is everything.” (General Dwight D. Eisenhower)

  3. Briefing Goals • To explain why the Test Community’s early involvement resolves issues that arise when ready to test. • To show how Spiral Development allows earlier testing to gain insight as to product development. • To Prepare the information OT&E needs in a clear manner that will allow them to effectively audit your DT results. • Requirements Management. • Database collection of DT data and analysis. • To learn the value of Scripting the tests events to ensure you are satisfying the test objectives.

  4. Early Involvement • Program Managers (PM) should seek OT&E membership on Initial Capability Document (ICD) development: • Ensure Requirements are testable. • (e.g. - Define “User Friendly”) • Get ICD clarification issues resolved early with MDA • Ensure Requirements are Clear and Concise. • (e.g. - Must operate with all Fleet Radios – Did you really intend the System to operate with 12 different radios?) • Ensure Scope is properly identified early on in the program. • OT&E member can guide the PM as to how the information can best be organized and presented for quicker OT&E analysis.

  5. System requirements Software requirements Software requirements Analysis Program design Coding Testing Operations Spiral Development – Earlier Testing Waterfall SPIRAL Engineering Production Stages Inception Elaboration Construction Transition Phases VS. Ideas Architecture Products Beta Releases Result • Testing Accomplished Near the end of the program. • 10x cost to affect changes discovered during testing. • Testing begins in the Elaboration phase. • Changes discovered during testing less costly to implement.

  6. SPIRAL Engineering Production Stages Inception Elaboration Construction Transition Phases Ideas Architecture Beta Releases Products Result Spiral Development – Earlier Testing Iterative Development Process 1 2 Planning and analysis Design 4 3 Assessment Implementation • Each Phase of development goes through one or many iterative development processes. • Testing – whether it be modeling and simulation or Developmental Testing can occur during these iterative developments. • Discoveries can be rolled into the next iterative cycle.

  7. F-16 AV-8B Iterative Cycle Test Events / INPUTS F/A-18 Planning and analysis Contractor DT Design Government DT Core C4I Implementation Improvements Inserted from early testing discovery START Assessment Spiral Development – Earlier Testing Block I Task: C4I System to Digitally talk to: Core C4I F/A-18 AV-8B F-16 Block II Block I

  8. = A specific Requirement within the document Planning For OT&E • The Project System Engineer should plan to show to the OT&E community how the system has been validated and is ready for test (thread the needle): • TEMP • Test Plans which are Requirements Driven. • Test Procedures which clearly trace back to the Test Plan. • This is accomplished through Requirements Management. ICD Perf Spec Sys Spec TEMP Test Plans Test Procedures Test Results SRR Testing Branch

  9. Collecting and Preparing Testing Data • The Test Team Lead needs to evaluate the type of data that needs to be collected and the format to best store that data. • This should be accomplished before any testing begins. • Data that needs to be sorted or analysis should be inputted into a database. • Requirements Management tools should reflect the data records used for analysis to show how the requirement was validated.

  10. Scripting Data During Test Events • Test Events should be scripted during DT to ensure that all variables that need to be tested are tested and are they tested in the proper method. • Scripting will also make it easier to answer the question of: How did we validate the requirement(s)?

  11. Test Result Reports • Test Result Reports should clearly tell the story of what was being tested, how it was tested, and the results of that testing: • The report should Identify whether the objective of a test run was satisfied. • It should call out the Higher-level requirement (ICD) and reflect which test runs were used in the analysis to satisfy or fail that requirement. • Should identify related areas to the test event which could not be tested. To assist with a follow-on test. • Should identify discrepancies with systems outside of the program in which the system has to Interoperate with.

  12. First Steps Have the Biggest Impact • Proper Planning up front will pay dividends later on in a program: • OT&E up-front involvement - will alleviate untestable requirements when the program gets to DT and OT&E. • Requirements Management - provides traceability • Can occur parallel to other developing efforts of a program. • Allows OT&E to effectively audit the program. • Test Plans – should design and organize what you want to accomplish. • Collection of Data - knowing what you need to do with the data collected will identify the best format to collect it. • The products produced (Test Result Reports) should tell a clear story. • Think the task through before you even take the first step.

  13. Questions??? Anthony Devino 540-653-3792 Devinoaj@nswc.navy.mil

More Related