1 / 23

How Do You Test an IT Service?

How Do You Test an IT Service?. Pete Green Tom Frey. Purpose. Explore evolving influence of IT systems and services on weapons systems Increasing complexity of testing Our experience foundation is in NMCI Foci are the test processes. How Did We Get Here?. NMCI Designated DOT&E Oversight

bchilders
Download Presentation

How Do You Test an IT Service?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. How Do You Test an IT Service? Pete Green Tom Frey

  2. Purpose • Explore evolving influence of IT systems and services on weapons systems • Increasing complexity of testing • Our experience foundation is in NMCI • Foci are the test processes

  3. How Did We Get Here? NMCI Designated DOT&E Oversight Program Master Test Plan Completed & Delivered NMCI Contract Awarded to EDS TESP Input Delivered Master Test Plan Development CT&E 2 Testing DASN Directs TESP for NMCI 2000 2001 2002 JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC JAN CT&E 3 Testing BSA conducted at 4 test sites Veridian sole sourced By EDS to conduct NMCI CT&E CT&E 1 Testing Conducted Interoperability Testing & Monitoring Plan Delivery Contract SLAs mapped to COIs

  4. Rolling In • Compressed Schedule • No Traditional Program Documentation • Our Response Based on 4-step Test Process • Build Up (Crawl, Walk, Run) • Concentric Testing • Familiar Fundamentals • Faces • Processes

  5. Test Approach • BSA • Phased Testing • CTE1 - DT (lab, bench) • CTE2 - DT (networks) • CTE3 - DT/OT (operational vignettes) • OPEVAL - OT • SLA based - derived CTPs, COIs,

  6. Test Approach

  7. Test Approach

  8. IT Testing • Similar to Early Aviation Testing • How fast can you go? • How high can you go? • OSI layer 1-3 centric • Parameter focus

  9. IT T&E and the OSI 7-Layer Model 7 6 5 4 3 2 1 Application “Is This System Effective and Suitable?” Where We need to be Mission Based Presentation Session Transport Network Present Commercial IT Testing “How Fast Can This Car Go?” Parametric Data Link Physical

  10. Product Line Maturity Systems Maturity Aircraft Networks Avionics Software 1950 1960 1970 1980 1990 2000

  11. C4I Testing • Success of testing tied to maturity of program execution • OTRR • Schedule Constraints • IP systems generate lots of data • Program Execution curve is fractal • Processes on critical path provide 1st order approximation • Use of surveys • Ao

  12. Testing and Program Execution

  13. Test Issues • NMCI equal to or better than legacy networks • Critical path processes • Seat cutover • Legacy applications • SLAs to COIs not a reversible function • Operationalizing the metrics • Early and combined CT/OT to minimize risk

  14. Baseline System Assessment • Innovative contract vehicle • Lack of traditional documentation • Needed to measure the performance of the legacy systems for comparison with NMCI

  15. BSA Tools • AMP • NetIQ • MIB scripts • Help Desk Logs • BelManage • Surveys • End user • CIO and IT professionals • Information Assurance • Use same tools and processes for OPEVAL

  16. BSA Lessons Learned • Configuration management is historical • Good management makes an immediate impact • Firewalls and IDS • Routers • Connectivity • Legacy applications

  17. TESP Process • Similar to a TEMP • More responsive • If it was not necessary, it was not included • If it was necessary, it was tailored to NMCI • From blank sheet to approval in under 11 months • Represented a strategy for success for NMCI

  18. The State of IT Testing Today • Operational assessments of interoperability and information assurance in fielded systems • Inverse correlation of interoperability and information assurance • Mission Thread Analysis • Blue, Green, and Red Teams • Common Toolkits • Information Operations Threat Capabilities Summaries • Feedback and feed forward

  19. Iop(ax) = by IA Interoperability vs. Information Assurance:A Complex Challenge Information Assurance (IA) Interoperability(Iop) An Enterprise System Needs to Balance Iop and IA

  20. Testing Frameworks System Testing Device Testing Mission-Focused Testing System of Systems Testing Z A Work Flow Threads

  21. Example Mission Thread(ATO Distribution) KCOIC KCOIC BCD GCC Prioritized Integrated KCOIC TWM & KCOIC ITO PLNS CTAPS CTB-AM CTAPS Sent to WS TL # 1,6 WS Server WS Target List Tasking Order 1 2 3 7 2 ACC Target GCCS-K Nomination C2/COP WS RELROK WAN HTACC RELROK WAN Training Facility CTAPS APS ROK AF ( Suwon ) GCCS-K Server Web HTACC CTAPS RELROK WAN Server CTAPS ADS CTAPS RELROK WAN WS Server TGT CTAPS WS ITO DB 4 Server 5 KUNSAN CP TANGO RELROK WAN CTAPS GCCS-K TGT C2/COP WS WS RELROK WAN 6 8

  22. System SMEs Performance Data Scenario OPFOR Campaign Operational Testers Mission SMEs Assessment Methodology Flow SYSTEM ISSUES Lessons Learned CCTI Strategies Joint Mission Areas UJTL Integrated Architectures Standards Assigned CoCom Missions Conditions Commanders Training Guidance C4ISP CoCom JMETs Joint Doctrine TT&Ps Exercise Comm/Sys Architecture Exercise Objectives PERFORMANCE ASSESSMENT MISSION ISSUES Assessment Plan Mission Functional Threads Training Objectives Situation MSEL Level of Perf BLUFOR OPLAN Collection Management Plan Control Plan Execution Performance Analysis After Action Review JTS EXERCISE ISSUES

  23. QUESTIONS?

More Related