1 / 48

Optimizing Test System Development

Optimizing Test System Development. Wyatt Meek VISTA Product Manager. Agenda. Boeing Test Engineering Priorities V I Engineering Overview Addressing Common Software Engineering Issues Configuration Management Project Management Reuse Discussion. Boeing Priorities.

maxime
Download Presentation

Optimizing Test System Development

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Optimizing Test System Development Wyatt Meek VISTA Product Manager National Instruments Confidential

  2. Agenda • Boeing Test Engineering Priorities • V I Engineering Overview • Addressing Common Software Engineering Issues • Configuration Management • Project Management • Reuse • Discussion

  3. Boeing Priorities • What are the top initiatives within the test engineering group? • How do you currently measure success/failure? • What issues does your group currently face? • Are your suppliers required to follow process/development guidelines?

  4. History of V I Engineering • Founded in 1992 • Select NI Alliance Member • Four Locations • Farmington Hills, MI (HQ) • St. Paul, MN • Indianapolis, IN • Los Angeles, CA • Business Units • Engineering Services • Engineering Information Management Systems (EIM) • VISTA

  5. Multi-Industry Experience

  6. research/definition Simplified Waterfall Model for Software Design requirements specification ship/maintain design coding testing

  7. All Too Often… LabVIEW makes it very easy to quickly put something together research/definition requirements specification ship/maintain design coding testing Developers do not follow a process

  8. Case Study Source: CMU/SEI-94-TR-013

  9. Case Study Source: CMU/SEI-94-TR-013

  10. Common Software Engineering Issues • Process Issues • Configuration Management • Project Management • Requirements Management • Quality Assurance (V&V) • Development Issues • Scalability • Reusability This leads to poor code quality and inefficient use of resources

  11. Addressing the Issues • SEI (Software Engineering Institute www.sei.cmu.edu) • SW-CMM • CMMI • IEEE Software Engineering Standards (www.standards.ieee.org/software) • 828, 830, 1028, 1220,1058.1 • ISO/IEC • 9001:2000, 12207, 15288 • Industry Guidelines • FDA • DoE • FAA

  12. Process Implementation • Process Analysis and Definition • Prioritize improvement initiatives • Define development process • Develop templates and guides • Select automation tools • Implement Automation Tools • Training • Software development process • Development and process improvement tools • Common programming techniques and architecture

  13. Challenges with Process Implementation for LabVIEW and TestStand • LabVIEW File Handling • Embedded Source and Object • Static Linkage • Metrics • Complexity • Size • Design Methodology • Style • Patterns

  14. Configuration Management

  15. Configuration Management • Why use Configuration Management? • To know what you’ve built (configuration) • To identify and track needed changes • To track status of configuration items • To deliver known and verified configurations • Four main Configuration Management Functions • Configuration identification • Configuration control • Configuration status accounting • Configuration authentication

  16. CM Terminology • SCC – Source Code Control • Configuration Item – files placed in SCC • Baseline – specific versions of CI’s at a specific time in the product’s lifecycle (ie Beta Release, Rev 2.0.1) • Status Accounting – knowing which version belongs in a baseline and why • Label – assigned as an attribute to a specific version of a file or application, may be used to show status accounting (Alpha, Beta, Reviewed, Verified, etc.)

  17. Example CM Plan and Process • Configuration Items (specifications, code, daq settings, support files, exe’s) • Configuration Control • User roles (developer, lead developer, project manager, operations) • Permissions (read/write access, labels) • Change procedures • Status Accounting • Labeling procedures • Authentication • Audits • Integrity checks

  18. CM Recommendations • Treat read only VIs as locked • Establish common development environment • Do not allow multiple checkouts (merging impossible) • Avoid branching SCC projects (cross linking and merging) • Do not use LLBs during development (can not locate modifications within) • Use labels during development (assign status) • Deploy exe or use integrity check

  19. CM Integration Solutions - demos • Basic • LV 6.0.2 PDS built in SCC • SCC repository independent of LV • Intermediate • LV 6.0.2 PDS and newer built in SCC with VSS or Perforce • Advanced • VISTA CM Tools for LV 6.1 (Base, Full, PDS) and newer with VSS or ClearCase (others in development)

  20. Project Management

  21. Importance of Metrics • Confirm Estimates • Project Monitoring • Cost Justification • Risk Analysis • Impact of Change • Track Improvement Initiatives • Reuse • Productivity • Diagnose root cause of project failure “You can’t manage what you can’t measure”

  22. Standard Metrics (text based languages) • Hours • SLOC • Function Points • Cyclomatic Complexity • EVA (Earned Value Analysis)

  23. LabVIEW Metrics • Hours • SLOC Equivalent Metric • Nodes (VI Metrics) • GOBs (Graphical OBjects calculator) • Cyclomatic Complexity (possible, but not yet available) • EVA

  24. TestStand Metrics • Hours • SLOC Equivalent Metric • Lines in seq file • EVA

  25. Metrics Trials • LabVIEW Developer Exams • GOBs more reliable than nodes • V I Engineering • ~10 GOBs/hr for product development and regulated industries • ~25 GOBs/hr for automotive solutions • Ball Aerospace • 8-10 GOBs/hr for 2 week to 2 man year projects • General Dynamics (CMMI Level 3) • 8.7 GOBs/hr avg for 8000 man hours of work

  26. Variables • Style • Design Patterns / Architecture used (GOOP vs event driven vs traditional) • Quality Level Requirement • Industry (regulated vs non-regulated) • Application Type (product, manufacturing test system, DVT, etc.)

  27. Establishing a Project Plan with Metrics • Review Requirements • Decompose Requirements into Tasks/Components • Complete preliminary design • MS Project example WBS • Estimate Development Tasks based on Historical Metrics and Risk (spreadsheet example) • Select Lifecycle • Identify Quality Goals • Assign Estimate to Non-development Tasks based on Historical Metrics based on Quality Goals and Lifecycle

  28. Establishing Project Control • Identify Project Deliverables for each SLM Phase • Build Master Project Schedule • Assign Planned Value to Deliverables with Project Schedule • SW deliverables back-fire GOBs from hours estimates

  29. Project Monitoring with Metrics • Track % complete for Deliverables • Use GOBs, SLM phase, status on tasks for objective feedback on software deliverables • Roll Up % Complete to EV • Track EV vs. Planned Value, Actual Cost

  30. Project Monitoring with Metrics • Track % complete for Deliverables • Use GOBs, SLM phase for objective feedback on software deliverables • Roll Up % Complete to EV • Track EV vs. Planned Value, Actual Cost • Manage by Budget at Completion vs. Estimate at Completion Variance, etc.

  31. Project Monitoring with Metrics • Track % complete for Deliverables • Use GOBs, SLM phase for objective feedback on software deliverables • Roll Up % Complete to EV • Track EV vs. Planned Value, Actual Cost • Manage by Budget at Completion vs. Estimate at Completion Variance, etc.

  32. Reuse

  33. Coding for Reuse in LabVIEW • Benefits of LabVIEW Reuse • What is Required for Effective Reuse with LabVIEW? • Reuse Tips

  34. Benefits of LabVIEW Reuse • Decreased Development Time • Reduced Risk • Improved Quality • Lower Overall Cost

  35. Requirements for Reuse • Reuse Development Philosophy • Reuse must be a part of the organizational culture • Coding Standards / Style Guides • Coding style needs to be familiar to entire team • Training • Source Code Control

  36. Reuse Development Philosophy • Always Evaluate Reuse Potential • Generalize Code; Avoid project specific code • Document for Reuse and Maintenance • Have Reuse Reviews at the Beginning and the End of the Project

  37. CM and Reuse • Avoid VIs with the Same Name • Put LabVIEW Reuse Libraries in user.lib (treat them similar to vi.lib) • Organize VIs based on code categories • Limit check-out/check-in access to reuse librarians • Categorize LabVIEW Libraries • General reuse, product line reuse, product specific • Maintain Backward Compatibility • Create a new VI before losing backward compatibility

  38. ..\VIE Library.lib\ArrayTools.llb ..\VIE Library.lib\ConfigUtilities.llb ..\VIE Library.lib\ControlFunctions.llb ..\VIE Library.lib\Dialogs.llb ..\VIE Library.lib\ErrorTools.llb ..\VIE Library.lib\FileUtilities.llb ..\VIE Library.lib\GUI.llb ..\VIE Library.lib\GUI.llb\Soft Keyboard.llb ..\VIE Library.lib\NumericTools.llb ..\VIE Library.lib\OS Utilities ..\VIE Library.lib\OS Utilities\Windows.llb ..\VIE Library.lib\SharedTYPE.llb ..\VIE Library.lib\Sound.llb ..\VIE Library.lib\StateMachine.llb ..\VIE Library.lib\StringTools.llb ..\VIE Library.lib\Time.llb ..\VIE Module.lib\AI and Calibration.llb ..\VIE Module.lib\DTS Engine.llb ..\VIE Module.lib\Formula Evaluator.llb ..\VIE Module.lib\SecurityModule.llb ..\VIE Module.lib\System Integrity Module.llb (.llb extensions are used for directory names to be consistent if converted to VI library files) Example Reuse Library Directory Structure in user.lib

  39. LabVIEW Style Affects Reuse • Using a common style and good VI documentation can significantly improve reuse • LabVIEW Development Guidelines, chapter 6 • devstyle.pdf in ..LabVIEW\Search the LabVIEW Bookshelf… • Use a code review checklist • Avoid overly complex code • Use templates to help standardize coding style between multiple developers

  40. Architecture Reuse Tips • Use layered architectures; limit interaction to adjacent layers • Avoid project specific dependencies; use loose coupling • Use Graphical Object-oriented Programming (GOOP) • Avoid global variables • Use standard configuration tools • Separate user-interface VIs from the main application • Think in terms of reusable components • Encapsulate functionality and data (use GOOP) • Remember that how features specific to an architecture are used may limit reuse to that architecture.

  41. VISTA Software engineering approach to the management, development and release of LabVIEW and TestStand software. • Process Consulting • Training • Development and Process Tools

  42. VISTA Configuration and Project Management Tool • Workspace • Metrics • CM Functionality • LV and TS integration • Baselines • Build with integrity (exe or native) • File comparison • File load check

  43. VISTA Productivity and Reuse Tools • Help Generator • Renaming and Relocation Tool • Documentation Tool • Reuse Library • Style Guide

  44. VISTA Quality Assurance • Node/Path Tracker • Cyclomatic Complexity • Traceability Tool

  45. VISTA GOOP Tools • GOOP Inheritance Toolkit • GOOP Wizard 3 with Inheritance • Icon Editor • Development Distribution Tool

  46. GOOP Background • Scalability – this non-OOP is not easy to grow or maintain

  47. GOOP Background • Scalability – layered, OOP approach

  48. Process Assessment Custom and Advanced Training Tools Project Support Architecture Design How Can V I Engineering Help You? Learn more at www.vista.viengineering.com

More Related