Software Assurance of Programmable Logic Devices

1. Software Assurance of Programmable Logic Devices Kalynnda Berens, SAIC/GRC Kalynnda.Berens@grc.nasa.gov

2. What is Programmable Logic • Programmable Logic Controllers (PLC) • Programmable Logic Devices • Field Programmable Gate Array (FPGA) • Application Specific Integrated Circuit (ASIC) • System-on-chip (SOC) • Complex PLD (CPLD) • others

3. Concerns • PLCs are “programmed”, usually by engineers. Programming languages may be graphical. • PL device complexity is similar to software, with many of the same problems • PL devices are ultimately “hardware”, but programmed like software. • Are PL firmware? • Current assurance activities may not be adequate for the complexity of these devices.

4. Surveys • Usage and Assurance surveys sent to engineers, assurance personnel, and others at NASA Centers • Distributing surveys was more difficult than anticipated • Targeted email requests produced adequate responses

5. Usage Survey • Identification • Programmable Logic (PL) Information • Types of PL Used and what for • Respondent PL familiarity • PL responsibility • Assurance • Standards • Procedures • Configuration Management • Testing • Assurance organizations • Safety

6. Assurance Survey • Identification • Project Programmable Logic Assurance • Center Programmable Logic Assurance • Review • Testing • Auditing • Witnessing • Respondent PL familiarity

7. Usage Survey Responses 2 surveys from non-NASA projects

8. Who filled out the Surveys? • Using PL: • Electronic/Computer engineers – 15 • Other Engineers – 27 • Management – 9 • Software engineer – 1 • Other – 3 • Assuring PL: • Safety – 6 • Software Assurance – 1 • Other assurance - 2

9. PL Used within NASA • PLC – 27 • FPGA – 26 • ASIC – 12 • EPLD, EEPLD – 24 • SOC – 6 • PAL - 5 • Others - 11 PL Usage Survey

10. Where is PL Used? • Facilities – 19 • Flight Systems – 21 • Ground Systems – 11 • Research – 7 • Facilities are nearly all PLC. • PLC used in 3 experiments and 2 ground systems PL Usage Survey

11. Respondent PL Duties • Use, program, and assure – 23 • Use and program – 10 • Do not use – 7 • Use, program, or assure – 2 each • Program and assure – 2 • Design – 3 • Other - 4 PL Usage Survey

12. PL Author • Hardware engineer – 31 • Software Engineer – 12 • Specialist – 9 • Center computer science – 2 • Outside source/contractor – 8 • Other – 10 PL Usage Survey

13. PL Programming PL Usage Survey

14. Standards None – 35 Project/Center – 12 National/International – 4 Development/Design Procedures None – 31 Project/Center – 18 National/International – 1 Project Guidelines Yes – 20 No – 33 Configuration Management Yes – 39 No – 15 Standards, Procedures, Guidelines, Oh my! PL Usage Survey

15. Who Tests? Engineer or HW team – 31 Team with Software – 8 Team (sw?) – 5 User – 1 Technicians – 3 Engineer w/ QA – 2 Other - 3 How Tested? Successful test – 26 Simulation – 16 Extensive testing – 16 Incremental testing - 13 Testing PL Usage Survey

16. Assurance Activities Other: Project reviews, Configuration management, FMEA PL Usage Survey

17. Safety PL Usage Survey

18. Supported projects with PL? 9 (90%) # projects? 29 total 1 respondent with 10 2 with 5 Safety-related Yes – 5 No – 3 PL used in projects PLC – 5 FPGA – 5 ASIC – 2 SOC – 2 Others - 6 Assurance Projects PL Assurance Survey

19. Who performs functions? 1 Vendor or contractor 2 Safety personnel PL Assurance Survey

20. PL Knowledge PL Assurance Survey

21. SA Support* *For PL safety-related activities PL Assurance Survey

22. Standards used • SCD • IEC 61508-3 • NASA-STD-8719.13 PL Assurance Survey

23. Conclusions • Need more data! • PL usage is wide-spread and varied within NASA • Center Assurance organizations not usually involved in PL activities • More assurance activities may be (vehemently) opposed by developers.

24. Interesting Comments • It is a serious mistake to equate VHDL programming to software.  At best it is firmware, but for the most part there isn't a good name for programming FPGA logic.  In a high performance design like our, the minute you forget that you are designing hardware and think you are writing software you fail. • Historically, existing standards have been outdated and limit the capability of firmware development.

25. Interesting Comments • It sounds like this survey is being taken to try and promote additional overhead and oversight.  The core problems that NASA is experiencing stem from excessive overhead and out of control bureaucracy. Attempting to set up a review process for firmware would result in having a committee of less than knowledgeable people assuming technical authority over personnel who would in general be more capable than those reviewing the firmware.  This would be a step in the wrong direction.

26. Interesting Comments • Small projects can't afford QA watching over designer's shoulder to see that he is doing his job properly. The use of Peer Reviews and Design Reviews should be adequate. Using Test Procedures and providing enough testing time would be of greater value of resources and added confidence.

27. Next Step • Gap Analysis • How does NASA compare to other groups? • Is the level of assurance adequate for how the PL is being used within NASA? • Is process assurance required, as well as product assurance? • Is there a complexity threshold? • FAA guidance (DO-254) • What are the standard practices in industry, the military, and other government agencies for PL assurance?

28. Please Take the Survey! http://osat-ext.grc.nasa.gov/rmo/plcsurvey If you have information, suggestions, or contacts, please email me at: Kalynnda.Berens@grc.nasa.gov