WILLIAM MAST GSFC/ WFF Code 598 June 27, 2019

1. What does QE even mean for a CubeSat or SmallSat Mission? WILLIAM MAST GSFC/ WFF Code 598 June 27, 2019

2. Contents • Introduction • Definitions • Risk Based Mission Assurance • Tailoring • Objective • Understanding The Mission Class • Define the Goals and Scope • Mission Objectives • Know your Platform • Tailoring • Parts Control • Schedule • Personnel • Conclusion

3. Introduction This is my response to a question I was asked in March at the Quality Leadership Forum. Information in this presentation is drawn from: • Techniques and Lessons Learned from LADEE, IceCube and other missions. • Risk Based SMA training from Dr. Jesse Leitner/300 • Input from Ron Perison/300 at WFF and Rich Williams/562 • Work done in the Wallops MPL Lab and in support of proposals “What does QE even mean for a CubeSat or SmallSat Mission?”

4. Definitions ASQ/ANSI/ISO 9000:2015: Quality -  as fitness for use, conformance to requirements, and the pursuit of excellence.      -What is built is what the engineers designed and works as intended Quality Assurance – all the planned and systematic activities implemented within the quality system that can be demonstrated to provide confidence that a product or service will fulfill requirements for quality. Quality Control – the operational techniques and activities used to fulfill requirements for quality. Quality assurance relates to how a process is performed or how a product is made, Quality control is more the inspection aspect of quality management

5. Definitions, Cont. Quality Engineering – shift from defining the role of quality from assurance to engineering so that instead of simply assuring quality at the end, integrate quality in the entire development lifecycle to engineer quality  Originated in the software world where code was broken into services and needed to be tested separately from the user interface. • Test driven development, Design tests as you are designing systems • Write code to test code • Move testing upstream in terms of unit component integration testing.  • Build test infrastructure, identify test scope, identify risks, define quality criteria, • start from unit to user acceptance testing, plan mitigation strategies up front -summarized from article by Nitin Mehra, Engineering Director at Indeed) Essentially, this is an additive process. Start with a design goal and design in quality controls with the mission design

6. Risk Based Mission Assurance Code 300 at GSFC has been trending from Quality Specialists to Quality Engineers The approach for SmallSats has been: Risk Based Mission Assurance The process of applying limited resources to maximize the chance for safety & mission success by focusing on mitigating specific risks that are applicable to the project vs. simply enforcing a set of requirements because they have always worked. ---The ability to omit (tailor) tests and activities that do not add quality to the project in proportion to the cost of the activity, or that can be otherwise mitigated. Assumes that you already have libraries of Test and Verification Plans, Design Standards, Engineering processes, etc. that are comprehensive and geared toward larger, more expensive projects. Essentially, this is a Subtractive process

7. Tailoring Tailoring is not: Deleting activities because they are difficult, time consuming, or expensive Tailoring is: Trimming away excess to achieve a personalized fit Reliability is tailorable, Safety is not

8. Objective Same Quality Objective Perform extensive Quality Planning at the start of the mission in order to reduce QA and QC costs throughout the Mission while tracking and understanding the resulting risks and mitigations. --plan and make decisions to conduct the least risky mission per resource spent. To meet this objective, you must fully understand the risks that you are incurring.  • How are the mission objectives affected if a latent defect is undiscovered?  • What would the project do if the defect were discovered? • How does the cost of mitigation compare with the cost of the test? To understand the risks, you must first fully understand the mission

9. Understanding the Mission Class Understand the Quality Expectations: This is usually defined in an AO and bounded by the Mission Class Class D: NPR7120.5 • Cost /schedule are equal or greater considerations compared to mission success • Technical risk is medium by design (may be dominated by yellow risks) • Many credible mission failure mechanisms may exist.  A failure to meet Level 1 requirements prior to minimum lifetime would be treated as a mishap • Examples:  LADEE, IRIS, NICER, and DSCOVR Streamlined Class D: • Class D projects under $150M w/o Launch Vehicle • Smaller Standing Review Board and only two Key Decision Points (KDPs) • Some Documents may be merged • KDP C decision on 60% confidence rather than 70% SubClass D:  7120.8 • Some level of failure at the project level is expected, failure is tracked at a program level, i.e. 15%, (QE is defined at the program level) • Very short mission life • Failure of an individual project prior to mission lifetime is considered an acceptable risk and would not constitute a mishap. • Examples: Sounding Rockets, Balloons “Do No Harm” • Rideshare User’s Guide Requirements defines the requirements which cannot be tailored • Quality and Reliability expectations are defined by the Stakeholder organizations. • No mishap would be declared if the experiment failed to function

10. Understand the Mission Class Cont. • Class D will have more resources for Quality Engineering and noncompliance recovery than the lower classes. • There is a floor of Quality Engineering/ Assurance that should be maintained regardless of mission class. A determination should be made if the project is sufficiently budgeted to meet stakeholder expectations. • Workmanship and attention to process matter and are always good investments.

11. Define your goals and scope Define your goals and scope at the  program start Goals:   What is the intention of the mission for each stakeholder? The intention of the IceCube mission was to conduct a science mission on a cubesat platform using commercially available hardware The intention of WFF was to develop expertise and experience designing and conducting a CubeSat mission. Enhance the capability of an existing larger spacecraft. Education, Training, Technology Development A cubesat mission that is intended to mature technology for a future mission will have a much different mission risks than the same mission intended to educate undergraduate students to work in technical teams. Scope:  What are the performance expectations and constraints? • Programmatic Risk vs. Technical Risk – the risk will be qualitative, not quantitative • Cost Cap, Budget Constraints • Mission Class:  Class D, Class D streamlined, SubClass D (7120.8), Do No Harm • Mission Lifetime, Launch window, Rideshare constraints • One Spacecraft or Constellation.  Series of launches? • Single String/ Selective Redundancy/ Graceful degradation

12. Mission Objectives Know where your goalposts are: • Mission Objectives, • Baseline and Threshold Success Criteria • Level 1 Science Requirements • How these flow down to the Mission and Spacecraft Requirements. • This will tell you what can be tailored and what cannot be tailored • This is the foundation to the mission design and allows you to define risks in terms of meeting mission objectives • Helps to build a de-scope plan describing the difference in cost, schedule and risk between the baseline mission to the threshold mission • The NASA AOs include a template to help with this process

13. Know your Platform Know what you have to work with Understand the Inherent Risks and Advantages of the SmallSat/CubeSat platform you are using, as well as the ground and flight environment This is where you will find most of your tailoring opportunities -Ex. Large mass limits for ESPA satellites allow for less expensive spacecraft structures, radiation shielding, sealed containers, which may require fewer or less severe tests. Other Characteristics to consider: • Simple, well defined launch interface • Small Radiating Surface Area limits power generation • Short mission lifetime • Short Development Schedules • Mass limit allows for radiation shielding • More parts and software reuse • Industry investment in capability development • Performance records from prior flights • Small teams, easy communication • Simplified and combined documentation

14. Tailoring Now you can begin tailoring Mass and Power Margins Margins applied to power and mass may be tailored based on the type and maturity of the hardware The more mature hardware is, the less margin is required. Acceptance testing/ Gap testing • Testing that is done by the vendor may be able to be substituted for testing that is done • Ensure that test results are available and environmental testing is done in flight-like environment and interfaces Environmental Testing • CubeSat Spacecraft and Instrument have about the same complexity as a single subsystem box on a larger spacecraft. Environmental testing is often first performed at the full spacecraft level • Start with GEVS, tailor tests that don’t apply • TV tests may be combined or cycles reduced • TV temperature transition rates can be much higher due to lower mass Performance Testing • Solar Array deployment tests can be conducted at temperature in ambient pressure since the arrays have negligible air resistance. • The Power system can be tested end-to-end by illuminating the arrays with LED panels • The Com System could be tested end-to-end by communicating with the MOC directly through a ground assett or TDRSS link.

15. Tailoring Parts Control Parts Control for CubeSats/SmallSats Obtain parts lists for each commercial board and identify parts that are not suited for the flight environment. • Understand relative families of parts, test records, history, and vulnerabilities • Aerospace and GSFC have databases of commercial parts that have undergone radiation testing • Assess the Risk, Effects on the mission, and replace or mitigate if another card is not available • Budget will likely limit selection to Commercial or Engineering Grade Parts • May need to purchase parts from industry leaders rather than performing our own screening and supply chain monitoring. • Parts Plan will balance risk vs. available funding

16. Tailoring Parts Control Cont. Lower Cost Missions have less Parts Control, less insight, and fewer options Must assess component reliability/ suitability on a board level. Parts lists may not be available, and the project may not have the resources to screen them Flight data and even prior flight information may not be available. Vendors often consider this information owned by their customer. Tracking and assessing the performance of components on NASA missions is even more important. How much Reliability can or should we outsource to vendors? • Gap testing - use vendor testing for acceptance instead of repeating tests • Parts list, radiation screening, supply chain – what can we really affect? • Flight performance – vendors often have large parametric databases, but make continuous improvements. • Determine where the line is that the vendor is trusted and where NASA takes overQA -Gauge the vendor’s depth of experience and the pedigree and performance history of the hardware. The Technology Readiness Level (TRL) definitions may be our besttool for evaluating risk based on hardware maturity and similarity of prior use.

17. Tailoring Schedule • Tailoring Mission Schedules • Combining milestone reviews, typically SRR/MDR, and PDR/CDR • Schedule time and cost for reviews is reduced • May allow earlier procurements at risk • May allow hardware to be fabricated early at risk • Discipline Reviews may be conducted individually at different times. • May allow a risky activity to begin sooner. • Can prevent a late subsystem from delaying other systems • Risks • May not actually reduce critical path. • Specific subsystems may diverge from the baseline design • Pauses that are normally built into a larger schedule which allow teams to catch up are removed

18. Tailoring Personnel Tailoring Personnel Team members perform multiple roles over the project lifecycle • Design team becomes the I&T team and then the Operations team • Systems Engineer becomes backup for I&T lead • More desirable to have one team member full time with multiple related roles than several people assigned at fractional FTEs to single roles. • Specialized engineering support will be fractional FTE Map the team roles to a resource loaded schedule • Ensure roles do not overlap • Ensure backups are availablefor critical tasks CubeSats are notorious for burning out teams, • Ensure that your resources are commensurate with the performance and quality expectations. • Thisis a Quality Issue

19. Conclusion Conclusions: • Quality Engineering can be an important tool to increase the likelihood of mission success. • Can be scaled for a CubeSat mission • Requires a deliberate effort • Requires that you fully understand your mission… which will also improve quality. Defining the mechanics of how to accomplish this on low cost missions is a work in progress. All comments and suggestions are appreciated