MRL and SBIR Success!

1. MRL and SBIR Success! IT’S NOT JUST THE SBIR TECHNOLOGY ! TRL MRL Manufacturing Readiness Level BR Readiness Levels - Technology - Manufacturing - Business - Customer CR Joe Graben, Director USM Business & Innovation Assistance Center May 2009 www.usm.edu/biac/

2. TRL = Technology Readiness Level MRL = Manufacturing Readiness Level BR = Business Readiness CR = Customer Readiness Definitions All four “readiness” levels are critical to successfully reaching transition to operational use.

3. Just because you have developed an SBIR technology through a phase II contract and there is a “customer” with a “need” doesn’t necessarily mean you can sell that technology as a part of system, sub-system, or sub-assembly and that the customer will readily insert your product into their operational use. There are many variables that must be addressed before you can succeed – you need to assure all your “ducks are in a row”. Success?

4. The ultimate “success” in the SBIR program is for the small business to be “commercializing” their SBIR technology by producing a product that is being used in either the military and/or civilian marketplaces. SUCCESS!

5. The ultimate “success” in the SBIR program is for the small business to be “commercializing” their SBIR technology by producing a product that is being used in either the military or civilian marketplaces. Basic SBIR Philosophy BR MRL TRL CR All four “readiness” levels are critical to successfully reaching transition to operational use.

6. Manufacturing Readiness Introduction to Manufacturing Readiness and Its Role in Successful SBIR Commercialization • What is the small business “business” plan? • Will the small business do the soup to nuts? • Will the small business produce the end product? • Will the small business license the technology (patents)? • Will the small business sell the technology? Just how does the small business plan on getting their SBIR Technology designed and manufactured into a usable end product the Government/Commercial markets can use?

7. The Problem: Risk A General Accounting Office (GAO) assessment of 54 major weapon programs found that the majority of programs were costing more and taking longer to develop than planned. Why did this happen? The programs were going ahead with less knowledge at critical junctures than suggested by best practices. These critical junctures known as “Knowledge Points” include: Technology maturity Design maturity Production maturity Lets look at these knowledge points. For more information see the GAO Report: GAO-05-301 at:http://www.gao.gov/new.items/d06391.pdf

8. What are Knowledge Points? Knowledge Points and associated indicators are defined as follows and all basically look at the maturity levels at a critical juncture: Technology is mature. This means that technologies need to meet essential product requirements and have been demonstrated to work in their intended environment. This requires a close matching of customer requirements and resources. A gap between industry best practices and actual technology maturity indicates risk. Product design is stable. This means that the design is stable at the system-level critical design review (midway through development). Best practices should have 90 percent of the drawings at the system-level completed. A gap between industry best practices and actual design stability indicates risk. Production processes are mature. This means that all key manufacturing processes are in statistical control (repeatable, sustainable and capable) at the start of production. A gap between industry best practices and actual production maturity indicates risk. What were the specific problems the GAO found with the Knowledge Points on the 54 programs they reviewed? Attainment of Product Knowledge Gap Indicates Risk Production, design & technology maturity Desired level of knowledge Desired level of knowledge Design & technology maturity Gap Indicates Risk Technology maturity Gap Indicates Risk DoD Design Review Production Decision Development Start Source: GAO Report GAO 05-301

9. The GAO study found - Immature Technologies: Eighty-five percent of the programs began development not having demonstrated all of their technologies as mature. There was a major gap between what they should have known at that point and what they knew. Going forward required a “leap of faith” that somehow a miracle would occur to solve these gaps. More often than not, programs sought to mature technologies well into system development when they should have been focusing on maturing the system design and preparing for production. These programs moved forward before the technologies were mature, but the miracle failed to appear and it caused problems. Program acquisition cost: Rose an average of 21 percent for those programs that preceded with immature technologies. Rose an average of only one percent for programs with mature technologies! What Were the Problems?

10. The GAO study found - Design Instability: Only 42 percent of programs held design reviews after achieving design stability. The majority moved forward with unstable designs. There was a major gap between what they should have known at that point and what they knew. Going forward required a “leap of faith” that somehow a miracle will occur to solve these gaps. These programs moved forward before the design was mature, the miracle did not occur and it caused problems. The mature programs experienced: A 6 percent increase in development costs and a schedule increase of 11 months Immature programs, those that did not achieve design stability by CDR experienced: A 46 percent increase in cost and a schedule slip of 29 months It should be noted that design stability cannot be attained if key technologies are not mature. What Were the Problems?(continued)

11. The GAO study found - Production Immaturity: Successful programs use statistical process control (SPC) as a best practice to bring manufacturing processes under control. Therefore, these processes are stable, capable and repeatable. Of the 54 programs reviewed, only 19 programs were in production or approaching a production decision within the next year. Of the 19, only two programs collected or even planned to collect statistical process control data. There was a major gap between what they should have known at that point and what they knew. Going forward required a “leap of faith” that somehow a miracle will occur to solve these gaps. These programs moved forward to production before the manufacturing processes were mature, the miracle failed to appear and it caused problems. Unfortunately the GAO only looked at this one manufacturing element (SPC) to judge maturity. The development of Manufacturing Readiness Levels includes the definition of nine different threads or maturity areas for evaluation. What Were the Problems?(continued)

12. MRLs provide a Common language and standard for assessing the manufacturing maturity of a technology or product and plans for its future maturation Complements existing Technology Readiness Levels Used to assess maturity and risk of a technology’s underlying manufacturing processes Enable rapid, affordable transition to weapon system programs Designed to address manufacturing risk mitigation Manufacturing Readiness Levels (MRLs) You can access the MRL Definitions at:https://acc.dau.mil/CommunityBrowser.aspx?id=23209

13. MRL – Why is it Important? Manufacturing Readiness is addressed in law: Title 10, Subtitle A, Part IV, Chapter 148 – National Defense Technology and Industrial Base, Defense Reinvestment, and Defense Conversion. “The Secretary of Defense shall establish a Manufacturing Technology Program to further the national security objectives of section 2501(a) of this title through the development and application of advanced manufacturing technologies and processes that will reduce the acquisition and supportability costs of defense weapon systems and reduce manufacturing and repair cycle times across the life cycles of such systems. Manufacturing Readiness Levels are also addressed in the Defense Acquisition Guidebook (DAG): “Engineering and Manufacturing Readiness Levels are a means of communicating the degree to which a technology is producible, reliable, and affordable. Their use is consistent with efforts to include the consideration of engineering, manufacturing, and sustainment issues early in a program. More information can be found in the Manager's Guide to Technology Transition in an Evolutionary Acquisition Environment. Application of EMRLs should be tightly integrated with the technical reviews detailed in Section 4.3. You can access the DAG at http://akss.dau.mil/dag/DoD5000.asp?view=document

14. Why is manufacturing readiness really important? Because sometimes we go to war, and for that we need systems, equipment and supplies. The quality, availability and performance of those systems, equipment and supplies is directly tied to the manufacturing capability. What if we are asked to surge? Do we have the capability to ramp up in time to meet deployment requirements? What if we are asked to meet a new challenge with improved performance? Can we develop and deploy the solution in a timely and cost effective manner? And sometimes things just happen, for example, The canopies you have been manufacturing for the last 10-years suddenly show up with problems because somewhere you lost the expertise. The Original Equipment Manufacturer (OEM) just notified you that they were no longer going to produce a critical part and now you need alternative sources. The only factory in the world that produces your critical part just had a fire and there goes your supply chain. MRL – Why is it Important?

15. Manufacturing Readiness is tied directly to producibility or to the design. Producibility can be defined as “the measure of the relative ease of manufacturing.” That is, “is it easy to make?” Producibility is a design accomplishment resulting from a coordinated effort by design engineering and all the functional engineering specialties to create a functional design that optimizes the ease and economy of fabrication, assembly, inspection, test, and acceptance without sacrificing function, performance or quality. One of the basic producibility principles is to focus on the simplicity of design. Simple designs are actually more elegant and take more effort to achieve than complex designs and include dictates such as: Use economical materials. Standardize materials and components. Minimize parts count. Eliminate or minimize special tooling and testing. Lets look at an example of complex design made into a simpler design. MRL – Why is it Important?

16. The item on the right is a Bailout Bottle Holder used in the F/A-18 Hornet fighter aircraft. The original design was more complex than it needed to be. This design was simplified using a technique called Design for Manufacturing and Assembly (DFMA). This technique asks three questions: During operation, does this part move relative to the part it is connected to ? Does this part need to be made from a different material than the pare it is connected to? Does this part need to be removed? If you can answer no to each of these three questions, then that part is a candidate for re-design. You start by comparing Part No.1 with Part No. 2, then do the same for 1 to 3, 1 to 4, etc., until all combinations have been assessed. What do you think the effect is of the simpler design on manufacturing and assembly of the design to the right? F/A-18 Bailout Bottle Holder MRL – Why is it Important?

17. Manufacturing Readiness Levels DoD and the services and agencies have been conducting Manufacturing Assessments for many years, and many of the services and agencies have checklists and other tools to help you to conduct these assessments. These assessments can include the following: • Production Readiness Review (PRR) • Manufacturing Management Production Capability Review (MMPCR) • Quality Assurance Surveys • Best Manufacturing Practices Center of Excellence Technical Risk Identification and Mitigation System (TRIMS) • Producibility Reviews • In-Process Reviews, etc.

18. Additional Definitions for MRLs Production relevant environment – An environment that contains key elements of production realism not normally found in the laboratory environment (e.g. uses production personnel, materials or equipment or tooling, or process steps, or work instructions, etc.). May occur in a laboratory or model shop if key elements of production realism are added. Production representative environment – An environment (probably on the shop floor) that contains most of the key elements (tooling, equipment, temperature, cleanliness, lighting, personnel skill levels, materials, work instructions, etc) that will be present in the shop floor production areas where low rate production will eventually take place. Pilot line environment – A shop floor production area that incorporates all of the key elements (equipment, personnel skill levels, materials, components, work instructions, tooling, etc.) required to produce production configuration items, subsystems or systems that meet design requirements in low rate production. To the maximum extent practical, the pilot line should be representative of processes to be used in rate production.

19. Manufacturing Readiness Levels Full Rate Production demonstrated and lean production practices in place Low Rate Production demonstrated. Capability in place to begin Full Rate Production Pilot line capability demonstrated. Ready to begin low rate production Capability to produce systems, subsystems or components in a production representative environment Capability to produce a prototype system or subsystem in a production relevant environment Capability to produce prototype components in a production relevant environment Capability to produce the technology in a laboratory environment Manufacturing concepts identified MRL 10 MRL 9 MRL 8 MRL 7 MRL 6 MRL 5 MRL 4 MRL 3 Production & Deployment System Development & Demonstration Technology Development Concept Refinement Pre-Concept Refinement

20. MRL 3 MRL 3 – Manufacturing Concepts Identified Identification of current manufacturing concepts or producibility needs based on laboratory studies. Materials characterized for manufacturability. Assumed that all corresponding TRL requirements are met for each MRL. Pre Concept Refinement Phase • Key Exit Criteria • Potential manufacturing sources identified (Commercial/Government and Domestic/Foreign) • Evaluate materials and processes for manufacturability and producibility • Budget request to achieve MRL 4 submitted • Materials assessed for manufacturability • Identification of manufacturing concepts/producibility needs • Industrial Base capability identified for key technologies

21. MRL 4 – Produce the Technology in a Laboratory Environment Required investments, such as manufacturing technology development identified. Processes to ensure manufacturability, producibility and quality are in place and are sufficient to produce technology demonstrators. Manufacturing risks identified for prototype build. Manufacturing cost drivers emerging. Producibility assessments of design concepts have been completed. Key Performance Parameters (KPP) identified. Special needs identified for tooling, facilities, material handling and skills. Concept Refinement (CR) Phase leading to a Milestone A decision MRL 4 • Key Exit Criteria • Identify Industrial Base capabilities and gaps/risks for Key Technologies • Assess Design for Producibility • Assessment of manufacturing processes for production and O&S • Key Performance Parameters (KPPs) identified • Budget request to achieve MRL 5 submitted • Prior use determined • Risk Reduction Plans in place for new materials • Assess current state-of-the-art of proposed processes • Yield and rate assessments complete • Quality strategy identified

22. MRL 5 – Produce Prototype Components in a Production Relevant Environment Manufacturing strategy refined and integrated with Risk Management Plan. Identification of enabling/critical technologies and components is complete. Prototype materials, tooling and test equipment, as well as personnel skills have been demonstrated on components in a production relevant environment, but many manufacturing processes and procedures are still in development. Manufacturing technology development efforts initiated or ongoing. Producibility assessments of key technologies and components ongoing. Component Design to Cost (DTC) goals set. Early Technology Development (TD) Phase MRL 5 • Key Exit Criteria • Industrial Base assessed for potential manufacturing sources • Initiate assessment of key technology/component producibility • Validate design choices against industrial base capabilities • Identification of enabling/critical technologies and components • Initiate evaluation of design Key Characteristics • Value Stream Map of cost model for all CTEs • Budget request to achieve MRL 6 submitted • Materials produced in a prototype environment • Supply chain sources identified • Process capability requirements identified • Yields and rates established for production and issues identified • Industrial Base capabilities/gaps identified • Special Tooling/Special Test Equipment identified

23. MRL 6 – Produce a Prototype System or Subsystem in a Production Relevant Environment Initial manufacturing approach developed. Majority of manufacturing processes have been defined and characterized, but there are still significant engineering/design changes. Preliminary design of critical components completed. Producibility assessments of key technologies complete. Prototype materials, tooling and test equipment, as well as personnel skills have been demonstrated on subsystems/systems in a production relevant environment. Production cost drivers/goals analyzed. Producibility considerations shape system development plans. System level DTC goals set. Long lead and key supply chain elements identified. Industrial Capabilities Assessment (ICA) for MS B completed. Later Technology Development (TD) Phase leading to a Milestone B decision MRL 6 • Key Exit Criteria • Industrial Capability Assessment for Milestone B completed • Producibility assessments completed • All enabling/critical technologies/components tested and validated • Budget request to achieve MRL 7 submitted • Supply chains in place • Yields and rates evaluated in a production relevant environment • Quality system in place • Workforce skills available • Integrated Master Plan/Integrated Master Schedule developed • Prototype tooling concepts demonstrated in a production relevant environment

24. MRL 7 – Produce Systems, Subsystems or Components in a Production Representative Environment Detailed design is underway. Material specifications are approved. Materials available to meet planned pilot line build schedule. Manufacturing processes and procedures demonstrated in a production representative environment. Detailed producibility trade studies and risk assessments underway. Cost reduction efforts underway, incentives in place. Supply chain and supplier QA assessed. Long lead procurement plans in place. Production tooling and test equipment design & development initiated. System Development and Demonstration (SDD) Phase leading to Design Readiness Review (DRR) MRL 7 • Key Exit Criteria • Industrial capability supports production • Detailed trade studies completed • Product requirements support detailed systems design • Budget request to achieve MRL 8 submitted • Long lead procurement identified/planned for LRIP • Effective supply chain management • Yields and rates estimated in production representative environment • Quality targets established • Workforce/training requirements identified • Facilities for LRIP and FRP identified

25. MRL 8 – Pilot Line Capability Demonstrated and Ready to Begin Low Rate Initial Production Detailed system design essentially complete and sufficiently stable to enter Low Rate Initial Production (LRIP). All materials are available to meet planned LRIP schedule. Manufacturing and quality processes and procedures proven in a pilot line environment, under control and ready for LRIP. Known producibility risks pose no significant risk for LRIP. Program has budget estimate to reach MRL 9, including investments for Full Rate Production (FRP). Supply chain established and stable. ICA for MS C completed. System Development and Demonstration (SDD) Phase leading to a Milestone C decision MRL 8 • Key Exit Criteria • Industrial Capabilities Assessment completed for Milestone C • Producibility improvements implemented • Detailed design complete • Budget request to achieve MRL 9 submitted • Materials proven and validated during SDD • Long lead procurement initiated for LRIP • Supply chain stable to support LRIP • Yields and rates verified for LRIP • Quality targets demonstrated • Workforce trained and certified • Manufacturing plan completed • Tooling and test equipment proven on pilot line

26. MRL 9 – Low Rate Initial Production Demonstrated and Ready to Begin Full Rate Production Major system design features are stable and proven in test and evaluation. Materials are available to meet planned full rate production schedules. Manufacturing processes and procedures are established and controlled to three-sigma or some other appropriate quality level to meet design key characteristic tolerances in a Low Rate Initial Production environment. Production risk monitoring ongoing. LRIP cost goals met, learning curve validated. Production and Deployment Phase leading to a Full Rate Production (FRP) decision MRL 9 • Key Exit Criteria • Industrial capability supports FRP • Producibility risks mitigated and do not threaten FRP • Three sigma or other appropriate quality levels • LRIP cost goals met • Budget estimate for Lean implementation during FRP • Long lead procurement initiated for FRP • Supply chain stable to support FRP • Yield and rate targets achieved • Workforce requirements for FRP identified • Manufacturing plan updated and validated for FRP • Tooling and test equipment proven during LRIP

27. MRL 10 – Full Rate Production Demonstrated and Lean Production Practices in Place This is the highest level of production readiness. Engineering/design changes are few and generally limited to quality and cost improvements. System, components or items are in rate production and meet all engineering, performance, quality and reliability requirements. All materials, manufacturing processes and procedures, inspection and test equipment are in production and controlled to six-sigma or some other appropriate quality level. Production actual costs meet FRP goals. Lean practices well established and continuous process improvements ongoing. Latter Stages of Production and Deployment Phase plus Operations and Support Phase MRL 10! • Key Exit Criteria • Industrial capability supports modifications, upgrades, surge, and other potential requirements • Ongoing producibility improvements • Product design stable • FRP cost goals met • Budget sufficient for production at required rates and schedule • Supply chain proven • Six sigma or other appropriate quality levels • Yield and rate targets achieved • Workforce skill sets maintained • Manufacturing risks mitigated • Proven tooling and testing equipment in place

28. The Mississippi Manufacturing Extension Program (MEP.ms) may be another useful source of assistance in helping to address MRL issues: http://mep.ms/ MRL Assist Tool Website A useful MRL assist tool developed under the direction of the Joint Defense Manufacturing Technology Panel is available on the web at: https://www.mrlassist.bmpcoe.org/

29. MRL/TRL Comparison TRLs should not be used interchangeably with MRLs. TRLs focus on a technologies maturity, while MRLs look at the maturity of the manufacturing system and processes that will deliver that design as a final product. A critical technology might have be very mature yet the manufacturing processes needed to produce it may be very immature. This will be especially true if manufacturing is not involved early in the design and development process. One of the classic roles of manufacturing is to take a look at the design and make it producible using tools like Design for Manufacturing and Assembly (DFMA). Some manufacturing considerations are:  Use economical materials. Design solutions seldom involve just one material. If you have a choice of materials that provide the same performance, then choose the least expensive material.  Use economical manufacturing techniques. If you have a choice on which machine or method to use to fabricate or assemble a product choose the least expensive approach.  Standardize materials and components. Often components or materials used in one product can be used in other products.  Design for process repeatability. Use quality control tools to make your processes more repeatable. If your factory floor processes are capable and in control then you stand a better chance of achieving your design goals.  Design for Product inspectability. Consider how you are going to inspect or verify that the product will meet its objectives. If you are inventing a new material you need to ask yourself how you are going to determine its acceptability. • Use acceptable materials and processes. You should be aware that some materials like methylene chloride is an ozone-depleting compound and is not an unacceptable material for use. So do not embed it in the product or use it in the manufacturing processes you employ. • Minimize your total part count. The number of parts drives a design’s efficiency, reliability, and maintainability. It is usually wise to study the design and work to reduce part count.  Minimize your skill levels required to manufacture. Henry Ford used simplicity to enable workers to focus on only one task. By doing this Ford was then able to connect the line to a chain and pull the vehicle through the factory giving the world its first moving assembly line.

30. Approach for Conducting MRAs • Hardware intensive • Critical mass of time to complete • Newly developed products INTRODUCE • Meet with PM to get buy-in and gather program info • Customize MRL approach for program • Train program IPT on manufacturing tools to support manufacturing lllmaturity efforts TRAIN • Define “end game” for ATD • Define what is to be assessed and why OBJECTIVE STATEMENT DEFINED • Determine current MRL • Develop plan, actions, and estimate costs to get to target MRL • Schedule for implementation ASSESS INCORPORATE • Incorporate MRL into program tasks and reporting • Manage overall process • Manage risk identification/reduction process • Manage manufacturing maturity to target MRL • Reassess as appropriate MANAGE

31. MRA Benefits to SBIRs Manufacturing risk/maturity is not the only cost/schedule/performance driver, but we need to manage manufacturing readiness integral to the acquisition process – increase successful technology transition The intent for SBIRs that address manufacturing is the same as for other programs – to make the product less risky for the customer Products made by mature manufacturing processes generally: Cost less Are less prone to quality problems Make the product / process perform the same, and perform better as a whole Are more reliable in service Have less difficult time delivering on schedule Likewise we want to address not just manufacturing technology readiness and the manufacturing capability, but also the business case for the SBIR Establish the Business Case Technology is ready for transition – and customers exist for product / process Long term agreements are potentially available for delivering product Capital investments--when does the company invest? Government?

32. Getting All the Ducks in a Row TRL - Technology mature and design stable TRL - Technology proven to work in the field MRL - Technology can be mass produced in an end product Enabling technology, sub-assembly, sub-system, etc BR - Company is in a position to sell & support product Not just an R&D firm CR - Product fills an AF need Improves performance, cost effective CR - The need lines up with the opportunity to insert the product Production block, lead times, at what chain of supplier tier CR - The product meets acquisition strategy Lowest cost, form fit function, production rates, etc.

33. It only takes one duck to be “out of line” to keep your SBIR company from successfully inserting your technology into operational use! Ultimate Success! This presentation based in part and a modification of original “Readiness Level” materials provided by the U.S. Air Force SBIR/STTR Program: http://www.sbirsttrmall.com/Library/Default.aspx

34. Contact • To learn more about how the MS-FAST Program can help your company compete in federal R&D programs contact: • Joe Graben, MBA • Director – USM/BIAC • Phone: (228) 688-2280 • E-mail: Joseph.Graben@usm.edu "This U.S. Small Business Administration (SBA) Cooperative Agreement is partially funded by the SBA. SBA's funding is not an endorsement of any products, opinions, or services. All SBA funded programs are extended to the public on a nondiscriminatory basis."