MRL and SBIR Success! IT’S NOT JUST THE SBIR TECHNOLOGY ! TRL MRL Manufacturing Readiness Level BR Readiness Levels - Technology - Manufacturing - Business - Customer CR Joe Graben, Director USM Business & Innovation Assistance Center May 2009 www.usm.edu/biac/
IT’S NOT JUST THE SBIR TECHNOLOGY !
Manufacturing Readiness Level
Joe Graben, Director
USM Business & Innovation Assistance Center
TRL = Technology Readiness Level
MRL = Manufacturing Readiness Level
BR = Business Readiness
CR = Customer Readiness
All four “readiness” levels are critical to successfully reaching transition to operational use.
Just because you have developed an SBIR technology through a phase II contract and there is a “customer” with a “need” doesn’t necessarily mean you can sell that technology as a part of system, sub-system, or sub-assembly and that the customer will readily insert your product into their operational use.
There are many variables that must be addressed before you can succeed – you need to assure all your “ducks are in a row”.
The ultimate “success” in the SBIR program is for the small business to be “commercializing” their SBIR technology by producing a product that is being used in either the military and/or civilian marketplaces.
The ultimate “success” in the SBIR program is for the small business to be “commercializing” their SBIR technology by producing a product that is being used in either the military or civilian marketplaces.
All four “readiness” levels are critical to successfully reaching transition to operational use.
Just how does the small business plan on getting their SBIR Technology designed and manufactured into a usable end product the Government/Commercial markets can use?
A General Accounting Office (GAO) assessment of 54 major weapon programs found that the majority of programs were costing more and taking longer to develop than planned. Why did this happen? The programs were going ahead with less knowledge at critical junctures than suggested by best practices. These critical junctures known as “Knowledge Points” include:
Lets look at these knowledge points.
For more information see the GAO Report: GAO-05-301 at:http://www.gao.gov/new.items/d06391.pdf
Knowledge Points and associated indicators are defined as follows and all basically look at the maturity levels at a critical juncture:
Technology is mature. This means that technologies need to meet essential product requirements and have been demonstrated to work in their intended environment. This requires a close matching of customer requirements and resources. A gap between industry best practices and actual technology maturity indicates risk.
Product design is stable. This means that the design is stable at the system-level critical design review (midway through development). Best practices should have 90 percent of the drawings at the system-level completed. A gap between industry best practices and actual design stability indicates risk.
Production processes are mature. This means that all key manufacturing processes are in statistical control (repeatable, sustainable and capable) at the start of production. A gap between industry best practices and actual production maturity indicates risk.
What were the specific problems the GAO found with the Knowledge Points on the 54 programs they reviewed?
Attainment of Product Knowledge
Production, design & technology maturity
Desired level of knowledge
Desired level of knowledge
Design & technology maturity
Source: GAO Report GAO 05-301
The GAO study found - Immature Technologies:
Eighty-five percent of the programs began development not having demonstrated all of their technologies as mature. There was a major gap between what they should have known at that point and what they knew. Going forward required a “leap of faith” that somehow a miracle would occur to solve these gaps.
More often than not, programs sought to mature technologies well into system development when they should have been focusing on maturing the system design and preparing for production. These programs moved forward before the technologies were mature, but the miracle failed to appear and it caused problems.
Program acquisition cost:
Rose an average of 21 percent for those programs that preceded with immature technologies.
Rose an average of only one percent for programs with mature technologies!
What Were the Problems?
The GAO study found - Design Instability:
Only 42 percent of programs held design reviews after achieving design stability. The majority moved forward with unstable designs. There was a major gap between what they should have known at that point and what they knew. Going forward required a “leap of faith” that somehow a miracle will occur to solve these gaps. These programs moved forward before the design was mature, the miracle did not occur and it caused problems.
The mature programs experienced:
A 6 percent increase in development costs and a schedule increase of 11 months
Immature programs, those that did not achieve design stability by CDR experienced:
A 46 percent increase in cost and a schedule slip of 29 months
It should be noted that design stability cannot be attained if key technologies are not mature.
What Were the Problems?(continued)
The GAO study found - Production Immaturity:
Successful programs use statistical process control (SPC) as a best practice to bring manufacturing processes under control. Therefore, these processes are stable, capable and repeatable. Of the 54 programs reviewed, only 19 programs were in production or approaching a production decision within the next year. Of the 19, only two programs collected or even planned to collect statistical process control data. There was a major gap between what they should have known at that point and what they knew. Going forward required a “leap of faith” that somehow a miracle will occur to solve these gaps. These programs moved forward to production before the manufacturing processes were mature, the miracle failed to appear and it caused problems.
Unfortunately the GAO only looked at this one manufacturing element (SPC) to judge maturity. The development of Manufacturing Readiness Levels includes the definition of nine different threads or maturity areas for evaluation.
What Were the Problems?(continued)
MRLs provide a Common language and standard for assessing the manufacturing maturity of a technology or product and plans for its future maturation
Complements existing Technology Readiness Levels
Used to assess maturity and risk of a technology’s underlying manufacturing processes
Enable rapid, affordable transition to weapon system programs
Designed to address manufacturing risk mitigation
Manufacturing Readiness Levels (MRLs)
You can access the MRL Definitions at:https://acc.dau.mil/CommunityBrowser.aspx?id=23209
Manufacturing Readiness is addressed in law:
Title 10, Subtitle A, Part IV, Chapter 148 – National Defense Technology and Industrial Base, Defense Reinvestment, and Defense Conversion.
“The Secretary of Defense shall establish a Manufacturing Technology Program to further the national security objectives of section 2501(a) of this title through the development and application of advanced manufacturing technologies and processes that will reduce the acquisition and supportability costs of defense weapon systems and reduce manufacturing and repair cycle times across the life cycles of such systems.
Manufacturing Readiness Levels are also addressed in the Defense Acquisition Guidebook (DAG): “Engineering and Manufacturing Readiness Levels are a means of communicating the degree to which a technology is producible, reliable, and affordable. Their use is consistent with efforts to include the consideration of engineering, manufacturing, and sustainment issues early in a program.
More information can be found in the Manager's Guide to Technology Transition in an Evolutionary Acquisition Environment. Application of EMRLs should be tightly integrated with the technical reviews detailed in Section 4.3.
You can access the DAG at http://akss.dau.mil/dag/DoD5000.asp?view=document
Why is manufacturing readiness really important? Because sometimes we go to war, and for that we need systems, equipment and supplies. The quality, availability and performance of those systems, equipment and supplies is directly tied to the manufacturing capability.
What if we are asked to surge? Do we have the capability to ramp up in time to meet deployment requirements?
What if we are asked to meet a new challenge with improved performance? Can we develop and deploy the solution in a timely and cost effective manner?
And sometimes things just happen, for example,
The canopies you have been manufacturing for the last 10-years suddenly show up with problems because somewhere you lost the expertise.
The Original Equipment Manufacturer (OEM) just notified you that they were no longer going to produce a critical part and now you need alternative sources.
The only factory in the world that produces your critical part just had a fire and there goes your supply chain.
MRL – Why is it Important?
Manufacturing Readiness is tied directly to producibility or to the design. Producibility can be defined as “the measure of the relative ease of manufacturing.” That is, “is it easy to make?”
Producibility is a design accomplishment resulting from a coordinated effort by design engineering and all the functional engineering specialties to create a functional design that optimizes the ease and economy of fabrication, assembly, inspection, test, and acceptance without sacrificing function, performance or quality.
One of the basic producibility principles is to focus on the simplicity of design. Simple designs are actually more elegant and take more effort to achieve than complex designs and include dictates such as:
Use economical materials.
Standardize materials and components.
Minimize parts count.
Eliminate or minimize special tooling and testing.
Lets look at an example of complex design made into a simpler design.
MRL – Why is it Important?
The item on the right is a Bailout Bottle Holder used in the F/A-18 Hornet fighter aircraft. The original design was more complex than it needed to be.
This design was simplified using a technique called Design for Manufacturing and Assembly (DFMA). This technique asks three questions:
During operation, does this part move relative to the part it is connected to ?
Does this part need to be made from a different material than the pare it is connected to?
Does this part need to be removed?
If you can answer no to each of these three questions, then that part is a candidate for re-design. You start by comparing Part No.1 with Part No. 2, then do the same for 1 to 3, 1 to 4, etc., until all combinations have been assessed.
What do you think the effect is of the simpler design on manufacturing and assembly of the design to the right?
Bailout Bottle Holder
MRL – Why is it Important?
DoD and the services and agencies have been conducting Manufacturing Assessments for many years, and many of the services and agencies have checklists and other tools to help you to conduct these assessments. These assessments can include the following:
Production relevant environment – An environment that contains key elements of production realism not normally found in the laboratory environment (e.g. uses production personnel, materials or equipment or tooling, or process steps, or work instructions, etc.). May occur in a laboratory or model shop if key elements of production realism are added.
Production representative environment – An environment (probably on the shop floor) that contains most of the key elements (tooling, equipment, temperature, cleanliness, lighting, personnel skill levels, materials, work instructions, etc) that will be present in the shop floor production areas where low rate production will eventually take place.
Pilot line environment – A shop floor production area that incorporates all of the key elements (equipment, personnel skill levels, materials, components, work instructions, tooling, etc.) required to produce production configuration items, subsystems or systems that meet design requirements in low rate production. To the maximum extent practical, the pilot line should be representative of processes to be used in rate production.
Manufacturing Readiness Levels
Full Rate Production demonstrated and lean production practices in place
Low Rate Production demonstrated. Capability in place to begin Full Rate Production
Pilot line capability demonstrated. Ready to begin low rate production
Capability to produce systems, subsystems or components in a production representative environment
Capability to produce a prototype system or subsystem in a production relevant environment
Capability to produce prototype components in a production relevant environment
Capability to produce the technology in a laboratory environment
Manufacturing concepts identified
MRL 3 – Manufacturing Concepts Identified
Identification of current manufacturing concepts or producibility needs based on laboratory studies. Materials characterized for manufacturability. Assumed that all corresponding TRL requirements are met for each MRL.
Pre Concept Refinement Phase
MRL 4 – Produce the Technology in a Laboratory Environment
Required investments, such as manufacturing technology development identified. Processes to ensure manufacturability, producibility and quality are in place and are sufficient to produce technology demonstrators. Manufacturing risks identified for prototype build. Manufacturing cost drivers emerging. Producibility assessments of design concepts have been completed. Key Performance Parameters (KPP) identified. Special needs identified for tooling, facilities, material handling and skills.
Concept Refinement (CR) Phase leading to a Milestone A decision
MRL 5 – Produce Prototype Components in a Production Relevant Environment
Manufacturing strategy refined and integrated with Risk Management Plan. Identification of enabling/critical technologies and components is complete. Prototype materials, tooling and test equipment, as well as personnel skills have been demonstrated on components in a production relevant environment, but many manufacturing processes and procedures are still in development. Manufacturing technology development efforts initiated or ongoing. Producibility assessments of key technologies and components ongoing. Component Design to Cost (DTC) goals set.
Early Technology Development (TD) Phase
MRL 6 – Produce a Prototype System or Subsystem in a Production Relevant Environment
Initial manufacturing approach developed. Majority of manufacturing processes have been defined and characterized, but there are still significant engineering/design changes. Preliminary design of critical components completed. Producibility assessments of key technologies complete. Prototype materials, tooling and test equipment, as well as personnel skills have been demonstrated on subsystems/systems in a production relevant environment. Production cost drivers/goals analyzed. Producibility considerations shape system development plans. System level DTC goals set. Long lead and key supply chain elements identified. Industrial Capabilities Assessment (ICA) for MS B completed.
Later Technology Development (TD) Phase leading to a Milestone B decision
MRL 7 – Produce Systems, Subsystems or Components in a Production Representative Environment
Detailed design is underway. Material specifications are approved. Materials available to meet planned pilot line build schedule. Manufacturing processes and procedures demonstrated in a production representative environment. Detailed producibility trade studies and risk assessments underway. Cost reduction efforts underway, incentives in place. Supply chain and supplier QA assessed. Long lead procurement plans in place. Production tooling and test equipment design & development initiated.
System Development and Demonstration (SDD) Phase leading to Design Readiness Review (DRR)
MRL 8 – Pilot Line Capability Demonstrated and Ready to Begin Low Rate Initial Production
Detailed system design essentially complete and sufficiently stable to enter Low Rate Initial Production (LRIP). All materials are available to meet planned LRIP schedule. Manufacturing and quality processes and procedures proven in a pilot line environment, under control and ready for LRIP. Known producibility risks pose no significant risk for LRIP. Program has budget estimate to reach MRL 9, including investments for Full Rate Production (FRP). Supply chain established and stable. ICA for MS C completed.
System Development and Demonstration (SDD) Phase leading to a Milestone C decision
MRL 9 – Low Rate Initial Production Demonstrated and Ready to Begin Full Rate Production
Major system design features are stable and proven in test and evaluation. Materials are available to meet planned full rate production schedules. Manufacturing processes and procedures are established and controlled to three-sigma or some other appropriate quality level to meet design key characteristic tolerances in a Low Rate Initial Production environment. Production risk monitoring ongoing. LRIP cost goals met, learning curve validated.
Production and Deployment Phase leading to a Full Rate Production (FRP) decision
MRL 10 – Full Rate Production Demonstrated and Lean Production Practices in Place
This is the highest level of production readiness. Engineering/design changes are few and generally limited to quality and cost improvements. System, components or items are in rate production and meet all engineering, performance, quality and reliability requirements. All materials, manufacturing processes and procedures, inspection and test equipment are in production and controlled to six-sigma or some other appropriate quality level. Production actual costs meet FRP goals. Lean practices well established and continuous process improvements ongoing.
Latter Stages of Production and Deployment Phase plus Operations and Support Phase
The Mississippi Manufacturing Extension Program (MEP.ms) may be another useful source of assistance in helping to address MRL issues:
MRL Assist Tool Website
A useful MRL assist tool developed under the direction of the Joint Defense Manufacturing Technology Panel is available on the web at: https://www.mrlassist.bmpcoe.org/
TRLs should not be used interchangeably with MRLs. TRLs focus on a technologies maturity, while MRLs look at the maturity of the manufacturing system and processes that will deliver that design as a final product. A critical technology might have be very mature yet the manufacturing processes needed to produce it may be very immature. This will be especially true if manufacturing is not involved early in the design and development process. One of the classic roles of manufacturing is to take a look at the design and make it producible using tools like Design for Manufacturing and Assembly (DFMA). Some manufacturing considerations are:
Use economical materials. Design solutions seldom involve just one material. If you have a choice of materials that provide the same performance, then choose the least expensive material.
Use economical manufacturing techniques. If you have a choice on which machine or method to use to fabricate or assemble a product choose the least expensive approach.
Standardize materials and components. Often components or materials used in one product can be used in other products.
Design for process repeatability. Use quality control tools to make your processes more repeatable. If your factory floor processes are capable and in control then you stand a better chance of achieving your design goals.
Design for Product inspectability. Consider how you are going to inspect or verify that the product will meet its objectives. If you are inventing a new material you need to ask yourself how you are going to determine its acceptability.
Minimize your skill levels required to manufacture. Henry Ford used simplicity to enable workers to focus on only one task. By doing this Ford was then able to connect the line to a chain and pull the vehicle through the factory giving the world its first moving assembly line.
Manufacturing risk/maturity is not the only cost/schedule/performance driver, but we need to manage manufacturing readiness integral to the acquisition process – increase successful technology transition
The intent for SBIRs that address manufacturing is the same as for other programs – to make the product less risky for the customer
Products made by mature manufacturing processes generally:
Are less prone to quality problems
Make the product / process perform the same, and perform better as a whole
Are more reliable in service
Have less difficult time delivering on schedule
Likewise we want to address not just manufacturing technology readiness and the manufacturing capability, but also the business case for the SBIR
Establish the Business Case
Technology is ready for transition – and customers exist for product / process
Long term agreements are potentially available for delivering product
Capital investments--when does the company invest? Government?
TRL -Technology mature and design stable
TRL - Technology proven to work in the field
MRL - Technology can be mass produced in an end product
Enabling technology, sub-assembly, sub-system, etc
BR - Company is in a position to sell & support product
Not just an R&D firm
CR - Product fills an AF need
Improves performance, cost effective
CR - The need lines up with the opportunity to insert the product
Production block, lead times, at what chain of supplier tier
CR - The product meets acquisition strategy
Lowest cost, form fit function, production rates, etc.
It only takes one duck to be “out of line” to keep your SBIR company from successfully inserting your technology into operational use!
This presentation based in part and a modification of original “Readiness Level” materials provided by the U.S. Air Force SBIR/STTR Program:
"This U.S. Small Business Administration (SBA) Cooperative Agreement
is partially funded by the SBA. SBA's funding is not an endorsement of
any products, opinions, or services. All SBA funded programs are
extended to the public on a nondiscriminatory basis."