Manufacturing Readiness Assessment Acquisition Training
This presentation is the property of its rightful owner.
Sponsored Links
1 / 41

Gary Stanley Manufacturing Technology Division PowerPoint PPT Presentation


  • 129 Views
  • Uploaded on
  • Presentation posted in: General

Manufacturing Readiness Assessment Acquisition Training. Gary Stanley Manufacturing Technology Division AFRL/RXMT Phone # 937-904-4398 [email protected] Public Release Case # 88ABW-2008-0329. Defense Policy Insertion Plans: Policy Targets. DoD documents

Download Presentation

Gary Stanley Manufacturing Technology Division

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Gary stanley manufacturing technology division

Manufacturing Readiness Assessment Acquisition Training

Gary Stanley

Manufacturing Technology Division

AFRL/RXMT Phone # [email protected]

Public Release Case # 88ABW-2008-0329


Defense policy insertion plans policy targets

Defense Policy Insertion Plans:Policy Targets

  • DoD documents

    1) Defense Acquisition Guidebook

    2) DoDI 5000.2

    3) DoDD 5000.1

  • Air Force documents

    • AFI 63-1201, “Disciplined Systems Engineering”

    • Secy of the Air Force for Acquisition Policy Memo

    • AFRL Advanced Technology Demonstrations

      • Program Baseline Development

      • AFMC Instruction 61-102, ATD Tech Transition Planning


What will the policy look like

What Will The Policy Look Like?

B

C

A

MRL 8

Pilot line capability demonstrated. Ready to

begin low rate production

MRL 7

Capability to produce systems, subsystems or components in a production representative environment

MRL 1

Basic mfg implications identified

MRL 3

Mfg proof of concept developed

MRL 2

Mfg concepts identified

MRL 9

Low rate production demonstrated. Capability in place to begin full rate production

MRL 5

Capability to produce prototype components in a production relevant environment

MRL 6

Capability to produce a prototype system or subsystem in a production relevant environment

MRL 10

Full rate production demonstrated and lean production practices in place

MRL 4

Capability to produce the technology in a laboratory environment

  • MRLs are linked very closely with TRLs

  • MRAs will be performed prior to each Milestone Decision

    • M/S A – MRL 4

    • M/S B – MRL 6

    • M/S C – MRL 8

    • Full Rate Production – MRL 9


Prr and mra

PRR and MRA

  • The areas evaluated in a PRR and MRA process are nearly the same

  • Major differences

    • Timing - PRRs usually occur in the SDD phase versus MRAs that occur throughout the Acquisition and S&T Phases

    • PRR will evaluate the total program's readiness to proceed into production vs MRA focusing on MRL ratings and MMPs

    • PRR focuses on a total program risk assessment versus MRA providing an objective score on the manufacturing maturity of the program and how to achieve required MRLs

  • Bottom Line – if you have done a PRR, you can do an MRA

    • Requires same skill base


Gary stanley manufacturing technology division

Manufacturing Readiness Assessment Acquisition Training

Jim Morgan

Manufacturing Technology Division

AFRL/RXMT Phone # [email protected]

Public Release Case # 88ABW-2008-0329


Session outline

Session Outline

  • What is a Manufacturing Readiness Assessment (MRA)?

  • Why Manufacturing Readiness?

  • What are Manufacturing Readiness Levels (MRLs) and how do they pertain to the Acquisition Life Cycle?

  • How to do an MRA

  • Sample Outputs and Deliverables

  • Finding and Conclusions

  • Additional Information


What is a manufacturing readiness assessment

What Is A Manufacturing Readiness Assessment?

  • An MRA is

    • An Assessment of a Program’s Readiness to Manufacture and Produce Its Intended Design

    • A Tool to Develop and Implement -

      • Manufacturing Risk Mitigation Plans

      • Business Strategies

        • Effects of Design Changes (Planned Upgrades, Spiral)

        • Pricing Agreements (Long Term vs. Single Lot)

        • Capital Investment Plans (Contractor and/or Government)

  • An MRA

    • Assigns Manufacturing Readiness Levels (MRLs) to Key System Components

    • Analogous to Technology Readiness Levels (TRLs)


Session outline1

Session Outline

  • What is a Manufacturing Readiness Assessment (MRA)?

  • Why Manufacturing Readiness?

  • What are Manufacturing Readiness Levels (MRLs) and how do they pertain to the Acquisition Life Cycle?

  • How to do an MRA

  • Sample Outputs and Deliverables

  • Finding and Conclusions

  • Additional Information


Why manufacturing readiness manufacturing industrial base challenge

Why Manufacturing Readiness? Manufacturing & Industrial Base Challenge

  • Consensus among Congress, OSD, CSAF, GAO:

    “Advanced weapon systems cost too much, take too long to field, and are too expensive to sustain”

  • GAO study of 54 weapons programs:

    • Core set of 26 programs: RDT&E costs up by 42% and schedule slipped by 20%

      • $42.7B total cost growth

      • 2.5 years slip on average

    • Characteristics of successful programs (GAO):

      • Mature technologies, stable designs, production processes in control

      • S&T organization responsible for maturing technologies, rather than program or product development manager

  • To mitigate impact of diminishing manufacturing infrastructure

    • People, policy, programs gutted

    • Lost recipe on how to manage manufacturing risk

    • Won’t get infrastructure back but still need to manage manufacturing risk


Why manufacturing readiness acquisition health and manufacturing readiness

Why Manufacturing Readiness? Acquisition Health and Manufacturing Readiness

Manufacturing risk/maturity is not the only cost/schedule/performance driver, but we need to manage manufacturing readiness integral to the overall acquisition process

  • Products made by immature manufacturing processes generally:

    • Cost more

    • Are prone to quality problems

    • May not all perform the same

    • Are less reliable in service

    • Have a hard time delivering on schedule


Technology readiness levels trls

Technology Readiness Levels(TRLs)

Provide a common language and widely-understood standard for:

  • Assessing the performance maturity of a technology and plans for its future maturation

  • Understanding the level of performance risk in trying to transition the technology into a weapon system application

    TRLs leave major transition questions unanswered:

  • Is the technology producible?

  • What will these cost in production?

  • Can these be made in a production environment?

  • Are key materials and components available?


Session outline2

Session Outline

  • What is a Manufacturing Readiness Assessment (MRA)?

  • Why Manufacturing Readiness?

  • What are Manufacturing Readiness Levels (MRLs) and how do they pertain to the Acquisition Life Cycle?

  • How to do an MRA

  • Sample Outputs and Deliverables

  • Finding and Conclusions

  • Additional Information


Gary stanley manufacturing technology division

Manufacturing Readiness Levels(MRL)

  • Common language and standard for

    • Assessing the manufacturing maturity of a technology or product and plans for its future maturation

    • Understanding the level of manufacturing risk in trying to produce a weapon system or transition the technology into a weapon system application

  • Designed to help set the agenda for manufacturing risk mitigation


Mrl relationships

MRL Relationships

Relationship to System Acquisition Milestones

Pre-Concept Refinement

Concept

Refine-ment

Technology Development

System Development & Demonstration

Production & Deployment

A

B

C

MRL 1

Basic Mfg Implications Identified

MRL 3

Mfg

Proof of Concept

Developed

MRL 4

Manufacturing

Processes

In Lab

Environment

MRL 5

Components

In Production

Relevant

Environment

MRL 6

System or

Subsystem

In Production

Relevant

Environment

MRL 7

System or

Subsystem

In Production

Representative

Environment

MRL 8

Pilot Line

Demonstrated

Ready for

LRIP

MRL 9

LRIP

Demonstrated

Ready for

FRP

MRL 10

FRP

Demonstrated

Lean Production

Practices in

place

MRL 2

Mfg

Concepts

Identified

TRL 1

Basic

Principles

Observed

TRL 2

Concept

Formulation

TRL 3

Proof

of

Concept

TRL 4

Breadboard

in

Lab

TRL 5

Breadboard

in Rep

Environment

TRL 6

Prototype

in Rep

Environment

TRL 7

Prototype

in Ops

Environment

TRL 8

System

Qual

TRL 9

Mission

Proven

Relationship to Technology Readiness Levels


Mrl definitions

MRL Definitions

B

C

A

MRL 8

Pilot line capability demonstrated. Ready to

begin low rate production

MRL 7

Capability to produce systems, subsystems or components in a production representative environment

MRL 1

Basic mfg implications identified

MRL 3

Mfg proof of concept developed

MRL 2

Mfg concepts identified

MRL 9

Low rate production demonstrated. Capability in place to begin full rate production

MRL 5

Capability to produce prototype components in a production relevant environment

MRL 6

Capability to produce a prototype system or subsystem in a production relevant environment

MRL 10

Full rate production demonstrated and lean production practices in place

MRL 4

Capability to produce the technology in a laboratory environment

  • Production relevant environment – An environment normally found during MRL 5 and 6 that contains key elements of production realism not normally found in the laboratory environment (e.g. uses production personnel, materials or equipment or tooling, or process steps, or work instructions, stated cycle time, etc.). May occur in a laboratory or model shop if key elements or production realism are added.

  • Production representative environment – An environment normally found during MRL 7 (probably on the manufacturing floor) that contains most of the key elements (tooling, equipment, temperature, cleanliness, lighting, personnel skill levels, materials, work instructions, etc) that will be present in the shop floor production areas where low rate production will eventually take place.

  • Pilot line environment – An environment normally found during MRL 8 in a manufacturing floor production area that incorporates all of the key elements (equipment, personnel skill levels, materials, components, work instructions, tooling, etc.) required to produce production configuration items, subsystems or systems that meet design requirements in low rate production. To the maximum extent practical, the pilot line should utilize rate production processes.


9 mrl evaluation criteria threads

9 MRL Evaluation Criteria(“Threads”)

  • Technology and Industrial Base

    • Technology maturity, technology transition to production, ManTech development

  • Design

    • Producibility program, design maturity

  • Cost and Funding

    • Production cost knowledge (cost modeling), cost analysis, mfg investment budget

  • Materials (raw matls, components, subassys, subsystems)

    • Maturity, availability, supply chain management, special handling

  • Process Capability and Control

    • Modeling & Simulation (product & process), mfg process maturity, process yields/rates

  • Quality Management, to include supplier quality

  • Manufacturing Personnel, to include specialization, training, & certification

  • Facilities, to include capacity and plant layout & design

  • Manufacturing Management

    • Manufacturing planning and scheduling

    • Materials planning

    • Tooling and special test equipment


Milestone b key manufacturing considerations

Milestone “ B ”Key Manufacturing Considerations

A

  • Industrial Base capabilities identified for key technologies and key processes

  • Producibility & Manufacturability assessment of design concepts completed

  • Establishment/validation of manufacturing capability and management of manufacturing risk for the product lifecycle

  • Initial Key Performance Parameters (KPPs) identified

  • Producibility cost risks assessed

  • Survey completed to determine if materials have been used before

  • Lead times identified for all materials

  • Survey completed for potential supply chain sources

  • Special handling requirements identified

  • Survey completed to determine the current state of proposed processes

  • Yield and Rates assessed on proposed processes

  • Quality strategy developed

  • Manufacturing skill sets identified

  • Specialized facility requirements/needs identified

  • Special Tooling/Special Test Equipment (STE) requirements are considered


Milestone b key manufacturing considerations1

Milestone “ B ”Key Manufacturing Considerations

B

  • Industrial capability in place to support manufacturing of development articles

  • Required manufacturing technology development solutions demonstrated in a production relevant environment

  • Producibility assessments of key technologies/components and producibility trade studies completed

  • Key Characteristics and tolerances established

  • Lead times have been identified for all materials

  • Cost model inputs include design requirements, material specifications, tolerances, integrated master schedule, results of system/subsystem simulations and production relevant demonstrations

  • Material maturity verified through technology demonstration articles

  • Availability issues addressed to meet technology demonstration articles

  • Supply chain plans in place

  • Plans to address special handling requirements complete

  • Initial simulation models developed at the technology, sub-system or system level

  • Manufacturing processes demonstrated in production relevant environment

  • Yields and Rates evaluated in production relevant environment

  • Initial Quality Plan and Quality Management System is in place

  • Manufacturing workforce skills available for production in a relevant environment

  • Manufacturing facility and facility development plans adequate to support SDD or Technology insertion

  • Manufacturing risk mitigation approach for SDD or Technology insertion Programs defined.

  • Most material decisions made (make/buy), material risk identified and plans made to mitigate

  • Prototype tooling concepts demonstrated in relevant manufacturing environment.


Milestone c key manufacturing considerations

Milestone “ C ”Key Manufacturing Considerations

C

  • Industrial Capability Assessment (ICA) for MS C has been completed. Industrial capability is in place to support LRIP.

  • Required manufacturing technology solutions validated on a pilot line

  • Known producibility issues have been resolved and pose no significant risk for LRIP

  • Detailed design of product features and interfaces is complete

  • Major product design features are sufficiently stable such that key LRIP manufacturing processes will be representative of those used in FRP

  • Engineering cost model driven by detailed design and validated with data from relevant environment

  • Cost analysis of proposed changes to requirements or configuration

  • Yields and Rates evaluated in production relevant environment

  • Materials proven and validated on System Demonstration and Development (SDD) production as adequate to support LRIP

  • Long Lead procurement initiated for LRIP. Availability issues pose no significant risk for LRIP

  • Most material decisions made (make/buy), material risk identified and plans made to mitigate

  • Prototype tooling concepts demonstrated in relevant mfg environment


Manufacturing considerations for full rate production decision mrl 9

Manufacturing Considerations forFull Rate Production Decision (MRL 9)

  • Industrial capability is in place to support start of FRP

  • Producibility issues/risks discovered in LRIP have been mitigated and pose no significant risk for FRP

  • Known producibility issues have been resolved and pose no significant risk for LRIP

  • Major product design features are stable and LRIP produced items are proven in product testing

  • Major product design features are sufficiently stable such that key LRIP manufacturing processes will be representative of those used in FRP

  • Variability experiments conducted to show FRP impact and potential for continuous improvement

  • Program has budget estimate for lean implementation during FRP

  • Special handling procedures demonstrated in LRIP

  • Manufacturing processes & procedures are established and controlled in production to 3-sigma or other appropriate quality level

  • Yield and rate targets achieved, yield improvements on-going

  • Quality targets verified on production line

  • FRP personnel requirements identified

  • Capacity plans adequate to support FRP decision

  • All manufacturing risks have been validated and mitigated using LRIP articles

  • All tooling, test and inspection equipment proven on LRIP


Session outline3

Session Outline

  • What is a Manufacturing Readiness Assessment (MRA)?

  • Why Manufacturing Readiness?

  • What are Manufacturing Readiness Levels (MRLs) and how do they pertain to the Acquisition Life Cycle?

  • How to do an MRA

  • Sample Outputs and Deliverables

  • Finding and Conclusions

  • Additional Information


Mr assessment process

MR Assessment Process

  • Assessment Lead Briefs PM on Manufacturing Assessment Efforts/Expectations

  • Works with PM to

    • Determine appropriate level for Manufacturing Readiness Assessment(s) (MRAs) -- System may contain several critical technologies/components/manufacturing cells

    • Schedule on-site MRA with contractor(s)

    • Send Orientation Package to contractor(s)

    • Define Assessment Team Membership

    • Define Deliverables of Assessment Results

    • Conduct on-site assessment with contractor(s)

    • Deliver final report/briefing


Manufacturing readiness implementation approach acats

Manufacturing Readiness ImplementationApproach (ACATs)

INTRODUCE

Meet with Wing/Program Management Team

And Other Stakeholders

TRAIN

  • Define Objectives

  • - Yield Improvement

  • New Variant (e.g. Spiral)

  • Increased Capacity (Surge)

OBJECTIVE

STATEMENT

DEFINED

ASSESS

  • Decompose the Problem Space

  • By Technology (i.e. Component)

  • By Supplier

  • - Handle Assembly & Test

INCORPORATE

Wing/PM Team owns the plan

MANAGE


Preparations

Preparations

  • Contact Defense Contract Management Agency (DCMA) to gather information on the contractor’s current and past performance.

  • Notify companies and send orientation package

    • Purpose, approach, questions, strawman agenda

    • MRL definitions/threads

    • Address contract issues if any

    • Self-Assessment

  • Select Assessment team(s)

    • Typically 2-6 members per team

    • Appropriate members (include Gov’t customer)

      • Specialists for key technologies (if needed)

  • Schedule On-site assessments

    • Months prior to key milestone decisions to establish a baseline and allow time to develop/implement risk mitigation plans

  • Team Orientation–Meet prior to on-site assessment


On site mra process review

On-site MRA Process Review

  • Contractor welcome, review of agenda and orientation to facility

  • Introduction of assessment team and contractor personnel

  • Government team lead briefing to contractor describing objectives and expectations for the on-site visit

  • Contractor overview and discussion of the results of their self-assessment

  • Shop-floor visits to key areas by individuals or small groups

  • One-on-one or small group discussions between assessment team members and contractor subject matter experts focused on key areas

  • Private meeting of Government assessment team to:

    • Prepare feedback and identify any action items

    • Initial assessment of current MRL (their area or overall)

    • Key strengths/risks/issues

    • Key missing data (if any)

    • Proposed action items

  • Outbriefing by Government team to contractor


Example process flow generic aircraft

Example Process FlowGeneric Aircraft

  • Large programs can require multiple MRAs

Deliver to Facility B

Install

Fuselage

Fuel System

Install

Cables

Composite

Assembly

Composite

Fabrication

Install

Oil System

Install

Avionics

Assemble

Tails

Engine

Delivery

Cable

Fabrication

INS

Delivery

Landing Gear

Build-up

Engine Build-up

Deliver to Facility C

Initial

Power-up

Checks

Assemble

Wings

Install

Power plant

Install

Landing Gear

Install

Brake System

Install Engine Cowls

Install

Wings/Tails

Move A/C

To Test

Colors represent supplier/facility location

Final

Inspection

System

Tests


Supplier mra plan

Supplier MRA Plan

  • Identify and prioritize critical suppliers

  • Develop common SOW for distribution to suppliers

    • Scope of MRA detailed

    • Method of MRA detailed

    • Output defined

  • Developed detailed MRA execution plan with each supplier (Schedule, format, personnel)

  • Execute MRA

  • Define/Plan/Execute MRL mitigations

  • Measure mitigation effectiveness, update assessment


Follow on activities

Follow-on Activities

  • Gather any key missing data

  • Convene team meeting -- Typically within 2 weeks of on-site assessment

    • Discuss and finalize assessment

    • Examine current program and manufacturing risk reduction plans

    • Agree on likely MRL at completion of milestone if current plan is followed

  • Share results with contractor

  • Identify the specific risk reduction activities necessary to reach the next milestone

  • Identify the funding, time-phasing and approach to carrying out each activity

  • Prepare and submit final report


Session outline4

Session Outline

  • What is a Manufacturing Readiness Assessment (MRA)?

  • Why Manufacturing Readiness?

  • What are Manufacturing Readiness Levels (MRLs) and how do they pertain to the Acquisition Life Cycle?

  • How to do an MRA

  • Sample Outputs and Deliverables

  • Finding and Conclusions

  • Additional Information


Mra process outputs

MRA Process Outputs

  • Baseline Manufacturing Readiness Levels (MRL)

  • Key Factors Where Manufacturing Readiness Falls Short Of Target

    • Define MRL Driving Issues

  • Identify Programs And Plans To Reach Target MRL

    • Identify Existing Investments and Additional Needs

    • Summarize Improvement In Manufacturing Plan

  • Assess Risk to Manufacturing Cost, Schedule, and Performance

  • Implement and Execute the Manufacturing Plan

  • Assess Effectiveness Of the Manufacturing Plan

    • Address Right Issues?

    • Timely? Adequately Funded?

    • Probability Of Success?

    • Options For Increased Effectiveness


Sample summary roll up of components

SAMPLE SUMMARY ROLL-UP OF COMPONENTS


Sample summary drill down

SAMPLE SUMMARY (Drill down)


Mra risk management

MRA Risk Management

  • Assessing Risk is independent of the MRL value assigned

    • Higher MRL value may be highest risk

      • Eg. Requires new equipment, high cost process

  • Risk Assessment should consider

    • Time needed to reach target MRL

    • Require new personnel, training, capital, or more POM samples to flush out the process

    • Leverage other programs

    • Captive or Merchant Supplier Dependency??

    • Part of a company’s core business

      • Leads into an industrial base assessment

  • Effective of use of Design for Manufacturing Tools and other simulation techniques.


Session outline5

Session Outline

  • What is a Manufacturing Readiness Assessment (MRA)?

  • Why Manufacturing Readiness?

  • What are Manufacturing Readiness Levels (MRLs) and how do they pertain to the Acquisition Life Cycle?

  • How to do an MRA

  • Sample Outputs and Deliverables

  • Finding and Conclusions

  • Additional Information


Afrl rxm mra deskbook

AFRL/RXM MRA Deskbook

  • The “how-to” of MRAs

  • First draft completed in March 07

  • Modeled after TRA Deskbook

    • Similarities

      • Achieving levels of readiness for risk reduction

      • Selection process for assessment areas

    • Differences

      • Readiness in S&T and Acquisition world

      • Rigorous assessment process

  • Draft revised based on lessons learned from Reaper MRA

    • Dec 07, Public releasable, on DAU website


Some mrl thoughts

SomeMRLThoughts

  • MRLs are not a report card

    • MRL 7 might not be good

    • MRL 3 might not be bad

  • MRLs are a tool to manage and mitigate manufacturing risk

    • A common language used to assess manufacturing maturity

    • Provide insight not oversight


Some mra lessons learned

Some MRA Lessons Learned

  • Process is more effective if company is actively engaged in the assessment

  • System integration and test operations are often ripe for maturation efforts

  • Resources required to conduct an MRA will vary significantly

  • Subject matter expertise is needed to “do it right”


Findings and conclusions

Findings and Conclusions

  • Looking at transitioning technology to production

    • Must incentivize good decision processes;

    • Unlike TRLs, going backwards on MRLs might be a good thing

  • A low MRL number may be ok

    • Is there time to raise the level?

    • Is there a new manufacturing process being pursued?

    • Replacing a manual process with an automated process

      • Encouraging repeatability, faster cycle time, etc.

  • Identify opportunities to validate manufacturing processes

    • Avoid accepting analogous process claims during the design phase and claiming fabrication is maturing

  • May never build enough units to reach MRL 10

    • Achieve a six sigma or equivalent process

    • Stable Line, may require a multi-product factory


Mra thoughts

MRA Thoughts

  • MRA process highlights areas needing attention to lower production risk

  • Easy Tracking for Prime contractor and Government as manufacturing proceeds

  • Detailed analysis rank ordered; Can be an investment strategy

  • Accomplished the goal as an acquisition test case

Manufacturing maturity through the MRA process enables efficient, cost effective manufacturing


Session outline6

Session Outline

  • What is a Manufacturing Readiness Assessment (MRA)?

  • Why Manufacturing Readiness?

  • What are Manufacturing Readiness Levels (MRLs) and how do they pertain to the Acquisition Life Cycle?

  • How to do an MRA

  • Sample Outputs and Deliverables

  • Finding and Conclusions

  • Additional Information


Additional information

Additional Information

DoD MRL web site

http://www.dodmrl.com

- MRA link to DAU CoP

- Contains MRL definitions, MRL criteria matrix, MRA Deskbook

and more

TRA Deskbook

http://www.dod.mil/ddre/doc/tra_deskbook.pdf

- MRLs contained in Appendix I

DAU PQM Community of Practice

https://acc.dau.mil/pqm

- Manufacturing Readiness folder

  • Look for MR definitions

  • Look for MR matrix

  • Look for MRL tutorial

  • Look for MRA Deskbook


  • Login