Welcome 88384
Download
1 / 30

Welcome - PowerPoint PPT Presentation


  • 188 Views
  • Updated On :
loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Welcome' - victoria


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Welcome l.jpg
Welcome

  • Welcome to the Programmatic Audit and Review (PA&R) Process training module. Module I begins with a basic introduction to the elements of the Office of Safety and Mission Assurance (OSMA) Review and Assessment Division (RAD) and provides a detailed description of the Programmatic Audit & Review (PA&R) Process.

  • The PA&R Process directly supports the OSMA/RAD charter to ensure lifecycle safety and mission success through compliance verification of Agency SMA requirements thus providing a robust foundation for the Safety and Mission Assurance (SMA) operational readiness decisions.


Training agenda l.jpg
Training Agenda

  • Module I - Introduction and PA&R Overview

  • Module II- Knowledge

    • Make Decision to Audit

    • Conduct Program Discovery

    • Understand Baseline SMA Requirements

  • Module III - Visibility

    • Verify Requirements Flow-down

    • Verify Process Capability

    • Verify Requirements Compliance

    • Provide Feedback

  • Module IV – Understanding

    • Support Operational Readiness Decision

  • Module V – Summary


Module i introduction overview l.jpg
Module I - Introduction & Overview

  • Why Conduct a PA&R?

  • What is a PA&R?

  • How Do We Implement the PA&R Process?


Why conduct a pa r l.jpg
Why Conduct a PA&R?

Module I

“ Prepare a detailed plan for defining, establishing, transitioning, and implementing an independent Technical Engineering Authority, independent safety program, and a reorganized Space Shuttle Integration Office…”

Recommendation R9.1-1 of the CAIB Report dated August 2003

The Programmatic Audit and Review (PA&R) Process was initiated in response to the Columbia Accident Investigation Board (CAIB) Findings and Recommendations to provide a robust, structured and formalized independent Safety and Mission Assurance Compliance Verification function in the Agency. It represents, in part, the Agency response to CAIB recommendation R9.1-1 and R7.5-2 on the need to increase independent assessment and compliance verification activities, and is in accordance with commitments by the Administrator to initiate Space Shuttle Program independent review and assessment activity prior to Return to Flight (RTF).

“NASA Headquarters Office of Safety and Mission Assurance should have a direct line authority over the entire Space Shuttle Program safety organization and should be independently resourced.”

Recommendation R7.5-2 of the CAIB Report dated August 2003

This PA&R function was established under a new Review and Assessment Division of the Office of Safety and Mission Assurance in 2003 and was chartered via the normal NPR processes.


Office of safety and mission assurance l.jpg

OSMA

Safety and Assurance Requirements Division (SARD)

Corporate leadership in definition and implementation of NASA's Agency-wide Safety and Mission Assurance Policies, Procedures, Standards, Tools, Techniques and Training.

Mission Support

Division

(MSD)

Primary interface with the customers of the Office of Safety and Mission Assurance

Assurance that appropriate SMA requirements are in place for each program or project.

Review and Assessment Division (RAD)

Primary Independent Corporate Compliance Verification Responsibility

Best Practices & Lessons Learned

Pre-Operations Decision Support

Knowledge, Understanding & Visibility of Compliance with Requirements, Process Capability, Technical Issues, Minority Opinions & Residual Risk

Office of Safety and Mission Assurance

Module I

The Office of Safety and Mission Assurance (OSMA), as well as Center Safety Mission Assurance (SMA) organizations, play a key role in the PA&R Process. The chart below depicts the PA&R-related responsibilities of OSMA. NASA Centers may have similar relationships. Although all divisions of OSMA have a role, RAD has the primary OSMA compliance verification responsibility.


The rad compliance verification landscape l.jpg

Compliance Verification of

Agency SMA Requirement Set

Agency SMA Review & Assessment

Functional Audits

Agency Requirements

Institutional / Facility / Operational (IFO) Safety Audit

Institutional Programmatic Support (IPS) Audit

Center Requirements

Program Requirements

Programmatic Audit & Review

(PA&R)

Contracts

Contractor Requirements

The RAD Compliance Verification Landscape

Module I


The rad compliance verification landscape8 l.jpg
The RAD Compliance Verification Landscape

Module I

Compliance Verification of

Agency SMA Requirement Set

Agency SMA Review & Assessment

Functional Audits

Agency Requirements

Institutional / Facility / Operational (IFO) Safety Audit

Institutional Programmatic Support (IPS) Audit

Center Requirements

Program Requirements

IFO Safety Audits are conducted on site at the NASA Centers to verify compliance with Federal, State, local, and Agency SMA and technical directives, policies, and requirements contained in applicable documents.

These requirement areas within the IFO Baseline Requirements Set (BRS) include, but are not limited to Public/Worker Safety, Facility System Safety, Mishap Investigation, GIDEP, Fire Protection, Equipment Safety, Explosives Safety, Range Safety, and Electrical Safety.

Programmatic Audit & Review

(PA&R)

Contracts

Contractor Requirements


The rad compliance verification landscape9 l.jpg
The RAD Compliance Verification Landscape

Module I

Compliance Verification of

Agency SMA Requirement Set

Agency SMA Review & Assessment

Functional Audits

Agency Requirements

Institutional / Facility / Operational (IFO) Safety Audit

Institutional Programmatic Support (IPS) Audit

Center Requirements

IPS Audits are conducted onsite at the NASA Centers to verify that Institution/Center policies, procedures and processes are in place to define, impose, and implement appropriate Agency SMA requirements for their respective programs and projects.

Program Requirements

  • The purpose of the IPS is twofold:

  • Provide management with an independent, objective, and constructive evaluation of the compliance of the institution with the applicable IPS BRS.

  • Assess the effectiveness of implementation of SMA process and technical directives, policies, and requirements in achieving desired outcomes relevant to the areas being audited.

Programmatic Audit & Review

(PA&R)

Contracts

Contractor Requirements


The rad compliance verification landscape10 l.jpg
The RAD Compliance Verification Landscape

Module I

Compliance Verification of

Agency SMA Requirement Set

Agency SMA Review & Assessment

Functional Audits

Agency Requirements

Institutional / Facility / Operational (IFO) Safety Audit

Institutional Programmatic Support (IPS) Audit

Center Requirements

Program Requirements

Programmatic Audit & Review

(PA&R)

Contracts

The PA&R process provides independent compliance verification of Agency SMA process, technical, and, as appropriate, engineering performance specification (EPS) requirements within the applicable program/project BRS.

Contractor Requirements

PA&Rs include analyses, reviews, and assessments in addition to the on site compliance verification audits at center, prime contractor, and subcontractor facilities as appropriate


The rad compliance verification landscape11 l.jpg

Compliance Verification of

Agency SMA Requirement Set

Agency SMA Review & Assessment

Functional Audits

Agency Requirements

Institutional / Facility / Operational (IFO) Safety Audit

Institutional Programmatic Support (IPS) Audit

Center Requirements

Program Requirements

Programmatic Audit & Review

(PA&R)

Contracts

Contractor Requirements

The difference in RAD programmatic audits is that IPS Audits focus on Center SMA support and the imposition of requirements on Center programs and projects, while PA&Rs focus on specific and unique program and project SMA functions and the implementation of requirements.

The RAD Compliance Verification Landscape

Module I


Authorizing documents l.jpg
Authorizing Documents

Module I

There are several Agency level documents that authorize compliance verification and the PA&R Process:

NASA Policy Directive NPD 1000.0, Section 3.2 states that “NASA employs a system of checks and balances for effective internal control and to ensure the successful achievement of missions, assigning proper levels of influence and action to different organizations. … The Safety & Mission Assurance organization maintains responsibility for verification of programmatic compliance through strategies, policies, and standards.”

NASA Policy Directive NPD 1000.3C, Section 4.6.2.2 (d) states that “the Chief, Safety and Mission Assurance is responsible for …(d) Verifying the effectiveness of SMA requirements, activities, and processes.”

NPR 7120.5D, paragraph 2.5.4 states that “NASA HQ SMA also has a Programmatic Audit and Review (PA&R) process described in NPR 8705.6, Safety and Mission Assurance Audits, Reviews, and Assessments. That process provides independent compliance verification for the applicable NASA SMA process and technical requirements within the program/project safety and mission assurance plan, the program baseline requirements set, and appropriate contract documentation. Program/Project Managers directly support the PA&R process (either Headquarters-led or Center-led) by providing logistics and resource support required for the successful execution of and response to PA&R process activities. They also coordinate with Center SMA and Center procurement officials to ensure that contracts provide for adequate contractor support for all PA&R activities, and they direct and authorize program/project contractors to support PA&R process activities.”

NPD 8700.1C, Section 1.e states that “it is NASA policy to verify and validate life-cycle implementation of SMA, RM, and mission- success requirements through ongoing surveillance of program, project, and contractor processes.”


Governing documents l.jpg

Office of Safety and Mission Assurance

Headquarters Office Work Instruction

HOWI 8700-GE037 Baseline

January 17, 2006

NASA PROCEDURAL REQUIREMENTS

NPR: 8705.6

Effective Date: October 04, 2005

Expiration Date: October 04, 2010

Programmatic Audit and Review (PA&R) Process

Safety and Mission Assurance Audits, Reviews, and Assessments

  • 1. Purpose

  • The purpose of this Office of Safety and Mission Assurance (OSMA) Headquarters Office Work Instruction (HOWI) is to document the process of preparing for and conducting Programmatic Audits and Reviews( PA&R) of NASA programs and projects. This HOWI is an integral part of the NASA Safety and Mission Assurance (SMA) Review and Assessment process, which also includes the NASA Headquarters Institutional/Facility/Operations (IFO) Safety Audit process and the Safety and Mission Success Review (SMSR) process, discussed in their respective HOWIs.

Responsible Office: Office of Safety and Mission Assurance

  • Section 3.2.7.1 [Program/Project Manager shall:] Incorporate PA&R process activities into program/project plans, including a program/project-unique mission assurance process map and matrix developed and maintained by the program/project with appropriate support and guidance of the Headquarters and/or Center SMA organization (Requirement).

  • 3.2.8.1 [SMA Managers Reporting (matrixed or direct) to the Program/Project Manager shall:] Support the program/project manager in the development and maintenance of the program Assurance Process Map and Matrix (Requirement).

Governing Documents

Module I

There are two Documents Governing the PA&R Process

NASA–wide NASA Procedural Requirement NPR 8705.6

Headquarters Office Work Instruction HOWI 8700-GE037

For complete copies of these documents, go to the NASA On-line Directive Information System (NODIS) Library at http://nodis.hq.nasa.gov/


What is a pa r l.jpg

What is a PA&R?

Module I

The Programmatic Audit and Review (PA&R) Process is designed to provide independent verification of NASA Center, Program, or Project compliance with Agency Safety and Mission Assurance (SMA) requirements as reflected in the applicable programmatic SMA Baseline Requirement Set (BRS). The PA&R Process

  • Is an Agency-wide process, elements of which are owned by a variety of HQ and Center organizations.

  • Helps preserve and protect value created by the program or project.

  • Is a presence throughout the program/project lifecycle, from initial planning stages to end-of-life.

  • Supports Operational Decisions with Audit/Review/Assessment Reports and the identification and qualitative assessment of SMA requirements risk.

A set of SMA requirements jointly negotiated among the program/project, engineering community, SMA community, and, as appropriate, institutional organizations imposed on a program/project. Typically, the BRS is a subset of Agency SMA process, technical, and engineering performance specification requirements uniquely applicable to a given NASA program, project, facility, or operation.


Further elaboration on the pa r process l.jpg
Further Elaboration on the PA&R Process (definition, imposition, implementation, and verification).

Module I

  • What PA&R is…

    • An audit, assessment, or review of a program or project activity conducted by a PA&R team of Subject Matter Experts (SMEs) independent of the activity

    • A focused look, in both time-line and scope, to provide specific knowledge, visibility, and understanding into a program or project

      • PA&Rs can look at any given level of the program or project organizational hierarchy at any phase

    • A process that results in specific findings, corrective actions, and a final report and out-briefing to the Chief, Safety and Mission Assurance and the Office of Chief Engineer (HQ-led PA&R) or the Center SMA Director (Center-led PA&R).

  • What PA&R is NOT…

    • Continuous, day-to-day, matrixed SMA support to a program or project


White hat vs black hat l.jpg
White Hat vs. Black Hat (definition, imposition, implementation, and verification).

Module I

  • PA&R is designed to be flexible in a “White Hat/Black Hat” application of audits, assessments and reviews. PA&R results vary by PA&R categories and include both recommendations and findings. These terms are formalized in the following slides.

White Hat:Provides advice/support/mentoring to the person/organization observed. White hat assessor looks at goodness of a process.

Black Hat: Assures that policies, procedures, and practices are in accordance with requirements through the implementation of rigorous audit processes.


Pa r categories l.jpg
PA&R Categories (definition, imposition, implementation, and verification).

Module I

Assessment:An evaluation and analysis of procedures, processes, and practices necessary to effectively implement requirements. The assessment findings are provided for information to the organization being assessed.

Review:An examination and analysis of documentation to evaluate the feasibility, appropriateness, relevance, and/or effectiveness of documentation contents. It is a necessary precursor to an audit or assessment. The review findings are provided for information to the organization being assessed.

Audit:An onsite in-process verification of compliance with policies, procedures, processes, and requirements. Audits have the responsibility for citing non-compliances, as well as observations, and require a follow-up corrective action process, including a corrective action plan and corrective action status report(s).


Pa r findings l.jpg
PA&R Findings (definition, imposition, implementation, and verification).

Module I

Critical Non-compliance (CNC): A failure to comply with a program/project SMA plan requirement, and/or a contract SMA requirement that could lead to: 1) loss of life or serious injury, and/or 2) loss of a high value asset, and/or 3) impairment of mission success. A CNC will require immediate remedial action.

Non-compliance (NC):A failure to comply with a program/ project SMA plan requirement, and/or a contractually imposed SMA requirement.

Observation (OBS): A finding which is technically not a non-compliance but has the potential to adversely affect safety and/or mission success. Observations may also include identification of potential best practices.


How do we implement the pa r process l.jpg

This section of the training generally describes how we implement the PA&R Process. It introduces the Principal Steps and how they fit into the traditional Program/Project Lifecycle

How Do We Implement the PA&R Process?

Module I


Top level pa r goals and process l.jpg

Knowledge implement the PA&R Process. It introduces the Principal Steps and how they fit into the traditional Program/Project Lifecycle

Knowledge

Develop

Develop

of Program SMA requirements

of Program SMA requirements

-

-

Visibility

Visibility

-

-

Gain

Gain

into SMA capability and compliance levels

into SMA capability and compliance levels

Understanding

Understanding

-

-

Communicate

Communicate

of associated

of associated

Residual Risks

Residual Risks

to support Operational Readiness decisions

to support Operational Readiness decisions

Knowledge

Knowledge

Visibility

Visibility

Understanding

Understanding

Support

Support

Make

Make

Conduct

Conduct

Understand

Understand

Verify

Verify

Verify

Verify

Verify Process

Verify Process

Operational

Operational

Decision

Decision

Program

Program

Baseline SMA

Baseline SMA

Requirements

Requirements

Requirements

Requirements

Capability

Capability

Readiness

Readiness

to Audit

to Audit

Discovery

Discovery

Requirements

Requirements

Flow

Flow

-

-

down

down

Compliance

Compliance

Decision

Decision

Provide

Provide

Feedback

Feedback

Top-Level PA&R Goals and Process

Module I

These are the Principal Steps of the PA&R Process. A general discussion follows. Modules II, III, and IV provide a detailed discussion of each of step.


Top level pa r goals and process21 l.jpg
Top-Level PA&R Goals and Process implement the PA&R Process. It introduces the Principal Steps and how they fit into the traditional Program/Project Lifecycle

Module I

  • Make risk-based selection determined by

    direction, request, or formal selection logic

  • Finalize schedule through PA&R Planning

    Workgroup

  • Notify program or project of selection

  • Develop initial scope

  • Establish the PA&R Team


Top level pa r goals and process22 l.jpg
Top-Level PA&R Goals and Process implement the PA&R Process. It introduces the Principal Steps and how they fit into the traditional Program/Project Lifecycle

Module I

Working with Program/Project POC, compile documentation and information to learn about the program:

  • PA&R team briefly summarizes program or project

    SMA processes

  • PA&R reviews Project/Program developed SMA

    Maps and Matrices which identify SMA roles,

    responsibilities, and relationships


Top level pa r goals and process23 l.jpg
Top-Level PA&R Goals and Process implement the PA&R Process. It introduces the Principal Steps and how they fit into the traditional Program/Project Lifecycle

Module I

  • Obtain the program-specific SMA

    baseline requirements set (BRS)

    for each program or project

  • Understand the risk associated

    with any SMA requirements gaps


Top level pa r goals and process24 l.jpg
Top-Level PA&R Goals and Process implement the PA&R Process. It introduces the Principal Steps and how they fit into the traditional Program/Project Lifecycle

Module I

  • Review the formal documented flow-down of the BRS through the organizational chain (program to project to prime contractor to subcontractor)

  • Propose recommendations, as appropriate.


Top level pa r goals and process25 l.jpg
Top-Level PA&R Goals and Process implement the PA&R Process. It introduces the Principal Steps and how they fit into the traditional Program/Project Lifecycle

Module I

  • Review key SMA Processes

  • Assess each implementing organization

    (any level in the organizational chain)

    to ensure there are sufficient SMA

    resources (i.e., staffing, skill mix, tools,

    funding, etc.)

  • Propose recommendations, as appropriate.


Top level pa r goals and process26 l.jpg
Top-Level PA&R Goals and Process implement the PA&R Process. It introduces the Principal Steps and how they fit into the traditional Program/Project Lifecycle

Module I

  • Conduct a rigorous, in-process, on-site

    compliance verification audit to verify

    implementation of baseline requirements

  • Document audit findings which consist of

    non-compliances and observations

  • Track and verify corrective action


Top level pa r goals and process27 l.jpg
Top-Level PA&R Goals and Process implement the PA&R Process. It introduces the Principal Steps and how they fit into the traditional Program/Project Lifecycle

Module I

Inform all SMA process stakeholders of the PA&R results and any suggested revisions to Agency-wide or program-specific requirements


Top level pa r goals and process28 l.jpg
Top-Level PA&R Goals and Process implement the PA&R Process. It introduces the Principal Steps and how they fit into the traditional Program/Project Lifecycle

Module I

Summarize PA&R findings and SMA requirement risks for the Chief, Safety and Mission Assurance, the Chief Engineer or appropriate Center-level decision maker


Pa r a lifecycle approach l.jpg

PA&R Process implement the PA&R Process. It introduces the Principal Steps and how they fit into the traditional Program/Project Lifecycle

1. Make

Decision

To Audit

3. Understand

Baseline SMA

Requirements

4. Verify

Requirements

Flow-down

5. Verify

Process

Capability

6. Verify

Requirements

Compliance

7. Support

Operational

Readiness

Decision

2. Conduct

Program

Discovery

7. Provide

Feedback

Audit / Review Opportunities

Throughout Program Lifecycle

Contract

Award

Build

Review

Launch

SRR

PDR

CDR

FRR

LRR

Program

Lifecycle

Mission Needs

and Conceptual

Studies

Concept

Definition

Design &

Development

Fabrication,

Integration,

Test & Evaluation

Pre-

Operations

Operations /

Disposal

PA&R – A Lifecycle Approach

Module I

  • The PA&R Process is designed to be applied during any phase or before/between any major decision points of a program or project lifecycle, e.g. Program Milestone Reviews such as SRR, PDR, CRR, FRR and LRR. Emphasis typically evolves from concentration on flow-down and capability verification early in a program or project to rigorous compliance verification in later lifecycle phases.

  • Because no two programs/projects are identical, the PA&R Process adapts to the unique complexity of each program or project, taking full advantage of information gained in earlier phases, as well as other internal and external audits and reviews.


Pbma kms provides key support to the pa r process l.jpg
PBMA-KMS Provides Key Support implement the PA&R Process. It introduces the Principal Steps and how they fit into the traditional Program/Project Lifecycleto the PA&R Process

Module I

  • The Process Based Mission Assurance Knowledge Management System (PBMA-KMS) provides quick and easy access to critical Safety and Mission Data to support program managers and SMA professionals.

  • PBMA-KMS functionality is being actively applied to manage the PA&R process:

  • Enhanced Security Work Groups are used to manage and support PA&R activities

    • Library functionality used to distribute team data, requirements documents, briefing materials, and audit reports to team members

    • Audit/Review Findings securely stored via Secure But Unclassified (SBU) capability

  • SecureMeeting is used to share real-time sensitive information and conduct meetings remotely via the Internet.

  • The Knowledge Registry is used to identify individuals with the best SMA / technical expertise to support PA&R audits/reviews.

  • To access PBMA go to:

http://pbma.nasa.gov


ad