2007 ssc survey analysis sub team summary october 23 2007
Download
1 / 48

2007 SSC Survey Analysis Sub-Team Summary October 23 - PowerPoint PPT Presentation


  • 390 Views
  • Uploaded on

2007 SSC Survey Analysis Sub-Team Summary October 23, 2007 Team: Susan Frailey (Lead), Arne Logan, Jacky Gualandri, John Haddock, EJ Kegerreis, John Stecklow and Dina Heasley SSC 2007 Survey Results In 2007 EVERY person registered in eAuditNet was invited to participate in the survey.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '2007 SSC Survey Analysis Sub-Team Summary October 23' - niveditha


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
2007 ssc survey analysis sub team summary october 23 2007 l.jpg

2007 SSC Survey AnalysisSub-Team SummaryOctober 23, 2007

Team: Susan Frailey (Lead), Arne Logan, Jacky Gualandri, John Haddock,EJ Kegerreis, John Stecklow and Dina Heasley


Slide2 l.jpg

SSC 2007 Survey Results

In 2007 EVERY person registered in eAuditNet was invited to participate in the survey.

In 2003 & 2005, invitations were sent only to eAuditNet Administrators.


Slide3 l.jpg

SSC 2007 Survey Results

  • Results shown are from the 1,085 respondents who provided their identity.

  • As with previous years, anonymous results are not reported for analysis purposes.


Slide4 l.jpg

Survey Addressed the following facets:

  • Audit Sites

  • Redundant Audits

  • Flow Down

  • Task Group

  • Audit Consistency

  • PRI

  • Participation and Awareness

  • General Comments


Slide5 l.jpg

Date Reported

Reported by

Health

2007 Supplier Survey Results

Health

Criteria

Facet & Question No:

Recommendations:

Survey questions:

Actions:

Assigned to:

Status

Conclusions:

10/23/07

Susan Frailey

N/A

Facet: Audit Sites

Questions: 6, 7, 8, 9 & 10

The 25 mile rule should be reconsidered based more on the Quality organisation and system in addition to distance. (NOTE: This pertains to approximately 18% of the respondents.)

There is no rating of this facet since these are “information gathering” questions.

6. NEW QUESTION: Does your company hold multiple

Nadcap approvals at different locations which fall outside of

the 25-mile rule (as defined in NIP 7-01), and are not

classed as satellite sites?

7. NEW QUESTION: How many different locations does your

company have Nadcap approvals for? Please list the

number below.

8. NEW QUESTION: If you have multiple Nadcap Approvals,

are these operating under the same approved quality

system as your main site?

9. NEW QUESTION: Is the quality system at the remote site

controlled by a central quality department?

10. NEW QUESTION: How many miles/kilometers separate

the central/main facility from the other locations? Please

list for each location

  • Supplier Support Committee to make proposal for revision of the 25 mile rule.

  • SSC and NMC working group to finalize an acceptable approach to multiple site operation.

  • SSC

  • NMC/SSC

  • In-work

  • In-work

  • Majority of respondents are single site 82%

  • 97% of respondents have 5 sites or less.

  • 195 responses indicated that where they had multiple sites, they were outside of the 25 mile rule.

  • Of those that operate multiple sites 81% control Quality from a central function but 40% of those sites have their own Quality System.


Slide6 l.jpg

Date Reported

Reported by

Health

2007 Supplier Survey Results

Health

Criteria

Facet & Question No:

Recommendations:

Survey questions:

Actions:

Assigned to:

Status

Conclusions:

10/23/07

Susan Frailey

Facet: Redundant Audits

RED

Questions: 12, 13 & 14

  • All Primes to review their internal systems to ensure they are not performing redundant audits

  • Create an eAuditNet feature for suppliers to log in prime visits, the reason for visit and number of visit days.

  • Create an eAuditNet feature for primes to input in general terms their findings without revealing the source of the findings, in order to validate Nadcap process.

12. REVISED QUESTION: Are Nadcap Prime User Members

performing special process/commodity audits which are

already covered within the scope of the Nadcap audit?

13. NEW QUESTION: You indicated that Nadcap Prime User

Members are performing special process/product audits

which are already covered within the scope of the Nadcap

audit. Is it a result of any of the following? Please select

all that apply (13a)

14. REVISED QUESTION: In 2005 and 2006, which Nadcap

Prime User Members performed special process/product

audits for which you were Nadcap accredited? Please list

in the box below. OPTIONAL (or if you would like a

response back), please list the date(s) of the audit(s) and

name(s) of the auditor(s).

  • Primes to investigate why they are performing redundant audits and report back.

  • Create eAuditNet feature for suppliers

  • Create eAuditNet feature for primes

  • Primes

  • PRI

  • PRI

  • Pending acceptance of action

  • Pending acceptance of action

  • Pending acceptance of action

  • Question 12 is that which the health is measured. Questions 13 & 14 are follow-up.

  • Primes are still conducting visits that suppliers (63%) believe are redundant to Nadcap. Little change from 2005.


Slide7 l.jpg

Date Reported

Reported by

Health

2007 Supplier Survey Results

Health

Criteria

Facet & Question No:

Recommendations:

Survey questions:

Actions:

Assigned to:

Status

Conclusions:

10/23/07

Susan Frailey

Facet: Flow Down

YELLOW

Questions: 15 & 16

  • Create a way for suppliers to electronically identify, who is not providing the specifications or flow down information and what is being or specifically not being provided.

  • Use contract flow templates as a recommended practice of obtaining required information (HT, CP)

15. Do you find flow down of specifications, their revisions,

and other requirements from Nadcap Prime User

Members and their Sub-Tiers a problem?

16. NEW QUESTION: How often does your company

experience a problem obtaining flow down of material,

temper condition, drawings, and other requirements from

the Nadcap Prime User Members and their Sub-Tiers?

  • NMC to review the current flow-down practices to ensure that suppliers are receiving the required information.

  • NMC

  • Pending acceptance of action

  • Health measured for both questions as they are independent questions

  • (Q15) Flow of specifications a problem: Slight improvement from 2005. 43% of suppliers indicated a flow down problem (this is a 24% improvement from 2005).

  • (Q16) Flow down of required information a problem: on average 59% of suppliers have problems obtaining required information to perform the process.


Slide8 l.jpg

Date Reported

Reported by

Health

2007 Supplier Survey Results

Health

Criteria

10/23/07

Susan Frailey

Facet & Question No:

Facet: Task Group

GREEN

Questions: 19, 20, 25 & 28

Recommendations:

Survey questions:

19 & 20: 1) Forward to the responsible groups the specifics/issues identified in the survey

responses for action;

2) SSC better define for the supplier the SSC process of getting issues from the suppliers

to the TG and the response to the Suppliers. Example: TG Supplier Voting Member

(SVM), we have NMC Supplier Voting members who are responsible for the supplier

TG voice;

3) SSC needs to educate suppliers of other data sources of clarifications…

(e.g. eAuditNet)

19. NEW QUESTION: Do the Nadcap checklist questions

need further clarification? If Yes, please identify the

checklists that require additional clarification.

20. NEW QUESTION: If you answered Yes, please identify

the checklists that require clarification.

25. NEW QUESTION: Is standardization among the Task

Groups a problem (ex. merit criteria, failure criteria,

classification of findings, etc.)? Please check all that

apply.

28. NEW QUESTION: Do the Nadcap Prime User Members

place sufficient priority on the globalization of standards?

Actions:

Assigned to:

Status

  • Review and respond to identified issues

  • Define process of getting supplier issues to TG and TG responses to suppliers

  • Educate suppliers of available data sources of check sheet clarifications

  • Task Groups

  • SSC

  • SSC

  • Pending acceptance of action

  • Pending acceptance of action

  • Pending acceptance of action

Conclusions:

  • Health is measured from questions 19 & 25. Question 20 is a follow-on to 19 and question 28 is insufficient to measure health.

  • Questions 19 & 20: 73% agree that the checklist require no further clarification.

  • 3) Question 25: 82% agree with Merit Criteria, 85% agree with Failure Criteria and 79% agree with the finding classifications.

  • 4) Question 28: 61% indicated sufficient priority was given to “Global Standardization”, but without knowing the how the term “Globalization of Standards” was interpreted, action cannot be taken from the data.


Slide9 l.jpg

Date Reported

Reported by

Health

2007 Supplier Survey Results

Health

Criteria

Facet & Question No:

Recommendations:

Survey questions:

Actions:

Assigned to:

Status

Conclusions:

10/23/07

Susan Frailey

Facet: Audit Consistency

YELLOW

Questions: 21, 22, 23 & 24

  • Maintain efforts on auditor training to develop common understanding from auditor to auditor.

  • Provide auditors with training for areas which may not be their specialty, especially within those commodities where there are a wide range of processes.

  • Provide more focused training on how to audit.

21. If your company has had more than one (1) Nadcap audit

within a special process/product, please rate the Nadcap

auditors for consistency in their interpretation of the

checklists.

22. If auditor to auditor consistency is a bigger problem in one

special process/product than another, please describe

below.

23. NEW QUESTION: In what areas do you find the Nadcap

auditors inconsistent? Please check all that apply

24. NEW QUESTION: Following your audit(s), do you find

consistency in how the Staff Engineer and the Nadcap

auditor interpret your finding(s)?

  • Task groups to focus on improving auditor consistency during training, including the interpretation of the checklist questions.

  • Task groups to consider how auditors can improve their knowledge in the wider subject content of their area.

  • Staff Engineers and Auditor Training to develop a more in-depth auditor selection and training.

  • Provide specific training on “how to conduct an effective audit”.

  • Task Groups

  • Task Groups

  • PRI Staff Engineers / Auditor Training

  • PRI

  • In-work

  • In-work

  • In-work

  • Pending acceptance of action.

  • There is a 17% improvement in auditor consistency over 2005 result.

  • 30% of respondents feel that there is inconsistency.

  • Two key areas of concern are variations between auditors when reviewing the same process and the auditors interpretation of the checklist.

  • Concerns raised about auditors auditing skills.

  • 74% of respondents felt Staff Engineer and Auditor were consistent with their interpretation.


Slide10 l.jpg

Date Reported

Reported by

Health

2007 Supplier Survey Results

Health

Criteria

Facet & Question No:

Recommendations:

Survey questions:

Actions:

Assigned to:

Status

Conclusions:

10/23/07

Susan Frailey

Facet: PRI

GREEN

Questions: 26, 27 & 31

  • Continue with efforts to translate check sheets into key languages.

  • Continue to endeavor to recruit auditors within key countries with appropriate language skills.

  • Improve flexibility in re-scheduling auto-scheduled audits.

26. NEW QUESTION: Have you experienced any problems

with Nadcap related to language barriers? If so, please

select all that apply.

27. If you have experienced a problem in this area, please list

any suggestions on how PRI can improve.

31. REVISED QUESTION: In 2004, auto-scheduling for

reaccreditation audits was implemented for most special

processes/products. If your company has had an audit

auto-scheduled, please indicate if this enhancement is

beneficial.

  • Globalization Sub-Team continues to work the language issues.

  • PRI to continue recruiting auditors with language skills to alleviate language barriers.

  • PRI to improve upon the re-scheduling flexibility of auto-scheduled audits.

  • Globalization Sub-Team

  • PRI / Auditor Hiring

  • PRI / eAuditNet Administrators & Scheduling

  • In-process

  • In-process

  • In-work

  • 72% have no problems with language. Of those that did have a problem during an audit, 54% were in the USA or UK.

  • Still a strong desire among some to see checklists and audits carried out in their local language.

  • 60% agreed that auto-scheduled audits are beneficial.

  • 9% felt the limited flexibility was a concern with audit scheduled during plant shut-down or scheduled with not enough lead time.


Slide11 l.jpg

Health

Criteria

Date Reported

Reported by

Health

2007 Supplier Survey Results

10/23/07

Susan Frailey

Facet & Question No:

Facet: Participation & Awareness

YELLOW

Question: 30, 32, 33 & 34

Recommendations:

Survey questions:

Questions 30 & 32: re-word questions so action can be taken from results.

Question 33: 1) Add link in eAuditNet to SCC section of PRI website to provide for greater exposure to the SSC and the benefits.

2) Simplify the navigation within the PRI website….too difficult to locate

information….needs be no more than 2 clicks away

Question 34: 3) Of those who want to be involved and gave the OK to contact them, need to be contacted;

4) For those not aware of opportunities need to come up with a way of making

opportunities better known;

5) Of those not interested and gave the OK to contact them, need to be contacted to

find out their reasons; 6) Next survey provide option of giving a reason why no

interest.

30. NEW QUESTION: How would you rate the effectiveness

of Nadcap WebEx meetings and/or trainings?

32. REVISED QUESTION: Do you plan to attend a Nadcap

meeting in the next twelve (12) months? Please select all

that apply…a) Yes, I will attend for the following

benefits…b) No, I will not attend for the following reasons

33. REVISED QUESTION: How well do you feel the Supplier

Support Committee (SSC) represents your interests and

concerns?

34. NEW QUESTION: The Nadcap process needs supplier

involvement in all areas of the process. Are you aware

that you can volunteer for Task Group, Supplier Support

Committee (SSC) and Nadcap Management Council

(NMC) activities and teams?

Actions:

Assigned to:

Status

  • Enhance eAuditNet with link to SSC information

  • Simplify navigation in PRI website to enhance supplier awareness

  • Create more awareness of SSC opportunities

  • Determine the level of interest of suppliers regarding their involvement in SSC.

  • PRI

  • PRI

  • SSC

  • SSC

  • Pending acceptance of action

  • Pending acceptance of action

  • Pending acceptance of action

  • Pending acceptance of action

Conclusions:

  • Health measured from Questions 30, 33, & 34. Question 32 is an informational question.

  • Question 30: Effectiveness of WebEx - 46% had taken advantage of WebEx and 72% of those indicated it was effective.

  • Question 32: Informational question about reasons for either attending Nadcap meetings or not.

  • Question 33: SSC representation - 57% responded they are aware of the SSC and 79% of those feel the SSC represents them Very Well or Well. However, 43% of total were not aware of the SSC.

  • Questions 34: Supplier involvement - 36% not aware of involvement opportunities; 17% want to be involved; 8% indicated they are currently involved; 39% had no interest and no option to provide reason why.


Slide12 l.jpg

Date Reported

Reported by

Health

2007 Supplier Survey Results

Health

Criteria

Facet & Question No:

Recommendations:

Survey questions:

Actions:

Assigned to:

Status

Conclusions:

10/23/07

Susan Frailey

Facet: General Comments

GREEN

Questions: 11, 18 & 29

  • PRI Staff, NMC, SSC, et al, should continue to work to ensure all suppliers’ best interests are being addressed.

  • Notify companies that have responded to SSC Survey of the results/action taken.

11. If your company has been accredited for one (1) year or

more, and your company measures business and/or

quality trends, have you seen improvement in this area(s)

related to your Nadcap accreditation(s)? Select all that

apply.

18. REVISED QUESTION: Prior to your audit, did you take

advantage of pre-audit preparation tools? If yes, which of

the following did you find helpful? Please check all that

apply.

29. How well do you think the Nadcap process takes the

interests of the suppliers into consideration?

  • SSC/PRI to consider how all responding companies will view the Survey Results / Actions Taken.

  • SSC / PRI

  • Pending acceptance of action.

  • 95% of respondents have been accredited for one year or more and measure business and quality trends.

  • 76% of respondents both reported an increase in quality and 45% noted a business increase. This is a 25% increase in business over 2005 data.

  • 97% of respondents utilized pre-audit preparation tools of some sort.

  • 74% of respondents felt their best interests were taken in to consideration. This is a 7% increase from 2005.

  • 26% of the respondents still feel their best interests are not taken into consideration. This is down from 31% in 2005 (Result is a 16% positive trend).




Health criteria l.jpg
Health Criteria

Not Applicable

Legend




Slide19 l.jpg

SSC 2007 Survey Results

Responses by sector.

2007

2005

Asia

9

2%

Asia

90

8%

Europe

70

20%

Americas

688

64%

Europe

306

28%

Americas689

64%

Americas

275

77%

Data not recorded in 2003.


Slide20 l.jpg

SSC 2007 Survey Results

Response by Task Group:

Results from 2005 & 2003 surveys that reviewed similar data are included as an embedded document.


Slide21 l.jpg

SSC 2007 Survey Results

NEW QUESTION: Does your company hold multiple Nadcap approvals at different locations which fall outside of the 25-mile rule (as defined in NIP 7-01), and are not classed as satellite sites?


Slide22 l.jpg

SSC 2007 Survey Results

NEW QUESTION: How many different locations does your company have Nadcap approvals for? Please list the number below.

4.5 is the average

Complete responses are enclosed on the document below.


Slide23 l.jpg

SSC 2007 Survey Results

NEW QUESTION:If you have multiple Nadcap Approvals, are these operating under the same approved quality system as your main site?

Responses are from the 18% who answered Yes to previous question.

117 responses

79 responses


Slide24 l.jpg

SSC 2007 Survey Results

NEW QUESTION:Is the quality system at the remote site controlled by a central quality department ?

159 responses

Responses are from the 18% who answered “Yes” to the first satellite question.

37 responses


Slide25 l.jpg

SSC 2007 Survey Results

NEW QUESTION: How many miles/kilometers separate the central/main facility from the other locations? Please list for each location.

Responses are from the 196 survey participants who indicated that they have a facility that falls outside of the satellite audit rule.

Average is 441 miles

Complete responses are on the document below.


Slide26 l.jpg

SSC 2007 Survey Results

REVISED QUESTION: Are Nadcap Prime User Members performing special process/commodity audits which are already covered within the scope of the Nadcap audit?

Results from 2005 & 2003 surveys that reviewed similar data are included as an embedded document.


Slide27 l.jpg

SSC 2007 Survey Results

NEW QUESTION: You indicated that Nadcap Prime User Members are performing special process/product audits which are already covered within the scope of the Nadcap audit. Is it a result of any of the following? Please select all that apply.

Responses are from the 63% who responded Yes to previous question


Slide28 l.jpg

SSC 2007 Survey Results

REVISED QUESTION:In 2005 and 2006, which Nadcap Prime User Members performed special process/product audits for which you were Nadcap accredited?

Companies listed 50+ times in order:

Results from 2005 survey that reviewed similar data is included as an embedded document. This question was not asked on the 2003 survey.


Slide29 l.jpg

SSC 2007 Survey Results

Do you find flow down of specifications, their revisions, and other requirements from Nadcap Prime User Members and their Sub-Tiers a problem?

In 2003, 5% selected “Other”


Slide30 l.jpg

SSC 2007 Survey Results

NEW QUESTION: How often does your company experience a problem obtaining flowdown of material, temper condition, drawings, and other requirements from the Nadcap Prime User Members and their Sub-Tiers?

Sub-Tier

Primes


Slide31 l.jpg

SSC 2007 Survey Results

NEW QUESTION: Do the Nadcap checklist questions need further clarification? If Yes, please identify the checklists that require additional clarification.

Checklists that were listed and number of times:

AQS 8 CP 69

CT 9 COMP 9

FLU 1 HT 47

MTL 36 NDT 43

NM 6 SEALS 4

SE 4 WLD 14

Additional comments and specific documents listed on document below.


Slide32 l.jpg

SSC 2007 Survey Results

NEW QUESTION:Is standardization among the Task Groups a problem (ex. merit criteria, failure criteria, classification of findings, etc.)? Please check all that apply.


Slide33 l.jpg

SSC 2007 Survey Results

NEW QUESTION: Do the Nadcap Prime User Members place sufficient priority on the globalization of standards?


Slide34 l.jpg

SSC 2007 Survey Results

If your company has had more than one (1) Nadcap audit within a special process/product, please rate the Nadcap auditors for consistency in their interpretation of the checklists.If auditor to auditor consistency is a bigger problem in one special process/product than another, please describe below.

162 Comments were made. Task Groups mentioned:

CP 36

HT 30

NDT 24

WLD 10

NM 6

MTL 5

CT 3

SE 2

Other comments are general to the process and included on the document below.


Slide35 l.jpg

SSC 2007 Survey Results

NEW QUESTION: In what areas do you find the Nadcap auditors inconsistent? Please check all that apply.


Slide36 l.jpg

SSC 2007 Survey Results

NEW QUESTION: Following your audit(s), do you find consistency in how the Staff Engineer and the Nadcap auditor interpret your finding(s)?


Slide37 l.jpg

SSC 2007 Survey Results

NEW QUESTION: Have you experienced any problems with Nadcap related to language barriers? If so, please select all that apply.If you have experienced a problem in this area, please list any suggestions on how PRI can improve.

130 Comments/ Suggestions were made; hire auditors that speak language of supplier, translate checklists in other languages, allow responses to be submitted in any language, etc.

All comments are included on the embedded document.

See next slide for

country breakdown


Slide38 l.jpg

SSC 2007 Survey Results

Language Question – Additional Data:

Countries of respondents who indicated they had a problem with language during a Nadcap audit:


Slide39 l.jpg

SSC 2007 Survey Results

REVISED QUESTION: In 2004, auto-scheduling for reaccreditation audits was implemented for most special processes/products. If your company has had an audit auto-scheduled, please indicate if this enhancement is beneficial.

75 Comments made; no opinion, audits are scheduled too far in advance, more flexibility is needed, etc. All comments are included on the document below.

2005 Results are included, data not collected in 2003.


Slide40 l.jpg

SSC 2007 Survey Results

NEW QUESTION: How would you rate the effectiveness of Nadcap WebEx meetings and/or trainings?


Slide41 l.jpg

SSC 2007 Survey Results

REVISED QUESTION: Do you plan to attend a Nadcap meeting in the next twelve (12) months? Please select all that apply.

Yes, I will attend for the following benefits…

Results from 2005 & 2003 surveys that reviewed similar data are included as an embedded document.


Slide42 l.jpg

SSC 2007 Survey Results

REVISED QUESTION: Do you plan to attend a Nadcap meeting in the next twelve (12) months? Please select all that apply.

No, I will not attend for the following reasons...


Slide43 l.jpg

SSC 2007 Survey Results

REVISED QUESTION: How well do you feel the Supplier Support Committee (SSC) represents your interests and concerns?

Results from 2005 & 2003 surveys that reviewed similar data are included as an embedded document.


Slide44 l.jpg

SSC 2007 Survey Results

NEW QUESTION: The Nadcap process needs supplier involvement in all areas of the process. Are you aware that you can volunteer for Task Group, Supplier Support Committee (SSC) and Nadcap Management Council (NMC) activities and teams?


Slide45 l.jpg

SSC 2007 Survey Results

If your company has been accredited for one (1) year or more, and your company measures business and/or quality trends, have you seen improvement in this area(s) related to your Nadcap accreditation(s)? Select all that apply.


Slide46 l.jpg

SSC 2007 Survey Results

REVISED QUESTION: Prior to your audit, did you take advantage of pre-audit preparation tools? If yes, which of the following did you find helpful? Please check all that apply.

Results from 2005 & 2003 surveys that reviewed similar data is included as an embedded document.


Slide47 l.jpg

SSC 2007 Survey Results

How well do you think the Nadcap process takes the interests of the suppliers into consideration?

Data not recorded in 2003.


Slide48 l.jpg

The slide 25 can be misleading because respondents can give multiple answers.  If we assume 10 people responded and 6 said there was no problem but the other 4 answered in the negative 3 times.

Problem1    Problem2    Problem3    no problem

4                4                    4                6                        total 10 respondents

40%            40%                40%            60%                response per question / 10  total does not add up to 100%

if the number of responses is used as the number respondents (i.e. response per question/18)

22%            22%                22%            33%

These give a different slant on the result and add up to 100% so it is likely that this is the way the calculation in the chart was made.


ad