1 / 24

QUALITY OF EVIDENCE

QUALITY OF EVIDENCE. FRCC Compliance Workshop September/October 2008. Evidence is essential. The Registered Entity has the burden to prove compliance through presentation of quality evidence to a standard or requirement. “Provide Quality Evidence ”. Objective .

chessa
Download Presentation

QUALITY OF EVIDENCE

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. QUALITY OF EVIDENCE FRCC Compliance Workshop September/October 2008

  2. Evidence is essential The Registered Entity has the burden to prove compliance through presentation of quality evidence to a standard or requirement. “Provide Quality Evidence ”

  3. Objective Objective – What is quality evidence? • Define evidence and the types of evidence. • Describe what constitutes quality evidence. • Define appropriate and sufficient evidence. • Provide a better understanding of evidence and its role in compliance monitoring activities.

  4. Definition of Evidence Evidence is: • Data or information on which factual statements can be based. • A collection of relevant and sufficient information offered to verify a presented fact. • Obtained through examining and analyzing records, observing events, or conducting interviews.

  5. Presenting Your Evidence • Auditors gather evidence to draw conclusions on findings in a logical method based upon factual evidence. • Facts are to be valid and relevant for the point needed to be proven. • A fact is something that can be objectively verified, is real and does exists. • Facts must be verifiable. • A verifiable fact is one whose data or information can be substantiated by comparison, investigation or confirmation. • Entities should consider presenting and submitting their evidence in the same manner.

  6. Types of Evidence Evidence can be provided as: • Physical evidence • Direct observations • Corroborating evidence • Corroborating evidence is evidence that tends to support a proposition that is already supported by some evidence. (Wikipedia 9/11/2008) • To strengthen or support with other evidence; make more certain (the Free dictionary 9/11/2008)

  7. Types of Evidence Physical Evidence is direct inspection of activities of people, property, or events: • Some examples include: • Schedules • Charts • Maps • Graphs • One-lines • Manuals • Procedures and Policies • Plans

  8. Types of Evidence Documentation: • Physical forms of evidence which already exists • Can be both external or internal • Some examples include: • Letters • Contracts • Assessments • Agreements • Spreadsheets • Database Extracts • E-mails • Studies

  9. Types of Evidence Direct Observations: • What an auditor sees during your review. • Strengthened through corroboration with other evidence. • By itself is almost always unreliable. • Is achieved by monitoring of personnel, facilities, equipment, and tools.

  10. Types of Evidence Corroborating evidence is: • Additional information from interviews, logs, etc provides clarity to make the appropriate compliance decision. • Used to establish the appropriateness of evidence. • Tests the reliability of evidence and provides confirmation. • Some examples include: • Reliability Coordinator or Neighboring questionnaires • Voice recordings • Logs • Interviews

  11. An Example of Corroborating Evidence The auditor requests evidence that a Registered Entity has followed all directives from the Reliability Coordinator • The Registered Entity states it has not received any directives: • Response is neither sufficient, nor adequate • Testimonial • Low quality • May not withstand scrutiny Corroborating evidence could be an e-mail from the Reliability Coordinator stating the Reliability Coordinator did: • Not send any directives to the Registered Entity during the specified time frame, or • Did send a directive and it was followed

  12. Quality Evidence Quality evidence is evaluated by its: • Appropriateness • relevant, valid and reliable to support findings. • Sufficiency • enough information to lead another person to the same conclusion. • Quality • high quality evidence to be used as proof of findings.

  13. Appropriateness of Evidence Quality evidence must be appropriate: • Relevant to the reliability standard requirements: • Is there enough to persuade the auditor his findings are reasonable? • Is it logically related to the requirement? • Valid as a document: • Is the evidence based upon sound reasoning and accurate information? • Reliable information: • Can the evidence be substantiated when tested?

  14. Example of Appropriate Evidence An agreement is offered as evidence. The auditor will seek to determine: • Is this the original document? • Does the agreement contain the Registered Entity letterhead, approval signature and date,revision date, etc? • Does the agreement address the subject of the requirement? • Is this version the current version?

  15. Sufficient Evidence Quality evidence must be sufficient: • Is there enough evidence to lead an auditor to the same findings that you have reached. • Strong evidence may be better than a large volume of weak evidence.

  16. Example of Sufficient Evidence • The auditor is provided three documents to determine if a procedure exists and is coordinated with neighboring entities: • A procedure • E-mails • Previous version of procedure • The above evidence is sufficient as: • A valid procedure is provided which addresses all sub-requirements even if they do not apply to the audited entity. • E-mails provide evidence the procedure was sent, received and confirmed as coordinated. • Previous version of procedure shows the procedure existed for the audit period.

  17. Quality Evidence • The evidence must be of the highest quality to be used as proof of findings. • Quality evidence should contain: • Documents title • Purpose • Date approved • Revision level • Effective date • Authorizing signatures

  18. Testimonial Evidence Written or oral statements in response to inquiries or interviews. • Examples are: • Reliability Coordinator or Neighboring questionnaires • Interviews • Letters sent for confirmation

  19. Computer Processed Evidence Computer evidence can be provided if: • The evidence can be determined to be sufficient and appropriate. • Is the information considered quality evidence? • Can the evidence be validated from electronic logs to confirm creation? • Some examples include: • Printouts (titled, dated) • Screen shots • Logs (titled, dated) • Study results (titled, dated)

  20. Professional Judgment All evidence reviewed requires some level of professional judgment. “Professional judgment of the auditors is based on their: • Knowledge • Skills • Experiences • Good Faith Application • Integrity

  21. Final Thoughts Some things to remember: • Who you are trying to convince, the auditors. • Compliance is based upon the evidence provided. • Would you objectively make a determination of compliance or possible violation? 

  22. Conclusion At the end of the review, enough quality evidence must have been provided to: Soundly substantiate the teams findings and Support your expected findings

  23. “Provide Quality Evidence ”

  24. Questions ?

More Related