HIGH IMPACT RESEARCH AUDIT NO. 1/2012

HIGH IMPACT RESEARCH AUDIT NO. 1/2012 The Audit Findings: A Summary Audit Panel Closing Meeting, HIR Audit No.1/2012 Auditorium API 4 June 2012

HIR Audit Plan

Thank You…… • Auditors for their commitment, preseverance and hard work • Auditees – VC, HIR Secretariat (Central & Faculty), • Researchers (PI-Co Researchers- RAs) • Faculty Heads • International & Corporate Relations Office

Aim & Scope of Audit Aim To ensure/check that the management of High Impact Research (HIR) projects conform to UM Quality Management System (QMS) requirements. Scope Management of research projects under the High Impact Research funding from both internal (UM) and Ministry of Higher Education. (Planning-Implementation-Monitoring-Improvement Actions) Period 7- 25 May 2012

Methods • Interviews • Observations • Documentation

Audit Location: Secretariat of HIR, VC Office & Recipient Faculties - Secretariat of PTj, Principal Investigators.

Instrument 1: Nothing (no documents, no plans, no evidence)present 2. Documents and plans available with no clear evidence of implementation 3. Documents and plans available with some evidence of implementation 4. Documents and plans available with clear evidence of implementation. 5.Excellent: example of good practices

Reporting • Commendations- best practices worthy of praise • Affirmations- good practices that should be maintained/enhanced. • Recommendations- Areas where There are concerns and need for Improvements

These findings were based on a sample of 63 projects from the 5 faculties and the HIR Secretariat. • The report today would not indicate any specific projects which contribute to the verdicts • The written detailed report would be handed in to the secretariat with the specific findings. • The presentation of findings would follow the P-D-C- A stages

Commendation 1: Overall vision The audit panel commends UM’s management in particular the avantgarde leadership of the Vice Chancellor, stewardship by the HIR Committee, commitment of the HIR secretariats and passion of the researchers who rose to the challenge in making this project a success and thus giving a new momentum to UM’s research.

Interview with the VC-24 May 2012 How long and how much time did you spend conceptualizing the HIR Project? Frankly speaking….there..has not been much thinking……it all comes from the heart What do you mean..by coming from the heart? I mean…its all based on INTUITION & “GUT FEELINGS”

Planning Stage Process of Application- Vetting- Approval Due to the instantaneous birth of the HIR project, the initial process of application-vetting and approval of research projects were not systematic and efficient. However, through time as the project developed as seen in the later cycles, there is seen to be great improvement in the process especially in the forms & guidelines, appointment and utilization of assessors recommendations. Affirmation 1 The panel affirms the various improvement efforts by the HIR secretariat and later by the various faculty HIR secretariat & in the process of research application- vetting & approval.

Planning Stage Appointment of Assessors It is noted that the assessors appointed were suggested by the Principal Investigators. In many instances they were mainly internal (national) assessors and there were also instances where the assessors form part of the research team. Whilst this is understandable at the beginning & it is noted that there were improvements in the later stages, it would be better if there exist a guideline and criteria for the appointment of assessors. Recommendation 1 The panel recommends that the HIR secretariat set specific criteria for assessors and formulate a guideline for appointment- this would ensure appointment of an independent and reliable assessors- those who would able to determine the “impact” and the viability of a project

Planning Stage Recommendation 1 The panel recommends that the HIR secretariat set specific criteria for assessors and formulate a guideline for appointment- this would ensure appointment of an independent and reliable assessors- those who would able to determine the “impact” and the viability of a project. The panel also recommends that the secretariat prepares a database of assessors formed based on suggestions from researchers or other parties.

Planning Stage Record keeping Documentation audit of research files indicated a lot of improvements needed in terms of record keeping. Some of the issues raised include: • forms of various sorts without any signature for verification purposes • duplication of application forms in most files, from which it is understood that revisions were made based on committee’s feedback- often without verification. • Duplication of letters of award • Bursar’s letters of notification to open account make with reference to VC’s letters of award but with incorrect dates Recommendation 2 It is recommended that the record keeping of research files is up-dated. Only the final and verified copies be kept- if there is a policy to keep all then the history noting the significance of each should be available and shown.

Do/Implementation Stage Execution of Research by PI and Team (1) Records show that most of the sample projects have been carried out close to the research plan and making satisfactory progress aligned to the milestones in the proposal. Some projects strong evidences on collaborative work. Commendation 2 The audit panel comments the principal investigator and the research team for their passion, commitment and perseverance in carrying out their research activities in a timely manner as stipulated in their proposal.

Do/Implementation Stage Execution of Research by PI and Team (2) Whilst there appear to be encouraging evidences of research execution, there are also indications of the lack of involvement of some members of the research team and in certain cases it shows that the research is only anchored by the PI and the RAs. Recommendation 3 The audit panel recommends that the principal investigator updates the research group members in the progress reports to ensure that inactive members or might those who leave the group are dropped out and the research team updated.

Do/Implementation Stage Execution of Research by PI and Team (3) Records indicated that in the execution of the research the PIs also travel abroad for various purposes- conferences, visiting icons and collaborators. However, there is no common practice when PIs travel abroad-some goes under cutipenyelidikanand others cutipersidangan- thus the conditions imposed differ. Although HIR requires a report within 2 weeks of return but evidences show that no such reports exist. Affirmation 2 • The panel applaud the HIR secretariat for formulating the requirement and and devising a travel report form UM.C/625?1/HIR/ and UM.C/625?1/HIR-MOHE/ but suggest that the travel arrangements be standardised and monitoring enhanced through the use of the forms.

Do/Implementation Stage Appointment of Research Assistants Appointment of RAs are follows a guideline and standardised application form. Documentation of appointment of RAs for most projects are in order. However, there seems to be discrepancies of the salary of the RAs with same qualifications- this mainly due to two reasons: firstly that the salary of RAs are suggested by the PI and secondly the range of salary of each cohort of RAs is to big. Affirmation 3 The appointment of Ras based on the guidelines should be continued but can be improved by the HIR committee might want to consider standardized salary scheme throughout. There should also be a contract for those RAs sent for training to ensure the ROIs of sending them for training.

Do/Implementation Stage Procurements Most procurements were made according to those stipulated in the proposals and subjected to the necessary procedures. There were also evidences of some which were not done according to the proposal but there were evedences of permission sougth and granted. However, there is no system to monitor amount spend and committed and discrepancies occur between records kept by bursar and actual expenditure. Some Pis keep manual records whilst there are some who does not have any record of expenditure and balance. Recommendation 4 It is recommended that a system similar to e-finance be set up for finance monitoring.

Do/Implementation Stage Research Space & Inventory All equipments bought were tagged and inventoried. There are records of training for installed equipments and a log book for the equipments available. All projects based at the faculties have no space problems, although there are cases where the equipment bought are still placed in the suppliers warehouse because the space allocated at the HIR building is not ready. Affirmation 4 The panel commends the Responsibility Centres and the HIR secretariat for assurance of space needed for research and for adhering to the requirements of traceability to equipments purchased.However, the panel would recommend the university to execute a research space audit to ensure future availability of research space and to avoid situations where space is not ready for equipmnents purchased.

Do/Implementation Stage Interfacing/Communication The panel applaud the HIR secretariat for adhering to the paperless administration by executing communication through e-mels and the web page. However, they must also be vigilant enough to ensure that this form of communication is effective and efficient- there must be consistency between SOPs on the web-page and the practices by Pis, all forms of communications must be, aligned, up-dated and records maintained. Affirmation 5 All information disseminated through e-mails must be concurrently up-dated on the web-page. Any communication done via e-mails must be kept as record.PIs must be make aware that feedbacks are necessary even though requests are made via e-mails.

Do/Implementation Stage Interfacing/Communication The setting-up of the HIR secretariat at the responsibility centre is recent, with the MOHE projects. The two secretariats seemed to know what they need to do although no documents were sighted to indicate all aspects of their roles. There now seems to exist a two track-system one for the internal projects and the other for the MOHE projects. In the case of the internal projects, communication is mainly between the HIR secretariat and the PIs, whilst the HOHE project has the RC’s secretariat in the loop. This often creates some confusion.

Do/Implementation Stage Recommendation 5 The panel recommends that there should be a realignment of management- to create a link between the - HIR Central secretariat – PI - RC secretariat. At the very least, the Rc’s secretariat should be involved in the loop of the communications between the stakeholders mentioned.

Check/Monitoring Monitoring of Output A system for output monitoring is in place- via a periodical review of progress. The progress report is monitored by the committee which appoints a HIR Technical Committee to assess the progress report. However, as evidenced from the progress report form, monitoring is only focused on the publication KPI, not the bright sparks and the academic icon. It is also discovered that the evidences attached with the progress report is not scrutinized- there are cases where the publication attached has nothing to do with the HIR grant, and also cases where the date of publication is before the HIR project being awarded. This is pertinent to ensure the ROIs especially where the funds is exorbitant

Check/Monitoring • It is also obvious that the data for monitoring for all HIR project is not easily available. Recommendation 6 The panel recommends that the progress Review Report form be re-examined and improve to ensure holistic monitoring of KPIs and specifically to include articles published in TIER 1 journals. Recommendation 7 The panel recommends that the HIR secretariat establish a database of KPI achievement.

Act/Corrective Actions taken • It is noted that a group of research has complete its cycle and under extension amongst which there are those whose records on KPIs are not complete .Yet there are no evidences of actions or follow-ups taken.

Good practices • Some faculties and certain projects to be highlighted in the main report,

Thank You.

HIGH IMPACT RESEARCH AUDIT NO. 1/2012