Discussant’s Comments By

1. Auditing Systems Development: Constructing the Meaning of “Systematic and Rational" in the Context of Legacy Code Migration for Vendor Incentives Discussant’s Comments By David Bateman, Sobey School of Business, Saint Mary’s University, Halifax, N.S. Canada

2. My Approach to this role • My focus for this review was to seek the answer to the question below. Were the case authors convincing enough for me to consider recommending the use of this case in my university’s (SMU’s) accounting program?

3. What I Plan to talk about • My answer first – the heck with suspense • Justification for my position- Why? • What I liked about the case • What I disliked • How I think the case might be improved • Summary

4. The Answer • Would I consider recommending the use of this case in our university’s (SMU’s) accounting program?YESIt would be a great learning experience for our students (with a few caveats)

5. Why ? • Consistent with my philosophy of teaching which has evolved over the last 25 years, namely, that students learn more and retain more through active participation in their own learning environment. • I have found that student learn more when challenged and engaged in unstructured learning activities/opportunities.

6. “From confusion comes learning” • I picked this quote up from somewhere and I have tried to find its origins without success. I tell my students that it is my job as an educator to challenge/confuse them by asking a series of often simple but hard to answer questions/problems that challenge their real understanding of an underlying concept. And that it is their job to resolve the confusion - through research, practice, investigative questions and most important of all through learning from theirs and others in the class mistakes. They tend to look at me in a very perplexed manner and some will challenge me with the statement that its my job to teach them the right stuff i.e. the answers.

7. I like unstructured learning activities • This audit simulation case is a great example of an unstructured learning activity. • Students are place into a realistic situational role (IT Auditors) with limited information and are forced to deal with the realities and ambiguities of an imperfect world. Linkage to Sarbenes Oxley issues adds to the relevance of the case as well as providing more challenge to resolve the ambiguities of law and regulation. • Students are challenged/confused and their efforts rise to the challenge will determine their real take-aways from the activity i.e. better understanding with longer retention and expandibility.(ability to apply to future learning or professional practice situations)

8. Case likes • Unstructured learning environment • Incremental / Inferential mental model building is necessary on the part of the student using information acquired from multiple sources, (limited case write up primarily in a simulated dialogue mode, limited investigative questioning of instructor (in class or via discussion forums, business press articles, web resources, prior learning (auditing, AIS & query capabilities), interpreting results of data analysis, and group collaborations • its not easy and students have to think and deal with imperfect information not the least of which is the critical requirement to properly shape the problem.

9. Likes cont’d • Clear well specified learning objectives directly linked to student activities helps provides students with a path to follow however how to accomplish it is left to the student to evolve. • Readiness pretest- the requirement to research/read the cases supporting documentation and the subsequent revealing readiness pretest of 5 m/c questions should alert the students that this case is different from other cases they may have taken in their academic careers. Results are likely to reveals weaknesses in their prerequisite understanding of critical concepts (Wake up Call and Heads Up that they have to digest and translate the significance of the materials they are asked to read)

10. Likes cont’d • Evolving case development- case and teaching notes shows the benefit of feedback - lessons learned from use -- with comments from students, instructors and the authors using the case (testimonials are powerful influencers) • Comprehensive package - case justification, implementation guide and teaching notes, marking guides are well research and extremely comprehensive. Supporting materials readings are from well recognized business press sources (adds credibility to the students) • Digital format feasible – facilitates dissemination to students and their easy of use. Care must be taken to ensure that web links remain active (live)

11. Likes cont’d • Answers Questions- Possible adopters questions such as those below are well answered • Prerequisite Knowledge requirements- audit & AIS • Prerequisite specific data query skills (Critical) • Total estimated time for students 8-15 hrs (probably low as (2-3 hours just to read materials provide) • Virtues of single assignment vs. multi-part assignment • The benefits of group collaboration in unstructured problem domain – learning from inferior solutions and peers is reinforced • Alternative software tools can be used – audit software(ACL) and IDEA or database query tools (MS Access)

12. Likes cont’d • As a potential instructor the authors have made a strong sales pitch as to the case’s pedagogical benefits and have identified important cognitive take aways for the students. (sharpens critical thinking which will be essential to their future professional careers) • I am sold • Now comes my “cognitive dissonance”-- is the “no pain no gain” approach the best for my students?

13. Dislikes • Learning vs Frustration – unrealistic unstructured learning can fail miserably. (With available student and faculty time being a real constraint)There is a fine line here between stretching students to learn more and frustrating them into submission/failure. Keys here is knowing your student’s (backgrounds/capabilities) and providing effective communication paths for guidance and clarification (not the “right” answers)

14. Dislikes • Audit of Systems Development • Audit Migration materials sufficient for students to come to reasonable assessments and conclusions. • Audit of systems development (generally) • Materials too sketchy and incomplete to really give students a positive learning experience - Moved from XP to something else (not really clear) • Logs of stage involvement not conclusive of problems.

15. Dislikes cont’d • Not enough company background info provided about the company Organfood , industry info, historical financial info , prior audits, CFO’s desire to be Sox compliant for possible sale of business. No real basis to assess materiality of findings which is outside the scope of the auditing assignment’s objectives but a real concern if errors detected might lead to financial statement restatement/fraud.

16. Dislikes cont’d • No real comment on student success level in the assignment except to say that most of the projects were acceptable. • Reusabilitymay be limited from year to year as students pass on info on solutions. Data could be changed but underlying problem more difficult to change. • Will students be motivated to perform at the level necessary for successful completion – no game/fun factor – turn it into a competition.

17. Opportunities for improvement • Provide more contextual info to students in the case • Organfood info • Historical financial summary (materiality issues) • Audit history • CFO’s involvement & desire for Sox compliance • Industry Info Provide more information on the grocery/food industry in the case (1-3 paragraph industry synopsis)

18. Opportunities for improvement 2. Auditing of systems development (generally) either provide more information • Extend dialogue in case with IT personal • Create & Provide IS dept self assessment reviews • Introduce internal audit assessment of IT development practices • Better reference material on XP and its impact on quality development OR Drop it from the audit objectives within the case

19. Opportunities for improvement 3. Clarify the role of the student relative to the case characters - Change the roles of Alexie to be the IS auditor and Betani to be his/her IS Audit Manager who is very busy. • Ask the students to take on the role of Alexie in the assignment with the instructor taking on the role of Betani who would be available for limited guidance and ultimate evaluation of the quality of the work performed) Create an additional information sheet for instructors to use to provide resources to inquisitive students

20. Opportunities for improvement 4. Design intermediate process/feedback loops to help reduce failure rates on the case. Allow each group 1-2 planning sessions with their IT Audit Manager to ask specific questions on their work assignment. The instructor/manager would provide enough information to refocus and re-energize groups whose frustration levels were getting too high. Schedule these early in the project schedule (fixed dates) so that students are forced to get started earlier. Online web resources, designated chat rooms for each group with coaching session time , etc would facilitate this activity as well.

21. Opportunities for Improvement 5. Attempt to increase student motivation to accept the challenge of an unstructured learning activity. 1. Link to job aspirations as IS auditor – long term 2. Make IT FUN – short term motivator- Consider creating a more competitive aspect to the case by inviting Professional IT auditors to sit as a judges panel for group presentations and perhaps sponsor a winning team prize of (dinner out and bragging rights/plaque)

22. Opportunities for Improvements 6. Learning Reflections – while case debriefing is essential and viable in a class setting because the projects were of a group nature it would be useful to have each student write a reflective learning journal on what they learned out of the activity After the class debriefing and how they think it will be useful to them. (1-2 page max). This process would hopeful require each student to put the pieces together in such a way as to demonstrate what they learned from the entire learning process.

23. Summary comments • Positives outweigh negatives by far. • Ensuring student have command of the prerequisite knowledge and skills will be essential such that they have a reasonable chance for success • Provide more formalized intermediate help (IT Audit planning sessions) - reduce/manage frustrations • Debriefing so as to learn from mistakes will be an important closure activity – learning journals extend this • Great contribution to available educational tools – simulation case which promotes the development professional judgment and the integration of information technology tools into the professional accountants tool chest.