Through the Looking Glass - How to do a Post Project Review

1. Through the Looking Glass - How to do a Post Project Review Barbara Purchia February 20, 2001 Rational Software bpurchia@rational.com

2. What is a Post Project Review? When to do it How to do it Post Project Review Meeting Real life problems What have we found so far Summary Overview Agenda

3. What Are They? A Rose by Any Other Name • Post Project Review (PPR) • Project Retrospective • Project Legacy • Project Summary • Post Mortem • Post Partum

4. What Are They? What is a Post Project Review? • A summary of project development, start to finish • A brief description and analysis of • Project activities • Results • Lessons learned • Recommendations • An improvement tool • NOT a record of failures

5. What Are They? Why Should We Do a Post Project Review? • Learn from project experiences • Retain knowledge for future projects • Understand how your project was done • Identify areas to improve • Understand successes • Provide team closure

6. ??? When to Do Them When Do I Do One? • For completed, terminated, or cancelled projects • No more than 30 days after project end • NOT before project release • When all critical participants can attend • During the project • After a major project milestone • At designated schedule points • When things are not going well

7. How To Do Them What’s Involved? • Collect the information • Technical • Managerial • Metrics • Schedules • Anything kept during the project • Manager or project logs • Emails • ...

8. How To Do Them What’s the Deliverable? • Concise documented information summary • Development activities • Project metrics • Estimates and actuals • Technical and managerial lessons learned • Recommendations for future projects • Focused only on the project and the processes

9. How To Do Them How Do I Start? • Hold a meeting including • ALL participating functions • Project management team • Affected functions AND/OR • Request input via email • Remote sites • Managerial lessons learned • Filtered to be unattributable

10. How To Do Them What Is a PPR Meeting? • A tool for reviewing and analyzing the project • Promote discussion on improvements • Involve a critical mass of those directly involved • Promote discussion of all project areas • Deliverables • Team dynamics • Processes • Provide input for the PPR document • Provide closure for the project team

11. How To Do Them What Doesn’t the PPR Meeting Do? • Assign blame to any individual or group • Provide an exhaustive study of the project • Provide solutions to all issues • Shoot the messenger

12. How To Do Them How to Hold a PPR Meeting • Use a facilitator • Set up a firm time • Use an agenda • Ensure Senior Management support/sponsorship • Keep the focus on the process not the people!!!

13. How To Do Them How to Plan for the PPR Meeting • Pick the meeting date • Identify attendees and ensure availability • Book the room in advance • Prepare an agenda • If using a questionnaire, review the questions and tailor it for the project • Send meeting announcement

14. Senior Management Ensure that a PPR is done PPR Sponsor Call the PPR meeting Ensure that information gathered Produce and post the PPR document Use the data Project team members Attend the meeting and participate Use the data Facilitator Non-project member Help plan and organize meeting Ensure meeting runs smoothly Administrator Capture the issues and recommendations How To Do Them Roles and Responsibilities

15. Managers Team Members Project Program Product QA/QE Development Documentation International Beta Service UI Manufacturing ... How To Do Them PPR Meeting Attendees All Project Participants

16. How To Do Them Facilitator Preparation • For each group, work with the sponsor to • Understand the environment • Understand the culture • Understand the people • Understand any previous PPR activities

17. Milestone 75 MPH Start How To Do Them Sponsor Preparation • Produce a brief project refresher • Time line of key milestones and/or significant activities • Review the PPR questionnaire with the Facilitator • Identify anticipated issue categories with the Facilitator (optional) • Review any previous PPRs • Documents • Improvement activities

18. How To Do Them Participant Preparation • Think about the project • Review the questionnaire • Review meeting minutes • Review emails, documentation, ..

19. How To Do Them Room Preparation • Done by the facilitator • Ensure that meeting aids are in the room • Flip charts • Markers • Post-its • Tape or tacks • Pens • Bring candy, donuts, or have lunch available after the meeting (optional)

20. The PRR Meeting PPR Meeting Logistics • Split into two parts • Looking backwards • Looking ahead

21. The PRR Meeting Example Two Hour Meeting Agenda • Introduction and Groundrules (5 minutes) • Project refresher (10-15 minutes) • What Went Well and Issues Gathering (40 minutes) • Prioritization (10 minutes) • Brainstorm solutions for top 3 issues (40 minutes) • Meeting Closure (5-10 minutes)

22. NOT! Off with her head!!! Off with his head!!! The PRR Meeting Groundrules • Be polite and gracious • Focus on problems and process • No finger pointing • No assigning blame • No “You’s” • Let each person have a turn • Avoid defensiveness • Respect everyone’s perspective

23. The PRR Meeting Project Refresher Example

24. The PRR Meeting What Went Well and Issues Gathering • Split into separate activities • Use Post-its • What Went Well • Use flip chart sheet(s) • Put “What Went Well” Header on flip chart sheet • Write down positives on Post-its • Read in round robin fashion until done • Facilitator puts on flip chart sheet • Group any duplicates Barb’s Lessons Learned

25. The PRR Meeting What Went Well and Issues Gathering (Continued) • Issues • Use flip chart sheets • Write potential issue categories on top of blank sheets • Have several blank sheets • Tape to wall • Write down issues on Post-its • Read in round robin fashion until done • Facilitator puts on flip chart sheet • Group any duplicates Barb’s Lessons Learned

26. The PRR Meeting Post-It Rules • One item per Post-It • Write as many Post-Its as you want • Write enough so that each idea stands on its own • Read 1 Post-It per person until we are done • Identify duplicates and pass them to the facilitator • One item per Post-It

27. The PRR Meeting Issue Prioritization • Everyone checks issues • Ensure grouped correctly • Ensure duplicates collected correctly • Everyone gets 3 votes • Used however person feels • Total all issues • Identify top 3 issues • Check in with group to see if agree Last minute changes to requirements (8) Finish critical components sooner (4) More Beta Test sites (4)

28. The PRR Meeting Brainstorm Solutions for Top 3 IssuesTechnique 1 • Use flip chart sheets • Identify the issue and page number on top right • Write issue on top of flip chart sheet • Write down all solutions • Get clarification as needed • Check in to see if captured correctly • Spend 10 - 15 minutes per issue • Check in if energy low

29. The PRR Meeting Brainstorm Solutions for Top 3 IssuesTechnique 2 • Identify volunteer facilitators for the top issues • Include similar items • Break up into groups • Volunteer facilitators should use flip chart sheets • Identify the issue and page number on top right • Write issue on top of flip chart sheet • Write down all solutions • Get clarification as needed • Check in to see if captured correctly

30. The PRR Meeting Meeting Closure • Thank people for attending • Answer any questions • Re-iterate the next steps • Thank people again THANKS!!

31. The PRR Meeting Facilitator Tips • Protect each speaker • Pay attention to the speaker • Keep the meeting positive • Don’t filter the information • Let people finish their thoughts • Help people continue the discussion, not sidebars • Check-in with group • Ensure that senior or vocal person does not shut down meeting • Make it fun

32. ??? The PRR Meeting Candidates for Questionnaire and Issues • Project objectives • Project requirements • Project planning, tracking, and risk management • Localization • Training, Knowledge, Tools • Functional specifications • Development strategy/process • Lessons learned • Communications

33. The PRR Meeting Sample Questionnaire • Project Objectives • What were the objectives? Were the objectives clearly stated? • Were the objectives of the project met? • Did everyone work from the same set of objectives? • Project Requirements • Was there a Requirements document? • Were the requirements known, understood and adhered to? • Did the requirements change? If yes, why and how often? • Project Planning and Risk Management • Was there a project plan? Was it maintained throughout the project? • At the outset, was the schedule realistic? • Did the project plan change? If so, why and how often?

34. After the PRR Meeting What Does the Sponsor Do After the PPR? • Collect metrics • Estimated and actual • Staffing • Milestones and schedules • Project cost • Code size • Quality (e.g. defects) • Any other useful metric information • Collect managerial lessons learned • Write the PPR document • Use an administrator to help • Have team review document • Post or distribute the PPR document

35. After the PRR Meeting What Else Does the Sponsor Do After the PPR? • Follow-up on action items • Use information in the next or similar projects • Communicate improvements

36. After the PRR Meeting Outline for PPR Document • Introduction • Results vs Plan • Status • Staffing • Milestones • Cost • Quality • Lessons Learned • What went well • What didn’t go well • Technical lessons • Managerial lessons • Recommendations for future projects

37. Problems Real Life Problems • The data is not used • The document is not produced • People feel that nothing will come from the meeting • The document takes forever to complete • Meeting gets out of control • Key people don’t provide input

38. Discoveries What Have We Found • Consistent issue areas • Issues moving further downstream in the process • Requirements and specifications • Communications • Quality • Project planning and tracking • Process • People expect follow-through • A focus on “management” instead of what “I” can do • Department and organizational project similarities

39. Discoveries How Have PPRs Helped Us? • Used the schedule information to plan the next project • Used actual cost information for project prediction • Organization metrics • Time between phases • Time to market • Bottleneck areas • Opportunities to share solutions

40. Valuable tool for capturing and learning from past projects Valuable tool for predicting future projects Good mechanism for project closure Great vehicle for tracking organizational information Good way for the facilitator to learn about the team Summary Try it. You’ll like it!!

41. Any Questions??