Overview of the PIPS Best-Value Process

1. This presentation is being provided for educational purposes onlyPlease refer to the RFP for specific instructionsIf there are any inconsistencies, the RFP and Amendments shall take precedence over this presentation

2. State of Idaho WIM and MDC Projects Overview of the PIPS Best-Value Process Kristen Barlish Arizona State University 07/18/2012

3. Agenda • Best Value Concepts • PIPS/Best Value Process • Phase 1 • Phase 2 • Phase 3 • Examples • Summary

4. Education Meeting WIM Project MDC Project Scope: “…services to carry out a Mobile Data Collection (MDC) solution…” Budget: $1.0M (State FY 2013) & $1.2M (FY 2014) Other considerations: expected benefits, project phases and dependencies • Scope: “…to provide and install a permanent Weigh in Motion Sensor System (WIM)…” • Budget: $420,000.00 • Other considerations: location requirements, permitting, and training

5. PBSRG’s Research Results • Worldwide as a leader in Best-Value Systems • 17 Years • 210 Publications • 550+ Presentations • 900+ Projects • $4.6 Billion Services & Construction • 71 Different Client (Federal, Public, Private) • 98% On-time, On-Budget, Customer Satisfaction • Various Awards (PMI, NIGP, IFMA, IPMA) • Clients: Federal, State, Local, School Districts, Private • Applications: Construction & Design/Engineering, IT Services, Facility Services, Business Services, University Services, Health Insurance, Medical Services, Manufacturing

6. Why Best-Value??? • Performance Issues • Project delays • Cost increases • Poor quality • Lack of accountability • Claims / Disputes • Litigation • Managerial Issues • Do more with less • Spend taxpayers money wisely ERROR

7. Seems Similar, but…. • The PIPS BV process is similar to traditional processes • Factors in Cost • Factors in a Proposal • Factors in Past Performance • Factors in Interviews • However, there are minor tweaks to each on of these factors along with an additional requirement called the Pre-Award or Clarification Period

8. Factors For Success • Fair (state/follow rules) • Open (no prequalification / open to all with experience) • Impartial (minimize evaluator bias) • Transparent (provide documentation and debriefing upon award) • Efficient (minimize efforts)

9. Best-Value Concepts

10. Not “Business As Usual” • Approach is different • Can’t wait until awarded the project to plan • Submittals are your opportunities • 28% Projects are Successful • 46% Projects are Unsuccessful • 26% Projects Completely Failed / Cancelled • 202% Average Cost Overrun • 111% Average Schedule Overrun The Standish Group, Chaos Reports - Collects case information on IT failures. Has studied over 70,000 projects in 15 years (Eveleens and Verhoef, 2010)

11. Example

12. Suggestions • Identify who your best people are (should be done through measurements) • Have your best people sit down in a room and think about this project • Adapt their comments/ideas into your Risk and Value Assessment Plans • Correlate any suggestions/ideas/solutions to documented performance • Keep marketing people away (risk of reformatting, names, words, etc.) • Stick to the page limit (look at the major items first) • Keep it simple and non-technical (so anybody can understand)

13. What does it mean? • “Expert” • “Dominance” • “Vision”

14. Expertise What the Owner Described What the Consultant Specified What the Contractors Installed What the Owner Really Needed FACTS: • Owners are not experts (they have an idea of what they want) • RFP’s are rarely 100% accurate • Experts should know more about the service than the owner • Experts should know what the Owners need (even if the Owner didn’t describe it) • We are looking for an expert to provide us with their vision and expertise 14

15. It Is More Important For The Vendor To Know What To Do Than It Is For Client To Know What The Vendor Should Do.

16. Picture the Happy Client – New Model • System meets the intent • Expectations are met/exceeded • Project meets cost and schedule • Satisfied Approach is to work backwards, how did you get there? This becomes your plan…

17. Assumption: Vendors are Not a Commodity …but how do we know who to select?

18. Which is more dominant? • Which Vendor would you pick for your construction project? • What is your criteria? • How do you justify? • Does everyone agree?

19. Dominant vs. Non-Dominant Non-Dominant • A decision needs to be made • Not everyone can understand • Evaluator must have technical expertise • Not everyone will agree • Difficult to justify a decision Dominant • Anybody can understand • Everybody should see the difference • No decision needs to be made (“no-brainer”) • Very easy to justify purchase

20. Vision Highly Experienced Can See End to Beginning Every vendor has “Vision” Uses information & logic to increase vision Project Risk Risk Risk Must be minimized from the beginning 20

21. PIPS Best-Value Process

22. PHASE 1 PHASE 2 PHASE 3 Identification of Potential Best-Value Pre Planning and Quality Control Award | Measurement & Documentation PIPS Best-Value Process

23. Request For Proposal (RFP) • Contents: • Current / Existing conditions • Desired outcomes / Objectives • Proposal requirements and selection criteria RFP

24. What If You Have a Question? • All proposers must meet the minimum requirements / objectives • If a proposer does not understand, they must ask a question prior to submitting a proposal. • If a proposer cannot meet a requirement, they must inform the State prior to proposing • Identify which requirement cannot be met • Provide recommendations or alternatives that the State may consider • The State will then issue an addendum with their response 24

25. Your Proposal Must Minimize Surprises Delivering something that doesn’t work Delivering something that isn’t what the client is expecting Delivering something that isn’t what the client needed Changes that add more cost Changes that add more time 25

26. PHASE 1 PHASE 2 PHASE 3 Identification of Potential Best-Value Pre Planning and Quality Control Award | Measurement & Documentation PIPS Best-Value Process

27. Proposal Package(Attachments) TECHNICAL PROPOSAL ENVELOPE • Attachment A – Proposal Cover Sheet • Attachment B – Proposal Form • Attachment C– Risk Assessment Plan • Attachment D – Value Assessment Plan • Attachment E– Reference List • Attachment F – Survey Questionnaires • Attachment G– Past Performance Information Scores • Idaho Signature Page • Affidavit - Concerning Alcohol and Drug-Free Workplace • Affidavit - Concerning Illegal Aliens COST PROPOSAL ENVELOPE • Attachment H– Cost Proposal Form

28. WIM – Criteria and Weights

29. MDC – Criteria and Weights

30. Evaluation Committee • Will be used to evaluate specific portions of the Proposal • Evaluators will not be provided with the names of any Proposers, product names, cost, or any additional information • Evaluators will independently review and score the items comparatively to one another • Objective of the scoring is to not make a decision (looking for “dominant” differential) • Evaluations will be scored on a 1/5/10 scale • “10” = Dominantly higher value than the average (clearly shows differential) • “5” = About average (insufficient information to make a clear decision) • “1” = Dominantly below the average (clearly shows differential)

31. Critical Formatting Requirements • Must use templates (“Attachments”) • NOT allowed to alter the templates (change font size, font type, boarder widths, add color, add images/graphics, re-format, re-create, etc.) • Risk Assessment and Value Assessment Plans (Attachments C and D) MUST NOT contain any names that can be used to identify who Proposer is (such as company names, personnel names, project names, or product names) and not identify the Proposer’s cost or fee for this project. • Proposal is limited to • 1 Page = Controllable Risk Assessment Plan • 1 Page = Non Controllable Risk Assessment Plan • 1 Page = Value Added Plan • Single side of page only!

32. Risk Assessment Plan – How To • Identify and prioritize all major risks (that the Proposer sees that are unique and applicable to this project) that may impact a successful delivery of the project. • Risk = not completing on time, not finished within budget, generating change orders, or sources of dissatisfaction to the owner. • The risk should be described in non-technical terms and should contain enough information to understand why the risk is a valid risk. Proposer must also explain how it will avoid or minimize the risks from occurring. • Solutions must be nontechnical, logical, easily understood, or contain verifiable performance information. • All risks/solutions must be included in the cost proposal

33. Risk Assessment Plan • Controllable Risk Assessment: risks, activities, or tasks that are controllable by Proposer, or by entities/individuals that are contracted to by Proposer. This includes things that are part of the technical scope of what Proposer is being hired to do. Project risk that other vendors have due to lack of experience and expertise • Non-Controllable Risk Assessment: risks, activities, or tasks that are not controllable by Proposer. This may include risks that are controlled by Agency, Agency’s representatives, or completely uncontrollable. Although these risks may not be controlled by Proposer, Proposer must identify a strategy that can be followed or used to mitigate these risks. • Expert’s greatest risk are things they don’t control

34. To Be Considered “Dominant” Solutions Must Be: • Clear and concise • Non-technical descriptions • Logical • Measurements that document time, quality, and cost • Document performance results

35. Risk Assessment Example Controllable Risk • VENDOR 1 • RISK: Noise from our demolition may result in student/staff complaints (since we will be doing demo in an in-operational library during finals week). • SOLUTION: We will work with the user to minimize the impact of noise from demolition. • VENDOR 2 • RISK: Noise from our demolition may result in student/staff complaints (since we will be doing demo in an in-operational library during finals week). • SOLUTION: To minimize this risk, we have planned to demolition during off hours and weekends. We will also install rubber sheets on the floors to diminish noise and vibrations. Both solutions can be performed within your budget.

36. Risk Assessment Example Non-Controllable Risk • VENDOR 1 • RISK: The local water company must have the water turned on by June in order for us to properly water the newly installed recreational fields (or the grass will die). • SOLUTION: We will coordinate and plan our schedule with the water company as soon as the award is made to make sure that we get water to the site to irrigate the fields. • VENDOR 2 • RISK: The local water company must have the water turned on by June in order for us to water the newly installed fields (or the grass will die). On past projects, the water company has failed to meet the schedule 90% of the time. • SOLUTION: To minimize this risk, we will coordinate our schedule with the water company as soon as we are awarded the project. If they fail to meet our schedule, we will setup and connect temporary waterlines to the nearby fire hydrants and we will also have water trucks on-site to irrigate the fields.

37. Risk Assessment Example Non-Controllable Risk • VENDOR 1 • RISK: Since this is a revenue based contract, the greatest risk is that the student population on campus does not grow as expected (which will impact our financial projections). • SOLUTION: We will implement a plan based on our past expertise to stimulate student growth and enhance our marketing plan to the students (which we have done at several accounts). • VENDOR 2 • RISK: Since this is a revenue based contract, the greatest risk is that the student population on campus does not grow as expected (which will impact our financial projections). • SOLUTION: We expect that we can still increase revenues even if the student population does not increase as predicted. At two similar accounts, we increased growth by more than 10% in two years, despite flattening enrollment, through concept and menu updates, increasing convenience (speed of service), convenience store additions, and expanding points of sale.

38. Risk Assessment Example Controllable Risk • VENDOR 1 • RISK 1: The existing ballasts and lamps are hazardous • SOLUTION 1: We will dispose of them properly • RISK 2: A 2950 lamp is not as bright as a 3150 • SOLUTION 2: We will use the 3150 lamp • RISK 3: Once power is turned off, it will be hard to see in between the stacks • SOLUTION 3: All employees will have headlamps • VENDOR 2 • RISK 1: There are sprinkler heads located 12” above the light fixtures we will be retrofitting. If a head gets hit with a ladder, conduit, or lamp, it is possible that the sprinkler could go off and damage the books on the shelves. • SOLUTION 1: To mitigate this risk, we will temporarily install sprinkler head guards around the sprinklers in the areas we will be working in.

39. Risk Assessment Example Controllable Risk • RISK: Major risk items typically associated with transit implementations revolve around change management and business process impact. New technology implementations create change for the users. Change often causes issues with technology adoption. Requirements and scope creep also creates challenges. Systems may have thought a certain technology or component was incorporated in the RFP and/or needs assessment process that is not included in the actual scope of work or contract. Communication is also an area that can be a challenge. • SOLUTION: A clearly defined scope of work and communication of the scope at the beginning of the project minimizes scope creep. If there is a discrepancy, scope or requirements can be discussed early on in the process versus at the end of the process. Communication is the key to successful implementations. Change management and business process re-engineering for organizations can be minimized at the technology and management levels. Management can get early buy-in at the “grass roots” level and include them in the technology planning process. The Team focuses on providing very configurable and flexible tools to minimize process re-engineering tasks. The Team focuses on automating existing business processes and providing additional tools to improve those processes that need to be improved such as data management….

40. Technical Information vs. Verifiable Performance Information RISK: A poor roofing system can result in roof leaks, which may inconvenience building occupants, increase complaints, increase maintenance, damage building contents, and be a source of mold issues. Vendor A Solution: • To minimize this risk, we are proposing a thermally-welded roofing system that has a tensile strength of 2,130 PSI, elongation of 300%, tear strength of 312lbs, has been tested for 10,000, and has a cold brittleness of -30°C. Vendor B Solution: • To minimize this risk, our proposed roofing system has been installed on over 400 roofs and has had an average roof age of 18 years, in which 99% of the roofs don’t leak and 100% of the end clients are satisfied.

41. Things to Avoid • Marketing data: • Our company is known worldwide as a leader in… • We will use our long history to… • We will use state-of-the-art process to… • Technical data: • The system we propose has 200% elongation and 600psi tensile strength. • The product will pass the ASTM-568a test. • The system can process up to 24GB per second • Transferring risk back to client: • We will work with the owner to resolve issues… • We will work with the user to establish the proper tests/procedures to ensure that data is transferred properly… • We will have team meetings / partnering meetings with the owner…

42. Avoid General Statements • Clear communication between all parties is essential • Project planning is vital to keeping the project on schedule and on budget. • Meetings are essential to gathering information to define and implement the solution. • Tracking tools will aid in keeping the project on schedule. • It is important to have buy-in from key stake holders. • An experienced project team will increase the chances of success • Complex systems may not be embraced by the end users

43. Value Assessment Plan – How To • Opportunity for the Proposer to identify any value added options or ideas that may benefit the State/Agency. • May also be referred to as additional or optional system functionality or services. This may include ideas or suggestions on alternatives in implementation strategies, timelines, project scope, modules, methodologies, or financing. • All value added ideas must be logical and/or based on verifiable performance metrics. • All value added options must be related to a cost or schedule impact. • Value added ideas must NOT be included in the cost proposal. Prior to award, the State will determine if the value added items will be accepted or rejected.

44. Example: Value Added Items • Reroofing this building will not stop all water leaks. The majority of the leaks are caused by cracks in the parapet walls, broken/missing glass, and poor caulking. For an additional $10K and 3 weeks in schedule we can replace and repair all of these items. 44

45. Example: Value Added Items • We may be able to increase revenues to the University by an additional 5% per year, if we are allowed to install and operate our own vending machines throughout campus.

46. Example: Value Added Items • Instead of purchasing “Named Licenses”, the Agency may want to consider purchasing “Concurrent Licenses”. In a “Named Licensing” model, the software designates a license per user and only that particular named user can use/access the license. If that named user is in meetings, on vacation, or not using the system, the license is not utilized. In a “Concurrent Licensing” model, the server keeps track of the total number of licenses and loans the licenses to users as they log in. If a user is inactive, the server releases the license and allocates the license to the next user. The advantage is that the Agency is not required to purchase licenses that are not being used, which can result in approximately 25% savings in cost.

47. Remember… • Be clear and concise • Non-technical descriptions / solutions • Logical • Measurements that document time, quality, and cost • Document performance results / verifiable results

48. Step 1 ENTITY Prepare Reference List Vendor Step 2 Prepare and Send Survey Questionnaires to Past Clients Step 3 Collect/Receive Completed Surveys Step 4 Enter data into Reference List Step 5 Package all material(Reference List and Surveys) and Submit Past Performance Information • PPI will be collected on the following Entities: • The Firm • Project Manager (Individual) • WIM additional requirements • Each Entity must prepare and submit a Reference List (Attachment E), Survey Questionnaire (Attachment F), and Past Performance Information Scores (Attachment G)

49. Reference List (Attachment E) • For each Entity, Proposer must prepare and submit a list of clients that will evaluate Entity's performance. • The maximum number of references that can be submitted is 5 for each Entity. • The past projects must be: • Installed and operational • Related or similar projects • The reference list must contain different: • Projects • Clients • The client must complete the survey • The State of Idaho or its employees cannot be used • Note: Each Entity can use the same references provided that they were used or applied on that particular project.

50. Survey Questionnaire (Attachment F) • For each Entity, Proposer must prepare, send out, and collect survey questionnaires to each individual listed on the Reference List. • Proposer must modify so that the surveys are returned back to the Proposer. • All returned surveys MUST be evaluated AND signed by the client. • Returned surveys must be packaged together and submitted with Proposer's proposal (Proposer should keep a copy of all returned surveys for Proposer's records).