Diversity Data & Metrics: Impact and accountability

1. Diversity Data & Metrics: Impact and accountability Michele Minter Princeton University June 3, 2013

2. What gets measured, gets managed. Peter Drucker

3. What are data and metrics used for? • To answer questions • To monitor activity and trends • To create accountability • To challenge assumptions and change behavior • To manage compliance obligations • To communicate to the community

4. Types of data and metrics • Survey/focus group/interview • Demographic • Benchmarking • Reporting activity • Compliance data

5. Good data and metrics • Capture the right things at the right level of granularity • Create accountability • Identify problems • Incentivize good behavior

6. But… Not everything that can be counted counts, and not everything that counts can be counted. Albert Einstein

7. Goal setting: measuring against what? • Before setting diversity goals, the definition and value of diversity must be clear • Diversity goals may measure: • Desired level and quality of activity • Aspirations to: • Be in the top percentage on a given measure, when compared with peers • Show numerical improvement in representation • Show improvement in measures of climate

8. Princeton’s diversity goal setting • Explains why diversity matters for Princeton • Asks departments to review their data and “diagnose” their challenges • Asks departments to define activities for which they are willing to be accountable • Imposes some priorities set by the central administration • Expects numerical improvement as a result of activity but doesn’t mandate it • Provides many tools and resources to help departments achieve goals • Asks departments to report on progress

9. In our lust for measurement, we frequently measure that which we can rather than that which we wish to measure... and forget that there is a difference. George UdnyYule

10. The challenges of data and metrics • “Garbage in, garbage out” • Data and metrics may not measure what you think they measure • “Apples and oranges” confusions • Correlation vs. causation confusions • Wrong level of granularity smoothes data or masks problems • Changing definitions create confusion • FERPA constraints or statistical invalidity can compromise the ability to measure • Metrics can motivate the wrong behavior • Metrics can become burdensome and bureaucratic

11. Elements of a Good Internal Dashboard • Survey data regarding campus climate and perceptions • Demographic snapshots + appropriate benchmarks • Demographic trend data + appropriate benchmarks • Flow data: hires, departures, candidate pools • Activity highlights • Ability to sort data in multiple ways, including at the institutional level, at the unit level, and by sub-populations

12. Data are just summaries of thousands of stories – tell a few of those stories to help make the data meaningful. Chip and Dan Heath

13. The External Dashboard • Demographic snapshots • Demographic trend data • Activity highlights • Climate snapshots • Plus lots of contextual information, including institutional values, why diversity matters, and voices of individual constituents

14. Things get done only if the data we gather can inform and inspire those in a position to make a difference. Mike Schmoker