1 / 41

Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management

Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management. Prof. Brian Boyd W. P. Carey School Arizona State University http://www.briankboyd.com. Roadmap. Background State of the art Consequences of measurement problems Application. Background.

judith-vega
Download Presentation

Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Current Issues in Individual, Group, and Organizational Level Measurement :Strategic Management Prof. Brian Boyd W. P. Carey School Arizona State University http://www.briankboyd.com

  2. Roadmap • Background • State of the art • Consequences of measurement problems • Application

  3. Background • Most of my papers have a strong measurement component, e.g., • SMJ 90: Environmental characteristics • JMS 91: Implications for planning  performance • SMJ 94: Board control • AMJ 98: Discretion and executive pay • SMJ 98: Planning systems • SMJ 01: CEO social capital • Reviewers often profoundly uninterested in measurement • What is the ‘state of the art’ for construct measurement in strategy? • What are the implications of this?

  4. How well does strategy research contribute to the goal of advancing the paradigm? Strategy and Paradigm Development • In a Kuhnian sense, management is a young discipline • Strategy is one of the youngest subsets of management • Empirics play a key role in advancing paradigm development

  5. Where Were We Two Decades Ago? • Frequent use of nominal and single-item scales • Inadequate assessment of validity • Inexplicit treatment of strategy as an organization-level construct Source: Venkatraman & Grant (1986) AMR

  6. Overall, the rigor of much strategy research is weak. Strategy & Methods:An Overview • Inattention to methodology and design (Bowen & Wiersma, 1999) • Few replication studies (Hubbard et al., 1997) • Insufficient power (Mone et al., 1996) • Little emphasis on construct measurement (Hitt, Boyd & Li, 2004) We’ll focus on these issues

  7. Type I error Type II error A Power Primer Example: Population correlation = .30, a = .05 N = 30, 50% chance of successful detection N = 50, 70% chance N = 100, 90+% chance

  8. This is the good news… Power in Strategy Research • Published strategy papers had only 4 in 10 chance of successful detection – half of recommended level (Mazen, Hemmasi & Lewis, 1987) • Review of one research stream – only 8% of studies had sufficient power (Ferguson & Ketchen, 1999) • SMJ has lower power than other journals (Mone, Mueller & Mauland, 1996) • Few studies report power statistics (Mone et al, 1996)

  9. Blalock’s (1979) Elements of Social Processes • A theoretical language explaining causal relations between constructs • An operational language for examining relationships between constructs using indicators • An integrative theory describing causal relationships between constructs and indicators

  10. Lack of consensus on measurement of basic strategy constructs Strategic planning Performance Diversification Board oversight Environment Discretion Low emphasis on measurement Prevalence of single measures Sporadic reporting of reliability and validity Quality of Strategy Measures?

  11. Weak indicators lead to measurement error, and attenuation. Power calculations are based on fully accurate constructs. In other words, the statistics presented earlier overestimate the power of strategy research Constructx Constructy Hypotheses framed Indicatorx Indicatory Hypotheses tested The Role of Measurement Quality

  12. 0.30 Constructx Constructy N = 150power = .80 .60 .60 0.10 Indicatorx Indicatory N = 150power = .33 An Example

  13. Study 1: Assessment Content analysis of several hundred published articles Multiple journals Multiple years Study 2: Consequences Replication of a prominent research question Large sample analysis of US firms Manually adjust levels of measurement precision State of the Art & Consequences of Measurement Problems in Strategy Research

  14. State of the Art

  15. Assessment • Identified allempirical strategy papers in key outlets: AMJ, ASQ, MS,and SMJ • Sampled articles published 1998 -- 2002 • Content coded articles for: • Sample size • Measurement schemes • Reliability • Statistical test is unit of analysis: 625 tests in 196 articles • High inter-rater reliability on selection & coding of studies

  16. Sample size limitations do not appear to constrain measurement Descriptives

  17. While reported reliabilities are strong, this information is typically not available in the article. Good News…Bad News

  18. Is this a salient problem,or a non-issue? Summary of Content Analysis • Most strategy studies rely on single indicators • Limited efforts made to report reliability data, when multiple indicators are used • Many studies report either partial or no correlation matrices, making post hoc asessment impossible

  19. Consequences

  20. “Though both sets of authors conduct similar empirical tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999 Why Do Firms Diversify? • Amihud & Lev (1981): Diversification is a result of agency conditions. • Lane, Cannella & Lubatkin (1998): No support for A&L agency model. • Denis , Denis & Sarin (1999): Both studies contaminated by methodological flaws. Substituting improved measures led to stronger results.

  21. Measurement Both A&L and LCL studies used broad ownership categories Denis et al found better results when using ratio level indicators Only single indicator measures used Power 3 IV regression, w/ a=.05 power = .80N = 34 for large effect76 for moderate547 for small LCL N=309Denis et al N=933 Bulk of these studies had insufficient power Methods Issues in the Agency Argument

  22. Duality Equity Assets Sales BoardControl Diversification Sample:640 Fortune firms, representing 50 2-digit SIC s, and 200 4-digit SICs Related Firm Size Unrelated (Boyd, 1994) Insiders -.16 [2.2] StockOwnership Owner Reps .11 [2.3] Director Pay (Palepu, 1985)

  23. Duality Equity Assets Sales Insiders BoardControl StockOwnership Diversification Owner Reps Director Pay Related Firm Size Unrelated (Boyd, 1994) Now, Let’s Introduce Some Shadows… -.15 [2.1]-.12 [1.7] .11 [2.3] (Palepu, 1985)

  24. Duality Equity Assets Sales Insiders BoardControl StockOwnership Diversification Owner Reps Director Pay Related Firm Size Unrelated (Boyd, 1994) …And Some More… .09 [2.2] .13 [2.9] -.07 [1.6] -.05 [1.3] .05 [1.2] .06 [1.5] .12 [2.7] -.06 [1.3] -.03 [0.7] -.02 [0.5] .11 [2.3] (Palepu, 1985)

  25. (a) Single indicator DV(2 models) Failure to detect effect in 1 of 2 models R2 drops by a 10X factor Path coefficient drops slightly (b) Single indicator DV and IV (10 models) Failure to detect effect in 7 of 10 models R2 drops by a 10X factor Path coefficient cut in half Bottom Line:The A&L and LCL debate is entirely an artifact of measurement error and attenuation. Quantifying Shadows…

  26. Do We Know What We Know? Measuring Firm Size • Strategy research has placed little emphasis on statistical power • Construct measurement is not a priority of empirical research • Many strategy constructs are broad • Weak consensus regarding measurement of key concepts How many valid research questions have been cast aside due to these method limitations?

  27. What’s Next?

  28. Two decades ago The field of marketing was criticized for failing to pay attention to construct validity and associated measurement issues Today Studies routinely provide reliabilities, factor models, and evidence of convergent and discriminant validity A Comparison: Marketing Source: Jarvis, Mackenzie & Podsakoff (2005) JCR

  29. Some Easy Steps • Sneak measurement into substantive topics • Less attention to knee-jerk criticism (why ROE?) and more critical review of published measures • Collect multiple measures • Pay attention to validity • Hang out with the OB crowd

  30. More Balance in Research Focus Constructx Constructy SubstantiveResearch ConstructMeasurementResearch Indicatorx Indicatory

  31. A C B X X X X X X X X X Factor Models for Better Measurement (a) Common Approach

  32. A C B X X X X X X X X X Factor Models for Better Measurement (b) Convergent and discriminant validity

  33. A X X X X X X X X X Factor Models for Better Measurement (b) CMV test

  34. Duality BoardControl Insiders StockOwnership Owner Reps Director Pay Constructs: Cause or Effect?

  35. Duality BoardControl Insiders StockOwnership Owner Reps Director Pay Constructs: Cause or Effect?

  36. Cause or Effect? Deciding…

  37. Duality BoardControl Insiders StockOwnership Owner Reps Director Pay What About Board Control? • Substitution, portfolios, and balancing (Rediker & Seth ’85 Finkelstein & D’Aveni ’84) all suggest reflective model • CED of CFA is 0.97 • All indicators load at .001, with some moderate loading • Supplementary model using a sixth indicator (CEO tenure) • Main model unaffected • 8 of 12 subsets NS: 67% versus 70% NS in 5-factor tests

  38. Slope of Sales Mean of Sales Levels of Measurement EnvironmentDimensions Objective Perceptual Munificence Industry Industry Discretion Outsiders Simulation Firm Insiders TMT Middle Managers Source: Boyd, Dess & Rasheed (1993) AMR

  39. Organizational Research MethodsSpecial issue on Strategic Management Sample topics: • Pitfalls of using archival proxies • Evaluation of construct validity of key metrics • Providing insights into studying ‘unobservables’ • Articles/notes and empirical/conceptual papers welcome • Click here for Call for Papers • Proposals due March 1, 2006

  40. Web Resources • Boyd, Gove & Hitt (2005). Construct measurement in strategic management research, SMJ • Boyd, Gove & Hitt (2005). Consequences of measurement problems in strategic management research, SMJ • Manuscript draft, reviews, and reviewer replies to BGH 2005a/b • Hitt, Boyd & Li (2004). The state of strategic management research, RMSM

More Related