Current issues in individual group and organizational level measurement strategic management
Download
1 / 41

Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management - PowerPoint PPT Presentation


  • 100 Views
  • Uploaded on

Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management. Prof. Brian Boyd W. P. Carey School Arizona State University http://www.briankboyd.com. Roadmap. Background State of the art Consequences of measurement problems Application. Background.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management' - judith-vega


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Current issues in individual group and organizational level measurement strategic management

Current Issues in Individual, Group, and Organizational Level Measurement :Strategic Management

Prof. Brian Boyd

W. P. Carey School

Arizona State University

http://www.briankboyd.com


Roadmap
Roadmap Level Measurement :

  • Background

  • State of the art

  • Consequences of measurement problems

  • Application


Current issues in individual group and organizational level measurement strategic management

Background Level Measurement :

  • Most of my papers have a strong measurement component, e.g.,

    • SMJ 90: Environmental characteristics

    • JMS 91: Implications for planning  performance

    • SMJ 94: Board control

    • AMJ 98: Discretion and executive pay

    • SMJ 98: Planning systems

    • SMJ 01: CEO social capital

  • Reviewers often profoundly uninterested in measurement

  • What is the ‘state of the art’ for construct measurement in strategy?

  • What are the implications of this?


Strategy and paradigm development

How well does strategy research contribute to the goal of advancing the paradigm?

Strategy and Paradigm Development

  • In a Kuhnian sense, management is a young discipline

  • Strategy is one of the youngest subsets of management

  • Empirics play a key role in advancing paradigm development


Where were we two decades ago
Where Were We Two Decades Ago? advancing the paradigm?

  • Frequent use of nominal and single-item scales

  • Inadequate assessment of validity

  • Inexplicit treatment of strategy as an organization-level construct

Source: Venkatraman & Grant (1986) AMR


Strategy methods an overview

Overall, the rigor of much strategy research is weak. advancing the paradigm?

Strategy & Methods:An Overview

  • Inattention to methodology and design (Bowen & Wiersma, 1999)

  • Few replication studies (Hubbard et al., 1997)

  • Insufficient power (Mone et al., 1996)

  • Little emphasis on construct measurement (Hitt, Boyd & Li, 2004)

We’ll focus on these issues


A power primer

Type I error advancing the paradigm?

Type II error

A Power Primer

Example: Population correlation = .30, a = .05

N = 30, 50% chance of successful detection

N = 50, 70% chance

N = 100, 90+% chance


Power in strategy research

This is the advancing the paradigm?good news…

Power in Strategy Research

  • Published strategy papers had only 4 in 10 chance of successful detection – half of recommended level (Mazen, Hemmasi & Lewis, 1987)

  • Review of one research stream – only 8% of studies had sufficient power (Ferguson & Ketchen, 1999)

  • SMJ has lower power than other journals (Mone, Mueller & Mauland, 1996)

  • Few studies report power statistics (Mone et al, 1996)


Blalock s 1979 elements of social processes
Blalock’s (1979) Elements of Social Processes advancing the paradigm?

  • A theoretical language explaining causal relations between constructs

  • An operational language for examining relationships between constructs using indicators

  • An integrative theory describing causal relationships between constructs and indicators


Quality of strategy measures

Lack of consensus on measurement of basic strategy constructs

Strategic planning

Performance

Diversification

Board oversight

Environment

Discretion

Low emphasis on measurement

Prevalence of single measures

Sporadic reporting of reliability and validity

Quality of Strategy Measures?


The role of measurement quality

Weak indicators lead to measurement error, and attenuation. Power calculations are based on fully accurate constructs.

In other words, the statistics presented earlier overestimate the power of strategy research

Constructx

Constructy

Hypotheses framed

Indicatorx

Indicatory

Hypotheses tested

The Role of Measurement Quality


An example

0.30 Power calculations are based on fully accurate constructs.

Constructx

Constructy

N = 150power = .80

.60

.60

0.10

Indicatorx

Indicatory

N = 150power = .33

An Example


State of the art consequences of measurement problems in strategy research

Study 1: Assessment Power calculations are based on fully accurate constructs.

Content analysis of several hundred published articles

Multiple journals

Multiple years

Study 2: Consequences

Replication of a prominent research question

Large sample analysis of US firms

Manually adjust levels of measurement precision

State of the Art & Consequences of Measurement Problems in Strategy Research


Current issues in individual group and organizational level measurement strategic management

State Power calculations are based on fully accurate constructs.of the Art


Assessment
Assessment Power calculations are based on fully accurate constructs.

  • Identified allempirical strategy papers in key outlets: AMJ, ASQ, MS,and SMJ

  • Sampled articles published 1998 -- 2002

  • Content coded articles for:

    • Sample size

    • Measurement schemes

    • Reliability

  • Statistical test is unit of analysis: 625 tests in 196 articles

  • High inter-rater reliability on selection & coding of studies



Good news bad news

While reported reliabilities are strong, this information is typically not available in the article.

Good News…Bad News


Summary of content analysis

Is this a salient problem, typically or a non-issue?

Summary of Content Analysis

  • Most strategy studies rely on single indicators

  • Limited efforts made to report reliability data, when multiple indicators are used

  • Many studies report either partial or no correlation matrices, making post hoc asessment impossible


Current issues in individual group and organizational level measurement strategic management

Consequences typically


Why do firms diversify

“Though both sets of authors conduct similar empirical tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999

Why Do Firms Diversify?

  • Amihud & Lev (1981): Diversification is a result of agency conditions.

  • Lane, Cannella & Lubatkin (1998): No support for A&L agency model.

  • Denis , Denis & Sarin (1999): Both studies contaminated by methodological flaws. Substituting improved measures led to stronger results.


Methods issues in the agency argument

Measurement tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999

Both A&L and LCL studies used broad ownership categories

Denis et al found better results when using ratio level indicators

Only single indicator measures used

Power

3 IV regression, w/ a=.05 power = .80N = 34 for large effect76 for moderate547 for small

LCL N=309Denis et al N=933

Bulk of these studies had insufficient power

Methods Issues in the Agency Argument


Current issues in individual group and organizational level measurement strategic management

Duality tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999

Equity

Assets

Sales

BoardControl

Diversification

Sample:640 Fortune firms, representing 50 2-digit SIC s, and 200 4-digit SICs

Related

Firm Size

Unrelated

(Boyd, 1994)

Insiders

-.16 [2.2]

StockOwnership

Owner Reps

.11 [2.3]

Director Pay

(Palepu, 1985)


Current issues in individual group and organizational level measurement strategic management

Duality tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999

Equity

Assets

Sales

Insiders

BoardControl

StockOwnership

Diversification

Owner Reps

Director Pay

Related

Firm Size

Unrelated

(Boyd, 1994)

Now, Let’s Introduce Some Shadows…

-.15 [2.1]-.12 [1.7]

.11 [2.3]

(Palepu, 1985)


Current issues in individual group and organizational level measurement strategic management

Duality tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999

Equity

Assets

Sales

Insiders

BoardControl

StockOwnership

Diversification

Owner Reps

Director Pay

Related

Firm Size

Unrelated

(Boyd, 1994)

…And Some More…

.09 [2.2] .13 [2.9] -.07 [1.6]

-.05 [1.3] .05 [1.2] .06 [1.5]

.12 [2.7] -.06 [1.3] -.03 [0.7]

-.02 [0.5]

.11 [2.3]

(Palepu, 1985)


Quantifying shadows

(a) Single indicator DV tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999(2 models)

Failure to detect effect in 1 of 2 models

R2 drops by a 10X factor

Path coefficient drops slightly

(b) Single indicator DV and IV (10 models)

Failure to detect effect in 7 of 10 models

R2 drops by a 10X factor

Path coefficient cut in half

Bottom Line:The A&L and LCL debate is entirely an artifact of measurement error and attenuation.

Quantifying Shadows…


Do we know what we know
Do We Know What We Know? tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999

Measuring Firm Size

  • Strategy research has placed little emphasis on statistical power

  • Construct measurement is not a priority of empirical research

  • Many strategy constructs are broad

  • Weak consensus regarding measurement of key concepts

    How many valid research questions have been cast aside due to these method limitations?


Current issues in individual group and organizational level measurement strategic management

What’s Next? tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999


A comparison marketing

Two decades ago tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999

The field of marketing was criticized for failing to pay attention to construct validity and associated measurement issues

Today

Studies routinely provide reliabilities, factor models, and evidence of convergent and discriminant validity

A Comparison: Marketing

Source: Jarvis, Mackenzie & Podsakoff (2005) JCR


Some easy steps
Some Easy Steps tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999

  • Sneak measurement into substantive topics

  • Less attention to knee-jerk criticism (why ROE?) and more critical review of published measures

  • Collect multiple measures

  • Pay attention to validity

  • Hang out with the OB crowd


More balance in research focus
More Balance in Research Focus tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999

Constructx

Constructy

SubstantiveResearch

ConstructMeasurementResearch

Indicatorx

Indicatory


Factor models for better measurement

A tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999

C

B

X

X

X

X

X

X

X

X

X

Factor Models for Better Measurement

(a) Common Approach


Factor models for better measurement1

A tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999

C

B

X

X

X

X

X

X

X

X

X

Factor Models for Better Measurement

(b) Convergent and discriminant validity


Factor models for better measurement2

A tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999

X

X

X

X

X

X

X

X

X

Factor Models for Better Measurement

(b) CMV test


Constructs cause or effect

Duality tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999

BoardControl

Insiders

StockOwnership

Owner Reps

Director Pay

Constructs: Cause or Effect?


Current issues in individual group and organizational level measurement strategic management

Duality tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999

BoardControl

Insiders

StockOwnership

Owner Reps

Director Pay

Constructs: Cause or Effect?


Cause or effect deciding
Cause or Effect? Deciding… tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999


What about board control

Duality tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999

BoardControl

Insiders

StockOwnership

Owner Reps

Director Pay

What About Board Control?

  • Substitution, portfolios, and balancing (Rediker & Seth ’85 Finkelstein & D’Aveni ’84) all suggest reflective model

  • CED of CFA is 0.97

  • All indicators load at .001, with some moderate loading

  • Supplementary model using a sixth indicator (CEO tenure)

    • Main model unaffected

    • 8 of 12 subsets NS: 67% versus 70% NS in 5-factor tests


Levels of measurement

Slope of Sales tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999

Mean of Sales

Levels of Measurement

EnvironmentDimensions

Objective

Perceptual

Munificence

Industry

Industry

Discretion

Outsiders

Simulation

Firm

Insiders

TMT

Middle Managers

Source: Boyd, Dess & Rasheed (1993) AMR


Organizational research methods special issue on strategic management
Organizational Research Methods tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999Special issue on Strategic Management

Sample topics:

  • Pitfalls of using archival proxies

  • Evaluation of construct validity of key metrics

  • Providing insights into studying ‘unobservables’

  • Articles/notes and empirical/conceptual papers welcome

  • Click here for Call for Papers

  • Proposals due March 1, 2006


Web resources
Web Resources tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999

  • Boyd, Gove & Hitt (2005). Construct measurement in strategic management research, SMJ

  • Boyd, Gove & Hitt (2005). Consequences of measurement problems in strategic management research, SMJ

  • Manuscript draft, reviews, and reviewer replies to BGH 2005a/b

  • Hitt, Boyd & Li (2004). The state of strategic management research, RMSM