current issues in individual group and organizational level measurement strategic management
Download
Skip this Video
Download Presentation
Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management

Loading in 2 Seconds...

play fullscreen
1 / 41

Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management - PowerPoint PPT Presentation


  • 100 Views
  • Uploaded on

Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management. Prof. Brian Boyd W. P. Carey School Arizona State University http://www.briankboyd.com. Roadmap. Background State of the art Consequences of measurement problems Application. Background.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Current Issues in Individual, Group, and Organizational Level Measurement : Strategic Management' - judith-vega


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
current issues in individual group and organizational level measurement strategic management

Current Issues in Individual, Group, and Organizational Level Measurement :Strategic Management

Prof. Brian Boyd

W. P. Carey School

Arizona State University

http://www.briankboyd.com

roadmap
Roadmap
  • Background
  • State of the art
  • Consequences of measurement problems
  • Application
slide4

Background

  • Most of my papers have a strong measurement component, e.g.,
    • SMJ 90: Environmental characteristics
    • JMS 91: Implications for planning  performance
    • SMJ 94: Board control
    • AMJ 98: Discretion and executive pay
    • SMJ 98: Planning systems
    • SMJ 01: CEO social capital
  • Reviewers often profoundly uninterested in measurement
  • What is the ‘state of the art’ for construct measurement in strategy?
  • What are the implications of this?
strategy and paradigm development

How well does strategy research contribute to the goal of advancing the paradigm?

Strategy and Paradigm Development
  • In a Kuhnian sense, management is a young discipline
  • Strategy is one of the youngest subsets of management
  • Empirics play a key role in advancing paradigm development
where were we two decades ago
Where Were We Two Decades Ago?
  • Frequent use of nominal and single-item scales
  • Inadequate assessment of validity
  • Inexplicit treatment of strategy as an organization-level construct

Source: Venkatraman & Grant (1986) AMR

strategy methods an overview

Overall, the rigor of much strategy research is weak.

Strategy & Methods:An Overview
  • Inattention to methodology and design (Bowen & Wiersma, 1999)
  • Few replication studies (Hubbard et al., 1997)
  • Insufficient power (Mone et al., 1996)
  • Little emphasis on construct measurement (Hitt, Boyd & Li, 2004)

We’ll focus on these issues

a power primer

Type I error

Type II error

A Power Primer

Example: Population correlation = .30, a = .05

N = 30, 50% chance of successful detection

N = 50, 70% chance

N = 100, 90+% chance

power in strategy research

This is the good news…

Power in Strategy Research
  • Published strategy papers had only 4 in 10 chance of successful detection – half of recommended level (Mazen, Hemmasi & Lewis, 1987)
  • Review of one research stream – only 8% of studies had sufficient power (Ferguson & Ketchen, 1999)
  • SMJ has lower power than other journals (Mone, Mueller & Mauland, 1996)
  • Few studies report power statistics (Mone et al, 1996)
blalock s 1979 elements of social processes
Blalock’s (1979) Elements of Social Processes
  • A theoretical language explaining causal relations between constructs
  • An operational language for examining relationships between constructs using indicators
  • An integrative theory describing causal relationships between constructs and indicators
quality of strategy measures
Lack of consensus on measurement of basic strategy constructs

Strategic planning

Performance

Diversification

Board oversight

Environment

Discretion

Low emphasis on measurement

Prevalence of single measures

Sporadic reporting of reliability and validity

Quality of Strategy Measures?
the role of measurement quality

Weak indicators lead to measurement error, and attenuation. Power calculations are based on fully accurate constructs.

In other words, the statistics presented earlier overestimate the power of strategy research

Constructx

Constructy

Hypotheses framed

Indicatorx

Indicatory

Hypotheses tested

The Role of Measurement Quality
an example

0.30

Constructx

Constructy

N = 150power = .80

.60

.60

0.10

Indicatorx

Indicatory

N = 150power = .33

An Example
state of the art consequences of measurement problems in strategy research
Study 1: Assessment

Content analysis of several hundred published articles

Multiple journals

Multiple years

Study 2: Consequences

Replication of a prominent research question

Large sample analysis of US firms

Manually adjust levels of measurement precision

State of the Art & Consequences of Measurement Problems in Strategy Research
assessment
Assessment
  • Identified allempirical strategy papers in key outlets: AMJ, ASQ, MS,and SMJ
  • Sampled articles published 1998 -- 2002
  • Content coded articles for:
    • Sample size
    • Measurement schemes
    • Reliability
  • Statistical test is unit of analysis: 625 tests in 196 articles
  • High inter-rater reliability on selection & coding of studies
summary of content analysis

Is this a salient problem,or a non-issue?

Summary of Content Analysis
  • Most strategy studies rely on single indicators
  • Limited efforts made to report reliability data, when multiple indicators are used
  • Many studies report either partial or no correlation matrices, making post hoc asessment impossible
why do firms diversify

“Though both sets of authors conduct similar empirical tests on virtually identical data, they arrive at completely different conclusions.” Denis et al., 1999

Why Do Firms Diversify?
  • Amihud & Lev (1981): Diversification is a result of agency conditions.
  • Lane, Cannella & Lubatkin (1998): No support for A&L agency model.
  • Denis , Denis & Sarin (1999): Both studies contaminated by methodological flaws. Substituting improved measures led to stronger results.
methods issues in the agency argument
Measurement

Both A&L and LCL studies used broad ownership categories

Denis et al found better results when using ratio level indicators

Only single indicator measures used

Power

3 IV regression, w/ a=.05 power = .80N = 34 for large effect76 for moderate547 for small

LCL N=309Denis et al N=933

Bulk of these studies had insufficient power

Methods Issues in the Agency Argument
slide23

Duality

Equity

Assets

Sales

BoardControl

Diversification

Sample:640 Fortune firms, representing 50 2-digit SIC s, and 200 4-digit SICs

Related

Firm Size

Unrelated

(Boyd, 1994)

Insiders

-.16 [2.2]

StockOwnership

Owner Reps

.11 [2.3]

Director Pay

(Palepu, 1985)

slide24

Duality

Equity

Assets

Sales

Insiders

BoardControl

StockOwnership

Diversification

Owner Reps

Director Pay

Related

Firm Size

Unrelated

(Boyd, 1994)

Now, Let’s Introduce Some Shadows…

-.15 [2.1]-.12 [1.7]

.11 [2.3]

(Palepu, 1985)

slide25

Duality

Equity

Assets

Sales

Insiders

BoardControl

StockOwnership

Diversification

Owner Reps

Director Pay

Related

Firm Size

Unrelated

(Boyd, 1994)

…And Some More…

.09 [2.2] .13 [2.9] -.07 [1.6]

-.05 [1.3] .05 [1.2] .06 [1.5]

.12 [2.7] -.06 [1.3] -.03 [0.7]

-.02 [0.5]

.11 [2.3]

(Palepu, 1985)

quantifying shadows
(a) Single indicator DV(2 models)

Failure to detect effect in 1 of 2 models

R2 drops by a 10X factor

Path coefficient drops slightly

(b) Single indicator DV and IV (10 models)

Failure to detect effect in 7 of 10 models

R2 drops by a 10X factor

Path coefficient cut in half

Bottom Line:The A&L and LCL debate is entirely an artifact of measurement error and attenuation.

Quantifying Shadows…
do we know what we know
Do We Know What We Know?

Measuring Firm Size

  • Strategy research has placed little emphasis on statistical power
  • Construct measurement is not a priority of empirical research
  • Many strategy constructs are broad
  • Weak consensus regarding measurement of key concepts

How many valid research questions have been cast aside due to these method limitations?

a comparison marketing
Two decades ago

The field of marketing was criticized for failing to pay attention to construct validity and associated measurement issues

Today

Studies routinely provide reliabilities, factor models, and evidence of convergent and discriminant validity

A Comparison: Marketing

Source: Jarvis, Mackenzie & Podsakoff (2005) JCR

some easy steps
Some Easy Steps
  • Sneak measurement into substantive topics
  • Less attention to knee-jerk criticism (why ROE?) and more critical review of published measures
  • Collect multiple measures
  • Pay attention to validity
  • Hang out with the OB crowd
more balance in research focus
More Balance in Research Focus

Constructx

Constructy

SubstantiveResearch

ConstructMeasurementResearch

Indicatorx

Indicatory

factor models for better measurement

A

C

B

X

X

X

X

X

X

X

X

X

Factor Models for Better Measurement

(a) Common Approach

factor models for better measurement1

A

C

B

X

X

X

X

X

X

X

X

X

Factor Models for Better Measurement

(b) Convergent and discriminant validity

constructs cause or effect

Duality

BoardControl

Insiders

StockOwnership

Owner Reps

Director Pay

Constructs: Cause or Effect?
slide36

Duality

BoardControl

Insiders

StockOwnership

Owner Reps

Director Pay

Constructs: Cause or Effect?

what about board control

Duality

BoardControl

Insiders

StockOwnership

Owner Reps

Director Pay

What About Board Control?
  • Substitution, portfolios, and balancing (Rediker & Seth ’85 Finkelstein & D’Aveni ’84) all suggest reflective model
  • CED of CFA is 0.97
  • All indicators load at .001, with some moderate loading
  • Supplementary model using a sixth indicator (CEO tenure)
    • Main model unaffected
    • 8 of 12 subsets NS: 67% versus 70% NS in 5-factor tests
levels of measurement

Slope of Sales

Mean of Sales

Levels of Measurement

EnvironmentDimensions

Objective

Perceptual

Munificence

Industry

Industry

Discretion

Outsiders

Simulation

Firm

Insiders

TMT

Middle Managers

Source: Boyd, Dess & Rasheed (1993) AMR

organizational research methods special issue on strategic management
Organizational Research MethodsSpecial issue on Strategic Management

Sample topics:

  • Pitfalls of using archival proxies
  • Evaluation of construct validity of key metrics
  • Providing insights into studying ‘unobservables’
  • Articles/notes and empirical/conceptual papers welcome
  • Click here for Call for Papers
  • Proposals due March 1, 2006
web resources
Web Resources
  • Boyd, Gove & Hitt (2005). Construct measurement in strategic management research, SMJ
  • Boyd, Gove & Hitt (2005). Consequences of measurement problems in strategic management research, SMJ
  • Manuscript draft, reviews, and reviewer replies to BGH 2005a/b
  • Hitt, Boyd & Li (2004). The state of strategic management research, RMSM
ad