research methods national research university hse economics ph d program dr c s leonard june 2011 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Lecture 1 What is a good question? PowerPoint Presentation
Download Presentation
Lecture 1 What is a good question?

Loading in 2 Seconds...

play fullscreen
1 / 53

Lecture 1 What is a good question? - PowerPoint PPT Presentation


  • 85 Views
  • Uploaded on

Research Methods National Research University HSE Economics, Ph. D Program Dr C S Leonard June 2011. Lecture 1 What is a good question?. Outline of lecture. Knowledge Claims Classical Approaches Post-positivism Strategies of inquiry Data mining Desig n-based research

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Lecture 1 What is a good question?' - shira


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
research methods national research university hse economics ph d program dr c s leonard june 2011

Research MethodsNational Research UniversityHSE Economics, Ph. D ProgramDr C S Leonard June 2011

Lecture 1

What is a good question?

outline of lecture
Outline of lecture
  • Knowledge Claims
    • Classical Approaches
    • Post-positivism
  • Strategies of inquiry
    • Data mining
    • Design-based research
  • Good questions emerge from good research designs

Research Methods

knowledge claims
Knowledge Claims

Research Methods

knowledge claims1
Knowledge Claims
  • What warrants knowledge?
  • How is scientific method applied?

Research Methods

two positions
Two Positions

Leonard GSOM Ph.D. Research Methods 2010

  • Two scientific positions, inductivism and deductivism
    • From which method emerges
    • Deductive (logic) argumentation
      • If premises are true and no fallacies in the argument, then conclusion will be true
      • Not concerned with truth or falsity
    • Inductive arguments may have true premises, but we cannot be certain that conclusions will also be true (ampliative reasoning)
inductive deductive
Inductive, Deductive

Leonard GSOM Ph.D. Research Methods 2010

  • Deductive, from general to particular, and inductive, from particulars to general
  • Inductive: Frances Bacon vs the medieval Church: purging ourselves of idols
    • Problem (Hume) Can we predict the future? No
    • Positivism is descended from Bacon
  • Research becomes historical, truth confined to a systematic empirical study, that might obtain general laws
  • Empirical findings worthless to some deductivists
falsification karl popper 1902 1994
Falsification: Karl Popper (1902-1994)

Leonard GSOM Ph.D. Research Methods 2010

  • Karl Popper’s critical rationalism hasgenerated much debate since the 1930s
  • Intellectual autobiography, Unended Quest
  • Background, early Marxist, training with Adler and Freudian theories convinced them that the theories were too broad
    • Later rejected psychologism
  • Favored theory of relativity, Einstein, could be tested, verified, falsified
logic of falsificationism
Logic of Falsificationism
  • Scientific theories are abstract
    • can be tested only indirectly, by reference to their implications.
  • Scientific theory, and human knowledge generally, is irreducibly conjectural or hypothetical
    • to solve problems that have arisen in specific historio-cultural settings
  • Logically, no number of positive outcomes at the level of experimental testing can confirm a scientific theory
  • A single counterexample is logically decisive: it shows the theory, from which the implication is derived, to be false.

Research Methods

vs inductivism
Vs inductivism

Leonard GSOM Ph.D. Research Methods 2010

Direct antithesis of inductivism

Growth of knowledge requires overturning previous beliefs

Have a theory, test it, falsify it and move on

Many economists seek to prove theories correct; the job of science is to disprove them

Don’t strive for certainty (verificationism)

Bans ad hoc adjustments to a theory to prevent it from being falsified

A new theory will possess greater empirical content than its predecessors

critics
Critics

Leonard GSOM Ph.D. Research Methods 2010

  • This hinders not promotes science
  • Can’t reject theories so easily, some theories are better at some things than others
    • Marxism, accommodates business cycles and disequilibrium better then orthodox Keynsianism, but the Monetarists better understand inflationary processes than the Keynsians, who argue that their theories and policies are more effective against unemployment
    • Should not critique a new theory too rigorously, because it may have something good in it
defender blaug
defender: Blaug
  • Blaug (be taken seriously)
    • Have a prediction about the future
    • Require a formal model
    • Falsification is essential
    • Be scientific, or not
    • Falsificationism: much tougher
    • Lay down restrictions on what Popper calls immunizing strategems

Research Methods

enormous influence
Enormous influence

Leonard GSOM Ph.D. Research Methods 2010

  • Mark Blaug
    • Even econometricians, however, econometric results difficult to falsify
    • Plain fact
      • Most economists tend to verify..
thomas kuhn
Thomas Kuhn

Leonard GSOM Ph.D. Research Methods 2010

  • Structure of Scientific Revolutions
    • Science is not good science unless it is working under the framework of a theory, makes no progress
    • It adopts a particular view of the world, and all subsequent research adds to that
    • Pre science—lots of theories hoping to explain the same thing
    • The paradigm: an achievement so important that it attracts an enduring group of adherents
      • Commitment and consensus are prerequisites for normal science
paradigms
Paradigms

Leonard GSOM Ph.D. Research Methods 2010

Older generations stay with their paradigms, new ones acquire new paradigms

Releases scientists from the necessity of debating fundamentals

They can then concentrate on subtle, esoteric aspects of their subject

kuhn normal science
Kuhn: Normal Science

Leonard GSOM Ph.D. Research Methods 2010

  • Mopping up
  • Determination of facts
  • Setting the facts within theory
    • Articulating the theory
  • Then, anomalies, followed by crisis, followed by fundamental change
applied econometrics
Applied econometrics

Leonard GSOM Ph.D. Research Methods 2010

Middle ground

Use theory, provide initial specification

Data exploration techniques to extend or refine it

Bridge theory and empirical data analysis

How do we know a theory is correct?

Different users have different tastes and beliefs

Complications with computation: large data sets numerous models possible

paradigms1
Paradigms

Leonard GSOM Ph.D. Research Methods 2010

How researchers will learn/what they will learn, assumptions

Philosophical assumptions, epistemologies (how we know something), ontologies (what is knowledge), axiology (what values go into knowledge), methodology (process for studying)

extensions
Extensions

Leonard GSOM Ph.D. Research Methods 2010

  • These debates shaped much social science theory about
    • Innovations
    • Science
    • Path dependence
    • Historical legacies
    • Nature of change
    • Nature of reform and timing
positivism
Positivism
  • Can we be positive about our claims of knowledge when studying behavior and actions (Comte, Mill, Durkheim, Newton and Locke)?
  • Causes probably determine effects?
  • Reductionism: reduce ideas into small discrete sets to be tested

Research Methods

positivism vs interpretivism
Positivism vsInterpretivism
  • Interpretivism: Weber (Verstehen)

Research Methods

positivism and its opponents
Positivism and its Opponents
  • Quantitative, positivist, post-positivist research, empirical science
  • Challenge to positivism: against the traditional notion of the absolute truth of knowledge; playing tennis with the net down

Research Methods

methodological pluralism
Methodological pluralism?
  • Bruce Caldwell (let 100 flowers bloom)
    • Little economics will survive if we take this seriously
    • Confirmationism Verification
    • Falsificationism is never practiced because it is unpracticeable

Research Methods

baysian methods
Baysian methods
  • Test, verify and refine the laws and theories governing behavior
  • Baysian methods: from theory, to test, to revision

Research Methods

1 post positivism
(1) Post Positivism
  • Knowledge is conjectural
  • Research is to make claims and refine or abandon them
  • Data, evidence and rational considerations shape knowledge
  • Being objective is key

Research Methods

2 social construction
(2) Social construction
  • Social construction
    • Mannheim, Burger, Luckmann, Neuman
    • Look at participants views
    • Judgments are subjective, meanings are varied and multiple
    • Interviews: open ended questioning, the more the interviewee talks spontaneously, the better
    • Participants allowed to construct meaning (rather than responding to concrete situations)
    • Process of interaction, context of work

Research Methods

3 pragmatism
(3) Pragmatism
  • Pierce, James and Dewey
  • Knowledge claims arise out of actions, situations, consequences, rather than ex ante conditions
  • Concern with what works
  • Pluralistic approach
  • Mixed methods, qualitative, quantitative
  • Research always occurs in social contexts
  • Stop asking questions about the laws of nature

Research Methods

logic of research design
Logic of Research Design

Influence Research Design

Research Methods

slide28

“Across most fields… applied economists are now less likely to pin a causal interpretation of the results on econometric methodology alone. Design-based studies are distinguished by their prima facie credibility and by the attention investigators devote to making both an institutional and a data-driven case for causality.” Joshua D. Angrist and Jörn-Steffen Pischke

Research Methods

general to specific methodology
General to Specific Methodology

“LSE” tradition of time-series econometrics that began in the 1960s at the London School of Economics

Mizon (1995) a brief history

The practitioners of LSE econometrics are now widely dispersed among academic institutions throughout Britain and the world.

The LSE approach is described sympathetically in Gilbert (1986), Hendry (1987,1995, esp. chs. 9-15), Pagan (1987), Phillips (1988), Ericsson, Campos and Tran(1990), and Mizon (1995).

For more sceptical accounts, see Hansen (1996) and Faust and Whiteman (1995, 1997)

Research Methods

general to specific
General to Specific

Context: Linear (cross-country growth) Use: Time-Series

Step 1.

General regression will include every possible variable -- all the information about the true determinants.

Step 2.

The information content is then sharpened by a more parsimonious regression – the specific regression

  • it is statistically well specified (for example, it has white noise errors);
  • that it is a valid restriction of the general regression, and
  • that it encompasses every other parsimonious regression that is a valid restriction of the general regression

Criticism: data-mining,

Research Methods

extreme bounds analysis the new critique
Extreme Bounds Analysis: the new critique
  • Edward Leamer’s “extreme-bounds analysis” (1983, 1985).
  • A coefficient of interest is robust only to the degree that it displays a small variation to the presence or absence of other regressors. Leamer and Leonard (1983) define the extreme-bounds for the coefficient of a particular variable within a search universe as ranging between the lowest estimate of its value minus two times its standard error to the highest estimate of its value plus two times its standard error, where the extreme values are drawn from the set of every possible subset of regressors that include the variable of interest. A variable is said to be robust if its extreme bounds lie strictly to one side or the other of zero.

Research Methods

judgment calls
Judgment Calls

The main difference between structural and experimental (or ``atheoretic'') approaches is not in the number of assumptions but the extent to which they are made explicit. (Michael Keane)

Research Methods

summary
Summary
  • Quantitative (numbers)
    • Experimental design--controls
    • Non experimental design, surveys
  • Qualitative (words)
    • Narratives, phenomenologies, ethnographies, grounded theory, case studies
  • Mixed methods
    • Sequential, concurrent, transformative

Research Methods

quantitative
Quantitative
  • Random assignment of subjects to treatment
  • Quasi experiments: non random designs
  • Surveys cross sectional and longitudinal, generalize from sample to population

Research Methods

qualitative
Qualitative
  • Ethnographies: researcher studies an intact cultural group in its setting over time (responses)
  • Grounded theory
    • Derive an abstract theory of a process, action or interaction, grounded in views of participants
  • Case Studies
    • Exploring in depth a program, event, activity, process, or individuals, bound in time, variety of procedures
  • Phenomenological: lived experiences
  • Narrative research: lives, stories, retellings

Research Methods

mixed
Mixed
  • Gets around biases in any one method used exclusively
  • Progress from one method to another
  • Illustrate
  • Determine what the concept is
  • Test assumptions on one case

Research Methods

logic
Logic
  • Qualitative:
    • Wider range of methods, non-numerical by definition
    • Small n, intensive interviews, depth analysis, discursive, account of event or unit
    • Focus on event, decision, institution, location, issue or legislation
    • Incident important in its own right (war, election, change in leadership, marketing strategy, community decision, etc)
  • Against bifurcation? mixed methods
    • Systematic, scientific research of all kinds
    • Most research does not neatly fit one or other category

Research Methods

goal of social science research
Goal of social science research
  • Inference (descriptive, explanatory)
    • Attempting to infer beyond immediate facts to something broader
    • Learning about causal effects from data
  • Public procedures (explicit, codified)
    • Replication
  • Conclusions are uncertain
  • Observes rules of inference

Research Methods

descriptive inference
Descriptive inference
  • Distinguish systematic from non-systematic features
  • Systematic from stochastic
  • Counter-factuals (what would have happened, had meters not struck the earth 65 million yrs ago)

Research Methods

rules for research design
Rules for Research Design
  • Intuition: Choice of better topics is idiosyncratic
  • Two ways to test if it is a good topic
    • Is it about something important in the real world
    • A research topic should make a specific contribution to an identifiable scholarly literature by increasing our ability to construct verified scientific explanations
    • ie: locating it within the framework of existing social science literature
    • This is the subject of the second lecture today—what makes a theory or theoretical contributions valuable to the community of editors of journals

Research Methods

caution
Caution
  • There may be reasons a theory is practicable, even though its long term scientific value has been questioned
  • Theoretically incoherent models used to forecast the US economy—diversion of macroeconomic theory and applied macroeconomics (see Mankiw 1990)
  • New theories, however, remain speculative

Research Methods

design based research
Design based Research
  • Leamer 1983 highlighted the benefits of sensitivity analysis, a procedure in which researchers show how their results change with changes in specification or functional form. Sensitivity analysis has had a salutary but not a revolutionary effect on econometric practice.
  • As we see it, the credibility revolution in empirical work can be traced to the rise of a design-based approach that emphasizes the identification of causal effects.

Research Methods

wow factor
Wow-Factor
  • Design-based studies typically feature either real or natural experiments and are distinguished by their prima facie credibility and by the attention investigators devote to making the case for a causal interpretation of the findings their designs generate.
  • Design-based studies are most often found in the microeconomic fields of Development, Education, Environment, Labor, Health, and Public Finance, but are still rare in Industrial Organization and Macroeconomics.

Research Methods

literature review
Literature Review

Research Methods

literature review1
Literature Review
  • Classical
  • Systematic review
  • Meta-analysis
  • Narrative review
  • Search issues
  • Presentation

Research Methods

classical literature review
Classical Literature Review
  • Large Disciplinary Differences
  • Sociology, Psychology, Business
  • Economics

Research Methods

systematic review
Systematic Review
  • A systematic review aims to provide an exhaustive summary of literature relevant to a research questions
  • The first step of a systematic review is a thorough search of the literature for relevant papers.
  • The Methodology section of the review will list the databases and citation indexes searched as well as any individual journals. Next, the titles and the abstracts of the identified articles are checked against pre-determined criteria for eligibility and relevance.

Research Methods

slide48
More
  • Ability to control for between-study variation
  • Including moderators to explain variation
  • Deal with information overload: the high number of articles published each year.
  • It combines several studies and will therefore be less influenced by local findings than single studies will be.
  • Makes it possible to show if a bias for published works exists.

Research Methods

systematic analysis
Systematic Analysis
  • Meta-Analysis
  • Meta-analysis leads to a shift of emphasis from single studies to multiple studies. It emphasizes the practical importance of the effect size instead of the statistical significance of individual studies. This shift in thinking has been termed "meta-analytic thinking"

Research Methods

meta analysis crit
Meta-Analysis: crit
  • About science, but not science
    • Statistical examination of scientific studies
  • Cannot propose ways to falsify a theory

Research Methods

meta narrative review
Meta-Narrative Review
  • Example: Connected Communities’,
    • a research programme of the Institute of Health and Human Development (IHHD) to investigate the meanings of community within and across research disciplines by adopting an innovative methodology based on a meta-narrative systematic review approach. Policy and academic interest in the concept of ‘community’ is longstanding and such interest has become central to policy making in the last two decades.
  • Meta-narrative review shows the diversity in the meaning of ‘communities’ --various conceptualisations and meanings of community across disciplines, over time, and within different cultures and contexts
    • (Greenhalgh et al, 2005) is a type of ‘systematic’ review rather than a traditional expert driven literature review
  • A focus on identifying the ‘storylines of research’ within and across disciplinary boundaries. Identifies the meta-narratives of each discipline and analyse the different ‘discourses’ and languages of ‘community’.

Research Methods

the end

THE END

Until Next Hour