Agenda

1. Harvard Extension School, Spring 2013SSCI E-100B – Section 2 (23667):Graduate Research Methods and Scholarly Writing in the Social Sciences: Government and HistoryJoe BondClass 5March 4, 2013

2. Agenda • Channels of analysis (structured, episodic, leadership-personality-world views) • Content analysis • Facilitation (Dave & Nick)

3. Applied Research • Definition: research where findings and conclusions are applied immediately to solve a problem or improve the effectiveness of an existing or proposed social program • Are agencies set up to help the poor succeeding? • How many parking meters should be put on main street? • Where are traffic lights most needed in a given city? • How can the US best reduce the incidents of state failure worldwide?

4. Africa Prospects: Predicting State Failure with Structural Data

5. Africa Prospects • Purpose:to assess the vulnerability of countries to conflict escalation based on its profile or set of structural indicators. • Overall Accuracy:is defined as the ratio of correct classifications (C) to all classifications (A). Accuracy = C/A * 100%. • Recall: is defined as the ratio of correct classifications (C) to the observed classification (O). Recall = C/O * 100% and represents the ability of the algorithm to classify the conflicts as they were observed. • Precision: is defined as the ratio of correct classifications (C) to correct (C) and incorrect classifications (I). Precision = C/(C+I) *100%. Illuminates the algorithm’s false positives; specifically, the higher the ratio the lower the false positives.

6. Forecasting Performance Metrics: Definitions and Illustrations Recall Precision Overall Accuracy # of correctly predicted conflicts # of conflicts that occurred # of correctly predicted conflicts # of conflicts predicted to occur # of correct predictions # of predictions made • Bad forecast model #1 (almost every country will be unstable) • High recall= 99% (1% miss rate) • Low precision = 5% (95% false positive rate) • Low accuracy =40-50% • NET IS CAST TOO BROADLY • Bad forecast model #2 (few countries will be unstable) • Low recall = 5% (95% miss rate) • High precision =100% (0% false positives) • Low accuracy =40-50% • NET IS CAST TOO NARROWLY • Near-perfect forecast model • High recall = 99% • High precision = 100% • High accuracy =99.5% Countries forecast to be unstable at some level of intensity False positives Countries that experience instability “Misses” misses Countries that DO NOT experience instability

7. 5-15 Year Validation of Forecasting Average Performance Scores For Different Training Sets / Forecast Periods Forecast Period Accuracy Recall Precision However, high recall scores indicate the net is cast wide enough to correctly forecast conflicts that DO occur (errors fall on the side of caution). Low precision scores (high false positive rates) in the out years indicate that the world was more stable than would have been expected given macro-structural conditions.

8. Independent Variables • Caloric Intake: Estimate of the average number of calories consumed per person, per day. • GDP per Capita: Annual gross domestic product per person measured in constant 1995 U.S. dollars. • Male/Female Infant Mortality: Number of deaths of male and female children under 1 year of age per 1,000 live births. • Life Expectancy: Average life expectancy (males and females combined). • Youth Bulge: Ratio of population aged 15-29 to those aged 30-69. • Among others……..

10. Dependent Variable: Index of Instability • Maximum level/intensity of conflict per country-year; source: KOSIMO Data Project, Heidelberg Institute of International Conflict Research (HIIK),1975-2003. http://www.hiik.de/de/index_d.htm • Represents a high threshold of instability 3 Levels of instability intensity Instability Levels High intensity (if combined probability > 67% Moderate intensity (if combined probability > 67% None/Low intensity (if combined probability > 67% Key Assumption: country is unstable if (and only if) the government or its opponent(s)threatens or initiates a conflict to restore equilibrium or harmony with respect to its internal or external relations.

11. Prospects’ Algorithm • Prospects tool uses a classification algorithm, dubbed FASE (Fuzzy Analysis of Statistical Evidence) that characterizes the relationship between a target variable and the explanatory variables. • This relationship has to be learned, and this learning is done by selecting a training set of historical data and optimizing the algorithm based on historical relationships between the known values of the explanatory and target variables (i.e. post-diction). • At the time of training, a separate but still historical test set is also selected to generate performance metrics. This test procedure is conducted to evaluate the algorithm’s ability to calculate known but masked (unknown to the tool) target variable values in the test set from the algorithm learned in training.

12. Steps • 1. Compile a time series data set of the selected target variable’s intensity • 2. Compile a time series data set of candidate indicators associated with the target’s intensity • 3. Train an algorithm that explains the historical target intensities with the candidate indicators • 4. Calculate performance measures of the explanation from a time series of historical test data • 5. Generate vulnerability scores based on projections from the current value of the indicators • 6. Calculate a confidence level for the forecasts using the likelihood of occurrence at each intensity • 7. Iterate from step #1 for experimentation with alternative target variables • 8. Iterate from step #2 for experimentation with alternative explanatory variables or indicators

13. Typologies • Typologies are a way of sorting out relationships and developing hypotheses • Definition: a way to analyze all of the logical combinations of at least two variables.

14. Example of a Typology: James David Barber's The Presidential Character. Predicting Performance in the White House Two Baselines: • "activity-passivity” • "positive-negative view“

15. Presidential Types • Active-Positives: T. Roosevelt, F. Roosevelt, Truman, Kennedy, • Active-Negatives: Wilson, Hoover, L. Johnson, Nixon • Passive-Positives: McKinley, Taft, Harding, Reagan • Passive-Negatives: Coolidge, Eisenhower

16. Presidential Character & Amnesty for the Last 17 Presidents) • Active-positives granted 55% of all amnesties • Active-negatives granted 35% of all amnesties • Passive-positive presidents granted 7.5% of all amnesties • Passive-negatives granted 2.5 % of all amnesties • Active Presidents combined (i.e. both positives and negatives) granted 90% of all amnesties What explains this? • Presidents have averaged over 200 acts of clemency per year

17. Amnesty by the Numbers: 1900-1993 • Ford: 409 clemency actions taken (382 pardons and 27 commutations) or 35% of all requests • Reagan: 406 (393 pardons, 13 commutations) or 13% of all requests • G.H.W. Bush: 77 (74 pardons, 3 commutations) or 5% of all requests • Wilson: 2550 (995 pardons, 1403 commutations) or 37% of all requests

18. Content Analysis

19. Content Analysis • Content Analysis is a systematic attempt to examine some form of verbal or visual communication such as newspapers, diaries, letters, speeches, movies, or television. • Can be inductive or deductive. • Objective is to classify content • Can be either qualitative or quantitative (e.g. frequency counts). • Manifest Content: what explicitly appears in a text. • Latent Content: meanings implied by the written content that do not actually appear in the text.

20. Examples • Content Analysis of Video Games • Look at “E” (like rated G) games and look for violence, killing, and the use of weapons in the course of normal play. • Analyze The Daily Show and Assess for: • % of the stories addressing political topics • % of the stories addressing a public policy theme • % of the stories addressing international news in some way • % of the stories having to do with the news media • % of the stories that address celebrity/entertainment news • % of the guests who could be labeled serious (e.g. politicians, government officials, authors, etc.) • % of stories that involved traditional news media or video footage vs. % coming from network of cable news shows

21. How Times Have Changed

29. How Times Have Changed, Or Not. 2013 Fiat 500 Abarth Ad

30. Verbal Behavior Analysis • Verbal Behavior Analysis (VBA) is a content analytic technique designed to tap "styles of speaking with patterns of thinking and behaving" (Weintraub, 1989: 7). • Weintraub devised a system to analyze samples of speech (monologues elicited by a standardized procedure) to obtain the frequencies of occurrence of members of fourteen categories, not all of which are, strictly speaking, "syntactic" (e.g. long pauses and the rate of speech). • Then groups representing "no pathology" and various psychopathological syndromes are compared with respect to the frequencies with which these categories appear in speech.

31. VBA, Continued • Fifteen indicators: 1) I, 2) We, 3) Me, 4) Negatives, 5) Qualifiers, 6) Retractors, 7) Direct References, 8) Explainers, 9) Expressions of Feeling, 10) Evaluators, 11) Adverbial Intensifiers, 12) Non-personal References, 13) Creative Expressions, 14) Rhetorical Questions, and 15) Interruptions

32. VBA, Continued • frequent use of evaluators are indicative of individuals possessing a punitive conscience • frequent use of retractors convey impulsivity • high adverbial intensifier scores indicate persons "who see the world in black and white terms;" • frequent use of explainers signify tendencies toward rationalization • high qualifiers scores indicate anxiety and avoidance to commitment • frequent use of negatives signify negation and denial • frequent use of rhetorical questions indicate aggressiveness • frequent use of direct references indicates that the speaker has difficulty speaking and prefers to divert the attention of the audience • low use expressions of feeling convey an impression of aloofness • frequent use of creative expressions indicate creativity

33. Quick PT Analyzer

34. Group Exercise (counts as your 4th in-class exercise) • Identify up to a half a dozen themes • Associates specific words and Phrases to each theme • Rate overall intensity of each theme

35. Content Analysis of Newsfeed

36. Integrated Data for Events Analysis (IDEA): A Third Generation Event Framework • Most domain specific events data frameworks have been or can be mapped to IDEA, including WEIS, PANDA, MIDS, etc. • IDEA is a multi-framework compatible data standard designed to facilitate the comparison of data developed by different conceptual frameworks • The IDEA framework is currently comprised of 249 social, economic, environmental and political events

37. IDEA Class Hierarchy (N=249) Level 0: All phenomena Level 1: Animal incidents, Human actions, Human conditions, Other incidents Level 2: Animal attack, Accident, Accuse, Animal death, Agree, Animal illness, Cognitive state, Complain, Consult, Human death, Demand, Demonstrate, Deny,Economic activity, Endorse, Economic status, Expel, Force Use, Grant,Human illness, Natural disaster, Other animal incident, Other human action, Other human condition, Other incident, A&E Performance, Promise, Propose, Reject, Request, Reward, Comment, Sanction, Seize, Sports contest, Threaten, Warn, Yield Level 3: Abduction, Affective state, Agree or accept, Alerts, Promise to mediate, Apologize, Arrest and detention, Ask for material aid, Request protection, Assure,Beliefs and values, Criticize or denounce,Balance of payments, Break relations, Call for action, Unconventional weapons attack, Acknowledge responsibility, Extreme climactic condition, Collaborate, Crowd control, Commodity prices, Demand aid, Decline comment, Demand ceasefire, Default on payment, Defy norms, Demand information, Demand mediation, Demand meeting, Demand protection, peacekeeping, Demand policy support, Demand rights, Demand withdrawal, Discussion, Drought, Declare war, Corporate Earnings, Earthquake, Ease sanctions, Extend economic aid, Extend humanitarian aid, Extend military aid, Empathize, Equity prices, Exchange rates, Formally complain, Flood, Forgive, Executive adjustment, Halt discussions, Infectious human illness, Host a meeting, Hurricane, Informally complain, Improve relations, Interest rates, Extend invitation, Judicial actions, Armed force blockade, Armed force mobilization, Armed force display, Covert monitoring, Armed force threats, Non-infectious human illness, Other physical force threats, Radioactive leak, Optimistic comment, Physical assault, Pessimistic comment, Protest demonstrations, Praise, Promise material support, Promise policy support, Offer to mediate, Offer to Negotiate, Offer peace proposal, Armed actions, Refuse to allow, Ratify a decision,Real estate prices, Reduce or stop aid, Reduce routine activity, Release or return, Currency reserves, Riot Reject proposal, Investigate Seize possession, Provide shelter, Solicit support, Hazardous material spill, Rally support, Strikes and boycotts, Tornado, Transactions, Sanctions threat, Tsunami, Non-specific threats, Give ultimatum, Travel to meet, Volcano, Elect representative, Wildfire, Yield to order, Yield position Level 4:Armed force air display, Missile attack, Ask for economic aid, Ask for humanitarian aid, Ask for armed assistance, Assassination, Agree to mediation, Agree to negotiate, Agree to peacekeeping, Agree to settlement, Impose restrictions, Beatings, Border fortification, Break law, Armed force border violation, Chem-bio attack,Private transactions, Private default on payments, Impose censorship, Armed battle, Bodily punishment, Coups and mutinies, Criminal arrests, De-mining, Demobilize armed forces, Earnings above expectations, Earnings below expectations, Ease economic sanctions, Ease military blockade, Equity prices down, Equity prices up, Evacuate victims, Grant asylum, Government transactions, Government default on payments, Artillery attack, Reduce or stop humanitarian assistance, Halt negotiation, Halt mediation, Reduce or stop economic assistance, Political flight, Reduce or stop military assistance, Reduce or stop peacekeeping, Hostage taking and kidnapping, Investigate human rights abuses, Security alert, Downward trend in interest rates, Investigate war crimes, Upward trend in interest rates, Hijacking, Torture, Armed force alert, Mediate talks, Mine explosion, Armed force activation, Armed force occupation, Armed force naval display, Engage in negotiation, Nuclear alert or test, Disclose information, Protest altruism, Small arms attack, Protest procession, Political arrests, Protest obstruction, Protest defacement, Promise economic support, Promise humanitarian support, Promise military support, Reject ceasefire, Relax curfew, Request mediation, Request an investigation, Reject mediation, Nuclear attack, Reject peacekeeping, Reject proposal to meet, Relax censorship, Reject request for material aid, Return, release person(s), Return, release property, Relax administrative sanction, Reject settlement, Request withdrawal or ceasefire, Suicide bombing, Sexual assault, Threaten forceful attack, Threaten forceful blockade, Threaten to boycott or embargo, Threaten biological or chemical attack, Armed force troops display, Threaten to halt negotiations, Threaten to halt mediation, Threaten nuclear attack, Threaten forceful occupation, Threaten to reduce or break relations, Threaten to reduce or stop aid, Observe truce, Threaten war, Vehicle bombing, Veto

38. Selected Branches (IDEA Event Framework) Allphenomena HumanactionHumanconditionsAnimalincidentsOtherincidents Other human conditionHumanillnessEconomicstatusHuman deathCognitivestate Infectious human Non-infectious Affective stateBelief illnesshuman illness sand values BalanceCommodityDebtEquityExchangeRealCurrency reserves of paymentspricesyieldspricesratesestate prices Red signifies a terminal event (i.e., the lowest node on a branch)

39. Basic Structure of Extracting Meaning from a Report Subject Verb Direct Object/Indirect Object Examples: US President George W. BushblastedIraqi President Saddam Hussein for defying the no-fly zone. The U.S.delivered$20 million of humanitarian aid to Afghanistan yesterday. Subject (source), Verb (event), Direct Object/Indirect Object (target)

40. Variables Coded from Each Clause 1) ID (auto-generated, unique ID, 2) Sentence ID, 3) Event ID, 3) Event Date, 4) Report Date, 5) Event Place (e.g. Baku), 6) Event Administration (e.g. Azerbaijan), 7) Source Value (e.g. President IlhamAliyev 's grip), 8) Source Name (e.g. AZJ), 9) Source Administration (AZJ), 10) Source Level (e.g. INDI), 11) Source Sector (e.g. NEXE), 12) Event Negated, 13) Event Status (e.g. past, ongoing, foreshadowing), 14) Event Type (e.g. conflict/cooperation), 15) Is Flagged (i.e. pre-defined search terms), 16) Event Form (i.e. IDEA code, 17) Event Value (literal value), 18) Target Value (literal value), 19) Target Name, 20) Target Admin, 21) Target Level, 22) Target Sector, 23) Information Value, 24) Information Name, 25) Information Admin, 26) Information Level, 27) Information Sector, 28) Locus, 29) Affect, 30) Mechanism, 31) Injury, 32) Damage From these 32 variables we create 50 + additional variables in a post-parse process

41. Example Description of an IDEA Event Form IDEA Event Code: 2122 Name: Criminal arrests and detentions Description: Arrests and detentions explicitly characterized as criminal Usage Notes: Example*: French police on Tuesday arrested a man trying to sneak through Paris airport customs with a boa snake hidden in his underpants, an airport spokeswoman said. *Source = blue, Event = red and Target = green

42. Wordnet’s 15 Senses for the Verb “kill”) 1. kill -- (cause to die; put to death, usually intentionally or knowingly; "This man killed several people when he tried to rob a bank"; "The farmer killed a pig for the holidays")2. kill, defeat, vote down, vote out -- (thwart the passage of; "kill a motion")3. kill -- (cause the death of, without intention; "She was killed in the collision of three cars")4. stamp out, kill -- (end or extinguish by forceful means; "Stamp out poverty!")5. kill -- (be fatal; "cigarettes kill"; "drunken driving kills")6. kill -- (be the source of great pain for; "These new shoes are killing me!")7. kill -- (overwhelm with hilarity, pleasure, or admiration; "The comedian was so funny, he was killing me!")8. kill -- (hit with so much force as to make a return impossible, in racket games; "She killed the ball")9. kill -- (hit with great force, in sports; "He killed the ball")10. kill -- (deprive of life; "AIDS has killed thousands in Africa")11. toss off, bolt down, belt down, pour down, down, drink down, kill -- (drink down entirely; "He downed three martinis before dinner"; "She killed a bottle of brandy that night")12. kill, obliterate, wipe out -- (mark for deletion, rub off, or erase, as of writings; "kill these lines in the President's speech")13. kill -- (tire out completely; "The daily stress of her work is killing her")14. kill -- (cause to cease operating; "kill the engine")15. kill -- (destroy a vitally essential quality of or in; "Eating artichokes kills the taste of all other foods") SeeWordNet

43. Sense 2 "Hypernyms (this is one way to...)" of verb "kill" Sense 2 kill, defeat, vote down, vote out -- (thwart the passage of; "kill a motion • veto, blackball  -- (vote against; refuse to endorse; refuse to assent) • oppose, controvert, contradict - (be resistant to; "The board opposed his motion.“ • refute, rebut - (overthrow by argument, evidence, or proof; “the speaker refuted his opponent's arguments") • renounce, repudiate (cast off or disown; "She renounced her husband“) • reject -- (refuse to accept or acknowledge; "I reject the idea of starting a war"; "The journal rejected the student's paper") • judge -- (form an opinion of or pass judgment on)

44. Sense 2: Container (synset) of verb "kill" judge reject renounce; repudiate refute; rebut oppose; controvert; contradict veto; blackball kill; defeat; vote down; vote out

45. Semantic Framework: Noun Classes Top Level All agents Level 1 True agents (political actors) Pseudo agents (other actors, like the environment) Level 2 Civil society agents, Government agents Intangible things, Tangible things Level 3 Armed civilian groups, Artists, Athletes, Body parts, Communication, Events, Human actions, Businesses, Candidates, Civic group agents, Human artifacts, Human cognition, Human attitudes, Criminals, Detainees, Diplomats, Educators, Natural environment, Status, Time-related Ethnic agents, Farmers, Health care agents, phenomena Judiciary, Legislators, Mass media, Migrants, Military, National executive, Nominal agents, Occupations, Officials, Political opposition, Political parties, Philanthropic agents, Religious agents, Royalty, Sub-national officials, Students, Unions Level 4 Arabs, Bosnian-Croats, Bosnian-Moslems, Animals, Ancient beliefs, Disease, Bosnian-Serbs, Christians, Cult, Christian-Orthodox, Food, Health conditions, Historical figures, Hindu, Insurgents, Jew, Kurds, Moslems, Ideology, Human languages, Legislation, Peace-keeping forces, Paramilitary groups, Location, Markets, Human-made Police (note that the entries at this level are for conditions, Monetary units, Non-gun weapons, illustration purposes only; a comprehensive list of Protest actions, Plants and flora, Polls & surveys, entries is included in actual data development) Travel to meet, Violent actions, Weapons, Weather conditions Level 5 (greater differentiation among agents is possible Accident, Apology, Assassination, Balance of through user specification) payments, Biological agent/weapon, Bombing action, CBR weapons use, Censorship, Chemical agent/weapon, Commodity prices, Debt yields, Drought, Earthquakes, Equity prices, Exchange rates, Explosive device, Floods, Firearms, Harassment, Hurricanes and typhoons, Infectious disease, Litigation, Military actions, Military hardware, Monetary reserves, Nuclear devices, Protest altruism, Real estate prices, Military raids, Riot, Rapes, Shooting, Strike and boycott action, Tornados, Tsunami, Volcanic events, Wild fires Level 6 Biological weapons use, Car bomb, Car bombing, De-mining vehicle, Grenade/RPG, Grenade/RPG use, Mine explosions, Mines, Missile, Missile attack

46. Synset Example: Selected Entries of Nouns and Noun Phrases Mapped to the Noun Class “Migrant” (WordNet) abandoned, abandoned person, bag lady, beggar, beggarman, beggarwoman, bird of passage, bum, castaway, deportee, derelict, displaced person, internally displaced person, dosser, down-and-out, DP, IDP, drifter, evacuee, exile, foundling, gamin, have-not, hobo, homeless, homeless person, immigrant, mendicant, migrant, nomad, orphan, outcast, outcaste, panhandler, pariah, poor person, profligate, ragamuffin, rake, refugee, rip, roamer, roue, rover, shipwreck survivors, quatter, squatter, stateless person, street arab, street person, sundowner, tatterdemalion, throwaway, tramp, transient, urchin, vagabond, vagrant, waif, wanderer

47. From Syntax to Events The Readergenerates and maps “events,” or who does what to whom when where & how • syntax (from the read module) • semantics (from an external dictionary) and • user-specified information (from a protocol) into data matrices that can be used in statistical and other analyses.

48. Mapped to (user defined) Events • The mapping procedure is guided by a user-defined set of dictionaries or protocol. The (IDEA) protocol maps specific words and phrases to their various meanings. • Relevant behavioral referents are considered events, around which the event data matrix is built. Each event is linked to its actors who are identified as individuals, groups, organizations or states.

49. The Reader & IDEA Operate together to support monitoring and interactive assessment of evolving conflict situations,

50. Automated Coding • Advantages of automated coding no longer in dispute (exceptions: idiosyncratic text, low N) • As good as humans (see King and Lowe article) • 100% Consistent (could be consistently bad but it is consistent) • 100% Transparent • Flexible & Extensible