Question 1 • How can we succeed? • in a Business School Context • Competition comes from Economics, Strategy, and Organizational Perspectives • which draw on more widely employed reference disciplines
Assumptions to Argue About • “Competition” implies a zero sum game • It is unusual to cite cognitive literature (see the work of experimental economics and behavioral accountants) • Our work does not cite economics, strategy, or organizational research: 3 dissertations: • Studied effects of different utility theory cost and probability parameters in softlifting experiment (Peace) • Studying effects of problem-solving cues on strategy development (Sampler) • Group spreadsheet debugging work cited group problem-solving literature (Joseph)
Assumptions to Argue About • Those fields are secure • Colleagues in strategy consider their field as the champion of identity crisis • If economics is Arnold Shwarzzeneger, strategy might be Woody Allen. • What is Cognitive IT? • Somewhere between Jean-Claude Van Damme and Regis Philbin • Our work does not cite widely-used reference disciplines • Which disciplines are we missing? Pure math? If that is required, we would have schools of Finance/Econ.
So, Regis, How do We Succeed? • Study Interesting Problems (or problems that will become interesting) • Attract something: • Students • Colleagues • Grants • Reporters
Publicity • Please visit your school’s PR function • Much of what we do is interesting to regular people, too. • I found new respect from others after: • Chatting with Chris Arnold on All Things Considered about applying results of response time research to Web browsing (1996—result of a Washington Post op-ed)
WSJ Report • Pam Sebastian’s column • Word of mouth’s interference on learning a package • Ironic Philippe Kahn juxtaposition • Also reported in Computerworld, etc. • ICIS & CACM articles
CNN TV • Study cited by CNN, Business Week, dozens of news-papers, 4 radio programs • To appear in CACM • Intended to show how a word processing aid needs to fit cognitive abilities of users
Question 2a: Are we Doomed always to be a Minority? • Yes, unless we act. • We must be represented on editorial boards and must not turn over all of the crown jewels to hostile forces • One person can make a difference: calmly noted the lack of any relevant track for HCI/cognitive research in ICIS 2003 to a program chair; now we have an entire mini-track as a direct result of that conversation • Our SIGs will help a great deal. • We should broaden our data collection beyond college sophomores at least once in a while • We should write carefully, write well, and explain effectively the relevance of our studies
Question 2b: What Happened to the Experiments of the 70s and 80s? • Experiments are hard to construct that have: • Basis in theory, but (like Goldilocks and the 3 Bears): • Not too soft (supported too little by previous studies): A stretch? • Not too hard (supported too much by previous studies): Obvious? • Relevance and Rigor • Realism of manipulation • Representativeness of sample • Reasonable magnitude of findings • Capture of appropriate constructs • How many studies can do all of these? Is it zero? Perhaps. • If not zero, do we read them as “guilty until proven innocent?” • A recent submission: a big flaw: we didn’t cite an article released 90 days after submission
Question 3: What Should We Do? • Let’s figure out how to fairly evaluate experiments: what bed is just right? • Let’s ask/plead/demand our journals to provide balanced editorial boards • Let’s market our work properly: • Don’t oversell it by promising silly implications • Don’t undersell it by hiding implications • Don’t try to sell it if no implications!
What to Do (continued)? • Make agreements to have regular, formal or informal research exchanges • Discuss experiments you are designing. • Be brutally critical at this stage; the best gift to your colleagues. • Withholding criticism is not being kind to them. • Pilot tests are not just for PhD Theses! Fix materials; debug procedures; ascertain magnitude of effect size to choose appropriate sample size.
What to Do (continued)? • Before a large investment in the experiment, inventory support for hypotheses • If support is meager, perhaps save your energy. • If support is widespread, emphasize your contribution • Before a large investment in subjects, • Do you own tasks before any other steps. • You will find obvious difficulties. Fix them. • Remove any confounds. • Make sure tasks are debugged.