1 / 23

IT Research Seminar February 10, 2003

IT Research Seminar February 10, 2003. Measuring IS Success: Quest for the Dependent Variable in IS Research The Journey Special IT Research Seminar William H. DeLone Kogod School of Business February 8, 2008. IS Success Research Stream.

quasim
Download Presentation

IT Research Seminar February 10, 2003

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. IT Research Seminar February 10, 2003 Measuring IS Success: Quest for the Dependent Variable in IS ResearchThe JourneySpecial IT Research SeminarWilliam H. DeLoneKogod School of BusinessFebruary 8, 2008

  2. IS Success Research Stream • UCLA Dissertation – Successful use of IS by SMEs; MISQ 1988 • IS Success: Quest for the Dependent Variable; ISR 1992 = “DeLone & McLean Success Model” • Ten Year Update; JMIS, Spring 2003 • Measuring E-Commerce Success, International Journal of Electronic Commerce, Fall 2004 • IS Success Models, Dimensions, Measures, and Interrelationships, under review European Journal of Information Systems

  3. Research Motivation • UCLA measurement course (Mason & Swanson) • Peter Keen’s 5 research challenges for MIS (ICIS 1) – What is the dependent variable? How does MIS establish a cumulative tradition? • Dependent variable for PhD study in SMEs; Use & Impact • If you can’t measure it, you can’t research it

  4. Quest for MIS Dependent Variable (ISR, 1992) – “IS Success” • Purposes • Organize & summarize MIS research related to defining the dependent variable • Measure progress on defining the dependent variable • Improve IS research practice • Contribute to “Cumulative Tradition” – compare apples to apples

  5. Theoretical Underpinnings • Mason’s 1978 article on measuring information output • Production->Product->Receipt->Influence on Recipient->Influence on System • Mason’s work was based on Shannon & Weaver’s 1949 Theory of Communications book – Levels of communications measurement = technical, semantic, effectiveness • DeLone & McLean Success Categories • System Quality->Information Quality-> Use->Satisfaction->Individual Impact-> Organization Impact

  6. Methodolgy • Literature review • IS Articles from 1981 to 1988 • Framework/model for organizing success measures • Empirical measures grouped into six success categories

  7. Figure 1 D&M IS Success Model System Quality Use Individual Impact Organizational Impact Information Quality User Satisfaction D & M IS Success Model

  8. Results/Conclusions • A simple and parsimonious framework for organizing IS success measures • IS Success - multi-dimensional and interdependent construct • Selection of measures is contingent on objectives and context of study • Reduce IS Success measures; build on existing measures=> “cumulative tradition” – comparison of results • Need for organizational impact measures

  9. The Challenge • “ This success model clearly needs further development and validation before it could serve as a basis for the selection of appropriate IS measures.” (p.88)

  10. Ten-Year Update (JMIS 2003) • Purposes • Model Utility • Validate the Model – Causal relationships • Update the Model to recognize the changes in IS • Assess progress in IS success measurement

  11. Utility of D&M Success Model • Cited by more than 285 refereed journal and proceedings articles between 1993 and 2002 (according to a recent study in CAIS vol. 20, ISR 1992 article is the most cited article in MIS over the last 15 years; > 400 citations) • Most articles used the Model “as a drunkard uses a lamppost for support rather than illumination.” Statement was rejected by editor.

  12. Model Validation • Seddon & Kiew (1994) validated 4 of the proposed associations • Rai et. al. validated overall model using goodness of fit tests (ISR 2002) • Fifteen additional empirical studies validated one or more proposed associations

  13. Updated Model • IS move from production function to production & service function => importance of service quality • Impacts on whom? Individuals, groups, org., industry, economy => Net Benefits dimension with contextual definition

  14. Updated Model

  15. Assessment & Conclusions • D&M IS Success Model supported and validated • More careful attention to multidimensionality of IS Success • Confusion between Independent & Dependent variables • Operationalization of the model is contextual (Seddon et. al.) • Progress in parsimonious measure development is slow • System Use is misunderstood and undervalued • Use and satisfaction are not a substitute for Net Benefits measures (Yuthas & Young, 1998)

  16. Recent Advances in Success Measurement

  17. Application of Model: E-Commerce Success (IJEC 2004) • Premise: E-commerce does not need a new measurement paradigm but some new measures • Apply the D&M IS Success Model for measuring E-Commerce Success • Literature Review and classification of emerging e-commerce effectiveness measures • Case examples of application of IS Success Model to E-Commerce success measurement

  18. Application of Model: ERP Success • Article by Sedera & Gable (ICIS 2004) – Government & University ERPs • Most comprehensive empirical test of model • Four dimensions of IS Success – • System quality • Information quality • Individual impact • Organizational impact • ** 27 Item measures

  19. Validated Measures for IS Success Source: Sedera and Gable (2004) • System Quality - ease of use, ease of learning, user requirements, system features, system accuracy, flexibility, sophistication, integration, and customization • Information Quality – availability, usability, understandability, relevance, format, and conciseness • Individual Impact – learning, awareness/recall, decision effectiveness, and individual productivity • Organizational Impact - organizational costs, staff requirements, cost reduction, overall productivity, improved outcomes/outputs, increased capacity, e-Government, and business process change • What happened to Use and User Satisfaction?

  20. Measuring Systems/Information Usage (Don Marchand) • Don Marchand, Professor of Strategy & Information Management at IMD – Switzerland • 20% of value realization is in deployment: 80% of value realization is in information and IT USAGE • Challenge – Information Usage is difficult to see, measure and manage

  21. Measuring System Usage (Burton-Jones & Straub, ISR 2006) • Problem – over-simplified measures of use; e.g. duration of use and breadth of use • Importance of context & purpose • Elements of usage include: systems, user and task • Proposed Dimensions of Systems Usage – Cognitive Absorption (engagement) + Deep Structure (use of system features that support a specific task) • Systems Usage empirically related to task performance

  22. Current Research • Measuring IS Success: Models, Dimensions, Measures and Interrelationships under review at European Journal of Information Systems • Assessing the state of IS Success Measurement (via D & M Model) • Summarizing empirical literature, 1992 to 2006 • Contributions • Summarizes measures used for each dimension of success • Validates significant relationships for 10 of the 15 causal relationships in the Updated Success Model based on empirical studies

  23. Conclusions • DeLone & McLean IS Success Model remains the most popular, comprehensive framework for guiding the development of the dependent variable in IS research and for comparing results • More IS researchers are using the D&M Model to inform and guide their measurement of the dependent variable rather than to merely justify their choice of measures • Information quality and information use are under studied • Satisfaction is over used as a surrogate for success; therefore much information is lost • Bias toward ease of data collection threatens rigor and understanding

More Related