slide1
Download
Skip this Video
Download Presentation
An Alternative Theoretical Framework to Analyze Failures in Decision Making: Application to Student Dropout in Asynchronous Learning Environments

Loading in 2 Seconds...

play fullscreen
1 / 39

An Alternative Theoretical Framework to Analyze Failures in ...An - PowerPoint PPT Presentation


  • 501 Views
  • Uploaded on

An Alternative Theoretical Framework to Analyze Failures in Decision Making: Application to Student Dropout in Asynchronous Learning Environments Naj Shaik PhD [email protected] University of Illinois at Urbana-Champaign Ninth Sloan-C International Conference on Asynchronous Learning Networks

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'An Alternative Theoretical Framework to Analyze Failures in ...An' - oshin


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

An Alternative Theoretical Framework to Analyze Failures in Decision Making: Application to Student Dropout in Asynchronous Learning Environments

Naj Shaik PhD

[email protected]

University of Illinois at Urbana-Champaign

Ninth Sloan-C International Conference on Asynchronous Learning Networks

Nov 14-16, 2003

slide2

Student dropout is rarely the result of a single point of decision failure but typically occur when the system breaks down at multiple points along the chain

    • Adapted from James Reason. Human Error. Cambridge University Press, 1990.
  • Retention is everybody\'s business
    • Provost, Virginia Commonwealth University
retention rates
Retention Rates

Historical Data

  • Between 1880 & 1980 the rate of retention in US was around 55% (Tinto, 1982) which is higher than Australia and UK.

Current Research

  • Traditional Students: Second year retention rate
    • 80% for member institutions, 87% for selective institutions and 69% for less selective institutions (Consortium for Student Retention Data Exchange 2001).
    • 74% for private and 72% for public 4-year institutions (ACT 2001).
  • Non-Traditional Students: Second year retention rate
    • Retention rates for distance education programs are lower than those for the traditional on-campus programs and courses (Dutton, Dutton, and Perry, 2001; Terry, 2001; Carr, 2000).
significance of retention issue
Significance of Retention Issue
  • A number of federal and state agencies request the reporting of retention data.
  • U.S. Department of Education plans to emphasize raising retention rates in higher education while working with Congress to reauthorize the Higher Education Act (reported in Chronicle of Higher Education, Dec 2002).
  • It is used as an indicator of academic quality in U.S. News and World Report annual college rankings (Graham and Morse, 1998).
  • A low retention rate reflect poorly on a program and impact program promotion and recruitment efforts.
conventional theoretical models frameworks
Conventional Theoretical Models / Frameworks
  • Over the last 3-decades a number of frameworks have been proposed to explain the student dropout phenomena based on student behavior.
    • Sociological Model (SM): Spady
    • Student Integration Model (SIM): Tinto
    • Social & Personal Beliefs Model (SPBM): Fishbein & Azjen
    • Student Attrition Model (SAM): Bean
    • Integrated Retention Model (IRM): Cabrera et. al.
    • Congruence Model (CM): Boshier
    • Chain of Response Model (CRM): Cross
    • Identity Theory Model (ITM): Witte et al.
    • Composite Persistence Model (CPM): Rovai
    • Rational Choice Model (RCM): Manski & Wise
student integration model sim
Student Integration Model (SIM)

Student Integration Model is based on

    • Durkheim (1951) theory of suicide
    • Van Gennep (1960) theory of rites of passage
    • Predictive version of Spady sociological model.
  • Individual pre-entry college attributes (family background, skill and ability, prior schooling) form individual goals and commitments.
  • The individual’s goals and commitments interact over time with institutional experiences (the formal and informal academic and social systems).
  • The extent to which the individual becomes academically and socially integrated into the academic and social systems of an institution over time determines the decision to dropout.
  • The absence of successful integration arises due to incongruence (odds with the institution) and isolation (disconnected from the institution).
enhancements to student integration model
Enhancements to Student Integration Model
  • Student Attrition Model (Bean): importance of external variables such as finances, family responsibility and support etc for non-traditional students.
  • Congruence Model (Boshier): Influence of environmental variables such as ….
  • Integrated Retention Model (Cabrera et al.): an integrated framework based on the overlapping factors between SIM (Tinto) & SAM (Bean).
  • Identity Theory (Witte et al.): non-traditional student’s identity development and its influence on student behavior.
  • Composite Persistence Model (Rovai): includes learning environment, nature of teaching and learning etc to the evolving framework.
synthesis conventional fit models
Synthesis: Conventional “Fit” Models
  • Dropout is explained through student-institution “FIT" by looking at student, institutional, and environmental variables.
  • Absence of successful integration is attributed to incongruence (odds with the institution) and isolation (disconnected from the institution). (Tinto)
  • Student Responsibility: “ … either low goal commitment or low institutional commitment can lead to dropout.” (Tinto, 1976. p.96).
observations conventional fit models
Observations: Conventional “Fit” Models
  • Incongruence: “… students must disassociate … from membership in the communities of the past, [such as] …. family, the local high school, [etc.] …(Tinto, 1987, p.95)” to become fully incorporated with the college community.
    • Implications: Students to “commit a form of cultural suicide to be academically successful” (Tierney, 1999, p.85).
  • Transition: The reference to theory of rights of passage is inconsistent with anthropological basis.
    • Rights of passage signifies movement within the same culture and not between cultures (Tierney 1992).
  • Integration / Fit appeals to commonsense but hides the complexity and over simplifies the issue (Draper, 2003).
observations conventional fit models10
Observations: Conventional “Fit” Models
  • Empirical Support (Draper, 2003)
    • Research is based on weakly consistent evidence.
    • No comparative research testing competing frameworks.
    • Absence of controlled experiments.
  • Absence of a psychometric-based standardized instrument for data collection.
  • Absence of standard protocols to administer instrument.
  • Generalization not possible due to data constraints.
  • In the absence of a common metrics valid time-series and cross-program comparisons on dropout rates are difficult to interpret (National Research Council, 1996).
  • The list of explanatory variables is growing as we look at the number of suggested enhancements to the ‘Fit’ framework.
alternative theoretical framework
Alternative Theoretical Framework
  • Based on cognitive architecture and draws on the field of cognitive psychology particularly the contributions of
    • Rasmussen: Skills- Rules- & Knowledge Framework
    • Norman: Error Taxonomy
    • Reason: Generic Error Modeling System
  • The dropout phenomenon is described as attributed to divergent origins from technological failure to managerial oversight to organizational weakness.
  • The goal is to highlight the importance of organizational environment relative to individual performance.
rasmussen s step ladder framework
Rasmussen’s Step Ladder Framework

Interpretation Evaluation

IdentificationGoal Selection

ObservationProcedure Selection

Problem  Activation  Execution Problem Solved

  • Represents constraints in the work environment.
  • Describes three levels of human behavior in terms of increasing order of cognitive processing.
  • Comprises a three level hierarchy defined by skilled-based, rule-based and knowledge-based behavior.
  • At the lower level actions are more specific and rigid while at the higher level actions are more general and flexible.
rasmussen s step ladder framework13
Rasmussen’s Step Ladder Framework
  • Skill-Based (SB) corresponds to effortless routine actions that take place as smooth, automated and highly integrated patterns of behavior (driving under normal weather conditions).
  • Rule-Based (RB) is applicable to tacking unfamiliar anticipated problems. Solutions to these problems are governed by stored rules which are learned by explicit training and by experience (driving on a slippery road).
  • Knowledge-Based (KB) is applicable to unfamiliar unanticipated problems. It requires reasoning to diagnose and solve problems, identify deep features about a situation, adapt plans and responses to the needs of the situation in the absence of pre-packaged solutions (driving during rush hours on Chicago lake shore in winter).
reason generic error modeling system
Reason: Generic Error Modeling System

Error Types: Slips, Lapses, and Mistakes

  • Slips relate to observable actions associated with attention failures (Skill-Based behavior).
  • Lapses involve memory failures (Skill-Based behavior).
  • Mistakes associated with complex problem solving and relate to a plan that is inadequate to achieve the objective (Rule-Based or Knowledge-Based)

Error Source: Latent and Active Errors

  • Latent errors are removed from the direct control of the front-line operators and relate to organization level decisions, communication processes, business practices and rules, deficiencies in training etc.
  • Active errors include operator errors at system front end.

Error Chain: Swiss cheese metaphor.

reason generic error modeling system15
Reason: Generic Error Modeling System

Error Chain & Swiss cheese metaphor

  • Numerous factors behind any single decision failure.
  • A single failure analyzed out of context might seem benign.
  • Swiss Cheese metaphor
    • to explain the influence of latent factors on decision failures.
    • need to work backwards to uncover latent factors on decision failures.
    • often a potential error chain is broken by redundancy in the system (feedback mechanisms).
  • Organizational Influences (latent errors)  Unsafe Supervision (latent error)  Preconditions for unsafe acts (latent error)  Unsafe acts (active failures).
reason s swiss cheese metaphor
Reason’s “Swiss Cheese” Metaphor

Organizational Influences

Management Hierarchy

Organization Culture

Organization Climate

Organizational Processes

Institutional Policies

Institutional Procedures

Rewards Structure

Nature of Inter-departmental Cooperation

Line Management Deficiencies

Instructional Technology Resources

Human Resource Constraints

Budget Constraints

Business Rules

Business Practices

Work Environment

Front-end System Users

User’s Knowledge, Experience, Training, & Workload

Team Work

Demands of the task

Latent Factors @ Blunt End

Active Triggers @ Sharp End

reason s swiss cheese metaphor17
Reason’s “Swiss Cheese” Metaphor

Blunt End

Sharp End

Latent Conditions

Organizational Factors

Latent Conditions

Latent & Active Conditions

Line Management Factors

Supervision

& Control

Triggers

Individual / Team Factors

Preconditions

Active Conditions

Active Conditions

Gaps or Weakness

in System Defenses

System Defenses Working

reason s swiss cheese metaphor18
Reason’s “Swiss Cheese” Metaphor

Blunt End

Sharp End

Latent Conditions

Organizational Factors

Latent Conditions

Latent & Active Conditions

Line Management Factors

Supervision

& Control

Triggers

Individual / Team Factors

Preconditions

Active Conditions

Active Conditions

Failure in Decision Making

Trajectory of system failure when Loopholes in the firewall of the system lineup

student dropout processes inventory
Student Dropout: Processes Inventory
  • Recruitment
  • Admissions / Registration
  • Orientation
  • Advising
  • Teaching & Learning Environment
recruitment process
Recruitment Process
  • Identification of Recruits
    • direct mailings, loading test score data, high school and community campus visits, and campus events.
    • graduate students recruitment is decentralized and may involve separate recruiting efforts.
    • international students
  • Managing potential recruits information
  • Assigning and Managing recruiters (includes personal interviews with the prospects)
  • Developing a customized communication plan
latent failures recruitment process
Latent Failures: Recruitment Process

Non-standardized Protocols

  • Personal interview with the recruits
  • Nature and scope of student communication plan
  • Address needs of special students groups: gender, ethnicity, athletes, economic status etc.
  • GPA computations for prospective graduate student (which degree? undergraduate, graduate, multiple degrees)

Data / information requirements (in a fragmented system)

  • Availability of test scores and percentiles
  • Data needs of recruiters, admission councilors, departments, special groups on campus.
  • Type and scope of tracking regarding the interventions / special programs
latent failures culture recruitment
Latent Failures (Culture): Recruitment

Management emphasis is on enrolment relative to retention even thought it is not a sound economic policy.

A small decrease in student attrition can result in a significant gain in revenue to the institution.

  • A public institution with 2000 freshman enrollment and a dropout rate of 30% can save $1 million for a 10% decrease in the dropout (Levitz, Noel & Richter 1999).
  • 1% increase in the freshman retention will result in a $500,000 gain in additional revenue to the institution (Johnson, 1997)
  • Community college with an enrollment of 15,000 students, and a full-time equivalent student bringing in state funds of $5,000, a 1% decrease in retention increases the tuition revenue by $750,000 (Dr. Jing Luan, Cabrillo College)
admission registration process
Admission / Registration Process
  • Evaluation of prospective student application
    • auto evaluate based on the criteria
    • flag applicants that do not meet the criteria
  • Review by administrators (initial review)
    • administrative officers
    • college / department
      • Look for other factors and background information
  • Review Committees (liberal arts, engineering, etc)
    • Look for additional details
    • Admission based on guidelines
    • Hold for dean’s review
    • Admission decision
  • Number of subjective factors are involved in admission decisions
latent failures culture registration
Latent Failures (Culture): Registration

Registration procedures to mask some attrition

  • British Open University beginning students register on a temporary basis. If they withdraw within the three months of starting the program, their official registration will not show up on the university records (Guri-Rosenblit, 1999).

Definition

  • Definition of a dropout varies widely among institutions. The terms retention, attrition, departure, withdrawal are sometimes used interchangeably.
  • Time-to-degree completion or average time to graduation is less accurate due to an increasing number of non-traditional students and the stop-out phenomena.
orientation process
Orientation Process

Agencies and Scope

  • University level, College level, & Department level
  • Special interest groups / programs
    • Minority Affairs Program
    • Athletics Program
    • Honor’s program
  • Time Duration (1-2 days)

Processes

  • Faculty: Meet in small groups with faculty.
  • Peers: Interact with peers and other freshmen.
  • Learn about the institution’s academic standard.
  • Support Services: Become familiar with services.
  • Advisor: Meeting to map out academic strategies.
  • Placement: Complete placement testing (if needed ).
  • Pre-Registration: Select fall classes, and register.
advising process
Advising Process
  • Decentralized process and varies from college to college.
  • Categories of academic advisors
    • Academic Professional (AP)
    • Faculty
    • Graduate Student
  • Advising Types
    • New Students
    • Continuing Students
      • Formal advising relationship with a faculty or AP
      • Informal advising relationship
  • Advisor and Student responsibilities
  • Student to advisor ratio
    • UIUC college: between 7 to 60 students per advisor
  • Information system to support Advising
latent failures advising
Latent Failures: Advising
  • 60% of students reported that they are not satisfied with the quality of advising received at their college (National Survey reported in Cuseo 2003; Astin 1993).
  • Insufficient information, getting wrong information relating to instruction, financial aid and billing were some of the negative comments (Survey by Heverly 1999).
  • Need for better Advising Formats & Response Time
    • 24 X 7 Online with 24-hour response
    • Face-to-face regular, evening and weekend hours
  • Student to advisor ratio & quality of student advising
    • UIUC college: between 7 to 60 students per advisor
  • Decentralized / Fragmented Advising System
    • Sharing of student data and advisor notes across campus advising agencies.
  • Degree Audit Reporting System (DARS) not accessible to students.
latent failures culture advising
Latent Failures (Culture): Advising
  • Faculty contracts do not emphasize the importance of advising as a faculty responsibility (Teague & Grites, 1980).
  • Reward structure
    • Often blocks the ability to reward faculty who are genuinely committed to advising” (Creamer & Scott, 2000)
    • Only 12% of postsecondary institutions offered incentives or rewards that recognize outstanding advising of first-year students (Survey: Policy Center on the First Year of College, 2003).
  • Faculty promotion and tenure - advising is typically recognized by giving it only minor consideration (Habley, 1988).
  • Supplementary programs for (at-risk and ethnic) students groups but none for faculty.
teaching learning environment
Teaching & Learning Environment

Who is teaching the course

  • Research faculty
  • Adjunct faculty
  • Graduate student

Course delivery format

  • Asynchronous online courses (no instructor contact)
  • Mixed format online courses (with instructor contact)
  • Face-to-face format

Instructional Design

  • Course structure & role of instructor
  • Assessment and feedback
  • Support for different learning styles
  • Peer interactions & instructor interactions
  • Virtual community support
latent failures learning environment
Latent Failures: Learning Environment

Institutional commitment to teaching

  • Four-year and junior colleges: teaching emphasis.
  • Universities: research emphasis relative to teaching.

Instructional Design

  • Rigid course structure with pre-determined course topics.
  • Assessment – multiple choice items graded by TA.
  • Little support for different learning styles

Nature of instruction

  • Freshman courses with large class size often taught by non-tenured faculty / graduate students.
  • Corresponding course taught without any instructor
  • Online courses with all content online and no scheduled instructor contact.
  • Online courses with scheduled lecture / contact hours taught by untrained professionals.
latent failures culture faculty research
Latent Failures (Culture): Faculty Research

Rational Choice Model

  • The decision to enroll is a decision to initiate an experiment, a possible outcome of which is dropout. Lowering dropout levels would not necessarily make society better off. (: Manski, 1989).
  • Graduation decision is part of the overall labor market optimizing problem based on an assessment of returns from graduation and the cost of persistence to the student. (DeBrock et al. 1996).
  • Dropout does not necessarily reflect a lack of ability on the part of the student. Dropout cannot be treated as a failure on the part of the educational system.

Attrition is recognised as a perfectly normal condition within tertiary education. (Tinto1982; Bean 1980)

application to student dropout structural layer
Application to Student Dropout: Structural Layer

Organizational Influences

Management Hierarchy

Regulations: State & Federal

Organizational Culture: Research Emphasis

Institutional Priorities: Sports & Entertainment

Rewards Structure: Emphasis on Research

Enrollment Management: Focus on Recruitment

Accountability: Dropout

Line Management Deficiencies

Information System: Decentralized and Fragmented

Human Resource Constraints: Temporary Staff & Graduate TAs

Budget Constraints: Quality of Advising

Pedagogy: Converting traditional courses to online format

Front-end System UsersInstructor Knowledge, Experience, Training, & Workload

Absence of Team Work & Inter-department Cooperation

Learning Environment

Latent Factors @ Blunt End

Active Triggers @ Sharp End

application to student dropout
Application to Student Dropout

Blunt End

Sharp End

Decentralized / Fragmented Information System

Vague Procedures

Latent Conditions

Insufficient Tutoring Services

Organizational Factors

Latent Conditions

Advisor (Expertise)

Latent & Active Conditions

Line Management Factors

Supervision

& Control

Triggers

Individual / Team Factors

Preconditions

Active Conditions

Non-traditional Student

Active Conditions

State & Federal Regulations

Tenured Faculty with Research Focus

Institutional Culture: Recruitment Emphasis

Immature Freshman

Incomplete Student Plan

Support Services: Social

Forced Online Pedagogy (Traditional Course)

Large Class Size

Student Dropout

Trajectory of system failure when Loopholes in the firewall of the system lineup

application to student dropout process layer
Application to Student Dropout: Process Layer

Triggers

Incomplete information about the Prospect

Incomplete Communication Plan

Staff/Recruiter Workload

Institutional factors

Recruitment

Over-whelmed with Advising Information

Admission

Decentralized / fragmented Information System

Orientation

Immature Freshman

Over-worked Athlete

Advising

Asynchronous Online Course

Learning Environment

Decentralized Information System

Vague Recruitment Guidelines

Instructor Experience (TA)

Federal & State Regulations

Placement Credits

Transfer Credits

Support Services

Freshman Course (large class size)

Student

Dropout

Trajectory of system failure

when Loopholes in the firewall of the system lineup

System Defenses working

application to student dropout process layer35
Application to Student Dropout: Process Layer

Triggers

Incomplete information about the Prospect

Incomplete Communication Plan

Staff/Recruiter Workload

Institutional factors

Recruitment

Over-whelmed with Advising Information

Admission

Decentralized / fragmented Information System

Orientation

Immature Freshman

Over-worked Athlete

Advising

Learning Environment w/:

Instructor as Coach

Small class-size

Team-based Peer Learning

Relevant Projects

Instructor Interactions

Assessment & Feedback

Decentralized Information System

Vague Recruitment Guidelines

Placement Credits

Transfer Credits

Support Services

Student

Completes Course

System Defenses working when Loopholes in the firewall of the system gets deflected

interventions national level
Interventions: National Level

Federal & State Initiatives

  • National priority (mission is to educate).
  • Address accountability and legal liability issues.
  • Appropriations and student performance.
  • Student Attrition Management System (Shaik, 2003a).
  • Culture of Retention (Shaik, 2003b).

Student Attrition Management System

  • Based on standardized metrics and failures typology.
  • Centralized national database on student attrition.
  • Online web-based system with a user friendly interface.
  • Reporting (voluntary and mandatory) of dropout incidents at (institutional) process level.
  • Recovery management framework.
interventions organization level
Interventions: Organization Level

Assessment of work environment

  • Methodologies:
    • Cognitive Task Analysis (What is);
    • Cognitive Work Analysis (What should be);
    • Fault-tree Methodology;
  • Goal: identify latent errors & redesign environment.

Reward structure

  • Goal: to encourage retention by faculty and staff.

Research

  • Validated and replicable using time-series and cross-program data from the national database.
  • Reports on dropout to go beyond data collection.
  • Pattern Analysis to identify the causes and to discover robust patterns which can then be used as forecasts from operational and reporting data.
selected references
Selected References
  • Bean, J. P. (1980). Dropouts and turnover: The synthesis and test of a causal model of student attrition. Research in Higher Education, 12, 155-187.
  • Boshier, R. (1973). Educational participation and dropout: A theoretical model. Adult Education, 23, 255–282.
  • Cabrera, A. F., Nora, A., and Castaneda, M. B. (1993). College persistence: Structural equations modeling test of an integrated model of student retention. Journal of Higher Education, 64(2), 123-139.
  • Cross, K. P. (1981). Adults as learners. San Francisco: Jossey-Bass.
  • Draper, S. (2003). Tinto’s model of student retention. URL: http://www.psy.gla.ac.uk/~steve/localed/tinto.html
  • Fishbein, M. and Ajzen, I (1975). Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research. Reading, MA: Addison-Wesley.
  • Rasmussen, J. (1983). Skills, Rules and Knowledge; Signals, Signs and Symbols and Other Distinctions in Human Performance Models. IEEE Transactions on System, Man and Cybernetics. SMC-13 (3). 257-266.
  • Norman, D. (2002). The Design of Everyday Things. Basic Books.
  • Reason J. (1990). Human Error. Cambridge University Press.
  • Shaik, N. (2003a). Student Attrition Management System: An Organization with Memory – A Conceptual Framework . Working paper, Division of Academic Outreach, University of Illinois.
  • Shaik, N. (2003b). Culture of Student Retention: Alternative Framework. Working paper, Division of Academic Outreach, University of Illinois.
selected references39
Selected References
  • Spady, W. G., (1971) Drop-outs from Higher Education: An Interdisciplinary Review & Synthesis, Interchange, 64-85.
  • Seidman, A. (1996). Retention Revisited: RET = E Id + (E + I + C)Iv. College and University, 71(4), 18-20.
  • Tinto, V. (1993). Leaving College: Rethinking the Causes and Cures of Student Attrition, Chicago: University of Chicago Press.
  • Vicente, K. and Rasmussen, J. (1992). Ecological Interface Design: Theoretical Foundations. IEEE Transactions on Systems, Man, and Cybernetics, 22(4), 589-606.
  • Virginia Commonwealth University (2000) Retention 2000 Recommendations. http://www.students.vcu.edu/dsa/retention2000/advising
  • Witte J. W., Forbes, S. E. & Witte, M. M (2000). Identity Theory and Persistence: A Tentative Synthesis of Tinto, Erikson, and Houle. Journal of Integrative Psyhcology. Vol 2.
ad