Qualitative Assessment Two Definitions.
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
1. Qualitative Assessment “Qualitative inquiry cultivates the most useful of all human capacities—the capacity to learn from others.”
Halcom’s Evaluation Laws. Cited in Patton (1990). Qualitative Evaluation and Research Methods. pp. 7.
Moving Forward with Assessment
Difficult Dialogues Initiative
Ford Foundation, NY, NY
October 16-17, 2008
2. Qualitative Assessment Two Definitions “An inquiry process of understanding a social or human problem, based on building a complex, holistic picture, formed with words, reporting detailed views of informants, and conducted in a natural setting.”
Creswell (1994). Research design: Qualitative & quantitative approaches. Cited in Detlor (2004). Towards Knowledge Portals.
“Explores and tries to understand people's beliefs, experiences, attitudes, behaviors and interactions. It generates non-numerical data…”
3. Qualitative Assessment Tidbits 1899: W.E. Dubois, undertook first social survey in the United States.
1900’s: Early phase of qualitative research. Rooted in Social Investigation.
1950’s: Government provided grants in support of educational research. Preferred modality is qualitative research.
Late 1960’s: Qualitative research came into being. Development of socially responsive programs.
Late 1900’s: Accountability movement in education results in increasing use of qualitative research and assessment.
4. Why DDI & Qualitative Assessment? Well suited in areas that are complex, multidimensional sensitive and nuanced.
Illustrative DDI Project Titles-
Engaging Controversy: Religion, freedom and the topics of identity
Imagining the Future: Dissent, dialogue and the freedom to inquire
Practicing Pluralism: Interactive theater, campus climate and academic freedom
The open nature of qualitative approach allows the subjects to respond according to their own framework.
“In assessing the impact of a faculty development workshop in adoption of new pedagogy, the teachers’ experience, reasons for participating in the workshop or the students that she teaches may impact her view of the value of the workshop and her sense of its applicability.”
Quote from grantee.
Helps to understand a program / situation as a whole.
“..the holistic approach assumes that the whole is understood as complex system that is greater than the sum of its parts.”
Patton (1990). Qualitative Evaluation and Research Methods.
5. Why DDI & Qualitative Assessment? Continued To provide a richer, more descriptive picture of the information being gathered or responses to the questions being asked. For example, assessing and describing in a rich manner the quality of increased capacity for exchange between Arab and Israeli students.
On college campuses addressing pluralism issues, qualitative assessment can in itself serve as a process of inclusion and appreciation for multiple values and views.
6. SOME CHARACTERISTICS OF THE QUALITATIVE PROCESS
Process extends through data reporting, data analysis, and utilization of findings.
Researcher’s lens can impact data gathering and interpretation. Expertise in both assessment and subject area are significant.
It is complex and requires multiple iterations to better uncover information (experiences, perspectives, views).
Large volumes of data can accumulate quickly. Allocation of time and resources for data collection and analysis is imperative.
Value: You get out what you put into it!
7. Qualitative Approaches Used by DDI
FOCUS GROUPS: In qualitative assessment you go to the source (naturalistic group)
INTERVIEWS: direct source
PARTICIPANT OBSERVATIONS: To fully describe the process, you want to observe it
PORTFOLIO ASSESSMENT: inductive
FILM DOCUMENTATION: profiles and pictures
8. Assessment at its Core is about Student Learning Outcomes Question: How do we get from here to there?
Pluralism is ongoing work
Assessment is an ongoing process
Tools to assess student learning outcomes:
In classroom exercise (presentations, debate)
9. ENHANCING DDI INDICATORS OF SUCCESS IN QUALITATIVE ASSESSMENT Ensuring indicators of success are sound: Important particularly for subjects and areas that are considered outside of the core curriculum or where the topics are controversial:
“Unearth narratives about the complex history of race”
Quote from grantee.
Members check-in: This process can also serve to inform the constituencies and to begin process of change.
Triangulation: Using multiple sources to confirm findings.
Enlisting assessment advice and support from experienced and knowledgeable colleagues: Serves as a form of triangulation and check-in. Can address gaps in assessment capacity.
Detailed reporting: Necessary to illustrate how the assessment team arrived at findings and to project the richness of the data.
10. DISTINGUISH ANECDOTAL EVIDENCE FROM QUALITATIVE DATA Unsolicited responses to programs in the form of criticism or praise is considered anecdotal evidence. Such evidence is important and worthy of consideration, but it does not constitute formal assessment. Qualitative approaches are not the same as random responses and are designed to collect data that can be verified and validated.
The Ford Foundation is interested in vigorous assessment that:
allows you to shape programs as they are implemented
demonstrates what has been achieved
enables you to examine what might be done differently as you continue to develop programs
11. ANALYSIS MIRRORS PROCESS INDUCTIVE
ENRICHED THROUGH MEMBERS CHECK-IN
12. Qualitative Reporting Reporting formats are shaped by assessment purpose and by the meaning that is being conveyed.
Typical qualitative reporting includes narrative accompanied by matrix or charts that demonstrate key issues or causal relationships.
Ford Foundation assessment reporting seeks to communicate outcomes, indicators of program success and lessons learned through DDI programs. Use of case studies and rich description is of particular interest.