9 th Annual Texas A&M University Assessment Conference, February 24, 2009

1. Developing Your Own Course-Specific Assessment Tool: Strategies for Teaching FacultyCS-5.10, Track 2 David J. SilvaVice Provost for Academic AffairsProfessor of Linguistics & Distinguished Teaching Professor 9th Annual Texas A&M University Assessment Conference, February 24, 2009

2. The Issue • Pressure to assess • … increasingly at the course level • Faculty concerns • administrative intrusion • perceived lack of utility (“just another hoop”) • lack of knowledge about assessment • lack of experience developing assessment tools • additional burden (“doing more with the same”)

3. Addressing Potential Tension • Faculty concerns • administrative intrusion • perceived lack of utility(“just another hoop”) • lack of knowledge about assessment • lack of experience developing assessment tools • additional burden (“doing more with the same”) • Admin Responses • administrative scaffolding • focus on benefits to instructors / programs (“knowledge= power”) • provide professional development • provide focused seminars and associ-ated incentives • focus on integration (“maximizing existing resources”)

4. Contextualized Approach: QEP • “Active Learning: Pathways to Higher Order Thinking at UT Arlington” • Ten Pilot Projects • Education: Literacy Studies • Engineering: First-Year, Electrical Engineering, Capstone • Honors College: Seminar • Liberal Arts: English/Libraries, History, Political Science • Nursing: Capstone • Science: Mathematics • In search of a second path for assessment (“Triangulation”) • In place: Criterion-based course rubrics • Problematic: Program-level assessment, multiple choice CT test • Developing appreciation for the role of context in assessing HOTS – Content Based Assessment (CBA)

5. Multifaceted Approach • Ongoing professional development • Workshop on Grading and Assessment; Generation NeXt • Teaching Circles for assessment as well as for active learning • Afternoon seminar / discussion group on active learning • Special One-Day QEP PI workshop on CBA • Continued appeal to faculty’s innate curiosity • Assessment = Research • Research earns reward • Resources for Innovation • Faculty award • Small seed grants • Faculty Development Leaves • Focus on integration: “maximizing existing resources”

6. QEP’s Existing Resources • Basic understanding of assessment • Appreciation for the cyclic nature of assessment • Clearly articulated student learning outcomes (SLOs) • Desire to optimize research results • Community of Practice • Culture of good citizenship

7. Goals of the CBA Workshop • Reconnect PIs with Bloom’s Taxonomy • (Re-)identify the primary SLO • Look toward authenticity: the discipline • “What sorts of … • knowledge • comprehension • application … … does a successful ________ draw upon?” • Begin to draft assessment questions

8. What Are We Creating? • Direct Assessment • Complements indirect assessment • Interim Questionnaires (engagement, effort) • Focus Group Data • Value Added Outcomes • Complements absolute outcomes • Rubric Data • Embedded Assessment (potentially) • Used as course-initial orientation tool • Used as part of final exam (optional) • Summative Assessment • Triangulation Thanks to Mary J. Allen for this framework

9. Let’s “Learn Active!” • Course Information • Key SLO • What’s the chief objective (Broadly Conceptualized)? • Disciplinary skills • What do people who do X do? • What do people who do X need to know? • Course-related skills • How does this course contribute to (3)? • Beginnings of possible questions

10. Bloom ↔ Discipline ↔ Course Let’s see a bit of what we’ve accomplished thus far…

11. HIST 1311 • “History of the United States” • Professor Stephanie Cole • 130 – 145 students • Innovation: Personal Response Systems

12. HIST 1311 In recent years, there has been considerable debate in the United States regarding the status of same sex marriage. In some states, such unions are legal. In others, they are prohibited by the state Constitution. From an historian’s perspective, which of the following events would provide the most insight when it comes to understanding the sort of conflict regarding federal vs. states rights? a) The Teapot Dome scandal of the 1920’s b) The Missouri Compromise of 1820 c) The ratification of the 14th Amendment the US Constitution in 1868 d) The Nullification Crisis of the 1830’s

13. POLS 2311 • “Government of the United States” • Professor Rebecca Deen • 60 – 110 students • Innovation: Campaign Service Module

14. POLS 2311 Which of the following is not a component of the Michigan Model of voting behavior? a) partisan identificationb) social groups c) candidate evaluations d) issue preference

15. LING 2301 • “Intro to the Study of Human Language” • Professor David Silva • 25 – 30 students • Innovations: Blog; Public Service Product

16. LING 2301 When a person suffers from the neurological condition known as aphasia, not only are his language capacities diminished in some regard, but in addition, he presents difficulties with: • moving the left side of his body (arm, leg, mouth, etc.) • moving the right side of his body (arm, leg, mouth, etc.) • his nighttime vision • remembering the names of close friends and family members • breathing normally, particularly on high-ozone days

17. LING 2301 As a parent, you are hoping to invest wisely in the linguistic development of your 3-year old. More specifically, you want your toddler to enter school as an English-Chinese bilingual. Hence, you: • take Chinese classes and share what you’ve learned with your child each day • subscribe to a Chinese-language satellite TV channel and have your child watch children’s programming each morning from 8:00 to 11:30 • hire a Chinese housekeeper, who takes your child when running errands each day, and who speaks Chinese with your child regularly • leave your child with a Chinese babysitter, who works from her home because she is raising a 1-year old and a 3-year old of her own • enroll your child in an ethnically diverse preschool, making sure that there are Chinese-speaking children in the program

18. LING 2301: Results

19. LING 2301: Surprise Which of the following conditions is not considered a prerequisite to successful language acquisition? • cognitive capacity (i.e., developmentally normal brain function) • access to data in the form of appropriate textbooks and/or electronic media (e.g., CDs) • opportunities for authentic interaction • integrative motivation • instrumental motivation  92% correct (Pre-test)  53% chosen on Post-test!

20. Still to be Done • Moving pedagogy into the higher domains as appropriate • Rethinking knowledge as something else • Working on how to write the questions • Learning from others, e.g. TTU’s CAT • Validating each CBA instrument • Continuing to assess the QEP as we move toward Phase II • Fostering a culture of assessment campus-wide

21. What Do We Have for Our Efforts? • Some answers to “WIIFM?” • Some more dedicated amateur assessors • note: the root of amateur is “love” • Some additional movement in our ongoing shift into a “culture of assessment” “If we don't change direction soon,we'll end up where we're going.” -Irwin Corey

22. Thank You ありがとうAsanteDankeDiolchGraciasGràcies Grazie감사합니다Köszönöm MahaloMerciObrigado СпасибоTackTerima Kasih谢谢 David J. Silvadjsilva@uta.edu Be A Maverick. ™