undergradute curriculum assessment evaluation report n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Undergradute Curriculum Assessment/Evaluation Report PowerPoint Presentation
Download Presentation
Undergradute Curriculum Assessment/Evaluation Report

Loading in 2 Seconds...

play fullscreen
1 / 27

Undergradute Curriculum Assessment/Evaluation Report - PowerPoint PPT Presentation


  • 96 Views
  • Uploaded on

Undergradute Curriculum Assessment/Evaluation Report. Ted Baker CS Faculty Workshop 18 August 2009. Overview. Training for instructors Review of student exit survey responses Accreditation status summary Things we need to do Not the most logical order, but the most critical items first .

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Undergradute Curriculum Assessment/Evaluation Report' - lahela


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
undergradute curriculum assessment evaluation report

Undergradute Curriculum Assessment/Evaluation Report

Ted Baker

CS Faculty Workshop

18 August 2009

overview
Overview
  • Training for instructors
  • Review of student exit survey responses
  • Accreditation status summary
  • Things we need to do

Not the most logical order, but the most critical items first.

training for instructors
Training for Instructors
  • Go to www.cs.fsu.edu/ugcc for
    • approved course objectives; include them.
    • SMALC outcomes that you must assess (if any)
      • Rubric you must use
      • Action plan from prior year
    • end-of term reports from prior instructors
  • Go to DOF website for most recent
    • Attendance statement
    • ADA statement
    • Honor code statement
  • Send syllabus to DUGS at start of semester
    • Preferably, in time for review before start of classes
  • We will soon (hopefully) have a web-based system for uploading the syllabi.
other beginning of term matters
Other Beginning-of-Term Matters
  • Confer with instructor from prior term, if possible. Get advice, materials.
  • Find out who is teaching the course for distance/PC campus, and communicate with them.
  • If you want to change textbooks, propose the change to the UGCC (but try for stability).
  • Beware of university and departmental requirement for final examination in all UG courses, given during final exam week.
  • If you have a TA, be sure to supervise, observe, mentor, and evaluate.
issues from exit survey
Issues from Exit Survey
  • See http://www.cs.fsu.edu/ugcc/smalcs09/surveys.html
  • Today, see spreadsheet.
issues raised by student advisory panel
Issues raised by student advisory panel
  • See http://www.cs.fsu.edu/ugcc/minutes/ugsac_02_10_09.html
  • Discrete math sequence is not working
    • Some math instructors are not teaching the courses well
    • Students are not motivated, because they don’t see the applications
    • We are not using the material enough in our courses soon enough, so student forget what they did learn
    • Significant repetition between and within Discrete Math courses, COP4530, and COP4531
    • Many students seem to be discourage dfrom major by the DM sequence
  • Have CS faculty teach discrete math
  • Integrate DM II into CS courses where it is used
slide7
More
  • COP4531 tries to rush through too many algorithms, without adequately developing the underlying design techniques
  • Courses that are logically connected, as prerequisites, need to be offered in contiguous semesters, so that students don’t forget what is needed from the preceding course
  • Overlapping course pairs not well coordinated and not consistent: sometimes duplication, and sometimes gaps. Need more instructor coordination, more consistency.
    • COP3014-COP3330
    • COP3330-COP4530
    • COP4530-COP4531
slide8
More
  • Problems with instructors who are new to a course. Suggest less turnover of instructors in a course.
  • Excessive use of Powerpoint makes lectures move to fast, too hard to follow. Suggest more hand-writing to engage students.
  • Need more exposure to event-driven programming
  • Electives are a problem. Department should poll students to see what electives they want.
  • Need more room for electives in the curriculum (less required courses).
  • Department seems to put more priority on graduate education than undergraduate.
  • Too much overlap between CDA3101 and CDA3100.
slide9
More
  • COP4710 requires students to use Java. This ends up being duplicative of the Java course.
  • Intro to Unix and Unix Tools courses sometimes overlap a lot, and vary a lot in content from term to term.
  • We are not making good use of recitation meetings. They should not just be another lecture, perhaps by the TA. They should be used to interact with students, with a focused activity.
  • COT4420 notes need updating.
  • More use of IDEs (along with command line) in courses beyond the introductory level.
  • COP3014 might attract more students if there were more interesting (e.g., GUI) assignments in it.
accreditation status
Accreditation Status
  • Interim report submitted in June, now being reviewed by ABET/CAC representative.
    • You can read the report if you go to http://www.cs.fsu.edu/ugcc/interim.pdf.
    • Username/password = smalcs/smalcs
  • If this report is approved (in July 2010), we will be approved for another 4 years.
  • Otherwise, we would probably be asked to do another interim report in June 2011.
issues in the report
Issues in the Report
  • Coverage of social and ethical issues related to computing.
  • Adequacy of elective offerings.
  • Sufficiency of university resources.
  • Outcomes and assessment process.
ethical social issues
Ethical & Social Issues
  • Added material to DB and SE courses
  • Created new course, CIS4250
  • This addresses the ABET complaint, but
    • DB and SE course treatment is now duplicative
    • It can’t be phased out until grandfathered students graduate,
    • and we find a way to handle distance students.
    • Distance students can’t do oral debates.
    • Oral debates can’t be made optional for CIS4250, since it is an “oral competency” course for SACS accreditation purposes.
  • One idea:
    • Create another ethics course, related to computing (maybe only 2 hrs, maybe only DL, maybe only taught by adjuncts)
    • Students who take that course must also take SPC2600 to cover oral competency requirement.
elective offerings
Elective Offerings
  • This is partly a student perception problem.
  • Report shows that we actually are offering more electives than the institution of the chair of the team that made the criticism.
  • Remaining issues:
    • Many are not practically available to most students, due to tough prerequisites (dual-listed graduate courses, other electives as prerequisite, senior-level courses as pre-requisite)
    • Distance students have fewer options
  • What we are doing:
    • Offer popular ones, without daunting pre-requisites, more often.
  • Things we might do:
    • Allow students to take some electives out of CS?
    • Work with other Florida universities to share distance course offerings?
adequacy of resources
Adequacy of Resources
  • ABET reviewers probably thought they were helping the program, by putting pressure on the FSU administration to replace lost faculty lines.
  • Our interim report explains effects of current economic distress on the University, and that
  • we are lucky to have suffered no layoffs, and to have been allowed to replace Ken Baldauf and Chrissy.
  • I have explained this also, in person, to the ABET reviewer.
  • We can hope that ABET will appreciate putting further pressure on FSU will not help.
assessment
Assessment
  • We have expanded our assessment process.
  • Tried to combine ABET requirements with SACS/SMALC requirements.
  • Doubled number of SMALC outcomes
    • to cover min. set of ABET/CAC-required outcomes
  • Added end-of-term instructor course reports
    • to provide more useful information for program improvements
  • Added focus-group meeting with undergraduate students
  • Added more formal review and reporting process
    • including this report, to the workshop
remaining problems
Remaining Problems
  • Faculty education.
    • Everyone needs to understand the process and the goal.
    • Therefore, this talk and more to come.
  • Increased workload (“death by assessment”)
    • Therefore, proposals to come, to reduce load.
  • Fundamental fallacy in ABET model
    • We can still make a good faith effort.
death by assessment
“Death by Assessment”
  • Gathering the assessment data is a lot of work.
  • This is distracting us from other essential duties, including the evaluation and planning that are supposed to be driven by the assessment process.
  • Faculty members still do not all understand what is expected.
  • The DUGS is overwhelmed by the workload.
    • The DUGS has already spent more than a man-month at this, dunning for data, meeting with people to explain what is needed, extracting files from e-mails and Web pages, editing, renaming to a standard system, copying into directories, reformatting into .pdf, renaming and reformatting them, creating web links to the files, etc. Moreover, thhis manual work is very error-prone.
    • Much of the supporting data files still need to be populated on our web site, to which the interim report refers.
    • The DUGS has been meeting with the webmaster since late fall 2008 to develop an on-line tool that would automate most of this, but the process seems to have stayed on the back burner.
assessement processes
Assessement Processes
  • State Mandated Academic Learning Compacts
    • We specify some specific minimal achievements (“student educational outcomes”, and “program outcome”) for our graduating students, which we must assess and report every year.
  • ABET
    • Starts similarly to SMALC, but goes further. We must demonstrate that we use the data from assessments as the basis for program improvements.
abet model
ABET Model
  • Engineering method should be applied to the process of education.
  • We define a set of desired outcomes for our educational program.
  • We gather data on how well those outcomes are being achieved.
  • We review and evaluate the data, which drives our plans to improve our program.
    • Not just the instructional process, but also the meta-processes (outcome statements, assessment methods, evaluation process, etc.)
fallacy in the model
Fallacy in the Model
  • Assumes we have a repeatable process.
    • So that we can vary one parameter, and assess how this has improved/hurt the program.
    • This seems to be called “regression-discontinuity design” by experimental design theorists.
    • We have different people teaching courses, sometimes in the same term.
    • Students coming into courses also have had different experiences in preceding courses.
    • There are so many differences in courses and students that we can’t isolate the effects of any specific change with reliability.
what can we do
What can we do?
  • Try to make the process more repeatable?
    • Not rotate instructors so frequently.
    • Communicate more between successive instructors, reusing text, slides, etc.
  • Take a more holistic view of the program assessment, recognizing that regression-discontinuity experimental method is not feasible in this environment.
  • Turn in syllabus before the term, and have it reviewed for consistency.
how to reduce workload
How to reduce workload
  • Increase priority of on-line data collection.
  • Collect SMALC data only once a year.
  • Reduce set of courses in which SMALC data is collected, to a few capstone courses.
  • Collect instructor reports in every course, and rely more on them for directing program improvements.
  • Extend survey to gather more feedback from alumni.
  • Continue
  • Do thorough review of assessment data less frequently, rotating through courses and outcomes on a multi-year cycle.
    • Which ones to start with? The math/theory sequence?
remaining instructor responsibilities
Remaining Instructor Responsibilities
  • SMALC data collection, in some courses.
  • End-of-term report, in every course.
  • Reviewing SMALC and other data, and making specific recommendations to DUGS and/or UGCC.
other ug curriculum issues
Other UG Curriculum Issues
  • New BA degree requirements
  • Relaxing prerequisites, to make courses more accessible
new ba requirements
New BA Requirements
  • Tougher than BS; requires 12 more courses than BS, in humanities.
  • 2 students graduated with BA in 2007, none in 2008 & 2009.
  • ABET considers this a separate degree; charges double fees, sends extra accreditation visitor; wants examples of student transcripts to prove program is vital.
  • We can’t provide example transcripts.
  • We decide to drop ABET accreditation of the BA degree.
  • This opens up possibilities
    • To attract more students to CS
    • By eliminating for the BA some requirements imposed solely to satisfy ABET/CAC
relaxing prerequisites
Relaxing Prerequisites
  • CS curriculum discourages cross-disciplinary studies, due to very deep prerequisite structure.
  • This also causes scheduling problems for students, and limits their choices of electives.
  • It is not clear that we actually depend on all these prerequisites.
  • It is time to review all prerequisites, to see if they are essential.
some changes already
Some changes, already
  • Java for non-majors
    • dropped C++ prerequisite
  • C++ for non-majors
    • Reduced math pre-requisite
  • COP4610
    • Reduced CDA3101 to co-requisite or instructor permission
  • Security courses
    • COP4530 replaced by CDA3100
    • Can we go further?