Computer Science Department Assessment 2006 Report

1. Computer Science Department Assessment2006 Report Slippery Rock University

2. CS, IS, and IT • Three Student Learning Outcomes • Critical Thinking and Problem Solving • Communication and Interpersonal Skills • Professional and Ethical Responsibilities • Three Assessment Methods • Surveys • Course embedded assessment • Internship appraisals

3. Program Outcomes • The following three slides show the newly approved student learning outcomes for each of the three programs in the department • Previously, the educational objective “Problem Solving” was separate from “Critical Thinking” • The individual program outcomes were re-worded and consolidated as needed • Communication outcomes were made more consistent across the three programs • The program outcome “continuing professional development” was added to the “Ethical and Professional Responsibilities” educational objective

4. Computer Science Student Learning Outcomes I. Problem Solving and Critical Thinking. Solve abstract and complex problems using software design methodology. Make informed choices among alternative solutions. The student will be able to: • Design an algorithmic solution to a problem using problem decomposition and step-wise refinement. • Implement an algorithm by creating a tested and debugged programmatic solution. • Examine and analyze alternative solutions to a problem. • Develop abstract models to simulate complex systems. • Determine appropriate hardware and software combinations for maximum efficiency. • Determine correctness and efficiency of a system design and implementation. II. Communication and Interpersonal Skills. Use written, oral, and electronic methods for effective communication. The student will be able to: • Document all aspects of a system precisely and clearly. • Use written, oral, and electronic communication to convey technical information effectively. • Devise effective user interfaces. • Work cooperatively in teams and with others. III. Ethical and Professional Responsibilities. Discern and articulate the impact of technologies on society. The student will be able to: • Plan for and ensure the security, privacy, and integrity of data. • Recognize the ethical, legal, and social implications of computing. • Demonstrate an understanding of the Association of Computing Machinery (ACM) Code of Professional Ethics. • Analyze the impact that computing has on the global society. • Recognize the need for continuing professional development.

5. Information Systems Student Learning Outcomes I. Problem Solving and Critical Thinking. Solve business related problems by analyzing existing systems and designing and implementing the most feasible alternative. The student will be able to: • Examine and analyze alternative solutions to a problem. • Make informed choices among alternative hardware and software configurations for the design, development, and implementation of an information system. • Integrate programming and design principles for effective problem solving to address the needs of an organization. • Perform critical analyses and write feasibility studies of system implementations. • Evaluate the effectiveness of information systems design and implementation. II. Communication and Interpersonal Skills. Use written, oral, and electronic methods for effective communication. The student will be able to: • Write clear and concise user documentation. • Summarize technical information for technical and non-technical personnel. • Use oral and electronic communication effectively. • Devise effective user interfaces for information systems. • Work cooperatively in teams and with others. III. Ethical and Professional Responsibilities. Discern and articulate the impact of technologies on society. The student will be able to: • Plan for and ensure the security, integrity, and privacy of data and the confidentiality of the end user. • Recognize the ethical, legal, and social implications of information processing. • Demonstrate an understanding of the Association for Computing Machinery (ACM) Code of Professional Ethics. • Analyze the impact that information systems have on the global society. • Recognize the need for continuing professional development.

6. Information Technology Student Learning Outcomes I. Problem Solving and Critical Thinking. Solve problems by creating client/server applications that are efficient, user-friendly, and portable. The student will be able to: • Apply programming and system management techniques to address information technology problems. • Integrate design and implementation principles to develop effective web pages. • Perform critical analyses of the impacts of decisions. • Implement IT solutions that consist of system and application software written in various programming languages. • Create efficient, graphical client/server applications. II. Communication and Interpersonal Skills. Use written, oral and electronic methods for effective communication. The student will be able to: • Document all aspects of a system precisely and clearly. • Use written, oral, and electronic communication to convey technical information effectively. • Devise effective user interfaces for the web. • Work cooperatively in teams and with others. III. Ethical and Professional Responsibilities. Discern and articulate the impact of technologies on society. The student will be able to: • Determine the economic and organizational effects of information technology on global society. • Recognize important legal issues and demonstrate appropriate social responsibilities in information technology. • Demonstrate an awareness of the codes of professional ethics in the information technology industry. • Plan for and ensure the security, privacy, and integrity of data. • Recognize the need for continuing professional development.

7. Methods Constituents Senior Junior Sophomore Surveys Students Outcomes Critical Thinking & Problem Solving 140, 150, 374 376, 464, 474, 488 Faculty Communication & Interpersonal Skills 140, 423, 488 Course Embedded Professional & Ethical Responsibilities Internship Appraisals Supervisor

8. Assessment Methods • Each program outcome is assessed by three assessment methods from different constituents. • Student Surveys • Prefer statistical sample of at least 25 • 75% very confident indicates good performance • 85% indicates excellent performance • Internship Appraisals • 85% indicates good performance • 95% indicates excellent performance • Course Embedded • 80% indicates good performance • 90% indicates excellent performance

9. Analysis of Assessment Data Surveys Course Embedded Internship Appraisals Discussion Constituents Students Faculty Advisory Board Report results Curriculum Changes Data Collection (3 methods)

10. 2005 – 2006 Curriculum Changes • Modified IS allied areas to reflect external ABET evaluation • Increased the breadth of the allied areas • Removed non-accredited allied areas • Modified CpSc 370 and the CS major • Updated the 370 course by removing the teaching of a language and instead teaching computer organization • Put 370 back in the core (a required course) • Changed core language from C++ to Java • National assessment data suggests that prior to Java, Alice should be used. As of Fall 2006, Alice will be used in CpSc 140 to decrease the drop-out rate; Java will be taught in 150, 343 and 374.

11. General Observations • Four educational objectives contained too many program outcomes to assess – reduced to three • Add program outcome “continuing professional development” to conform to ABET’s new outcomes • Revise student surveys for new outcomes • New surveys will be the same for all levels (So, Jr, Sr) and new thresholds will be developed • Survey results will be examined for increasing improvement from one level to the next • Survey questions do not address program outcomes • External appraisals group all three programs together and naming is not consistent with educational objectives

12. Observation Overview • The following slides are organized as follows: • Specific observations for each major based on problems/limitations reported in Spring 2005 • 2005 observations for the Communication and Interpersonal Skills Educational Objective for each major • Only observations revealing possible problems/limitations are reported

13. Computer ScienceUnresolved Observations • Student Surveys • CS Ethical and Professional Responsibilities continue improving • 58% very confident as Soph. (up from 57% in 2004) • 54% very confident as Sr. (up from 46% in 2004) • Course Embedded Assessment • Withdrawal rate in CpSc 140 is high (Fall 2005) • External Appraisals • In 2004, 49% of students were considered “exceptionally mature” (judgment category), and 47% were considered above average. • In 2005, these numbers decreased to 45% and 30%, respectively. • Board of Visitors Fall 2005 meeting: Board felt the sum total was well within acceptable limits

14. Information SystemsUnresolved Observations • Student Surveys • IS Critical Thinking: 50% very confident as Sr. (same as 50% in 2004) • IS Ethical Responsibilities and Professional Responsibilities • 48% very confident as Sr. (up from 45% in 2004) • Course Embedded Assessment • Faculty observation (circa 2003): students in CpSc 130 struggling • Withdrawal rate in CpSc140 is high (Fall 2005) • CpSc 443 assessment data for critical thinking is at 75% • In 2003, this course was added to the IS major to improve critical thinking. • External Appraisals • In 2004, 49% of students were considered “exceptionally mature” (judgment category), and 47% were considered above average. • In 2005, these numbers decreased to 45% and 30%, respectively. • Board of Visitors Fall 2005 meeting: Board felt the sum total was well within acceptable limits

15. Information TechnologyUnresolved Observations • Student Surveys • IT Critical Thinking – 62% very confident as Sr. (up from 56% in 2004) • IT Ethical and Professional Responsibilities • 51% very confident as Soph. (up from 46% in 2004) • 63% very confident as Sr. (up from 58% in 2004) • Course Embedded Assessment • Faculty observation (circa 2003): students in CpSc 130 struggling • Withdrawal rate in CpSc 140 is high (Fall 2005) • CpSc 443 assessment data for critical thinking is at 75% • In 2003 this course was added to the IT major to improve critical thinking • External Appraisals • In 2004, 49% of students were considered “exceptionally mature” (judgment category), and 47% were considered above average. • In 2005, these numbers decreased to 45% and 30%, respectively. • Board of Visitors Fall 2005 meeting: Board felt the sum total was well within acceptable limits

16. Computer ScienceCommunication Observations • External Appraisals • 47% of students are outstanding for “relating to others”; 53% are very good • Student Surveys • 60% of sophomores, 82% of juniors, and 76% of seniors are very confident using their communication skills. • The sum of confident and very confident is 93%, 94%, and 94% for sophomores, juniors, and seniors, respectively. • Course Embedded Assessment • Average scores for group projects fluctuate more than individual assignments. • Standard deviation for group projects in CpSc 150 is 11.58 (Spring 2005). • Standard deviation for individual assignments in CpSc 140 is 4.69 (Spring 2005). Although all the average scores for individual assignments in CpSc 140 are above 80, the average score decreases from 93.71 to 82.86 towards the end of the semester. • The standard deviation for group projects in CpSc 150 is 9.77 (Fall 2005). • The standard deviation for group projects in CpSc 423 is 6.48 (Fall 2005).

17. Information SystemsCommunication Observations • External Appraisals • 47% of students are outstanding for “relating to others”; 53% are very good • Student Surveys • 67% of sophomores, 58% of juniors, and 65% of seniors are very confident using their communication skills. • The sum of confident and very confident is 98%, 100%, and 97% for sophomores, juniors, and seniors, respectively. • Course Embedded Assessment • Average scores for group projects fluctuate more than individual assignments. • Standard deviation for group projects in CpSc 150 is 11.58 (Spring 2005). • Standard deviation for individual assignments in CpSc 140 is 4.69 (Spring 2005). Although all the average scores for individual assignments in CpSc 140 are above 80, the average score decreases from 93.71 to 82.86 towards the end of the semester. • The standard deviation for group projects in CpSc 150 is 9.77 (Fall 2005). • The standard deviation for group projects in CpSc 423 is 6.48 (Fall 2005).

18. Information TechnologyCommunication Observations • External Appraisals • 47% of students are outstanding for “relating to others”; 53% are very good • Student Surveys • 67% of sophomores, 74% of juniors, and 77% of seniors are very confident using their communication skills. • The sum of confident and very confident is 99%, 98%, and 97% for sophomores, juniors, and seniors, respectively. • Course Embedded Assessment • The average scores for individual assignments are better than those of group assignments. • The average score is 86.97 for individual assignments in CpSc 140 (Spring 2005). • The average score is 71.67 for group projects in CpSc 327. • The average score is 81.81 for group projects in CpSc 443. However, the average score decreases from 93.71 to 82.86 towards the end of the semester.

19. Committee Recommendations • Separate measurements for each assessment method per program outcome need to be created (e.g., 85% of senior will respond with very confident to “How confident are you working cooperatively in teams and with others”) • Modify external appraisal form: • Add an evaluation of student intern’s Oral & Written Communication. • Add an evaluation of student intern’s Planning & Organizational Skills. • Change ‘Relationship with Others’ heading to ‘Interpersonal Skills’ heading. • Add check-box to note student intern’s major field of study so that assessment data from surveys can be sorted accordingly. • Change wording of open-ended questions on 2nd page to reflect employer concerns about additional skill sets needed by student interns. • In Response to Fall 2005 Board of Visitor Recommendations • Poll Internship Employers for questions to be included in their suggested, new ‘Internship Completion Questionnaire’ • Student Interns would complete questionnaire to assist Internship Employers in providing an excellent learning experience and valuable knowledge acquisition. • Poll current internship students for possible questions to be included in this questionnaire. • Data would be analyzed annually and provided to Internship Employers.

20. Next Year • Fall meeting with faculty to make recommendations to improve weaknesses • Re-map course outcomes to program outcomes • Revise course outcomes as necessary • 2006 data for all assessment methods will be evaluated with Communication and Interpersonal Skills as the priority • Implement new student surveys in Fall 2006 • Beginning with the Fall Semester 2006, students surveys will be completed in designated class sessions after their Sophomore, Junior, and Senior years. • Following the student survey completion instructors will : • explain the purpose of the surveys, • share with the students the results of past surveys, • allow students an opportunity to offer input into survey design • document student feedback so that future surveys can be improved • Report to Board of Visitors at Spring meeting