1 / 55

Evaluating & Choosing Instructional Software

Evaluating & Choosing Instructional Software. Edited by: Sandra Lebrón-Lozada, M. Ed. May, 2006. Introduction Objectives and Expectations. Workshop Objectives. By the end of this workshop you should: Know the distinctions among software applications used in instruction.

mala
Download Presentation

Evaluating & Choosing Instructional Software

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Evaluating & Choosing Instructional Software Edited by: Sandra Lebrón-Lozada, M. Ed. May, 2006

  2. IntroductionObjectives and Expectations

  3. Workshop Objectives • By the end of this workshop you should: • Know the distinctions among software applications used in instruction. • Be aware of pedagogical issues regarding the effectiveness of instructional software in different disciplines or content areas. • Overview the evaluation process including the development of evaluation criteria, the selection of an evaluation instrument, how to analyze evaluation results, and who needs to be involved.

  4. Lesson OneGood Reasons to Use Software

  5. Improve upon ExistingGoals and Outcomes • Learning Outcomes – broad goals focusing on course goals and core competencies in a course or curriculum; for example improving critical thinking skills. • Learning Bottlenecks – specific problem areas that are difficult to learn and if not mastered can hinder progress in a course; for example mastering fundamental math concepts. “A Taxonomy of Goals,” Ehrmann, 2002

  6. Establish New Goals that Depend upon Technology • Computer Literacy – Basic knowledge of computer use for communication, authoring and information • Web Research – The ability to find and evaluate information from web-based sources • Learning Online – How to communicate and work effectively in asynchronous, networked environments “A Taxonomy of Goals,” Ehrmann, 2002

  7. Activity 1: Finding Good Reasons • Sketch out your thoughts on the following questions… • What are the current ways in which software is used in your course (s)? • Have these uses improved instruction over more traditional methods? • What new course goals or learning bottlenecks could benefit from the use of technology?

  8. Value Defined: • “Software is valuable if it has the capability of being used to help improve teaching and learning. As the cases show, indicators of value can include evaluation results, awards won, testimonials from users, and the like. Notice that our definition of software's ‘value’ is independent of the extent to which it is used.” “On Value, Viability and Success,” Ehrmann, 1994

  9. Viability Defined: • “Software is viable if it is used by enough people for a long enough period of time that all its investors (original developers, fonders, publishers, institutional support staff, faculty, and students) can justly feel that they each have received an adequate return on their own investments in developing, acquiring, and/or learning to use the software.” “On Value, Viability and Success,” Ehrmann, 1994

  10. Some Finds on Viability • From studies published in Valuable, Viable Software in Education, funded through the Annenberg/CPB Project • “If computer software is not adequately upgraded to cope with changes in operating systems, it probably will not last long enough to achieve viability.” • “…most of the valuable viable software…fell into the category of worldware…” “On Value, Viability and Success,” Ehrmann, 1994

  11. A Typology forInstructional Software Courseware Learning Management System (LMS) Course Management System (CMS) Open Source Content Management Systems (OS/CMS) Worldware Low Threshold Applications (LTA)

  12. A Broad Definition: • “Instructional software can be defined • as any software used for instruction.”

  13. 1. Courseware- Educational Software Made for the teaching and learning of specific content. Ex. Math, Science, Social Sciences, English, etc.

  14. Definition of Courseware: • “Software designed specifically for • use in any educational setting: classroom, computer labs, network, and web-based that can be access from different devices”

  15. Virtual Laboratories

  16. Virtual Laboratories: Lab View

  17. Virtual Laboratories: Quiz View

  18. DiscoverEcon

  19. DiscoverEcon: Chapter View

  20. DiscoverEcon: Exercise View

  21. Learning Management Systems (LMS) and Course Management Systems (CMS) “Software that is used to design, deliver and manage an online course.”

  22. 2. Learning Management Systems (LMS) • Learning management systems (LMS) refers to software that primarily acts as an electronic registrar by electronically performing various enrollment and related tasks. • LMSs were originally designed for workplace learning environments, and specifically perform some or all of the following tasks: • registration • track participation (classroom attendance, sign-on of online courses) • track of completions (including final scores or grades) • testing • follow-up discussions with participants • aggregated reports, such as the number of people registered for particular courses • transfer of information to other systems, such as human resource information systems • process charges for courses, such as tuition payments and transfer payments among departments • course catalog • skills management. http://www.learningcircuits.org/2005/nov2005/carliner.htm

  23. Mymathlab: To login as an instructor to preview an existing HCC course go to: http://www.coursecompass.com/ccindex.html Login Name: AWMML27 Password: AWMML27 DistanceLearning.org Glossary http://www.distancelearning.org/glossary/

  24. 3. Course Management Systems (CMS) • Course management systems (CMSs) are online systems that were originally designed to support classroom learning in academic settings, such as universities and high schools. CMSs provide instructors with the ability to perform the following tasks: • Place course materials online. Most CMSs provide pre-programmed buttons for the course syllabus, course schedule, and course materials linked to specific lessons, such as copies of readings and PowerPoint slides from lectures. • Track student progress through assessment features, which enable instructors to give quizzes and tests online, and an online gradebook, where instructors can post student grades. • Discussion board, where instructors and students can discuss readings and continue class discussions between formal class sessions.

  25. 3. Course Management Systems (CMS) • Other communications tools, which let instructors send announcements to classes and communicate individually with students • Lock box for students, where students can store class materials in a safe place—either a presentation to give later in class or backing up class assignments in a safe place. • Course statistics, which provide information on the use of the course site, including who used the course site and when. http://www.learningcircuits.org/2005/nov2005/carliner.htm

  26. WebCT Course Management @HCC http://webct.hccs.edu/

  27. WebCT Course Management

  28. WebCT Tools http://www.colostate.edu/webct/webct_tools.htm

  29. WebCT with e-Pack http://www.blackboard.com/extend/Digital_Content/Instructors.htm

  30. More about epacks • ..\WebCTepacks\wbct-texas2004-epacks.htm

  31. 4. Open Source/Course Management System (OS/CMS) “ Free” software that is used to design, deliver and manage an online course using open sources available on the Internet…

  32. Open Source/Course Management System (OS/CMS) Benefits? http://www.openknowledge.org/writing/open-source/scb/why-open-source.html http://open-source.gbdirect.co.uk/migration/benefit.html Myth? http://www.cio.com/archive/030104/open.html

  33. Open Source/Course Management System (OS/CMS) “ Free” software that is used to design, deliver and manage an online course using open sources available on the Internet…

  34. 5. Worldware Software that “is developed for purposes other than instruction but is also used for teaching and learning.”

  35. Benefits: • Widely available • Have a real-world application • Easy to correlate to curriculum “Low Threshold Applications (LTAs),” TLT Group

  36. A Sampling of Worldware • Word processors • Spreadsheets, statistical and other mathematical software • Software for managing library services • Communications software • Tools for design and composition in fields as diverse as architecture, desktop publishing and music • Tools for modeling and simulation • Research databases online and on discs • Programming languages “On Value, Viability and Success,” Ehrmann, 1994

  37. 6. Low Threshold Applications - LTA Software that “is reliable, accessible, easy to learn, non-intimidating and (incrementally) inexpensive.” “Low Threshold Applications (LTAs),” TLT Group

  38. LTA Benefits: • Ubiquitous • Reliable • Easy to Get Started • Offer Immediate Results • Precipitate Long Term Changes “Low Threshold Applications (LTAs),” TLT Group

  39. Possible LTAs • Communication Software(Email / Listserv / Chat / Discussion Boards) • Web Software(Browser, Search Engine) • Productivity Software(Word Processor, Spreadsheet, Presentation) • Collaborative Software(Wikis, Weblogs)

  40. Activity 3: Consider all options • Sketch out your thoughts on the following questions… • Based on your notes from Activities 1 , what software options do you think are most viable for your course(s). Keep in mind your course goals and desired learning outcomes.

  41. The Next StepThe Software Evaluation Process Using Instructional Design Principles Analyze/content, instruction, typology, and resources Design/ Evaluation Plan and Timeline Develop/Criteria and Eval. Instrument Implement/ Conduct Assessment Evaluate/Results and Recommendations

  42. Sample Standard Criteria • The ERMES (EuRopean Multimedia Educational Software network approach: • http://acm.org/ubiquity/views/g_avellis_1.html • Educational features of the software • Technical features • Ease of use (usability) • Content • http://www.ifets.info/journals/5_2/avellis.html • the content which is to be taught; • the deliverymedia used to provide information; • the user interface, that is the way the educational software presents itself to the user; • the interaction devices, by which the user interacts with the computer, making choices, answering questions or performing activities, and is provided with feedback to each response; and, • the instructional strategy adopted.  

  43. Evaluation Review Criteria- various • http://www.evalutech.sreb.org/criteria/index.asp

  44. Selecting Evaluation Instruments • Heuristic Approach for evaluating • Interface design heuristics [after Nielsen ] • Educational design heuristics [after Quinn ] • Content design • http://www.usq.edu.au/users/albion/papers/ascilite99.html

  45. Rating Scales Surveys • http://www.ed.brocku.ca/~jkerr/sftwreva.htm • Teaching, Learning, and Technology Group. http://www.tltgroup.org

  46. Rubricsfor Evaluating Instructional Software • WebCT Exemplary Rubric – http://www.webct.com/exemplary/ • Texas Higher Education Coordinating Board guide for Incorporating the Principles of Good Practice into electronically-based Courses – http://distance.hccs.edu/links/fac_res.htm • http://www.arches.uga.edu/~eshealy/rubric.htm • http://www.ncsu.edu/midlink/sw.eval.rubric2.doc • http://hagar.up.ac.za/catts/learner/eel/Conc/conceot.htm

  47. OS/CMS Assessment Evaluation Process Adapted from Univ. of Northern Iowa http://fp.uni.edu/its/et/lms/cms_eval.pdf • Needs Assessment • •What are we doing this? Determine the purpose. • •Agree on the evaluation methodology and process. • • Identify evaluators • • Identify resources • • Search and select evaluation method, criteria and process • • Gather information on products/vendors • • Determine any costs • • Create timeline

  48. OS/CMS Assessment Evaluation Process – cont. Adapted from Univ. of Northern Iowa http://fp.uni.edu/its/et/lms/cms_eval.pdf • Evaluation Process • •Identify products – from selected list • •Choose features - http://www.edutools.info • • Assign ranks (1-5) for weighting – as criteria to create a short list of products • Choose criteria for evaluating the features- i.e. rubric w/scores + scores + weight • • Perform individual or group evaluations in three areas: • Learner Tools • Support Tools • Technical Specifications • •Report findings and make recommendations

  49. The Pilot Test • Who should be involved to make your pilot a success? All stateholders. • Students • Faculty and Colleagues • Technologists • Administration

  50. Students • Have your students actively participate by… • Giving first impressions about the software’sease of use, accessibility, and its currency • Adding an assessment of the software to the evaluation of the course

More Related