1 / 48

Technology for automated assessment: The World-Wide-Mind

Technology for automated assessment: The World-Wide-Mind. Ciarán O’Leary Dublin Institute of Technology 22 nd May 2003. Quick Overview. Requirements Large amounts of software online Large number of participants in project Diversity is key! Need to

asasia
Download Presentation

Technology for automated assessment: The World-Wide-Mind

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Technology for automated assessment: The World-Wide-Mind Ciarán O’Leary Dublin Institute of Technology 22nd May 2003

  2. Quick Overview • Requirements • Large amounts of software online • Large number of participants in project • Diversity is key! • Need to • Provide architecture and protocol that will make this software available • Have a simple entry level – like the WWW

  3. e-Learning Overview • Reusability of components • Potential for automated assessment • Potential for comparisons with other students

  4. Agenda • World-Wide-Mind • Introduction to goal of project • Definition of a service • Available software • Blocks World example • Usage in learning environment • Comparison with existing models • Possibilities for future development

  5. Part I

  6. World-Wide-Mind • Dublin City University • Dr. Mark Humphrys • Artificial Intelligence Project • Distributed • Artificial Minds

  7. The Jolly Little Creature Agent

  8. The Jolly Little Creature’s Mind • Multiple (massively diverse) modules • Since, according to faculty psychologists, the mental causation of behaviour typically involves the simultaneous activity of a variety of distinct psychological mechanisms, the best research strategy would seem to be divide and conquer: first study the intrinsic characteristics of each of the presumed faculties, then study the ways in which they interact • Jerry Fodor – The Modularity of Mind, p1.

  9. Building JLC’s Mind • Divide and conquer • Required diversity through multiple authors • The more authors the more diversity • Make modules available online • Researchers • Amateurs • Students!

  10. Building JLC’s Mind • Requirements • Architecture • Standard/protocol to use to integrate components • Requirement for protocol • No barriers to entry • Simple • Not tied to any platform

  11. Summary Part I • World-Wide-Mind • Distributed Artificial Intelligent Project • Success dependent on participation of large number of very different researchers • Require diversity • Need an architecture and protocol for components to be plugged together • Major goal is simplicity

  12. Part II

  13. JLC’s Mind Mind Perceive state Select action Body State Take action World State New state

  14. JLC’s Mind Online Mind Perceive state Select action Body State Take action World State New state Internet

  15. World-Wide-Mind Architecture Mind Server World Server Client

  16. World-Wide-Mind Architecture Mind Server Mind Server World Server Mind Server Mind Server World Server Mind Server Mind Server Client Mind Server

  17. Service Methods World Service • newrun • getstate • takeaction • endrun • Mind Service • newrun • getaction • endrun

  18. World-Wide-Mind Protocol • “The term Web services describes a standardized way of integrating Web-based applications using the XML, SOAP, WSDL and UDDI open standards over an Internet protocol backbone. XML is used to tag the data, SOAP is used to transfer the data, WSDL is used for describing the services available and UDDI is used for listing what services are available” • http://www.webopaedia.com • Society of Mind Markup Language (SOML) • Lightweight Web Services

  19. SOML Message 1. <soml version="0.9"> 2. <request type="getaction" runid="123456"> 3. <param name="state"> 4. (0, 0, 0, 9, 7, 8, 2, 0, 4) 5. </param> 6. <argument name="usemem" value=“true”/> 7. </request> 8. </soml>

  20. Web Based Client – Java Applet

  21. Web Based Client – Java Applet

  22. Constructing complex minds, online Mind Server MindM Server World Server Mind Server Client

  23. Who builds the sub-minds? • Everyone • Professional researchers • Students • Amateurs • Anyone interested in AI!

  24. Who builds the parent minds • Everyone • Anyone can put software online • Anyone can put a world online • Anyone can use my mind as part of their mind • Best minds are the ones that will be re-used

  25. How does this make the AI problem easier? • Society A solves problem A • Society B solves problem B • Problem A + B is solved by writing parent mind to arbitrate between A and B’s parent minds

  26. World A+B Client MindM A+B World A World B MindM A MindM B Mind Mind Mind Mind Mind Mind Society of Mind

  27. Summary Part II • Distribute the mind on the Internet • Use server-centric architecture • Use existing HTTP protocol • Use simple XML-like messages • Simple entry level • Potential for provision of large numbers of components required for project

  28. Part III

  29. World-Wide-Mind in learning environment • 10 Students presented with an online problem • Have to interact with the online problem using client software • Need to create “mind” or solution

  30. Goal F E Start D D C F C B A B E A First Implementation • Blocks World

  31. Blocks World Service • Action • State empty(1)&ontable(A,2)& clear(A)&ontable(B,3)& on(F,B)&on(D,F)& clear(D)&ontable(E,4)& on(C,E)&clear(C)& empty(5)&gripper() grip(A) ungrip(3)

  32. newrun getaction getaction getaction getaction getaction endrun takeaction takeaction takeaction endrun newrun takeaction getstate Blocks World Run Mind Server Blocks World Server Client

  33. Blocks World Service • Supports • newrun • Provides runid • getstate • Provides state in form shown • takeaction • Receives action sent by client • Returns updated state following execution of action • endrun • Ends run, clears up server • All messages carry error code information

  34. Building a blocks world “mind” • Create service that supports • newrun • Provides runid • Can also provide startup-parameters • getaction • Receives state from client (which had got it from world service) • Provides state in form shown • endrun • Ends run, clears up server • All messages should carry error code information

  35. Scoreboard • Scores of best performing minds recorded in a list on the “world” site • Score is the number of moves needed to stack blocks • Any mind that interacts with world will have its score recorded and displayed • Others can then run that mind in the world • Performance based assessment • Performance of algorithm can be judged from score board

  36. Technical Requirements • Web Server space • One of… • Java Servlets • ASP • JSP • PHP • CGI – C++/Perl/any language • To write own client (or to integrate another mind), need to know how to connect to URL • Simple code in most languages

  37. Connect to URL (Java) String getResponseFromService(String serviceURL, String request) { URLConnection connection = new URL(serviceURL).openConnection(); PrintWriter outs = new PrintWriter(connection.getOutputStream()); outs.println(request); // SOML message outs.close(); StringBuffer returnedText = new StringBuffer(); BufferedReader ins = new BufferedReader(new InputStreamReader(connection.getInputStream())); String line = ""; while((line = ins.readLine()) != null) { returnedText.append(line); } return returnedText.toString(); }

  38. Connect to URL (Perl) sub getResponseFromService() { $serviceURL = @_[0]; $request = @_[0]; new LWP::UserAgent->request( POST “$serviceURL”, Content_Type => 'form-data', Content => [ q(q) => "$request"])->content; }

  39. Summary Part III • Simple blocks world problem online using protocol/architecture developed for World-Wide-Mind • Software still available online for others to interact with • Client software also available • Students put minds online • Minimal knowledge of network programming required • Only really need to read from stdin and write to stdout. • Scoreboard records best performing algorithms/minds

  40. Part IV

  41. Comments on Architecture • Lightweight web services • Low entry level • Protocol simple • Technical requirements minimal • Software continually available • Problem can be re-used remotely by others • At the very least it can be tested remotely

  42. Comments on Architecture • Comparison with World-Wide-Web • Students put project work (HTML pages/PDF) online (DCU/DIT) • Others can review their project work • Very simple entry level • Before there were tools (MS Front Page) there was simple (terrible) HTML

  43. Reusability • Why don’t we use other peoples software? • Platform issues • Programming language isues • Its in our favour for others to make software available • What is their incentive? • If you’re putting it online anyway… • Learning objects • SCORM • PEARL • TOIA (IMS QTI compatible tools)

  44. Summary Part IV • We need to re-use other peoples software • Local installation has many issues • Complex standards require tools • Low entry level

  45. Part V

  46. Future Development • Existing standard • Portal site will provide • Sample code • Instructions on creating services • Services • Client software

  47. Future Development • Possible • Integration at higher levels with other standards • SOAP • XML • WSDL • DAML • e-Learning standards • PEARL uses HTTP

  48. Questions?

More Related