1 / 25

Introduction to Grid Computing

Deployment of a Language Detector Grid Service. Introduction to Grid Computing. Felix Hageloh Roberto Valenti. University of Amsterdam, 02-11-2005. Overview. Introduction Required Steps Our Service Introduction The basic idea Use Case Interface Implementation Problems Encountered

garygary
Download Presentation

Introduction to Grid Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Deployment of a Language Detector Grid Service Introduction to Grid Computing Felix Hageloh Roberto Valenti University of Amsterdam, 02-11-2005

  2. Overview • Introduction • Required Steps • Our Service • Introduction • The basic idea • Use Case • Interface • Implementation • Problems Encountered • Future Work • Conclusions • Questions

  3. Introduction • Our chosen task: Grid Services • Task Goals: • Build a grid service. • Aggregate the service with another to provide additional, higher-level services

  4. Steps • Get access to the systems • Authentication • Security issues • Obtain User Certificate • Obtain Host Certificate • Implement the service • Create required files • WSDL • QNames • WSDD • JNDI • Compile and create GAR file • As Globus user: • Deploy service • Start container

  5. But you all know this… So… we jump to our service.

  6. Our Service

  7. Our Service: Introduction • We were requested to implement a useful service which could be integrated on other services • We are AI students so… Let’s Merge AI and Grid Computing!!

  8. Our Service: The Basic Idea • Idea: Language Detection Is a necessary first step in a multitude of applications • Useful Web Service Examples: • Email filtering • Information retrieval • Spell checkers • Can also be component of an aggregated grid service

  9. Our Service: The Basic Idea • What about creating a Language Detector on the Grid? • Training and Testing can be extremely time consuming running on a single machine • Data difficult to obtain -> can be shared on the Grid • Duplicate data for parallel computing

  10. Our Service: Use Case Simple Interface: • Receives a piece of text • Returns a string indicating the language

  11. Our Service: Adding States • Grid services can have states (as opposed to web services) • Not necessary for our service but for the learning factor • Added “dummy” states to our service: • Last Operation • Times Used

  12. Our Service: Statefull Use Case

  13. Our Service: Interface • Requests and Responses <xsd:element name="detect" type="xsd:string" /> <xsd:element name="detectResponse“ type="xsd:string“ /> <xsd:element name="getLanguageRP"> <xsd:complexType /> </xsd:element> <xsd:element name="getLanguageRPResponse" type="xsd:string" /> <xsd:element name="getLastOpRP"> <xsd:complexType /> </xsd:element> <xsd:element name="getLastOpRPResponse" type="xsd:string" /> <xsd:element name="getTimesUsedRP"> <xsd:complexType /> </xsd:element> <xsd:element name="getTimesUsedRPResponse" type="xsd:int" />

  14. Our Service: Interface • Port Types <portType name="LanguageDetectorPortType" … > <operation name="detect"> <input message="tns:DetectInputMessage" /> <output message="tns:DetectOutputMessage" /> </operation> <operation name="getLanguageRP"> <input message="tns:GetLanguageRPInputMessage" /> <output message="tns:GetLanguageRPOutputMessage" /> </operation> <operation name="getLastOpRP"> <input message="tns:GetLastOpRPInputMessage" /> <output message="tns:GetLastOpRPOutputMessage" /> </operation> <operation name="getTimesUsedRP"> <input message="tns:GetTimesUsedRPInputMessage" /> <output message="tns:GetTimesUsedRPOutputMessage" /> </operation> </portType>

  15. Our Service Implementation

  16. Language Detection: Basic Idea • Essentially based on probabilities of character combinations • Every language has typical character combinations that are very frequent in that language • “th” in english • “ij” in dutch • Easy for humans to detect a language even when we don’t know that specific language

  17. Language Learning: Standard Process • Standard machine learning process

  18. Language Learning: Markov Models • Basic Markov Model • kth order Markov Model

  19. Language Detection: Classification • Transitional probabilities estimated as • Classification

  20. Language Detection: Example • The training text for a language consists of the string • Learned model: • the probability of the string would be: test text test ( ^^, t, 1.0 ) ( ^t, e, 1.0 ) ( te, s, 0.5 ) ( es, t, 1.0 ) ( st, , 1.0 ) ( te, x, 0.5 ) ( ex, t, 1.0 ) ( xt, , 1.0 ) P(test|L) = P(t|^^)*P(e|^t)*P(s|te) *P(t|es)*P(_|st) = 1*1*0.5*1*1=0.5

  21. Language Detection: Performance

  22. Problems Encountered • Necessary tools had to be installed (ANT) • Problems on our machine (GRAM) • Conflicts with other team • Buggy shell script to build gar file • Sensitive to path lengths/ names

  23. Future Work • Connect with other services • Make training and evaluation a grid service • Make it part of a multi lingual retrieval engine • Web interface (interactive)

  24. Conclusions • Successfully managed to create and deploy our own web service • Broke loose from the tutorial web service structure • Merged Grid Computing with AI • Got hands on experience with Grid applications and structure • A lot of possibilities to integrate and/or extend the implemented service

  25. Questions ?

More Related