Interoperability of Distributed Component Systems

1. Interoperability of Distributed Component Systems Midterm Presentation CSE333 – Distributed Component Systems Professor Steven Demurjian Research Team Bryan Bentz Jason Hayden Upsorn Praphamontripong Paul Vandal

2. As of Mid-Term Review: Project Direction We had examined different applications of XML: • Software component re-use • Open Office documents • Common Data Warehouse • Semantic Web The issue was: What do these have to do with each other?

3. Endemic XML Problem XML is like HTML, but users may define tags:<Title>Example Document</Title> <Created>December 1, 2005</Created> But this second line could as easily be:<CreationDate>12/1/2005</CreationDate>

4. All XML representations share this problem: in what contexts does the metadata make sense? In a single context, this usually isn’t a problem (though systems do grow and change); Metadata Metadata Data: code, text, databases, services Data: code, text, databases, services

5. All XML representations share this problem: in what contexts does the metadata make sense? For unanticipated interactions between systems, this requires new mechanisms. ? Metadata Metadata Data: code, text, databases, services New web service

6. Project Direction • While a universal XML problem, this consistency issue is central to the Semantic Web; • We chose to use Semantic Web ideas as a unifying theme in what we were doing. • We have been looking at different areas, but this problem of XML and interoperability is the common thread.

7. Semantic Web: The Common Thread Topics to cover: • RDF: Resource Description Framework • Practical RDF • Our Semantic Web server experiments

8. Resource Description Framework • RDF originated with the work of R.V. Guha at Apple, on what was called the Meta Content Framework. • RDF is a major component of the Semantic Web; • It is used to describe resources and their properties; • It has its own XML representation; • It consists of triples: Why triples?

9. Semantic Network Representation • A very general and powerful representation; • Invented by Richard Richens of Cambridge Language Research Unit in 1956 for natural language work/machine translation; widely used since then. • Strongly influenced the ‘link’ concept in HTML.

10. RDF Triples Encode a Semantic Network

11. Practical RDF • Want to use common set of objects and relationships – namespaces, repositories – when possible. • An alternative approach is to provide translations (e.g. synonyms) to map homegrown properties to universally accepted properties. • One example of universal components: • Dublin Core.

12. Practical RDF: The Dublin Core Dublin Core Metadata Element Set: • Title2. Creator3. Subject4. Description5. Publisher6. Contributor7. Date8. Type 9. Format10. Identifier11. Source12. Language13. Relation14. Coverage15. Rights (Properties may be further qualified.) Terms are described in detail at http://dublincore.org/documents/dcmi-terms/

13. RDF/XML <?xml version="1.0"?> <!DOCTYPE rdf:RDF PUBLIC "-//DUBLIN CORE//DCMES DTD 2002/07/31//EN" "http://dublincore.org/documents/2002/07/31/dcmes-xml/dcmes-xml-dtd.dtd"> <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:dc="http://purl.org/dc/elements/1.1/"> <rdf:Description rdf:about="http://example.org/"> <dc:title>My Home Page</dc:title> </rdf:Description> How to read this: XML declarationReferencing the XML DTDDeclaring the use of RDFMarking the title.

14. Building A Semantic Web Server • We used RDF Gateway, by Intellidimension (www.intellidimension.com), a platform for development and deployment of Semantic Web applications. • We chose to represent software components and associated documentation, and the relationships between these elements. • We built RDF triples describing these relationships, and exercised them using inference rules.

15. RDF Gateway:

16. Components Represented: • FFT code in C++ • FIR filter code in C • Hidden Markov Model code in C • Wavelet implementation in Java Each component had one or more source code files and/or documentation files; e.g. for the HMM, we had: hmm.c The basic code filehmm.h The header filehmmrand.c Platform-independent random nums.hmmutils.c File I/O, matrix codehmmtut.ps Postscript documentation

17. Semantic Network Fragment Fast Fourier Transform FIR filters filter.c FFT.c++ Language: C FFT.h FFT.doc Language: C++ Hmmtut.ps hmm.c Language: Java Hmmrand.c hmm.h Wavelet.java Hmmutils.c Wavelets sysutils.h Hidden Markov Models

18. Example RDF triples: “source/hmm.c” requires “source/hmmutils.c”“source/hmmtut.ps” documents “source/hmm.c”“source/hmmutils.c” requires “sysutils.h”… • RDF triples were instantiated using the RDFQL interface in RDF Gateway; • There were no global software-engineering-related predicates we could draw on.

19. Using the ‘Semantic’ Part of the Semantic Web Inference Rule Example: INFER {?A 'requires' ?C} FROM {?A 'requires' ?B} AND {?B 'requires' ?C}; Rule is recursively applied to identify all dependencies for a given file. Result when passed hmm.c is:source/hmmutils.c source/hmmrand.c source/hmm.h source/sysutils.h

20. Further Inference • A similar rule returns appropriate documentation for any of the files returned. • The inference engine is quite powerful, and may be used to examine database schemas, rewrite metadata, or reformat data. • We used the output of inferences to write HTML on the fly – the inference engine became a window into the Semantic Web.

21. Review So Far • We chose the problem of disparate XML as the unifying theme; • Semantic Web technologies exist specifically to address this problem; • This is the unifying thread in our different research areas.

22. OpenDocument Introduction • What is it? • Why is it important? • Who created it? • How is XML used to contain information? • Testing the portability of documents • OpenDocument and RDF • Conclusions

23. What is OpenDocument? • Open source document file format for saving and exchanging documents across multiple editors. • Editors include: OpenOffice, StarOffice, KOffice, IBM Workplace, and Abiword. • All information is stored in XML files • Constantly evolving to incorporate newest ideas • Currently trying to become ISO certified

24. Why is it important? • Allows the use of multiple document editors • Stored in XML compared to .doc binaries allows easy access to data. • Example pre MS Word 95 unreadable in Office 2003 • European Union requiring document formats to be approved by standards body. • Microsoft – No • OpenDocument - being reviewed • Massachusetts recommending all state documents being in non proprietary format. Debate ongoing.

25. Who created it? • Developed by OASIS consortium • OASIS is composed of multiple large vendors including: Adobe, Corel, IBM, KDE, and Sun • Based off old OpenOffice file format but modified per input from different vendors

26. How is XML used to Store information? • OpenDocument file is a compressed archive file containing the following items: • Meta.xml • Settings.xml • Styles.xml • Content.xml • Manifest.xml • Pictures folder • Thumbnail.png • Configuration2 folder • Only 2 of the above files are needed to create a valid OpenDocument file • XML files contain a combination of Dublin Core and custom elements

27. Meta.xml • Used to store information in XML describing the document. • Some information stored is title, creation date, and creator

28. Settings.xml • Saves settings and options used by the editor that last saved the document. • Application independent and not required • Examples would be curser location and page when file saved

29. Styles.xml • Stores information about the format and layout of text that are used within the document. • Styles can be applied to anything ranging from document to character. • Information about styles can also be stored in content.xml.

30. Content.xml • One of the 2 required files • Contains all the content that is entered in a document. • Can store styles instead of styles.xml but not encouraged.

31. Other Components • Pictures folder – stores all images used within document. .bmp are converted • Thumbnail.png – thumbnail picture of entire document. Print preview. • Configurations2 – not currently used. Planned for future use • Manifest.xml – 2nd required file. Used by the document editor to know which XML files are created and where they are located.

32. Testing Portability • The ability to create and edit office documents in variety of editors is tested. • Four different documents (2 text and 2 spreadsheet) are created in OpenOffice 2.0 and tested for portability with StarOffice 8.0 and Koffice 1.4.2 • Text and spreadsheet used because received most attention and most used in business. • OpenOffice, StarOffice, and KOffice are the largest vendors supporting OpenDocument.

33. Simple Text Document • Text document that says “Open Document test” in variety of fonts and styles. • The file opened correctly in all three applications.

34. Advanced Text Document • Advanced text document that has a table, image, shading, and variety of fonts, sizes, and justifications. • Opened correctly in all three applications

35. Simple Spreadsheet Document • Simple spreadsheet that performs addition, subtraction, multiplication, and division. • Opened correctly in all three applications

36. Advanced Spreadsheet Document • The complex equation is entered:=(SUM(A1;A3;AVERAGE(A1:A5)+LOG(A1+10;10)-SQRT(A4*SIN(0.75)+MOD(A3;A4))) + (POWER(3;3)/LN(A4))) • Opened correctly in OpenOffice and StarOffice • KOffice had ERR:511 in the cell where the equation appears

37. Advanced Spreadsheet Document(2) • Problems occur because OpenDocument does not define a standard for functions, only formatting. • Currently each application defines their own standard for function formatting. • New research into extending OpenDocument to developing a standard to exchange formulas in spreadsheet called OpenFormula proposed by David Wheeler. Currently in draft status.

38. OpenDocument and RDF • With the use of RDF, documents can be easily found by subject, author, creation date, or keywords. • Information can be retrieved through accessing the Dublin Core elements in the meta.xml file. • Information sharing and searching are easily achieved without requiring user input during insertion into the database • An ongoing topic of research in ways to efficiently parse the XML files for information.

39. OpenDocument Conclusions • The documents are not yet universally portable but work is continuing to improve this ability. • Information stored in XML results in many benefits over a binary file. • With the use of RDF, an exciting future is ahead with the ability to find documents extremely efficiently and easily.

40. Library Component Specification Lexical Analyzer Syntactic Analyzer Extracted library component specification Semantic Analyzer Component Analyzer Retrieved components Match Maker Query Component Specification Lexical Analyzer Syntactic Analyzer Extracted query component specification Semantic Analyzer Component Analyzer Software Component Retrieval: Basic Architecture

41. XML-based software component specification A component – method with a set of individual conditions <component> <cname> component_name </cname> <method> <mname> method_name </mname> <paramNo> no_of_parameter </paramNo> <param> <pname> parameter_name </pname> <ptype> type_of_parameter </ptype> <psize> size_of_parameter </psize> </param> … <precond> <left> left_operand1 </left> <operator> operator1</operator> <right> right_operand1</right> </precond> <precond> <left> left_operand2 </left> <operator> operator2</operator> <right> right_operand2</right> </precond> … <postcond> <left> left_operand1 </left> <operator> operator1</operator> <right> right_operand1</right> </postcond> <postcond> <left> left_operand2 </left> <operator> operator2</operator> <right> right_operand2</right> </postcond> … <return> return_type </return> </method> … </component> A component – method with Boolean conditions <postcond> <subcond> <left> left_operand1 </left> <operator> operator1</operator> <right> right_operand1</right> </subcond> <and> <subcond> <left> left_operand2 </left> <operator> operator2</operator> <right> right_operand2</right> </subcond> </and> <and> <subcond> <left> left_operand3 </left> <operator> operator3</operator> <right> right_operand3</right> </subcond> </and> … </postcond>

42. XML-based software component specification A component – method with multi-level conditions (Boolean join) <postcond> <subcond> <left> left_operand1 </left> <operator> operator1</operator> <right> right_operand1</right> <and> <subcond> <left> left_operand2 </left> <operator> operator2</operator> <right> right_operand2</right> <and> <subcond> <left> left_operand3 </left> <operator> operator3</operator> <right> right_operand3</right> </subcond> … </and> </subcond> </and> </subcond> </postcond> An “isa” component <component> <cname> component_name1 </cname> … </component> <component> <cname> component_name2 </cname> <isa> component_name1 </isa> <method> <mname> method_name </mname> <paramNo> no_of_parameter </paramNo> <param> <pname> parameter_name </pname> <ptype> type_of_parameter </ptype> <psize> size_of_parameter </psize> </param> … <precond> … </precond> <postcond> … </postcond> <return> return_type </return> </method> … </component> A component – method with multi-level conditions <postcond> <left> <subcond> <left> left_operand1 </left> <operator> operator1</operator> <right> right_operand1</right> </subcond> </left> <operator> operator2 </operator> <right> right_operand2</right> </postcond>

43. Software Component Retrieval: Determine Matching • Exact Match MatchExact(Q, L) = (QpreLpre) (QpostLpost)  (QinLin)  (QoutLout) • Generalization Match MatchGen(Q, L) = (Lpre  Qpre  Lpost  Qpost) • Specialization Match MatchSpec(Q, L) = (QpreLpreQpostLpost) • Partial Match MatchPart(Q, L) = (MatchGen(Q, L) MatchSpec(Q, L))  (L’inLin (L’in = Qin) L’outLout (L’out = Qout)) • Reference Match MatchRef(Q, L) = (MatchExact(Q, l) MatchPartial(Q, l))  (MatchExact(q, L) MatchPartial(q, L)) Q and q denote queries L and l denote existing components in the library/repository

44. Software Component Retrieval: Compute Similarity

45. <query> <component> <cname> q2 </cname> <method> <mname> m2 </mname> <paramNo> 2 </paramNo> <param> <pname> k </pname> <ptype> real </ptype> </param> <param> <pname> j </pname> <ptype> real </ptype> </param> <precond> <left> j </left> <operator> greater_or_equal </operator> <right> 0 </right> </precond> <postcond> <left> k </left> <operator> greater_or_equal </operator> <right> 0 </right> </postcond> <postcond> <left> <subcond> <left> k </left> <operator> power </operator> <right> 2 </right> </subcond> </left> <operator> equal </operator> <right> j </right> </postcond> <return> k </return> </method> </component> </query> Component name: q2 Parameter ----> Method: m2 k Method: m2 j Pre condition ----> Method: m2 weight =100.00% j greater_or_equal 0 Post condition ----> Method: m2 weight =50.00% k greater_or_equal 0 Method: m2 weight =50.00% ( k power 2 ) equal j Return value ----> Method: m2 k Display Similarity Table Format: component name, method name, Sim-Post, Sim-Pre, Sim-Param, Sim-Return, method-sim, match_type, ref_match =========================================================q2, math1, sqr, 50.0, 100.0, 100.0, 100.0, 75.0, generalization q2, math2, sqr2, 100.0, 100.0, 100.0, 100.0, 100.0, exact q2, math3, sqr, 0.0, 100.0, 100.0, 100.0, 50.0, partial Software Component Retrieval: Illustration Repository/Library: lib1.xml Query: q2.xml Weight: input: 10 output: 10 precondition: 30 postcondition: 50

46. Software Component Retrieval: Illustration Component name: q_math1 Parameter ----> Method: q_sqr x Method: q_sqr r Pre condition ----> Method: q_sqr weight =100.00% x greater_or_equal 0 Post condition ----> Method: q_sqr weight =100.00% ( r power 2 ) equal x Return value ----> Method: q_sqr r Component name: isa_test1 Parameter ----> Method: find_max param1 Method: find_max param2 Pre condition ----> Method: find_max weight =50.00% param1 greater 0 Method: find_max weight =50.00% param2 greater 0 Post condition ----> Method: find_max weight =100.00% param1 greater param2 Return value ----> Method: find_max param1 Display Similarity Table Format: component name, method name, Sim-Post, Sim-Pre, Sim-Param, Sim-Return, method-sim, match_type, ref_match ================================================================ q_math1, math1, sqr, 100.0, 100.0, 100.0, 100.0, 100.0, exact q_math1, math2, sqr2, 100.0, 100.0, 100.0, 100.0, 100.0, specialization q_math1, math3, sqr, 0.0, 100.0, 100.0, 100.0, 55.0, partial isa_test1, list_operation, add_list, 0.0, 0.0, 50.0, 0.0, 5.0, partial isa_test1, simple_array, add_to_array, 0.0, 0.0, 100.0, 0.0, 10.0, partial isa_test1, simple_array, find_max, 100.0, 0.0, 100.0, 100.0, 65.0, partial isa_test1, stack_operation, pop, 0.0, 50.0, 50.0, 0.0, 22.5, partial isa_test1, stack_operation, push, 0.0, 0.0, 100.0, 0.0, 10.0, partial isa_test1, graphic_obj, pop, 0.0, 50.0, 0.0, 0.0, 17.5, partial isa_test1, math1, sqr, 100.0, 100.0, 100.0, 100.0, 100.0, generalization-reference isa_test1, math2, sqr2, 100.0, 100.0, 100.0, 100.0, 100.0, generalization-reference Repository/Library: lib1.xml Query: q_isa_1.xml Weight: input: 10 output: 10 precondition: 35 postcondition: 45

47. Component Description Repository Component Integrators Component Vendors Matchmakers M1 … … Query Interface M2 Publish Interface … Mn Licensing server Software Component Retrieval: Issues (Varadarajan, et. al. “Componentxchange: An E-Exchange for Software Components”, WWW’10, 2001)

48. input-type (relation) datatype (relation) (relation) add C1 insert C2 Software Component Retrieval: RDF • Different tags refer to the same element <input-type> string </input-type> and <datatype> string </datatype> • Different terms refer to the same content • For an array’s operations: <operator> add </operator> and <operator> insert </operator> RDF: INFER {?s ?B ?o} FROM {?A is-a-synonym-of ?B} AND {?s ?A ?o}; • Integration of software component specifications • Same standards of specifications • different standards of specifications

49. Integration of software component specifications described the same standards <component> <cname> simple_array </cname> <method> <mname> find_max </mname> … <precond> <subcond> <left> param2 </left> <operator> in </operator> <right> temp_array </right> <and> <subcond> <left> param2 </left> <operator> greater </operator> <right> 0 </right> </subcond> </and> </subcond> </precond> <postcond> <left> param1 </left> <operator> greater </operator> <right> param2 </right> </postcond> <return> param1 </return> </method> <method> <mname> add </mname> … <precond> <left> x </left> <operator> not_in </operator> <right> A </right> <and> <subcond> <left> A </left> <operator> not </operator> <right> full </right> </subcond> </and> </precond> <postcond> <left> x </left> <operator> in </operator> <right> A </right> </postcond> <return> none </return> </method> </component> <component> <cname> stack_operation </cname> <method> <mname> push </mname> … <precond> <left> n </left> <operator> greater_equal </operator> <right> 0 </right> </precond> <precond> <left> S </left> <operator> not </operator> <right> full </right> </precond> <postcond> <left> x </left> <operator> in </operator> <right> S </right> </postcond> <postcond> <left> x </left> <operator> top </operator> <right> S </right> </postcond> <postcond> <left> n </left> <operator> increase </operator> <right> 1 </right> </postcond> <return> none </return> </method> <method> <mname> pop </mname> … <precond> <left> n </left> <operator> greater </operator> <right> 0 </right> </precond> <postcond> <left> n </left> <operator> decrease </operator> <right> 1 </right> </postcond> <return> peak </return> </method> </component>

50. Integration of software component specifications described with different standards <component> <cname> simple_array </cname> <method> <mname> find_max </mname> … <precond> <subcond> <left> param2 </left> <operator> in </operator> <right> temp_array </right> <and> <subcond> <left> param2 </left> <operator> greater </operator> <right> 0 </right> </subcond> </and> </subcond> </precond> <postcond> <left> param1 </left> <operator> greater </operator> <right> param2 </right> </postcond> <return> param1 </return> </method> <method> <mname> add </mname> … <precond> <left> x </left> <operator> not_in </operator> <right> A </right> <and> <subcond> <left> A </left> <operator> not </operator> <right> full </right> </subcond> </and> </precond> <postcond> <left> x </left> <operator> in </operator> <right> A </right> </postcond> <return> none </return> </method> </component>