1 / 48

Towards a Language-Independent Universal Digital Library

Explore challenges in breaking language barriers, the UNL system, and requirements for universal knowledge representation in this insightful conference summary.

lcastaneda
Download Presentation

Towards a Language-Independent Universal Digital Library

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Towards a Language-Independent Universal Digital Library The Second International Conference on Universal Digital Libraries (ICUDL 2006) 17-19-2006 November, Alexandria, Egypt Sameh Alansary Magdy NagiNoha Adly sameh.alansary@bibalex.orgmagdy.nagi@bibalex.orgnoha.adly@bibalex.org Bibliotheca Alexandrina

  2. Introduction • IT made the full text libraries’ assets available digitally (Independent of time, place and copy). e.g. - Million Book Project. - Nasser Digital Library. UDL • Digitization only does not lead to “universality” in its optimum sense. • A new dimension of universality should be added: Independency of Language

  3. Language-dependency blocks information dissemination • Language dependency holds language barriers. • If it is always possible for everyone to read in everyone’s mother tongue, this will help in: • Dissemination of knowledge. - Preservation of nationality and identity. - Preventing cultural hegemony. • 80% of books and e-materials is written in English and 20% is written in other languages.

  4. Attempts to break language barriers • Translation systems have been introduced (NLP): Approaches: 1- Direct translation approach. 2- Transfer approach. 3- Interlingual approach. • Examples of Systems: • Google translation:http://www.google.ch/language_tools • Fujitsu systems:http://www.fujitsu.com/global/services/translation

  5. Drawback of MT systems 1- The quality of results is often inadequate. 2- Work for a limited number of language combinations. 3- Hold an overload on the network: To translate from and to only 10 languages, 10 grammars, 10 lexicons, 90 translation dictionaries and 90 sets of translation rules will be needed, plus the need for semantic processing in each language.

  6. Towards a universal system for knowledge representation

  7. Some questions may bear in mind: • How can we represent natural language materials in a language independent format? (a format required) • What is the system suitable for representing knowledge in the format selected? (a system required) • How is this system going to work?

  8. Requirements for a universal representation of knowledge: 1- The content of the original material (meaning) must not be lost. 2- This universal format should be understandable by various platforms over the network. 3- This universal format should be decodable to any natural language.

  9. UNL System

  10. What is UNL? (1) • The Universal Networking Language (UNL) is an artificial language for computers to express information and knowledge that can be expressed in natural language. • Started in 1996, as an initiative of the UNU/IAS in Japan • R&D in UNL - Development on 15 languages: Arabic, Chinese, English, French, German, Hindi, Indonesian, Italian, Japanese, Korean, Portuguese, Russian, Spanish, Thai, Swahili. - Transferred to the UNDL Foundation in 2001.

  11. What is UNL? (2) • It expresses information or knowledge of natural language (NL) in the form of semantic network with hyper-node. {UNL} agt(go(icl>move).@entry.@past, :01) plt(go(icl>occur).@entry.@past, school(icl>institution)) agt:01(work(icl>do), boy(icl>person.@entry)) plc:01(work(icl>do),here) {/UNL} The boy who works here went to school Example: UNL expression:

  12. The boy who works here went to school go(icl>move) @ entry @ past agt plt boy(icl>person) @ entry school(icl>institution) agt here plc work(icl>do) :01 UNL-hyper graph

  13. The UNL System Formalism System Components Knowledge representation

  14. USER The UNL-system components UNL LANGUAGE SERVER Enconverter =  Deconverter (EnCO) (EnCO) Language Server UNL <- >Chinese EnCO DeCO UNL document Language Server UNL <-> Arabic UNL Editor UNL Viewer UNL Proxy Language Server UNL <-> Spanish Internet 1 2 3 Language Server UNL <-> Hindi EnCO DeCO Language Server UNL <- >Japanese Language Server UNL <- > English EnCO DeCO EnCO DeCO

  15. Natural Language A) Language servers: Analysis Rules Web Server with UNL document EnConverter UNL UNL-language Dictionary Knowledge Base UNL UNL Language Server DeConverter NL Concurrence Dictionary Generation Rules

  16. C) UNL Proxy Server: B) UNL Tools: 1- UNL viewer. • Searches for UNL at the web, send it to the language server and displays it on the user’s chosen language. 2- UNL editor. 3- UNL verifier.

  17. Mechanism of conversion between NL and UNL Annotated Natural Language texts Annotation Editor Universal Parser UNL Verifier EnConverter Natural Language texts UNL Document Grammatical Rules Word Dictionary Co- Occurrence Dictionary UW Dictionary UNL KB Web server HTML+XML Natural Language texts UNL Document DeConverter

  18. UNL as a formal language: How does it represent knowledge? 1- Universal words (UW): to represent concepts. Example: boy(icl>person) hear(icl>perceive(agt>person,obj>thing)) 2- relations: 38 semantic relations can be distinguished. Example: agt, aoj, bas, con, coo, dur, … etc. 3- Attributes: to express subjectivity of the speaker. Example: @past, @emphasis, @def, @not, … etc.

  19. 4- Knowledge base (UNLKB). • Define the Universal Word. • Provide linguistic knowledge of concepts

  20. Ibrahim Shihata UNL Arabic Center (ISUAC) • It is established at Bibliotheca Alexandrina. • It is responsible for designing, implementing, and maintaining the various components of the Arabic language server. • The Arabic language server will be capable of: - Enconverting the Arabic texts to the universal format. - Deconverting the universal materials produce by other language centers to Arabic.

  21. The Achievements of the ISUAC A) Arabic language resources and tools. B) Developing tools. C) Arabic language-based universal materials.

  22. A) Arabic language resources and tools: 1- The Arabic Dictionary: It is a repository of information for all UNL Arabic grammars. Dictionary Head Words (Vocabulary of Arabic) Universal words (Vocabulary of UNL) Linguistics Features (Linguistic info about HWs)

  23. 2- Arabic EnConversion Rules: • It is responsible for Enconverting Arabic to UNL. • Arabic EnConversion Rules are able to: 1- Perform morphological analysis to extract concepts the Arabic words refer to. 2- Assign exact semantic relation between concepts as being expressed in the context of the Arabic sentence.

  24. Simulation of how Enconverter works ولد جمال عبد الناصر في 15 يناير 1918 في 18 شارع قنوات حي باكوس بالإسكندرية. ولد / /جمال عبد الناصر/ /في/ /15/يناير/ /1918/ /في/ /18/ /شارع/ /قنوات/ /حي/ /باكوس/ /بال/إسكندرية/. delete delete mod mod mod obj tim mod tim plc plc plc

  25. UNL Network:

  26. 3- Arabic DeConversion Rules: • It is responsible for generating Arabic sentences out of UNL networks. • Arabic DeConversion Rules are able to: 1- Select Arabic words that represent universal concepts. 2- Arrange the concepts of the UNL network in a syntactically well-formed sentence.

  27. Simulation of how the Deconverter works description(icl>action) obj Egypt aoj outcome(icl>resul).@entry collaboration(icl>action) mod agt bas aoj More (aoj>thing) 150 scientist(icl>scholar) .@entrry aoj prominent(aoj>thing) and Egypt gol scholar(icl>person) accompany(agt>thing,obj>thing) 1798 agt tim Bonaparte(iof>person) obj و مرموق تعاون وا باحث الذين عالم ة صاحب أكثر إلى في بونابرت مصر مصر وصف محصل من 150 1798 وصف مصر محصلة تعاون أكثر من 150 باحث و عالم مرموق الذين صاحبوا بونابرت في 1789 إلى مصر

  28. 4- A Corpus for Modern Standard Arabic: • A representative sample (100 Millions) that reflects the empirical usage of Modern Standard Arabic. • It plays a principle role in enhancing and updating both EnConversion and DeConversion rules.

  29. B) Developing tools: 1- Integrated Development Environment (IDE)

  30. 2- Corpus analysis software (GATE)

  31. C) Arabic language-based universal materials. Library of Alexandria: the Fourth Pyramid. Abou Simple: The Temple of the Sun. Nasser Digital Library The Encyclopaedia of Famous Persons

  32. An example of an Arabic Sentence in UNL (universal) format

  33. وكان جمال عبد الناصر الابن الأكبر لعبد الناصر حسين الذي ولد في عام 1888 في قرية بني مر في صعيد مصر في أسرة من الفلاحين، ولكنه حصل على قدر من التعليم سمح له بأن يلتحق بوظيفة في مصلحة البريد بالإسكندرية، وكان مرتبه يكفي بصعوبة لسداد ضرورات الحياة. {unl} aoj(son(icl>person):0I.@def.@entry, Gamal Abdel Nasser(iof>person):00) mod(son(icl>person):0I.@def.@entry, Abd El-Naser Hosen(iof>person):23.@topic) aoj(old(aoj>thing):1J, son(icl>person):0I.@def) man(old(aoj>thing):1J, most(icl>how):15) obj(born(obj>thing):31.@past, Abd El-Naser Hossain(iof>person):23.@topic) and(get(agt>thing,obj>thing):6S.@past.@contrast, born(obj>thing):31.@past) scn(born(obj>thing):31.@past, family(icl>group):5Q) plc(born(obj>thing):31.@past, village(icl>region):4D) tim(born(obj>thing):31.@past, year(icl>period):3M) mod(year(icl>period):3M, 1888:41) plc(village(icl>region):4D, upper Egypt(iof>place):58) mod(village(icl>region):4D, Bani Morr(iof>village):4S) mod(family(icl>group):5Q, farmer(icl>person):65.@pl.@def) obj(get(agt>thing,obj>thing):6S.@past.@contrast, degree(icl>abstract thing):7N) agt(allow(agt>thing,gol>thing,obj>thing):8M.@past, degree(icl>abstract thing):7N) mod(degree(icl>abstract thing):7N, education(icl>activity):82.@def) gol(allow(agt>thing,gol>thing,obj>thing):8M.@past, join(agt>person,obj>thing):9I.@present) obj(allow(agt>thing,gol>thing,obj>thing):8M.@past, his(pos>he):97) and(suffice(aoj>thing,obj>thing):CM.@present, join(agt>person,obj>thing):9I.@present) obj(join(agt>person,obj>thing):9I.@present, job(icl>work):A7) plc(job(icl>work):A7, postal service{icl>service ):AN) plc(postal service{icl>service ):AN, Alexandria(iof>city):BB) aoj(suffice(aoj>thing,obj>thing):CM.@present, salary(icl>money):BV) mod(salary(icl>money):BV, his(pos>he):CB) obj(suffice(aoj>thing,obj>thing):CM.@present, satisfy(agt>thing,obj>thing):DQ) man(suffice(aoj>thing,obj>thing):CM.@present, hardly:DA) obj(satisfy(agt>thing,obj>thing):DQ, demand(icl>wants):E6.@pl.@def) mod(demand(icl>wants):E6.@pl.@def, life(icl>activity):EV.@def) {/unl} Language -Independent Format

  34. Is it going to work this way?!! • Are there language servers ready to work? • Are the universal materials deconvertable to other languages? What about Arabic?? • Is the Arabic language server able to enconvert Arabic texts to universal format? • Is it also able to deconvert the universal materials back to Arabic?

  35. A proof of the concept

  36. UNL-based Library Information System (UNL-LIS) • It is a system to search in a digital library catalogs. • It is built on the UNL KI, therefore: - Query is in Natural Language (two languages) • Answer is also in Natural Language (7 languages)

  37. UNL LIS Core Architecture User Question LIS Language Server Enco rules + Dic Question in NL MARC21 Records Enconversion Process Question in UNL UNL KB MARC21 Importing Process Query Engine Encyclopedia Answer in UNL Concepts Definitions Deconversion Process Language Server Deco rules + Dic Answer in NL

  38. Demo: Screen Shots

  39. 1. Enter query 2. Press to search Encyclopedia 3. Specify result's language (Arabic) 4. View results here (Naguib Mahfouz). Click for more information. 5. A link to the UNL document {unl} agt(begin(agt>thing,obj>action):12.@past.@entry, Naguib Mahfouz(iof>person):0N.@topic) obj(begin(agt>thing,obj>action):12.@past.@entry, writing(icl>action):18) tim(begin(agt>thing,obj>action):12.@past.@entry, year old:1S.@past) aoj(year old:1S.@past, Naguib Mahfouz(iof>person):0N.@topic) qua(year old:1S.@past, 17) plc(born(aoj>thing):00, Cairo(iof>city):08) aoj(born(aoj>thing):00, Naguib Mahfouz(iof>person):0N.@topic) tim(born(aoj>thing):00, 1911:0H) {/unl} [/S] ;;Time 1.4 Sec ;;Done! {unl} and(write(agt>thing,obj>thing):1K.@past.@entry, publish(agt>thing,obj>thing):0K.@past) obj(write(agt>thing,obj>thing):1K.@past.@entry, novel(icl>tale):1B.@pl.@topic) tim(write(agt>thing,obj>thing):1K.@past.@entry, before(icl>how(obj>thing)):1S) aoj(more(icl>additional):1A, novel(icl>tale):1B.@pl.@topic) qua(novel(icl>tale):1B.@pl.@topic, 10:16) [/S]

  40. Conclusion

  41. Conclusion • Independency of language is a very important dimension that should be considered in storing and retrieving texts for a UDL • The UNL system is a promising formalism for representing knowledge in a universal format. • The ISAUC less than 2 years old, however, it is one of the very active language centres in designing and implementing UNL materials and tools. • The UNL LIS has proved feasibility of the concept of language independency.

  42. Thank YouAny question is welcomed.

More Related