Software Applications for Cultural Diversity

1. Software Applications for Cultural Diversity Rod Davis, SIL International Comp_dept_mali@sil.org, rod_davis@sil.org

2. About SIL International • faith-based International NGO • language-based development • research, translation and literacy • active in 20% of the world’s languages

3. SIL Language Software Development • 53 software development personnel • Member of the Unicode Consortium and very active on behalf of minority languages • 60 software titles to support the work of language fieldworkers

4. One of the Biggest Challenges: Dealing with Complex Scripts • Encoding • Transition to Unicode • Input (Keyboarding) • Complexity • Extensibility • Type Design • Unicode based • Smart-font compiler technology • Rendering • Diacritic placement, Contextual shaping, Ligatures, Reordering/splitting, Bi-directionality

5. Solutions in Dealing with Complex Scripts • Encoding • TechKit, a utility to convert legacy encodings to Unicode • http://scripts.sil.org: SIL resource site with information, tutorials, utilities, etc. for making the transition to Unicode • Input • Keyman (Windows); http://tavultesoft.com • KMFL (a Linux ‘Keyman’ still under development) • IMEs (Input Method Editors) • Type Design • Graphite Compiler adds Graphite rendering tables to a TrueType font, giving it ‘smart-font’ capability • Rendering • Graphite rendering engine: Can handle basic display of any complex script in use today. • NOTE: These technologies can benefit not only ‘complex scripts’, but also orthographies with special diacritics or a few non-Roman characters.

6. What is a Smart Font? • A font containing data describing how the glyphs are to displayed. • The smart font data is in the form of tables inside the font file itself. • A rendering engine interprets the font tables to appropriately render the glyphs. • In contrast, a ‘dumb font’ has only a direct correspondence between the data characters and the displayed glyphs.

7. What is Graphite? • A package, developed by SIL International, that can be embedded into other applications, adding "smart font behavior" to that application. • This package includes the following: • A programming language (GDL) for specifying the font's behavior • A compiler for building the smart font • A rendering engine for displaying text using the smart font • Note that it is open-source:http://graphite.sil.org/http://sourceforge.net/projects/silgraphite

8. What Makes Graphite Special? • It differs from other complex script technologies: • Unlike OpenType, it does NOT assume that script-specific information is incorporated at the application or operating system level (as in Uniscribe). • It is extensible, and provides support for characters in Unicode’s Private Use Area ranges … unlike Uniscribe, which handles only script behaviors already part of the Unicode standard. • It is an open-source solution versus proprietary. • It can handle basic display of any complex script in use today. • The following slides demonstrate the types of rendering difficulties that Graphite was designed to handle:

9. http://scripts.sil.org/

10. http://scripts.sil.org/

11. http://scripts.sil.org/

12. http://scripts.sil.org/

13. http://scripts.sil.org/

14. SIL’s Work with UNESCO • In 2003, SIL International and UNESCO engaged in a cooperative project as part of UNESCO’s Initiative B@bel effort. http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&cat_id=Babel • Goal: Enable the development of complex script support in information and communication technologies (ICTs). • Project SILA: A Graphite-enabled version of Mozilla • Project Goal: To enable minority language communities to publish on the Internet … • Graphite-enabled Edit Control Version 0.9 • basic Graphite edit control for data input applications • SDK for Windows 2000/XP, developer’s guide and well-commented source code • Modified version of WorldPad -- Graphite-Enabled Text Processor • Simple text processor for Windows 2000/XP

15. Graphite-enabled Mozilla Contrast Internet Explorer on the left with Graphite-enabled Mozilla on the right. The Graphite-enabled version of Mozilla displays the multiple diacritics separately as intended. http://sila.mozdev.org/silab2.htm

16. Graphite-enabled Mozilla E-mail

17. Graphite-enabled Mozilla Instant Messaging

18. Graphite-enabled WorldPad http://www.ethnologue.com/tools_docs/fieldworks.asp

19. Lessons learned We are participating in or have created several open source development efforts. We have learned that… • FLOSS (Free/Libre Open Source Software) Linux is very attractive to low income groups we work with. • Setup and maintenance of open source OS and software is too complex for most of our end users. • Open source development is complex. • Getting support for complex minority scripts into the “core” builds of open source software is hard. • Localization of software is complex and time consuming. • Despite the difficulties, the benefits are great and we are heading in this direction on a number of fronts.

20. What is Keyman? • Keyman allows you to enter text in Windows®-based applications in other languages without changing your physical keyboard (or system keyboard). • It does this by remapping the character keys according to the font for the language you wish to use. • It works with both ANSI” and Unicode. It is in wide-use and well tested. • Keyman 6.1 is a commercial program available at a reasonable price: http://tavultesoft.com/keyman/ • The development tool called ‘Keyman Developer’ is sold separately, and uses a rule-based programming language. • Note: There are some issues with very large character sets that are handled better with IMEs (Input Method Editors). However, the IMEs are more complicated and are not really user modifiable or extensible.

21. Technical overview of Keyman Operating system keystrokes Keyman Program (.KMN) Keyman Executable (.KMX) TIKE Compiler Driver Config- uration Program characters Application software

22. SIL Unicode-based Fonts • Doulos SIL 4.0: A linguist’s general use font … This is a comprehensive inventory of glyphs needed for almost any Roman- or Cyrillic-based writing system, whether used for phonetic or orthographic needs. Status: Released but not a ‘full-family’ font (regular type-face only) • Gentium is a typeface family designed to enable the diverse ethnic groups around the world who use the Latin script to produce readable, high-quality publications. It supports a wide range of Latin-based alphabets and includes glyphs that correspond to all the Latin ranges of Unicode. Status: Released and in wide use; Linux installation also released. • Additional Unicode-based fonts, including one specifically tuned for literacy use, are under development. • http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&cat_id=FontDownloads(Just remember http://scripts.sil.org )

23. Language and Culture tools SIL International has also developed software tools that serve the following needs: • Lexical database programs that facilitate: • Grammatical/morphological analysis • Dictionary creation and printing • Web publication of lexicons • Entry and analysis of cultural data • Language Survey tools for doing statistical comparison of related languages/dialects • Speech Tools for doing phonetic analysis of speech (including tonal analysis) • An extensive CD-Rom based reference tool that covers all aspects of language and cultural field work

24. FieldWorks Data Notebook for analyzing cultural data http://www.ethnologue.com/tools_docs/fieldworks.asp

25. ToolBox for language data management http://www.sil.org/computing/toolbox/

26. Lexique Pro for publishing interactive lexicons http://www.lexiquepro.com/

27. WordSurv and PalmSurvto compare language survey data http://wordsurv.css.tayloru.edu http://palmsurv.cyboreal.com

28. Speech Toolsfor analyzing speech data See http://www.sil.org/computing/catalog/index.asp

29. LinguaLinks Libraryfor electronic reference materials to support language and culture fieldwork • Anthropology • Consulting • General Reference Works • Language Learning • Linguistics • Literacy • Sociolinguistics • Translation • http://www.ethnologue.com/lingualinks.asp

30. Partnering in Script Development • Things SIL International could potentially offer: • Support script development for languages currently unsupported • Partner in the development of keyboarding solutions • Continued development of the Graphite rendering technology • Support incorporating these technologies into current open source or commercial tools, e.g., Open Offiice • Things SIL International would benefit from: • Resources necessary to actually develop the scripts (people, money, hardware/software) • Community cooperation in the standardization of scripts • Help getting open source software groups to incorporate these technologies into their core software efforts

31. Web-sites of Interest • SIL International: http://sil.org/ • Multilingual Computing: http://www.sil.org/computing/multilingual.htmlContent: Here are Links to some SIL contributions to research and development in the area of multilingual computing, links to additional resources, and a glossary of Character Encoding and Rendering. • Fonts in Cyberspace: http://www.sil.org/computing/fonts/Content : This is a guide to finding language fonts on the Internet. Containing more than 400 sources for 123 languages. • NRSI: Computers & Writing Systems: http://scripts.sil.orgContent: Character Encoding, Understanding Unicode, Keyboard Design And Keyboarding Utilities, Type Design, and Script Rendering Technologies • SIL Software Catalog: http://www.sil.org/computing/catalog/index.aspContent: SIL has developed more than 60 pieces of software to support the work of its fieldworkers; most available to the public for free download. • Keyman 6.1: http://tavultesoft.com/keyman/Content: Keyman is a commercial Windows keyboard mapping software that, with Keyman Developer, is used to design your own keyboard layouts. • SIL FieldWorks 2.0: http://www.ethnologue.com/tools_docs/fieldworks.aspSIL FieldWorks is a suite of software tools that work together to help language teams worldwide. The current released suite of programs includes WorldPad and the SIL FieldWorks Data Notebook. Additional applications are under development. • Field Linguist’s Toolbox: http://www.sil.org/computing/toolbox/Toolbox is a data management and analysis tool for field linguists. It is especially useful for maintaining lexical data, and for parsing and interlinearizing text, but it can be used to manage virtually any kind of data. • Lexique Pro: http://lexiquepro.comLexique Pro is an interactive lexicon viewer, with hyperlinks between entries, category views, dictionary reversal, search, and export tools. It's designed to display your data in a user-friendly format so you can distribute it to others.