Annual Project Review

1. Annual Project Review 12 October 2000 IST-1999-10500

2. Introduction Nick Lodge ITC

3. Presentation Structure • Introduction • Project Management Overview • Linguistic Advances for Deaf People • Progress on Internet Services • Progress on Broadcast Services • Commercial Strategies • Video of Counter Services

5. Project Management Overview John Glauert UEA, Norwich, UK

6. Project Structure • User Focus • Application Focus • Technology Focus

7. Evaluation Exploitation WWW High Street Broadcast Language Animation

8. WWW High Street Broadcast Application Focus • Internet • Information and Education for Deaf People (IvD, IDGS) • High Street • Counter Services – Post Office (UKPO) • Television • Regulation and Standards (ITC, INT, IRT)

9. Evaluation Exploitation User Focus • Evaluation • Close Involvement of Deaf People (RNID) • Exploitation • Broadcasters and Service Providers (ITC, UKPO, Televirtual)

10. Language Animation Technology Focus • Sign Language Linguistics • Use of natural sign language (IDGS, UEA, IvD, RNID) • Animation • Increased realism in sign generation (Televirtual, INT, UEA)

11. Workplan Structure Project Phases • Technology Transfer • Prototype Applications • Advanced Applications

12. Technology Transfer • First 12 Months • Exchange Existing Prototypes • Establish Collaborative Working • Largely Complete

13. Prototype Applications • Months 9 to 24 • Enhancement of Prototypes • Extended Flexibility • Focus on Well Defined Domains • Excellent Progress

14. Advanced Applications • Months 18 to 36 • Increased Animation Realism • Flexible Sign Generation • Advanced Linguistic Processing • Address Wide Ranging Domains

15. Workpackage Progress • Achievements in Review Period • All Deliverables achieved • Milestones achieved or rescheduled with no impact • Progress on Deliverables • Reported here for Prototype Applications • Planned Changes • Response to Exploitation Opportunities will be presented

16. Management System • Workpackage Leadership • Consortium Meetings • Workpackage Meetings

17. Workpackage Leadership • 9 Partners • 8 Workpackages • Workpackage Leadership • Coordination of Partners • Workplan in Single Partner Tasks • Lightweight use of Microsoft Project

18. Consortium Meetings • Team of Partner Leaders • Quarterly Meetings • First day for Management and Planning • Second day allows for Technical Discussions

19. Workpackage Meetings • Meetings as required by Workpackage Leader • Extensive use of Email • Website for Document Archive and Exchange

21. Addressing the needs of Deaf people using natural sign language - Overview of linguistic processing in ViSiCAST Thomas Hanke, IDGS, U Hamburg

22. Why Sign Language? • Natural language of most Deaf, i.e. not an artificial language • 1‰ in Europe pre-lingually deaf, only 10% of them Deaf of Deaf • Most of them leave school functionally illiterate

23. What is Sign Language? • A language with its own syntax & lexicon • Not a 1:1 translation from spoken language as is SSE • Not universal, but much closer to each other than spoken languages

24. What is so special about Sign Language? • Parallel use of articulators • Use of space to encode grammatical function • Iconicity and productivity

25. Sign Language and Text • No established written form for any sign language • “phonetic” transcription systems such as HamNoSys • Glossing: GIVE-1-3a

26. HamNoSys • Flexible enough to handle at least the European sign languages, will be verified for DGS, NGT, BSL • Further developed in the project to include nonmanual activities • Embedded into XML: SiGML

27. SiGML • Three Levels: • Glosses: mapping more or less 1:1 to the animation machines’ database of signs • HamNoSys: phonetic description • Motion capture files: for uninterpreted direct transmission thru SiGML

28. SiGML (2) • <signed_utterance> • <sign> • <manual_activity gloss=“GIVE-BOOK-1-3a”> • <face attitude=IRONIC> • </sign> • <sign>… • </signed_utterance>

29. How does it fit in? Web pages English text to Sign Notation Transport & Animation English Spoken lang to text

30. Standard AI Machine Translation Approach English text Sign Notation (XML) Lexicon Lexicon Grammar Grammar Semantic Structure

31. To reduce complexity • Restricted Domains • Handling objects in a kitchen • Appartment descriptions • Semi-automatic process • User can help both with signs and word order

33. Progress on Internet Services Margriet Verlinden IvD, The Netherlands

34. Workpackage 2 • objective • produce tools that will allow a deaf citizen to access multimedia resources through sign language • deliverables • Browser plug-in (month 12) • Web-page-sign (month 30) • Signing tutor (month 30)

35. Signing on the WWW • What will it look like? • How does it work? • How is it created? • How far are we now?

36. link to project info. daily updated signing avatar controls for signing What will it look like? MOCK-UP PLAYPAUSESTOP

37. How does it work? text from meteo-rological institute Internet-browser + plugin for motion player (=avatar) SERVER CLIENT standard sentences playlist of motion data Internet-page with SiGML

38. How is it created? text from meteorological institute • update-facility • model for weather forecasts • mapping from stand. senten-ces to sign sequences (3x) • recording + post-editing of signs (3x) • XML for signs: SiGML • user-interface (webpage+plug-in) standard sentences playlist of motion data Internet-page with SiGML

39. How far are we? • basic functionality implemented • motion data is being post-edited • additional functionality in progress DEMO

41. Progress on Broadcast Services Mark Wells Televirtual, Norwich, UK

42. The Problem: • Legislation (UK, soon European) requires on-screen signing • “Open Signing” (ever present) is obtrusive • Even “Closed Signing” uses bandwidth: Broadcasters run a tight bandwidth economy

43. The Solution: Virtual Human Signing • Can be Open or Closed • Picture Quality not compromised by compression • Potential for User Interaction • Very Cheap in terms of bandwidth

44. Virtual Humans / Avatars • Entertainment • Communications • Transactions • Guides

45. Virtual Humans / Avatars • Entertainment

46. Virtual Humans / Avatars • Communications

47. Signing Virtual Humans

49. Recording Signing

50. Recording Signing Real time Face, Hand, Body, Tracking