Multimedia Research in E-Education

1. Multimedia ResearchinE-Education • Veljko Milutinović, Fellow of the IEEE An Overview of the Ongoing Projects http://galeb.etf.bg.ac.yu/~vm e-mail: vm@etf.bg.ac.yu

2. Major R&D Bottlenecks • Integrated Educational Systems • Concept Understanding and Interactive Cooperation • Intelligent Search and Retrieval for Educational Purposes • Innovative Applications

3. U. of California at Berkeley • Integrated Systems: Intel MES Grant • Concept Understanding: WISE • Intelligent Search: GIST • Innovative Applications: BMRC

4. MIT • Integrated Systems: Microsoft I-Campus • Concept Understanding: OXYGEN • Intelligent Search: NCAM • Innovative Applications: NREN

5. Stanford University • Integrated Systems: Challenge 2000 Multimedia • Concept Understanding: Media X • Intelligent Search: CBIR • Innovative Applications: SUMMIT, CERAS, MM Collab, Shakespeare

6. Project Name (University) • Project Leader(s) • Project URL • Project Essence in ASCII • Project Essence in JPEG/MPEG

7. Intel MES Grant (Berkeley) • James Demmel • http://www.intel.com/pressroom/archive/releases/CO081897.HTMA • RISC approach to MES

8. WISE (Berkeley) • Eric Baumgartner • http://wise.berkeley.edu/pages/intro/wiseIntro01.html • Risk approach to K12

9. GIST (Berkeley) • Joe Hellerstein • http://now.cs.berkeley.edu/AM • Risk approach to generalized search tree for secondary and multimedia storage; supports any lookup over that data

10. BMRC (Berkeley) • Larry Rowe • http://www-plateau.cs.berkeley.edu/ • Risk approach to contents and technology management

11. Microsoft I-Campus (MIT) • Thomas L. Magnanti • http://web.mit.edu/newsoffice/tt/1999/oct06/microsoft.html • Risk approach to $25 million

12. OXYGEN (MIT) • Anant Agarval, John Ancorn, Krste Asanovic, Rodney Brooks, … • http://oxygen.lcs.mit.edu/ • Risk approach to bringing abundant computation, multimedia, and communication naturally into people's lives, through an infrastructure of mobile and stationary devices connected by a self-configuring network

13. NCAM (MIT) • Geoff Freed • http://www.ncddr.org/du/researchexchange/v06n03/multi.html • Risk approach to accessible multimedia and distant education

14. NREN (MIT) • MIT, ARPA, DOE, NASA, NSF • http://www.ccic.gov/pubs/blue94/section.3.2.html • Risk approach to gigabit communications infrastructure for e-research and e-education

15. Challenge 2000 MM (Stanford) • Shari Golan, Barbara Means, Bill Penuel • http://ctl.sri.com/projects/displayProject.jsp?Nick=ch2000mm • Risk approach to the next decade challenges in MES

16. Media X (Stanford) • John Perry • http://mediax.stanford.edu/about/education.html • Risk approach to multimedia courses in interdisciplinary major undergraduate and graduate programs

17. CBIR (Stanford) • Jia Li, James Z. Vang, Gio Wiederhold • http://www-db.stanford.edu/IMAGE/ • Risk approach to contents based image retrieval: semantics-sensitive integrated matching for picture libraries, wavelet-based image indexing and searching

18. SUMMIT (Stanford) • Parvati Dev • http://www-smi.stanford.edu/projects/summit.html • Risk approach to MES in medicine

19. Potentials of R&D in MESthrough Italy/Serbia Cooperation An Overview of Belgrade University Projects for High-Tech Computer Industry in the USA and EU (IPSI)

20. Some of the Recent Projects • NCR, Compaq, SUN, Intel, • Comshare, Zycad, QSI, Virtual, • TechnologyConnect, BioPop, eT, MainStreetNetworks, • Salerno, Pisa, Siena, L’Aquila, ...

21. Response: Industry

22. Flynn, M. J., Computer Architecture, Jones and Bartlett, USA (96)position 1 (12 citations) Bartee, T. C., Computer Architecture and Logic Design, McGraw-Hill, USA (91)position 1 (2 citations) Tabak, D., RISC Systems (RISC Processor Architecture), Wiley, USA (91)position 1s (6 citations) Stallings, W., Reduced Instruction Set Computers (RISC Architecture), IEEE CS Press, Los Alamitos, California, USA (90)position 1s (3 citations) Heudin, J. C., Panetto, C., RISC Architectures, Chapman-Hall, London, England (92)position 3s (2 citations) van de Goor, A. J., Computer Architecture and Design, Addison Wesley, Reading, Massachusetts, USA (2nd printing, 91)position 4s (3 citations) Tannenbaum, A., Structured Computer Organization (Advanced Computer Architecures), Prentice-Hall, USA (90)position 5s (4 citations) Feldman, J. M., Retter, C. T., Computer Architecture, McGraw-Hill, USA (94)position 7s (2 citations) Stallings, W., Computer Organization and Architecture, Prentice-Hall, USA (96)position 9s (3 citations) Murray, W., Computer and Digital System Architecture, Prentice-Hall, USA (90)position >10s (2 citations) Wilkinson, B., Computer Architecture, Prentice-Hall, USA (91)position >10 (2 citations) Decegama, A., The Technology of Parallel Processing (Parallel Processing Architectures), Prentice-Hall, USA (90)position >10s (2 citations) Baron, R. J., Higbie, L., Computer Architecture, Addison-Wesley, USA (92)position >10s (1 citation) Tabak, D., Advanced Microprocessors (Microcomputer Architecture), McGraw-Hill, USA (95)position >10s (1 citation) Zargham, M. R., Computer Architecture, Prentice-Hall, USA (96)position >10s (1 citation) Hennessy, J. L., Patterson, D. A., Computer Architecture: A Quantitative Approach, Morgan-Kaufmann, USA (96)na (0 citations) Hwang, K., Advanced Computer Architecture, McGraw-Hill, USA (93)na (0 citations) Kain, K., Computer Architecture, Addison-Wesley, USA (95)na (0 citations) Response: Academia

23. N.B. ERRORS MADE & LESSONS LEARNED

24. 1

25. 2

26. 3

27. Summary The world’s best journals – IEEE (50): A European record in ICT Books with 7 Nobel Laureates: Kenneth Wilson, Ohio (North-Holland) Leon Cooper, Brown (Prentice-Hall) Robert Richardson, Cornell (Kluwer-Academics) Jerome Friedman, MIT (Kluwer-Academics) Herb Simon, CMU (IOS) Arno Penzias, AT&T (IOS) Harold Kroto, England (IOS)

28. Multimedia Internet Gallery Funding: Fraunhofer IPSI, Darmstadt, Germany Implementation: IPSI, Belgrade, Serbia Project Termination Date: August 31, 2003. Users: Germany, Serbia, USA, Canada,… Demo: www.ipsi.co.yu

29. IPSI Belgrade, Ltd. 3D@3D MULTIMEDIA ARTSHOP GALLERY vm@ipsi.co.yu, vm@etf.bg.ac.yuhttp://galeb.etf.bg.ac.yu/vm, http://www.ipsi.co.yu

30. Authors Marinkovic Ivan Stojanovski Aleksandar Nikezic Gavro Radakovic Miroslav Skundric Nikola Milutinovic Darko Toskov Ivan Vujovic Ivana Milutinovic Veljko Anucojic Goran

31. Introduction – IPSI Belgrade IPSI Belgrade is a company jointly founded by German and Serbian capital Partners: • IPSI Fraunhofer, Darmstadt, Germany • Telecom Italia Learning Services, Italy • NYU, School of Continuous Professional Studies, USA • Purdue University, School of Technology, USA

32. Introduction – IPSI Belgrade • Multimedia Workspaces of the Future- Multimedia Applications for the Web- Environments for Cooperative Working and Learning- Virtual Information and Knowledge Environments- Mobile Interactive Media- Open Adaptive Information Management Systems- Publication Engineering and Technology- Hardware Design and Operating Systems- Networks and WWW- Semantic Web and Datamining

33. Introduction – IPSI Belgrade Products: • Advanced Multimedia Virtual Gallery • Tools for B2B Matchmaking on the Web • Web security for P2P, The injection cache, The STS cache, Genetic Search with Spatial/Temporal Mutations, Customer Browser Satisfaction Web Search, Browser Acceleration, Technology Transfer, Testing Infrastructure for EBI, Distant Web Educating Machine, e-Tourism, …

34. MMAG: Problem Statement - Creating Web based art gallery with “look and feel” of the real world exhibitions • Visitor moves through the gallery by “walking with options” • 2D on 3D and 3D on 3D, with multimedia options

35. Existing Solutions • Musee national des Arts asiatiques http://www.museeguimet.fr/tour-guimet/index.html • Web Server of the Galleria degli Uffizi in Florence http://www.uffizi.firenze.it • The Distributed Interactive Virtual Environment (DIVE) http://www.sics.se/dive/ • The Web3D Repository http://www.web3d.org/vrml/artgal.htm

36. Proposed Solution - Virtual reality gallery: Multimedia In Action - Advanced search capabilities: Dream Search - Artist’s criteria room generation: Do It By Yourself

37. Why is it better? - Dynamically generated/exploitable gallery - Content based search engine - User satisfaction

38. Conditions and Assumptions • PC- Internet connection- Internet Explorer 5.0 or higher - Netscape 7.0 - Cortona VRML plug-in for IE- Basic multimedia tools and standards

39. Analysis and Implementation - Application is written in ASP.NET using C# as code-behind, and ADO.NET for database access. - Database server is SQL Server 2000. • Communication with the database is entirely made through XML (using SQLXML3.0 framework). • Queries are made in XPath, while adding, changing and deleting of the records is done through UpdateGrams. - Application is optimized for Internet Explorer 5.0 or higher, at the 1024x768 screen resolution. Netscape 7.0 or higher is also supported. - 3D gallery is completely generated on the server side (dynamically) using VRML.

40. Track 1 Track Requirements: • To stand as the integrative part for the other two tracks • To provide: • User interaction • Database connectivity (database independent) • Search functions (simple and advanced using Track3 output) • Information brokering between artists and buyers • Administration tools • Artworks management tools, etc. • Thin client (3D scene generation on server side)

41. Track 1 Development Tools: • Application server platform: • Windows XP Professional • IIS 5.1 • MS SQL Server 2000 • Development platform: • ASP.NET • C# as code-behind. • Communication with the underlying database: • XML & XSD using XPath queries (DB independent) • Currently using SQLXML3.0 add-on for ADO.NET

42. Track 1 Database Design:

43. Track 1 Administrator Tools: • Separate entry point:http://<server_address>/artshop/admin

44. Track 1 Users & Exhibitors: • Entry point:http://<server_address>/artshop/index.htm

45. Track 1 Interesting Details: Native XML DBMS under development at IPSI Fraunhofer Practical testing of the XML/XPath database access Dynamic addition (to the system) of new multimedia types 3D view of search results

46. Track 1 Interesting Details: • Application that can connect on the fly to any DBMS which supports XML/XPath is an interesting and possibly useful idea (user just has to set one XML file containing local field mapping, and one XSD to map the database fields to the pre-defined scheme) • Cons: • XPath queries are lot less powerful then standard SQL queries • Inherently, loss of speed (one complex SQL query had to be simulated with couple of XPath queries and additional processing in the code). • For now, SQLXML3.0 does not support complete XPath standard.

47. Track 1 Errors Made: • Initially, content analysis, picture processing, and adding data to database were completely separated (as specified in the contract), with the idea of later (partial) integration. • Turned out to be a bad idea(required a lot of interventionfrom the ArtShop system administrator when adding artworks).

48. Track 1 Lessons Learned: • Problem solved by complete integration of forementioned tasks into the one system process which monitors input directory, automatically schedules picture processing and content analysis, and takes care of updating of all necessary fields in all required databases. • With that, we achieved maximum automation, reduced time needed for artwork addition, and reduced amount of data transferred through the Internet (between the administrator’s machine and the application host).

49. Track 2 Image-Content-Oriented Search Track requirements: Images used for extracting objects are artistic paintings Image analyses Extraction of the features Create XML file for each image Fetch the database with the features

50. Load parameters, input and output directory Open the picture Determine the filter value Put the picture into the reduced matrix Determine histogram Create objects Merge objects into bigger objects Create sorted array of objects Create database objects, prepare them, and put them in XML file and tables in database Track 2 Algorithm: