Multimedia Systems

1. Multimedia Systems Course Instructor Samana Zehra

2. What will be covered today? • Course Introduction • Course Objective • Contents • Resources • Grading Policy • Lecture no.1

3. Course Objective • To learn what are … • Different forms of multimedia • Principles of multimedia processing and presentation in computers • Multimedia applications • Multimedia networking

4. Course Contents • Multimedia Basics • Multimedia Formats • Image Fundamentals • Color Science • Sound/Audio • Video • Compression Fundamentals • Multimedia Applications • Multimedia Networking

5. Resources • Ze-Nian Li & Mark S. Drew, "Fundamentals of Multimedia", Pearson Education, 2004 • Steinmetz R & Nahrstedt K, “Multimedia: Computing, Communication & Applications”, Pearson Education, 2001 • F. Halsall, "Multimedia Communications: Applications, Networks, Protocols, and Standards", first edition, Addison-Wesley, 2000 • James F. Kurose & Keith W. Ross, “Computer Networking – A top-down approach featuring the internet”, Third edition, Pearson Education, 2005

6. Grading Policy • Theory Exam (100 marks) • Sessionals (50 marks) • Internal (25 marks) • Quizzes (4 to 5) • Assignments (2 to 3) • can include presentations • External (25 marks) • Viva • May include some tasks/presentations

7. Lecture -1 The introduction begins…

8. Introduction • Multimedia definitions:- • Multi means many; much; multiple • Medium means: • a substance regarded as the means of transmission of a force or effect; a channel or system of communication, information or entertainment (Merriam-Webster Dictionary) • Medium means: • an intervening substance through which something is transmitted or carried on (American Heritage Electronic Dictionary) • So Multimedia is medium having multiple content forms.

9. Introduction (Contd.) • Multimedia includes a combination of text, audio, still images, animation, video, and interactivity content forms. • Multimedia is usually recorded & played, displayed or accessed by computerized and electronic devices, but can also be part of a live performance. • Multimedia also describes electronic media devices used to store and experience multimedia content. ( source: Wikipedia )

10. A PowerPoint presentation used in a corporate office VVO Multimedia-terminal in Germany Virtual reality uses multimedia content

11. Introduction (Contd.) • Based on time-dimension in the representation space, media can be • Time-independent (Discrete) • Text, Graphics • Time dependent (Continuous) • Audio, Video • Video, sequence of frames (images) presented to the user periodically. • Time dependent aperiodic media is not continuous!! • Note: Discrete & Continuous have no connection with internal representation.

12. History of Multimedia • Newspaper - perhaps the first mass communication medium to employ Multimedia - they used mostly text, graphics, and images. • 1895 - Marconi sent his first wireless radio transmission at Pontecchio, Italy. • 1901 - Maroni detected radio waves beamed across the Atlantic. Initially invented for telegraph, radio is now a major medium for audio broadcasting. • Television - new media for the 20th century. It brings the video and has changed the world of mass communications. • Some of the important events in relation to Multimedia in Computing include: • 1945 - Bush wrote about Memex • 1967 - Negroponte formed the Architecture Machine Group at MIT • 1969 - Nelson & Van Dam hypertext editor at Brown • Birth of The Internet

13. History of Multimedia (Contd.) • 1971 - Email • 1976 - Architecture Machine Group proposal to DARPA: Multiple Media • 1980 - Lippman & Mohl: Aspen Movie Map • 1983 - Backer: Electronic Book • 1985 - Negroponte, Wiesner: opened MIT Media Lab • 1989 - Tim Berners-Lee proposed the World Wide Web to CERN (European Council for Nuclear Research) • 1990 - K. Hooper Woolsey, Apple Multimedia Lab, 100 people, educ. • 1991 - Apple Multimedia Lab: Visual Almanac, Classroom MM Kiosk • 1992 - the first M-bone audio multicast on the Net • 1993 - U. Illinois National Center for Supercomputing Applications: NCSA Mosaic • 1994 - Jim Clark and Marc Andreesen: Netscape • 1995 - JAVA for platform-independent application development. Duke is the first applet. • 1996 - Microsoft, Internet Explorer.

14. Multimedia Authoring Tools • Multimedia elements are composed into a project using authoring tools. • Multimedia Authoring tools are those programs that provide the capability for creating a complete multimedia presentations by linking together objects such as a paragraph of text (song), an illustration, an audio, with appropriate interactive user control

15. Multimedia Authoring Tools (Contd.) • By defining the objects' relationships to each other, and by sequencing them in an appropriate order, authors (who use authoring tools) can produce attractive and useful graphical applications. • To name a few authoring tools • Macromedia Flash • Macromedia Director • Authorware • The hardware and the software that govern the limits of what can happen are multimedia platform or environment

16. Multimedia and Interactivity • Multimedia is interactive when the end-user is allowed to control what and when the elements are delivered. • Interactive Multimedia is Hypermedia, when the end-user is provided with the structure of linked elements through which he/she can navigate.

17. Multimedia and Interactivity • Multimedia is linear, when it is not interactive and the users just sit and watch as if it is a movie, e.g., cinema presentation • Multimedia is nonlinear, when the users are given the navigational control and can browse the contents at will, e.g., computer game, computer-based training.

18. Multimedia System • A MultimediaSystem is a system capable of processing multimedia data and applications. • A Multimedia System is characterised by the processing, storage, generation, manipulation and rendition of Multimedia information.

19. Characteristics of Multimedia System • A Multimedia system has four basic characteristics: • Multimedia systems must be computer controlled. • Multimedia systems are integrated. • The information they handle must be represented digitally. • The interface to the final presentation of media is usually interactive.

20. Challenges for Multimedia System • Supporting multimedia applications over a computer network renders the application distributed. • Multimedia systems may have to render a variety of media at the same instant; a distinction from normal applications. • There is a temporal relationship between many forms of media (e.g. Video and Audio). In this case problems are: • Sequencing within the media - playing frames in correct order/time frame in video • Synchronisation - inter-media scheduling (e.g. Video and Audio). Lip synchronisation is clearly important for humans to watch playback of video and audio and even animation and audio.

21. Challenges for Multimedia Systems • The key issues that multimedia systems need to deal with are: • How to represent and store temporal information. • How to strictly maintain the temporal relationships on play back/retrieval • What process are involved in the above. • Data has to be represented digitally so many initial source of data needs to be digitised - translated from analog source to digital representation. This will involve scanning (graphics, still images), sampling (audio/video) although digital cameras now exist for direct scene to digital capture of images and video. • The data is large (several Mb) for audio and video - therefore storage, transfer (bandwidth) and processing overheads are high. Data compression techniques very common.

22. Desirable Features • Given the (mentioned) challenges the following features are desirable (if not a prerequisite) for a Multimedia System: • Very High Processing Power • needed to deal with large data processing and real time delivery of media. Special hardware commonplace. • Multimedia Capable File System • needed to deliver real-time media e.g. Video/Audio Streaming. Special Hardware/Software needed e.g RAID technology. • Data Representations/File Formats that support multimedia • Data representations/file formats should be easy to handle yet allow for compression/decompression in real-time. • Efficient and High I/O • input and output to the file subsystem needs to be efficient and fast. Needs to allow for real-time recording as well as playback of data. e.g. Direct to Disk recording systems.

23. Desirable Features • Special Operating System • to allow access to file system and process data efficiently and quickly. Needs to support direct transfers to disk, real-time scheduling, fast interrupt processing, I/O streaming etc. • Storage and Memory • large storage units (of the order of 50 -100 Gb or more) and large memory (50 -100 Mb or more). Large Caches also required and frequently of Level 2 and 3 hierarchy for efficient management. • Network Support • Client-server systems common as distributed systems common. • Software Tools • user friendly tools needed to handle media, design and develop applications, deliver media.

24. Multimedia Applications • Multimedia finds its application in various areas, a few are: • Education • Computer-based trainings, reference e-books, documents imaging, distant learning. • Entertainment • Interactive games • Medicine • Training to doctors through virtual surgery • Business • Employee training, advertisements, customer support, selling products. • Arts • Art exhibitions, literature presentations • Scientific Research • Modeling and simulations

25. Multimedia Applications (Contd.) In MedicineSource: Cardiac Imaging,YALE centre for advanced cardiac imaging

26. Multimedia Applications (Contd.) In training

27. Multimedia Applications (Contd.) Public awareness campaignSourceInteractive Multimedia Project Department of food science& nutrition, Colorado State Univ

28. Multimedia Applications (Contd.) • Other Applications include: • World Wide Web • Video conferencing • Video-on-demand • Interactive TV • Home shopping • Games • Virtual reality • Digital video editing and production systems • Multimedia Database systems