Mipmapped High Resolution Video for Immersive Virtual 3D Environments

1. Mipmapped High Resolution Video for Immersive Virtual 3D Environments Andrew Prudhomme Advisor: Jurgen Schulze Simplified Plug-in Structure Introduction Video Frames VideoLOD Node The goal of this project was to create an efficient way of displaying 2D high resolution video in immersive 3D environments. An aspect of this goal was the implementation of mipmapping on the video stream. Since the stream can be moved towards or away from the viewer and oriented for different degrees of visibility, the area in pixels of the video in the plane of the screen was used to determine what level of detail was required. This project took the form of a C++ COVISE plug-in. The data sets were sequences of prerecorded and preprocessed frames of 2k and 4k resolutions. Another important aspect of the project was to create a solution that scaled well to an environment where one logical display is being run by multiple computers, such as a tiled display wall. Each frame of the video is pre-encoded for use with the plug-in. The image is run through several tools that generate the frame’s mipmap levels. In addition, the images are broken up into multiple tiles. Each of these tiles are saved into a single archives as a binary representation of a scene graph node structure. SLOD Node SLOD Node Image Tile Image Tile Group Group Image Tile Image Tile Image Tile Image Tile Image composed of eight tiles Results Mipmapping Image Tile Visibility The testing for this plug-in was performed on both a single and multi node system. The testing system was a cluster of 16 computers consisting of a single master node and the rest driving a tiled display wall. In this wall, the nodes were arranged in three columns with five nodes each. Each node drove four displays arranged horizontally. The wall was used for multi node testing and the masternode alone was used for single node testing. Determining which tiles are visible to the display allows the program to selectively load only the image data that is needed. The visibility calculation use the same pixel coordinates used to determine the mipmap level. There are three possible conditions for a visible tile. Mipmapping involves using a lower quality version of an image when possible. The data sets used for test were selected to be long enough to eliminate the computers' ability to cache the entire stream. The base data set was a set of 3000 4k video frames. From these frames, four video sets were created and processed as seen on the chart. The level of detail needed for each image tile is determined by the pixel area of the tile in the plane of the screen. The corner coordinates of each tile are translated into pixel coordinates on screen through a series of matrix multiplications. The area is found by bounding the tile with a rectangle and subtracting the extra pieces. A tile corner is in the display area. Performance on Tiled Display Wall A tile edge passes through the display area. The ratio of the screen pixel area to the actual pixel area determines the level of detail required for the tile. A tile entirely encompasses the display area.