Elements of Scientific Visualization Using OpenDX Jon Johansson Academic Information and

1. Elements of Scientific Visualization Using OpenDX Jon Johansson Academic Information and Communication Technologies May 2, 2007

2. Working Directory • the data files for this session unpack to about 200 MB • lets work in scratch space so we don’t fill up our AFS accounts cd /scratch mkdir <yourCCID> cd <yourCCID> • work that you do in scratch is not guaranteed to stay so back it up if you want to keep it

3. Working Directory • download the files for the workshop from • http://www.ualberta.ca/CNS/RESEARCH/Courses/2007/Workshop/_08_VisUsingOpenDX/ • get the file: AICT0708.IntroToOpenDX.Data.zip • to upcompress the file use unzip AICT0708.IntroToOpenDX.Data.zip • another file: ElectricPotentialData.zip is data that you can use to create the visualization developed in the last third of the presentation

4. Viz Packages • OpenDX, VTK are AVS use a pipelined, component-based architecture • a user can quickly assemble modular software components into a “finished application.” • these systems are flexible in the sense that components can be combined in a multitude of ways, thereby allowing an application developer to accomplish a wide variety of visualization tasks • they are extensible as they offer the means for developers to add new components to the system, thereby extending the system’s functionality

5. OpenDX • information on OpenDX can be found at • http://www.opendx.org/ • this site has the latest version of dx for download • IBM has a site with older information • http://www.research.ibm.com/dx/

6. OpenDX • 1991, IBM Research introduced as Visualization Data Explorer (dx) • May 24 1999, IBM made Visualization Data Explorer (dx) open source as IBM Open Visualization Data Explorer (OpenDX) • the last commercial version of dx was 3.1.4B • OpenDX is now at 3.4.2

7. OpenDX • OpenDX provides a full set of tools for manipulating, transforming, processing, realizing, rendering and animating data • allows for visualization and analysis methods based on points, lines, areas, volumes, images or geometric primitives in any combination

8. OpenDX Architecture • dx uses a client-server execution model • the client process is the graphics user interface: • it always runs on a workstation • the server process does all of the computation: • it may reside on the same, or a different machine

9. OpenDX Architecture

10. Starting OpenDX • to start the dx process: • on a Linux/Unix machine, type • dx • on Windows click an OpenDX icon • Start Menu -> OpenDX -> DX • this will bring up the OpenDX Startup Window

11. OpenDX Interface • from the startup window we can: • import data through a wizard • run previously written visual programs • create or edit visual programs • run the OpenDX tutorial • run sample programs • process “startupui”

12. OpenDX Data Import • OpenDX import module will read the following data formats: • DX – Data Explorer native format • CDF (Common Data Format) • from the National Space Science Data Center • a self-describing data abstraction for the storage and manipulation of multidimensional data in a discipline-independent fashion. • http://nssdc.gsfc.nasa.gov/cdf/cdf_home.html • NetCDF (network Common Data Format) • from the Unidata Program Center in Boulder, Colorado • an interface for array-oriented data access and a library that provides an implementation of the interface • http://www.unidata.ucar.edu/packages/netcdf/

13. OpenDX Data Import • OpenDX import module will read the following data formats: • HDF – Hierarchical Data Format (HDF version 4.1 only for Windows) • from the National Center for Supercomputing Applications • software and file formats for scientific data management • http://hdf.ncsa.uiuc.edu/ • Image – TIFF, MIFF, GIF, RGB, R+G+B, YUV • Spreadsheet • typically non-spatial data, for example:

14. OpenDX Data Import • OpenDX import module will read the following data formats: • General • describe the data file with a “general array header file” which can describe a wide variety of formats for data • use the Data Prompter tool started by the Import Data button to describe your data and create the header file

15. OpenDX Data Import • the Data Prompter • describe your data to OpenDX • the Data Prompter with create a “.general” file which describes your data file • the import module reads the general file

16. OpenDX Data Import

17. OpenDX Data Import • the “.general” file is ascii text • it contains keyword statements that describe important aspects of the data file • pathname to file • grid structure • data type • format • more …

18. OpenDX Data Import • a data file might consist of ascii values, each line containing 4 values … -8.50000000 14.72243186 48.00000000 326.96078431 -8.50000000 14.72243186 49.00000000 326.96078431 -8.50000000 14.72243186 50.00000000 326.96078431 -9.50627936 14.09363873 -50.00000000 365.66830060 … • if the coordinate values aren’t present, it’s important know how the file was written: • row major: for( i = 0; i < nx; i++){ for( j = 0; j < ny; j++){ for( k = 0; k < nz; k++){ fprintf( outfh, "%12.8f %12.8f %12.8f %12.8f \n", V ); } } } • row major is defined by the last index varying the fastest

19. OpenDX Data Import • We have a data set that has regular spacing in the x, y and z directions • We don’t need to read the coordinates for each point from a file • The general header file to read in this data is: file = ./pot_cart.dat grid = 101 x 101 x 101 format = ascii interleaving = field majority = row header = lines 0 field = potential structure = scalar type = float dependency = positions positions = regular, regular, regular, -50.0, 1.0, -50.0, 1.0, -50.0, 1.0 end

20. OpenDX Data Import • DX supports many techniques for generating renderable geometry from data. • For scalar data these include: • color and opacity mapping • contours and isosurfaces • histograms • two-dimensional and three-dimensional plotting • surface deformation, etc • For vector data • arrow plots • streamlines, streaklines, etc

21. OpenDX Data Import • Visualizations can be annotated with: • ribbons • tubes • axes • glyphs • text • display of data locations, meshes and boundaries • Data interactions are also supported: • probing – (selecting a location anywhere in a volume) • picking – (selecting a location on the surface of an object) • arbitrary surface and volume sampling • arbitrary cutting/mapping planes

22. OpenDX Visual Program Editor • Click on “New Visual Program …” in the Startup Window • starts the VPE • create “Visual Programs” by connecting modules • connections represent data flow

23. OpenDX Visual Program Editor

24. OpenDX Visual Program Editor • to move a module from the palette on the left to the canvas on the right of the VPE: • click on the module to highlight it • move the cursor to the canvas – the cursor becomes a right-angle to show where the upper left corner of the module will be • left-click to place the module

25. OpenDX Visual Program Editor

26. OpenDX Visual Program Editor • input ports are at the top of the module • output ports are at the bottom • some modules have no input or no output ports • double click on a module to see the interface for the module

27. OpenDX Visual Program Editor • the Import module reads data from a file • three inputs (from left to right): • file name • the variable to read from the file (depends on the data format) • data format (select general) • one output • the data (as a field)

28. OpenDX Visual Program Editor • the interface to the Import module allows you to set values

29. Data Import • A big hurdle in using any visualization system can be getting the data into the package • Once your data is in the package the data can be manipulated using the tools built into the package • The form of the data is frequently dictated by the form of the geometry in the problem • E.g. Spherically symmetric problem results in spherical-polar coords in the data set

30. Data Import • Each visualization package has defined a set of data types upon which the algorithms depend to function • A well chosen data structure can permit the development of powerful algorithms • A poorly chosen data structure can hinder the performance of the package

31. Data Import • Data sets come at many scales and in many formats: • Computer simulations of cellular processes • Electro-magnetic fields around the Earth and through the Solar system • Nuclear interactions • Crystal formation • Scans of human bodies: CT, MRI, PET • need a taxonomy for data

32. Data Sets – Uniform Mesh • represents data organized in a regular array • only need to enter the min and max values of the axes: (x0, y0) and (xmax, ymax) • can describe the data file with another text file having a “general” extension

33. Data Sets – Rectilinear Mesh • represents data organized on orthogonal axes, but the spacing between data points may be irregular • need to provide lists of the x and y coords and the grid is constructed

34. Data Sets – Irregular Mesh • less symmetry than uniform or rectilinear meshes • provide the coordinates of all the points in the mesh • the connectivity can be inferred

35. Data Sets – Structured Mesh • the meshes above are all classified as “Structured” • the points can be characterized by dimensions along some axes • the grid below is 7x3 – that’s enough to infer the connectivity

36. Data Sets – Unstructured Mesh • represents data points that aren’t organized in a regular pattern • must provide all coordinates for points as well as explicit connection information • data and its description in a file with the extension “.dx” • this is referred to as “Unstructured Cell Data” • the 12 points below can be connected in many ways, for example: 5 triangles and 4 quads 1 triangle and 6 quads 13 triangles

37. Data Sets – Scattered Points • only concerned with the points in space – not with the connections • read the points in as a list – connections are implied P0 → P1 → … → Pn-1 • points in 3D have 1D connections • use either .general or .dx – the .general is easier

38. Data Sets – Important Parameters • nspace – number of spatial coordinates per node: (x, y, z) nspace = 3 • ndim – number of grid dimensions (also called the computational dimension). This is the space of the connections. For scattered points ndim = 1. • dims – an array of grid dimensions {nx, ny, nz}

39. OpenDX Data Model • Fields are the fundamental objects in the Data Explorer data model. • A field represents a mapping from some domain to some data space. • The domain of the mapping is specified by a set of positions and (generally) a set of connections that allow interpolation of data values for points between positions.

40. OpenDX Data Model • Positions represent what can be thought of as (and often really are) locations in space; the data are the values associated with the space of the positions. • The mapping at all points in a domain (not just those specified by the given positions) is represented implicitly by specifying that the data are dependent on (located at) the sample points or on the connections between points.

41. OpenDX Data Model • the positions and data are said to be components of a field, and every field must contain at least a "positions" component and a "data" component. Fields may also contain other components (e.g., "connections, colors"). • note that you need to create a color component before the field can be rendered • An OpenDX field consists of data and the additional components needed to describe that data so that OpenDX can process it.

42. OpenDX Data Import • If your data is basically a number of arrays you can use the General Array Importer • To import data through the General Array Importer, you must be able to answer the following questions:

43. OpenDX Data Import • What are the independent and dependent variables? • Scalar field and positions? • independent variable constitute the "positions" component of a data field • What is the dimensionality of the positions and data components ? • 1-D 2-D 3-D …

44. OpenDX Data Import • How is the independent variable (the set of positions) to be described? • regular grid (which can be completely described by an origin and a set of deltas) • an explicit list (which may or may not be part of the data file)? • E.g. data measurements might be on a grid of 1-degree increments in latitude and 5-degree increments in longitude

45. OpenDX Data Types • How are the positions connected to one another? • Are they connected? • For example, a regular grid of positions might be connected by a regular grid of connections (lines, quads, or cubes). The connections specify how data values should be interpolated between positions. Positions that are explicitly specified (i.e., not regular) can also be connected by a regular grid of connections (e.g., if the grid is deformed, or warped)

46. OpenDX Data Types

47. OpenDX Data Types • top row represent surfaces • bottom row represent volumes • the three types of grid are (left to right) • irregular (irregular positions, irregular connections), • deformed regular (irregular positions, regular connections) • regular (regular positions, regular connections),

48. OpenDX Data Import • What is the format of the stored data values • ASCII or binary? • floating point, integer, signed or unsigned byte, etc.? • What is the order of the data items with respect to the grid? • column major (first index varies fastest) • row major (last index varies fastest)

49. OpenDX Data Import • Are the data dependent on "positions" or on "connections"? • i.e. are the data values associated one-to-one with positions or with the connections between positions? • Data associated with connections are often referred to as "cell-centered"

50. OpenDX Data Import • What kind of embedded text (comments, etc.) in the data file must be "skipped" when the data values are read • # of lines of header? • # of bytes of header?