The HDF5 Virtual File Layer (VFL) and Virtual File Drivers (VFDs) - PowerPoint PPT Presentation

the hdf5 virtual file layer vfl and virtual file drivers vfds n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
The HDF5 Virtual File Layer (VFL) and Virtual File Drivers (VFDs) PowerPoint Presentation
Download Presentation
The HDF5 Virtual File Layer (VFL) and Virtual File Drivers (VFDs)

play fullscreen
1 / 11
The HDF5 Virtual File Layer (VFL) and Virtual File Drivers (VFDs)
481 Views
Download Presentation
ledell
Download Presentation

The HDF5 Virtual File Layer (VFL) and Virtual File Drivers (VFDs)

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. The HDF5Virtual File Layer (VFL)andVirtual File Drivers (VFDs) Dana Robinson The HDF Group Efficient Use of HDF5 With High Data Rate X-Ray Detectors Paul ScherrerInstitut

  2. HDF5 Library HDF5 API Virtual File Layer VFD VFD User-Supplied VFD disk

  3. HDF5 API internal generic I/O call H5FDwrite() Virtual File Layer function pointer Virtual File Driver VFD-specific I/O call H5FD_sec2_write() disk

  4. HDF5 VFDs log split multi family sec21(default) Windows2 STDIO3 core (in-memory) MPI MPI-POSIX "logical" VFDs which perform no I/O themselves 1) Uses POSIX I/O (sec2 = "POSIX section 2") 2) Currently a wrapper for SEC2. There is no driver which uses Win32 API calls. 3) "How to write a VFD" demo driver. Not intended for production use.

  5. The core VFD allows you to create/open HDF5 files in memory. HDF5 API H5FDwrite() Virtual File Layer disk Core VFD H5FD_core_write() memory File is (optionally) written to disk on close

  6. The family VFD allows you to split a logical HDF5 file among many smaller physical files. HDF5 API H5FDwrite() Virtual File Layer Family VFD H5FD_family_write() disk file A file B file C … file Z

  7. The multi (and split) VFD allows you to direct various categories of HDF5 data to different files and disks. Multi VFD H5FD_multi_write() global and local heap data, B-trees, object headers raw data superblock disk 1 disk 2 file A file B file C

  8. Selecting a "Terminal" VFD A "terminal" VFD does not require a second VFD 1) Create the property list. hid_tfapl_id = H5Pcreate(H5P_FILE_ACCESS); 2) Set the VFD using the appropriate API calls. herr_terr = H5Pset_fapl_sec2(fapl_id); 3) Create/open your file. hid_t fid = H5Fcreate("foo.h5", 0, H5P_DEFAULT, fapl_id);

  9. Selecting a "Logical" VFD A "logical" VFD requires a second, underlying VFD Step 1 Create a file access property list (FAPL) and set the VFD to use the terminal VFD. Step 2 Create a second FAPL and set the VFD to use the logical VFD (passing in the first VFD). Step 3 Open your file using the second (logical) FAPL. Logical VFD Terminal VFD Disk

  10. 1) Create the first property list. hid_tfapl_id = H5Pcreate(H5P_FILE_ACCESS); 2) Set the terminal VFD using the appropriate API calls. herr_terr = H5Pset_fapl_sec2(terminal); 3) Create the second file create/access property list. hid_tfapl_id = H5Pcreate(H5P_FILE_ACCESS); 4) Set the logical VFD using the appropriate API calls. herr_terr = H5Pset_fapl_family(logical, SIZE, terminal); 5) Create/open your file. hid_t fid = H5Fcreate("foo.h5", 0, H5P_DEFAULT, logical);

  11. Stackable VFDs Virtual File Layer Non-Terminal VFD Non-Terminal VFD Terminal VFD Disk