1 / 12

Tools Development and Demonstration: North Carolina Geospatial Data Archiving Project

Jim Tuttle North Carolina State University Libraries. Tools Development and Demonstration: North Carolina Geospatial Data Archiving Project. Process Overview . Data transfer Threat and format analysis, validation Archive package organization Selective format migration

espen
Download Presentation

Tools Development and Demonstration: North Carolina Geospatial Data Archiving Project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Jim Tuttle North Carolina State University Libraries Tools Development and Demonstration:North Carolina Geospatial Data Archiving Project

  2. Process Overview Data transfer Threat and format analysis, validation Archive package organization Selective format migration Metadata normalization and supplementation Source metadata translation Statistics collection Extra-repository AIP management

  3. Data Transfer Python Md5sum comparison 'Transfer set' metadata capture in 'Seed file'

  4. Threat and format analysis, validation Python wrappers for the following: Virus – ClamAV Compressed files (tar, zip, gzip, bzip)‏ Geodatabases (extension and size)‏ Executable files (magic numbers)‏ Jhove validation

  5. Archive package organization ESRI ArcGIS toolbar for selected formats

  6. Archive package organization • Rule-based python logic • filestem • extension relationships ( multi-file format validation)‏ • directory structure • Manual intervention • metadata.doc • NOID assignment

  7. Selective Format Migration Coversions using ArcGIS toolbar e00 interchange to coverage to shapefile geodatabase to raster, shapefile, etc Original files retained

  8. Metadata Normalization & Supplementation Agency-specific XML templates in ArcCatalog with synchronization flags Provenance and curation metadata scripted

  9. Source Metadata Translation • Hub-and-spoke model a la Echo Depository • repository agnostic • modular conversion hub • facilitate repository software migration & inter-archive exchange

  10. Statistics Collection Python scripted statistics generation: number of files by format cumulative size by format mean file size collection size agency contribution

  11. Extra-repository AIP management Workflow Management Database populated as a spoke on the metadata/ingest hub External tracking of NOID, Handle, ISO keywords, other metadata for interaction with other systems

  12. Questions? Jim Tuttle Geospatial Data Librarian &Project Coordinator NCGDAP NCSU Libraries jim_tuttle at ncsu dot edu http://www.lib.ncsu.edu/ncgdap/

More Related