1 / 15

North Carolina Geospatial Data Archiving Project

Workflow, Tools, and Resources Jim Tuttle, Geospatial Data Librarian. North Carolina Geospatial Data Archiving Project. Project Overview. Partnership between university library (NCSU) and state agency (NCCGIA)

darrellscott
Download Presentation

North Carolina Geospatial Data Archiving Project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Workflow, Tools, and Resources Jim Tuttle, Geospatial Data Librarian North Carolina Geospatial Data Archiving Project Digital Preservation in State Government – Wilmington, NC

  2. Digital Preservation in State Government - Wilmington Project Overview Partnership between university library (NCSU) and state agency (NCCGIA) One of eight projects in the first NDIIPP funding round: "Building a Network of Partners" Focus on state and local geospatial content in North Carolina Objective: engage existing state/federal geospatial data infrastructures in preservation

  3. Digital Preservation in State Government - Wilmington Content Complexity Multi file objects Spatial databases Ancillary data files Time-versioning Diverse data sources/metadata practices

  4. Digital Preservation in State Government - Wilmington Workflow Overview Acquisition Format Migration Submission Information Package (SIP) Creation Ingest Metadata

  5. Digital Preservation in State Government - Wilmington Acquisition: Workflow Collection creation/declaration File Manifest Metadata Seed File Transfer data to processing machine

  6. Digital Preservation in State Government - Wilmington Acquisition: Tools and Resources PHP/PostgreSQL form Python automation scripting Threat analysis ClamAV Unix ‘file’ utility jjtuttle@dli:~/$ file putty putty: MS-DOS executable (EXE), OS/2 or MS Windows

  7. Digital Preservation in State Government - Wilmington Acquisition: Tools and Resources Md5 checksum jjtuttle@dli:~/$ md5sum O-view.vsd 69b3e2f6cff1537bd607f5522d0c5c4d O-view.vsd Jhove Format registries PRONOM (UK National Archives), GDFR (Harvard/Mellon), Fred (LC)

  8. Digital Preservation in State Government - Wilmington Format Migration: Workflow On-receipt migration of selected formats Object-level metadata creation/augmentation

  9. Digital Preservation in State Government - Wilmington Format Migration: Tools and Resources Python batch process wrappers ArcCatalog metadata templates

  10. Digital Preservation in State Government - Wilmington SIP Item Creation: Workflow • Submission Information Package grouping • Ontology logic based on defined multi-file complex format components and directory structure • Repository-agnostic item grouping

  11. Digital Preservation in State Government - Wilmington SIP Item Creation: Tools and Resources Python scripts highly dependent on: Explicit understanding of ontological relationships of complex format components Logical directory structure as dictated by data-producer software Spreadsheet illustrating item assignment for manual review Automated revision of assignment based on spreadsheet modifications

  12. Digital Preservation in State Government - Wilmington Ingest Metadata: Workflow Extraction of elements from multiple sources Crosswalk metadata to archive ingest record (DSpace Qualified Dublin Core), METS, and external Workflow Management Database

  13. Digital Preservation in State Government - Wilmington Ingest Metadata: Tools and Resources Python XML libraries XSL/XSLT NOID (Nice Opaque Identifier) Persistent Identifier

  14. Digital Preservation in State Government - Wilmington Conclusion Plenty of free, open source tools The robustness of an ingest process must be inversely proportionate to the demands placed on data producers in preparation for ingest Finding the balance between cost-saving automation and the accuracy and flexibility of human intervention is difficult

  15. Digital Preservation in State Government - Wilmington For More Information NCGDAP – North Carolina Data Archiving Project http://www.lib.ncsu.edu/ncgdap/ NDIIPP – National Digital Information Infrastructure Preservation Program http://www.digitalpreservation.gov/ ClamAV http://www.clamav.net/ Unix File utility: ‘man file’ JHOVE – JSTOR Harvard Object Validation Environment http://hul.harvard.edu/jhove/ PRONOM Format Registry http://www.nationalarchives.gov.uk/pronom/ GDFR – Global Digital Format Registry (in planning) http://hul.harvard.edu/gdfr/ Fred Format Registry (proof-of-concept) http://tom.library.upenn.edu/cgi-bin/fred?cmd=Default&sid=ca21d10e67b269a75a98fe369d2ab670 XSLT – eXtensible Stylesheet Language Transformations http://www.w3.org/TR/xslt NOID – Nice Opaque IDentifier http://www.cdlib.org/inside/diglib/ark/ Jim Tuttle, Geospatial Data Librarian jim_tuttle at ncsu dot edu

More Related