1 / 45

The New DRS: Plan for Metadata Migration

The New DRS: Plan for Metadata Migration. Harvard Library & Library Technology Services February 26, 2014. Agenda. Welcome and introduction …... Franziska Frey Migration challenges .…………... Randy Stern Creating the plan ………………..... Kate Bowers

lorant
Download Presentation

The New DRS: Plan for Metadata Migration

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The New DRS: Plan for Metadata Migration Harvard Library & Library Technology Services February 26, 2014

  2. Agenda Welcome and introduction …... Franziska Frey Migration challenges .…………... Randy Stern Creating the plan ………………..... Kate Bowers Walkthrough of plan …………….. Andrea Goethals Imaging Services: minimizing disruption……….… Bill Comstock Wrap-up & next steps ………...… Kate Bowers, Andrea Goethals Q & A ……………………………….…... All

  3. Franziska Frey, Tracey Robinson Welcome & Introduction

  4. The DRS Advisory Group… …provides oversight and guidance during the rollout phase of the DRS2 project and ensures that the user community of active DRS depositors and content owners contributes to decisions about the rollout. Amy Benson Kate Bowers Bill Comstock Franziska Frey (chair) Andrea Goethals Wendy Gogel Tracey Robinson Randy Stern

  5. Why a New DRS? • Upgrade to best-in-breed technologies • Adopt digital preservation best practices and standards • Preserve metadata better • Improve collection management • Support preservation planning & activities • Improve access to content & metadata • Support more formats & genres

  6. Preservation Capability Before and After the DRS2 Project = already compliant = will be compliant after the DRS2 project Based on the NDSA Levels of Digital Preservation

  7. Evolution of the DRS New DRS infrastructure development Current DRS in production New DRS in production DRS enhancements 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 New DRS metadata migration & user adoption

  8. New DRS - Completed hardware in production migrated content to new hardware Infrastructure Development beta 3 software in production early release beta 1 beta 2 2015 2011 2012 2013 2014 2009 2010 users trained, phase 1 Metadata Migration & User Adoption Fedora assessment DuraCloud pilot test convened DRS Advisory Group first object deposited to the new DRS

  9. New DRS - Upcoming metadata migration tools created Infrastructure Development metadata migrated users moved 2015 2011 2012 2013 2014 2009 2010 Metadata Migration & User Adoption

  10. Randy Stern Migration Challenges

  11. Why “Metadata” Migration? Why not “content” migration?

  12. Pre-migration Current DRS Database DRS Content

  13. Post-migration Current DRS Database New DRS Database New DRS Index New DRS Object Descriptors DRS Content

  14. New DRS Data Model • Not a simple metadata conversion • A new DRS object is a logical intellectual entity that unifies multiple DRS files • Still image objects - archival and production masters, and deliverables including thumbnails • Audio objects - archival and production masters and deliverables • PDS objects - page image and text files

  15. Object Descriptors • METS files generated for each object • Standards-based internal schemas (PREMIS, MODS, MIX, etc.) • Metadata gathered from multiple sources • Current DRS database • Every content file • HOLLIS records

  16. Technical Challenges • Many formats • Images, audio, text, digitized books, web sites, documents, biomedical image stacks, opaque files • Unique migration rules per format • technical metadata, roles, relationships • Large (>5000 file) PDS documents • 45+ million DRS files

  17. Technical Challenges • At 1 sec/file, 45 million files would take 520 days! • We are designing the migration software tools for parallel processing • We are configuring multiple servers to run the migration

  18. Kate Bowers Creating the Plan

  19. Formulating a Migration Strategy • Analysis of: • DRS content • Technical (relationships, etc. for building objects) • Volume and type by repository • metadata for mapping • user activity in the DRS • survey of highest volume, active users • training and testing registration lists

  20. Migration Strategy Factors • Combines needs of users with technical requirements • User sequencing will be based on: • Current deposit & administrative activity • Level of preparation (training and participation in beta testing)

  21. Andrea Goethals Walkthrough of Plan

  22. Migrating Content in 5 Stages Migrate 1st : Tier 1 content Migrate 2nd: Tier 2 content Migrate 3rd: Tier 3 content Migrate 4th: Tier 4 content Migrate 5th: Tier 5 content

  23. Migrating Content in 5 Stages simpler objects Migrate 1st : Tier 1 content Migrate 2nd: Tier 2 content Migrate 3rd: Tier 3 content Migrate 4th: Tier 4 content Migrate 5th: Tier 5 content more complex objects

  24. Migrating Content in 5 Stages dependencies between tiers Migrate 1st : Tier 1 content Migrate 2nd: Tier 2 content Migrate 3rd: Tier 3 content Migrate 4th: Tier 4 content Migrate 5th: Tier 5 content dependencies within tiers

  25. Migrating Content in 5 Stages

  26. Migrating Content in 5 Stages

  27. Migrating Content in 5 Stages Tiers 1, 3, 4, 5: Migrate across all DRS owner codes at one time Tier 2: Migrate one DRS owner code at a time

  28. Tier 2: Sequence by DRS Owner Code • Migrate just your unit’s PDS document and still image content • Minimize the amount of time the content you manage the most is in 2 different systems

  29. DRS Owner Codes to be Migrated DOAK.RESLIB FCOR.CARP FMUS.ORC HPPM.PIRC FMUS.FARL GSE.GUTMN FMUS.SEM DIV.LIBR FHCL.SLV FHCL.MUSI FHCL.JUD VIT.BERE FMUS.PEAB FMUS.HUH FCOR.URI FCOR.HCO HBS.BAKR HUL.PRES FHCL.HOUGH DOAK.MUS FHCL.GOV FCOR.FORST FMUS.ARN FHCL.CAB HUL.ARCH FMUS.WARE FHCL.FUNG FCOR.REISCH HUAM.SARDIS KSG.LIBR FMUS.MCZ GSD.LIBR HUL.GGL HLNC.LIBR FHCL.DAVIS RAD.SCHL HPRE.WARD HLS.LIBR ARB.AAHOD RAD.ARCH HMS.COUNT FHCL.TOZ FHCL.YENCH FHCL.COLL FHCL.ENV FHCL.MAPS FMUS.GRAY FCOR.WOLBACH DOAK.LIBR HUL.OIS FHCL.MED HUAM.MUSE FHCL.LITT FHCL.FAL

  30. Timing • Current estimates: • Building & testing migration tools: Now • Begin Tier 1 content: Spring 2014 • Begin Tier 2 content: Summer 2014 • Units will be contacted about their Tier 2 migration schedule

  31. After Your Tier 2 Migration • You, and anyone depositing on your behalf, will begin depositing only to the new DRS • All of your management tasks will be done only in the new DRS

  32. Bill Comstock Imaging Services: Minimizing Disruption

  33. Minimizing Disruption • Testing by Imaging Services • Uninterrupted services • Migration sequencing • Participating as a “pioneer”

  34. Testing by Imaging Services Alpha and beta testing: • Depositing processes • DRS content maintenance tools • Searching and assembling content for download • Editing PDS objects

  35. Uninterrupted Services Providing services before and after your migration • Content needs to be deposited • Content needs to be searched • Content needs to be assembled • may need to be edited • may need to be downloaded

  36. Migration Sequencing • We will synchronize deposits with your migration • start depositing for you in the new DRS after your Tier 2 content is migrated

  37. Imaging Services as Pioneers As pioneers, we: • Learn to use the new tools • Refine the new depositing workflows • Identify bugs • Suggest improvements • Create a group of local experts that can support those that follow We’ll wear the scars so that you can stay pretty!

  38. Kate Bowers and Andrea Goethals Wrap-Up And Next Steps

  39. Nine Pioneers • Limited number of first depositors to new DRS • Factors • New DRS-ready content from new systems • EAS (Electronic Archiving Service), ACORN (Weissman Preservation Center conservation treatments), DASH (for ETD) • Prepared and trained staff • No content to migrate • HUA example: opaque objects

  40. First Deposit in the New DRS 大藏經DaZang Jing - Buddhist sutra, Qing dynasty (1644 -1911), China, Tibetan language

  41. Email List drs-migration@hulmail.harvard.edu

  42. http://hul.harvard.edu/ois/systems/drs/drs2.html

  43. Coming Attractions • Open meetings • Technical aspects brown bag (March) • Digital preservation & DRS intro (Summer) • Training and instruction • Refresher training • New training • Onsite assistance

  44. Thanks! Q & A

More Related