Policy on digital records preservation in the NSW public sector Cassie Findlay Senior Project Officer, Government Recordkeeping
Key concepts • Digital records: evidence of business in the form of emails, web pages, Word documents, scanned documents, spreadsheets etc. • Preservation: Process and operations involved in ensuring the technical and intellectual survival of authentic records over time (AS ISO 15489 Part 1 Clause 3.14) • Migration:involves a set of organised tasks designed to periodically transfer records from one hardware or software configuration to another, or from one generation of technology to another, while maintaining the records’ authenticity, integrity, reliability and useability’.
The preservation challenge of digital records • Many digital formats are proprietary • Even widely accepted formats cannot be read after a few generations of technological change. • A 2005 survey showed that 38% of public offices had some technology dependent records that were either not accessible or accessible only with some difficulty. • State Records and public offices have a shared responsibility to preserve accessible and authentic digital records of government business.
Background • Renewed strategic focus on digital recordkeeping and archiving + Proposed project to build State’s first digital archive • Need for a policy basis for both public offices’ and State Records’ approaches to the preservation of digital records
Policy development timeline • Feb-March 2007: Research into Australasian and international approaches and frameworks • May 2007: Workshops with State Records staff • July 2007: Draft policy released for public comment to NSW public sector and archival community • Sept-Oct 2007: Revising policy based on comments received • Dec 2007: Policy to be submitted to State Records’ Board • 2008: Follow up with updated guidance on digital records preservation
Policy scope • Preservation activities undertaken by public offices on their own records eg. migrating records in an EDRMS forward; copying digital records to a different storage medium; taking inactive records ‘offline’ And • Preservation activities that may be taken in the future by State Records in order to preserve digital State archives
Policy informed by.. • Key threats to digital records • Government’s and State Records’ requirements • Available tools and techniques for preservation
Key threats to digital records • Loss of authenticity if they are not adequately controlled and protected • Becoming unreadable as software applications and hardware change • Physical deterioration of media
Loss of authenticity • If records are not kept in official recordkeeping systems they can be vulnerable to tampering or accidental loss • There should be an unbroken ‘chain of custody’ • If a record is not linked to the business it relates to it loses its meaning • Records that are not authentic cannot be relied upon for legal and other purposes
Technology obsolescence • Records are dependent on the software used to create and maintain them • Usually software companies eg Microsoft control the software • Systems are decommissioned and organisations purchase new software not compatible with the old • Storage media changes:
Physical deterioration of media • Threats to digital records media include: • Magnetic fields • Temperature & humidity • Dust • Inappropriate housing or shelving • Disaster
Government / State Records requirements • Legal requirements: • State Records Act 1998 • Evidence Act 1995 • Electronic Transactions Act 2000 • Size and complexity of the NSW public sector • Technologies public offices are using now • The need to preserve both State archives and temporary value records • The maturity and ‘fit’ of available preservation techniques
Available tools and techniques • Digital preservation has advanced rapidly in recent years (NAA, PROV already operational) • A collaborative effort from libraries, archives, universities and others (ADRI, APSR) • Tools & techniques we have focused on: • OAIS model • Open standards eg ODF • Tools for normalisation and management of digital preservation processes • Emerging techniques eg emulation to be monitored
Principle 1: Digital State records should be migrated forward as technologies change • Monitor records to detect risk of obsolescence • Plan and manage migration projects • Migrate long term value / archival records into stable formats • Where no migration paths are available contact State Records to discuss alternative preservation techniques • State Records proposes to migrate archival records to open standard formats plus XML using Xena
Principle 2: The content and essential characteristics of digital State records must remain unchanged throughout the preservation process • Testing should be employed to check that the content and essential characteristics are not compromised by the preservation process • The responsible public office needs to define what ‘content and essential characteristics’ means for its own records • State Records will require this information when accepting transfers of digital State archives
Principle 3: Digital State records must be preserved in context • Information needed to understand and use records should be linked to otherwise associated with them throughout the preservation process • The digital records preservation process itself must be recorded
Principle 4: Digital State records must be secure and tracked throughout the preservation process. • The preserver should implement security measures to ensure that the records being preserved are not compromised during any preservation process • It must be possible to demonstrate an unbroken chain of custody throughout the preservation process.
Principle 5: Digital records preservation programs should be flexible • State Records will maintain digital State archives in bitstream in addition to archival formats • The preserver should seek to based preservation approaches on the use of non-proprietary technologies to avoid loss of control because of changed commercial arrangements
Responsibilities • Chief executives: • ensure all records that require technology to be read or understood remain accessible for as long as they are required (State Records Act 1998, section 14) • Nominated senior officers: • address the preservation of digital records in organisational policy and procedure • Ensure essential characteristics are defined prior to preservation processes • State Records: • Determine arrangements for the preservation and management of digital State archives • Provide public offices with information and advice on digital records preservation
For more information • Contact us: • Ph: (02) 9673 1788 • Email: firstname.lastname@example.org • WWW: www.records.nsw.gov.au