Public Records in the Digital Age. Salvador Barragan Curator of Government Records Nebraska State Historical Society. A ‘record’ is the complete set of documentation required to provide evidence of a business transaction . What is a record?.
Curator of Government Records
Nebraska State Historical Society
A ‘record’ is the complete set of documentation required to provide evidence of a business transaction.What is a record?
In our current period records of historical permanency were stored on paper and kept in filing cabinets
When the cabinet was full, records were sent to file room.
Now records are stored electronically on computers
When the computer is ‘full’ – add more hard drives or servers.
Basic skills to manage and maintain records has been lost, replaced by infinite storageShifting Media
5.Create electronic records that are accessible and usable over time (non-proprietary formats).
6.Integrate diverse document forms and formats into records.
7.Identify need for internal and external primary and secondary access to records.
1. Management & PreservationRecords Capture – Records are created or captured and identified to support the business process and meet all records management requirements related to the process.
2. Management & PreservationRecords Maintenance and Accessibility – Electronic records are maintained so that they are accessible and retain their integrity for as long as they are needed.
3. Management & PreservationSystem Reliability – A system is administrated in accordance with best practices in the information resource management field to ensure the reliability of the records it produces.
What happens when you do not have a Management & Preservation
As electronic records become more integrated into society, producers of those records will be held to higher standards of conduct
Federal and State Mandates
Case LawHigher Standards
002.01 Record. The Records Management Act (Revised Statutes of Nebraska, Chapter
84, Article 12) defines a record as: "any book, document, paper, photograph, microfilm,
sound recording, magnetic storage medium, optical storage medium, or other material
regardless of physical form or characteristics created or received pursuant to law, charter,
or ordinance or in connection with any other activity relating to or having an effect upon
the transaction of public business." A record is information that is inscribed on a tangible
medium or that is stored in an electronic or other medium and is retrievable in
The foundation of democracy in America is government accountability to the people and permanency of our culture and heritage.Records Retention
So the question becomes… who takes care of the records, and do they have the knowledge understanding of the new technology?
Historically records sent to file room, staff maintained access to records and managed lifecycle based on need and legal requirements
Now records are managed by users and IT staff, based on capacity and cost.Caretakers of Information
www.digitalarchives.wa.gov function of Records, what are we to do?
When ever possible follow the prevailing best practices and standards.
File Room of the 21 function of Records, what are we to do?st century
Capacity and Speed double every 18 months
First Immutable Law of Digital Archiving
“What hardware you use today will be obsolete within four years”Hardware
Network – Cisco Backbone end to end function of Records, what are we to do?
LAN and SAN
EMC – SAN storage
5 TB now, 20TB by end of Year
HP – Servers and desktops
ADIC – Tape Library for offsite, disaster recovery (nightly or weekly back up, remember Katrina and 9-11)
Microsoft – Software and DevelopmentDigital Archives Hardware
Native function of Records, what are we to do?
PDF/A (Used by the Federal Courts)
XML http://www.thexmltoolkit.org/guides.asp(metadata and interoperability)
DoD 5015.2-STD compliant system
Nebraska State Records Guidelines
Whenever possible seek the
Open, documented solution!
Remember WordStar and DBase II ???Archival Software and File Format Standards
Cross cultural and contextual boundaries function of Records, what are we to do?
Interoperability & Metadata schemaMetadata & Interoperability
Essential to maintain control of the information explosion function of Records, what are we to do?
Allows hard coded rules and information exchange
BUT still requires a strong knowledge, understanding and implementation of basic records management
Second Immutable Law of Digital Archiving:
“Data is Data, a Record is a Record, It is the content that drives retention, not the media”Content Management
DoD 5015.2-STD compliant system function of Records, what are we to do?
Wrap original file in native format
Wrap XML copy
Apply metadata & XML for indexing, searching & retrieval
Provide chain of custody & authenticity
Microsoft Solution function of Records, what are we to do?
SQL Server back end
BizTalk translation utility
SSH Tectia for secure transport
Washington State Archives Case Study‘Content Management’
Maintain Chain of Custody function of Records, what are we to do?
In the care of trusted 3rd party
Received from trusted, known sourceAuthenticity
Encrypted SSH FTP transmission function of Records, what are we to do?
Issue Digital Certificate
Verify IP and computer information
MD5 Hash on all original files
Copy of FTP on tape prior to ingestion
DB backups on tape
Record Level Security for confidential InfoData Security
Restrict records at item, field or series level function of Records, what are we to do?
Restrict to individual, dept, office or global
Uses authenticated login to reveal fields
Anonymous users see ‘Restricted’Record Level Security
Deep Storage XML function of Records, what are we to do?
MUST be flexible function of Records, what are we to do?
Microsoft BizTalk 2004
Transforms, adds metadata based on business rules
Creates ‘deep storage’ copy wrapping original file in XML, with Hash
Creates ‘web’ version of original fileIngestion Process
Designed around latest industry standards function of Records, what are we to do?
Open source, non-proprietary file storage
Applies metadata ‘tags’ to save information about record
creator, date, agency, subject, etc.
Provides chain of custody & authenticity of record
Allow search and retrieval of archival records through a web pageArchive Database
Distributed, non-standardized environment function of Records, what are we to do?
Limited technology expertise in some agencies
Unpredictable data growth rate
Few business models
Limited internal expertiseRisks
Authenticity of record function of Records, what are we to do?
File naming conventions
Start small with e-mail, web page
Use existing retention schedules
Shift AWAY from desktops…
…And move to central servers
Management Software is a must!
Privacy of sensitive dataManagement Issues
“Anything that you do today, will need major overhaul in two years or sooner”
Technology and industry changing at unprecedented rates… But, more records are ‘lost’ every day!
Key is to be flexible and address with systematic forethoughtThird Immutable Law
Ecommerce Functionality two years or sooner”
Add to Shopping cartAdd to Shopping Cart
Comply with statutory & regulatory mandates. two years or sooner”
The Law requires preservation of certain public records – it doesn’t specify whether those records are paper or electronic. All records must be given the same care.
Avoid loss of legal & historical records
As technology changes, the older media (5 ¼” floppy disks, for instance) become harder to read.
Centralization means uniformity in maintenance
‘Trained professionals’ serve as caretakers
Preserve rare and ‘at-risk’ paper records
Improved access for citizens
By centralizing historical electronic records in one location, ‘one-stop shopping’ will provide the information quicker and easierWhy a Digital Archives?
Dr. Ed Papenfuse, State Archivist for the Maryland State Archives.
Andrea Falling, State Archivist for the Nebraska State Historical Society.
Cathy Danahy, Assistant Director of the Nebraska Records Management Division.Acknowledgements