Public Records in the Digital Age. Salvador Barragan Curator of Government Records Nebraska State Historical Society. A ‘record’ is the complete set of documentation required to provide evidence of a business transaction . What is a record?.
Public Records in the Digital Age
Curator of Government Records
Nebraska State Historical Society
A ‘record’ is the complete set of documentation required to provide evidence of a business transaction.
Before Paper we stored valuable historical data on stone and papyrus.
In our current period records of historical permanency were stored on paper and kept in filing cabinets
When the cabinet was full, records were sent to file room.
Now records are stored electronically on computers
When the computer is ‘full’ – add more hard drives or servers.
Basic skills to manage and maintain records has been lost, replaced by infinite storage
5.Create electronic records that are accessible and usable over time (non-proprietary formats).
6.Integrate diverse document forms and formats into records.
7.Identify need for internal and external primary and secondary access to records.
1.Records Capture – Records are created or captured and identified to support the business process and meet all records management requirements related to the process.
2.Records Maintenance and Accessibility – Electronic records are maintained so that they are accessible and retain their integrity for as long as they are needed.
3.System Reliability – A system is administrated in accordance with best practices in the information resource management field to ensure the reliability of the records it produces.
What happens when you do not have a
As electronic records become more integrated into society, producers of those records will be held to higher standards of conduct
Federal and State Mandates
002.01 Record. The Records Management Act (Revised Statutes of Nebraska, Chapter
84, Article 12) defines a record as: "any book, document, paper, photograph, microfilm,
sound recording, magnetic storage medium, optical storage medium, or other material
regardless of physical form or characteristics created or received pursuant to law, charter,
or ordinance or in connection with any other activity relating to or having an effect upon
the transaction of public business." A record is information that is inscribed on a tangible
medium or that is stored in an electronic or other medium and is retrievable in
The foundation of democracy in America is government accountability to the people and permanency of our culture and heritage.
So the question becomes… who takes care of the records, and do they have the knowledge understanding of the new technology?
Historically records sent to file room, staff maintained access to records and managed lifecycle based on need and legal requirements
Now records are managed by users and IT staff, based on capacity and cost.
When ever possible follow the prevailing best practices and standards.
File Room of the 21st century
Capacity and Speed double every 18 months
First Immutable Law of Digital Archiving
“What hardware you use today will be obsolete within four years”
Network – Cisco Backbone end to end
LAN and SAN
EMC – SAN storage
5 TB now, 20TB by end of Year
HP – Servers and desktops
ADIC – Tape Library for offsite, disaster recovery (nightly or weekly back up, remember Katrina and 9-11)
Microsoft – Software and Development
PDF/A (Used by the Federal Courts)
XML http://www.thexmltoolkit.org/guides.asp(metadata and interoperability)
DoD 5015.2-STD compliant system
Nebraska State Records Guidelines
Whenever possible seek the
Open, documented solution!
Remember WordStar and DBase II ???
Cross cultural and contextual boundaries
Interoperability & Metadata schema
Essential to maintain control of the information explosion
Allows hard coded rules and information exchange
BUT still requires a strong knowledge, understanding and implementation of basic records management
Second Immutable Law of Digital Archiving:
“Data is Data, a Record is a Record, It is the content that drives retention, not the media”
DoD 5015.2-STD compliant system
Wrap original file in native format
Wrap XML copy
Apply metadata & XML for indexing, searching & retrieval
Provide chain of custody & authenticity
SQL Server back end
BizTalk translation utility
SSH Tectia for secure transport
Washington State Archives Case Study
Maintain Chain of Custody
In the care of trusted 3rd party
Received from trusted, known source
Encrypted SSH FTP transmission
Issue Digital Certificate
Verify IP and computer information
MD5 Hash on all original files
Copy of FTP on tape prior to ingestion
DB backups on tape
Record Level Security for confidential Info
Restrict records at item, field or series level
Restrict to individual, dept, office or global
Uses authenticated login to reveal fields
Anonymous users see ‘Restricted’
Deep Storage XML
MUST be flexible
Microsoft BizTalk 2004
Transforms, adds metadata based on business rules
Creates ‘deep storage’ copy wrapping original file in XML, with Hash
Creates ‘web’ version of original file
Designed around latest industry standards
Open source, non-proprietary file storage
Applies metadata ‘tags’ to save information about record
creator, date, agency, subject, etc.
Provides chain of custody & authenticity of record
Allow search and retrieval of archival records through a web page
Distributed, non-standardized environment
Limited technology expertise in some agencies
Unpredictable data growth rate
Few business models
Limited internal expertise
Authenticity of record
File naming conventions
Start small with e-mail, web page
Use existing retention schedules
Shift AWAY from desktops…
…And move to central servers
Management Software is a must!
Privacy of sensitive data
“Anything that you do today, will need major overhaul in two years or sooner”
Technology and industry changing at unprecedented rates… But, more records are ‘lost’ every day!
Key is to be flexible and address with systematic forethought
Add to Shopping cart
Comply with statutory & regulatory mandates.
The Law requires preservation of certain public records – it doesn’t specify whether those records are paper or electronic. All records must be given the same care.
Avoid loss of legal & historical records
As technology changes, the older media (5 ¼” floppy disks, for instance) become harder to read.
Centralization means uniformity in maintenance
‘Trained professionals’ serve as caretakers
Preserve rare and ‘at-risk’ paper records
Improved access for citizens
By centralizing historical electronic records in one location, ‘one-stop shopping’ will provide the information quicker and easier
Adam Jansen, Digital Archivist for the Washington State Archives.
Dr. Ed Papenfuse, State Archivist for the Maryland State Archives.
Andrea Falling, State Archivist for the Nebraska State Historical Society.
Cathy Danahy, Assistant Director of the Nebraska Records Management Division.