news     faq     contact us     prov

4.2 Loss of context, authenticity and integrity

There is an extensive treatment of the concepts of context, authenticity, reliability and integrity of electronic records in the archival literature.

The context of a record equates to how the record relates to other records held by an organisation. Context is critical to the use of a record. Frequently, an answer to a question will not be given by one record. Instead, a user is interested in understanding a story which is documented in a collection of related records. The context of a record allows the discovery of these related records.

An authentic record is one that is capable of being proved to be what it purports to be (i.e. the content is what it appears to be, it was created by the person who appears to have created it, and it was created at the time it appears to have been created).

A reliable record is one that contains a full and reliable representation of the facts which the record documents. Note that a record can be authentic, but not reliable. A record is not reliable, for example, if the author of the record left out material facts, misrepresented the position, or simply lied. Such a record would still be authentic as the content is as the author intended and it was created by the apparent author at the apparent time. Authenticity is concerned with the truth of the record as an object; reliability is concerned with the truth of the contents of the record.

Integrity refers to the record being complete and without unauthorised alterations. Note that records can be altered and retain their integrity, provided the alterations are allowed by the policy of the organisation, are authorised, and are documented.

These properties (context, authenticity, reliability, and integrity) are independent of whether the record is paper or electronic. In both paper and electronic records these properties are not contained in the content of the record. Instead, they are partially represented by information associated with the record content (this information is normally known as 'metadata' when dealing with electronic records). Authenticity, reliability, and integrity are also partially dependent on the processes used to capture and manage the records.

The challenge in preserving electronic records is ensuring that the systems that manage the electronic records hold sufficient metadata and implement suitable processes to ensure the long-term retention of context, authenticity, reliability, and integrity.

In a traditional paper-based recordkeeping system these properties are largely demonstrated by the procedures involved in the creation, storage, and handling of the record. For example, reliability is shown by the fact that the record was created for future reference as part of a standard business procedure. Authenticity and integrity is shown by the procedures involved in managing and controlling access to records. Ultimately, these procedures are backed up by conventional forensic tests such as tests on signatures, the age of the paper, type of typewriter, and ink.

This reliance on procedures can be transferred to many electronic records, particularly those managed by application specific systems. Consider a financial system, for example. The records would be considered reliable because they are automatically generated by the system as a side-effect of carrying out financial tasks. They are authentic because the actions can only be carried out via the financial system and the system keeps logs of who carried out the task, when it was carried out, and how the tasks are related. Finally, the logs record any changes to the records, and hence the records have integrity.

However, many electronic records are not managed in such a formal way. This particularly applies to those records held in generic software applications (e.g. email systems) or in the general file system. Fundamentally, the problem is that these systems are not designed to ensure authentic records or to ensure their integrity once created. These records can be the most important held by an agency; for example, they may document the development of government policy.

One method of ensuring authenticity and integrity of these records is to install an application that is designed to manage records and to ensure their authenticity and integrity (a recordkeeping system). Once records are registered with the recordkeeping system, the system can ensure that the record retains integrity. Essentially, the recordkeeping system acts as a vault, mediating and recording access to the records. Just like the financial system, the recordkeeping system only allows certain operations on the registered records, only allows authorised users to perform those operations, and keeps audit trails of all operations.

However, there are several issues with using a recordkeeping system to ensure the reliability and integrity of records.

The effectiveness of a recordkeeping system depends on users placing their records under the control of the system. At some point, for example, users must move their emails from their mailbox to the recordkeeping system. This is to be contrasted to a financial system, for example, where the system is used to carry out the tasks associated with managing money, the records being automatically generated as a side-effect. With a recordkeeping system, the tasks are carried out in other applications and users have to consciously decide to place the records under the control of the system.

Care needs to be taken that users with special access cannot subvert systems holding records. Typical special access users are records administrators or (computer) system administrators. However, it should be noted that such users can equally subvert traditional paper-based records systems, so this issue is no different in the electronic environment. The question is whether advantage should be taken of technology to close this hole.

Management by a recordkeeping system should be viewed as a medium-term solution. Any computer system has a relatively short life - say five to ten years - and there must be a plan to extract records from a system and to migrate them to a replacement system (or to manage them by some other mechanism if there is no replacement system). This migration is likely to be complex, as it is necessary to preserve sufficient information to show that the record was properly managed to ensure authenticity and integrity when under control of the original system. A particular concern about migration is that this may have to occur under extreme time or budgetary constraints. These constraints typically occur if an agency (or section) is closed and the records are no longer considered of operational interest. An example would be a Royal Commission. Funding for migration is likely to be minimal in these circumstances, and the time available for migration very short.

VERS defines a standard set of metadata that holds the information necessary to show the context, authenticity, reliability, and integrity of a record. The metadata is based upon that defined by the National Archives of Australia [NAA]. The VERS Standard requires that this metadata be encapsulated with the record content in a single object (the VERS Encapsulated Object, or VEO) upon export to PROV. The full VERS metadata is defined in PROS 99/007 Specification 2: VERS Metadata Scheme. PROS 99/007 Specification 3: VERS Standard Electronic Record Format defines the standard format of the VERS Encapsulated Object and the standard representation of the metadata.

A significant benefit of specifying a standard metadata scheme is in enforcing data normalisation. An archive will receive records from many agencies. This potentially leads to very serious problems of consistency of metadata. It would be almost impossible to provide a unified view of the collection if, for example, each agency used a different metadata element to contain the title. One benefit of defining standard metadata is that the agency performs the normalisation of the metadata. The agency understands the record and the source recordkeeping system.

A second benefit of this approach is that the metadata is encapsulated with the record content in a single object (the VERS Encapsulated Object, or VEO). The importance of this is that it is far less likely for the metadata to become separated from the record content. This is to be contrasted to the situation where the metadata is held in a database separate from the content. In this situation it is easily possible to lose the metadata, or to lose the linkage between the metadata and the content. If either of these situations occur, the record context, authenticity, reliability, and integrity are lost.

VERS uses digital signatures to show that a record has not been altered. A digital signature is the result of applying a mathematical function to the record and is a secret known only to the signer. A related mathematical function can be used to verify the digital signature. The VERS Standard contains metadata elements that contain the necessary information required to validate digital signatures. The way digital signatures are applied to VEOs is defined in PROS 99/007 Specification 3: VERS Standard Electronic Record Format.

Many archives do not specify that it is necessary to digitally sign records. Instead, integrity is shown by custody in an archival system. This has been the traditional approach to showing authenticity and integrity of paper records held by an archive. The reason PROV feels that this is inappropriate for electronic records is that custody was always backed up by forensic tests with paper records. Such tests are in their infancy with electronic records. Further, a digital archive is a far less benign environment than a paper repository and records can easily be altered by software bugs and hardware failures. Such failures can systematically affect large parts of the collection. It was felt that a verification mechanism independent of the digital archive was desirable.

back to top

printer friendly

VERS Story | Standard | Assessment | Projects | Digital Archive | Training | Toolkit | Publications

Site last reviewed: 23 Sep 2009
© State of Victoria 2009 | Disclaimer | Privacy Statement