Catalogue description Historic Mortality Datasets

Hardware: In 1985 the data was held by the Office of Population Censuses and Surveys (OPCS) on an IBM mainframe and an ICL 2900 series mainframe. For most of this period the original processing of mortality data and other vital statistics data was done on an IBM 1401 mainframe, which came into use in 1963 and was still being used in 1969.

Operating system: VME; The software formats in which the 1997 version was supplied to purchasers would normally have presupposed a DOS or Windows type operating system.

Application software: OPCS held the 1901-1992 dataset as ASCII text files.

Logical structure and schema: Both datasets contain Historic Deaths tables for each ICD revision (and the period 1901-1910), as well as a single Population table. In addition, the 1901-1995 dataset includes ICD Dictionary tables corresponding to each Historic Deaths table, which explain the codes used in the Historic Deaths tables for causes of death.

Data sources: The data in the datasets is taken from a mixture of published and unpublished sources. The information in all of these sources was originally gathered for purposes other than the compilation of the database. In both the 1901-1992 and the 1901-1995 datasets, data in the Population tables was taken from mid-year population estimates which were periodically issued by OPCS and its predecessors through several series of publications. These included the OPCS Monitor Series, the Registrar General's Quarterly Return for England and Wales, the Registrar General's Statistical Review, the Registrar General's Decennial Supplements and the 73rd Report of the Registrar General (1910). The population estimates also reflect periodic revisions made in light of data from the decennial Census.

Deaths data in the Historic Deaths tables for the period 1901-1958 was derived from published sources. Information on the numbers and causes of deaths in these years was manually transcribed from tables published annually in the Registrar General's Statistical Review. From 1959 onwards mortality data was available in electronic formats. For the period 1959-1967, these took the form of archived computer tapes of data on individual deaths, which had been used in the production of annual reference volumes. For deaths data after 1967, the compilers of the database were able to use computer summaries of mortality data which had already been created for routine tabulation purposes.

Original validation: The data in the Historic Deaths tables was systematically checked against the sources used to compile the data. For data from the period 1901-1958, this involved summing the data on computer by age group for each cause of death, and by cause of death for each age group, and checking the results against the published sources. Where the process of transcribing the data detected printing errors in the published sources, the data in the database was adjusted to achieve consistency rather than to agree with the incorrect published figures. It was also noted that 265 deaths in a colliery disaster in 1934 had not been registered until 1938-39. These deaths were allocated to 1934 in the database. Data for 1959-1967 was checked against published figures, to detect any errors arising from the corruption and loss of data on the archived computer tapes. For data from 1968 onwards, checking involved making sure that every ICD cause of death had been carried across.

Data coding: The creation of the 1901-1992 dataset involved converting ICD codes for causes of death into computer codes, which are substantially different in some cases from ICD codes. These considerations do not affect the 1901-1995 dataset, in which the actual ICD codes are reproduced with relatively minor modifications.

Constraints arising from published sources: These constraints affect data in the Historic Deaths tables for 1901-1958, in both datasets. The age groups into which the data is divided were determined by the age groups used in the Registrar General's Statistical Review in this period. These age groups, in turn, are the basis of the age groups used in the Population tables covering the period for 1901-1958. Similarly, data on causes of death is limited to those causes which were reported in the Registrar General's Statistical Review. In other words, where no incidences of a particular cause of death were recorded for a particular year, that cause will not appear in the datasets.

General constraints affecting mortality data: Regardless of whether published or unpublished sources were the immediate source of data in the Historic Deaths tables, the data was ultimately derived from the system for registering deaths in England and Wales. Data in the Historic Deaths tables was affected by changes in the methods of certifying deaths and of identifying the underlying cause of death.

Medical certificates and coroner's certificates were transmitted periodically by superintendent registrars to the Registrar General. The format of medical certificates changed over time. The most radical change occurred in 1927, when a two part medical certificate was introduced: in the first part the doctor recorded the disease or condition leading directly to death and causes antecedent to it, while the second part was reserved for"other significant conditions contributing to the death, but not related to the disease or condition causing it". From 1940 onwards the entry in the first part of the certificate was taken to be the underlying cause of death.

Further changes to the methods of identifying underlying cause of death occurred in 1984. OPCS adopted a broader interpretation than that previously used of a WHO coding rule, that when the cause of death in the first part of the death certificate was a direct sequel to a condition mentioned in the second part, the latter condition should be preferred as the underlying cause of death. This resulted in an artificial decrease in the numbers of deaths from certain causes (e.g. bronchopneumonia) and corresponding increases in other causes. The anomaly was reversed in 1993, when an overhaul of OPCS's computer systems led to the introduction of an automated system for coding cause of death which followed the internationally agreed interpretation of the WHO's rules for selecting underlying cause.

Two other factors affect mortality data from specific periods:

• During the periods 1915-1920 and 1940-1949, death statistics only record deaths in the civilian population.
• Mortality data for 1981-1982 was affected by industrial action by some registrars. This particularly affected the coding of deaths due to accidents or violence, and cases where more detailed coding was required.

The Historic Mortality Data Files database is believed to have originated in the Medical Statistics Division of the Office of Population Censuses and Surveys (OPCS) as a basic tool for researchers studying mortality in England and Wales. The two versions of the database which have been transferred record the numbers of deaths registered in England and Wales from 1901 onwards by year, sex, age group and underlying cause of death. They also provide estimates of the population at risk of dying by year, sex and comparable age groups. The data was designed to allow for the calculation of national mortality rates, the analysis of trends in mortality and differences in mortality by age and sex, epidemiological research, and local studies of mortality which required national rates for comparative purposes.

The creation of the database was prompted by the creation by the World Health Organisation (WHO), in the mid 1970s, of a database of death rates from 1950 onwards for a number of countries by year, sex and a limited range of causes. This led other institutions to construct similar but more detailed databases to allow for the computer analysis of their own national mortality data. In 1979 OPCS decided to construct a mortality database for England and Wales using readily available sources in published form and on computer. The database was updated annually by OPCS and its successor, the Office for National Statistics (ONS), to include additional years of data. It was distributed commercially to outside purchasers from at least 1985 onwards.

In 1997 the database was redesigned by ONS, and a new version was issued to the public on CD-ROM under the title 'Twentieth Century Mortality Files'. This included data down to 1995, and had a number of features which distinguished it from the earlier version of the database. After 1997, annual update CDs were issued by ONS covering data from 1996 until 1999. In 2003 ONS issued a revision of Twentieth Century Mortality Files covering data from 1901 to 2000, which incorporated revised population estimates based on the 2001 Census.