Department of Education and Science and successors: Statistics Branch: Schools' Census (Form 7) Datasets
|Title:||Department of Education and Science and successors: Statistics Branch: Schools' Census (Form 7) Datasets|
The Schools' Census datasets contain data on individual schools derived from their Schools' Census returns. Data is currently available for the following years: 1975-2001. The data for the years 1975-1994 are divided into datasets which are based on the major categories of schools covered by the census. The following list outlines the sub-series of datasets which are currently available:
After 1994, the Department continued to collect information on all the above school types, but no longer maintained the data in separate datafiles.
The datasets for 1975-1977 contain records on schools in England and Wales. From 1978 onwards only English schools are included, except in the datasets for independent schools, which continue to include Welsh schools up until 1985. For reasons which are unclear, the independent school datasets for 1975-1976 also contain data on LEA-maintained and direct-grant nursery schools. From 1977 onwards, data on nursery schools is held in separate datasets.
The datasets provide the following types of information:
The datasets in this series are available to download. Links to individual datasets can be found at piece level.
Hardware: ICL (possibly an ICL 2900) until c 1993, when the mainframe was decommissioned. Thereafter networked PCs. ASCII files exported from the SIR database were stored on magnetic tapes until c 1992, when migrated to optical disks; these were inherited by Analytical Services Directorate in 1999.
Operating System: GEORGE III until 1988, when replaced by VME. After the decommissioning of the mainframe, SIR ran on UNIX. Also MS DOS and Microsoft Windows (various).
Application Software: The production and subsequent use of Schools' Census datasets at the DfEE and its predecessors involved two separate operations, for which different packages were used. These operations were:
The SIR relational database management system manufactured by SIR Pty Ltd is thought to have been for the first purpose from the 1970s. These datasets were also imported into the analysis packages used by Analytical Services' statisticians. During the 1970s and 1980s they used two applications, thought to have been developed in-house, about which little information is available: 'Stages' (used in the 1970s) is thought to have been a batch processing based system, while its successor 'Fasolt' (standing for 'facility for on-line tabulation') was used in the 1980s. It is believed that by the early 1990s these utilities had been replaced by QuickTab. By 2000 the DfEE were analysing Schools' Census data using QuickTab's successor, QStat. The Department for Education and Skills intended to replace QStat with SPSS for data analysis purposes once PLASC was introduced to all LEA-maintained primary, middle, secondary and special schools. By October 2001 the Data Collection Unit was also in transition from using SIR to using Microsoft SQL Server for building 'collecting' databases for the surveys with which it dealt.
Logical structure and schema: The datasets normally consist of a single table per dataset. The exceptions to this are the 1976, 1992, 1993 and 1994 datasets for primary, middle and secondary schools. For 1976, tables were created by NDAD to facilitate access to the data. 1992 and 1993 have three tables per dataset; the 1994 dataset has four tables. The datasets for 1995-1997 have six tables, while those for 1998-2001 have seven.
How data was originally captured and validated: Schools' Census data was gathered through forms which were completed by schools in January each year. Most questions asked schools to supply data reflecting the situation on an enumeration date, traditionally the third Thursday in January. In later years schools were increasingly asked to supply data for periods other than the enumeration date. The forms for LEA-maintained schools were distributed and collected by LEAs, which would then send the forms to the DfEE and its predecessors, sometimes after checking the forms and extracting data. Independent schools, grant maintained schools, and direct grant schools received their forms from the DfEE and its predecessors, to which they returned their completed forms.
The following Schools' Census forms were used:
During the 1990s it became increasingly common for schools to generate their returns in digital form. This enabled validation checks to pick up discrepancies in the data. By 2000 about 70 percent of Schools' Census returns were being submitted digitally. Schools could send their digital returns on floppy disk. During the late 1990s data from paper returns was input for the DfEE by a data preparation bureau and transmitted to the DfEE electronically. Data from both the paper and digital forms was entered into a collecting database by the Data Collection Unit using SIR. The data was checked and validated, and datasets produced which were both archived (as ASCII files) and imported into the analysis packages used by Analytical Services' statisticians.
As the statisticians also kept the datasets in the analysis formats which they were using, this meant that there were effectively two 'archives' of Schools' Census data in the DfEE:
Constraints on the reliability of the data: The metadata transferred with the 1975-1991 datasets has a number of defects. Many of the field descriptions in the data dictionary files transferred with the datasets show signs of having been truncated, often at around 60 or 70 characters. This truncation is thought to have occurred in 1991-1992 when the datasets held by the Department's statisticians were migrated from Fasolt to QuickTab format.
However, even where field descriptions have not been affected by truncation, the descriptions do not always provide a complete picture of the functions of fields. It is not unusual for more than one field to have the same description even though the fields contain different data.
For the 1975-1991 datasets, the National Digital Archive of Datasets (NDAD) supplemented the original field descriptions by comparing the data with examples of completed Schools' Census forms. Although this work has clarified the functions of a number of fields, some ambiguities remain which will have implications for future use of the data.
In addition to the problem of truncated or inadequate field descriptions, some of the datasets for the nursery schools and independent schools also exhibit a problem with certain fields where the data should be a real number (i.e. a decimal), but the decimal place appears to be either missing or in the 'wrong' location. This problem does not affect the datasets for 1986 onwards.
Validation performed after transfer: Details of the content and transformation validation checks performed by NDAD on the Schools' Census datasets are contained in the catalogues of individual datasets.
Similar censuses have been carried out since the beginning of this century but the results have only survived in printed form. The following datasets are related:
A database of Grant Maintained Schools is in ED 278
A Register of Educational Establishments is in NV 2
Data gathered in the 1978 for LEA -maintained primary, middle and secondary schools has been deposited at the UK Data Archive. Statistical returns for 1944-1955 produced by the Ministry of Education on the basis of information gathered in the Schools' Census are held in
|Held by:||The National Archives, Kew|
|Former reference in The National Archives:||CRDA/13|
|Legal status:||Public Records unless otherwise stated|
Department for Education, 1992-1995
Department for Education and Employment, 1995-2001
Department of Education and Science, 1964-1992
|Physical description:||86 datasets and documentation|
|Restrictions on use:||The 1985-1991 datasets were formerly closed for 20 years. Following the Freedom of Information Act coming into force, the Department agreed that no FOI exemptions apply to the data. The datasets were opened on 10 February 2005. Crown Copyright; copies may be made for private study and research purposes only.|
|Access conditions:||Open unless otherwise stated|
|Immediate source of acquisition:||
In 2010 United Kingdom National Digital Archive of Datasets
|Custodial history:||The Schools' Census datasets and related dataset documentation were originally transferred from the Department for Education. The United Kingdom National Digital Archive of Datasets (NDAD) then held the datasets until 2010 when they were transferred to The National Archives (TNA).|
|Accruals:||Further accruals are not anticipated.|
Statistics derived from the Schools' Census were made available by the DfEE and its predecessors in a variety of publications. Up to and including the Ministry of Education's annual report for 1960, statistics were included in part II of the annual report. In 1962 the Ministry introduced a new series of annual Statistics of Education volumes: this has been issued in series covering the years 1961-1965, 1966-1979 and 1980 onwards.
|Unpublished finding aids:||
Extent of documentation: 553 documents, Dates of creation of documentation: 1974-2001
|Administrative / biographical background:||
Within the DfEE and its predecessors responsibility for conducting the Schools' Census and analyzing the data passed successively through a number of divisions. At the time of the census in January 2001 this responsibility rested with Qualifications, Pupil Assessment and Information Technology Support Division (QPAI) within the Analytical Services Directorate of the DfEE's Finance and Analytical Services Directorate. Two teams within the QPAI dealt with the Schools' Census: the Data Collection Unit, which prepared and validated the data; and the Schools Statistical Unit, whose statisticians designed the survey and analysed the data. During the 1990s the Information Technology Branch and its successors, the Information Systems Branch, the Information Systems Division and the Information Services Division appear to have temporarily absorbed the IT functions of Analytical Services Branch, including the Data Collection Unit. From ca. 1992 until 1999 they also ran a Corporate Data Archive in which Schools' Census data and data from other surveys was preserved. Responsibility for the Archive was transferred to Analytical Services Directorate in 1999 as they 'owned' most of the data.
Little is known about the early history of the Schools' Census. Education statistics based on annual returns submitted by schools have been published at a national level since at least 1905. Form 7 returns for LEA-maintained schools are thought to have been introduced in 1945-1946. Independent schools were required to provide information for the Schools' Census as a result of provisions in the Education Act 1944 and the Independent Schools Registration Regulations 1957, which required independent schools to furnish annual returns of changes to their pupils, boarders and teaching staff. In more recent years, the statutory basis of the Schools' Census was apparently provided by section 537 of the Education Act 1996. This required the governing bodies of LEA-maintained schools and the proprietors of independent schools to provide such information as might be required by the Secretary of State for Education.
Data from the Schools' Census was used to calculate schools' per capita allocation of funding, leading to a high response rate (around 95 percent in 1998). The information was used for a variety of purposes within the DfEE and its predecessors, including setting priorities and answering questions from official, parliamentary and private sources. Data on pupil numbers was used to assess LEAs' rate support grants and to project pupil numbers in future years; data on the numbers of teachers was used to assess national staffing levels. LEAs also used the data for their own capitation purposes, staffing allocations and for projecting pupil numbers and staffing levels in the future. Information derived from the Schools' Census was used by the DfEE and its predecessors to produce a number of statistical publications.
The final Schools' Census was conducted by the DfEE in January 2001. In 2000 and 2001 pilots were carried out in selected schools of a Pupil Level Annual Schools' Census (PLASC), intended to replace the Schools' Census in all LEA-maintained primary, middle, secondary and special schools in England from January 2002. Most of the data in PLASC returns will be generated automatically from schools' information management systems. Unlike Schools' Census returns, records on individual pupils will be included.
Context of this recordBrowse by Reference
- ED - Records created or inherited by the Department of Education and Science, and of related bodies
- Division within ED - Records of the Statistics, Research and Intelligence, and Planning Branches
- ED 267 - Department of Education and Science and successors: Statistics Branch: Schools' Census (Form 7) Datasets