Department of Education and Science and successors: Statistics Branch: Schools' Census (Form 7) Datasets
The Schools' Census datasets contain data on individual schools derived from their Schools' Census returns. Data is currently available for the following years: 1975-2001. The data for the years 1975-1994 are divided into datasets which are based on the major categories of schools covered by the census. The following list outlines the sub-series of datasets which are currently available:
- Primary, middle and secondary schools: 1975-1994;
- Special schools: 1975-1994;
- Nursery schools: 1977-1994;
- Independent schools: 1975-1994.
After 1994, the Department continued to collect information on all the above school types, but no longer maintained the data in separate datafiles.
The datasets for 1975-1977 contain records on schools in England and Wales. From 1978 onwards only English schools are included, except in the datasets for independent schools, which continue to include Welsh schools up until 1985. For reasons which are unclear, the independent school datasets for 1975-1976 also contain data on LEA-maintained and direct-grant nursery schools. From 1977 onwards, data on nursery schools is held in separate datasets.
The datasets provide the following types of information:
- Basic details about schools, such as the school's LEA and region, the type of school, any denominational affiliation, the sex of the pupils, and the school's establishment number. Establishment numbers (known as ESTAB or SCH numbers) act as a unique identifier for each school when combined with its LEA number. The names of schools are not included in the datasets, although it may be possible to identify schools using another database transferred to NDAD, the Register of Educational Establishments (see Related Material).
- Numbers of pupils by age, sex, whether full-time or part-time, and (for pupils with special educational needs) by type of disability. The latter data is not included in the 1989-1994 datasets.
- Numbers of staff by sex, type and whether full-time or part-time, plus the full-time equivalents of part-time staff.
- Data on classes on the registers of the school and classes taught at a particular period of the school day, by type of class, numbers of pupils, and numbers of teachers teaching.
- Data on courses of study in secondary and independent schools, including the numbers of pupils by age and sex pursuing examination courses ('A' level, 'O'/CSE level etc). This data is not included in the datasets for 1986-1991 inclusive; after 1992 it is reinstated.
The datasets in this series are available to download. Links to individual datasets can be found at piece level.
Hardware: ICL (possibly an ICL 2900) until c 1993, when the mainframe was decommissioned. Thereafter networked PCs. ASCII files exported from the SIR database were stored on magnetic tapes until c 1992, when migrated to optical disks; these were inherited by Analytical Services Directorate in 1999.
Operating System: GEORGE III until 1988, when replaced by VME. After the decommissioning of the mainframe, SIR ran on UNIX. Also MS DOS and Microsoft Windows (various).
Application Software: The production and subsequent use of Schools' Census datasets at the DfEE and its predecessors involved two separate operations, for which different packages were used. These operations were:
- The initial inputting and validation of the data.
- The subsequent analysis and tabulation of the data to produce published statistics.
The SIR relational database management system manufactured by SIR Pty Ltd is thought to have been for the first purpose from the 1970s. These datasets were also imported into the analysis packages used by Analytical Services' statisticians. During the 1970s and 1980s they used two applications, thought to have been developed in-house, about which little information is available: 'Stages' (used in the 1970s) is thought to have been a batch processing based system, while its successor 'Fasolt' (standing for 'facility for on-line tabulation') was used in the 1980s. It is believed that by the early 1990s these utilities had been replaced by QuickTab. By 2000 the DfEE were analysing Schools' Census data using QuickTab's successor, QStat. The Department for Education and Skills intended to replace QStat with SPSS for data analysis purposes once PLASC was introduced to all LEA-maintained primary, middle, secondary and special schools. By October 2001 the Data Collection Unit was also in transition from using SIR to using Microsoft SQL Server for building 'collecting' databases for the surveys with which it dealt.
Logical structure and schema: The datasets normally consist of a single table per dataset. The exceptions to this are the 1976, 1992, 1993 and 1994 datasets for primary, middle and secondary schools. For 1976, tables were created by NDAD to facilitate access to the data. 1992 and 1993 have three tables per dataset; the 1994 dataset has four tables. The datasets for 1995-1997 have six tables, while those for 1998-2001 have seven.
How data was originally captured and validated: Schools' Census data was gathered through forms which were completed by schools in January each year. Most questions asked schools to supply data reflecting the situation on an enumeration date, traditionally the third Thursday in January. In later years schools were increasingly asked to supply data for periods other than the enumeration date. The forms for LEA-maintained schools were distributed and collected by LEAs, which would then send the forms to the DfEE and its predecessors, sometimes after checking the forms and extracting data. Independent schools, grant maintained schools, and direct grant schools received their forms from the DfEE and its predecessors, to which they returned their completed forms.
The following Schools' Census forms were used:
- Form 7: Different versions were used for LEA-maintained primary, middle and secondary schools, and later city technology colleges.
- Form 7M: For special schools and hospital schools.
- Form 7R: For pupil referral units, a type of LEA-maintained school established by the Education Act 1993 to provide temporary education for certain types of pupils who were not able to attend a mainstream school.
- Form 9/26: Apparently used during the 1970s for direct grant technical schools.
- Form 11: For LEA-maintained nursery schools and direct grant nursery schools.
- Form 16: For direct grant grammar schools up to 1980.
- Form 30: Up to and including the 1978 census for independent schools 'recognised as efficient', i.e. independent schools which had 'sought and obtained recognition as efficient after inspection by Her Majesty's Inspectors of Schools'. In 1978 the arrangements for recognition as efficient were brought to an end. From 1979 onwards independent schools which had previously fallen into this category completed Form 101a.
- Form 101a: For 'other' independent schools up to 1978, and for all independent schools from 1979 onwards.
During the 1990s it became increasingly common for schools to generate their returns in digital form. This enabled validation checks to pick up discrepancies in the data. By 2000 about 70 percent of Schools' Census returns were being submitted digitally. Schools could send their digital returns on floppy disk. During the late 1990s data from paper returns was input for the DfEE by a data preparation bureau and transmitted to the DfEE electronically. Data from both the paper and digital forms was entered into a collecting database by the Data Collection Unit using SIR. The data was checked and validated, and datasets produced which were both archived (as ASCII files) and imported into the analysis packages used by Analytical Services' statisticians.
As the statisticians also kept the datasets in the analysis formats which they were using, this meant that there were effectively two 'archives' of Schools' Census data in the DfEE:
- ASCII files exported from the SIR collecting database.
- Files in QuickTab/QStat format which were 'owned' by the statisticians in Analytical Services Directorate and used by them for analysis.
Constraints on the reliability of the data: The metadata transferred with the 1975-1991 datasets has a number of defects. Many of the field descriptions in the data dictionary files transferred with the datasets show signs of having been truncated, often at around 60 or 70 characters. This truncation is thought to have occurred in 1991-1992 when the datasets held by the Department's statisticians were migrated from Fasolt to QuickTab format.
However, even where field descriptions have not been affected by truncation, the descriptions do not always provide a complete picture of the functions of fields. It is not unusual for more than one field to have the same description even though the fields contain different data.
For the 1975-1991 datasets, the National Digital Archive of Datasets (NDAD) supplemented the original field descriptions by comparing the data with examples of completed Schools' Census forms. Although this work has clarified the functions of a number of fields, some ambiguities remain which will have implications for future use of the data.
In addition to the problem of truncated or inadequate field descriptions, some of the datasets for the nursery schools and independent schools also exhibit a problem with certain fields where the data should be a real number (i.e. a decimal), but the decimal place appears to be either missing or in the 'wrong' location. This problem does not affect the datasets for 1986 onwards.
Validation performed after transfer: Details of the content and transformation validation checks performed by NDAD on the Schools' Census datasets are contained in the catalogues of individual datasets.
Department for Education and Employment, 1995-2001
Department of Education and Science, 1964-1992
Within the DfEE and its predecessors responsibility for conducting the Schools' Census and analyzing the data passed successively through a number of divisions. At the time of the census in January 2001 this responsibility rested with Qualifications, Pupil Assessment and Information Technology Support Division (QPAI) within the Analytical Services Directorate of the DfEE's Finance and Analytical Services Directorate. Two teams within the QPAI dealt with the Schools' Census: the Data Collection Unit, which prepared and validated the data; and the Schools Statistical Unit, whose statisticians designed the survey and analysed the data. During the 1990s the Information Technology Branch and its successors, the Information Systems Branch, the Information Systems Division and the Information Services Division appear to have temporarily absorbed the IT functions of Analytical Services Branch, including the Data Collection Unit. From ca. 1992 until 1999 they also ran a Corporate Data Archive in which Schools' Census data and data from other surveys was preserved. Responsibility for the Archive was transferred to Analytical Services Directorate in 1999 as they 'owned' most of the data.
Little is known about the early history of the Schools' Census. Education statistics based on annual returns submitted by schools have been published at a national level since at least 1905. Form 7 returns for LEA-maintained schools are thought to have been introduced in 1945-1946. Independent schools were required to provide information for the Schools' Census as a result of provisions in the Education Act 1944 and the Independent Schools Registration Regulations 1957, which required independent schools to furnish annual returns of changes to their pupils, boarders and teaching staff. In more recent years, the statutory basis of the Schools' Census was apparently provided by section 537 of the Education Act 1996. This required the governing bodies of LEA-maintained schools and the proprietors of independent schools to provide such information as might be required by the Secretary of State for Education.
Data from the Schools' Census was used to calculate schools' per capita allocation of funding, leading to a high response rate (around 95 percent in 1998). The information was used for a variety of purposes within the DfEE and its predecessors, including setting priorities and answering questions from official, parliamentary and private sources. Data on pupil numbers was used to assess LEAs' rate support grants and to project pupil numbers in future years; data on the numbers of teachers was used to assess national staffing levels. LEAs also used the data for their own capitation purposes, staffing allocations and for projecting pupil numbers and staffing levels in the future. Information derived from the Schools' Census was used by the DfEE and its predecessors to produce a number of statistical publications.
The final Schools' Census was conducted by the DfEE in January 2001. In 2000 and 2001 pilots were carried out in selected schools of a Pupil Level Annual Schools' Census (PLASC), intended to replace the Schools' Census in all LEA-maintained primary, middle, secondary and special schools in England from January 2002. Most of the data in PLASC returns will be generated automatically from schools' information management systems. Unlike Schools' Census returns, records on individual pupils will be included.
Found an error? Suggest a correction to help improve our descriptions.
Users have not yet tagged this record
Use the form below to include your knowledge of this record. If your description includes any citable references, please add these to the "Citable References?" field on the right, remebering to seperate each reference with a comma, for example (FO 176/1/1, FO 178/2/1, and so on...)
- ED Records created or inherited by the Department of Education and Science, and of related...
- Records of the Statistics, Research and Intelligence, and Planning Branches
- ED 267 Department of Education and Science and successors: Statistics Branch: Schools' Census...