Comparing the Reliability of Paper and Computer-based Tests
January 2004 - June 2004(Contact: Dr Bronwen Swinnerton)
Funded By: QCA
Other project members:
Dr Matt Homer
This research examines the performance data from four sittings of World Class Tests in mathematics for 9 and 13 year olds. These tests are composed of a paper and a computer component and the research will assess whether the tests as a whole, and also by component, are ‘reliable’, and more specifically whether the computer component is as ‘reliable’ as the paper component.
Inextricably linked to the ‘reliability’ of a test is the ‘validity’ of a test. When a pupil sits a test, the pupil, the teacher, parents and the test setter want to be confident that the test measures what it purports to measure. The results of these tests may not have crucial consequences for the pupil sitting the test, but all those involved want to be confident that any interpretations of the test results are accurate.
The research also addresses a number of specific questions:
- how reliable are the World Class Tests in mathematics?
- how do the reliability of the paper and computer components of the World Class Tests in mathematics compare?
- does placing mathematics tests on computer affect reliability?
- can performance in one medium be predicted from performance in the other medium?
- how do pupil’s perform relatively in the two media?
- do items of a particular kind, or items involving a particular content area predict performance more or less well than others?
This project last updated by Matt Homer on 12th August 2005.