Tests in Research

Tests and assessments are often used in research to quantify characteristics of individuals and groups. This topics addressed in this resource include:

• Introduction to testing
• Types of tests
• Key concepts in testing:
  • Domain- referenced tests
  • Criterion- referenced tests
  • Norm-referenced tests
• Key concepts in testing:
  • Validity
  • Reliability
• Developing test instruments
• Recommended reading

Testing and measurement is a complex and technical field of study and, of necessity, requires knowledge of statistics.  This resource serves as an introduction. If you wish to develop your own instruments or come to terms with the more technical and statistical aspects of measurement you will find it helpful to refer to relevant handbooks on educational testing and measurement that are listed in the recommended reading.

Introduction to testing

Tests are tools often used in research for data collection.  Human testing and measurement has a long history and for much of the 20th century it was the major element in educational research.  Educational testing paralleled the growth of measurement in psychology (psychometrics). Testing and measurement practices developed alongside statistical procedures and early research was based on the central assumption that intangible aspects of people could be measured and quantified.

Psychometrics

Psychometrics (the ‘measurement of the mind’) is generally considered to have begun in the late 19th century with Francis Galton’s attempts to measure differences between individuals in perception.  During the 20th century, the field of psychometrics developed increasingly sophisticated ways of quantifying subjective human factors in a range of domains.

The key aspect of psychometrics is that it contains within it models of both human mental function and theories of measurement.  Because mental factors are not directly observable, explicit models of their function must be articulated and then tested with quantifiable data.  Developments in psychometrics have been linked to this process of alternatively articulating and then testing such models.  The model then contributes in turn to measurement theory.  Thus models of human mental processes and theories of measurement have interacted with each other and developed over time.

The best known domains in which psychometrics has excelled are those of intelligence (or rational thinking) and personality.  However, psychometrics has developed such powerful models of both mental function and its measurement that its methods and instruments have been widely adopted in educational research and practice.  Scientific educational testing was founded on the models and theories of psychometrics and educational tests in this tradition have been shown to be sound, provided they are used with an awareness of psychometrics and its limitations. Two important points to bear in mind are that:

• psychometric tests are developed to measure specific traits, abilities or characteristics and any one test does not measure everything; results from tests need to be interpreted alongside other things that are known about the person.
• all tests are based on sampling of the domain being tested and of the person’s performance and so all measurement contains error (more of this later) and therefore scores are an estimate.

At its simplest, measurement theory in the context of education relates to devising tests.  Educational tests that measure knowledge or learner characteristics will produce scores whose meaning is derived from population characteristics or norms.  Aspects of measurement theory that  are covered in this unit include the defining of what is to be measured, developing instruments and selecting relevant items, addressing the reliability of the instruments and the validity of inferences made from the results and the use of population norms.

Testing in educational practice and research

Testing is an important feature in current educational practice.  For example, some local authorities use standardised tests in addition to teacher assessments to monitor progress and the development of pupil abilities in key curricular areas such as reading and maths.

Testing is often a feature of research into the impact of interventions using an experimental or quasi-experimental design (see unit X), where the purpose is to see if the intervention has led to an improvement in participant performance in the issue which is the target of the intervention.  For example, studies which investigate the benefits of different approaches to learning how to read make use of standardised tests in reading before, during and after the teaching of children using the different methods.

A recent well publicised (and perhaps contentious) study of approaches to reading is the use of synthetic phonics, for which the researchers used the British Picture Vocabulary Scale, the Clay ‘Ready to Read’ Word Test, The British Ability Scales Word Reading Test, The Primary Reading Test and the Schonell Spelling Test, as well as tests devised by themselves.

Resources for Research in Education by the Applied Education Research Scheme, University of Strathclyde, is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 2.5 UK: Scotland License