Attitude scales: Basic Concepts, Measures of Reliability and Validity

Like all research instruments, a good attitude scale must fulfil three principles.

• It must discriminate. Just as a good watch can tell you the difference between it being 11 a.m. and 3 p.m. A so-called “Attitude to Politics” scale on which Margaret Thatcher and Tony Benn get the same score probably lacks discrimination.
• It must show reliability. Just as, if I live in Greenwich, my watch (on BST) will always show me 1.00 p.m. as the sun is directly south in the sky.
• It must have validity. Just as, in my car with the cruise control at 60 mph, my car clock will tick on exactly one minute as the milometer advances by exactly one mile

Test reliability

1. An attitude scale, as we have seen, consists of a number of component items and a score which can be attributed to each (e.g. 1 to 5 on a typical Likert-scale item). The individual scores are then added up to give an overall attitude score.
2. Most measures of checking the reliability of an attitude scale essentially ask the question:  Do the separate items broadly measure the same quality?
3. The statistic which measures this is usually a correlation coefficient of some kind. While this resource is not about correlation coefficients, it is sufficient for our purposes to know that a correlation coefficient provides an answer to the kind of question we have seen above on a scale of 0 to 1: – where 0 means “Not at all”, 1 means “yes perfectly” and scores like, for example, 0.7 mean “Not perfectly, but to a very considerable extent”

Measures of reliability

We will give three common ways of how measures of reliability are done:-

• Split-Half Reliability: A measure of consistency where a test is randomly split in two halves and the scores for each half of the test are compared with one another. A correlation coefficient is calculated, measuring the extent to which people who score high on one half also score high on the other half (and of course conversely, the extent to which low scores on one half tend ro be associated with low scores on the other half). Essentially, if one half of the test appears to measure the same thing as the other half, then we have evidence that the whole test is indeed measuring something reliably.
• Cronbach’s Alpha  Less easy to understand than split-half reliability, and more difficult to compute (not that, with computing power, that matters today). But again it is a rather sophisticated measure of whether different bits of the test or scale are measuring the same thing
• Item-whole scale correlation This one, because it gives a score for each item, allows one to improve the scale by dropping the worse performing items. [See the Item Shredding section below for a prototype “attitudes to nurse education degrees” which gave the results when it was initially piloted on 45 nursing students in Paisley University (Sheward & Smith, 2007)]

Item shredding

Item-whole scale correlation This gives a score for each item, allows one to improve the scale by dropping the worse performing items.  Here we will use an example of how we might improve the data results through data shredding.
Example 1:    See below a prototype “attitudes to nurse education degrees” which gave the following results when it was initially piloted on 45 nursing students in Paisley University (Sheward & Smith, 2007)

		item-whole scale correlation
1	Will help nurses develop their career	.041
2	Will help nurses consider jobs outside nursing	.326
3	Will increase self confidence	.221
4	Will increase knowledge	.352
5	Will be more satisfying than obtaining a nursing diploma	.217
6	Ought to be essential for all student nurses	.386
7	Ought to be essential for all nurses	.438
8	Will increase professional competence	.454
9	Is only essential for nurses who are ambitious	.014
10	Will not make nurses better at their jobs	.325
11	Will increase professional status	.323
12	Will improve the patient care provided by nurses	.432
13	Will not help nurses consider a wider range of job opportunities in nursing	.391
14	Is the gold standard for nurses	.258
15	Is what employers of nurses want	.282
16	Is essential to be a good nurse	.405
17	Is easier as a student nurse because in studying mode	.415
18	Will be difficult for many nurses because they lack the academic ability	.057
19	Will increase nurses clinical skills	.341
20	Is more likely if nurses are funded	-.033
21	May put some nurses off training to be nurses if becomes essential	-.194
22	May cause problems with qualified staff who do not have a degree	-.054
23	May cause problems with other students who are not studying for a degree	.218
24	Will make nursing a more attractive career option if becomes essential	.405
25	Will be easier as a student than post registration	.566
26	Will increase choice of nursing jobs	.118
27	Will not improve job opportunities in the future	-.120
28	Is an excellent opportunity as a student nurse	.119
29	Should be recognised on the Nursing and Midwifery Council Register	.236
30	Should mean higher pay than for nurses with a diploma	.221
31	Will promote a two tier system in nurse education	-.045
32	Is necessary to further a career in nursing	.547
33	Will be difficult for many nurses because they lack confidence in their academic ability	-.069

Cronbach alpha = 0.39 Split-half 0.27
There is no simple answer as to how to improve this data. Attitude scales should not have items where the item-whole scale correlation is small. On the other hand, other things being equal, a longer scale is more reliable than a shorter one: so one should not throw out too many items.

We removed the following items in three steps:-

• Step 1: We removed items 1,9,18,20,22,31 and 33 i.e. all those whose correlation with the whole score was less than 0.1. There was only a modest improvement in Cronbach to 0.40; but the spilt-half correlation rose to 0.41
• Step 2 : We removed items 21, 26, 27 and 28 i.e. all those whose correlation with the whole score was less than 0.2. Cronbach’s alpha rose still modestly to 0.42; but the split-half correlation rose to 0.54
• Step 3: We removed items 3,5,14,15,23,29 and 30 i.e. all those whose correlation with the whole score was less than 0.3. Cronbach’s Alpha now rose to 0.85. And the split-half coefficient rose to 0.70

The resulting 15-item attitude scale is a much more reliable instrument than the initial 33-item scale
In general terms, you should look for published attitude tests/scales with Cronbach’s alpha and/or split-half (or similar) correlation coefficients of 0.7 or more.

Test /Scale Validity

An attitude scale may well be reliable, but still leave one in doubt as to what it is measuring. In the case of the test shown above, for example, we did indeed find that it discriminated well between nurse students who elected for a graduate training route and those who settled for a diploma. That is evidence that the scale measured what is supposed to measure – which captures what the idea of validity is about.
In general terms, there are four ways of tackling the issue of validity

• a)  Content validity
• b)  Concurrent validity
• c)  Predictive validity
• d)  Construct validity

(Oppenheim 1992)

Back

Resources for Research in Education by the Applied Education Research Scheme, University of Strathclyde, is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 2.5 UK: Scotland License