Measuring attitudes

An attitude is a person's feeling toward and evaluation of some object or event. Attitudes have two important aspects: Direction (positive/negative, for or against) and Intensity (strength of feeling). For example, you might like horses - thus, your attitude towards horses has a positive direction. If you are crazy about horses, your attitude toward them has a high level of intensity. You would be intensely positive toward horses.

Because attitudes are so much a part of human behavior, researchers have spent a great deal of time figuring out ways to measure attitudes. There are numerous attitude scales.* We will describe one of the more common approaches here, the Likert scale.

Constructing a Likert scale

The first step is to specify the attitude to be measured. In this example we will use attitude toward mathematics.

Step 1. Collect statements

Generate as many statements as possible covering all aspects of the issue (both pro and con). Do in-depth interviews on the topic, ask colleagues, survey the literature. Here are some examples:

 Math is one of my worst subjects. I like doing math. Math is a science. Men are better at math than women. I am no good in math. I got good grades in math. I need mathematics for my future career. Math is difficult for me. I am confident that I can learn math. Math is an important subject to learn. I can handle most subjects, but not math. These days math instruction at the high school level is of poor quality. I will not need much math when I get out of school.

 Step 2. Judge direction For this step you need to recruit some judges (20 or more people). You will ask them to rate the direction of the statement. Does the statement reflect a positive or negative attitude toward math? You do NOT want their opinion on the item -- this sometimes takes a bit of convincing. Present the collected items in the following format:

Instructions: Please rate each of the following items with regard to its favorability toward math (circle the appropriate number). Do not respond in terms of your own agreement or disagreement with the statements; rather, respond in terms of the judged degree of favorableness or unfavorableness.

 Item Very unfavorable Unfavorable Neutral/Unable to judge Favorable Very favorable 1. Math is one of my worst subjects. 1 2 3 4 5 2. Math is a science. 1 2 3 4 5 3. I am confident that I can learn math. 1 2 3 4 5

Step 3. Discard neutral (or unable to judge) statements

Keep only the items where at least 90% of the judges agree as to direction (favorability rating). Eliminate the statements rated as Neutral/Unable to judge, or those for which judges differ in their opinions (less than 90% agreement). The following statements are not directly for or against math and would be eliminated

Math is a science.
These days math instruction at the high school level is of poor quality.
Men are better at math than women.

 Step 4. Format items to measure intensity. NOTE the different instructions and labels at the top of the columns. This is how the final attitude scale will be presented to respondents.

Instructions: Please indicate your level of agreement with each of the following items (circle the appropriate number).

 Item Strongly disagree Disagree Neutral Agree Strongly agree 1. Math is one of my worst subjects. 1 2 3 4 5 2. I am confident that I can learn math. 1 2 3 4 5 3. Math is difficult for me. 1 2 3 4 5

Counterbalance the items by alternating positive and negative statements. It is OK to have more of one type than the other, but be sure to mix them up on the form.

Step 5. Pilot test (pre-test)

Before printing the final version, pretest the form on a few people that will not be in your final sample. There will ALWAYS be something that needs to be corrected - unclear directions, an ambiguous item, incorrect numbering, typos, etc. It is best to find them before you print hundreds of copies.

Scoring

After the respondent fills out the attitude survey, the researcher must reverse score the negative items (determined in Step 2 above) so that all of the individual item scores lie on the same scale with regard to direction. In reverse scoring, the 5 becomes 1, 4 becomes 2, 3 stays the same, 2 becomes 4 and 1 becomes 5. The reason is that we want to obtain a single score reflecting the intensity in a single direction - that is, we want a high overall score to reflect a positive attitude and a low overall score to indicate a negative attitude. If someone strongly agrees with "Math is difficult for me," the attitude toward math is negative. Although the person has circled 5 on the form, that item (being negative) is scored as a 1.

After the scores on the negative items are reversed, sum the individual ratings. Either a total score or the average is used to characterize the individual's attitude.

Validating the scale

There are 3 ways to demonstrate that a Likert scale is valid, that is, that it measures the attitude that it purports to measure in a credible way.

Item/whole score comparison

The form with all the statements is given to at least 100 respondents. For the final scale, keep only those statements that differentiate between the highest-scoring 25% (most positive toward math) and the lowest-scoring 25% (most negative toward math) of respondents. A drawback of this approach is that it requires generating a lot of items, in order to be sure to have some that differentiate the two extreme attitude groupings.

External criteria

Locate groups of people likely to have strong attitudes for and against the issue, for example engineering versus art history majors with regard to the importance of mathematics. Collect their opinions, and, as above, for the final attitude scale keep only the statements that differentiated the engineers from the art historians.

Factor analysis

Factor analysis is a statistical technique for identifying items that hang together. It requires a large sample, and knowledge of the statistical procedure.

General points regarding Likert scales

Respondents rate their degree of agreement with the statement. Their response shows both the direction (for or against) and intensity (strength) of their attitude.

All statements on the scale must be either positive or negative. The respondents may feel neutral about the statement, but the statement itself cannot be neutral.

Wording on the alternatives can vary, as can the direction. The numbers can go in either direction, but keep the direction the same for all the items.

 Strongly disapprove Disapprove Uncertain Approve Strongly approve 1 2 3 4 5 Strongly agree Agree Neutral Disagree Strongly disagree 1 2 3 4 5 Favor Slightly favor Neither favor nor oppose Slightly oppose Oppose 5 4 3 2 1

If the neutral or uncertain category is omitted, the item becomes a forced choice option. Some researchers assume that everyone has an opinion on everything and therefore should not be allowed to avoid making a choice. Others feel that the respondent may indeed feel neutral towards something, and should be given that response option.

How many alternatives? The consensus is that 5-7 works best, but the rule is not rigid. Sometimes having 3 alternatives -- Agree, Neutral, Disagree -- may be sufficient. In other cases the number can be extended to 9 or 11. Increasing the number is useful when respondents are likely to avoid checking the extreme options.

Limitations

All items, regardless of intensity, are given the same weight. With regard to the final attitude score, strongly agreeing with "I am confident I can learn math" carries the same weight at strongly agreeing with "Math is an important subject to learn."

The format does not lend itself to dealing with mixed or complex attitudes. For example, "Math is an important skill for computer programming, but of less use in politics." Statements that fit the requirements for a Likert scale may not be getting at more complex attitudes and feelings.

Attitude scales are of limited validity. They don't predict behavior very well. Words on a printed page or computer screen bear little resemblance to actual situations. Opinions on a topics such as marijuana use or hate speech restrictions are complex and multidimensional. They might not be reducible to a series of one dimensional items. As with questionnaires in general, self-reports of attitudes and behavior are strongly influenced by the context, format, and wording of items.

A scale can be useful when included on a questionnaire along with additional items for increasing validity, for example, using an attitudes toward marijuana scale, along with open-ended questions about when marijuana use might be OK, who should have access, or how its use affects communities.

Enrichment #2 (not required): Thurstone scale

Next section: Semantic differential

* Likert (described here), Thorndike, Guttman, and Bogardus are the more commonly-used attitude scale formats.