For more information:
Call: 1-800-447-3269
E-mail: info-ncs@pearson.com
Survey.PearsonAssessments.com
The Importance of Validity
Validity is one of the most important concepts in survey research. Without validity, you have meaningless results and have wasted a great deal of time, energy, and money. Much of the confusion surrounding validity is probably due to the cavalier manner in which the term frequently is used. This article presents general issues surrounding validity, as well as different types of validity.
Validity Defined
For many consumers their only concern with validity is that their survey has it. After all, isn't validity just another of 'those statistical things'? According to the American Psychological Association, validity "...refers to the appropriateness, meaningfulness, and usefulness of the specific inferences made from test scores." (Standards for Psychological and Educational Testing, 1985, p. 9). In other words, if your findings need to be appropriate, meaningful and useful, they need to be valid.
For the purposes of survey research, this definition raises two main points. First, a survey item or scale is not "valid" in and of itself. Rather, you need to consider the purpose and context of the survey items to determine whether or not the inferences you have made are appropriate.
Second, using a particular survey item may be valid for one interpretive purpose but not for another. Don't assume that just because an item was valid in one case, that it will be valid any time you want to use it. Take, as example, a survey designed to measure employee perceptions of the amount of communication in the organization. If you validated the items by demonstrating that resulting data is related to objective measures, then the survey would be "valid" for measuring the amount of communication. However, these same items may not be appropriate for measuring other communication issues such as quality of communication or satisfaction with communication.
It is important to note that it is common to refer to a 'valid' item on a survey. This indicates that inferences based on the item will be valid in that particular interpretive context.
Types of Validity
Everyone agrees that validity is important, but how do you assess validity? Researchers talk about three main types of validity: content, criterion-related, and construct validity. All the different approaches provide evidence about the appropriateness of the inferences made about items.
Content Validity
Content validity determines if the survey items are representative of the topic being measured. As a first step, you must clearly state what you are interested in measuring, then you must judge whether the items are representative of the topic.
In A Nutshell:
- Define what you are interested in measuring, for example 'Quality.'
- Choose the specific aspects which require feedback, for example, 'Error Rate.'
- Judge whether your items relate to the definitions you developed and adequately cover all aspects.
Criterion-Related Validity
Criterion-related validation relies on statistical analyses rather than judgments as in content validation. Criterion-related validation involves calculating a 'validity coefficient' by correlating the survey items with another measure (criteria) already known to be related to other aspects of the attribute. For example, if satisfaction with the service department relates to the number of friends one refers to the service department, then we could correlate scores on a measure of satisfaction to an index of referrals.
Criterion measures are often complex and it is difficult to ensure they cover the scope of the survey; therefore, they are often the most difficult part of the process. If the criterion is not well developed, then there is a limit to the confidence you can have in the validity of your items. As a result, take great care to ensure that both the criterion and the items are developed carefully.
In A Nutshell:
- Develop criterion.
- Measure correlation between survey and criterion.
- Construct Validity
Construct Validity
Construct validation attempts to understand what is being measured by examining the relationship between constructs (an abstract idea used as an explanatory concept--such as motivation or satisfaction).
In essence, construct validation concerns theories and the relationship between constructs proposed by a theory. Construct validation involves understanding why items are related by examining the underlying concepts. The process is hypothesizing a relationship then collecting data to test the hypotheses.
In order to use construct validation, you must understand the theoretical relationship of the construct to other constructs—what it should and should not be related to. Develop relationship patterns using the survey and other measures (either other surveys, observations, ratings, or other data), then systematically examine that pattern.
In A Nutshell:
- Determine the construct to be measured, for example, 'Quality.'
- Determine relationship between the construct and other constructs, for example, 'Satisfaction.'
- Examine pattern of relationships.
For more information:
Call: 1-800-447-3269
E-mail: info-ncs@pearson.com
Survey.PearsonAssessments.com
Copyright © 1995-2007, Pearson Education, Inc. or its affiliates. All rights reserved.This document may not be photocopied, reproduced, translated, or converted to any electronic or machine readable form in whole or in part without prior written approval. If portions of this document are quoted in scholarly research, credit must be attributed to Pearson Education, Inc.