This is Bill. Bill wants to hire a smart young person to work for his business. The problem is that he finds it hard to gauge people’s intelligence. First, he found a kid with the biggest, thickest eyeglasses he could find, but he was shocked to find out that he wasn’t so smart at all. It turned out that the kid was just nearsighted.
So then he hired a kid who graduated from Yale. The kid had a C average, sure, but he went to Yale. That turned out disappointing too. It turned out that his dad was a U.S. senator and a major donor. Oh well.
Then he found someone with a perfect GPA and SAT score. This person was smart in terms of being able to do math problems or regurgitate whatever she was reading, but she wasn’t the type to think on her feet, to think outside of the box, or to use the street smarts type of intelligence that Bill had in mind when he was looking for someone who he thought was smart.
Bill’s problem was that he was using bad signals to guess people’s intelligence. He was using measures with validity problems. Validity means that you’re measuring what you’re trying to measure in your research project. In this video, I’ll describe some often encountered validity problems. If you know what they are, then you can be cognizant about them in your research design, and hopefully, you’ll avoid creating and fielding surveys with invalid measures.
I’ll talk about four types of validity: face, content, criterion, and construct. I’ll explain what they are and how to assess your measures in light of these four criteria.
Face validity is the most straightforward validity question that you can ask about a question or a measure. Basically, face validity asks, “Does the measure make sense?” Does it make sense that the question you’re asking will operationalize the concept that you’re trying to measure?
Content validity is concerned with whether or not you are measuring a more complicated concept with enough measures to capture its full meaning. We have content validity problems when we take a very complex concept and use a measure that only captures a small slice of what that concept intends to mean.
Criterion validity is when we take outside measures that we know to be related to the concept that we’re trying to operationalize. If our measures are good, then they should be related to those outside measures.
Finally, there’s construct validity. Construct validity is something to think about when we’re using multiple questions or multiple variables to operationalize the same concept. If you have several variables all measuring the same thing, they should be related. These variables should also be somewhat distinct from other measures that aren’t measuring the same underlying concept.
In conclusion, there are four types of validity: face, content, criterion, and construct. When you’re developing a measurement scheme, take a moment to think about these different types of validity and whether or not the measures that you’re using are adequately operationalizing the concepts that you’re trying to study.