Without question, in order to understand effective social science research (or any kind of scientific research), you have to understand the concepts of "validity" and "reliability."
To put it simply, "validity" refers to whether a measure actually measures what you purport to be measuring.
For example, if you create a concept called "television use" and then decide to measure it by asking people how many TVs they own, that MIGHT be an indicator of how much TV they watch, but you definitely have some validity problems, right? Why? (post your thoughts as a comment to this entry).
It would probably be better to measure the concept by asking people how many hours of TV per day they watch, on average (or better still, you might have them go hour by hour thinking only of yesterday and to report what they watched during the day).
I'm sure you can see that using a measurement like this is better than just asking how many TVs someone owns...
Reliability, meanwhile, simply refers to how often you can repeat the measurement and get the same result.
Let's take an example and put the two concepts together--
Suppose you have a digital bathroom scale and you step up on it and weigh yourself and it reads "145 lbs."
Now let's suppose you repeat that process ten times in a row and you get results like this--
1. 145
2. 144
3. 144.5
4. 145
5. 145.1
6. 144.5
7. 144.8
8. 145.2
9. 145
10. 144
Acting reasonably, we should see these results and say "this scale has come pretty close to giving me the same reading 10 times in a row, so I conclude it's reliable." If so, its "reliability" is strong and is not in question.
What we do not know, however, is if the scale is RIGHT. What if it's wrong by, say, 10 lbs. and you REALLy weigh closer to 155 lbs.?
The scale's readings are reliable, but we can't say for sure if the scale is valid.
To confirm that the scale really is measuring pounds, we might "test" it by weighing other items whose weights we already know. For example, a 10 lb. bag of potatoes, a weight (from a weight room) of 25 lbs., a 50 lb. bag of rock salt, the official rod of steel used as the standard to determine a "pound" and so on.
Now, if we weigh all those items and each time the scale gives us readings that are really close to what we should expect, we can conclude that the scale is indeed valid.
Get it?
Let's practice. Below are several sample ways of measuring certain media concepts. Indicate to me, as comments, whether you think the measurements are valid (valid or NOT valid) and why or why not. In each case, one measurement is more valid than the other. Identify the valid measurement and indicate why the other is NOT valid.
Concept Measurement
media use
A) how much time you spend reading a daily newspaper
B) combination of how much time you spend on all media
exposure to TV violence
A) how many hours of TV you watch each day
B) how many hours of TV you watch each day broken up by TYPE of tv show watched (coded for content)
internet use
A) how many times a day you check Facebook
B) how many hours per day you spend online via computer or other handheld device
You may be thinking to yourself-- "this is pretty obvious stuff," but you'd be surprised at the number of reports professionals put together that suffer from problems of validity.
To answer the earlier inquiry, I believe the reason why the TV question would have validity problems is because owning a lot of TVs doesn't mean that you necessarily watch them. Some people just like to hoard things and may not have any of the TVs hooked up at all. Therefore the television usage aiming to be measured would not be at play. The data would not really answer the intended question.
ReplyDeleteFor concept measurement questions:
1A: NOT VALID. While newspapers are a form of media, they are not the best way of measuring what "media use" entails. While some further specifications might be helpful, as it stands. The second option is much more valid.
1B: VALID. I would say this option is more valid because it encompasses all the various forms of media that someone might use in a given day and quantifies it. This is a more appropriate form of measuring the concept.
2A: NOT VALID. While the option might give interesting data, it neglects to answer the purposed concept in a meaningful manner. Instead, it measures TV consumption instead of consumption with regard to violence as the topic suggests.
2B: VALID. Because this option specifies what type of TV the viewer is watching, it is easier to understand how that exposure relates to violence. Without the specificity, you can't determine if the content is violent or not.
3A: NOT VALID. Internet usage encompasses more than just social media, like Facebook. Therefore this doesn't really measure what it is supposed to.
3B: VALID. This option broadens its scope to include the various platforms and websites someone might visit when using the Internet. Therefore it more adequately measure Internet usage than the other option.