[Home Page] [Previous Page]
[Old Roundtable Archives] [New Roundtable]
Posted by Mike Curtis on January 27, 2000 at 15:26:36:
In Reply to: weighting posted by Val Keel on January 27, 2000 at 06:25:59:
You pay a penalty for weighting data, and your weights are very severe. The ratio of your weights 1.944/.056=34.71 is very large. That is, your are applying a weight to one set of your data that is 35 times larger than the one being applied to your other group. To think of it another way - you have two people who completed your questionnaire and you are going to count the first person's data 35 times for every one time you count the second person's data.
The penalty you pay for weighting data is that the level of precision for a given sample size is lower than it would be if the respondents were sampled in an equal probability (random) draw. The more severe the weighting scheme, the bigger the penaly you pay. So how big is big?
You can think in terms of an "effective sample size". If you interview a random sample of 500 respondents, your effective sample size is 500, and your confidence interval around any point estimate is based on a sample size of 500. Using your weighting scheme, your total sample of 500 respondents has an effective sample size of 264 (don't ask me for the formula I used for the calculation). That is, your total sample of 500 respondents only provides the precision of a random sample of 264 people. In effect, you have interviewed and thrown away 236 interviews from your total sample in order to have a 250 sample size in your smaller group. I hope there was a good reason!
If you just needed a group size that was large enough to analyze, a better distribution may have been a total sample size of 300 split 260 vs. 40. If you could have gotten away with analyzing only 40 people in your smaller group, your effective sample size overall would have gone up (to 274) even though you interviewed 200 fewer people! Why? Because your weighting scheme isn't nearly as extreme (about 5.3:1). Definitely something to think about during the sample design phase of any study using non-random sampling procedures.
I also realize you probably had no control over the situation you are now finding yourself in, but I come across this kind of thing so often when it doesn't need to happen that I just wanted to comment on it.
Subject: