Monday, February 22

Hamburgers and sampling bias

I ate lunch at Harvey's yesterday. The last time I ate there was about three years ago, when their chocolate milkshake made me quite sick. But I got hungry while shopping at the Home Depot, and since there was a Harvey's right in the store, I figured I'd give them another chance. Thankfully, I did not get sick this time (I avoided the chocolate milkshake), but the meal was still pretty disappointing and the service was slow.

While I was eating, I noticed a request for feedback on the receipt. You can complete a survey by going online or calling a 1-800 number. There was an incentive for completing the survey: I'd be entered in a bi-weekly draw for a $500 Harvey's gift certificate.

I considered completing the survey (I like surveys), but the more I thought about the incentive, the more I thought it wasn't worth my time. The thought of winning a gift card that could buy me about 250 more Harvey's hamburgers just didn't sound that appetizing. I could have tried reselling the gift card if I won, but that would have involved a lot of effort in finding a buyer on Craigslist or eBay and shipping them the gift card. I'm sure I wouldn't get anywhere near $500 for the card.

The Harvey's survey is a perfect example of how sampling bias can screw up a survey. If you don't like Harvey's, you have very little incentive to complete their survey. So the responses Harvey's gets to their survey will probably reflect a more favourable view of their restaurants than if they had a proper random sample.

I know Harvey's isn't the only restaurant to do this. Swiss Chalet offers you a free appetizer on your next order when you complete their survey. I complete it every time (usually with very favourable responses) because I love their chicken noodle soup. But if you had a bad experience and hate their appetizers, you're not going to complete the survey.

I wonder if restaurants who conduct surveys with food incentives are actually interested in honest feedback. If so, I wonder why don't they offer cash incentives instead.