Home

Polls: Land lines account for most survey responses but are they representative?

July 5, 2010

When is the last time you called a residential land line?

The way we communicate is changing daily and while these changes are recognized by some polling organizations, they are not being adequately addressed resulting in poll data that may not be representative.  Without confidence in the representativeness of the sample it is not possible to draw meaningful inferences from the poll results.

The polling industry maintains that there is no cause for concern that the ever increasing number of households without a land line telephone may be skewing their results.  I’m not so sure.  The number of households that have only wireless communication devices, e.g. wireless substitution,  continues to accelerate at a rapid pace.  Furthermore, the demographics associated with wireless only households introduces variability that should not be dismissed as inconsequential.

Figure 1. Nearly 1 in 4 households do not have a land line telephone. (Wireless Substitution: Early Release of Estimates From the National Health Interview Survey, July–December 2009, Stephen J. Blumberg, Ph.D., and Julian V. Luke, Division of Health Interview Statistics, National Center for Health Statistics)

Whenever I see polling data I wonder if the organization conducting the poll called cell phones or just land lines.  It makes a difference and the percent of cell phones called is not usually specified in the polling data.  The degree to which wireless substitution skews poll results remains an open issue and one that should not be dismissed when evaluating the representativeness of poll responses.  Inferences and conclusions based on non-representative data are of no value in advancing our understanding of the subject of the poll.

It’s not just that 1 in 4 households do not have a land line at all.  The differences between wireless only households and land line households touch many areas including state of residence, age, and socioeconomic status.  I suspect that these differences may be reflected in your own life and the lives of those around you.  How many of your friends and family do you call on their cell phone versus their land line?    Do usage patterns vary between land lines and cell phones? If you have a land line how likely are you to answers it versus letting it go to voice mail?

Figure 2. State-level comparisons of the percentage of wireless-only households, modeled estimates: United States, 2007. National Health Statistics Reports n Number 14 n March 11, 2009

These differences are challenging and expensive for legitimate polling organizations to control.  Land line calls are free while calling a cell phone incurs a charge, usually paid by the polling organization.  Additional restrictions on the availability of cell phone numbers and restrictions on unsolicited cell phone calls further restrict the sample universe and impact the representativeness of the sample.

This is not a new observation but one that may be overlooked by consumers when they are exposed to polling results.  It is the responsibility of the consumer to be aware of issues affecting the representativeness of poll data.  Given the ubiquitous nature of polls and pseudo-polls there is no substitute for critical thinking when deciding which one are valid and which ones have meaning.

Selected references regarding wireless substitution (cell phone only households) and the impact on polling:

http://pewresearch.org/pubs/1601/assessing-cell-phone-challenge-in-public-opinion-surveys

http://pewresearch.org/pubs/714/the-impact-of–cell-onlys-on-public-opinion-polls

http://pewresearch.org/pubs/901/cell-phones-polling-election-2008

http://www.cdc.gov/news/2007/07/SurveySystem.html

http://www.pollster.com/blogs/cell_phones_and_coverage_bias.php?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+pollster%2Fno

http://www.factcheck.org/askfactcheck/are_polls_skewed_because_many_people_only.html

http://www.cbsnews.com/stories/2008/10/11/politics/uwire/main4514944.shtml

http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201005.pdf

http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201005.htm

Mirror: http://confrontingmediocrity.com/Sbarefoot.nsf/dx/wirelesssubstitution.htm

Advertisements

One Response to “Polls: Land lines account for most survey responses but are they representative?”


  1. […] don’t vote they have undue influence because some digital dialer has connected with them on their landline and asked their opinion and the politicians listen. We should be better at governing ourselves than […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: