Biases in Telephone Surveys in Latin America

There are many modes in which consumer surveys can be conducted: door-to-door face-to-face interviews, telephone interviews, mail surveys, internet surveys, public place intercepts, etc.  In much of Latin America, surveys are conducted by personal interview.  In North America, door-to-door face-to-face interviewing is rarely done anymore due to the high labor costs; instead, many major surveys are done via telephones.  But why is telephone surveys not common in Latin America?

In North America (namely, Canada and the United States), the incidence of telephone ownership is near 100%.  Thus, the coverage of a telephone frame is nearly equivalent to the true population.  As a result, telephone surveys will not suffer from large biases due to incomplete frame coverage.  In Latin America, telephone ownership may be significantly less than 100%, and there may be some large biases from telephone surveys.  The purpose of this article is to empirically demonstrate these biases with actual data.

We will use the 2004 TGI Argentina study for illustration.  This is a survey of 5,154 persons between the ages of 12 to 75 years old interviewed during 2004.  Within this 67.9% said that their households own a telephone.  If telephone ownership is examined by socio-economic level, then we see that the incidences are 92.5 among ABC1, 89.1% among C2, 74.6% among C3, 63.2% among D1 and 48.2% among D2.  Therefore, any telephone survey sample will be skewed towards the affluent section of the population.

From the 2004 TGI Argentina study, we are going to select a list of survey variables and we are going to show the incidences for the total sample and the telephone-owning subsample.  The results are shown in the table below.  All of these varibles are positively correlated with socio-economic level (and hence telephone ownership).  Therefore, the incidences are always higher among the telephone-owning sub-sample than the total sample.  This shows that we cannot use the telephone-owning sub-sample to represent the population as a whole.

One might wonder if there is a way of weighting the telephone sub-sample to the total sample.  We observe these figures:

• Within the total sample, ABC1 is 11.2%, C2 is 10.2%, C3 is 24.5%, D1 is 27.5% and D2 is 26.6%
• Within the telephone sub-sample, ABC1 is 15.3%, C2 is 13.4%, C3 is 26.9%, D1 is 25.6% and D2 is 18.9%.

We can take the results from ABC1 in the telephone sub-sample and weight them down by 11.2/15.3, the resutls from C2 can be weighted by 10.2/13.4 and so on.  When we look at the new weighted socio-economic level distributin of the telephone sub-sample, we will find that it matches that of the total sample, by definition.  Has everything been fixed?  The results from this weighted analysis is shown in the next table.

The weighted numbers are still mostly higher than the total values, although we have reduced the sizes of the differences.  This says that weighting by socio-economic level will not completely eliminate the biases, although it is better than doing no weighting at all.  Is there any more weighting that can be done?  There may be, but we should not expect to completely eliminate the biases for all possible variables.

After all, we know that there is at least one variable for which no amount of weighting will ever correct for bias.  Which variable might that be?  Telephone ownership!  By definition, the telephone sub-sample has 100% incidence.  No amount of weighting is going to change that incidence --- it will always be 100%. 

(posted by Roland Soong, 7/9/2004)