Magazine Measurement in Extreme Environments
Presented by Paul Donato, Roland Soong and Pablo
of Audits & Surveys Worldwide
at the Worldwide Readership Symposium, Berlin, 1995
For so many years, a central debate at these readership conferences has been the
validity of the "Recent Reading" method versus "Through The Book"
method. At the Hong Kong symposium, Dick Lysaker won the best paper for the ARF Gold
Standard work on a method which could be used, in effect, to judge the validity of each of
the two competing methods.
The power of the Gold Standard was that it could produce estimates which were on average within about 5% of a concrete definition of readership --- that is, the reader is observed picking up and opening the pages of a magazine. So, the standard against which we claimed readership should be measured was the observation method. For the most part, we were completely pre-occupied with the question of validity --- that is, about which method was right. The only real environmental contingency that we studied was the differential impact of readership methods on monthlies versus. weeklies.
This paper is about methods, but, in fact, it is more about publishing environments and how they can change the way that we look at methods. In this paper, we will show that validity in one publishing environment not only fails to insure validity in a different environment, but, in fact, may insure invalidity in another environment.
Previous Analysis of Environments
At the San Francisco symposium, Don McGlathery discussed the impact of logo confusion between two similar titles under the condition that one title is much larger than the other in terms of audience. These two magazines appear in the same logo deck. By virtue of the similarity of the logos, there is a certain amount of logo confusion, especially among the less frequent, less knowledgeable readers. The problem would not be so insidious, except for the fact that if the rate of confusion is constant (for example, 10%), then the absolute number of mis-screens falling incorrectly to the smaller magazine is much larger than the absolute number of mis-screens falling to the larger magazine.
The point of raising this issue at this time is that it illustrates the impact of publishing environment and how this may be at least as important as respondent issues such as memory loss, burden telescoping and the like.
In describing a publishing environment, we include those factors that characterized the nature of the publishing landscape rather than just the respondent or methodology. These factors might include: circulation, distribution, cost and household size.
· Circulation - relative to the size of the population (per capita
circulation), and number of issues in concurrent circulation,
· Structure of Distribution - % Subscriber vs. % Newsstand,
· Relative Cost of the Magazine - indexed to the median incomes of the projected populations, and
· Household Size - or, specifically, the number of persons available to be part of a primary audience given the fact that the copy was obtained by a household member.
To a great extent, average issue readership (AIR) has developed or evolved within a publishing environment which is assumed to be relatively homogeneous in character across space and time. We do not talk a lot about differences in the environmental factors listed above. However, there are significant differences on these factors among different regions of the world. Yet, because the evolution of readership techniques has been very dependent on the consensus expressed at these symposia, we have not had the benefit of watching how very different but isolated publishing environments can logically lead to the adoption of very different measurement methods --- in the same way that ecological isolation can lead to very different forms on an archipelago.
With the exception of Brazil, the development of readership methodological research in Latin America has been very isolated. However, in 1994, a consortium of the largest international advertising agencies, cable networks and magazines funded the first annual multimedia planning survey of the 19 countries of Latin America called Los Medios y Mercados de LatinoAmerica.
Previous regional studies have tended to be done in large cities in the larger countries. In the study conducted by Audits & Surveys Worldwide, the universe covered approximately 90% of the population between the ages of 12 and 64. In total, this projected to 273 million out of the 450 million people of all ages of Latin America.
The readership methodology reflected the consensus of a committee of sponsors who believed that a sample size of under 6000 might have a very hard time documenting the existence of any (or almost any) readership, given the fact that the per capita circulation figures in all of Latin America of the measured magazines are small.
The largest magazine to be measured in Latin America had a total circulation of 1,111,000, which represents a per capita (measured) circulation 0.4%. The smallest magazine in the study had a circulation of 11,000 or a measured per capita circulation of 0.004%.
The relatively small circulation figures of these magazines can be appreciated when compared to the range of per capita circulation figures typically measured by the U.S. syndicated services. In truth, the largest national magazines in a Latin American country have larger per capita circulation figures within that country, but we were principally interested in the regional environment. As you can see, the largest regional magazine in Los Medios y Mercados de Latinoamerica was only slightly larger than the smallest magazines measured as part of the U.S. syndicated services.
In view of these low per capita circulation figures, our natural fear was that we would not find many readers with a sample of under 6000. As it turned out, we totally underestimated the differences in the publishing environment of Latin America when compared with the North American and European conditions. These differences in publishing environment interacted with the methods that were mainly developed and tested in North America and Europe.
We implemented a standard methodology --- namely, Recent Reading, with a 12 month screen and then asking "When was the last time that you read any issue of (NAME OF MAGAZINE)?" --- for 30 magazines on a regional basis.
We and everyone in the readership community understand many of the
characteristics of Recent Reading as it performs in the North American and European
environments. We did not know the questions that it would raise in Latin America, not just
about its performance in the Latin American environment. Latin America would serve as a
stress test or proving ground --- an extreme environment that would illustrate weaknesses
of a methodology which would not be apparent in the environment in which the methodology
was originally designed and usually tested within.
While there are many other factors that might affect the performance of any methodology in Latin America, the following factors
· Circulation Size
· Distribution Structure
· Relative Cost
· Household Size
would show most clearly how Recent Reading makes assumptions about the publishing environment and may perform poorly if these assumptions are violated.
1. Circulation Size
We mentioned before that Don McGlatherys analysis of title confusion was one of the first treatments of the publishing environment, despite the fact that Don focused on respondent limitations and method. Its relevance is found in the fact that there could be two parallel explanations for the phenomena that Don observed. One is that as circulation becomes smaller and approaches zero, the Readers-Per-Copy (RPC) estimate for such a publication becomes very large. The easiest way to understand this is to imagine the inclusion of a non-existent logo into the deck. Will anyone screen into this logo? Some studies have suggested that item response error (that is, answering a "Yes/No" question incorrectly) averages about 5%. That is, for any given question, about 5% of the respondents will answer incorrectly. This is not too insidious when random errors can cancel each other out. However, as circulation approaches zero, underclaiming does not occur and, even if one person overclaims, the result can be over 1,500 readers per copy.
Sounds ridiculous? Things never get that extreme. Popular Hot Rod --- the major beneficiary of title confusion in Don McGlatherys paper --- had a 0.1% per capita circulation and yielded 16 readers per copy. In Los Medios y Mercados de LatinoAmerica, the smallest magazine has a 0.004% per capita circulation. This means that the possibility and potential impact of overclaiming is much higher in Latin America.
The amazing thing is not that you can produce RPC estimates of 25 or above. The amazing thing is that these estimates do not actually approach infinity for the smallest magazine in a study where the average respondent weight is four times the circulation of the smallest magazine.
What is more amazing is the fact that the readership question --- "when was
the last time that you read any issue of (screened magazine)?" --- did in fact
eliminate so many of the screens. This fact is illustrated in figure 4 where it can be
seen that the smaller circulation weekly magazines did in fact realize much higher screen
levels, only to be reduced by the readership question to yield much lower Read-to-Screen
ratios than the higher circulation monthly regional magazines. The short of it is that,
you do have almost 5% overclaiming during the screen --- the generally expected level of
item response error --- for the very low circulation weeklies (usually less than 0.02% per
capita circulation), but you find more traditional Read-to-Screen levels for the higher
The impact of size, quite apart from title confusion and all other environmental factors, starts to become clearer when we examine the RPC characteristics of the larger national magazines, each of which was measured in their home country. Of course, the fewer copies available, the more passalong readership occurs. Also, title confusion could account for some of the differences. However, the mathematics of a unidirectional item response bias --- that is, overclaiming is possible while underclaiming is almost impossible.
The critical mass nature of this phenomena is supported by the fact that
circulation figures beyond 0.5% of the population all yield what we would consider
"reasonable" RPCs. The scatterplot implies that, below this level, there
is a critical threshold of per capita circulation at which the RPC increases
It is important to remember that these magazines are mostly general interest magazine publications. It is easier to compare these results than those in some of the U.S. and European studies, where the smallest publications are the hobbyist or vertical publications.
Ordinarily, we would be encouraged to fit a function and more precisely define the critical threshold below which the RPC takes off, except for a very significant confounding factor. In Latin America, as in many emerging regions, the concept that a single issue is in active circulation at any one time is simply incorrect.
By virtue of their international nature, regional magazines have a special appeal in Latin America as vehicles of fashion, news and even as aids to learn English. It is not at all uncommon to find multiple issues of a regional publication on sale --- new and unused --- at a newsstand. The accompanying photograph shows a Mexico City stand selling 3 different issues of PC Magazine at the same time.
We do not have hard data on the degree to which multiple issues are sold. It
would require a full audit of newsstands in Latin America to estimate the true size of the
concurrent circulation. However, our experience in these markets has been that one can
find back issues for sale in about one of six newsstands in many major Latin American
cities. This suggests a reversal of how audience estimates and circulation figures are
used to validate each other. In many places in the world, the circulation is essentially
used to validate the logic of readership estimates. But regarding regional magazines in
Latin America, readership levels are used to generate questions about how older issues are
kept in active circulation through traditional means such as passalong and non-traditional
means such as continued newsstand sales.
In short, the methodological committee responsible for developing and modifying next years survey believed that the readership estimates generated by a twelve-month screen and last-time-read variant of Recent Reading yield much higher RPC levels than in other publishing environments, because as per capita circulation approaches zero, overclaiming begins to dominated the resulting estimate, as underclaiming has no significant impact. Additionally, we were aware that the level of concomitant active circulation --- especially for regional magazines --- is much higher than the published single issue circulation but we cannot accurately quantify this concomitant circulation.
2. Structure of Distribution
At the beginning of the study, the methodological committee believed that there was a real likelihood of not finding readers to the measured publications. As a result, the question format was designed to minimize respondent burden in every way. As such, the only questions which were asked were the 12 month screen, Last Time Read and overall opinion for each magazine read. Without frequency data, it was difficult to demonstrate the impact of what is perhaps the most important feature of the Latin American publishing environment --- the general fact that 80 to 90% of all magazine sales are via newsstands.
This should not be surprising given the failure to develop effective national postal systems in many Latin American countries. The only viable way to operate subscriber fulfillment is to use private delivery services, which are obviously very expensive. Additionally, there was a strong disincentive to promote magazine subscriptions during the years of hyperinflation, as the circulation departments had to struggle with complex models of future costs and revenues.
Structure of Magazine Distribution
· NORTH AMERICA
Approximately 80% Subscriber
Frequency (3+ out of 4)
· LATIN AMERICA
Approximately 80% Newsstand
Frequency (1-2 out of 4)
This structure of distribution in Latin America is the opposite of the structure in North America. For the regional magazines measured in this study, their North American circulation figures --- generally their home markets ---average approximately 80% subscriber.
In last years study, we did not collect the source of copy and the place
of reading, but we will be doing it this year. Had we done so, we could have analyzed
frequency by subscriber (even with the limitations of the source questions). However, from
other sources, we know that the number of issues obtained is theoretically 4 out of 4 for
subscribing households and approximately 2 out of 4 for newsstand sales in the United
States. In Latin America, according to the publisher Editorial America, the costs of
magazines are such that frequency of purchase on the newsstand is less or about 1.5 out of
2.b Other Factors Which Interact With Distribution
There are two more factors that make Latin Americas publishing environment so unique. These factors interact with the predominantly newsstand distribution structure. The factors are the relative cost of each magazine and the household size. Both factors have been discussed in previous symposia, but the authors never would have had a reason to address the extreme nature of these factors in Latin American. Since much of what we produced has been compared to readership characteristics in the United States, we will compare the Latin American environment with the U.S. in order to illustrate how differences in environmental factors could have caused differences in audience estimates.
Household sizes in Latin America are almost twice as large as they are in the United States. This has an immediate impact on primary readership in its own right. This fact materialized in our estimates of newspaper readership --- which, as in the rest of the world, appears to center on a RPC estimate which is five to ten percent higher than the average household size.
The second factor that interacts with distribution is the cost of single issue copies in Latin America. For the most part, the cover prices of regional magazines in the study were the same as they had been in North America. In some cases, the cover prices were some 20 to 30% more in Latin America. Regardless of the actual price, the real household incomes in Latin America --- measured in U.S. dollars --- are some one-sixth what they are in the United States. The impact of this is to drive up the effective price of magazines to a level of six to eight times the cost in the North American environment.
So, the general Latin American publishing environment is one where magazines are generally purchased on the newsstand, at a cover price which by Latin American income standards is equivalent to the cost of a book and then brought home to a household which is almost twice as large of the United States.
Together, these factors should interact (potentially under the influence of a multiplier effect) to radically affect the assumed balance between replicated and parallel reading.
The logic is simple but cannot be tested properly in Latin America until data about the frequency of reading and the source of copy are collected. The argument is as follows:
· In a publishing environment which is 80% newsstand, the frequency of purchase and therefore reading must be significantly lower. In Latin America, this has been estimated through custom studies to be 1.5 out of 4.
· In an environment in which sales are 80% subscription, the frequency of purchase is theoretically almost 4 out of 4 and therefore the frequency of reading must be significantly higher.
· Many studies have suggested that replicated reading is at least of the same order of magnitude as parallel reading (Landgrebe in San Francisco 1993 showed evidence that suggests replicated exceeds parallel reading by 20%)
· If the assumed equivalence holds under a subscriber environment, it cannot hold in a newsstand environment where the frequency of purchase is less than half as much and magazines cost six times as much.
· Therefore, the validity of the assumed equivalence (replicated = parallel) in one environment almost assures its invalidity in the other.
Fundamentally, the argument suggests that if people buy half as many copies at six times the real price, they are much more likely to retain the copy for a longer period of time. Whatever effect this has, it operates under a household size multiplier which magnifies the impact.
Contrast with Local Methodologies
The findings of Los Medios y Mercados de LatinoAmerica are not the first opportunity to look at the performance of Recent Reading in this environment. MARPLAN currently uses Recent Reading in Brazil but projects to the population of approximately 10 markets which would make this diagnostic analysis difficult to conduct. IPSA Argentina tried a form of Recent Reading in Argentina several years ago, at which time the method also yielded an average of 15 to 20 readers per copy, even for the larger circulation national magazines. Roper conducted a pilot in four countries about ten years ago and, even with the limited sample used, it was clear that the estimates were of the same magnitude.
In the opening section of this paper, we discussed how sometimes publishing environments, if isolated, might yield their own forms of audience estimation --- perhaps more suited to the local environment than those methods developed in very different environments. To a great extent, this is what has happened in Latin America.
Having tried recent reading, IPSA dropped it in favor of the most widely used measure of readership in practice --- the two-issue cume ("Do you read magazines?" IF YES, "Have you read either of the last two issues of (NAME OF MAGAZINE)?"). The screen is a wholesale eliminator of two-thirds of the respondents who do not generally read magazines. The readership question does nothing more than reflect the attitude about the definition of a current issue --- "Have you read either of the last two issues?" --- and makes an explicit attempt to filter out anything older that the last two issues. It should be recalled that there may be multiple copies available on many newsstands. In Mexico, Argentina and several other countries, the method produces readers per active copies of about 10. Consciously or unconsciously, the adoption of this method explicitly rejects the assumption that, in some environments,
The readership of an average issue
is equal to
the readership of any issue in a one issue period.