Government Statistics
Government Statistics
Organization of statistical services
Statistical data generated by government sources are referred to and cited in many articles in this encyclopedia, including Census; Economic data; Fallacies, Statistical; International trade; Sample surveys; Vital statistics; and the biographies of Queteletand Willcox.This article is confined to over-all aspects: the relation of statistics to the establishment of national states; the range of substantive matters on which statistics are collected; the ways in which that collection is organized in different parts of the world; the business and professional environment within which gov ernment statisticians work; the problems and diffi culties they face in both developed and underdeveloped countries.
Government statistics in the modern state are an essential part of a wider information system. The first compilers of statistics did not make a sharp distinction between numerical and other facts, and it is still true that numerical data complement other kinds of information in the process by which decisions are made in private and public undertakings. These data are as indispensable in a centralized as in a pluralistic society; as important in the deliberations of government itself as they are in the firms, commissariats, or other groupings concerned with getting out a product. The effectiveness of the market mechanism of a free society depends on the quality of the information on which its entities base their decisions. Western European economic planning consists in large measure in the provision of supplementary information to private enterprise. A tightly controlled, centralized system needs, more than anything else, a feedback—of which the statistics it collects are a pivotal element—if it is not to be the unwitting victim of its own concentration of power.
Statistics and the state
The history, the present condition, and the problems yet to be solved in government statistics reflect the circumstances of modern national states. A governmental administrative apparatus is a principal user of official statistics, and its existence and efficiency are a main condition for securing them. While sporadic attempts at counting people or goods were known in classical times and in the ancient empires of the Middle East, there is virtually no continuity between these and modern official compilations.
Modern history . Like so much else that pertains to the modern state, its statistical system emerged about the time of the French Revolution. One of the early acts of the Constituent Assembly was to see to the publication of a statistical account of the resources of France, prepared by Lavoisier. In 1800 the Bureau de Statistique was created in Paris. The assembly required a census (Articles I and II of the Law of July 22, 1791, according to Faure 1918, p. 277), which was duly taken in 1791. François de Neufchâteau, in a circular dated the 15th of Fructidor of the year VI (presumably September 1, 1799), saw the census—with characteristic revolutionary exaggeration—as “the measure of the strength, the source of the wealth, the political thermometer of the power of states” (Faure 1918, p. 284).
Just as British nationality erupted less violently than did that of France, so its statistical system had a more gradual inception. Early landmarks were the Domesday Book of William the Conqueror in 1086; the record of customs dues collected in the Port of London in the time of Edward III the Tudor counts of men and resources in the face of the danger of war; and the registration of deaths, initiated by Henry VIII in 1532, when there was widespread fear of the plague.
The dependence of statehood on a statistical system has become more and more clear with the passing of time. The mercantilist writers of the epoch of the absolute monarchs were concerned with the power of the state in peace and war; the monarch and his advisers had to have measures of the stock of men and other resources [seeEconomic thought, article onmercantilist thought]. The British and French colonial administrations in North America, and later the British in India, took censuses, that of the Canadian province of Quebec in 1666 being the first of modern censuses (Linder 1959, p. 330). With democracy there came to be other reasons for statistical compilations. In the first of the major federal constitutions, that of the United States, a means of determining the political weight of the contracting entities was required; their relative populations seemed to provide this, and the first of the regular series of decennial censuses of the United States was taken in 1790, although a printed schedule was not employed until 1830. Confederation in Canada in 1867 was based on a similar provision for representation by population and was followed by a census in 1871; the Australian colonies were united in 1900, and the formation of a statistical office followed in 1905.
Since World War II dynastic, colonial, tribal, and other political forms have been displaced by national organizations throughout the world. Along with attempts to provide themselves with constitutions, elections, and the beginnings of modern industry, new countries have set up statistical systems, both as a precondition for development and as a symbol of nationhood. The new statistical style is far more ambitious than the colonial model that preceded, in proportion as over-all national aims are more extended than colonial aims. An important role in promoting the extension and quality of statistical work was played by the League of Nations, through its Committee of Statistical Experts, and subsequently by the United Nations, through its statistical and population commissions. Under UN auspices a world population census was attempted about 1950 and again in 1960, with a high degree of national cooperation. In the decade of the 1860s censuses were taken in which 17 per cent of the estimated population of the planet was counted; over ten times as many people were counted in censuses around 1960, and these constituted 67 per cent of the population of the globe (Demographic Yearbook 1962, p. 1). Results are not uniformly satisfactory; a study of the quality of statistical organization and statistical output of the various countries would undoubtedly show a close relation to the quality of governmental administration in general. Chile is better organized statistically than Burma; Burma than Cambodia.
Government statistics classified
A wide range of statistical data has come to be regarded as appropriate for government compilation and publication.
Classification by source. One way to classify government statistics is according to source: households for population censuses, family-budget surveys, employment and unemployment counts; business establishments for production and employment data; incorporated and unincorporated firms for profit figures; national-government revenue departments for foreign trade and income tax accounts. The categories are not exclusive; households and commercial establishments often provide complementary data bearing on the same matter.
Where economic statistics are essentially a summary record of transactions, they are in principle available through questions addressed to either of the parties to the transaction; retail prices may be ascertained through surveys of retail stores with respect to goods sold or through household surveys with respect to goods bought. If the business establishment is more often used, this is a matter of the greater availability of records and the larger number of transactions implicitly covered by one report; in underdeveloped countries, where business is less organized, questions addressed to the consumer are relatively more favored. For data on employment and unemployment, the household seems on the whole as satisfactory a source as the business establishment. In North America the contest between rival figures of unemployment has been an impor tant stimulus to improvement not only of the sampling and questioning techniques used in household surveys but also of the administration of unemployment insurance itself, in respects that go far beyond statistics.
Classification by time of publication. A grosser form of classification is by the indication of temporal trends that series give. Some are issued promptly and tell the latest news: weekly railway carloadings, department-store sales, stocks of wheat in central elevators, stocks of the several metals. At the other extreme are full censuses of industry and population, taken only at long intervals; their results are released over a period of time, beginning within a few months after enumeration and continuing for some years after the data are collected, and they are valuable for the cross-sectional relations that they reveal. Because of this configuration of prompt summary data and delayed details, the national economic picture of a country with respect to any moment of time is only gradually filled in, over the five or ten years subsequent to that time.
Government statistical activities include analysis and interpretation as well as data collection. Although knowledge of analysis helps in gathering data, and vice versa, the two specializations are different. An example of the division of labor on this basis is that existing between the United States Bureau of the Census and the Department of Labor on statistics of employment and unemployment; the former agency has the responsibility for gathering the material, the latter for the official analysis.
Expansion in modernization
A statistical system seems to start with population censuses and foreign trade as the main items; other kinds of data—for example, counts of starts and completions of residential construction—are added as time goes on. As a country develops, the need for statistics mounts with the variety and difficulty of decisions required by an increased division of labor in the economy. A demand for factual justification, in terms of which people can explain their decisions to themselves and to others, seems to have characterized the North American continent from an early date. General Francis A. Walker, speaking before the International Statistical Institute, point ed this out in 1893: “A strong passion for statistics early developed itself in the life of our people, and such statesmen and publicists as Hamilton …became working statisticians. … No government in the world has ever lavished money and labor …more cheerfully and patiently in this respect” (quoted in Cummings 1918, p. 573). But this culture of facts is no longer confined to the United States, to those of English speech, or to Europeans; it is becoming world-wide.
The proposition that the statistics collected by any government are a function of its interests and responsibilities can be exemplified in the process of decolonization. When much of the world was under the hegemony of the states of western Europe, the statistical system centered on foreign trade. What was important for British administrators was the amount of rice exported from Burma, of jute exported from Bengal; the Netherlands wanted to know the amounts of coffee, sugar, and rubber exported from Java in the periods when each of these commodities was at its height. Another phase of colonial development was the land tax; the land tax made it both necessary and possible to have figures on acreages and production of the main crops. A further stage was some rudimentary concern with people, expressed by population censuses. The building of railways in India and Java, for instance, was followed by the collection of transport statistics—data that no one would have collected when oxcarts, proas, and the backs of men were the principal means of shipping commodities. And the evolution that occurred in the colonies in Asia was paralleled in Latin America, where the several governments, although independent, had interests nearly as restricted as those of the colonial powers.
The advent of independence in Asia and of the welfare state everywhere has increased the range of government statistics. Every country concerned with development is trying to expand its educational system, and statistics of schools, teachers, and pupils are nearly universal. In England and Wales educational statistics of a kind date back to the 1820s and 1830s, when public money was first given to the schools (Baines 1918, p. 377). Vital statistics are being gradually improved, a process that requires the inculcation of the habit of recording births and deaths not only on the part of the hierarchy of the civil registration system but also on the part of the medical profession and the public at large; such institutionalization will take at least a generation or two in the new countries. Mean while, sample surveys are being introduced, to ob serve the birth rate and its changes, as well as many other population and economic phenomena. Family-budget surveys, which had been foreshadowed by colonial governments in their moments of welfare-mindedness (a coolie-budget survey was made in the Netherlands Indies in the 1930s), are becoming more frequent; the government of India, through the Indian Statistical Institute in Calcutta, has been particularly active in this and other types of household survey.
Organization of statistical services
Every government has sooner or later to consider the organizational framework within which its statistics will be collected. When left to themselves, individual departments of the central government tend to collect whatever data their administration generates, and they hardly separate the collection of such information from administrative control of their operations. When the act regulating the collection of duties in the United States was passed in 1789 (Cummings 1918, p. 579), it required the collector of customs to record ships’ manifests and other information connected with trade; the series of foreign trade statistics for the United States accordingly dates from 1789. The British Post Office collects and publishes the returns of the postal service; the British Department of Inland Revenues, established in 1849, collects the estate and stamp duties, land taxes, and income taxes. But with increase of scale, such departments come to separate their statistical from their administrative work; the British Board of Trade formed within itself a statistical branch as early as 1833 (Baines 1918, p. 374). Subsequently, labor, local departments of agriculture, and many other departments or boards came to have their own statistical units, not only in the United Kingdom but in the United States, the British dominions, and the countries of western Europe. The Interstate Commerce Commission (ICC) of the United States was from the start insistent on securing data from the railways it regulated, and ICC publications constitute an important series.
Sometimes one of these agencies would be as signed the taking of a decennial census, a periodic task required of the registrar-general in England and Wales, whose continuing work was the civil register and vital statistics. In Canada the decennial census provided for in the British North America Act was the charge of an office established in the Department of Agriculture in 1905. But in all countries, increasing attention to statistics led to proposals for centralization in a specialized agency.
Degree of centralization
With the establishment of even small statistical offices within gov ernment departments, the work of statistics began to benefit from some degree of separation from the day-to-day exigencies of administration.
Centralization in national government. The collection into a single office of all the statistical activities of the national government was considered, in order to secure even more of the gains of specialization. To the argument of efficiency (concentration of specialized personnel and equipment to deal with the large scale of centralized statistical work) was added that of avoidance of duplication, a perennial hazard of government work and one which appeared in statistical work from the earliest days. Separating statistical work from the other work of government lessens the pressure on the statistician to distort his results to protect political or administrative interests—reference is made below to the judicial aspect of the statistician’s function. The better coordination of a central system, moreover, ought to permit the recognition of gaps in data. It should also make easier the adoption of uniform classifications; it is highly desirable that production and exports, for instance, be recorded on the same commodity classification, so that comparisons may be possible. But anyone who knows the tendency to autonomy of the divisions of a governmental organization will realize that having the several compilations under a single roof is no automatic guarantee of uniformity; the several sections can pursue different courses. One special danger of a centralized organization is that it will become so self-contained as to be immune to the needs of the users of its data. Centralization offers high returns when the central organization is well coordinated internally and alert to the changing requirements of outside users.
The administrative argument for centralization was early carried into effect, in one degree or another, in the Netherlands, Canada, Australia, Germany, and Italy. About the middle of the 19th century the Netherlands took important steps in this direction; in 1848 a statistical bureau was established in the Department of Home Affairs whose responsibilities were far wider than the tabulation of data generated in the department. Canada and Australia committed themselves to a statistical system that was centralized in the double sense that statistics were at least as much the affair of the central government as of the states or provinces or of any lower level of government and that among the agencies of the central government there was one with pre-eminence in the collection and tabulation of statistics in a number of different fields. The Canadian system has been in principle entirely centralized since the founding of the Dominion Bureau of Statistics in 1918. A degree of centralization was arranged in Germany with the establishment of the Imperial Statistical Office in 1872, the year after the establishment of the German Empire; in Italy centralization was begun in 1861, the year of the constitution of the monarchy, with a directorate of statistics that was the ancestor of the present Istituto Centrale di Statistica.
In the United States and the United Kingdom practice has evolved closer to the decentralized pole of the centralized-decentralized continuum. The means of coordination in the United Kingdom is an office in the Cabinet Secretariat that operates through a series of understandings with the heads of statistical divisions in the Board of Trade, the Ministry of Food, the Ministry of Agriculture, etc. All the statistical divisions in departments are directed by statistician members of the Professional, Scientific, and Technical Class, one of the five classes of the British civil service; appointments anywhere are open to members of the statistician class irrespective of where they are serving. The Treasury approves the expenditures of statistical departments, and on new expenditures they are advised by the Central Statistical Office.
The American arrangement is more formal; the Office of Statistical Standards of the Bureau of the Budget not only reviews all requests for funds to carry out statistical work within the federal government; its approval is required before any federal government questionnaire can be sent to ten or more respondents. This requirement dates back to the Federal Reports Act of 1942 and was intended to reduce the burden on respondents and to eliminate duplication. Where duplication exists, it not only is wasteful of government funds but also arouses the fiercest resentment of respondents who are required to answer the same questions more than once.
The best coordination does not consist exclusively of the negative injunctions of an enforcement agency. An example of a more positive kind is the collaboration of the U.S. Social Security Administration, essentially an operating agency, and the Bureau of the Census in the creation of a publication showing county business patterns based on Social Security Administration records. The U.S. Bureau of the Census has used tax records to eliminate business-census questionnaires for a million retail establishments without employees. By using Social Security records as a source of lists, the Bureau of the Census is able to take the economic censuses at reduced cost.
Coordination requires fine judgment at a thousand points. Should both the U.S. Bureau of Mines and the Bureau of the Census continue to secure mineral statistics? The Bureau of the Census collects benchmark data on an establishment basis every five years from all establishments within the scope of the census, assigning to the mining industry the output of all those that are classifiable as mining. The Bureau of Mines collects information more frequently, on the output of minerals as well as on engineering and technical matters. These data are thought to be independently valuable and different enough not to seem to constitute duplication.
Division of responsibility. Along quite a different dimension is the division of responsibility between the national government and the local governments of states or provinces and cities. In this dimension the United Kingdom is highly centralized, while Germany has problems because of the division of statistical activities among the Länder, and even in the Netherlands there is some devolution of statistical activities to organizations in the cities.
Every national statistical system must depend on local sources for its information. At the one extreme is complete central control, as found in the national census of countries such as the United States and Canada, in which the local officers are appointed and instructed by the center through a training ladder that inculcates definitions and procedures that are, at least in principle, fully determined centrally, and which is backed by a central budget from which all participants are paid. More decentralized is the census in countries such as Argentina, where the plans are made by a national statistical office but where the budget does not include the funds for paying enumerators, who are government employees on the payroll of other departments of the center or the provinces (for example, schoolteachers), co-opted for one or several days’ work on the enumeration. Coordination of such an operation requires feats of diplomacy. Further down on the scale of decentralization is the system of vital registration, used in the United States and other countries, in which local registry officials report to a state authority and whatever national uniformity exists is the result of negotiations in which the national office can take leadership more by virtue of its persuasiveness than because of any legal or budgetary power.
Comparability and change
The government statistical agency must have both continuity and adaptation of its output to changing needs, a combination of virtues easier to prescribe than to follow. In some instances an agency is very rigid; the disposition to continue collecting the same data in the same fashion year after year is both the strength and the weakness of government agencies. But at other times it changes arbitrarily; A. L. Bowley charged that the United Kingdom census of 1921 was deliberately made noncomparable with anything that went before, a statement challenged by Greenwood et al. (1932, p. 279). Again, judgment is required; the elimination of old series and the initiation of new ones and the modification of definitions that make a given series more useful for the future but lessen its comparability with the past must be discussed on a more specific level than this article can attain.
Practices that are good in one place and time may not be so in a different economy and society. Attempts to transfer categories of statistical compilation from a country in which money economy dominates to one in which households produce for their own use rather than for exchange may not be satisfactory. Some of the greatest difficulties in collecting statistics within the developed countries are in the sector of subsistence agriculture. The relation of statistics to industrial practice is brought home especially strikingly by the present situation in the United States, where the National Bureau of Economic Research and the Bureau of the Census, among other agencies, are trying to find how individual series can be improved or replaced. With the shift of traffic to trucks and airlines, carloadings have become much less useful. Department-store sales mean less when many of the lines of merchandise are also sold by discount houses and drugstores. The inventive spirit that discovered department-store sales and carloadings as key figures in the economy is needed now to go beyond these.
Interactions with environment
The government statistical agency is by no means a free agent but depends on the administrative practices of other governmental agencies and of business concerns. Statistics of imports depend on valuations made for customs purposes; comparing the exports of newsprint or metals from Canada to the United States, for instance, with the imports of nominally the same commodities into the United States from Canada will show how substantial is the effect of varying definitions of commodities as well as of sources and destinations. Since definitions are often part of procedures for customs valuation, and these are embedded in laws around which substantial material interests have developed, discrepancies between countries are likely to persist despite many conventions and attempts at accommodation, from that of Brussels in 1910 to those of the United Nations Statistical Commission.
Effects of accounting practices. One important network in which the statistical agency is enmeshed is the accounting practices of industry, themselves in part the consequence of the regulations of the government income tax department. The art of statistical collection includes finding modes of definition of the entities about which inquiry is made that are most in conformity with the accounting practices of business and provide the most useful information to the public. The indus trial establishment, as against the firm, is generally taken as the primary element for collection of industrial statistics and is defined in practice as the smallest production unit that maintains more or less complete records. The statistical agency does not always play a purely passive role; it is often in a position to exercise influence and initiative. In Canada the federal agency concerned with financial statistics of the 5,000 municipalities of the country was able to persuade them to adopt a standard accounting procedure, which benefited them as well as national statistics.
National professional associations. The environment within which a statistics-producing office of government has to work includes, besides the operating agencies of the same government and the productive units of the country at large, a host of professional associations, as well as individuals interested in statistics. The American Statistical Association in 1844 petitioned the Congress of the United States to recompile the census of 1840; its policy of protesting poor work and supporting good in the federal statistical field has operated to the present day, although the need for its intervention is not as great as when the government service was staffed almost exclusively by amateurs (Bowman 1964).
International professional associations. To the national associations have now been added important international ones. First among these was the International Statistical Institute, from the nineteenth century on a persistent and wholesome influence in favor of high standards and comparable classifications. The Economic and Social Council of the United Nations, through its statistical and population commissions, has had an effect in stimulating countries to take censuses; to adopt a minimum set of questions in these, for which a serious attempt would be made to attain international comparability; to use modern techniques, in the interest of accuracy and promptness of release of data. Sensitive to the desire of nations to be independent of outside interference in statistics as in other fields, the council has made its chief weapon the argument of international comparability in statistics of production, as well as of population. Also of influence have been the regional agencies of the United Nations for Asia, Europe, Latin America, and now Africa; the Inter-American Statistical Association; the specialized agencies of the United Nations, including the United Nations Educational, Scientific, and Cultural Organization (UNESCO), the Food and Agriculture Organization (FAO), the International Labour Organisation (ILO), etc., in their respective fields. The resolutions and the conventions that these have drawn up often suffer from being stated in highly abstract terms, but they are supplemented by technical assistance services to less developed members, including the sending out of advisers and the providing of training facilities. The publications of international bodies have not only been directly useful to scholars and others but also have been an incentive to members to secure data. FAO conferences of groups of neighboring countries have constituted an effective form of pressure for improvement of agricultural statistics.
Other influences. Series of government statistics have often been initiated from the outside. The Metropolitan Life Insurance Company began collecting statistics of labor turnover in the United States in 1926, and these compilations were later taken over by the United States Department of Labor (Hauser & Leonard 1946, p. 363). In Canada statistics of wage rates collected by the Bell Telephone Company for its own purposes have influenced federal government collections. Statistics of stocks on hand have often been started by associations of manufacturers and then continued by a public statistical agency. In the Netherlands one of the influences that led to the centralized system was the private Union for Statistics, founded in 1856 (Stuart 1918, p. 435), which published many volumes of data, although it did no primary collecting. Later this private union was subsidized by the government, and ultimately all of its activities were taken over. Especially well known is the pressure that Quetelet, the Belgian statistician, exerted through his researches as a private citizen between about 1825 and 1841 (when he became a public official). His avid interest in the regularity of certain social phenomena, e.g., the “budget of the scaffold,” finds its monument in the extensive series of criminal statistics produced in Belgium and many other countries (Julin 1918, p. 128). In England the first edition of the essay by Malthus that attracted so much attention to population was followed within three years by the census of 1801 [see the biographies ofQuetelet and Malthus].
There are many other examples of the influence of men and ideas—as well as outside organizations —on government statistics. The collection of economic data during the present generation has been altered by the recognition of national accounts as a general framework. The concept was developed by scholars in universities in the United States and Great Britain, following suggestions of J. M. Keynes. The notion of national income makes it possible to arrange a great variety of existing series in the pigeonholes of a national-accounts framework; the output of the statistical systems ceases to appear arbitrary, for each portion measures a contribution to the gross national product; certain gaps become visible—for instance, personal services—and there is pressure to fill them. Furthermore, the examination in this fashion of economic statistics as a whole permits the grand total of the country’s economic activities to be calculated in at least three ways—by value added, by income received, and by expenditure—which have important elements of independence as to source of data. The confrontation of calculations of a total derived from more or less independent statistical sources has been an incentive to improve the accuracy of all components.
The current prestige of economic planning has had important effects on government statistical work. Planning may be good or bad, effective or ineffective, but in all instances it requires data. One cannot even go through the motions of planning without statistics. Government planning agencies requiring data are in an especially strong position to see that the necessary resources are allocated to their collection.
The statistical profession and government
Underlying recent developments is a rapid professionalization of statistics. Times have changed greatly since Francis A. Walker lamented, in an address before the American Statistical Association in 1896, “I do not know of a single man now holding, or who has ever held, a position in this country as the head of a statistical bureau, or as chief of a statistical service, or as a statistician, who had any elementary training for his work” (quoted in Cummings 1918, p. 574). A recent Bureau of the Census survey of its staff showed 576 employees in professional statistical positions, of whom all but 22 had an academic degree, including 36 ph.d.s and 102 m.a.s (Bowman 1964, p. 14). The change has come about through the extension of the field. Statistical method is applied to acceptance sampling and quality control in industry, to experimental design in agricultural trials, to bio-assay, operations research, and sample surveys of national populations. The interchange among these applications and between each application and a rapidly expanding mathematical statistical theory has had a decisive effect. Backed by a theory based on probability and relatively well-established methods of application of the theory; with a rapidly growing body of literature in books and professional journals; with departments of statistics in a number of universities, and courses in nearly all; there is less and less need for the government statistical agency to depend on the gifted amateur like Quetelet in Belgium, Knibbs in Australia, or Coats in Canada. And yet, present-day expansion and progress can be matched in quality by some of the early work. Florence Nightingale had much effect on the British War Office through her statistics showing deaths from battle and from disease separately. Her data on the army hospitals in India led to important reforms. It is not that the modern professional is better than the old-fashioned gifted amateur, but the former is more certain to be on hand when needed and commands the tools that have now been created.
Sampling in the new professionalization. Among the large-scale sample surveys now conducted are those on a monthly basis in the United States and Canada, quarterly in the Federal Republic of Germany and other European countries, and virtually continuously in India. The topics of survey are almost as wide as the topics of statistics itself [seeSample surveys].
Like any important change in technique, sampling does not merely attain the earlier objectives at lower cost; it brings a radically new viewpoint on the whole process of data collection. Instead of thinking of himself as charged with passively compiling given data, the statistician orients himself to the purpose of the compilation and the degree of accuracy required if the purpose is to be served; thus, he is led to take account of the nonsampling as well as the sampling error to which his work is subject. He sees himself at his professional best in measuring error, controlling quality, and evaluating results. He comes to measure information, not by the mass of data he can turn out, but rather by the accuracy attained in a single figure. The percentage of the labor force unemployed, for example, may be the occasion for a major decision in regard to the economy. If there is an allotment to improve the figure, he must decide what amount should be spent to improve the accuracy of response and what to increase the sample. In other words, he must decide if there will be more return from applying a given effort to the sampling or to the nonsampling error. The changed attitude toward error—the view that it is both inevitable and to be constantly combated—is seen in the United States Bureau of the Census, where the 1950 and subsequent population censuses have been accompanied by an official estimate of the degree of underenumeration, an estimate in part made by having superior enumerators survey a sample of areas. This practice has been applied in Canada since 1951. It is the mark of the professional statistician that he does not assert that his results are exact; even when he has done the survey by the best means that are available, he is satisfied to consider his one survey as an arbitrary selection from all the surveys that might have been carried out at about the same time and with equally acceptable definitions [seeErrors, article onNonsam pling errors].
Enduring problems of official statistics
A survey of the statistical agencies of any government, even those of a very advanced country, would show the continuance of some out-of-date practices, the publication of some figures that are not usable because they are based on purely administrative definitions of the entities counted or because the error is not stated by the agency but may be presumed to be large. Besides the valuable statistics they contain, library shelves are weighted with irrelevant, useless, and inaccurate statistics, a situation to be deplored in proportion as good techniques are available.
Government statistics’ judicial function. Especially pernicious are those inaccuracies intended to serve some political purpose. The strongest argument for statistical centralization relates to what may be called the judicial function of the official statistician. The statistics of foreign trade may show that the government’s trade policy is going badly; the statistics of prices may show that its monetary policy is leading to inflation. Protection of the public requires that the agency responsible for the policy not be the one with exclusive control over the statistics; to give it such control is to ask of it superhuman strength to withstand temptation. In a hundred minor decisions of statistical compilation, an operating agency may be swayed by noting how the result will come out. Presentation also requires impartiality. On issues such as whether a price index ought to be released now, when it has been scheduled to appear, or next week, after a national election has taken place, hangs the virtue of the statistician. Without this virtue and the resulting public confidence in statistics, much of the cost of collecting data is wasted; statistics that are not believed are of no use. In a statistician, as in a judge, ignorance is less dangerous than corruption. The statistician will be most useful to the community when his statistical honor is not in conflict with some nonstatistical responsibility. At a higher administrative level, the political head of the department that includes the statistical office must accept the self-discipline of permitting decisions regarding statistics to be made on statistical considerations, not because such decisions are always correct but because, when they are wrong, they are wrong in a disinterested way. One method that lessens the need for self-discipline is to have a commission entirely outside of politics control the statistical office, as is done in the Netherlands.
Much less subtle than the issues of the preceding paragraph are the distortions as a matter of policy that have been seen in some totalitarian countries. During the Stalin period in the Soviet Union, statistics were not considered to pertain to science but to revolution and to mobilization of the people for national construction. If the statistics do not show progress, the masses will become discouraged and their effort will flag. The same attitude was found in China in the 1950s (Li 1962). The competition between districts to be in the van of socialist production was extended to competition with respect to the figures of production, and this local zeal for declaring high figures reached its peak in 1960, when total production of cereals was announced as 275 million tons, apparently an exaggeration of over 100 per cent. It had been thought democratic to bring the masses into the work of compiling statistics; the consequent socialist rivalry seems to have genuinely deceived the regime, as well as its subjects, with subsequent disastrous consequences. There are clear signs that in the Soviet Union, as in older industrial societies, trustworthy statistics are now recognized as something more than a bourgeois luxury; whether the dependency of good decisions on good statistics is now understood in China is not clear.
Technical contributions
Extensive contributions have been made to technique in nearly every division of statistics by agencies attempting to implement efficiently their legislative assignment to collect data. One may mention the methods of calculating life tables; the deseasonalization of economic time series; the theory and practice of index-number calculation, as it has been developed in the United States Bureau of Labor Statistics and in the Dominion Bureau of Statistics in Canada; the classification of occupations, industries, and commodities (Coats 1925). Particularly important for the growth of demography were the technical contributions during the nineteenth century by William Farr and other statisticians of the United Kingdom General Register Office [seeIndex numbers; Life tables; Population; Time series].
The most striking single example of these contributions is the intimate relation of the U.S. Bureau of the Census to the development of tabulating and computing equipment, from the primitive punch-card devices of the 1890s, built in the Census Bureau itself by Herman Hollerith, to the Univac, built privately with bureau encouragement and financing and technical help from the Bureau of Standards. The Canadian census office also pioneered in this, with some highly original electric and compressed-air tabulators, the result of the ingenuity of Fernand Belisle, a lifelong employee of the Canadian Bureau of Statistics. Both Canadian and United States census offices have in recent years built or stimulated the building of input devices to avoid the keypunching of data, the most tedious part of the processing of large-scale surveys.
Besides having had a part in the development of computers, government statistical agencies have shown great initiative in using them. Such use, whether in an insurance office, an oil refinery, or a government statistical office, requires extensive rethinking of processes that have developed in the course of decades. Several of the more advanced of the world’s statistical offices have drastically reorganized their work in order to exploit electronic computation, with consequences of more extensive cross tabulation, better control of error in processing, and economy. In the United States the demand for machine-readable results has increased as users of statistics have acquired their own computers. The Bureau of the Census has an extensive catalogue of results available on punch cards or electronic tape; this mode of publication is sure to become widespread.
Publication of results
Prompt and full publication is the highest of virtues in government statistics—provided that the data published are accurate as well as relevant. This follows from the place of statistics in a decision-making process: the appropriateness of the decision—for example, to build a factory—will depend on the situation that exists at some future date; and if the statistician is not in a position to count the population of five or 25 years hence, he can at least describe what the situation was up to as recent a moment as possible. On the content of what is published—what cross classifications of distributions, what percentages, what time comparisons, what charts—the judgment of the government statistician becomes better the greater his contact with users of his results. With the multiplicity of governmental and private sources, the need increases for good indexing within any publication, as well as for cross referencing of comparable data in other publications.
Government statistical offices have entered the field of social science in their efforts to interpret their data. This runs from comment on statistical tables that enables the reader to know how the entities counted have been defined, along with measures of sampling and nonsampling error, through calculations distributing not-stated ages and deseasonalizing economic series, to full-scale monographs that have become a regular part of the census, in the United States and Canada going back at least a full generation. India is preparing a set of monographs based on the 1961 census, and Pakistan is planning to do likewise. If one thinks of the continuum, from the collection of raw statistical data to its final use as an ingredient either in scientific investigation or in the making of decisions, then there is room for differences of opinion as to where the role of the government statistical office ends and that of the user begins. The vital minimum is that the government statistician describe all aspects of his collection procedures that can possibly affect the interpretation of the results. It is to be hoped that in some future time no survey will be issued without realistic estimates of the accuracy of its figures, but that time still seems to be far in the future.
Notwithstanding all this, a certain abstemiousness is forced on the official statistician by virtue of his position; he cannot afford to take the sorts of risks in the interpretation of data that newspapermen cantake so freely. Public confidence in the accuracy of his figures, and hence their usefulness, would be jeopardized by palpable arbitrariness in judgments expressed in his text. This consideration often drives him to the writing of text that is no more than a gloss on what is obviously revealed in the tables and is perhaps less clear than the figures themselves. On the other hand, the government statistician can argue with justice that interpretation, lively or dull, made without his intimate knowledge of the basis of the figures is likely to be misleading. The answer is that, as a minimum, he give enough detail on his procedures to put the reader in a position to interpret correctly.
Secrecy
The matter of secrecy enters into statistical work in two ways, to which exactly opposite principles apply. In the report made by the individual to a government agency, it is generally considered that the most honest reporting will be secured if a guarantee of confidentiality is provided, giving the supplier assurance that his contribution will not be identified outside the statistical office and that only the statistical aggregates of which it is a part will be made public. The government statistician is endowed with the power to enforce reporting on the part of the persons or corporate bodies to which he addresses his questionnaires; the obverse of this power is the obligation to keep individual returns secret (Dobrovits 1947). In the large number of government surveys for which reporting is not mandatory, the degree of voluntary response may be proportional to the public confidence in provisions for maintaining secrecy.
On the other hand, the aggregate results of every survey ought to be given the widest possible publicity. One of the difficulties of government publication is that the government does not have complete access to the distribution channels of the private book trade; sometimes a solution is possible through private publication, as was done for the monographs on the 1950 census of the United States and as is done in the Netherlands and Germany. The principle of equal access of the public to statistics is important; this is safeguarded by preannounced release times, at which statistics are simultaneously made available to all who are interested. In many countries there is much diffusion of government statistical results through nongovernmental intermediate sources, including trade yearbooks and scientific journals; it is customary to allow republishing without specific permission.
A serious problem arises from the desire of government to keep certain statistical results secret. This was a regrettable necessity in the United States and Canada during World War II in respect, for instance, to detailed foreign trade figures, because the information would have been used by the Axis powers in their submarine campaign against Allied shipping. Aside from such exceptional cases, it is fair to say that there are no instances in which the public as a whole benefits from concealment of statistical reports that provide accurate data, although governments and particular departments within them may indeed benefit at the expense of the public. The discussion of the publication of government statistics merges at this point with the wider question of the free flow of information in the society as a whole.
Nathan Keyfitz
[See alsoElections, article onElectoral Systems.]
BIBLIOGRAPHY
Baines, Athelstane 1918 The History and Development of Statistics in Great Britain and Ireland. Pages 365-389 in John Koren (editor), The History of Statistics. New York: Macmillan.
Bowman, Raymond T. 1964 The American Statistical Association and Federal Statistics. Journal of the American Statistical Association 59:1–17.
Coats, Robert H. 1925 The Classification Problem in Statistics. International Labor Review 11:509–525.
Cummings, John 1918 Statistical Work of the Federal Government of the United States. Pages 573-689 in John Koren (editor), The History of Statistics. New York: Macmillan.
Demographic Yearbook. → Issued by the United Nations since 1948.
Dobbovits, Alexandre 1947 Sur le secret en statistique. Volume 3, pages 769-778 in International Statistical Conference, Washington, D.C., Proceedings. Calcutta: Eka.
Faure, Fernand 1918 The Development and Progress of Statistics in France. Pages 217-329 in John Koren (editor), The History of Statistics. New York: Macmillan.
Godfrey, Ernest H. 1918 The History and Development of Statistics in Canada. Pages 179-198 in John Koren (editor), The History of Statistics. New York: Macmillan.
Greenwood, Major et al. 1932 Discussion on the Quantity and Quality of Official Statistical Publications. Journal of the Royal Statistical Society 95:279–302.
Hauser, Philip M. 1963 Statistics and Society. Journal of the American Statistical Association 58:1–12.
Hauser, Philip M.; and Leonard, William R. (editors) (1946) 1956 Government Statistics for Business Use. New York: Wiley.
Holmes, Oliver W. 1960 “Public Records”—Who Knows What They Are? American Archivist 23:3–26.
Julin, Arm AND 1918 The History and Development of Statistics in Belgium. Pages 125-175 in John Koren (editor), The History of Statistics. New York: Mac millan.
Knibbs, George H. 1918 The History and Development of the Statistical System of Australia. Pages 55-81 in John Koren (editor), The History of Statistics. New York: Macmillan.
Koren, John (editor) 1918 The History of Statistics. New York: Macmillan.
Leonard, William R. 1958 An Outlook Report. Journal of the American Statistical Association 53:1–10.
Li, Choh-Ming 1962 The Statistical System of Communist China. Berkeley: Univ. of California Press.
Linder, Forrest E. 1959 World Demographic Data. Pages 321-360 in Philip M. Hauser and Otis Dudley Duncan (editors), The Study of Population: An Inventory and Appraisal. Univ. of Chicago Press.
Meitzen, August (1886) 1891 History, Theory, and Technique of Statistics. Philadelphia: American Academy of Political and Social Science. → First published in German.
Mills, Frederick C.J and Long, Clarence D. 1949 The Statistical Agencies of the Federal Government. Washington: Government Printing Office.
Morgenstern, Oskar (1950) 1963 On the Accuracy of Economic Observations. 2d ed. Princeton Univ. Press.
Nixon, James W. 1960 A History of the International Statistical Institute: 3885–1960. The Hague: International Statistical Institute.
North, S. N. D. 1918 Seventy-five Years of Progress in Statistics: The Outlook for the Future. Pages 15-49 in John Koren (editor), The History of Statistics. New York: Macmillan.
Stern, Johanna 1958 [Review of] Th. L. Galland’s Statistik der Beschaeftigten und Arbeitslosen. Journal of the American Statistical Association 53:1040–1043.
Stuart, C. A. V. 1918 The History and Development of Statistics in the Netherlands. Pages 429-444 in John Koren (editor), The History of Statistics. New York: Macmillan.
United Nations, Statistical Office 1954 Handbook of Statistical Organization. Studies in Methods, Series F, No. 6. New York: United Nations.
U.S. Bureau OF The Budget, Office OF Statistical Standards (1947) 1963 Statistical Services of the United States Government. Rev. ed. Washington: Government Printing Office.
Westergaard, Harald L. 1932 Contributions to the History of Statistics. London: King.
Wurzburger, Eugene 1918 The History and Development of Official Statistics in the German Empire. Pages 333-362 in John Koren (editor), The History of Statistics. New York: Macmillan.