Raw capture — raw/science-and-statistics-box.md
url: https://www-sop.inria.fr/members/Ian.Jermyn/philosophy/writings/Boxonmaths.pdf date_fetched: 2026-08-09
Science and Statistics George E. P. Box Journal of the American Statistical Association, Vol. 71, No. 356. (Dec., 1976), pp. 791-799. Stable URL: http://links.jstor.org/sici?sici=0162-1459%28197612%2971%3A356%3C791%3ASAS%3E2.0.CO%3B2-W Journal of the American Statistical Association is currently published by American Statistical Association.
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at http://www.jstor.org/about/terms.html. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use. Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at http://www.jstor.org/journals/astata.html. Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission.
JSTOR is an independent not-for-profit organization dedicated to and preserving a digital archive of scholarly journals. For more information regarding JSTOR, please contact support@jstor.org.
http://www.jstor.org Tue Jun 19 10:48:57 2007
Science and Statistics GEORGE E. P. BOX*
Aspects of scientific method are discussed: i n particular, its representation as a motivated iteration i n which, in succession, practice confronts theory, and theory, practice. Rapid progress requires sufficient flexibility to profit from such confrontations, and the ability to devise parsimonious but effective models, to worry selectively about model inadequacies and to employ mathematics skillfully but appropriately. The development of statistical methods at Rothamsted Experimental Station by Sir Ronald Fisher is used to illustrate these themes.
on the one hand, nor by the undirected accumulation of practical factson the other, but rather by a motivated iteration betmeen theory and practice such as is illustrated in Figure A(1). A. The Advancement of Learning A(1) An Iteration Between Theory and Practice A(2) A Feedback Loop
Fisher was introduced by the title which he himself would have chosen-not as a statistician but as a scientist, and this was certainly just, since more than half of his published papers were on subjects other than statistics and mathematics. 1 I y theme then will be first to show the part that his being a good scientist played in his astonishing ingenuity, originality, inventiveness, and productivity as a statistician, and second to consider what message that has for us now.
c
A
E
\
DEDUCTION
A
INDUCTION
HYPOTHESES MODEL CONJECTURE THEORY
H,
-
H, r---L--
1 MODIFIED 'HYPOTHESIS H
La--
I , - , J DEDUCTION
Matters of fact can lead to a tentative theory. Deductions from this tentative theory may be found to be discrepant with certain known or specially acquired facts. These discrepancies can then induce a modified, or in some cases a different, theory. Deductions made from the modified theory now may or may not be in conflict with fact, and so on. In reality this main iteration is accompanied by many simu1taneo;s subiterations (see, e.g., 21)
2.2 Flexibility
On this view efficient scientific iteration evidently requires unhampered feedback. The iterative scheme is shown as a feedback loop in Figure A(2). In any feedback 2.1 Iteration Between Theory and Practice loop it is, of course, the error signal-for example, the One important idea is that science is a means whereby discrepancy between what tentative theory suggests learning is achieved, not by mere theoretical speculation should be so and what practice says i s so-that can produce learning. The good scientist must have the flexibility * George E.P. Box is R.A. Fisher Professor of Statistics, University of Wisconsin, and courage to seeli out, recognize, and exploit such Madison, WI 53706. Research was supported by the United States Army under errors-especially his own. In particular, using Bacon's Grant DAHC04-76-G-0010. This is the written version of the R.B. Fisher Memorial Lecture presented a t the joint statistical meetings of the American Statistical Association and Biomztric Society given a t St. Louis in 1974. The author gratefully acknowledges the assistance of his wife Joan who generously shared the results of her research on her father's life and made available the manuscript of her biography of Fisher.
0 Journal of the American Statistical Association December 1976, Volume 71, Number 356 Applications Section
792
Journal of the American Statistical Association, December 1976
analogy, he must not be like Pygmalion and fall in love with his model.
2.3 Parsimony Since all models are wrong the scientist cannot obtain a "correct" one by excessive elaboration. On the contrary following William of Occam he should seek an economical description of natural phenomena. Just as the ability to devise simple but evocative models is the signature of the great scientist so overelaboration and overparameterization is often the mark of mediocrity.
2.4 Worrying Selectively Since all models are wrong the scientist must be alert to what is importantly wrong. I t is inappropriate to be concerned about mice when there are tigers abroad.
2.5 Role of Mathematics in Science Pure mathematics is concerned with propositions like "given that A is true, does B necessarily follow?" Since the statement is a conditional one, it has nothing whatsoever to do with the truth of A nor of the consequences B in relation to real life. The pure mathematician, acting in that capacity, need not, and perhaps should not, have any contact with practical matters at all. In applying mathematics to subjects such as physics or statistics we make tentative assumptions about the real world which we know are false but which we believe may be useful nonetheless. The physicist knows that particles have mass and yet certain results, approximating what really happens, may be derived from the assuhption that they do not. Equally, the statistician knows, for example, that in nature there never was a normal distribution, there never was a straight line, yet with normal and linear assumptions, known to be false, he can often derive results which match, to a useful approximation, those found in the real world. It follows that, although rigorous derivation of logical consequences is of great importance to statistics, such derivations are necessarily encapsulated in the knowledge that premise, and hence consequence, do not describe natural truth. I t follows that we cannot know that any statistical technique we develop is useful unless we use it. Major advances in science and in the science of statistics in particular, usually occur, therefore, as the result of the theory-practice iteration. The researcher hoping to break new ground in the theory of experimental design should involve himself in the design of actual experiments. The investigator who hopes to revolutionize decision theory should observe and take part in the making of important decisions. An appropriately chosen environment can suggest to such an investigator new theories or models worthy to be entertained. ;\lathematics artfully employed1 can then enable him to derive the logical consequences of his tentative 1 The researcher's purely mathematical ingenuity is likely to be exercised more, not less, by the fact of his dealing with genuine problems.
hypotheses and his strategically selected environment will allow him to compare these consequences with practical reality. In this way he can begin an iteration that can eventually achieve his goal. An alternative is to redefine design ,and decision so that such words as e z p e r i ~ ~ z e n t a l mathematical solutions which do not necessarily have any relevance to reality may be declared optimal.
SCIENTIST
With these ideas in mind let us see how Fisher qualifies as a scientist, using for illustration some of the events occurring during his stay at Rothamsted Experimental Station.
3.1 Rothamsted In 1919, Fisher had rejected the security and prestige of working under Karl Pearson in the most distinguished st.atistica1 laboratory in Britain and at that time certainly in the world. Instead, he took up a temporary job as the sole statistician is a small agricultural research station in the country. He was then already 29 years old and he later said that he was aware that he had failed at both the jobs (teacher and actuary) that he had so far attempted. Sir John Russell, then Director of Rothamsted, later recalled 117, p. 3261
.. . when I first saw him in 1919 he was out of a job. Before deciding anything I wrote to his tutor a t Caius college . . . about his mathematical ability. The answer was that he could have been a first class mathematician had he "stuck to the ropes" but he would not. That looked like the type of man we wanted.. . I had only £200 and suggested he should stay as long as he thought that should suffice.. . . He reported to me weekly a t tea at my house.. . . I t took me a very short time to realize that he was more than a man of great ability, he was in fact a genius.
.
At the end of a year, Fisher, who had a wife and child, had used up twice the £200, but by that time he had been given a permanent post.
3.2 Weighing the Baby For the theory-practice iteration to work, the scientist must be, as it were, mentally ambidextrous; fascinated equally on the one hand by possible meanings, theories, and tentative models to be induced from data and the practical reality of the real world, and on the other with the factual implications deducible from tentative theories, models and hypotheses. Fisher had great interest in practical matters. For example, he begins the real business of his book Statistical Methods for Research W o r k e r s in Chapter 2, by discussing different ways of plotting data. His first example is introduced as follows [12, p. 251: "Figure 1 represents the growth of a baby weighed to the nearest ounce at equal intervals from birth." He does not say that this is any particular baby. Recently I was fortunate to see the Fisher family records in which in Fisher's own hand are recorded the weight from birth of every one of his nine
793
Science and Statistics
children, weighed by himself, with the results carefully graphed. Comparison shows that the child is his second son, Harry Leonard, who was born in 1923 shortly before the first edition of the book was written. The next leg of the scientific iteration is hinted at as he goes on to discuss how best to plot the data so as to make "a rough examination of the agreement of observation with any (proposed) law of increase."
3.3 Find the Lady The extraordinary extent to which Fisher's actual every day experience was grist to the mill of his inductive mind is further illustrated in the famous opening lines of Chapter I1 of Fisher's book The Design of Ezperinzents [ll, p. 111: "A lady declares that by tasting a cup of tea . . . she can discriminate whether the milk or the tea infusion was first added to the cup. We will consider the problem of designing an experiment by means of which this assertion can be tested." Fisher proceeds to use this example to explain and illustrate the basic principles of good statistical design. There was, of course, a real lady. This incident happened many years before the book was written and just after Fisher came to Rothamsted. The lady was Dr. Muriel Bristol, the algologist, and she had declined the cup of tea that Fisher had offered her because he had added the tea first. Fisher declared it made no difference. To which she replied "Of course it did." Her future husband, William Roach, who was close at hand said "Let's test her," they did, and according to him she made nearly every choice correctly. In this she behaved similarly to the lady in the book who got one n-rong. 3.4 From Soil Bacteria to Nonlinear Design The tea urn was a great catalyst to iteration. There, each afternoon, Fisher conversed with members of the scientific staff and with visitors and became involved in their problems, often with dramatic consequences. One scientist who came to Rothanzsted about the same time as Fisher and became his intimate friend was the bacteriologist, Gerard Thornton. I t was he who first interested Fisher in improving the time consuming dilution methods for making bacterial counts. This resulted in Fisher's pioneering work on nonlinear design in 1922 mentioned by Cochran 141.
3.5 From Cotton to Extreme Values One of the early visitors to Rothamsted mas L.H.C. Tippett from the Cotton Research Institute. A matter of great practical concern to him was the strength of cotton yarn. Since the breaking strength of a piece of cotton is the strength of the weakest link, he was faced with what we should now call the extreme value problem. Tippett had first studied with Karl Pearson and had earlier approximated the distribution using the method of moments. I n cooperation with Fisher the problem lvas tackled rather differently. The authors note [14, p. 1801
that, "the limiting distribution must be such that the extreme member of a sample of n from such a distribution has itself a similar distribution." This simple but remarkable insight leads to a functional equation which yields as its solution the basic limiting forms. From these forms almost all subsequent work on the subject springs. The theory has applications in such different fields as the design of dams and the reliability of components. Like so many of Fisher's brain children this is now regarded as a distinct field of study. I will use for further illustration work that Fisher did at Rothamsted b e h e e n 1919 and 1927 which began with regression analysis and ended with a complete and elegant theory of experimental design which is still the basis for most statistically planned experiments. This work was published in a series of papers having the general title "Studies in Crop Variation" and numbered I, 11, IV, and VI [7, 13, 8, 5 , 61.
3.6 From Dung to Orthogonal Polynomials and Residual Analysis
By 1919 13 plots on Broadbalk wheat fields had received thirteen different manurial treatments uniformly for 67 years. In "Studies in Crop Variation I" 171, Fisher begins by presenting a workmanlike discussion, which lasts for twelve pages, of the responses to the thirteen different manures revealed by his analysis of the Broadbalk data. I n particular, he concludes that there is really nothing like plain dung. I t gives a high yield with no significant diminution of its effect over the years. He then quite suddenly shifts from manure to mathenlatics revealing where his analysis has come from. I n the next few pages he introduces orthogonal polynomials, presents formulas for their calculation from equispaced data, obtains the distributional properties of the coefficients, and shows how their significance may be judged. Without calling it that he presents the appropriate analysis of variance which he has used in fitting fifth degree polynomials to the annual yields. Most interesting of all, he discusses the properties of the residuals y - Q from a fitted polynomial of any degree - allowing us to see him in the guise of what some people now call a data analyst. Data analysis, a subiteration in the process of investigation, is illustrated here.
TENTATIVE
INFEREN':
TENTATIVE
RIT1cls
In the inferential stage, the analyst acts as a sponsor for the model. Conditional on the assumption of its truth he selects the best statistical procedures for analysis of the data. Having completed the analysis, however, he must switch his role from sponsor to critic3 Conditional now on the contrary assumption that the model may be 2 This paper [81 was presented to the Royal Society without the general title but was mysteriously labelled I11 and had clearly been originally intended for this series. a The apt christening of statistical criticism is due to Cuthbert Daniel.
794
Journal of the American Statistical Association, December 1976
seriously faulty in one or more suspected or unsuspected ways he applies appropriate diagnostic checks, involving various kinds of residual analysis. In order to conduct his analysis of the residuals from the fitted polynomials, Fisher obtained
+
i the average value of V ( y - 8) as ( I - ( r l)/n)$, ii. the individual variances of the residuals y, - 8, for the 67 observations, iii. the identity
+
u2 = V ( Y , ) = V ( Y ,- zl,) V($,) , j = 1,2, . . ., n , iv. an approximate formula for the autocorrelations of residuals from a fitted polynomial of any degree.
The average value of V(Q) from (i) and (iii) is u2(r l)/n. Thus, Fisher says if we want to have a small variance for Q we should keep r small-a demonstration of the value of parsimony, helping to justify his use of polynolnials of only fifth degree. Fisher plots the variances V (y, - Q,) for the individual residuals against their time order j. Because of relation (iii) the graph looked at upside down is also a plot of 17(Q,). Using this he notes the deceptive reduction of V(y, - Q,) at the extremities of the scale and the corresponding increase of V(Q,) and says [7, p. 1231 "it is a neakness of the polynomial form that the extreme terms should be so much affected." Finally, mentioning that overfitting and underfitting are both to be avoided he uses the matching of theoretical and empirical autoco~relationsof residuals to check when a polynomial of sufficiently high degree has been fitted. In particular, he compares theoretical and observed autocorrelations of residuals from polynomials of degree zero and five to shon the inadequacy of the former and the satisfactory fit of the latter. This application of serial correlation of residuals to the an kward problem of deciding a t what point adequacy of fit has been achieved has great freshness and interest 55 years later.
+
Education Acts of 1876 and 1880 made attendance at school compulsory." We are left to speculate whether the low wheat yields occurred after that time because the hands of the little boys who pulled the foxtail grass were now covered with ink and not ~vithearth.
3.8 From Rainfall and Wheat Yield to Distributed Lags I n 1924, in the third paper of the series 181, he used the Broadbalk data to demonstrate the influence of rainfall on the wheat yield. At the beginning of the paper he seemed to fear that he might be expected to accouilt for the effects not only of rainfall but also for such other variables as maximum and minimum temperature, dew point, and hours of bright sunshine. But he points out that allowances for the effect of each of these on the final harvested yield 11-ould need to be included a t least for each month separately. And he says if so many regressors are included a very high proportion of the total variation can seem to be accounted for by chance alone. In case some dissident reader might doubt it, he thereupon outlines the derivation of the distribution of the multiple correlation coefficient in one paragraph flat using n-dimensional geometry and on the next page produces a short table of tail areas for R. He then goes on to discuss the misleading effects of selection in what would now be called step-wise regression. Fisher's data \$-ereas follovis : i. for each of the 13 Broadbalk plots he had yields for each of 60 years,4 ii. for each of these 60 years he had daily rainfall records and for convenience he aggregated these for each year into 61 six-day periods (6 X 61 = 366) beginning immediately after the harvest.
In a remarkable demonstration of parsimonious modeling he first suggests that the yield of wheat in the jth year, zu, say, might be represented by
3.7 Weeds and the Education Acts Fisher was perplexed by the shapes of his fitted yield graphs. These showed a pattern of significant slow changes common to all the 13 Broadballr plots. I n particular, there was a common tendency for ~ O T T yields roughly in the period 1870-1880. This common pattern was not due to weather; a similar 'analysis he conducted for successive yields of experimental wheat a t Woburn, wheat averages for the whole of Hertfordshire, and for barley and grass from experimental plots a t Rothamsted, failed to show it. He speculates [7, p. 1291, "Of all the organic factors which influence the yield of wheat it is probable that weeds alone change sufficiently slowly to explain the changes at Broadbalk." He goes on to describe, as only a dedicated gardener could, all the various weeds that were found there. He notes that old records show that, in 1853, 211 man-days and 714 boy-days were spent in weeding the field. I n particular, the boys probably held in check by hand weeding the slender foxtail grass Alopecuris agrestis. But he says 17, p. 1313 "it may be remembered that the
In this model the coefficient at provides the average effect on eventual harvested yield of one inch of rain in the tth time period. I n modern parlance (3.1) might be called a "transfer function" model expressing the memory" of the system. Economists later called it a "distributed lag" model but they seem to have been unaware of Fisher's prior work or of his ingenious way of proceeding using orthogonal polynomials. As it stands (3.1) is highly nonparsimonious. Fisher decided, therefore, to represent the rainfall data r,, by orthogonal polynomials of fifth degree. He now notes that the coefficients a t should also f o l l o ~a ~smooth curve which might be represented in the same way. Thus, ( (
4 Five years 1890, 1891, 1905, 1906, and 1915 were omitted because the plots in these years had special treatment.
795
Science and Statistics
But if the orthogonal functions Tit are chosen so that Ti? = 1 ; then, after summing, (3.1) may be written
xt
wj = c f aopoj f a l p l j
f a5P5j .
The a's which determine the lagged weights in the transfer function can thus be obtained by regressing the wj onto the estimated 0's. Having carried through the necessary heavy calculations and graphed his results Fisher conducts a very extensive discussion and comparison of the polynomial distributed lag curves for the differently manured plots from which, in particular, he adduces the predominant effect of rain in reducing soil nitrates. One feels that his love of parsimony mas certainly not lessened by the fact that the computations were performed by hand by himself and his assistant. Indeed, much can still be learnt from his discussion about economical ~rocessesof calculation and appropriate checks 18, p. 111-31.
3.9 From Fertilizer and Potatoes to the Analysis of Variance
form of the appropriate correction came quickly in the first edition of Statistical Methods in 1925 (see [12, p. 2381). Using part of the same data, Fisher there gives the correct analysis and points out that it is essential to use separate error variance estimates (for between and within plot comparisons) and shows that one is indeed significantly larger than the other. iv. In this very first paper on the analysis of variance, Fisher demonstrated the flexibility of his thought by questioning the linear model (which almost everybody else has ever since accepted as representing received truth). The authors say [13, p. 3161, "the above test is only given as an illustration of the method; the summation formula for combining the effects of variety and manurial treatment is evidently quite unsuitable for the purpose. No one would expect to obtain from a low yielding variety the same actual increase in yield which a high yielding variety would give . . a far more natural assumption is that,the yield should be the product of two factors one de~endinkon the varietv and one on the manure." With the possibility of transformation so much a part of Fisher's everyday thought,, we might expect him now to wroceed along that route but in fact he derives the avpropriate nonlinear analysis, devising met,hods which have only recently been rediscovered [IS].
.
-
-
About this time Fisher was getting rather tired of 3.10 Mice, Tigers, and Randomization analyzing old records-he later described it as "raking A man in daily muddy contact with field experiments over the muck heap." I n "Studies in Crop Variation 11," could not be expected to have much faith in any direct jointly authored with his assistant, Miss W.A. Mac- assumption of independently distributed normal errors.' Kenzie, and subtitled "The Manurial Response of Differ- While the supposition of marginal normality for the errors ent Potato Varieties," [13] he tried his hand at analyzing might be regarded as innocuous, the idea that errors from some experimental data from Rothamsted. The authors adjacent plots of land could be treated as independent remark that it would be convenient if (contrary to some would be obviously absurd and dangerous. This was one expert opinion) different varieties of plants did not react important reason for Fisher's insistence (i) on the physical differently to fertilizers, or as we should say now, if there act of randomization as a necessary condition for the were no interaction between variety and fertilizer. validity of any experiment and (ii) that given that An experiment had recently been run by Thomas Eden, randomization had been carried out inferences should be a crop ecologist at Rothanisted, in which each of t~velve made from the appropriate randomization distribution; varieties of potatoes were tested with six different com- to which, however, standard normal theory often probinations of manure. This experiment was analyzed as if vided an adequate approximation. it were a thrice replicated and randomized 12 X 6 facTo guarantee the exact validity of the usual null tests torial. (It wasn't, but we return to that later.) made with the standard linear model it is not, of course, From the analysis of variance which is presented, the necessary that the density function of the error vector e answer to the question, '(Is there significant interaction be spherically normal, it is necessary only that it be between varieties and manures?" appears to be No! i.e., the density function be of spherically syn~nietric,~ There are some remarkable things about this paper, the form f(ele). The fact that standard normal theory however : often provides an adequate approximation to that given by randomization theory is not because the density for i. The analysis of variance, hinted a t earlier, appears here for the first time in its completeness. I t arrives quite suddenly randomized errors is necessarily approximated by that of and unannounced in the middle of the paper after the discus- independent normal deviates. I t is rather because, in the sion of agricultural questions. I t is, of course, not even appropriate vector space, the symmetry induced by mentioned in the title. ii. After the algebraic identity between the tot'al swn of squares randomization is approximated by spherical symmetry. Fisher showed some irritation with later workers who and the within and between treatment-s sun1 of suuares has been written down, the statement is made [13, p. 3151 "If saw only a rich source of purely mathematical developall the plots were undifferentiated, as if the numbers had ment in his jTork, In workers on what has been mixed up and written down in random order, the aver- come to be called "distribution-free,, tests have often age value of each of the two parts is proportional to the number of degrees of freedom in the variation of which it is failed to emphasize and soinetimes perhaps even to ~ ha t ~ the very ~ beginning, randomization, an realize the limitations imposed by the necessary assump, important flag under which Fisher will sail, is firmly nailed tion of symmetry of the joint error distribution. The to the mast. iii. The analysis is wrong, because in fact the trial was actually run as what is now called a split plot design. Feedback in the
5 Obviou~ly,this must be true for any criterion which is a homogeneous function of the data of degree zero.
Journal of the American Statistical Association, December 1976
796-
validity of this assumption could, of course, only be guaranteed by randomization. Otherwise, the derived procedures, far from being distribution free, would be almost as restrictive as those derived on the assumption of normal independent errors. I t is true that long usage has seemed to sanctify the proposition that density funcf (yz) or at least that tions are of the form p(y) = p (y) = S (y), where S is some symmetric function of the elements of y. These propositions have come to be treated almost as natural laws or a t least as rules of the game that no sportsman would q ~ e s t i o n .I n ~ fact, of course, experiments where errors cannot be expected to be independent are very common. These points are not new but if we are to appreciate Fisher's point of view they need to be brought together and illustrated together. For this latter purpose the results of a simple sampling experiment are shown in the table. Two samples of 10 observations from identical populations of the forms indicated were taken and subjected to a t-test (t) and a I1Iann-Whitney test (MW). The sampling was repeated 1,000 times and the number of results significant at the 5 percent point was recorded. Ideally, this number should be 50 (that is, 5 percent of the total) but it has a standard deviation of about 7 because of sampling errors. More accurate results may be obtained by taking larger samples or by analytical procedures, however, since there is no practical difference between a significance level of say 4 percent and 6 percent, the present investigation suffices for illustration. Autocorrelation between adjacent values was introduced by generating observations from a moving average model of the form y, = u, - Out-l. I n this model the u t were
The frequencies shown under NR are those obtained for a nonrandomized test. The frequencies under R are those obtained when the observations n-ere randomly allocated to the two groups. As is to be expected the significance level of the t-test is affected remarkably little by the drastic changes made in the marginal parent distribution-changes for which the distribution-free test provides insurance. Unfortunately, of course, both tests are equally impaired by error dependence unless randomization is introduced when they do about equally well. The point is, of course, that it is the act of randomization that is of major importance here not the introduction of the distribution-free test function. 3.11 From M u c k Raking to Group Theory
Eden's potato data served to illustrate the method of analysis of variance but Fisher appears to have had no hand in planning that experiment. The design is not randomized nor blocked and its very deficiencies call for appropriate remedies. When Fisher's friend Gosset saw the paper, he wrote to Fisher [15, Letter No. 291, "The experiment seems to me to be quite badly planned, you should give them a hand in that . . .." Fisher later notes Gosset's "suggesting that I should start designing experiments" 115, summary of Letter No. 291. This he proceeded to do. The iterative process including the design aspect is sketched in Figure B.
B. Data Analysis and Data Getting in the Process of Scientific Investigationa
Frequency in 1,000 Trials of Significance at the 5 Percent Level Using the t-Test ( t )and the Mann-Whitney Test ( M W ) with No Randomization (NR) and Randomization ( R ) Parent distribution
PI
Test
Rectangular
-NR
R
Normal NR
)xq AV
Chi-squarea R
NR
R
Independent observations
H,
-
-_
REPLACES /
H,
HYPOTHESIS H,
L
Autocorrelated successive observations
ABLE
?':I-
k E
CONSEQUENCES
P
a
The parent chi-square distribution hasfourdegreesof freedom and is thus highly skewed.
independently and identically distributed about zero in the forms indicated in the table. Values of O were chosen so that pl, the first serial correlation, had values of -0.4 and +0.4. 6
Except in the study of time series.
Between 1919 and 1928 an iterative sequence occurred that went through three main stages, each leading logically to the next via interaction of theory and practice. The analysis of existing records led to the analysis of experimental trials which then led to the design of experimental trials. There were different but interactive aspects to this development. We can see (i) sequential evolution of the
Science and Statistics new methods in response to unfolding realizations of need, (ii) the persuading of practitioners to try the new techniques, and (iii) the changing role of the statistician implied by the development.
3.12 Evolution of the New Methods Fisher's attempts to analyze experimental data quickly led him to the essential principles of experimental design. The need for randomization to achieve validity; for replication to provide a valid estimate of error; for blocking extraneous sources of disturbance to achieve accuracy. Blocking in two directions sin~ultaneously(by randomized Latin squares) was particularly appealing. Fisher would have been brought to see the enormous advantages of the unorthodox factorial arrangements as an economical way to assess the effects of variables in combination by, for example, his early attempts to impart meaning to the differences associated with the 13 differently manured Broadbalk plots to which fertilizers had been applied in a highly nonbalanced manner. However, while the efficiency of factorial designs could be increased by packing in more factors, larger factorial designs required bigger blocks and hence produced greater inhomogeneity in the experimental material, giving larger experimental errors. The answer which quickly followed was confounding.
3.1 3 Persuading Practitioners The blessings of feedback were only available if scientists would try out his designs but, not surprisingly, Fisher a t first did not have an easy job selling his revolutionary ideas a t Rothamsted. Indeed, the first design run to his specification (in 1924) was not done a t Rothamsted a t all. I t was a randomized Latin Square design run at Bagshot for the Forestry Commission who had asked for and acted on his advice. But between 1924 and 1929, as described in "Studies in Crop Variation IV and VI" [5, 61, there is a rapid development of ideas which were quickly put into practice. I t is clear that Eden had become a convinced disciple during this period and it is refreshing, but alas unfamiliar, to see publication of new designs simultaneous with data obtained from their successful use. By the end of this period data were being collected from designs of great accuracy and beauty which included all of Fisher's ideas. In spite of all this in 1926 the Director of Rothamsted, Sir John Russell, wrote a paper 1161 in the Journal o f the M i n i s t r y o f Agriculture about agricultural experimentation which almost totally ignored the ideas of his proteg6. However, in the next issue [9] in a paper notable for its brevity and clarity, Fisher outlined h i s philosophy on the subject, setting his boss to rights and anyone else who would listen.
3.14 A New Heritage for Statisticians The original concept that the research station needed a statistician was revolutionary, but certainly the role
797 initially envisaged in 1919 for the statistician was a passive and possibly even a temporary one. Russell wondered if anything more could be extracted from the existing records. Fisher's work gradually made clear that the statistician's job did not begin when all the work was over-it began long before it was started. The statistician was not a curator of dusty relics. His responsibility to the scientific team was that of the architect with the crucial job of ensuring that the investigational structure of a brand new experiment was sound and economical. The latter role is much more fun than the former. He himself relished it and we should thank him for bequeathing it to us. It calls for abilities of a high order. I t requires among other things the wit to comprehend complicated scientific problems, the patience to listen, the penetration to ask the right questions, and the wisdom to see what is, and what is not, important. Finally, it requires from the statistitian the courage to wager his reputation each time an experiment is run. For the time must come when all the data are in and conclusions must be drawn; a t this stage oversights in the design, if they exist, will become embarrassingly evident.
4.1 Cookbookery and Mathematistry The maladies which result may be called coolcboolcery and ?natheli~atistry.T he symptoms of the former are a tendency to force all problems into the molds of one or two routine techniques, insufficient thought being given to the real objectives of the investigation or to the relevance of the assumptions ibmplied by the imposed methods. Concerning the latter, Fisher's apparently bivalent attitude towards mathematicians has often been remarked and has been the cause of perplexity and annoyance. He himself was an artist in the use of mathematics and emphasized the importance of mathematical training for statisticians-the more mathematics known the greater the potential to be a good statistician. Why then did he sometimes seem to refer so slightingly to mathematicians? The answer I think is that his real target was "mathematistry." I t is to make the distinction that the word is introduced here. Mathematistry is characterized by development of theory for theory's sake, which since it seldom touches down with practice, has a tendency to redefine the problem rather than solve it. Typically, there has once been a statistical problem with scientific relevance but this has long since been lost sight of. Fisher felt strongly about
Journal of the American Statistical Association, December 1976
798
this last point, particularly when he himself had produced the originally useful idea. I have cited already the development of distribution-free tests which, he felt, misused ideas initiated in Chapter I11 of his book Design of Experiments [ll, p. 481. Another annoyance was the generalization to what he felt was absurdity of his applications of group theory and combinatorial mathematics to experimental design. The penalty for scientific irrelevance is, of course, that the statistician's work is ignored by the scientific community. But this does not come to the notice of a statistician who has no contact with that community. I t is sometimes alleged that there is no actual harm in mathematistry. A group of people can be kept quite happy, playing with a problem that may once have had relevance and proposing solutions never to be exposed to the dangerous test of usefulness. They enjoy reading papers to each other at meetings and they are usually quite inoffensive. But we must surely regret that valuable talents are wasted at a period in history when they could be put to good use. Furthermore, there is unhappy evidence that mathematistry is not harmless. I n such areas as sociology, psychology, education, and even, I sadly say, engineering, investigators who are not themselves statisticians sometimes take mathematistry seriously. Overawed by what they do not understand, they mistakenly distrust their own common sense and adopt inappropriate procedures devised by mathematicians with no scientific experience. An even more serious consequence of mathematistry concerns the training of statisticians. We have recently been passing through a period where nothing very much was expected of the statistician. A great deal of research money was available and one had the curious situation where the highest objective of the teacher of statistics was to produce a student who would be another teacher of statistics. I t was thus possible for successive generations of teachers to be produced with no practical knowledge of the subject whatever. Although statistics departments in universities are now commonplace there continues to be a severe shortage of statisticians competent to deal with real problems. But such are needed.
4.2 Meeting the Challenge As long ago as 1950, Fisher, delivering the Eddington Memorial Lecture at Cambridge, said [lo, p. 221 For the future, so far as we can see it, it appears to be unquestionable that the activity of the human race will provide the major factor in the environment of almost every evolving organism. Whether they act consciously or unconsciously human initiative and human choice have become the major channels of creative activity on this planet. Inadequately prepared we unquestionably are for the new responsibilities, which with the rapid extension of human control over the productive resources of the world have been, as it were, suddenly thrust upon us.
One by one, the various crises which the world faces become more obvious and the need for hard facts on which to take sensible action becomes inescapable. The demand
for competent statisticians who can tease out the facts by analyzing data, planning investigations, and developing the necessary new theory and techniques will, therefore, continue to increase. 4.3 Training of Statisticians Competent statisticians will be front line troops in our war for survival-but how do we get them? I think there is now a wide readiness to agree that what we want are neither mere theorem provers nor mere users of a cookbook. A proper balance of theory and practice is needed and, most important, statisticians mast learn how to be good scientists; a talent which has to be acquired by experience and example. To quote Fisher once more, in 1952, in a letter concerning a proposed Statistics Center to be set up in Scotland he said: "I have no hesitation in advising that such a centre as you have under discussion should plan to integrate, teaching closely with project work in which practical experience'can be gained by those who are capable of learning from i t ; in contradistinction to the ruinous process of segregating the keener minds into a completely sterile atmosphere" [3]. I t is encouraging that at more and more statistical centers such advice is now being taken seriously.
It is surely because he was all of these that he was much more than the sum of the parts. He provides an example we can seek to follow. [Received May 1976.1 REFERENCES [I] Box, G.E.P. and Tiao, G.C., Bavsian Inference in Statistical Analysis, Reading, Mass. : Addison-Wesley Publishing Co., 1973. [2] -- and Youle, P.V., "The Exploration ,and Exploitation of Response Surfaces: An Example of the Link Between the Fitted Surface and the Basic Mechanism of the System," Biometries, 11, No. 3 (1955), 287-323. [a] Box, Joan Fischer, Fisher, The Life of a Scientist, New York: John Wiley & Sons, Inc. In press. [4] Cochran, W.G.,. "Experiments for Nonlinear Functions," Journal of the American Statistical Association, 68, No. 344 (1973), 771-81. [5] Eden, T. and Fisher, R.A., "Studies in Crop Variation IV. The Experimental Determination of the Value of Top Dressings with Cereals," Journal of Agricultural Science, 17 (1927), 54862. I31 --- and Fisher, R.A., "Studies in Crop Variation VI. Experiments on the Response of the Potato to Potash and Nitrogen," Journal of Agricultural Science, 19 (1929), 201-13. [7] Fisher, R.A., "Studies in Crop Variation I. An Examination of the Yield of Dressed Grain from Broadbalk," Journal of Agricultural Science, 11 (1921), 107-35.
Science and Statistics "Studies in Crop Variation 111. The Influence of Rainfall on the Yield of Wheat a t Rothamsted," Philosophical Transactions of the Royal Society of London, B, No. 213 (1924), 89-142. [9] ----, "The Arrangement of Field Experiments," Journal of the Ministry of Agriculture, 33 (1926), 503-13. "Creative Aspects of Natural Law," The Eddington [lo] Memorial Lecture, Cambridge, Eng. : Cambridge University Press, 1950. [Ill , The Design of Experiments, (8th Ed.), Edinburgh: Oliver & Boyd, Ltd., 1966. [12] ----, Statistical Methods for Research Workers, (14th Ed.), Edinburgh: Oliver & Boyd, Ltd., 1970. [13] -- and MacKenzie, W.A., "Studies in Crop Variation 11. The Manurial Response of Different Potato Varieties," Journal of Agricultural Science, 13 (1923)) 311-320. [8]
-
El41 ---and Tippett, L.H.C., "Limiting Forms of the Frequency Distribution of the Largest and Smallest Member of a Sample," Proceedings of the Cambridge Philosophical Society, 24 (1928), 180-90. [15] Gosset, W.S., Letters from W.S. Gosset to R.A. Fisher, 19161936, with summaries by R.A. Fisher and a foreword by L. McMullen, (2nd Ed.), Privately circulated, 1970. [l6] Russell, E. John, "Field Experiments: How They Are Made and What They Are," Journal of the Ministry of Agriculture, 32 (1926), 989-1001. ~171--, A History of Agricultural Research in Great Britain, London : Allyn and Unwin, Ltd., 1966. [IS] Wold, H., "Nonlinear Estimation by Iterative Least Squares Procedures," in F.N. David, ed., Research Papers in Statistics, Festschrift for J. Neyman, New York: John Wiley & Sons, Inc., 1966, 411-44.