A hypothetical experiment and Monte Carlo simulations were used to examine the effectiveness of statistical design of experiments methods in identifying from the experimental data the correct terms in postulated regression models for a variety of experimental conditions. Two analysis of variance techniques (components of variance and pooled mean square error) combined with F-test statistics were investigated with first-order and second-order regression models. It was concluded that there are experimental conditions for which one or the other of the procedures results in model identification with high confidence, but there are also other conditions in which neither procedure is successful. The ability of the statistical approaches to identify the correct models varies so drastically, depending on experimental conditions, that it seems unlikely that arbitrarily choosing a method and applying it will lead to identification of the effects that are significant with a reasonable degree of confidence. It is concluded that before designing and conducting an experiment, one should use simulations of the proposed experiment with postulated truths in order to determine which statistical design of experiments approach, if any, will identify the correct model from the experimental data with an acceptable degree of confidence. In addition, no significant change in the effectiveness of the methods in identifying the correct model was observed when systematic uncertainties of up to 10 percent in the independent variables and in the response were introduced into the simulations. An explanation is that the systematic errors in the simulation data caused a shift of the whole response surface up or down from the true value, without a significant change in shape. [S0098-2202(00)03102-3]

1.
Hicks, C. R., 1964, Fundamental Concepts in the Design of Experiments, Holt, Rinehart, and Winston, New York.
2.
Montgomery, D. C., 1997, Design and Analysis of Experiments, Fourth Edition, Wiley, New York.
3.
Brown, K. K., 1996, “Assessment of the Experimental Uncertainty Associated with Regressions,” Ph.D. dissertation, Department of Mechanical and Aerospace Engineering, University of Alabama in Huntsville.
4.
Brown
,
K. K.
,
Coleman
,
H. W.
, and
Steele
,
W. G.
,
1998
, “
A Methodology for Determining the Experimental Uncertainty Associated with Regressions
,”
ASME J. Fluids Eng.
,
120
, No.
3
, pp.
445
456
.
5.
Box
,
G. E. P.
, and
Draper
,
N. R.
,
1959
, “
A Basis for the Selection of a Response Surface Design
,”
J. Am. Stat. Assoc.
,
54
, No.
287
, pp.
622
654
.
6.
Box
,
G. E. P.
, and
Draper
,
N. R.
,
1963
, “
The Choice of a Second Order Rotatable Design
,”
Biometrika
,
50
, Parts 3 and 4, pp.
335
352
.
7.
Graybill, F. A., 1961, An Introduction to Linear Statistical Models, McGraw-Hill, New York.
8.
Coleman, H. W., and Steele, W. G., 1999, Experimentation and Uncertainty Analysis For Engineers, Second Edition, Wiley, New York.
9.
Montgomery, D. C., and Peck, E. A., 1992, Introduction to Linear Regression Analysis, Second Edition, Wiley, New York.
10.
Deaconu, S, 1997, “Statistical Design of Experiments and the Validation of Regression Models,” Masters thesis, Department of Mechanical and Aerospace Engineering, University of Alabama in Huntsville.
You do not currently have access to this content.