Principle of marginality
{{Short description|Statistical principle}}
{{Lead rewrite|date=December 2021}}
{{Use dmy dates|date=December 2023}}
In statistics, the principle of marginality, sometimes called hierarchical principle, is the fact that the average (or main) effects of variables in an analysis are marginal to their interaction effect—that is, the main effect of one explanatory variable captures the effect of that variable averaged over all values of a second explanatory variable whose value influences the first variable's effect.{{cite book |last1=James |first1=Gareth |last2=Witten |first2=Daniela |last3=Hastie |first3=Trevor |last4=Tibshirani |first4=Robert |title=An introduction to statistical learning: with applications in R |date=2021 |publisher=Springer |location=New York, NY |isbn=978-1-0716-1418-1 |page=103 |edition=Second |url=https://link.springer.com/book/10.1007/978-1-0716-1418-1 |access-date=29 October 2024}} The principle of marginality implies that, in general, it is wrong to test, estimate, or interpret main effects of explanatory variables where the variables interact or, similarly, to model interaction effects but delete main effects that are marginal to
them.Fox, J. [http://socserv.mcmaster.ca/jfox/Courses/SPIDA/dummy-regression-notes.pdf Regression Notes]. While such models are interpretable, they lack applicability, as they ignore the dependence of a variable's effect upon another variable's value.
Nelder{{Cite journal | last1 = Nelder | first1 = J. A. |author-link = John Nelder| title = A Reformulation of Linear Models | journal = Journal of the Royal Statistical Society | volume = 140 | issue = 1 | pages = 48–77 | doi = 10.2307/2344517 | year = 1977 | jstor = 2344517 }} (Section 2.1: The Neglect of Marginality) and VenablesVenables, W.N. (1998). [http://www.stats.ox.ac.uk/pub/MASS3/Exegeses.pdf "Exegeses on Linear Models"]. Paper presented to the S-PLUS User's Conference Washington, DC, 8–9 October 1998. have argued strongly for the importance of this principle in regression analysis.
Regression form
If two independent continuous variables, say x and z, both influence a dependent variable y, and if the extent of the effect of each independent variable depends on the level of the other independent variable then the regression equation can be written as:
:
where i indexes observations, a is the intercept term, b, c, and d are effect size parameters to be estimated, and e is the error term.
If this is the correct model, then the omission of any of the right-side terms would be incorrect, resulting in misleading interpretation of the regression results.
With this model, the effect of x upon y is given by the partial derivative of y with respect to x; this is , which depends on the specific value at which the partial derivative is being evaluated. Hence, the main effect of x – the effect averaged over all values of z – is meaningless as it depends on the design of the experiment (specifically on the relative frequencies of the various values of z) and not just on the underlying relationships. Hence:
- In the case of interaction, it is wrong to try to test, estimate, or interpret a "main effect" coefficient b or c, omitting the interaction term.See Venables, p.13: "... testing main effects in the presence of an interaction is a violation of the marginality principle".
In addition:
- In the case of interaction, it is wrong to not include b or c, because this will give incorrect estimates of the interaction.See Venables, p.14/15, about the S-Plus command
drop1, which does not drop the main effect terms from a model with interaction: "To my delight I see that marginality constraints between factor terms are by default honoured". In R, the marginality requirement of thedroptermfunction (in package MASS) is stated in the Reference Manual.The above regression model, with two independent continuous variables, is presented with a numerical example, in Stata, as Case 3 in [http://stats.idre.ucla.edu/stata/faq/what-happens-if-you-omit-the-main-effect-in-a-regression-model-with-an-interaction What happens if you omit the main effect in a regression model with an interaction?].
See also
{{Portal|Mathematics}}