Congeneric reliability

A quantity similar (but not mathematically equivalent) to congeneric reliability first appears in the appendix to McDonald's 1970 paper on factor analysis, labeled $\backslash theta$.Although McDonald, R. P. (1985). Factor analysis and related methods. Hillsdale, NJ: Lawrence Erlbaum and (1999). Test theory. Mahwah, NJ: Lawrence Erlbaum claim that {{Harvnb|McDonald|1970}} invented congeneric reliability, there is a subtle difference between the formula given there and the modern one. As discussed in {{Harvnb|Cho|Chun|2018}}, McDonald's denominator totals observed covariances, but the modern definition divides by the sum of fitted covariances. In McDonald's work, the new quantity is primarily a mathematical convenience: a well-behaved intermediate that separates two values.{{wikicite|reference=McDonald, R. P. (1970). Theoretical canonical foundations of principal factor analysis, canonical factor analysis, and alpha factor analysis. British Journal of Mathematical and Statistical Psychology, 23, 1-21. doi:10.1111/j.2044-8317.1970.tb00432.x.|ref={{harvid|McDonald|1970}}}}{{wikicite|reference=Cho, E. and Chun, S. (2018), Fixing a broken clock: A historical review of the originators of reliability coefficients including Cronbach’s alpha. Survey Research, 19(2), 23–54.|ref={{harvid|Cho|Chun|2018}}}} Seemingly unaware of McDonald's work, Jöreskog first analyzed a quantity equivalent to congeneric reliability in a paper the following year.Jöreskog, K. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36(2), 109–133. https://doi.org/10.1007/BF02291393 Jöreskog defined congeneric reliability (now labeled ρ) with coordinate-free notation, and three years later, Werts gave the modern, coordinatized formula for the same.Werts, C. E., Linn, R. L., & Jöreskog, K. G. (1974). [https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=243cf2b14d7645f41accb973c1a830b6fda6887a Intraclass reliability estimates: Testing structural assumptions]. Educational and Psychological Measurement, 34, 25–33. {{doi|10.1177/001316447403400104}}

Both of the latter two papers named the new quantity simply "reliability". The modern name originates with Jöreskog's name for the model whence he derived $\backslash rho\_\{C\}$: a "congeneric model".Graham, J. M. (2006). Congeneric and (Essentially) Tau-Equivalent Estimates of Score Reliability What They Are and How to Use Them. Educational and Psychological Measurement, 66(6), 930–944. https://doi.org/10.1177/0013164406288165Lucke, J. F. (2005). “Rassling the Hog”: The Influence of Correlated Item Error on Internal Consistency, Classical Reliability, and Congeneric Reliability. Applied Psychological Measurement, 29(2), 106–125. https://doi.org/10.1177/0146621604272739

Applied statisticians have subsequently coined many names for $\{\backslash rho\}\_\{C\}$. "Composite reliability" emphasizes that $\{\backslash rho\}\_\{C\}$ measures the statistical reliability of composite scores.Werts, C. E., Rock, D. R., Linn, R. L., & Jöreskog, K. G. (1978). A general method of estimating the reliability of a composite. Educational and Psychological Measurement, 38(4), 933–938. https://doi.org/10.1177/001316447803800412 As psychology calls "constructs" any latent characteristics only measurable through composite scores,Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302. https://doi.org/10.1037/h0040957 $\{\backslash rho\}\_\{C\}$ has also been called "construct reliability".Hair, J. F., Babin, B. J., Anderson, R. E., & Black, W. C. (2018). Multivariate data analysis (8th ed.). Cengage. Following McDonald's more recent expository work on testing theory, some SEM-based reliability coefficients, including congeneric reliability, are referred to as "reliability coefficient $\backslash omega$", often without a definition.Padilla, M. (2019). A Primer on Reliability via Coefficient Alpha and Omega. Archives of Psychology, 3(8), Article 8. https://doi.org/10.31296/aop.v3i8.125Revelle, W., & Zinbarg, R. E. (2009). Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma. Psychometrika, 74(1), 145–154. https://doi.org/10.1007/s11336-008-9102-z

File:Congeneric measurement model.png

Congeneric reliability applies to datasets of vectors: each row {{mvar|X}} in the dataset is a list {{math|X_i}} of numerical scores corresponding to one individual. The congeneric model supposes that there is a single underlying property ("factor") of the individual {{mvar|F}}, such that each numerical score {{math|X_i}} is a noisy measurement of {{mvar|F}}. Moreover, that the relationship between {{mvar|X}} and {{mvar|F}} is approximately linear: there exist (non-random) vectors {{math|λ}} and {{math|μ}} such that $$X\_i=\backslash lambda\_iF+\backslash mu\_i+E\_i\backslash text\{,\}$$ where {{math|E_i}} is a statistically independent noise term.

In this context, {{math|λ_i}} is often referred to as the factor loading on item {{mvar|i}}.

Because {{math|λ}} and {{math|μ}} are free parameters, the model exhibits affine invariance, and {{mvar|F}} may be normalized to mean {{math|0}} and variance {{math|1}} without loss of generality. The fraction of variance explained in item {{math|X_i}} by {{mvar|F}} is then simply $$\backslash rho\_i=\backslash frac\{\backslash lambda\_i^2\}\{\backslash lambda\_i^2+\backslash mathbb\{V\}[E\_i]\}\backslash text\{.\}$$ More generally, given any covector {{mvar|w}}, the proportion of variance in {{math|wX}} explained by {{mvar|F}} is $$\backslash rho=\backslash frac\{(w\backslash lambda)^2\}\{(w\backslash lambda)^2+\backslash mathbb\{E\}[(wE)^2]\}\backslash text\{,\}$$ which is maximized when {{math|w ∝ {{mathbb|E}}[EE^*]^-1λ}}.

{{math|ρ_C}} is this proportion of explained variance in the case where {{math|w ∝ [1 1 ... 1]}} (all components of {{mvar|X}} equally important): $$\backslash rho\_C\; =\; \backslash frac\{\; \backslash left(\; \backslash sum\_\{i=1\}^k\; \backslash lambda\_i\; \backslash right)^2\; \}\{\; \backslash left(\; \backslash sum\_\{i=1\}^k\; \backslash lambda\_i\; \backslash right)^2\; +\; \backslash sum\_\{i=1\}^k\; \backslash sigma^2\_\{E\_i\}\; \}$$

class="wikitable" style="text-align: center;" |+ Fitted/implied covariance matrix
! $X\_1$ ! $X\_2$ ! $X\_3$ ! $X\_4$
$X\_1$ | $10.00$
$X\_2$ | $4.42$ || $11.00$
$X\_3$ | $4.98$ || $5.71$ || $12.00$
$X\_4$ | $6.98$ || $7.99$ || $9.01$ || $13.00$
$\backslash Sigma$ | colspan="4" | $124.23\; =\; \backslash Sigma\_\{diagonal\}\; +\; 2\; \backslash times\; \backslash Sigma\_\{subdiagonal\}$

These are the estimates of the factor loadings and errors:

class="wikitable" style="text-align: center;" |+ Factor loadings and errors
! $\backslash hat\{\backslash lambda\}\_i$ ! $\backslash hat\{\backslash sigma\}^\{2\}\_\{e\_i\}$
$X\_1$ | $1.96$ || $6.13$
$X\_2$ | $2.25$ || $5.92$
$X\_3$ | $2.53$ || $5.56$
$X\_4$ | $3.55$ || $.37$
$\backslash Sigma$ | $10.30$ || $18.01$
$\backslash Sigma^2$ | $106.22$

:$\backslash hat\{\backslash rho\}\_\{C\}\; =\; \backslash frac\{\; \backslash left(\; \backslash sum\_\{i=1\}^k\; \backslash hat\{\backslash lambda\}\_i\; \backslash right)^2\; \}\{\; \backslash hat\{\backslash sigma\}^\{2\}\_\{X\}\; \}\; =\; \backslash frac\{\; 106.22\; \}\{\; 124.23\; \}\; =\; .8550$

:$\backslash hat\{\backslash rho\}\_\{C\}\; =\; \backslash frac\{\; \backslash left(\; \backslash sum\_\{i=1\}^k\; \backslash hat\{\backslash lambda\}\_i\; \backslash right)^2\; \}\{\; \backslash left(\; \backslash sum\_\{i=1\}^k\; \backslash hat\{\backslash lambda\}\_i\; \backslash right)^2\; +\; \backslash sum\_\{i=1\}^k\; \backslash hat\{\backslash sigma\}^\{2\}\_\{e\_i\}\; \}\; =\; \backslash frac\{\; 106.22\; \}\{\; 106.22\; +\; 18.01\; \}\; =\; .8550$

Compare this value with the value of applying tau-equivalent reliability to the same data.

Tau-equivalent reliability ($\backslash rho\_T$), which has traditionally been called "Cronbach's $\backslash alpha$", assumes that all factor loadings are equal (i.e. $\backslash lambda\_1=\backslash lambda\_2=...=\backslash lambda\_k$). In reality, this is rarely the case and, thus, it systematically underestimates the reliability. In contrast, congeneric reliability ($\backslash rho\_C$) explicitly acknowledges the existence of different factor loadings. According to Bagozzi & Yi (1988), $\backslash rho\_C$ should have a value of at least around 0.6.Bagozzi & Yi (1988), https://dx.doi.org/10.1177/009207038801600107 Often, higher values are desirable. However, such values should not be misunderstood as strict cutoff boundaries between "good" and "bad".Guide & Ketokivi (2015), https://dx.doi.org/10.1016/S0272-6963(15)00056-X Moreover, $\backslash rho\_\{C\}$ values close to 1 might indicate that items are too similar. Another property of a "good" measurement model besides reliability is construct validity.

A related coefficient is average variance extracted.

• [http://relcalc.blogspot.de/2016/05/how-to-obtain-and-use-relcalc.html RelCalc], tools to calculate congeneric reliability and other coefficients.
• [http://en.wikibooks.org/wiki/Handbook_of_Management_Scales Handbook of Management Scales], Wikibook that contains management related measurement models, their indicators and often congeneric reliability.

Category:Comparison of assessments

Category:Psychometrics

Category:Statistical reliability