Estimates Fleiss and Brennan-Prediger coefficients when each row is a
subject and each column a category, with x[i, k] the number of raters who
put subject i in category k. Rows may have different totals, which is
how missing ratings show up in this format. Raters are anonymous, so they
are treated as exchangeable and Conger is not reported. For rater-identified
data use agreement(); ratings_to_counts() converts it to this format.
"pairwise"(default, recommended): the moment estimator of Fleiss and Cuzick (1979), extended to linear and quadratic weights. It averages agreement over the rater pairs within each subject, the counts-format analogue of"pairwise"inagreement(). Consistent under MCAR, anyweight."cat_fiml": full-information maximum likelihood over the category compositions, consistent under MAR. It is the"cat_fiml"ofagreement()with the raters constrained to be exchangeable (the same likelihood), which is why it shares the name."nt_fiml_cs": normal-theory compound-symmetry FIML, quadratic weight only.
Rows with fewer than two ratings carry no agreement information; at least one row needs two or more.
Arguments
- x
A subjects-by-categories matrix of non-negative counts.
- estimator
"pairwise"(default),"cat_fiml", or"nt_fiml_cs".- weight
"nominal"(default),"linear", or"quadratic"."nt_fiml_cs"always uses"quadratic".- values
Optional numeric category scores for the linear and quadratic weights. Defaults to
1:C.- r_total
For
"cat_fiml": the number of raters each subject would have with no missing ratings. Defaults to the largest row total.- em_options
Named list of EM settings for the likelihood estimators:
tolandmax_iter, and for"cat_fiml"alsoprune_tol,start_alpha, andinfo_rcond.
Value
A misskappa_estimate with Fleiss and Brennan-Prediger; see
agreement().
References
Fleiss, J. L., & Cuzick, J. (1979). The reliability of dichotomous judgments: Unequal numbers of judges per subject. Applied Psychological Measurement, 3(4), 537-542.
Moss, J. (in preparation). Quadratically weighted agreement coefficients.
Examples
# Fleiss (1971): 30 subjects, 6 raters, 5 diagnostic categories.
agreement_counts(dat.fleiss1971)
#> misskappa: estimator = pairwise, weight = nominal, n = 30
#> estimate se lower upper
#> Fleiss 0.4302 0.0533 0.3258 0.5347
#> Brennan-Prediger 0.4444 0.0542 0.3382 0.5507
# Ordered categories with quadratic weights.
agreement_counts(dat.fleiss1971, weight = "quadratic")
#> misskappa: estimator = pairwise, weight = quadratic, n = 30
#> estimate se lower upper
#> Fleiss 0.2841 0.1093 0.0698 0.4983
#> Brennan-Prediger 0.3339 0.1019 0.1342 0.5336
