Estimates Conger / Fleiss / Brennan-Prediger weighted agreement coefficients for raw ratings with missing entries. Pick the estimator with `estimator`:
`"ipw"` — inverse-probability-weighted moment estimator for categorical ratings under MCAR, consistent under non-exchangeable raters with rater-varying observation rates;
`"cat_fiml"` — saturated-multinomial full-information maximum likelihood for categorical ratings under ignorable missingness;
`"pairwise"` — pairwise-available moment estimator for the quadratically weighted (scored) coefficient under MCAR;
`"nt_fiml"` — robust normal-theory FIML for the quadratically weighted coefficient under ignorable missingness.
`"ipw"` and `"cat_fiml"` take integer category codes and the nominal, linear, or quadratic `weight`. `"pairwise"` and `"nt_fiml"` are quadratic by construction: they treat `x` as numeric scores (pass scores directly, or integer codes with `values`) and require `weight = "quadratic"`.
Two generalizations are reached through the same call. Set `g > 2` for the Frechet / Hubert g-wise (multirater) kernels, which report the Conger-type (distinct rater combinations) and Fleiss-type coefficients. Pass a subjects-by-raters-by-features array to score vector-valued ratings with a component-separable loss. See the arguments for the estimator / weight combinations supported in each mode.
For scalar fixed-rater coefficients with missing entries, the observed-data pattern must identify the requested functional. For ordinary pairwise kappa (`g = 2`, and all quadratic requests), every rater must be observed at least once and every rater pair must be jointly observed by at least one subject. For non-quadratic `g > 2`, every requested rater g-tuple must be jointly observed by at least one subject. Violations are reported as non-identification before the estimator runs.
Arguments
- x
A subjects-by-raters matrix or data frame of scalar ratings, or a subjects-by-raters-by-features 3-D array of vector-valued ratings (the array form is auto-detected). Integer category codes for `"ipw"` / `"cat_fiml"`; numeric scores for `"pairwise"` / `"nt_fiml"`. `NA` marks missing entries.
- estimator
One of `"ipw"`, `"cat_fiml"`, `"pairwise"`, `"nt_fiml"`. Vector-valued ratings support `"pairwise"` and `"ipw"` only; continuous g-wise kernels (`weight = "linear"`, `g > 2`) support `"pairwise"` and `"ipw"` only.
- weight
Weighting scheme for `"ipw"` / `"cat_fiml"`: `"nominal"` (the default), `"linear"`, or `"quadratic"`. `"pairwise"` / `"nt_fiml"` require `"quadratic"`. For `g > 2` only `"nominal"`, `"linear"`, `"hubert"`, and `"quadratic"` are defined, where `"hubert"` is the all-raters-equal multirater kernel (g > 2 only). For vector-valued ratings `weight` selects the component loss (`"nominal"` -> Hamming, `"linear"` -> L1, `"quadratic"` -> squared).
- g
Arity of the multirater disagreement kernel. `g = 2` (the default) is ordinary pairwise kappa; `g > 2` uses the Frechet / Hubert g-wise family. `weight = "quadratic"` is g-invariant, so `g` is ignored there and the cheap closed form is used. Ignored for vector-valued ratings.
- values
Optional numeric vector of category scores used by the metric weightings (and by the quadratic / continuous g-wise estimators to map integer codes to scores). Defaults to the sorted unique observed categories.
- em_options
Named list of EM options for the likelihood estimators (`"cat_fiml"`, `"nt_fiml"`). For `"cat_fiml"`: `tol`, `max_iter`, `prune_tol`, `start_alpha`, `info_rcond`, and `flatten` (total Dirichlet pseudo-mass spread over the complete pattern table; any positive value selects the unique interior posterior mode when the saturated likelihood is flat, shrinking the fitted table toward uniform with weight `flatten / (n + flatten)`; `0`, the default, is strict ML). Flattening is a uniqueness device, not an inference upgrade: it leaves the point estimate essentially unchanged but makes the reported standard errors conservative (roughly 50% too wide in calibration). Pass any subset.
- ...
Mode-specific extras: `feature_weights` and `loss` (e.g. `"rms"`) for vector-valued ratings, and `max_chance_tuples` (cap on the number of chance tuples enumerated) for `g > 2`.
Value
An object of class `misskappa_estimate` carrying the named coefficient estimates and their asymptotic covariance matrix. Methods: `print`, `coef`, `vcov`, `confint`, `as.data.frame`, and `stats::influence`. The object also carries a `psi` component — the n-by-K matrix of per-subject influence functions, satisfying `vcov == crossprod(psi) / n^2` — for power users who want to build their own contrasts or joint tests.
Details
The public R API deliberately exposes only the main weighting schemes used in current agreement work: `"nominal"`, `"linear"`, and `"quadratic"` (plus the `"hubert"` g-wise kernel for `g > 2`). Older Krippendorff / irrCAC-style categorical schemes (`"ordinal"`, `"radical"`, `"ratio"`, `"circular"`, `"bipolar"`) remain in the internal C++ layer and Rcpp glue for validation and parity work, but are unsupported and not part of the recommended R interface.
References
Krippendorff, K. (2011). Computing Krippendorff's alpha-reliability.
Gwet, K. L. (2019). irrCAC: Computing chance-corrected agreement coefficients among raters.
Examples
# Categorical ratings with missing entries (Gwet 2014): the
# inverse-probability-weighted estimator under MCAR, with linear weights.
kappa(dat.gwet2014, estimator = "ipw", weight = "linear")
#> Warning: rater pair(s) rater4-rater5 co-observed by only one subject; the corresponding pairwise covariance is degenerate and the standard error unreliable.
#> misskappa: estimator=ipw, weight=linear
#> estimate se lower upper
#> Conger 0.5879 0.0772 0.4365 0.7393
#> Fleiss 0.5858 0.0774 0.4341 0.7375
#> Brennan-Prediger 0.6397 0.0767 0.4894 0.7900
# Scored coefficient (quadratic loss) via the pairwise-available moment
# estimator; treats the columns as numeric scores.
kappa(dat.zapf2016, estimator = "pairwise")
#> misskappa: estimator=pairwise, weight=quadratic
#> estimate se lower upper
#> Conger 0.8985 0.0278 0.8439 0.953
#> Fleiss 0.8984 0.0279 0.8437 0.953
# g-wise (multirater) nominal kappa over triples of raters.
kappa(dat.gwet2014, estimator = "ipw", weight = "nominal", g = 3)
#> Warning: rater 3-tuple(s) rater1-rater4-rater5, rater2-rater4-rater5, rater3-rater4-rater5 co-observed by only one subject; the corresponding observed-disagreement term is degenerate and the standard error unreliable.
#> misskappa: estimator=ipw, weight=nominal, g=3
#> estimate se lower upper
#> Conger 0.4369 0.0850 0.2703 0.6034
#> Fleiss 0.4308 0.0861 0.2620 0.5997
# Vector-valued ratings: a subjects-by-raters-by-features array. Each of the
# four expert observers classifies crackles at six chest sites, so a rating
# is a six-vector; the default Hamming loss counts disagreeing sites.
kappa(dat.vanbelle2019[, 1:4, ], estimator = "ipw")
#> misskappa: estimator=ipw, weight=hamming
#> estimate se lower upper
#> Conger 0.5192 0.0794 0.3635 0.6748
#> Fleiss 0.5150 0.0813 0.3557 0.6743