# Each subject has exactly three of the six ratings.
table(rowSums(is.finite(dat.tenhove2025)))
#>
#> 3
#> 60
# Off-diagonal zeros are rater pairs that never rate the same subject.
co <- crossprod(is.finite(dat.tenhove2025))
diag(co) <- 0
co
#> rater1 rater2 rater3 rater4 rater5 rater6
#> rater1 0 20 20 0 20 20
#> rater2 20 0 20 0 0 0
#> rater3 20 20 0 20 20 0
#> rater4 0 0 20 0 20 0
#> rater5 20 0 20 20 0 20
#> rater6 20 0 0 0 20 0The package distinguishes missing completely at random (MCAR) estimators from missing at random (MAR) likelihood estimators. In raw rater matrices, NA means the rating was not observed. In the C++ API, categorical missingness is encoded by misskappa::na_code.
The available-case estimator uses the observed rater pairs directly:
\[ \widehat D_o = \frac{\sum_i \sum_{r<s} R_{ir}R_{is} \ell_{X_{ir},X_{is}}} {\sum_i \sum_{r<s} R_{ir}R_{is}}, \]
where \(R_{ir}=1\) when the rating is observed. This targets the pairwise-observed estimand unless missingness is MCAR.
The IPW estimator reweights observed pairs by estimated observation probabilities. In schematic form,
\[ \widehat D_o^{\mathrm{IPW}} = \frac{1}{n}\sum_i \sum_{r<s} \frac{R_{ir}R_{is}}{\widehat\pi_r\widehat\pi_s} \ell_{X_{ir},X_{is}}, \]
with corresponding weighted marginal estimates for chance disagreement. This is the MCAR correction used for the raw categorical estimator.
The FIML estimator fits a categorical full-data distribution under MAR by EM. For observed pattern \(o_i\), the likelihood contribution is
\[ L_i(\theta) = \sum_{x_{\mathrm{mis}}} p_\theta(x_{\mathrm{obs},i}, x_{\mathrm{mis}}), \]
and agreement coefficients are computed from the fitted full-data distribution. The implementation returns Wald covariance estimates through Louis’ observed information when available.
For scored (quadratically weighted) ratings the normal-theory FIML fits a Gaussian mean and covariance by EM and reads the coefficient off the fitted moments with a sandwich delta-method standard error. Two covariance models are available, selected by the estimator name: "nt_fiml" fits a saturated covariance, while "nt_fiml_cs" fits the compound-symmetry \(\Sigma = \psi J + e I\) (the fixed-rater variance-component model, with free rater means). Compound symmetry needs only a connected co-observation graph and is consistent and slightly more efficient when that structure holds, and is robust under MCAR even when the covariance is misspecified. The saturated model is the safer choice under MAR when the covariance may depart from compound symmetry.
Gwet’s estimator is retained for comparison with the agreement-coefficient literature. It uses its own chance-disagreement convention and should be interpreted as a comparator rather than the default missing-data target.
Compound symmetry on a planned-incomplete design
The saturated coefficients ("pairwise", "nt_fiml", "cat_fiml") are functions of every pairwise second moment, so they are identified only when every rater pair is jointly observed by some subject, a complete co-observation graph. The compound-symmetry estimators need only a connected graph, and that is the whole point of a planned-incomplete design: you can rate each subject with a rotating subset of raters and still recover the coefficient.
The synthetic dat.tenhove2025 data (60 subjects, 6 raters, ordinal 1–5) is such a design, in the style of the planned-incomplete ICC studies of ten Hove et al. (2025). Each subject is rated by three of the six raters in overlapping panels, so the graph is connected but some pairs (raters 2 and 4, for example) never co-occur:
Asking for a saturated coefficient on this pattern stops with an identifiability error before the estimator runs:
kappa(dat.tenhove2025, estimator = "nt_fiml")
#> Error:
#> ! rater pair(s) rater1-rater4, rater2-rater4, rater2-rater5, rater2-rater6, rater3-rater6, rater4-rater6 never jointly observed, so Conger's kappa (and Brennan-Prediger) is not identified from this missing-data pattern (the co-observation graph is incomplete). Drop an offending rater, or provide data in which every rater pair is co-observed.The compound-symmetry estimators fit the same data. The likelihood ("nt_fiml_cs") and its distribution-free MCAR moment companion ("pairwise_cs") agree closely:
kappa(dat.tenhove2025, estimator = "nt_fiml_cs")
#> misskappa: estimator=nt_fiml_cs, weight=quadratic
#> estimate se lower upper
#> Conger 0.6845 0.0539 0.5789 0.7902
#> Fleiss 0.6837 0.0542 0.5773 0.7900
#> Brennan-Prediger 0.7180 0.0395 0.6405 0.7955
kappa(dat.tenhove2025, estimator = "pairwise_cs")
#> misskappa: estimator=pairwise_cs, weight=quadratic
#> estimate se lower upper
#> Conger 0.6920 0.0541 0.5859 0.7981
#> Fleiss 0.6920 0.0542 0.5858 0.7983
#> Brennan-Prediger 0.7301 0.0390 0.6537 0.8065Both report the usual Conger, Fleiss, and Brennan-Prediger coefficients with sandwich standard errors, recovered from a design where the saturated estimators cannot be computed at all.