Cross-coefficient panel asymmetry diagnostic

check_asymmetry() takes an N x k binary ratings matrix and returns a scalar delta_hat (in quality percentage points, "pp"): the max-min spread of the implied panel qualities across the three agreement coefficients (PABAK, mean AC1, Fleiss kappa). Each coefficient inverts to its own q_hat on the shared (q, pi_+) reference; if the calibration DGP held exactly, all three would imply the same quality, so the spread measures cross-coefficient model discordance in interpretable units of quality. (Option B, ratified 2026-07-05; the previous definition – spread of surface percentiles across four coefficients including Krippendorff alpha – ran through the retired nearest-cell percentile machinery, whose sawtooth inflated delta_hat with quantization noise. Alpha left the panel at 0.6.0; ICC never enters delta_hat, its reference being distribution-sensitive in ways the agreement family is not.)

Usage

check_asymmetry(
  ratings,
  axis = c("inter", "intra"),
  occasion = NULL,
  fit_icc = TRUE,
  ...
)

Arguments

ratings: User input: an N x k binary matrix, an N x k data.frame whose columns are 0/1 / logical / 2-level factor, or a list of two equal-length 0/1 vectors (k = 2 paired form). See ?normalize_ratings for accepted shapes.
axis: "inter" (default) or "intra". Selects the surface family.
occasion: Reserved for axis = "intra" (a vector / factor identifying viewing occasion); ignored when axis = "inter".
fit_icc: If FALSE, skip the lme4::glmer fit behind icc and drop ICC from the panel. icc never enters delta_hat, and the fit draws no random numbers, so a caller that reports only delta_hat and the implied qualities gets identical results at a fraction of the cost. The Monte Carlo null loop sets this; interactive users should not.
...: Forwarded to position_on_surface() (e.g. reference_type).

Value

An S3 object of class grass_asymmetry_panel with fields:

delta_hat: scalar implied-quality spread, in pp of quality
delta_percentile: delta_hat's percentile on the matched (k, N, q_hat) null ECDF (NA if the null is uncalibrated at the design)
flag: one of "aligned", "caution", "divergent"
matched_null: list describing the matched null cell (k, N, q, prev — the bridged true-prevalence estimate the lookup conditioned on, prev_apparent — the panel's raw positive rate, prev_bridged, q_hat_panel, n_draws, snapped, interpolated, unstable_tail), or NULL if uncalibrated
thresholds: named numeric vector of the implied (caution, divergent) pp cuts (95th/99th of the matched null)
thresholds_source: one of "matched_null_ecdf", "not_applicable_k2", "not_calibrated"
panel: data.frame with coefficient, observed, implied_q, percentile_pp (pooled percentile), clamped, in_delta_hat
notes: character vector of unique caveats from the underlying surface positioning calls (e.g. nearest-neighbor gaps, ICC unavailability, matched-null provenance)

Details

delta_hat is a split-bias detector. It fires when raters tilt in different directions across (Se, Sp) – e.g., one rater high-Se / low-Sp, another high-Sp / low-Se – because the three coefficients respond to heterogeneous per-rater behavior differently and end up implying different panel qualities. The framework's other failure mode, shared/uniform bias (every rater tilts the same direction, e.g., a panel trained on one protocol all favoring specificity over sensitivity), produces uniform degradation across coefficients: small delta_hat, low implied quality. Shared bias is detected by the panel's implied quality (and its consistency band), not by delta_hat. A divergent flag therefore identifies a specific kind of disagreement – cross-coefficient inversion from heterogeneous rater behavior – and routes the user to the per-rater pairwise PABAK matrix and pooled-reference (Se_tilde, Sp_tilde) diagnostic.

Each coefficient is positioned on its reference surface via position_on_surface(), which reports its implied q_hat. The panel diagnostic is delta_hat = max(q_hat) - min(q_hat) (in pp of quality), computed over agreement-family coefficients whose observed value sits within the achievable range of their reference surface (see Surface-envelope clamp below).

Surface-envelope clamp (v0.2.1+)

If an observed coefficient value falls outside the achievable range of its reference surface at the study's design (pi_hat, k, N), the inversion to q_hat clamps to the boundary. Including such clamped implied qualities in the max-min delta_hat would inflate the panel spread purely because of the clamp, not because the panel disagrees on quality. Since v0.2.1 the function therefore excludes clamped coefficients from delta_hat whenever at least two unclamped agreement-family coefficients remain. The affected coefficients are still shown in the returned panel data.frame with clamped = TRUE, and a note in $notes names which coefficients were excluded. This matters most often for ICC (which never enters delta_hat anyway) and at designs beyond the bundled reference range, where a coefficient clamps to the achievable boundary. If fewer than two unclamped agreement-family coefficients remain, delta_hat falls back to the raw spread including clamped values and the note records the fallback.

Flag from the matched null (v0.7.0/0.7.1)

The per-(k, N) size-alpha threshold table is retired. The flag is delta_hat's percentile on the null distribution of delta_hat at the matched (k, N, q_hat) cell of the bundled delta_null_ecdf, with the cut convention >= 95th caution, >= 99th divergent. The panel's q_hat is resolved first (median of the agreement-family implied qualities), then the matched null cell is looked up; the reported thresholds carry the implied pp cuts (95th/99th of that null) as context, and thresholds_source records how the flag was resolved. The three flags are:

aligned (below the 95th percentile of the matched null): the panel agrees on the implied quality. Any single coefficient is a stable summary; the primary coefficient (Table 2) carries the headline.
caution (>= 95th, < 99th): the panel is mildly inconsistent. Report the primary coefficient with a caution flag and the delta_hat value.
divergent (>= 99th): no single coefficient is a stable summary. Use latent_class_fit() to recover per-rater (Se, Sp) and report those instead.

The new check_asymmetry(ratings, ...) signature replaces the v0.1.x check_asymmetry(se, sp, ...) per-rater signature. For backward compatibility, calling check_asymmetry() with se = ... and sp = ... named arguments emits a one-time deprecation hint and routes the call to check_rater_asymmetry(). Supplying both ratings = ... and per-rater se = / sp = is an error.

Examples

set.seed(1)
# Build a 5x200 symmetric panel -- should print as 'aligned'.
Y <- matrix(rbinom(5 * 200, 1, 0.30), nrow = 200, ncol = 5)
check_asymmetry(Y)
#> GRASS panel asymmetry diagnostic
#> 
#>   delta_hat = 0.0 pp  (spread of the implied panel qualities)
#>   flag      = aligned  (0.5 percentile of matched null: k=5, N=200, q=0.70)
#> 
#>   panel:
#>     coefficient        observed   implied q   pooled pctile   in delta_hat
#>     pabak              0.16       0.697       33.7            yes
#>     mean_ac1           0.27       0.697       33.7            yes
#>     fleiss_kappa       0.00       0.697       33.8            yes
#>     icc                0.00       0.500       0.0            no [distribution-sensitive]
#> 
#>   Surface caveats:
#>     - Fitted-ICC F_key picked via glmer: mu_hat=-0.831, tau2_hat=0.011 -> F_key tau2=0.0625, mu=-0.847.
#>     - Fitted-ICC reference (GLMM-gap corrected) at F_key=LN_mu=-0.847_tau2=0.0625, k=5, N=200 (family=logit_normal, M1=0.303).
#>     - obs_value 0.0034 below achievable minimum (0.0834); q_hat clamped.
#>     - Delta-method SE undefined: dE/dq near zero at q_hat.
#>     - Observed value below the sampling range of the lowest calibrated quality (q = 0.55); consistency band open below the calibrated grid.
#>     - delta_hat is the implied-quality spread over the agreement family (PABAK, mean AC1, Fleiss kappa). ICC is reported on the panel but does not enter delta_hat (v0.5.0 scope: ICC's reference depends on the full subject-prevalence distribution F and does not share the (q, pi_+) sufficient statistic the agreement family does).
#>     - flag from delta_hat's percentile on the matched null (k=5, N=200, q=0.70, prev=0.05; 50,000 draws); design snapped to the calibrated grid; null interpolated between calibrated grid nodes.