Position an observed agreement coefficient on its DGP-calibrated surface

position_on_surface() is the Target-2 reporting primitive for the merged GRASS binary-rater-reliability paper. Given an observed coefficient value and the study design (pi_hat, k, N), it inverts the coefficient to an internal q_hat (rater operating quality on the Se = Sp diagonal under the clustered latent-class DGP), computes where the observed value sits on its reference sampling distribution (the surface percentile), and assigns a four-band adjective plus a confidence qualifier derived from the sampling-probability mass over bands.

Usage

position_on_surface(
  obs_value = NULL,
  metric,
  pi_hat = NULL,
  k = NULL,
  N = NULL,
  method = c("empirical", "delta"),
  bands = c(0.5, 0.625, 0.75, 0.875, 1),
  band_labels = c("Poor", "Moderate", "Strong", "Excellent"),
  surface_data = NULL,
  ratings = NULL,
  reference_type = c("fitted", "oracle"),
  ...
)

Arguments

obs_value

Numeric scalar. The observed agreement coefficient. Optional when ratings is supplied (auto-derived via compute_observed(metric, Y)).

metric

Character scalar. One of "pabak", "fleiss_kappa", "mean_ac1", "krippendorff_a", "icc".

pi_hat

Numeric scalar in (0, 1). The panel-identified marginal positive rate. Optional when ratings is supplied (auto-derived via mean(Y)); otherwise estimate from the rating matrix via grass_prevalence() or directly from rater marginals.

k

Integer >= 2. Number of raters. Optional when ratings is supplied (auto-derived as ncol(Y)).

N

Integer >= 1. Number of subjects. Optional when ratings is supplied (auto-derived as nrow(Y)).

method

One of "empirical" (default; uses the bundled sim-derived empirical q_hat sampling distribution) or "delta" (closed-form normal approximation from delta-method SE).

bands

Numeric vector of length 5 giving band boundaries on q. Must be strictly increasing, start at 0.5, end at 1.0. Default c(0.5, 0.625, 0.75, 0.875, 1.0).

band_labels

Character vector of length 4, one label per band. Default c("Poor", "Moderate", "Strong", "Excellent").

surface_data

Optional. A list with one or more of the following components, used when method = "empirical":

per_rep: a matrix of per-rep metric values at the closest simulated (q, pi_hat, k, N) cell – this is the empirical sampling distribution at that design. Used for the percentile of obs_value.
q_grid_per_rep: a matrix of dimension n_q x n_rep giving the empirical sampling distribution of the metric at each q-grid point, which the function uses to integrate P(q in band_j) against the inferred posterior shape of q_hat.
q_grid: numeric vector of q values for the rows of q_grid_per_rep. When surface_data is NULL and method = "empirical", the function signals an error with guidance.

ratings

Optional. From v0.2.0 this is the primary input for all metrics (not just ICC): supplying an N x k rating matrix auto-derives obs_value, pi_hat, k, and N. Accepts an N x k integer matrix of 0/1 values (rows = subjects, cols = raters), a data.frame with k rater columns, or a length-2 list of equal-length 0/1 vectors (k = 2). For metric = "icc", supplying ratings (also accepted as a long data.frame with columns subject and rating) additionally enables a glmer fit for (mu, tau2) that pins down the correct F_key for ICC inversion; without ratings, ICC falls back to a nearest-M1 F_key lookup with a prominent caveat note (tau2 is unidentified from pi_hat alone). The glmer path requires lme4 (Suggests).

reference_type

For metric = "icc" only. One of "fitted" (default; GLMM-gap-corrected reference matching what practitioners compute via glmer) or "oracle" (closed-form sigma^2_subject / (sigma^2_subject + pi^2/3) with sigma^2_subject known from F). Use "oracle" only if obs_value was computed via oracle variance decomposition (non-standard for applied work). For N beyond the fitted-reference sim range (currently N > 200), the function auto-falls-back to oracle with an explanatory note.

...

Reserved for future extension.

Value

A list of class grass_surface_position with fields:

observed_value – echo of obs_value
metric – echo of metric
design – list(pi_hat, k, N)
q_hat – inverted rater quality (internal scaffold)
se_q_hat – delta-method SE of q_hat
percentile – surface percentile of obs_value in [0, 1]
band_probabilities – named numeric vector of P(q in band_j)
modal_band – integer index 1..4 of the highest-probability band
modal_band_label – character adjective at the modal band
confidence – "decisive" / "moderate" / "weak"
sampling_method – which method was used
notes – character vector of caveats (e.g. nearest-neighbor gaps)

Details

The function implements the reporting convention locked on 2026-04-22: the practitioner cites the observed coefficient, its surface percentile, the band adjective, and the confidence qualifier. q_hat survives as internal scaffolding (surface parameterization and delta-method SE carrier) rather than as the labeled output.

Label construction (Sec.0.6.5)

Four adjacent-exclusive bands partition the above-chance quality range q in [0.5, 1.0] on equal width: Poor [0.5, 0.625), Moderate [0.625, 0.75), Strong [0.75, 0.875), Excellent [0.875, 1.0]. The geometric partition carries no a-priori weight; users may supply custom cut points through bands and matching band_labels.

The confidence qualifier is derived from the modal band's sampling probability p_star:

decisive – p_star >= 0.90
moderate – p_star in [0.60, 0.90)
weak – p_star < 0.60

Sampling-probability construction

Two methods are implemented.

Delta method (method = "delta", default): assumes q_hat is approximately normal with mean q_hat and standard deviation se_q_hat obtained from the delta-method SE of the monotone metric inversion, and computes P(q in band_j) by pnorm() on the band boundaries. This is the summary-stats-only fallback; it does not require the full simulation outputs.

Empirical method (method = "empirical", default): uses the 2,000-replication sampling distribution from multirater_sim_v3 (the symmetric-rater reference sim backing Paper 2) to form the empirical sampling distribution of q_hat_rep at the nearest matched scenario cell (metric, F_key nearest to pi_hat, k nearest, N nearest, q_true nearest to q_hat). The full per-rep data is not bundled (~300 MB); instead the package ships a precomputed 13-point empirical-quantile summary of the per-cell q_hat_rep vector. Band probabilities P(q_hat_rep in band_j) are recovered by piecewise-linear interpolation on the empirical CDF. Callers who want to pass their own per-rep vector may supply surface_data$per_rep or surface_data$q_grid_per_rep; those paths continue to override the bundled lookup. When the design (pi_hat, k, N) falls outside the simulated grid, nearest-neighbour clamping is applied and flagged in notes.

Internal reference-surface arithmetic

Under the clustered latent-class DGP with symmetric raters (Se = Sp = q), the large-N closed forms for PABAK, Fleiss kappa, AC1, and Krippendorff's alpha depend on q and the marginal positive rate pi_+ only. This function uses pi_hat as a plug-in for pi_+ and inverts the observed value on a 501-point q-grid on [0.5, 1] (matching paper2/code/12_q_inversion.R resolution). ICC requires the full subject-prevalence distribution F; in the absence of surface_data containing an ICC lookup, ICC requests fall through to a warning-noted delta-method approximation using the caller-supplied q_hat_override / se_q_hat_override if present, or stop with a clear message.

Ratings-primary path

From v0.2.0, the preferred entry point is to hand the rating matrix directly: position_on_surface(ratings = Y, metric = "pabak"). When ratings is supplied, the function auto-derives obs_value (via compute_observed(metric, Y)), pi_hat (mean(Y)), k (ncol(Y)), and N (nrow(Y)); any of those four arguments still supplied by the caller wins. This collapses the audit-style scalar-input path used in v0.1.x to a single matrix argument while keeping the scalar path callable for reproducibility checks. ratings accepts an N x k integer matrix in {0, 1}, a data.frame with k rater columns, or a length-2 list of equal-length 0/1 vectors (k = 2). Round-trip equality with the scalar path is a tested invariant.

Examples

# Ratings-primary path: just hand it the matrix.
set.seed(1)
Y <- matrix(rbinom(1000, 1, 0.3), nrow = 200, ncol = 5)
position_on_surface(ratings = Y, metric = "pabak")
#> grass surface-position report (Target-2 reporting convention)
#>   metric               : pabak
#>   observed value       : 0.156
#>   design (pi_hat,k,N)  : (0.304, 5, 200)
#>   q_hat (scaffold)     : 0.697 +/- 0.024
#>   surface percentile   : 0.475
#>   band probabilities   :
#>     Poor       0.000
#>     Moderate   1.000 <- modal
#>     Strong     0.000
#>     Excellent  0.000
#>   label                : Moderate (decisive)
#>   sampling method      : empirical

# Equivalent scalar-input path (audit):
position_on_surface(
  obs_value = 2 * mean(Y[, 1] == Y[, 2]) - 1,  # PABAK on first pair
  metric = "pabak", pi_hat = mean(Y), k = ncol(Y), N = nrow(Y)
)
#> grass surface-position report (Target-2 reporting convention)
#>   metric               : pabak
#>   observed value       : 0.110
#>   design (pi_hat,k,N)  : (0.304, 5, 200)
#>   q_hat (scaffold)     : 0.666 +/- 0.029
#>   surface percentile   : 0.767
#>   band probabilities   :
#>     Poor       0.163
#>     Moderate   0.837 <- modal
#>     Strong     0.000
#>     Excellent  0.000
#>   label                : Moderate (moderate)
#>   sampling method      : empirical

# Delta-method positioning -- no sim data needed, summary-stats only.
r <- position_on_surface(
  obs_value = 0.62,
  metric    = "pabak",
  pi_hat    = 0.42,
  k         = 5,
  N         = 50
)
r$modal_band_label
#> [1] "Excellent"
r$confidence
#> [1] "moderate"

# Fleiss kappa at imbalanced prevalence.
position_on_surface(
  obs_value = 0.18,
  metric    = "fleiss_kappa",
  pi_hat    = 0.08,
  k         = 3,
  N         = 200
)
#> grass surface-position report (Target-2 reporting convention)
#>   metric               : fleiss_kappa
#>   observed value       : 0.180
#>   design (pi_hat,k,N)  : (0.080, 3, 200)
#>   q_hat (scaffold)     : 0.935 +/- 0.013
#>   surface percentile   : 0.233
#>   band probabilities   :
#>     Poor       0.000
#>     Moderate   0.000
#>     Strong     0.000
#>     Excellent  1.000 <- modal
#>   label                : Excellent (decisive)
#>   sampling method      : empirical