Position an observed agreement coefficient on its DGP-calibrated surface

position_on_surface() is the Target-2 reporting primitive for the merged GRASS binary-rater-reliability paper. Given an observed coefficient value and the study design (pi_hat, k, N), it inverts the coefficient to an implied panel quality q_hat (rater operating quality on the Se = Sp diagonal under the clustered latent-class DGP) and evaluates the observed value against EVERY calibrated quality level at the matched design – a sweep – from which it derives three read-outs: the pooled percentile, the consistency band on quality, and the p(q) sweep profile.

Usage

position_on_surface(
  obs_value = NULL,
  metric,
  pi_hat = NULL,
  k = NULL,
  N = NULL,
  method = c("empirical", "delta"),
  surface_data = NULL,
  ratings = NULL,
  reference_type = c("fitted", "oracle"),
  ...
)

Arguments

obs_value

Numeric scalar. The observed agreement coefficient. Optional when ratings is supplied (auto-derived via compute_observed(metric, Y)).

metric

Character scalar. One of "pabak", "fleiss_kappa", "mean_ac1", "krippendorff_a", "icc".

pi_hat

Numeric scalar in (0, 1). The panel-identified marginal positive rate. Optional when ratings is supplied (auto-derived via mean(Y)); otherwise estimate from the rating matrix via grass_prevalence() or directly from rater marginals.

k

Integer >= 2. Number of raters. Optional when ratings is supplied (auto-derived as ncol(Y)).

N

Integer >= 1. Number of subjects. Optional when ratings is supplied (auto-derived as nrow(Y)).

method

One of "empirical" (default; uses the bundled sim-derived empirical q_hat sampling distribution) or "delta" (closed-form normal approximation from delta-method SE).

surface_data

Optional. A list with one or more of the following components, used when method = "empirical":

per_rep: a vector of per-rep metric values at the caller's own (q, pi_hat, k, N) cell – an empirical sampling distribution at that design. Honored for reproducibility audits; yields a plain cohort percentile with no sweep or consistency band.
q_grid_per_rep, q_grid: legacy per-q-grid empirical inputs. Retained for backward compatibility but no longer consumed internally (the sweep convention consults the bundled per-cell quantile surface); supplying them draws a note. When surface_data is NULL and method = "empirical", the function uses the bundled empirical q_hat surface, falling back to the delta-method sweep when that surface is unavailable at the design.

ratings

Optional. From v0.2.0 this is the primary input for all metrics (not just ICC): supplying an N x k rating matrix auto-derives obs_value, pi_hat, k, and N. Accepts an N x k integer matrix of 0/1 values (rows = subjects, cols = raters), a data.frame with k rater columns, or a length-2 list of equal-length 0/1 vectors (k = 2). For metric = "icc", supplying ratings (also accepted as a long data.frame with columns subject and rating) additionally enables a glmer fit for (mu, tau2) that pins down the correct F_key for ICC inversion; without ratings, ICC falls back to a nearest-M1 F_key lookup with a prominent caveat note (tau2 is unidentified from pi_hat alone). The glmer path requires lme4 (Suggests).

reference_type

For metric = "icc" only. One of "fitted" (default; GLMM-gap-corrected reference matching what practitioners compute via glmer) or "oracle" (closed-form sigma^2_subject / (sigma^2_subject + pi^2/3) with sigma^2_subject known from F). Use "oracle" only if obs_value was computed via oracle variance decomposition (non-standard for applied work). For N beyond the fitted-reference sim range (currently N > 200), the function auto-falls-back to oracle with an explanatory note.

...

Reserved for future extension.

Value

A list of class grass_surface_position with fields:

observed_value – echo of obs_value
metric – echo of metric
design – list(pi_hat, k, N)
q_hat – implied panel quality (coefficient inverted on the reference curve); the point estimate the consistency band surrounds
se_q_hat – delta-method SE of q_hat
percentile – POOLED percentile in [0, 1]: the observed coefficient's position within the design's full achievable range (trapezoid-weighted mixture over every calibrated quality level). Monotone in obs_value by construction.
percentile_basis – provenance string for percentile
band – 95% test-inversion consistency band on quality: list(lo, hi, level, open_low, open_high, note). The quality levels whose sampling distributions are consistent with the observed value at this design.
sweep – data.frame(q, p): the full profile p(q) = P(coefficient <= obs_value | quality q, this design)
sampling_method – which method was used
reference_used – which reference produced the curve
notes – character vector of caveats (e.g. nearest-neighbor gaps)

Details

The function implements the v0.7.1 sweep convention (ratified 2026-07-05): the practitioner cites the observed coefficient, its pooled percentile (position within the design's achievable range), and the consistency band on panel quality. q_hat is promoted to the card via the consistency band; it also carries the surface parameterization and delta-method SE. The stipulated four-band adjective (Poor/Moderate/Strong/Excellent) and the modal-band confidence qualifier (decisive/moderate/weak) are retired.

The three read-outs of the sweep

At the matched design (F/pi_hat, k, N) the observed coefficient is evaluated against every calibrated quality level q with no cell selection and no q snapping. Write p(q) = P(coefficient <= obs_value | panel of quality q, this design).

percentile – the pooled percentile: a trapezoid-weighted average of p(q) over the calibrated quality axis (trapezoid because the grid is non-uniform). It reads as the observed coefficient's position within the design's full achievable agreement range and is monotone in obs_value by construction. Returned in [0, 1]; callers such as grass_report() print it on the [0, 100] scale. This replaces the retired nearest-q_hat-cell percentile, whose cohort selection by a statistic derived from the coefficient made it a non-monotone sawtooth (panel review 2026-07-05).
band – the 95% test-inversion consistency band on quality: the quality levels q with 0.025 <= p(q) <= 0.975, endpoints interpolated where p(q) crosses 0.975 (lower) and 0.025 (upper). Open-ended at the grid boundary is reported with a boundary flag.
sweep – the full data.frame(q, p) profile, the object the sweep-ridgeline graphic renders.

Sweep construction

Two methods are implemented.

Empirical method (method = "empirical", default): at the matched (F_key, k, N) cell of the bundled empirical_q_hat_surface, ranks q_hat within each calibrated quality cell's q_hat_rep distribution (monotone-equivalent to ranking obs_value within the cell's coefficient distribution). The full per-rep data is not bundled (~300 MB); the package ships a precomputed multi-point empirical-quantile summary per cell. The whole quality axis is consulted – there is deliberately no q selection. When the design (pi_hat, k, N) falls outside the simulated grid, nearest-neighbour clamping is applied and flagged in notes.

Delta method (method = "delta"): the summary-stats-only fallback (also used for ICC, whose reference curve carries the F-shape conditioning). At each swept q it approximates the sampling distribution as Normal(E[metric](q), sd_metric(q)) and evaluates p(q) = pnorm(obs_value; mean, sd) on the calibrated q axis. A caller-supplied surface_data$per_rep single-cohort vector is still honored for reproducibility audits, yielding a plain cohort percentile with no sweep or band.

Internal reference-surface arithmetic

Under the clustered latent-class DGP with symmetric raters (Se = Sp = q), the large-N closed forms for PABAK, Fleiss kappa, AC1, and Krippendorff's alpha depend on q and the marginal positive rate pi_+ only. This function uses pi_hat as a plug-in for pi_+ and inverts the observed value on a 501-point q-grid on [0.5, 1] (matching paper2/code/12_q_inversion.R resolution). ICC requires the full subject-prevalence distribution F; in the absence of surface_data containing an ICC lookup, ICC requests fall through to a warning-noted delta-method approximation using the caller-supplied q_hat_override / se_q_hat_override if present, or stop with a clear message.

Ratings-primary path

From v0.2.0, the preferred entry point is to hand the rating matrix directly: position_on_surface(ratings = Y, metric = "pabak"). When ratings is supplied, the function auto-derives obs_value (via compute_observed(metric, Y)), pi_hat (mean(Y)), k (ncol(Y)), and N (nrow(Y)); any of those four arguments still supplied by the caller wins. This collapses the audit-style scalar-input path used in v0.1.x to a single matrix argument while keeping the scalar path callable for reproducibility checks. ratings accepts an N x k integer matrix in {0, 1}, a data.frame with k rater columns, or a length-2 list of equal-length 0/1 vectors (k = 2). Round-trip equality with the scalar path is a tested invariant.

Examples

# Ratings-primary path: just hand it the matrix.
set.seed(1)
Y <- matrix(rbinom(1000, 1, 0.3), nrow = 200, ncol = 5)
position_on_surface(ratings = Y, metric = "pabak")
#> grass surface-position report (sweep convention, v0.7.1)
#>   metric               : pabak
#>   observed value       : 0.156
#>   design (pi_hat,k,N)  : (0.304, 5, 200)
#>   implied quality q_hat: 0.697 +/- 0.024
#>   pooled percentile    : 33.7 (of the design's achievable range)
#>   consistency band     : consistent with panel quality 0.65-0.75 (95%)
#>   sampling method      : empirical

# Equivalent scalar-input path (audit):
position_on_surface(
  obs_value = 2 * mean(Y[, 1] == Y[, 2]) - 1,  # PABAK on first pair
  metric = "pabak", pi_hat = mean(Y), k = ncol(Y), N = nrow(Y)
)
#> grass surface-position report (sweep convention, v0.7.1)
#>   metric               : pabak
#>   observed value       : 0.110
#>   design (pi_hat,k,N)  : (0.304, 5, 200)
#>   implied quality q_hat: 0.666 +/- 0.029
#>   pooled percentile    : 26.3 (of the design's achievable range)
#>   consistency band     : consistent with panel quality 0.61-0.72 (95%)
#>   sampling method      : empirical

# Scalar path -- the three read-outs of the sweep convention.
r <- position_on_surface(
  obs_value = 0.62,
  metric    = "pabak",
  pi_hat    = 0.42,
  k         = 5,
  N         = 50
)
r$percentile   # pooled percentile of the achievable range
#> [1] 0.7681099
r$band         # consistency band on panel quality
#> $lo
#> [1] 0.8501462
#> 
#> $hi
#> [1] 0.9342085
#> 
#> $level
#> [1] 0.95
#> 
#> $open_low
#> [1] FALSE
#> 
#> $open_high
#> [1] FALSE
#> 
#> $note
#> NULL
#> 
head(r$sweep)  # the full p(q) profile
#>      q p
#> 1 0.55 1
#> 2 0.60 1
#> 3 0.65 1
#> 4 0.70 1
#> 5 0.75 1
#> 6 0.80 1

# Fleiss kappa at imbalanced prevalence.
position_on_surface(
  obs_value = 0.18,
  metric    = "fleiss_kappa",
  pi_hat    = 0.08,
  k         = 3,
  N         = 200
)
#> grass surface-position report (sweep convention, v0.7.1)
#>   metric               : fleiss_kappa
#>   observed value       : 0.180
#>   design (pi_hat,k,N)  : (0.080, 3, 200)
#>   implied quality q_hat: 0.935 +/- 0.013
#>   pooled percentile    : 87.6 (of the design's achievable range)
#>   consistency band     : consistent with panel quality 0.90-0.97 (95%)
#>   sampling method      : empirical