| block | cells | families | errors | G | signal | truth | D | model | covariate | basis |
|---|---|---|---|---|---|---|---|---|---|---|
| core | 42 | Gaussian, Poisson, binary | iid, OU, smooth | 40, 100 | low, mid, high | smooth | 61 | ff(X) | rich | default |
| warp | 12 | Gaussian, Poisson, binary | misreg. | 40, 100 | high | smooth, wiggly | 61 | ff(X) | rich | default |
| dense_grid | 9 | Gaussian, Poisson, binary | iid, OU, smooth | 100 | mid | smooth | 241 | ff(X) | rich | default |
| rough_truth | 14 | Gaussian, Poisson, binary | iid, smooth | 100 | low, mid, high | wiggly | 61 | ff(X) | rich | default |
| term_type | 9 | Gaussian, Poisson, binary | iid, OU, smooth | 100 | mid | smooth | 61 | f(x,t) | rich | default |
| families | 8 | scaled t, beta, negative binomial, negative binomial (fitted as Poisson) | iid, smooth | 100 | mid | smooth | 61 | ff(X) | rich | default |
| ar1_home | 1 | Gaussian | AR(1) | 100 | mid | smooth | 61 | ff(X) | rich | default |
| oscillating | 3 | Gaussian, Poisson, binary | sign-chg. | 100 | mid | smooth | 61 | ff(X) | rich | default |
| warp_ar1 | 1 | Gaussian | misreg. | 100 | high | wiggly | 61 | ff(X) | rich | default |
| heteroskedastic | 6 | Gaussian | var(t), var(z), var(t,z) | 40, 100 | mid | smooth | 61 | ff(X) | rich | default |
| lowrank_covariate | 6 | Gaussian, Poisson, binary | iid, smooth | 100 | mid | smooth | 61 | ff(X) | lowrank | default |
| basis_size | 12 | Gaussian, Poisson, binary | iid, smooth | 100 | mid | smooth | 61 | ff(X) | rich | large, xlarge |
Function-on-function regression under within-curve dependence: detailed results
Synthetic study (123 cells) and plasmode study (9 datasets), with a cross-study assessment
1 Notation and reading guide
Fits. REML: the ordinary penalized fit; NCV: the same model with smoothing parameters chosen by curve-blocked neighbourhood cross-validation (one curve left out at a time). Intervals. model-based: the fit’s own Bayesian covariance; CL2: curve-clustered sandwich covariance with the exact Bell–McCaffrey leverage adjustment; bias-aware: the NCV fit’s CL2 interval widened in quadrature by the NCV–REML difference δ; hybrid: the NCV estimate with the REML fit’s CL2 standard error; AR(1): a working AR(1) error model with ρ profiled (definitions in Section 3.1.1 and Section 3.6). Recipes. REML + CL2 (the recommended recipe; “fallback” in the recommendation rule) and NCV + CL2 + bias allowance (“proposal”). Estimands. the conditional mean E(Y | X) of a frozen test cohort on the response scale, the bivariate coefficient surface β(s,t) of the functional covariate term ff(X) (or f(x,t) of a scalar-covariate smooth), the functional intercept α(t) and the scalar-covariate effect γ(t). Coverage is the average over the evaluation grid of pointwise 95% intervals; the 5% quantile of pointwise coverage is the lower tail over grid points. CE (calibration error): root-mean-square undercoverage over a pool of cells with each cell’s Monte Carlo variance subtracted; a recipe is adequate if CE ≤ 2 pp. Rule classes per dataset: P = proposal, F = fallback, E = practically equivalent (simpler recipe), N = neither adequate, U = unresolved (a bootstrap interval straddles a threshold; by the rule an N is never downgraded to U, so running and DTI β are N although their fallback CE intervals straddle 2 pp); question 1: C = CL2 beats model-based by ≥ 2 pp. “Dependent” synthetic cells have the rough (OU) or smooth (DTI-FPC) error process; “core” cells form the family × error × G × signal factorial.
Interval score. For a 95% interval \([l, u]\) and truth \(y\) the score is \((u - l) + \frac{2}{0.05}\{(l - y)\,1(y < l) + (y - u)\,1(y > u)\}\), averaged over grid points and replicates: a proper scoring rule that rewards narrow intervals and penalises each miss in proportion to its distance from the interval. Relative error of an estimate: √(grid mean of (estimate − truth)² / grid mean of truth²), the truth centred at its grid mean for the mean and α(t); detection: the share of grid points with |truth| > 25% of max |truth| where the interval excludes zero on the side of the truth (Section 3.3.4).
Interval-score ratios (bias-aware or hybrid vs REML + CL2, proposal vs fallback) are always the geometric mean over the cells concerned of the per-cell ratio of mean interval scores, the definition used by the recommendation rule; widths likewise.
Monte Carlo precision. Every cell has 200 replicates. The stored MC SE of a grid-average coverage is 0.21–0.33 pp for REML + CL2 and 0.42–0.86 pp for the model-based intervals in the dependent core cells at G = 100 (plasmode REML + CL2: 0.17–1.95 pp); the grid average is far more precise than a single grid point, whose binomial SE near 0.95 is 1.5 pp. Bootstrap intervals in brackets are 95% replicate-bootstrap intervals of pooled or paired quantities.
2 Summary of findings
Model-based intervals fail under within-curve dependence; curve-clustered CL2 on the REML fit repairs them. Synthetic: in the 14 dependent core cells at G = 100 the model-based intervals cover 0.63–0.82 for the mean and β; REML + CL2 covers 0.94–0.95. On a four times denser grid model-based coverage of β falls by a further 30.7–36.9 pp while REML + CL2 moves by at most 1.3 pp. Under independent errors at G = 100 CL2 costs at most 1.1 pp of coverage (at G = 40 up to 2.6 pp, binary mean). This holds in the synthetic study for six response families, four estimands and both term types. Plasmode (Gaussian, ff model only): pooled model-based coverage 0.51–0.75 on the six counted datasets and 0.34–0.54 on the three stress tests; REML + CL2 0.92–0.95 and 0.88–0.93. Against it: REML + CL2 covers binary α(t) and γ(t) at 0.92–0.95 (model-based 0.94–0.96 under iid; for α(t) partly a link-scale bias of the GLM intercept, studentised mean -0.46 to -0.22 under dependence), the misregistered Poisson mean at 0.90–0.91 (every recipe undercovers there), and its lower tail for the mean at G = 40 under iid errors is 0.825.
The limits of REML + CL2. On the counted datasets its β intervals are 0.6–3.1 pp short of nominal (stress tests 2.3–6.8 pp), its 5% pointwise quantile is 0.87–0.90 for β (synthetic dependent cells at G = 100: 0.91–0.93), and at G = 40 it loses a further 0.4–1.7 pp (synthetic dependent cells: 0.92–0.94 at G = 40 vs 0.94–0.95 at G = 100). Against it: under REML-fitted truths the lower tail on running, DTI and electricity is 0.845–0.860. A curve-robust term for the uncertainty of the selected smoothing parameters raises REML + CL2’s β coverage in the dependent synthetic core cells from 0.937 to 0.951 at an interval score 1.002 times CL2’s (Section 9).
Estimation: curve-blocked NCV improves the bivariate surface under dependent errors in the synthetic study; in the plasmode study the gain depends on the truth source. Synthetic, dependent core cells at G = 100: NCV/REML MSE ratio for β 0.12–0.23, mean 0.59–0.80, γ(t) 0.93–1.04, α(t) 0.83–0.98; under independent errors 0.55–1.06 for β. Plasmode (geometric mean over residual sources, G = all): β 0.04–0.78 under the NCV-fitted truth, 0.05–0.90 under the log-midpoint truth, 0.32–2.24 under the REML-fitted truth (NCV-derived truths align the target with the NCV fit; the weather data, where NCV is rougher than REML yet wins under all truths, show that this is not a pure smoothness mechanism). Against it: NCV loses for β under the REML-fitted truth on 4 of 9 datasets, and the plasmode gains on α(t) reach 0.38 on one dataset, so the gain is not confined to surfaces there.
Bias-aware NCV intervals and the NCV/REML hybrid are not reliably calibrated. The allowance inherits REML’s noise (β coverage 0.997–0.998 with the largest bases, width 0.77–0.78 times REML + CL2’s) and cannot see bias shared by both fits: under real REML-fitted truths its β coverage on the counted plasmode datasets is 0.84–0.93 (0.93–0.97 under NCV-fitted truths), the shortfall sitting at the truth’s curvature peaks. Averaged over all nine datasets, under REML-fitted truths the hybrid is worse than REML + CL2 on every metric (β coverage 0.906 vs 0.924, 5% quantile 0.67 vs 0.87, interval score 1.04 times); it is better only under NCV- and MID-derived truths, partly by overcovering (0.970, 0.962). Against it: the recommendation rule with the undercoverage-only criterion selects the bias-aware proposal for both estimands in all three families on the synthetic pool (interval-score ratios 0.69–0.92); on the plasmode datasets it does not (Section 3.4, Section 10.5). Corrections of the NCV-centred interval do not change this on real data: the one-step bias correction combined with the smoothing-parameter term covers β at 0.65–0.94 per plasmode dataset (Section 8, Section 9).
AR(1) working models fail in the plasmode study: 0.44–0.93 for the mean and β on the nine plasmode datasets (real covariates and residual curves, frozen fitted truths) (log-midpoint truth, REML residuals; synthetic misregistration: mean 0.84, β 0.86–0.88). Against it: on stationary homoskedastic synthetic errors AR(1) is conservative and competitive (0.93–0.99).
Under within-curve dependence the REML estimate of β is poor, so REML + CL2 is valid but much less informative there. Synthetic dependent cells: median relative error (√ of mean squared error over mean squared truth) of β 0.67 for REML vs 0.21 for NCV; REML + CL2 half-widths are 1.35 times the effect’s size and detect 0.54 of the clearly non-zero grid points (NCV + CL2: 0.37, 0.95). The REML/NCV gap is present at every signal level (largest at low signal). Plasmode: relative error of β 0.63–3.42 (REML) and 0.47–0.81 (NCV) per dataset; REML + CL2 detects 0.08–0.64. Against it: NCV-centred intervals do not cover reliably (item 4), so the better estimate comes without a calibrated interval. All metrics side by side, with interval scores: Section 12.
Modelling the residual covariance (pcre, GLS) does not replace CL2 on real residual curves; the curve bootstrap covers at a much higher cost. Under dependent synthetic errors the pcre curve effect and GLS with an FPCA residual covariance cover β at 0.93–0.97 and 0.93–0.97; on the plasmode comparator cells at 0.59–0.86 and 0.60–0.86, against 0.91–0.94 for REML + CL2. CL2 on the GLS or pcre fit covers β at 0.63–0.92 and 0.66–0.94 there (Section 7.1); GLS with the raw residual covariance undercovers even under independent errors (0.59–0.89); the curve bootstrap (percentile, 199 refits) covers 0.93–0.97 on the plasmode cells at a median of 535 s per data set (Section 7, Section 14). Against it: on dependent synthetic data pcre and GLS (FPCA) cover β like REML + CL2 (0.93–0.94) and estimate it with an MSE 1.41–2.53 times NCV’s.
Cost. Gaussian, G = 100, one core: REML fit 2.0 s plus 1.0 s for CL2; curve-blocked NCV fit 8.6 s; AR(1) with ρ profiled 39 s (Section 14).
3 Synthetic study
3.1 Design
The synthetic study has 123 cells in 12 blocks (Table 1), 200 replicates each, 24600 tasks, none failed. Each cell fits the model by REML and by curve-blocked NCV and scores the conditional mean of a frozen test cohort, the surface β(s,t) (or f(x,t)), α(t) and γ(t). Every cell uses the same random draws per replicate, so contrasts between cells are paired. NCV fits converged in 100.0% of Gaussian tasks; the scaled-t family’s NCV “non-convergence” (22% converged) is a flat optimum, not a wrong fit (refits from different starts agree to < 6e-04 in the coefficients).
3.1.1 Data-generating process
Models. For curves i = 1, …, G and the family’s link g: M1 (default) \(g\{\mathrm E\,Y_i(t)\} = \alpha(t) + \int X_i(s)\beta(s,t)\,ds + z_i\gamma(t)\) with \(z_i \sim N(0, 1)\); M2 (term-type block) \(g\{\mathrm E\,Y_i(t)\} = \alpha(t) + f(x_i, t) + z_i\gamma(t)\) with \(x_i \sim U(0, 1)\), a smooth effect of a scalar covariate that varies over t. pffr fits f with \(\sum_i \hat f(x_i, t) = 0\) at every t, so f is scored against its truth centred on each data set’s own \(x_i\), and α is not scored in M2.
Grids. Responses are generated on 241 equidistant points in t ∈ [0, 1] and the functional covariate on 51 points in s; the fitted grid keeps every fourth t point (D = 61) or all of them (D = 241, dense-grid block), so the grids are nested. Estimands are evaluated on fixed coarser grids, identical for every D: 31 points in t, 26 in s (β), 19 points x ∈ [0.05, 0.95] (f), and the conditional mean for the 50 curves of a frozen test cohort at the 31 t points.
Covariate. \(X_i(s) = \xi_{i0} + \sum_{j=1}^{29} \xi_{ij}\sqrt 2\cos(j\pi s)\), \(\xi_{ij} \sim N(0, (j+1)^{-2})\); the constant term keeps the s-constant part of β identified. The centred spectrum has a first-PC share of 62%, 88% for the first four and a 99.5% rank of 24 (close to the DTI covariate’s 67%, 86% and 26). The low-rank variant uses variances (j+1)⁻³ (99.5% rank 9, below 1.5 times the ff basis’s s-dimension, like the running data’s knee angle).
Truths. \(\beta(s,t) = \cos(\pi s)\sin(\pi t) + (s - \tfrac12)\cos(2\pi t)\); \(f(x,t) = 2(x - \tfrac12)\cos(\pi t) + \sin(\pi x)\,t\); \(\gamma(t) = \sin(2\pi t)\); the rough (“wiggly”) truth adds \(0.6\sin(3\pi s)\sin(3\pi t)\) to β (and f) and \(0.5\sin(5\pi t)\) to γ. Each truth is projected by least squares (on 101 points per axis) onto the fitted default bases, so it is exactly representable; the misspecification experiment (Section 6) relaxes this. The truths are scaled so that the non-intercept signal \(S = \int \mathrm{Var}_{X,z}\{\eta(t) - \alpha(t)\}\,dt\) splits 80% / 20% between the main term and γ, and multiplied by √S. α(t) is a Gaussian bump (centre 0.5, sd 0.1, height √S), projected onto the intercept basis, plus a constant that sets each GLM’s average response level (found by root search on a population sample of 2000 covariate draws).
Signal and families. Gaussian: error variance set for R² ∈ {0.2, 0.5, 0.8}. Poisson: √S ∈ {0.5, 1} on the log scale, average mean count 3. Binary (Bernoulli per grid point): √S ∈ {0.5, 1} on the logit scale, average prevalence 0.2. Extended families: scaled t with 4 df (as Gaussian at R² 0.5), beta (φ = 20, average mean 0.3) and negative binomial (size 5, average mean 3, also fitted as Poisson) at √S = 0.5. The default level is R² 0.5 for Gaussian and √S 0.5 otherwise.
Within-curve dependence. A zero-mean process \(Z_i \sim N(0, R)\) with correlation kernel R on the grid is scaled for Gaussian errors or used as the latent layer of a Gaussian copula for the other families, \(Y_{ij} = F_{ij}^{-1}(\Phi(Z_{ij}))\), so every marginal is exactly the fitted family with mean \(g^{-1}(\eta_{ij})\) and the working-independence model is wrong only about the dependence.
| error process | kernel R(s, t) | role |
|---|---|---|
| iid | 1{s = t} | control |
| OU | 0.8 exp(−|s − t|/0.089) + 0.2·1{s = t} (Gaussian design effect 8 at D = 61) | rough, stationary |
| smooth | residual covariance of the DTI application (rcst ~ ff(cca), 92 curves) by FPCA: 24 eigenfunctions plus 6% white noise, standardised to a correlation (design effect 11.8 at D = 61, 47 at D = 241) |
smooth, realistic |
| AR(1) | exp(−|s − t|/0.071), exactly AR(1) on the grid | the AR(1) model’s own case |
| sign-changing | 0.8 exp(−|s − t|/0.3) cos(2π|s − t|/0.4) + 0.2·1{s = t} | stationary, correlations change sign with lag |
| var(t), var(z), var(t,z) | the smooth process with the DTI variance profile along t (variances 0.46–2.42, average 1); with subject i’s errors scaled by exp(0.5 z_i − 0.25); both | heteroskedastic (Gaussian) |
| misregistration | below | phase variability |
Misregistration. The model holds on each curve’s own clock and is observed on the common clock: \(Y_i(t_j) = \eta_i(h_i(t_j)) + \varepsilon_i(h_i(t_j))\) with \(h_i(t) = t + a_i\sin(\pi t)/\pi\), \(a_i \sim N(0, 0.314^2)\) truncated at \(|a_i| \le 0.9\) (monotone warps with fixed endpoints, maximal displacement SD 10%) and iid ε. For Gaussian responses the common-clock coefficients are the smeared functions \(\tilde\alpha(t) = \mathrm E_h\,\alpha(h(t))\), \(\tilde\beta(s,t) = \mathrm E_h\,\beta(s, h(t))\), \(\tilde\gamma(t) = \mathrm E_h\,\gamma(h(t))\) (quadrature over a with 201 nodes), and every estimand is scored against them; for GLMs the observed-clock mean is not \(g^{-1}\) of a linear predictor, so GLM misregistration cells score only the test-cohort mean, against its exact quadrature. These cells run at high signal.
Fits. Every data set is fitted by pffr() (refund) with mgcv’s gam() twice: by REML and by NCV with each curve as one leave-out block. Bases: cubic P-splines (mgcv ps, m = (2, 1), i.e. first-order difference penalties); intercept and γ(t) k = 12; ff term 8 (s) × 10 (t); M2’s f(x, t) 8 (x) × 12 (t) with the x knots fixed to [0, 1]; in the basis-size block 21 and 13 × 17 (large) or 39 and 23 × 31 (xlarge), nested spline spaces. Extra fits: pointwise NCV (mgcv’s default neighbourhoods; Gaussian core cells at G = 100 with iid and smooth errors, and the Gaussian sign-changing cell) and the AR(1) working model (Section 3.6).
Intervals. All intervals are pointwise at nominal 95%, estimate ± 1.96 SE, for an estimand Lθ (rows of the lpmatrix for the mean, the term’s basis at the evaluation points for β, f, α, γ; the inverse link applied to the interval ends for the mean): model-based (the REML fit’s Bayesian covariance with mgcv’s smoothing-parameter correction, V_c; V_p for NCV fits, whose V_c is not a valid covariance in the mgcv build used), CL2 (curve-clustered sandwich with the exact Bell–McCaffrey adjustment of each curve’s residual block, Bayesian form V_p − V_e added), bias-aware (NCV + CL2 widened in quadrature by δ = L(θ̂_NCV − θ̂_REML)), its variant on the frequentist CL2, the hybrid (NCV estimate with REML + CL2’s SE) and the AR(1) fit’s model-based interval.
Replicates. R = 200 per cell. Replicate r uses seed 20260925 + r in every cell, and all draws are made at full size (G = 100 curves, dense grid; G = 40 uses the first 40 curves, D = 61 the nested subgrid), so all cells share their random draws (common random numbers) and contrasts between cells are paired by replicate. The test cohort (50 covariate draws, seed 4711) is the same in every cell. Software: refund pffr-refactor commit 79a346fb and mgcv 1.9.5 (pre-release), both asserted in every task.
3.2 Inference
3.2.1 Model-based intervals versus CL2 by family and dependence
Model-based intervals undercover in every dependent cell and every family (Figure 1): at G = 100, Gaussian 0.63–0.67, Poisson 0.65–0.71, binary 0.72–0.82. REML + CL2 covers 0.94, 0.94–0.95 and 0.94–0.95. The signal level does not change this (REML + CL2’s core-factorial average per signal level is within 1.6 pp of nominal; Table 96 lists every cell). On the recommendation pool (dependent core cells at G = 100 plus the Gaussian misregistration cell) the calibration error of the model-based intervals is 0.15–0.30, that of REML + CL2 0.00–0.01 (Table 2). NCV + CL2 without the bias allowance is less well calibrated (0.02–0.06; binary responses worst). Under independent errors at G = 100, CL2 costs 0.3–1.1 pp of coverage relative to the model-based intervals, which overcover there (0.96–0.99); at G = 40 the cost is 0.4–2.6 pp (largest for the binary mean).
| family | estimand | CE model-based (REML) | CE REML + CL2 | CE model-based (NCV) | CE NCV + CL2 | gain REML | gain NCV | iid cost REML | iid cost NCV |
|---|---|---|---|---|---|---|---|---|---|
| Gaussian | E(Y | X) | 0.299 [0.292, 0.306] | 0.007 [0.004, 0.010] | 0.327 [0.319, 0.335] | 0.022 [0.018, 0.026] | 0.292 [0.287, 0.297] | 0.305 [0.300, 0.310] | 0.006 [0.005, 0.007] | 0.003 [0.002, 0.004] |
| Gaussian | beta(s,t) | 0.276 [0.265, 0.287] | 0.007 [0.001, 0.012] | 0.255 [0.242, 0.269] | 0.018 [0.012, 0.025] | 0.269 [0.262, 0.277] | 0.237 [0.230, 0.245] | 0.003 [0.003, 0.004] | 0.001 [0.000, 0.002] |
| Poisson | E(Y | X) | 0.287 [0.280, 0.295] | 0.007 [0.003, 0.011] | 0.321 [0.311, 0.330] | 0.025 [0.020, 0.030] | 0.281 [0.276, 0.285] | 0.296 [0.291, 0.301] | 0.007 [0.006, 0.008] | 0.005 [0.003, 0.006] |
| Poisson | beta(s,t) | 0.255 [0.243, 0.266] | 0.007 [0.000, 0.012] | 0.236 [0.224, 0.248] | 0.017 [0.011, 0.023] | 0.248 [0.240, 0.255] | 0.220 [0.212, 0.227] | 0.005 [0.004, 0.005] | 0.002 [0.001, 0.003] |
| binary | E(Y | X) | 0.208 [0.201, 0.216] | 0.008 [0.004, 0.012] | 0.305 [0.294, 0.314] | 0.057 [0.051, 0.063] | 0.200 [0.196, 0.205] | 0.247 [0.242, 0.252] | 0.011 [0.010, 0.012] | 0.004 [0.003, 0.005] |
| binary | beta(s,t) | 0.151 [0.139, 0.163] | 0.000 [0.000, 0.003] | 0.205 [0.189, 0.221] | 0.039 [0.029, 0.051] | 0.151 [0.139, 0.161] | 0.166 [0.158, 0.174] | 0.004 [0.003, 0.005] | 0.001 [0.000, 0.001] |
3.2.2 Lower tails of pointwise coverage
Grid averages hide where an interval fails. The 5% quantile of pointwise coverage (Figure 2; per-arm minima and medians in Table 79) is 0.91 or higher for REML + CL2 in the dependent cells at G = 100 (worst cell) and 0.89 at G = 40; for the model-based intervals it is as low as 0.54. NCV + CL2 without the allowance reaches 0.600 for β at G = 40 (binary responses). Under independent errors REML + CL2’s lower tail for the mean is 0.825 at G = 40.
3.2.3 Studentised errors of the REML fit
The studentised error z = (estimate − truth) / SE of the REML fit, pooled over grid points and replicates, is a descriptive diagnostic: an SD of z above 1 with a centred mean indicates SE underestimation, a shifted mean a bias, but the pooling over grid points cannot separate a global scale error from spatially varying bias (local biases of opposite sign cancel in the mean and inflate the SD), so the share of |z| > 1.96 is reported beside the SD (a normal pivot with that SD would give a predictable share). It is computed for β, α(t) and γ(t) in every cell (link scale) and for the mean in Gaussian cells (the stored mean records hold link-scale estimates next to response-scale truths). Means over cells (Table 94; per-cell values in the CSV): in the dependent core cells at G = 100 the SD of z with the CL2 SE is 0.99–1.03 for β and 1.03 for the Gaussian mean (mean of z -0.02 to 0.01; 4.8–5.9% of |z| > 1.96), against 1.48–2.20 with the model-based SE; at G = 40 it is 1.05–1.09 for β and 1.07 for the mean, which matches the 1–2 pp coverage loss of REML + CL2 there. Under independent errors the CL2 SE is conservative for β (SD 0.75–0.81 at G = 100), matching its coverage near 0.98. Across the dependent extension cells (misregistration, rough truth, dense grid, sign-changing, extended families, heteroskedastic, low-rank, smooth-effect) the SD of z for β is 0.95–1.09 and the mean of z -0.00 to 0.00; with the largest bases the SD falls to 0.96. For β and the Gaussian mean, therefore, the studentised error is centred and its SD exceeds 1 by a few per cent wherever REML + CL2 falls short. The GLM intercept α(t) is different: its studentised error is shifted under dependence, mean of z -0.25 to -0.22 for binary responses at G = 100 (-0.46 to -0.39 at G = 40; dense grid -0.42 to -0.39; large and xlarge bases -0.38 and -0.59) and -0.12 to -0.11 for Poisson (independent errors: 0.02–0.06 and 0.02–0.04), while Gaussian α(t) and every γ(t) stay within 0.02–0.08: a link-scale bias of the GLM intercept that grows with the basis size and is not corrected by CL2, which explains part of the binary α(t) shortfall.
3.2.4 Grid density
Denser grids make model-based intervals worse, not better: from D = 61 to 241 their coverage of β drops by 30.7–36.9 pp under dependence (0.1–0.9 pp under independent errors), while REML + CL2 changes by -1.3 to 0.0 pp and the bias-aware NCV interval by 0.4–1.5 pp (Figure 3, Table 3). NCV’s estimation advantage grows with the grid: the log NCV/REML MSE ratio falls by 1.15–1.80 under dependence, a factor of 3.2–6.1.
| family | error | model-based | REML + CL2 | NCV + CL2 | bias-aware | log MSE ratio |
|---|---|---|---|---|---|---|
| Gaussian | iid | -0.009 [-0.012, -0.007] | -0.010 [-0.013, -0.007] | -0.006 [-0.009, -0.003] | -0.001 [-0.004, 0.001] | -0.120 [-0.207, -0.003] |
| Gaussian | OU | -0.312 [-0.324, -0.301] | -0.001 [-0.005, 0.003] | -0.017 [-0.021, -0.013] | 0.005 [0.003, 0.007] | -1.153 [-1.305, -1.001] |
| Gaussian | smooth | -0.307 [-0.318, -0.296] | 0.000 [-0.003, 0.004] | -0.018 [-0.020, -0.015] | 0.005 [0.004, 0.007] | -1.151 [-1.320, -0.972] |
| Poisson | iid | -0.007 [-0.010, -0.005] | -0.007 [-0.011, -0.005] | -0.004 [-0.007, -0.001] | -0.001 [-0.003, 0.001] | -0.221 [-0.320, -0.134] |
| Poisson | OU | -0.324 [-0.336, -0.312] | -0.004 [-0.008, -0.000] | -0.021 [-0.024, -0.018] | 0.005 [0.003, 0.007] | -1.188 [-1.317, -1.056] |
| Poisson | smooth | -0.313 [-0.325, -0.301] | -0.002 [-0.006, 0.002] | -0.020 [-0.022, -0.018] | 0.004 [0.002, 0.006] | -1.196 [-1.317, -1.086] |
| binary | iid | -0.001 [-0.003, 0.001] | -0.000 [-0.003, 0.002] | 0.002 [-0.002, 0.007] | 0.003 [-0.000, 0.006] | -0.225 [-0.360, -0.081] |
| binary | OU | -0.369 [-0.382, -0.355] | -0.013 [-0.018, -0.008] | -0.034 [-0.047, -0.020] | 0.015 [0.010, 0.020] | -1.803 [-1.990, -1.622] |
| binary | smooth | -0.359 [-0.374, -0.344] | -0.009 [-0.014, -0.004] | -0.030 [-0.037, -0.022] | 0.014 [0.010, 0.018] | -1.675 [-1.857, -1.498] |
3.2.5 Number of curves
With 40 instead of 100 curves REML + CL2 loses 1.1–1.8 pp of coverage in the dependent cells (averaged over signal levels and error processes per family and estimand; Table 80), the bias-aware NCV interval 0.1–1.4 pp. NCV + CL2 without the allowance loses 1.0–6.9 pp, most for binary responses, where it covers β at 0.844 at G = 40. The model-based intervals are already far off at both G.
3.2.6 Signal strength and truth roughness
Signal strength changes little; truth roughness does (Figure 4, Table 81). With the rough truth and dependent errors NCV + CL2 covers β at 0.80–0.90 (smooth truth: 0.90–0.94), because NCV’s extra smoothing now buys bias; the bias allowance restores 0.93–0.95, still short of REML + CL2’s 0.94–0.95. With the smooth truth the allowance overcovers β (0.97–0.98). The frequentist-CL2 variant of the allowance undercovers under independent errors with a rough truth (0.78–0.92).
3.2.7 Estimands and term types
The pattern is the same for all four estimands (Table 4): in the dependent cells at G = 100 and default signal the model-based intervals cover α(t) at 0.65–0.73, γ(t) at 0.66–0.73, β at 0.66–0.80 and the mean at 0.64–0.74; REML + CL2 covers them at 0.92–0.95 (the low end is binary α(t)/γ(t)). Replacing the functional covariate by a smooth effect f(x,t) of a scalar covariate changes REML + CL2’s coverage by -0.7 to 0.5 pp under dependence (paired difference to the matching ff cell, Table 5); NCV’s relative advantage for the surface is smaller in that model (the log MSE ratio is higher by 0.77–1.29, i.e. the ratio is 2.2–3.6 times larger), but still below one.
| dependence | family | estimand | NCV + CL2 | NCV + CL2, bias-aware | REML + CL2 | REML, model-based |
|---|---|---|---|---|---|---|
| dependent | Gaussian | E(Y | X) | 0.930 | 0.958 | 0.943 | 0.640 |
| dependent | Gaussian | beta(s,t) | 0.934 | 0.983 | 0.942 | 0.664 |
| dependent | Gaussian | alpha(t) | 0.940 | 0.947 | 0.951 | 0.650 |
| dependent | Gaussian | gamma(t) | 0.930 | 0.941 | 0.943 | 0.659 |
| dependent | Poisson | E(Y | X) | 0.928 | 0.957 | 0.944 | 0.659 |
| dependent | Poisson | beta(s,t) | 0.935 | 0.983 | 0.943 | 0.690 |
| dependent | Poisson | alpha(t) | 0.931 | 0.948 | 0.949 | 0.666 |
| dependent | Poisson | gamma(t) | 0.927 | 0.938 | 0.941 | 0.670 |
| dependent | binary | E(Y | X) | 0.880 | 0.936 | 0.940 | 0.737 |
| dependent | binary | beta(s,t) | 0.899 | 0.971 | 0.951 | 0.799 |
| dependent | binary | alpha(t) | 0.846 | 0.905 | 0.923 | 0.725 |
| dependent | binary | gamma(t) | 0.840 | 0.901 | 0.931 | 0.735 |
| independent | Gaussian | E(Y | X) | 0.950 | 0.958 | 0.961 | 0.967 |
| independent | Gaussian | beta(s,t) | 0.982 | 0.989 | 0.986 | 0.989 |
| independent | Gaussian | alpha(t) | 0.945 | 0.949 | 0.956 | 0.956 |
| independent | Gaussian | gamma(t) | 0.946 | 0.951 | 0.950 | 0.960 |
| independent | Poisson | E(Y | X) | 0.948 | 0.956 | 0.960 | 0.966 |
| independent | Poisson | beta(s,t) | 0.982 | 0.990 | 0.986 | 0.989 |
| independent | Poisson | alpha(t) | 0.936 | 0.943 | 0.951 | 0.954 |
| independent | Poisson | gamma(t) | 0.947 | 0.951 | 0.955 | 0.963 |
| independent | binary | E(Y | X) | 0.939 | 0.945 | 0.949 | 0.963 |
| independent | binary | beta(s,t) | 0.982 | 0.987 | 0.990 | 0.994 |
| independent | binary | alpha(t) | 0.912 | 0.920 | 0.924 | 0.943 |
| independent | binary | gamma(t) | 0.912 | 0.917 | 0.923 | 0.945 |
| family | error | model-based | REML + CL2 | bias-aware | log MSE ratio |
|---|---|---|---|---|---|
| Gaussian | iid | -0.013 [-0.016, -0.010] | -0.015 [-0.018, -0.012] | -0.006 [-0.009, -0.004] | 0.118 [0.062, 0.169] |
| Gaussian | OU | 0.006 [-0.013, 0.022] | 0.005 [-0.003, 0.012] | -0.008 [-0.012, -0.004] | 1.083 [0.927, 1.239] |
| Gaussian | smooth | -0.007 [-0.024, 0.011] | -0.002 [-0.010, 0.006] | -0.010 [-0.015, -0.005] | 1.177 [1.003, 1.366] |
| Poisson | iid | -0.012 [-0.014, -0.009] | -0.013 [-0.016, -0.010] | -0.013 [-0.017, -0.011] | 0.259 [0.156, 0.346] |
| Poisson | OU | -0.001 [-0.019, 0.015] | 0.003 [-0.005, 0.011] | -0.018 [-0.024, -0.012] | 1.148 [0.936, 1.365] |
| Poisson | smooth | -0.011 [-0.028, 0.007] | -0.002 [-0.010, 0.006] | -0.020 [-0.026, -0.014] | 1.292 [1.061, 1.523] |
| binary | iid | -0.006 [-0.009, -0.003] | -0.008 [-0.012, -0.005] | -0.011 [-0.016, -0.007] | 0.013 [-0.100, 0.120] |
| binary | OU | -0.027 [-0.046, -0.009] | -0.003 [-0.011, 0.006] | -0.013 [-0.024, -0.003] | 0.771 [0.593, 0.948] |
| binary | smooth | -0.025 [-0.045, -0.004] | -0.007 [-0.015, 0.002] | -0.006 [-0.014, 0.002] | 0.897 [0.687, 1.111] |
3.2.8 Heteroskedasticity, misregistration and sign-changing correlation
A variance profile along t leaves every recipe where it was (Table 82). Variance that depends on the scalar covariate breaks the model-based intervals for γ(t) (0.49–0.51) and the AR(1) model’s (0.88–0.89), while REML + CL2 covers γ at 0.93–0.94.
Misregistration (each curve observed on its own clock, the estimand being the observed-clock mean and its smeared coefficients) gives model-based coverage of 0.71–0.73 for the Gaussian mean and REML + CL2 coverage of 0.93–0.94 (Table 6). For misregistered Poisson data every recipe undercovers the mean (REML + CL2 0.90–0.91, bias-aware NCV 0.91–0.92); binary responses are unaffected (0.93–0.95). Sign-changing correlations behave like the other dependent processes: model-based 0.65–0.88, REML + CL2 0.94–0.96 (Table 83).
| family | G | signal | truth | estimand | REML, model-based | REML + CL2 | NCV + CL2 | NCV + CL2, bias-aware | NCV + CL2 (freq.), bias-aware | AR(1) working model |
|---|---|---|---|---|---|---|---|---|---|---|
| binary | 40 | high | smooth | E(Y | X) | 0.945 | 0.944 | 0.919 | 0.932 | 0.884 | |
| binary | 40 | high | wiggly | E(Y | X) | 0.924 | 0.925 | 0.892 | 0.908 | 0.854 | |
| binary | 100 | high | smooth | E(Y | X) | 0.943 | 0.954 | 0.931 | 0.941 | 0.902 | |
| binary | 100 | high | wiggly | E(Y | X) | 0.918 | 0.932 | 0.909 | 0.919 | 0.878 | |
| Gaussian | 40 | high | smooth | beta(s,t) | 0.761 | 0.949 | 0.937 | 0.974 | 0.956 | |
| Gaussian | 40 | high | smooth | E(Y | X) | 0.729 | 0.938 | 0.919 | 0.946 | 0.934 | |
| Gaussian | 40 | high | wiggly | beta(s,t) | 0.739 | 0.945 | 0.820 | 0.928 | 0.901 | |
| Gaussian | 40 | high | wiggly | E(Y | X) | 0.706 | 0.934 | 0.885 | 0.932 | 0.918 | |
| Gaussian | 100 | high | smooth | beta(s,t) | 0.753 | 0.951 | 0.937 | 0.976 | 0.959 | 0.875 |
| Gaussian | 100 | high | smooth | E(Y | X) | 0.730 | 0.942 | 0.930 | 0.952 | 0.941 | 0.844 |
| Gaussian | 100 | high | wiggly | beta(s,t) | 0.729 | 0.944 | 0.863 | 0.934 | 0.908 | 0.860 |
| Gaussian | 100 | high | wiggly | E(Y | X) | 0.706 | 0.937 | 0.911 | 0.938 | 0.926 | 0.838 |
| Poisson | 40 | high | smooth | E(Y | X) | 0.634 | 0.902 | 0.792 | 0.910 | 0.900 | |
| Poisson | 40 | high | wiggly | E(Y | X) | 0.614 | 0.899 | 0.773 | 0.910 | 0.902 | |
| Poisson | 100 | high | smooth | E(Y | X) | 0.571 | 0.910 | 0.801 | 0.922 | 0.915 | |
| Poisson | 100 | high | wiggly | E(Y | X) | 0.558 | 0.906 | 0.780 | 0.918 | 0.912 |
3.2.9 Extended families
Scaled-t, beta and negative-binomial responses repeat the pattern (Table 7): under the smooth error process model-based coverage is 0.63–0.67 and REML + CL2’s 0.94–0.95; the NCV/REML MSE ratio for β is 0.18–0.22. Fitting negative-binomial data as Poisson costs the model-based intervals most (0.53–0.95), REML + CL2 little (0.94–0.97).
| family | fit_family | error | estimand | REML, model-based | REML + CL2 | NCV + CL2 | NCV + CL2, bias-aware | MSE ratio NCV/REML |
|---|---|---|---|---|---|---|---|---|
| beta | beta | smooth | beta(s,t) | 0.659 | 0.941 | 0.929 | 0.982 | 0.18 [0.15, 0.22] |
| beta | beta | smooth | E(Y | X) | 0.635 | 0.941 | 0.924 | 0.956 | 0.70 [0.68, 0.72] |
| beta | beta | iid | beta(s,t) | 0.989 | 0.984 | 0.979 | 0.988 | 0.75 [0.65, 0.91] |
| beta | beta | iid | E(Y | X) | 0.966 | 0.958 | 0.946 | 0.956 | 0.98 [0.96, 0.99] |
| negative binomial | negative binomial | smooth | beta(s,t) | 0.669 | 0.943 | 0.932 | 0.983 | 0.18 [0.14, 0.21] |
| negative binomial | Poisson | smooth | beta(s,t) | 0.531 | 0.938 | 0.913 | 0.984 | 0.09 [0.07, 0.11] |
| negative binomial | negative binomial | smooth | E(Y | X) | 0.641 | 0.943 | 0.925 | 0.956 | 0.69 [0.64, 0.74] |
| negative binomial | Poisson | smooth | E(Y | X) | 0.529 | 0.939 | 0.913 | 0.957 | 0.55 [0.52, 0.59] |
| negative binomial | negative binomial | iid | beta(s,t) | 0.991 | 0.988 | 0.982 | 0.990 | 0.89 [0.75, 1.05] |
| negative binomial | Poisson | iid | beta(s,t) | 0.945 | 0.972 | 0.962 | 0.982 | 0.53 [0.49, 0.59] |
| negative binomial | negative binomial | iid | E(Y | X) | 0.968 | 0.962 | 0.950 | 0.958 | 1.06 [1.03, 1.09] |
| negative binomial | Poisson | iid | E(Y | X) | 0.898 | 0.952 | 0.934 | 0.950 | 0.90 [0.87, 0.93] |
| scaled t | scaled t | smooth | beta(s,t) | 0.671 | 0.944 | 0.940 | 0.985 | 0.22 [0.18, 0.26] |
| scaled t | scaled t | smooth | E(Y | X) | 0.644 | 0.947 | 0.938 | 0.962 | 0.71 [0.70, 0.74] |
| scaled t | scaled t | iid | beta(s,t) | 0.988 | 0.985 | 0.982 | 0.990 | 0.60 [0.58, 0.63] |
| scaled t | scaled t | iid | E(Y | X) | 0.965 | 0.960 | 0.950 | 0.959 | 0.95 [0.94, 0.97] |
3.2.10 Basis size
As the bases grow, REML’s ff EDF under the smooth error process rises from 27–42 (default) to 56–89 (large) and 113–176 (xlarge) across the three families, while NCV’s rises from 12–21 to 15–30 and 18–41: REML absorbs the dependence into the surface. The NCV/REML MSE ratio for β falls to 0.01 with the xlarge basis (Figure 5, Table 8). REML + CL2 stays calibrated (β 0.95–0.96 at xlarge), whereas the bias-aware NCV interval overcovers more with every step (0.97–0.98 → 0.99 → 1.00) because the allowance is the difference of two estimates and inherits REML’s growing noise; its width relative to REML + CL2 is 0.80–0.83 (default) and 0.77–0.78 (xlarge), and its interval score still beats REML + CL2’s (log score ratio -0.08 to -0.04 under the smooth process) because REML + CL2’s intervals widen with REML’s variance. Under independent errors every arm overcovers β with the larger bases (1.00).
| family | error | basis | cov: NCV + CL2 | cov: NCV + CL2, bias-aware | cov: REML + CL2 | cov: REML, model-based | IS: NCV + CL2 | IS: NCV + CL2, bias-aware | IS: REML + CL2 | IS: REML, model-based | width: NCV + CL2 | width: NCV + CL2, bias-aware | width: REML + CL2 | width: REML, model-based | mse_ratio_beta |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gaussian | independent | default | 0.98 | 0.99 | 0.99 | 0.99 | 0.76 | 0.82 | 1.02 | 1.02 | 0.72 | 0.80 | 0.98 | 1.00 | 0.65 |
| Gaussian | independent | large | 0.99 | 1.00 | 1.00 | 1.00 | 1.04 | 1.14 | 1.55 | 1.57 | 1.03 | 1.13 | 1.54 | 1.56 | 0.56 |
| Gaussian | independent | xlarge | 1.00 | 1.00 | 1.00 | 1.00 | 1.36 | 1.47 | 2.15 | 2.18 | 1.35 | 1.47 | 2.15 | 2.18 | 0.49 |
| Gaussian | smooth (DTI FPCs) | default | 0.93 | 0.98 | 0.94 | 0.66 | 1.48 | 2.42 | 3.49 | 7.23 | 1.09 | 2.31 | 2.81 | 1.37 | 0.16 |
| Gaussian | smooth (DTI FPCs) | large | 0.95 | 0.99 | 0.95 | 0.67 | 1.56 | 5.19 | 7.80 | 15.49 | 1.27 | 5.15 | 6.24 | 3.11 | 0.03 |
| Gaussian | smooth (DTI FPCs) | xlarge | 0.97 | 1.00 | 0.95 | 0.67 | 1.70 | 10.33 | 15.51 | 29.38 | 1.48 | 10.31 | 13.15 | 6.41 | 0.01 |
| Poisson | independent | default | 0.98 | 0.99 | 0.99 | 0.99 | 0.44 | 0.47 | 0.57 | 0.58 | 0.41 | 0.45 | 0.55 | 0.56 | 0.71 |
| Poisson | independent | large | 0.99 | 1.00 | 1.00 | 1.00 | 0.58 | 0.63 | 0.85 | 0.87 | 0.57 | 0.62 | 0.85 | 0.86 | 0.55 |
| Poisson | independent | xlarge | 1.00 | 1.00 | 1.00 | 1.00 | 0.75 | 0.81 | 1.17 | 1.19 | 0.75 | 0.80 | 1.17 | 1.19 | 0.52 |
| Poisson | smooth (DTI FPCs) | default | 0.93 | 0.98 | 0.94 | 0.68 | 0.80 | 1.35 | 1.94 | 3.74 | 0.60 | 1.29 | 1.57 | 0.80 | 0.16 |
| Poisson | smooth (DTI FPCs) | large | 0.95 | 0.99 | 0.95 | 0.70 | 0.83 | 2.81 | 4.21 | 7.70 | 0.69 | 2.79 | 3.43 | 1.80 | 0.03 |
| Poisson | smooth (DTI FPCs) | xlarge | 0.97 | 1.00 | 0.95 | 0.71 | 0.91 | 5.45 | 8.18 | 13.98 | 0.81 | 5.44 | 6.94 | 3.66 | 0.01 |
| binary | independent | default | 0.98 | 0.99 | 0.99 | 0.99 | 1.30 | 1.32 | 1.41 | 1.46 | 1.23 | 1.27 | 1.37 | 1.44 | 1.06 |
| binary | independent | large | 0.99 | 1.00 | 1.00 | 1.00 | 1.64 | 1.68 | 1.86 | 1.94 | 1.61 | 1.66 | 1.84 | 1.94 | 1.05 |
| binary | independent | xlarge | 1.00 | 1.00 | 1.00 | 1.00 | 2.02 | 2.06 | 2.33 | 2.44 | 2.02 | 2.06 | 2.32 | 2.44 | 1.03 |
| binary | smooth (DTI FPCs) | default | 0.90 | 0.97 | 0.95 | 0.78 | 2.54 | 3.74 | 5.23 | 8.00 | 1.51 | 3.47 | 4.31 | 2.65 | 0.17 |
| binary | smooth (DTI FPCs) | large | 0.92 | 0.99 | 0.96 | 0.79 | 2.53 | 7.75 | 11.57 | 16.64 | 1.70 | 7.66 | 9.79 | 5.96 | 0.04 |
| binary | smooth (DTI FPCs) | xlarge | 0.93 | 1.00 | 0.96 | 0.79 | 2.59 | 16.87 | 25.64 | 35.34 | 1.90 | 16.84 | 21.80 | 13.30 | 0.01 |
3.3 Estimation
3.3.1 NCV versus REML by factor and estimand
NCV’s gain is a dependence effect concentrated on the bivariate surface (Figure 6, Table 9). Averaged over the whole core factorial (both G, all signal levels), the NCV/REML MSE ratio for β is 0.12–0.21 under dependent errors and 0.69–1.05 under independent errors; for the mean 0.58–0.74 and 0.97–1.06. In the dependent cells at G = 100 (the pool of the summary) the ratios are 0.12–0.23 for β, 0.59–0.80 for the mean, 0.93–1.04 for γ(t) and 0.83–0.98 for α(t). With the rough truth the gain for β shrinks to 0.21–0.43 (smooth process), the mean’s to 0.67–0.83. A low-rank covariate (spectrum like the running data’s knee angle) makes the β gain larger under dependence (0.09–0.12) and removes it under independent errors (0.71–1.26). The ratio is nearly the same at G = 40 and G = 100 (β: 0.21–0.33 vs 0.26–0.34).
| family | estimand | iid | ou | fpc |
|---|---|---|---|---|
| Gaussian | E(Y | X) | 0.97 [0.96, 0.98] | 0.66 [0.64, 0.68] | 0.64 [0.62, 0.65] |
| Gaussian | beta(s,t) | 0.73 [0.69, 0.78] | 0.15 [0.13, 0.16] | 0.12 [0.10, 0.14] |
| Poisson | E(Y | X) | 0.97 [0.93, 1.01] | 0.59 [0.53, 0.66] | 0.58 [0.52, 0.64] |
| Poisson | beta(s,t) | 0.69 [0.66, 0.72] | 0.17 [0.14, 0.19] | 0.15 [0.13, 0.17] |
| binary | E(Y | X) | 1.06 [1.04, 1.07] | 0.74 [0.72, 0.76] | 0.68 [0.66, 0.70] |
| binary | beta(s,t) | 1.05 [0.96, 1.15] | 0.21 [0.18, 0.24] | 0.17 [0.14, 0.19] |
3.3.2 Bias and variance
REML’s excess error under dependence is variance, not bias: in the dependent core cells at G = 100 the squared-bias share of REML’s β error is 0.01, of NCV’s 0.12–0.20 (Figure 7). With the rough truth NCV’s bias share for β rises to 0.26–0.42 (REML 0.01–0.02), the mechanism behind NCV + CL2’s undercoverage there. Per-cell values with the MC SEs of the MSE and of the squared-bias estimate are in synthetic-bias-variance.csv.
3.3.3 Effective degrees of freedom
Under dependent errors REML’s ff EDF exceeds NCV’s by a factor of 1.92–2.31 (independent errors: 1.09–1.31); for α(t) the factor is 1.15–1.83 and for γ(t) 1.21–1.60 (Table 10). NCV’s gain is largest where the EDF gap is largest.
| family | error | term | NCV | REML | ratio |
|---|---|---|---|---|---|
| binary | smooth | alpha(t) | 2.83 | 5.06 | 1.786 |
| binary | iid | alpha(t) | 4.70 | 4.47 | 0.951 |
| binary | OU | alpha(t) | 2.74 | 5.03 | 1.832 |
| Gaussian | smooth | alpha(t) | 7.26 | 8.46 | 1.166 |
| Gaussian | iid | alpha(t) | 8.09 | 8.33 | 1.030 |
| Gaussian | OU | alpha(t) | 7.33 | 8.41 | 1.148 |
| Poisson | smooth | alpha(t) | 6.63 | 8.09 | 1.220 |
| Poisson | iid | alpha(t) | 7.53 | 7.97 | 1.059 |
| Poisson | OU | alpha(t) | 6.69 | 8.05 | 1.203 |
| binary | smooth | beta(s,t) | 11.60 | 26.77 | 2.308 |
| binary | iid | beta(s,t) | 15.72 | 17.07 | 1.086 |
| binary | OU | beta(s,t) | 11.99 | 25.39 | 2.117 |
| Gaussian | smooth | beta(s,t) | 21.03 | 41.84 | 1.990 |
| Gaussian | iid | beta(s,t) | 25.48 | 33.50 | 1.315 |
| Gaussian | OU | beta(s,t) | 21.35 | 41.09 | 1.925 |
| Poisson | smooth | beta(s,t) | 19.64 | 39.37 | 2.004 |
| Poisson | iid | beta(s,t) | 24.30 | 31.42 | 1.293 |
| Poisson | OU | beta(s,t) | 20.07 | 38.71 | 1.928 |
| binary | smooth | gamma(t) | 3.70 | 5.93 | 1.602 |
| binary | iid | gamma(t) | 5.33 | 5.21 | 0.979 |
| binary | OU | gamma(t) | 3.69 | 5.72 | 1.548 |
| Gaussian | smooth | gamma(t) | 7.44 | 9.18 | 1.233 |
| Gaussian | iid | gamma(t) | 8.39 | 9.02 | 1.075 |
| Gaussian | OU | gamma(t) | 7.49 | 9.12 | 1.218 |
| Poisson | smooth | gamma(t) | 7.26 | 8.82 | 1.215 |
| Poisson | iid | gamma(t) | 8.15 | 8.68 | 1.065 |
| Poisson | OU | gamma(t) | 7.25 | 8.75 | 1.207 |
3.3.4 Relative error and informativeness of the intervals
MSE ratios compare the two fits but not the size of the error relative to the effect. Per replicate and estimand (analysis/relative-error.R → summaries/final/relative-error.csv):
- relative error √(grid mean of (estimate − truth)² / grid mean of truth²); for the mean and α(t) the denominator uses the truth centred at its grid mean (their level is not the effect), while the numerator keeps any level error; 1 means the error is as large as the effect;
- relative half-width, the interval’s root-mean-square half-width on the same scale;
- detection: the share of grid points with |truth| > 25% of max |truth| where the interval excludes zero on the side of the truth (β, γ, f);
- wrong sign: the share of all grid points where the interval excludes zero on the wrong side.
Cell values are medians over replicates (relative error, half-width) or means (detection, wrong sign, coverage).
Under independent errors both fits estimate β well: median relative error 0.16 (REML) and 0.12 (NCV) over the cells. Under dependent errors REML’s error is 0.67 (maximum 5.8), above 0.5 in 40 of 69 cells, NCV’s 0.21 (Figure 8, Table 11). In the core cells the REML/NCV ratio of the relative error is 1.1–1.4 under independent and 2.1–3.7 under dependent errors (Table 12); the gap is present at every signal level and largest at low signal. The spread of the selected smoothing parameters does not explain it: the SD of log λ̂ of the ff penalties over replicates is 0.1–0.9 (independent) and 1.1–1.7 (dependent) for REML, 0.7–1.4 and 1.0–4.7 for NCV (largest for binary responses at G = 40; Table 13), and log-scale spreads at different levels of smoothing are not comparable. What matters is where λ̂ lands: within a cell, REML’s bad replicates are those with a small λ̂ (λ-correction run on the 18 core cells at the default signal, lambda-correction/flag-check.R: Spearman correlation between small log λ̂ of one of the two ff penalties and the relative error 0.94–0.95 under dependence, mean over cells). The other estimands are estimated well by both fits (Table 11).
The intervals mirror this. Under dependence REML + CL2 covers β (median 0.943) with a half-width of 1.35 times the size of the effect and detects 0.54 of the clearly non-zero grid points; NCV + CL2 (coverage 0.924) has half-width 0.37 and detects 0.95; the bias-aware interval inherits REML’s noise through δ (half-width 1.14, detection 0.65). Wrong-sign exclusions stay rare for the CL2 intervals (at most 0.023 of the grid for REML + CL2, 0.040 for NCV + CL2) but reach 0.20 for model-based REML intervals.
| estimand | dep. | arm | cells | cov. | rel | rel max | cells >0.5 | reps >1 | hw | det | det min | wrong max |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| E(Y | X) | dep | REML, model-based | 83 | 0.65 | 0.28 | 1.19 | 18 | 0.04 | 0.27 | |||
| E(Y | X) | dep | REML + CL2 | 83 | 0.94 | 0.28 | 1.19 | 18 | 0.04 | 0.58 | |||
| E(Y | X) | dep | NCV + CL2 | 83 | 0.92 | 0.23 | 0.91 | 17 | 0.01 | 0.42 | |||
| E(Y | X) | dep | NCV + CL2, bias-aware | 83 | 0.95 | 0.23 | 0.91 | 17 | 0.01 | 0.53 | |||
| E(Y | X) | dep | AR(1) working model | 9 | 0.95 | 0.18 | 0.21 | 0 | 0.00 | 0.38 | |||
| E(Y | X) | ind | REML, model-based | 40 | 0.97 | 0.11 | 0.49 | 0 | 0.00 | 0.24 | |||
| E(Y | X) | ind | REML + CL2 | 40 | 0.96 | 0.11 | 0.49 | 0 | 0.00 | 0.24 | |||
| E(Y | X) | ind | NCV + CL2 | 40 | 0.95 | 0.10 | 0.50 | 1 | 0.00 | 0.22 | |||
| E(Y | X) | ind | NCV + CL2, bias-aware | 40 | 0.95 | 0.10 | 0.50 | 1 | 0.00 | 0.23 | |||
| beta(s,t) | dep | REML, model-based | 69 | 0.67 | 0.67 | 5.83 | 40 | 0.28 | 0.57 | 0.83 | 0.16 | 0.20 |
| beta(s,t) | dep | REML + CL2 | 69 | 0.94 | 0.67 | 5.83 | 40 | 0.28 | 1.35 | 0.54 | 0.04 | 0.02 |
| beta(s,t) | dep | NCV + CL2 | 69 | 0.92 | 0.21 | 0.74 | 3 | 0.02 | 0.37 | 0.95 | 0.40 | 0.04 |
| beta(s,t) | dep | NCV + CL2, bias-aware | 69 | 0.98 | 0.21 | 0.74 | 3 | 0.02 | 1.14 | 0.65 | 0.04 | 0.01 |
| beta(s,t) | dep | AR(1) working model | 9 | 0.99 | 0.19 | 0.21 | 0 | 0.00 | 0.50 | 0.94 | 0.93 | 0.00 |
| beta(s,t) | ind | REML, model-based | 37 | 0.99 | 0.16 | 0.40 | 0 | 0.00 | 0.40 | 0.96 | 0.43 | 0.00 |
| beta(s,t) | ind | REML + CL2 | 37 | 0.99 | 0.16 | 0.40 | 0 | 0.00 | 0.40 | 0.96 | 0.47 | 0.00 |
| beta(s,t) | ind | NCV + CL2 | 37 | 0.98 | 0.12 | 0.41 | 0 | 0.00 | 0.31 | 0.97 | 0.56 | 0.00 |
| beta(s,t) | ind | NCV + CL2, bias-aware | 37 | 0.99 | 0.12 | 0.41 | 0 | 0.00 | 0.32 | 0.97 | 0.55 | 0.00 |
| alpha(t) | dep | REML, model-based | 69 | 0.66 | 0.31 | 1.60 | 18 | 0.11 | 0.31 | |||
| alpha(t) | dep | REML + CL2 | 69 | 0.95 | 0.31 | 1.60 | 18 | 0.11 | 0.65 | |||
| alpha(t) | dep | NCV + CL2 | 69 | 0.92 | 0.28 | 1.23 | 17 | 0.09 | 0.57 | |||
| alpha(t) | dep | NCV + CL2, bias-aware | 69 | 0.94 | 0.28 | 1.23 | 17 | 0.09 | 0.64 | |||
| alpha(t) | dep | AR(1) working model | 9 | 0.95 | 0.23 | 0.24 | 0 | 0.00 | 0.49 | |||
| alpha(t) | ind | REML, model-based | 37 | 0.96 | 0.16 | 0.74 | 1 | 0.00 | 0.34 | |||
| alpha(t) | ind | REML + CL2 | 37 | 0.95 | 0.16 | 0.74 | 1 | 0.00 | 0.35 | |||
| alpha(t) | ind | NCV + CL2 | 37 | 0.94 | 0.16 | 0.74 | 4 | 0.01 | 0.32 | |||
| alpha(t) | ind | NCV + CL2, bias-aware | 37 | 0.94 | 0.16 | 0.74 | 4 | 0.01 | 0.34 | |||
| gamma(t) | dep | REML, model-based | 75 | 0.66 | 0.22 | 1.14 | 17 | 0.04 | 0.23 | 0.99 | 0.41 | 0.07 |
| gamma(t) | dep | REML + CL2 | 75 | 0.94 | 0.22 | 1.14 | 17 | 0.04 | 0.46 | 0.94 | 0.15 | 0.01 |
| gamma(t) | dep | NCV + CL2 | 75 | 0.92 | 0.22 | 1.00 | 17 | 0.03 | 0.42 | 0.96 | 0.12 | 0.01 |
| gamma(t) | dep | NCV + CL2, bias-aware | 75 | 0.93 | 0.22 | 1.00 | 17 | 0.03 | 0.44 | 0.95 | 0.06 | 0.01 |
| gamma(t) | dep | AR(1) working model | 9 | 0.92 | 0.17 | 0.24 | 0 | 0.00 | 0.37 | 0.98 | 0.94 | 0.01 |
| gamma(t) | ind | REML, model-based | 40 | 0.96 | 0.11 | 0.54 | 1 | 0.00 | 0.24 | 1.00 | 0.35 | 0.01 |
| gamma(t) | ind | REML + CL2 | 40 | 0.95 | 0.11 | 0.54 | 1 | 0.00 | 0.25 | 1.00 | 0.45 | 0.01 |
| gamma(t) | ind | NCV + CL2 | 40 | 0.94 | 0.12 | 0.58 | 1 | 0.00 | 0.23 | 1.00 | 0.44 | 0.01 |
| gamma(t) | ind | NCV + CL2, bias-aware | 40 | 0.94 | 0.12 | 0.58 | 1 | 0.00 | 0.23 | 1.00 | 0.42 | 0.01 |
| family | signal | error | G | NCV | REML | ratio |
|---|---|---|---|---|---|---|
| binomial | high | fpc | 40 | 0.43 | 1.21 | 2.85 |
| binomial | mid | fpc | 40 | 0.74 | 2.22 | 2.99 |
| binomial | high | fpc | 100 | 0.29 | 0.67 | 2.33 |
| binomial | mid | fpc | 100 | 0.45 | 1.18 | 2.64 |
| binomial | high | iid | 40 | 0.25 | 0.28 | 1.09 |
| binomial | mid | iid | 40 | 0.38 | 0.40 | 1.06 |
| binomial | high | iid | 100 | 0.18 | 0.20 | 1.10 |
| binomial | mid | iid | 100 | 0.27 | 0.29 | 1.06 |
| binomial | high | ou | 40 | 0.40 | 0.97 | 2.44 |
| binomial | mid | ou | 40 | 0.73 | 1.75 | 2.40 |
| binomial | high | ou | 100 | 0.28 | 0.60 | 2.16 |
| binomial | mid | ou | 100 | 0.43 | 1.04 | 2.40 |
| gaussian | high | fpc | 40 | 0.13 | 0.35 | 2.62 |
| gaussian | low | fpc | 40 | 0.38 | 1.39 | 3.67 |
| gaussian | mid | fpc | 40 | 0.22 | 0.70 | 3.22 |
| gaussian | high | fpc | 100 | 0.09 | 0.20 | 2.17 |
| gaussian | low | fpc | 100 | 0.26 | 0.70 | 2.69 |
| gaussian | mid | fpc | 100 | 0.15 | 0.36 | 2.35 |
| gaussian | high | iid | 40 | 0.07 | 0.10 | 1.37 |
| gaussian | low | iid | 40 | 0.19 | 0.22 | 1.12 |
| gaussian | mid | iid | 40 | 0.12 | 0.15 | 1.26 |
| gaussian | high | iid | 100 | 0.05 | 0.07 | 1.36 |
| gaussian | low | iid | 100 | 0.13 | 0.16 | 1.22 |
| gaussian | mid | iid | 100 | 0.08 | 0.11 | 1.31 |
| gaussian | high | ou | 40 | 0.13 | 0.33 | 2.60 |
| gaussian | low | ou | 40 | 0.37 | 1.37 | 3.71 |
| gaussian | mid | ou | 40 | 0.21 | 0.66 | 3.09 |
| gaussian | high | ou | 100 | 0.09 | 0.19 | 2.08 |
| gaussian | low | ou | 100 | 0.26 | 0.68 | 2.67 |
| gaussian | mid | ou | 100 | 0.15 | 0.35 | 2.30 |
| poisson | high | fpc | 40 | 0.17 | 0.45 | 2.62 |
| poisson | mid | fpc | 40 | 0.24 | 0.76 | 3.13 |
| poisson | high | fpc | 100 | 0.12 | 0.25 | 2.13 |
| poisson | mid | fpc | 100 | 0.17 | 0.41 | 2.46 |
| poisson | high | iid | 40 | 0.10 | 0.13 | 1.24 |
| poisson | mid | iid | 40 | 0.13 | 0.16 | 1.22 |
| poisson | high | iid | 100 | 0.07 | 0.09 | 1.32 |
| poisson | mid | iid | 100 | 0.09 | 0.12 | 1.30 |
| poisson | high | ou | 40 | 0.17 | 0.42 | 2.42 |
| poisson | mid | ou | 40 | 0.24 | 0.69 | 2.87 |
| poisson | high | ou | 100 | 0.12 | 0.25 | 2.11 |
| poisson | mid | ou | 100 | 0.17 | 0.40 | 2.40 |
| family | error | G | NCV | REML |
|---|---|---|---|---|
| binomial | fpc | 40 | 4.67 | 1.74 |
| binomial | fpc | 100 | 1.91 | 1.34 |
| binomial | iid | 40 | 1.42 | 0.91 |
| binomial | iid | 100 | 1.12 | 0.27 |
| binomial | ou | 40 | 4.26 | 1.50 |
| binomial | ou | 100 | 1.85 | 1.13 |
| gaussian | fpc | 40 | 1.01 | 1.51 |
| gaussian | fpc | 100 | 1.06 | 1.28 |
| gaussian | iid | 40 | 0.89 | 0.13 |
| gaussian | iid | 100 | 0.66 | 0.10 |
| gaussian | ou | 40 | 1.37 | 1.42 |
| gaussian | ou | 100 | 0.97 | 1.17 |
| poisson | fpc | 40 | 1.11 | 1.48 |
| poisson | fpc | 100 | 1.05 | 1.27 |
| poisson | iid | 40 | 0.76 | 0.13 |
| poisson | iid | 100 | 0.89 | 0.11 |
| poisson | ou | 40 | 1.45 | 1.37 |
| poisson | ou | 100 | 1.13 | 1.16 |
3.3.5 Pointwise NCV and the low-rank covariate
Ordinary pointwise NCV (mgcv’s default neighbourhoods) is no substitute for curve blocks: under the smooth and the sign-changing error processes its β MSE is 2.86–4.86 times REML’s and its mean MSE 1.34–1.45 times (Table 14), because a held-out point is predicted by its correlated neighbours.
| error | estimand | MSE pointwise NCV / REML |
|---|---|---|
| iid | beta(s,t) | 0.65 [0.62, 0.68] |
| iid | E(Y | X) | 0.96 [0.95, 0.97] |
| smooth | beta(s,t) | 2.86 [2.57, 3.21] |
| smooth | E(Y | X) | 1.34 [1.32, 1.37] |
| sign-chg. | beta(s,t) | 4.86 [4.35, 5.38] |
| sign-chg. | E(Y | X) | 1.45 [1.42, 1.47] |
3.4 The recommendation rule under both criteria
The recommendation rule compares the proposal (NCV + bias-aware CL2) with the fallback (REML + CL2) on the pool per family and primary estimand: a recipe is adequate if its CE is ≤ 2 pp, and among adequate recipes the interval-score ratio decides (≤ 0.95 proposal, ≥ 1/0.95 fallback, otherwise the simpler fallback). Two calibration criteria are reported (Table 15): a symmetric one that counts over- and undercoverage alike, and one that counts undercoverage only, since width is already priced by the interval score. Under the symmetric criterion the proposal fails adequacy for β in every family (CE 0.03, from overcoverage near 0.98) and the rule picks the fallback for β and the proposal for the mean; under the undercoverage-only criterion the proposal’s CE is 0.00–0.01 and it is chosen for both estimands in all three families on interval score (β ratio 0.69–0.70, mean 0.87–0.92), at every signal level (synthetic-recommendation-by-signal.csv).
The rule’s synthetic output is not the recommendation. On the nine plasmode datasets the undercoverage-only criterion does not select the proposal (Section 10.5: no consistent class, and the proposal undercovers β under REML-fitted truths), and the proposal’s interval-score advantage comes from a construction whose calibration depends on the truth’s smoothness (Section 3.5).
| family | estimand | criterion | ce_proposal | ce_fallback | score_ratio | decision |
|---|---|---|---|---|---|---|
| binary | beta(s,t) | symmetric | 0.026 | 0.000 | 0.704 | fallback |
| binary | beta(s,t) | under only | 0.000 | 0.000 | 0.704 | proposal |
| binary | E(Y | X) | symmetric | 0.010 | 0.008 | 0.916 | proposal |
| binary | E(Y | X) | under only | 0.010 | 0.008 | 0.916 | proposal |
| Gaussian | beta(s,t) | symmetric | 0.032 | 0.007 | 0.692 | fallback |
| Gaussian | beta(s,t) | under only | 0.000 | 0.007 | 0.692 | proposal |
| Gaussian | E(Y | X) | symmetric | 0.007 | 0.007 | 0.873 | proposal |
| Gaussian | E(Y | X) | under only | 0.000 | 0.007 | 0.873 | proposal |
| Poisson | beta(s,t) | symmetric | 0.032 | 0.007 | 0.688 | fallback |
| Poisson | beta(s,t) | under only | 0.000 | 0.007 | 0.688 | proposal |
| Poisson | E(Y | X) | symmetric | 0.005 | 0.007 | 0.877 | proposal |
| Poisson | E(Y | X) | under only | 0.000 | 0.007 | 0.877 | proposal |
3.5 Bias-aware intervals: two failure mechanisms
The allowance δ = L(β̂_NCV − β̂_REML), added in quadrature to the CL2 standard error of the NCV fit, fails in two ways.
Inflation by REML’s noise. δ is a difference of two estimates and carries the sampling variance of the REML fit, which the interval ignores. Where REML is noisy the interval is too wide: in the dependent core cells at G = 100 the bias-aware β interval covers 0.981 against REML + CL2’s 0.945 at 0.82 times the width (interval score ratio 0.69); with the larger bases the coverage reaches 0.997–0.998 (Table 16).
Blindness to bias shared by both fits. δ measures only how far NCV moved away from REML. Where the truth is rougher than both fits, both are biased in the same direction and δ is small. In the rough-truth cells with the smooth error process NCV + CL2 covers β at 0.80–0.90 with 36–68% of grid points below 0.90; the allowance lifts the average to 0.93–0.95 but leaves 9–27% of points below 0.90, and the remaining shortfall follows the truth’s curvature (Spearman correlation of the pointwise shortfall with |∂²β/∂s²| + |∂²β/∂t²|: 0.48–0.80, against -0.00 to 0.32 for REML + CL2; Figure 9). The bias-aware interval’s rough-truth lower tail (0.828) is no better than NCV + CL2’s smooth-truth tail (0.85). The frequentist-CL2 variant is worse wherever the Bayesian one is short (rough truth, β: 0.78–0.94).
The hybrid. The NCV estimate with the REML fit’s CL2 standard error covers β at 0.997 in the dependent core cells (5% quantile 0.988, width 1.00 and interval score 0.81 times REML + CL2’s) and at 0.988 with the rough truth (5% quantile 0.959 against REML + CL2’s 0.917). Its real-data results are in Section 10.10.
| setting | estimand | n_cells | cov REML+CL2 | cov bias-aware | cov hybrid | q05 REML+CL2 | q05 bias-aware | q05 hybrid | width bias-aware/REML+CL2 | width hybrid/REML+CL2 | IS bias-aware/REML+CL2 | IS hybrid/REML+CL2 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| dependent core, G = 100 | beta(s,t) | 14 | 0.945 | 0.981 | 0.997 | 0.919 | 0.944 | 0.988 | 0.816 | 1.00 | 0.690 | 0.814 |
| dependent core, G = 100 | E(Y | X) | 14 | 0.943 | 0.952 | 0.973 | 0.913 | 0.909 | 0.935 | 0.928 | 1.01 | 0.884 | 0.889 |
| rough truth, smooth errors | beta(s,t) | 7 | 0.943 | 0.938 | 0.988 | 0.917 | 0.828 | 0.959 | 0.856 | 1.00 | 0.824 | 0.831 |
| rough truth, smooth errors | E(Y | X) | 7 | 0.943 | 0.945 | 0.964 | 0.914 | 0.890 | 0.917 | 0.943 | 1.01 | 0.927 | 0.921 |
3.6 The AR(1) working model
Fit. The AR(1) working model is pffr(algorithm = "bam", method = "fREML", rho = rho): mgcv’s bam() with AR(1) errors that restart at the first grid point of every curve, so curves stay independent and each curve’s errors are whitened with the same ρ. ρ is profiled. The model is fitted at every ρ in {0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0.95, 0.99}; the criterion is bam’s fREML score, which already contains the whitening log-determinant \(-(n - G)\log(1/\sqrt{1 - \rho^2})\) (n data points, G curves), i.e. the restricted likelihood of the AR(1) model. The minimising grid value is refined by a one-dimensional search (optimize()) between its two neighbouring grid values, and the refined ρ replaces it if its criterion is lower. A ρ̂ at either end of the grid (0 or 0.99) is flagged as a boundary solution. Intervals use the AR(1) fit’s model-based covariance. The arm is Gaussian only and runs in the Gaussian cells at G = 100 with the OU (default signal), smooth (default signal), exact-AR(1), sign-changing, heteroskedastic and misregistration (both truths) errors, and in the plasmode cells listed in Table 91. Each fit costs 12 grid fits plus the refinement (Section 14). Mean ρ̂ per cell and the share of boundary solutions are in Table 18.
Modelling the dependence as AR(1) (ρ profiled) is conservative and competitive when the errors are stationary and homoskedastic: coverage 0.93–0.99 on the OU, smooth, exact-AR(1) and sign-changing processes, with interval scores for β 0.74–0.84 times the bias-aware NCV interval’s (Table 17). It fails under misregistration (mean 0.84, β 0.86–0.88; ρ̂ averages 0.32–0.38) and for γ(t) under covariate-dependent variance (above). Fit times: median 39 s per AR(1) fit against 2.0 s (REML) and 8.6 s (NCV) plus about 1.0 s per CL2 covariance, Gaussian G = 100 (10 replicates per error process).
| cell | error | G | truth | estimand | AR(1) working model | NCV + CL2, bias-aware | REML + CL2 | REML, model-based | IS AR(1) / bias-aware NCV | MSE AR(1) / NCV |
|---|---|---|---|---|---|---|---|---|---|---|
| 12 | smooth | 100 | smooth | beta(s,t) | 0.993 | 0.983 | 0.944 | 0.657 | 0.77 [0.73, 0.80] | 0.98 [0.82, 1.18] |
| 12 | smooth | 100 | smooth | E(Y | X) | 0.972 | 0.959 | 0.944 | 0.634 | 0.98 [0.97, 0.99] | 1.04 [1.02, 1.05] |
| 103 | var(z) | 100 | smooth | beta(s,t) | 0.992 | 0.984 | 0.949 | 0.660 | 0.78 [0.74, 0.82] | 0.92 [0.76, 1.09] |
| 103 | var(z) | 100 | smooth | E(Y | X) | 0.949 | 0.955 | 0.944 | 0.606 | 1.04 [1.02, 1.07] | 1.05 [1.03, 1.07] |
| 101 | var(t) | 100 | smooth | beta(s,t) | 0.991 | 0.983 | 0.944 | 0.662 | 0.80 [0.77, 0.84] | 1.00 [0.80, 1.21] |
| 101 | var(t) | 100 | smooth | E(Y | X) | 0.972 | 0.959 | 0.944 | 0.642 | 1.04 [1.03, 1.05] | 1.10 [1.08, 1.12] |
| 105 | var(t,z) | 100 | smooth | beta(s,t) | 0.990 | 0.984 | 0.950 | 0.666 | 0.81 [0.78, 0.86] | 1.06 [0.88, 1.25] |
| 105 | var(t,z) | 100 | smooth | E(Y | X) | 0.951 | 0.955 | 0.944 | 0.615 | 1.11 [1.08, 1.13] | 1.12 [1.10, 1.14] |
| 96 | sign-chg. | 100 | smooth | beta(s,t) | 0.992 | 0.985 | 0.942 | 0.700 | 0.84 [0.80, 0.86] | 1.13 [1.01, 1.27] |
| 96 | sign-chg. | 100 | smooth | E(Y | X) | 0.963 | 0.957 | 0.942 | 0.646 | 0.95 [0.94, 0.96] | 1.03 [1.01, 1.06] |
| 10 | OU | 100 | smooth | beta(s,t) | 0.972 | 0.983 | 0.940 | 0.672 | 0.78 [0.74, 0.81] | 1.30 [1.11, 1.49] |
| 10 | OU | 100 | smooth | E(Y | X) | 0.934 | 0.958 | 0.942 | 0.646 | 0.96 [0.95, 0.98] | 1.04 [1.03, 1.06] |
| 95 | AR(1) | 100 | smooth | beta(s,t) | 0.989 | 0.984 | 0.940 | 0.651 | 0.74 [0.71, 0.77] | 1.12 [0.96, 1.27] |
| 95 | AR(1) | 100 | smooth | E(Y | X) | 0.961 | 0.959 | 0.942 | 0.625 | 0.94 [0.93, 0.95] | 1.02 [1.01, 1.04] |
| 45 | misreg. | 100 | smooth | beta(s,t) | 0.875 | 0.976 | 0.951 | 0.753 | 1.41 [1.32, 1.50] | 1.29 [1.00, 1.62] |
| 45 | misreg. | 100 | smooth | E(Y | X) | 0.844 | 0.952 | 0.942 | 0.730 | 1.49 [1.43, 1.54] | 1.09 [1.07, 1.11] |
| 99 | misreg. | 100 | wiggly | beta(s,t) | 0.860 | 0.934 | 0.944 | 0.729 | 1.22 [1.15, 1.30] | 0.84 [0.70, 0.98] |
| 99 | misreg. | 100 | wiggly | E(Y | X) | 0.838 | 0.938 | 0.937 | 0.706 | 1.45 [1.41, 1.50] | 1.00 [0.99, 1.01] |
| cell | error | truth | mean rho | boundary share | median time s |
|---|---|---|---|---|---|
| 10 | OU | smooth | 0.659 | 0 | 50.2 |
| 12 | smooth | smooth | 0.850 | 0 | 51.6 |
| 45 | misreg. | smooth | 0.320 | 0 | 50.2 |
| 95 | AR(1) | smooth | 0.790 | 0 | 52.8 |
| 96 | sign-chg. | smooth | 0.728 | 0 | 46.0 |
| 99 | misreg. | wiggly | 0.384 | 0 | 44.8 |
| 101 | var(t) | smooth | 0.863 | 0 | 48.9 |
| 103 | var(z) | smooth | 0.849 | 0 | 47.9 |
| 105 | var(t,z) | smooth | 0.862 | 0 | 49.5 |
4 Covariance components, reference distributions and NCV covariances
Three experiments on the 18 core cells at the default signal (family × error process × G, D = 61, R = 200; the study’s replicates and REML fits, reproduced to < 1e-8 relative difference against the stored task records): the covariance ablation (ablation/), per-point Satterthwaite degrees of freedom (satterthwaite/), and mgcv’s own covariance for curve-blocked NCV fits (ncv-jack-recheck/, 3 cells × 20 replicates). Coverage is the grid-average of nominal 95% pointwise intervals; MC SEs are per cell.
4.1 Covariance ablation
Constructions, all on the replicate’s REML fit: model-based V_c; CR1, refund’s curve-clustered sandwich (sandwich = "cluster") without leverage adjustment or small-sample factor; CL2 with the diagonal-leverage approximation of each curve’s adjustment (“I − H_gg approx.”) or with the full-block adjustment, each in the Bayesian form (+ V_p − V_e) and the frequentist form; and the model-based and full-block CL2 intervals of a refit with every smoothing parameter fixed at the cell’s geometric-mean REML value. Critical values z and \(t_{G-1}\).
What each component contributes (paired contrasts over replicates, mean and β, dependent cells): the leverage adjustment over CR1 1.0–4.2 pp, the full block over the \(\mathbf I - \tilde{\mathbf H}_{gg}\) approximation 0.1–0.4 pp, the Bayesian term \(\mathbf V_p - \mathbf V_e\) 0.5–3.0 pp (independent errors: 2.2–10.4 pp), a \(t_{G-1}\) critical value 0.3–0.8 pp, and fixing λ 0.4–2.2 pp for CL2 against -2.3 to 1.5 pp for the model-based intervals. For the binary intercept with smooth errors at G = 40, CL2 covers 0.885 and 0.925 with λ fixed; under independent errors 0.859 and 0.867. The pooled MC SEs of averaged contrasts in the CSV treat cells as independent although they share replicates; per-cell SEs (ablation-contrasts.csv) are exact.
| contrast | estimand | dependence | G | diff_min | diff_max | diff |
|---|---|---|---|---|---|---|
| Bayes - freq (CL2) | alpha | dependent | 40 | 0.005 | 0.016 | 0.008 |
| Bayes - freq (CL2) | alpha | dependent | 100 | 0.002 | 0.010 | 0.005 |
| Bayes - freq (CL2) | alpha | independent | 40 | 0.020 | 0.060 | 0.034 |
| Bayes - freq (CL2) | alpha | independent | 100 | 0.016 | 0.049 | 0.027 |
| Bayes - freq (CL2) | beta | dependent | 40 | 0.011 | 0.030 | 0.018 |
| Bayes - freq (CL2) | beta | dependent | 100 | 0.009 | 0.022 | 0.014 |
| Bayes - freq (CL2) | beta | independent | 40 | 0.058 | 0.104 | 0.075 |
| Bayes - freq (CL2) | beta | independent | 100 | 0.045 | 0.073 | 0.056 |
| Bayes - freq (CL2) | gamma | dependent | 40 | 0.003 | 0.018 | 0.009 |
| Bayes - freq (CL2) | gamma | dependent | 100 | 0.004 | 0.011 | 0.006 |
| Bayes - freq (CL2) | gamma | independent | 40 | 0.028 | 0.069 | 0.043 |
| Bayes - freq (CL2) | gamma | independent | 100 | 0.013 | 0.052 | 0.028 |
| Bayes - freq (CL2) | mean | dependent | 40 | 0.006 | 0.018 | 0.010 |
| Bayes - freq (CL2) | mean | dependent | 100 | 0.004 | 0.012 | 0.007 |
| Bayes - freq (CL2) | mean | independent | 40 | 0.032 | 0.068 | 0.045 |
| Bayes - freq (CL2) | mean | independent | 100 | 0.022 | 0.051 | 0.033 |
| Bayes - freq (CR1) | alpha | dependent | 40 | 0.005 | 0.021 | 0.011 |
| Bayes - freq (CR1) | alpha | dependent | 100 | 0.002 | 0.013 | 0.006 |
| Bayes - freq (CR1) | alpha | independent | 40 | 0.032 | 0.072 | 0.045 |
| Bayes - freq (CR1) | alpha | independent | 100 | 0.016 | 0.052 | 0.028 |
| Bayes - freq (CR1) | beta | dependent | 40 | 0.017 | 0.046 | 0.028 |
| Bayes - freq (CR1) | beta | dependent | 100 | 0.011 | 0.027 | 0.017 |
| Bayes - freq (CR1) | beta | independent | 40 | 0.083 | 0.130 | 0.101 |
| Bayes - freq (CR1) | beta | independent | 100 | 0.054 | 0.084 | 0.066 |
| Bayes - freq (CR1) | gamma | dependent | 40 | 0.008 | 0.024 | 0.014 |
| Bayes - freq (CR1) | gamma | dependent | 100 | 0.003 | 0.013 | 0.007 |
| Bayes - freq (CR1) | gamma | independent | 40 | 0.036 | 0.087 | 0.057 |
| Bayes - freq (CR1) | gamma | independent | 100 | 0.016 | 0.059 | 0.033 |
| Bayes - freq (CR1) | mean | dependent | 40 | 0.009 | 0.025 | 0.015 |
| Bayes - freq (CR1) | mean | dependent | 100 | 0.005 | 0.015 | 0.009 |
| Bayes - freq (CR1) | mean | independent | 40 | 0.046 | 0.084 | 0.060 |
| Bayes - freq (CR1) | mean | independent | 100 | 0.026 | 0.058 | 0.038 |
| CL2 - CR1 | alpha | dependent | 40 | 0.025 | 0.034 | 0.029 |
| CL2 - CR1 | alpha | dependent | 100 | 0.007 | 0.012 | 0.009 |
| CL2 - CR1 | alpha | independent | 40 | 0.011 | 0.014 | 0.013 |
| CL2 - CR1 | alpha | independent | 100 | 0.004 | 0.010 | 0.007 |
| CL2 - CR1 | beta | dependent | 40 | 0.031 | 0.042 | 0.038 |
| CL2 - CR1 | beta | dependent | 100 | 0.010 | 0.014 | 0.012 |
| CL2 - CR1 | beta | independent | 40 | 0.004 | 0.005 | 0.005 |
| CL2 - CR1 | beta | independent | 100 | 0.001 | 0.003 | 0.002 |
| CL2 - CR1 | gamma | dependent | 40 | 0.032 | 0.048 | 0.039 |
| CL2 - CR1 | gamma | dependent | 100 | 0.011 | 0.016 | 0.013 |
| CL2 - CR1 | gamma | independent | 40 | 0.018 | 0.024 | 0.022 |
| CL2 - CR1 | gamma | independent | 100 | 0.009 | 0.010 | 0.009 |
| CL2 - CR1 | mean | dependent | 40 | 0.033 | 0.038 | 0.036 |
| CL2 - CR1 | mean | dependent | 100 | 0.011 | 0.014 | 0.012 |
| CL2 - CR1 | mean | independent | 40 | 0.014 | 0.016 | 0.015 |
| CL2 - CR1 | mean | independent | 100 | 0.006 | 0.007 | 0.006 |
| exact - shortcut | alpha | dependent | 40 | 0.002 | 0.004 | 0.003 |
| exact - shortcut | alpha | dependent | 100 | 0.000 | 0.001 | 0.001 |
| exact - shortcut | alpha | independent | 40 | 0.001 | 0.003 | 0.002 |
| exact - shortcut | alpha | independent | 100 | 0.001 | 0.002 | 0.001 |
| exact - shortcut | beta | dependent | 40 | 0.003 | 0.004 | 0.003 |
| exact - shortcut | beta | dependent | 100 | 0.001 | 0.001 | 0.001 |
| exact - shortcut | beta | independent | 40 | 0.001 | 0.001 | 0.001 |
| exact - shortcut | beta | independent | 100 | 0.000 | 0.000 | 0.000 |
| exact - shortcut | gamma | dependent | 40 | 0.002 | 0.006 | 0.004 |
| exact - shortcut | gamma | dependent | 100 | 0.000 | 0.002 | 0.001 |
| exact - shortcut | gamma | independent | 40 | 0.003 | 0.004 | 0.003 |
| exact - shortcut | gamma | independent | 100 | 0.001 | 0.002 | 0.001 |
| exact - shortcut | mean | dependent | 40 | 0.003 | 0.004 | 0.003 |
| exact - shortcut | mean | dependent | 100 | 0.001 | 0.001 | 0.001 |
| exact - shortcut | mean | independent | 40 | 0.002 | 0.002 | 0.002 |
| exact - shortcut | mean | independent | 100 | 0.001 | 0.001 | 0.001 |
| fixed - reselected (CL2) | alpha | dependent | 40 | 0.004 | 0.040 | 0.014 |
| fixed - reselected (CL2) | alpha | dependent | 100 | 0.001 | 0.016 | 0.006 |
| fixed - reselected (CL2) | alpha | independent | 40 | 0.004 | 0.008 | 0.005 |
| fixed - reselected (CL2) | alpha | independent | 100 | 0.002 | 0.015 | 0.006 |
| fixed - reselected (CL2) | beta | dependent | 40 | 0.012 | 0.022 | 0.016 |
| fixed - reselected (CL2) | beta | dependent | 100 | 0.007 | 0.012 | 0.009 |
| fixed - reselected (CL2) | beta | independent | 40 | 0.001 | 0.017 | 0.006 |
| fixed - reselected (CL2) | beta | independent | 100 | 0.000 | 0.002 | 0.001 |
| fixed - reselected (CL2) | gamma | dependent | 40 | 0.004 | 0.031 | 0.013 |
| fixed - reselected (CL2) | gamma | dependent | 100 | 0.002 | 0.022 | 0.008 |
| fixed - reselected (CL2) | gamma | independent | 40 | 0.001 | 0.039 | 0.014 |
| fixed - reselected (CL2) | gamma | independent | 100 | 0.001 | 0.026 | 0.010 |
| fixed - reselected (CL2) | mean | dependent | 40 | 0.006 | 0.020 | 0.012 |
| fixed - reselected (CL2) | mean | dependent | 100 | 0.004 | 0.012 | 0.006 |
| fixed - reselected (CL2) | mean | independent | 40 | 0.002 | 0.018 | 0.007 |
| fixed - reselected (CL2) | mean | independent | 100 | 0.001 | 0.011 | 0.004 |
| fixed - reselected (model) | alpha | dependent | 40 | -0.001 | 0.008 | 0.003 |
| fixed - reselected (model) | alpha | dependent | 100 | -0.012 | 0.000 | -0.004 |
| fixed - reselected (model) | alpha | independent | 40 | -0.035 | 0.000 | -0.012 |
| fixed - reselected (model) | alpha | independent | 100 | -0.003 | 0.001 | -0.001 |
| fixed - reselected (model) | beta | dependent | 40 | -0.018 | 0.015 | -0.004 |
| fixed - reselected (model) | beta | dependent | 100 | -0.023 | 0.004 | -0.008 |
| fixed - reselected (model) | beta | independent | 40 | -0.002 | 0.007 | 0.001 |
| fixed - reselected (model) | beta | independent | 100 | -0.001 | -0.001 | -0.001 |
| fixed - reselected (model) | gamma | dependent | 40 | -0.003 | 0.012 | 0.001 |
| fixed - reselected (model) | gamma | dependent | 100 | -0.008 | 0.007 | -0.002 |
| fixed - reselected (model) | gamma | independent | 40 | 0.000 | 0.003 | 0.002 |
| fixed - reselected (model) | gamma | independent | 100 | 0.000 | 0.007 | 0.002 |
| fixed - reselected (model) | mean | dependent | 40 | -0.005 | 0.003 | -0.002 |
| fixed - reselected (model) | mean | dependent | 100 | -0.007 | -0.001 | -0.003 |
| fixed - reselected (model) | mean | independent | 40 | -0.006 | -0.001 | -0.003 |
| fixed - reselected (model) | mean | independent | 100 | -0.001 | 0.000 | 0.000 |
| t - z (CL2) | alpha | dependent | 40 | 0.008 | 0.012 | 0.009 |
| t - z (CL2) | alpha | dependent | 100 | 0.002 | 0.004 | 0.003 |
| t - z (CL2) | alpha | independent | 40 | 0.004 | 0.009 | 0.007 |
| t - z (CL2) | alpha | independent | 100 | 0.002 | 0.003 | 0.002 |
| t - z (CL2) | beta | dependent | 40 | 0.007 | 0.008 | 0.008 |
| t - z (CL2) | beta | dependent | 100 | 0.003 | 0.003 | 0.003 |
| t - z (CL2) | beta | independent | 40 | 0.002 | 0.003 | 0.002 |
| t - z (CL2) | beta | independent | 100 | 0.001 | 0.001 | 0.001 |
| t - z (CL2) | gamma | dependent | 40 | 0.006 | 0.011 | 0.008 |
| t - z (CL2) | gamma | dependent | 100 | 0.002 | 0.004 | 0.003 |
| t - z (CL2) | gamma | independent | 40 | 0.006 | 0.010 | 0.008 |
| t - z (CL2) | gamma | independent | 100 | 0.003 | 0.004 | 0.003 |
| t - z (CL2) | mean | dependent | 40 | 0.007 | 0.008 | 0.008 |
| t - z (CL2) | mean | dependent | 100 | 0.003 | 0.003 | 0.003 |
| t - z (CL2) | mean | independent | 40 | 0.005 | 0.008 | 0.006 |
| t - z (CL2) | mean | independent | 100 | 0.002 | 0.003 | 0.002 |
| arm | dependence | estimand | se_sd | width |
|---|---|---|---|---|
| CL2 (full block) | dependent | beta | 0.943 | 3.752 |
| CL2 (full block) | dependent | mean | 0.990 | 1.093 |
| CL2 (full block) | independent | beta | 1.370 | 1.137 |
| CL2 (full block) | independent | mean | 1.108 | 0.462 |
| CL2 (full block, freq.) | dependent | beta | 0.899 | 3.528 |
| CL2 (full block, freq.) | dependent | mean | 0.965 | 1.069 |
| CL2 (full block, freq.) | independent | beta | 0.982 | 0.791 |
| CL2 (full block, freq.) | independent | mean | 0.970 | 0.409 |
| CL2, I - H_gg approx. | dependent | beta | 0.935 | 3.714 |
| CL2, I - H_gg approx. | dependent | mean | 0.982 | 1.082 |
| CL2, I - H_gg approx. | independent | beta | 1.362 | 1.130 |
| CL2, I - H_gg approx. | independent | mean | 1.099 | 0.458 |
| CL2, I - H_gg approx. (freq.) | dependent | beta | 0.891 | 3.488 |
| CL2, I - H_gg approx. (freq.) | dependent | mean | 0.956 | 1.057 |
| CL2, I - H_gg approx. (freq.) | independent | beta | 0.970 | 0.782 |
| CL2, I - H_gg approx. (freq.) | independent | mean | 0.960 | 0.404 |
| CR1 | dependent | beta | 0.861 | 3.407 |
| CR1 | dependent | mean | 0.908 | 0.983 |
| CR1 | independent | beta | 1.320 | 1.097 |
| CR1 | independent | mean | 1.051 | 0.434 |
| CR1 (freq.) | dependent | beta | 0.813 | 3.161 |
| CR1 (freq.) | dependent | mean | 0.880 | 0.956 |
| CR1 (freq.) | independent | beta | 0.911 | 0.734 |
| CR1 (freq.) | independent | mean | 0.905 | 0.377 |
| fixed lambda, CL2 | dependent | beta | 1.058 | 3.729 |
| fixed lambda, CL2 | dependent | mean | 1.039 | 1.103 |
| fixed lambda, CL2 | independent | beta | 1.422 | 1.133 |
| fixed lambda, CL2 | independent | mean | 1.156 | 0.462 |
| fixed lambda, model-based | dependent | beta | 0.539 | 1.962 |
| fixed lambda, model-based | dependent | mean | 0.508 | 0.516 |
| fixed lambda, model-based | independent | beta | 1.413 | 1.129 |
| fixed lambda, model-based | independent | mean | 1.146 | 0.462 |
| model-based (Vc) | dependent | beta | 0.504 | 2.119 |
| model-based (Vc) | dependent | mean | 0.505 | 0.528 |
| model-based (Vc) | independent | beta | 1.414 | 1.185 |
| model-based (Vc) | independent | mean | 1.135 | 0.471 |
4.2 Satterthwaite degrees of freedom
The per-point Satterthwaite df (median per cell) are 13–27 at G = 40 and 32–84 at G = 100 (relative to G − 1: 0.32–0.85). Satterthwaite adds 0.6–2.7 pp over z at G = 40 and 0.2–1.2 pp at G = 100 for the mean, β and γ, and 0.2–1.8 pp over \(t_{G-1}\). The worst cell under Satterthwaite is binomial alpha with iid errors at G = 40 (0.872): the binary intercept and γ shortfalls are a bias, not a scale problem. refund’s df are computed for the sandwich part only; the variant that accounts for \(\mathbf V_p - \mathbf V_e\) (“B2”) has larger df and lies between \(t_{G-1}\) and the plain Satterthwaite reference (satterthwaite-contrasts-summary.csv). The diagonal Gram matrix gives practically the same df.
| contrast | estimand | dependence | G | diff_min | diff_max | diff | width_ratio |
|---|---|---|---|---|---|---|---|
| Satt - t | alpha | dependent | 40 | 0.002 | 0.007 | 0.004 | 1.017 |
| Satt - t | alpha | dependent | 100 | 0.000 | 0.001 | 0.000 | 1.003 |
| Satt - t | alpha | independent | 40 | 0.002 | 0.004 | 0.003 | 1.013 |
| Satt - t | alpha | independent | 100 | 0.000 | 0.001 | 0.000 | 1.002 |
| Satt - t | beta | dependent | 40 | 0.015 | 0.017 | 0.016 | 1.074 |
| Satt - t | beta | dependent | 100 | 0.005 | 0.007 | 0.006 | 1.028 |
| Satt - t | beta | independent | 40 | 0.004 | 0.005 | 0.004 | 1.071 |
| Satt - t | beta | independent | 100 | 0.002 | 0.003 | 0.002 | 1.028 |
| Satt - t | gamma | dependent | 40 | 0.012 | 0.018 | 0.015 | 1.077 |
| Satt - t | gamma | dependent | 100 | 0.005 | 0.009 | 0.007 | 1.028 |
| Satt - t | gamma | independent | 40 | 0.010 | 0.017 | 0.014 | 1.076 |
| Satt - t | gamma | independent | 100 | 0.005 | 0.006 | 0.006 | 1.028 |
| Satt - t | mean | dependent | 40 | 0.013 | 0.014 | 0.014 | 1.072 |
| Satt - t | mean | dependent | 100 | 0.005 | 0.006 | 0.005 | 1.027 |
| Satt - t | mean | independent | 40 | 0.010 | 0.013 | 0.011 | 1.071 |
| Satt - t | mean | independent | 100 | 0.004 | 0.005 | 0.004 | 1.026 |
| Satt - z | alpha | dependent | 40 | 0.010 | 0.018 | 0.013 | 1.050 |
| Satt - z | alpha | dependent | 100 | 0.002 | 0.006 | 0.003 | 1.015 |
| Satt - z | alpha | independent | 40 | 0.008 | 0.013 | 0.010 | 1.045 |
| Satt - z | alpha | independent | 100 | 0.003 | 0.003 | 0.003 | 1.014 |
| Satt - z | beta | dependent | 40 | 0.022 | 0.025 | 0.024 | 1.108 |
| Satt - z | beta | dependent | 100 | 0.008 | 0.010 | 0.009 | 1.041 |
| Satt - z | beta | independent | 40 | 0.006 | 0.009 | 0.007 | 1.106 |
| Satt - z | beta | independent | 100 | 0.002 | 0.004 | 0.003 | 1.041 |
| Satt - z | gamma | dependent | 40 | 0.019 | 0.025 | 0.022 | 1.111 |
| Satt - z | gamma | dependent | 100 | 0.009 | 0.012 | 0.010 | 1.041 |
| Satt - z | gamma | independent | 40 | 0.017 | 0.027 | 0.021 | 1.110 |
| Satt - z | gamma | independent | 100 | 0.009 | 0.009 | 0.009 | 1.041 |
| Satt - z | mean | dependent | 40 | 0.021 | 0.022 | 0.022 | 1.107 |
| Satt - z | mean | dependent | 100 | 0.008 | 0.009 | 0.008 | 1.040 |
| Satt - z | mean | independent | 40 | 0.015 | 0.020 | 0.017 | 1.105 |
| Satt - z | mean | independent | 100 | 0.006 | 0.008 | 0.007 | 1.039 |
| Satt B2 - t | alpha | dependent | 40 | 0.002 | 0.005 | 0.003 | 1.013 |
| Satt B2 - t | alpha | dependent | 100 | 0.000 | 0.000 | 0.000 | 1.001 |
| Satt B2 - t | alpha | independent | 40 | -0.004 | 0.000 | -0.001 | 0.998 |
| Satt B2 - t | alpha | independent | 100 | -0.001 | 0.000 | 0.000 | 0.998 |
| Satt B2 - t | beta | dependent | 40 | 0.008 | 0.012 | 0.011 | 1.051 |
| Satt B2 - t | beta | dependent | 100 | 0.003 | 0.005 | 0.004 | 1.020 |
| Satt B2 - t | beta | independent | 40 | -0.001 | 0.000 | 0.000 | 0.993 |
| Satt B2 - t | beta | independent | 100 | 0.000 | 0.000 | 0.000 | 1.000 |
| Satt B2 - t | gamma | dependent | 40 | 0.011 | 0.016 | 0.012 | 1.065 |
| Satt B2 - t | gamma | dependent | 100 | 0.004 | 0.007 | 0.006 | 1.025 |
| Satt B2 - t | gamma | independent | 40 | 0.004 | 0.007 | 0.005 | 1.034 |
| Satt B2 - t | gamma | independent | 100 | 0.002 | 0.004 | 0.003 | 1.016 |
| Satt B2 - t | mean | dependent | 40 | 0.009 | 0.012 | 0.011 | 1.060 |
| Satt B2 - t | mean | dependent | 100 | 0.004 | 0.005 | 0.005 | 1.023 |
| Satt B2 - t | mean | independent | 40 | 0.002 | 0.004 | 0.003 | 1.025 |
| Satt B2 - t | mean | independent | 100 | 0.001 | 0.002 | 0.002 | 1.012 |
| Satt diag - Satt | alpha | dependent | 40 | -0.002 | 0.000 | -0.001 | 0.996 |
| Satt diag - Satt | alpha | dependent | 100 | 0.000 | 0.000 | 0.000 | 0.999 |
| Satt diag - Satt | alpha | independent | 40 | -0.001 | 0.000 | -0.001 | 0.997 |
| Satt diag - Satt | alpha | independent | 100 | 0.000 | 0.000 | 0.000 | 0.999 |
| Satt diag - Satt | beta | dependent | 40 | -0.001 | 0.001 | 0.000 | 1.001 |
| Satt diag - Satt | beta | dependent | 100 | 0.000 | 0.001 | 0.000 | 1.001 |
| Satt diag - Satt | beta | independent | 40 | 0.000 | 0.000 | 0.000 | 1.003 |
| Satt diag - Satt | beta | independent | 100 | 0.000 | 0.000 | 0.000 | 1.001 |
| Satt diag - Satt | gamma | dependent | 40 | -0.001 | 0.002 | 0.000 | 1.004 |
| Satt diag - Satt | gamma | dependent | 100 | 0.000 | 0.001 | 0.000 | 1.001 |
| Satt diag - Satt | gamma | independent | 40 | -0.001 | 0.002 | 0.000 | 1.005 |
| Satt diag - Satt | gamma | independent | 100 | 0.000 | 0.000 | 0.000 | 1.001 |
| Satt diag - Satt | mean | dependent | 40 | -0.001 | 0.001 | 0.000 | 1.003 |
| Satt diag - Satt | mean | dependent | 100 | 0.000 | 0.000 | 0.000 | 1.001 |
| Satt diag - Satt | mean | independent | 40 | 0.000 | 0.000 | 0.000 | 1.004 |
| Satt diag - Satt | mean | independent | 100 | 0.000 | 0.000 | 0.000 | 1.001 |
| estimand | G | df_q05 | df_median | df_q95 | df_rel | df_b2 |
|---|---|---|---|---|---|---|
| alpha | 40 | 20.85 | 27.1 | 32.3 | 0.69 | 34.2 |
| alpha | 100 | 73.13 | 84.0 | 91.3 | 0.85 | 100.3 |
| beta | 40 | 8.58 | 13.2 | 17.4 | 0.34 | 30.6 |
| beta | 100 | 22.92 | 32.3 | 40.6 | 0.33 | 65.7 |
| gamma | 40 | 8.35 | 12.8 | 17.0 | 0.33 | 16.5 |
| gamma | 100 | 22.44 | 32.0 | 40.4 | 0.32 | 38.8 |
| mean | 40 | 8.93 | 14.0 | 19.1 | 0.36 | 19.2 |
| mean | 100 | 24.35 | 35.2 | 46.5 | 0.36 | 45.4 |
4.3 mgcv’s covariance for curve-blocked NCV fits
With nei$jackknife = TRUE, mgcv replaces \(\mathbf V_p\) of an NCV fit by a neighbourhood-aware jackknife covariance. For curve neighbourhoods (smooth errors, G = 100, current patched mgcv 1.9-5, the same NCV fits) its standard errors are 0.49–0.77 times the Bayesian ones, and it covers the mean and β at 0.32–0.58, against 0.88–0.95 for CL2 on the same fits (Table 23). mgcv 1.9-1 and 1.9-3 give the same result.
| family | estimand | arm | coverage | coverage_se | se_ratio_to_bayes | width | reps |
|---|---|---|---|---|---|---|---|
| binary | beta | Bayesian Vp | 0.704 | 1.000 | 0.993 | 20 | |
| binary | beta | CL2 | 0.890 | 1.577 | 1.581 | 20 | |
| binary | beta | jackknife Vc (patched mgcv) | 0.615 | 0.828 | 0.835 | 20 | |
| binary | beta | jackknife Vp (jackknife = TRUE) | 0.583 | 0.765 | 0.751 | 20 | |
| binary | mean | Bayesian Vp | 0.608 | 1.000 | 0.088 | 20 | |
| binary | mean | CL2 | 0.878 | 1.867 | 0.163 | 20 | |
| binary | mean | jackknife Vc (patched mgcv) | 0.445 | 0.667 | 0.060 | 20 | |
| binary | mean | jackknife Vp (jackknife = TRUE) | 0.373 | 0.557 | 0.049 | 20 | |
| Gaussian | beta | Bayesian Vp | 0.699 | 1.000 | 0.599 | 20 | |
| Gaussian | beta | CL2 | 0.948 | 1.844 | 1.097 | 20 | |
| Gaussian | beta | jackknife Vc (patched mgcv) | 0.559 | 0.755 | 0.454 | 20 | |
| Gaussian | beta | jackknife Vp (jackknife = TRUE) | 0.538 | 0.726 | 0.427 | 20 | |
| Gaussian | mean | Bayesian Vp | 0.619 | 1.000 | 0.306 | 20 | |
| Gaussian | mean | CL2 | 0.931 | 2.135 | 0.644 | 20 | |
| Gaussian | mean | jackknife Vc (patched mgcv) | 0.365 | 0.543 | 0.168 | 20 | |
| Gaussian | mean | jackknife Vp (jackknife = TRUE) | 0.324 | 0.485 | 0.148 | 20 | |
| Poisson | beta | Bayesian Vp | 0.698 | 1.000 | 0.341 | 20 | |
| Poisson | beta | CL2 | 0.937 | 1.781 | 0.606 | 20 | |
| Poisson | beta | jackknife Vc (patched mgcv) | 0.568 | 0.771 | 0.266 | 20 | |
| Poisson | beta | jackknife Vp (jackknife = TRUE) | 0.545 | 0.734 | 0.246 | 20 | |
| Poisson | mean | Bayesian Vp | 0.613 | 1.000 | 0.521 | 20 | |
| Poisson | mean | CL2 | 0.917 | 2.061 | 1.076 | 20 | |
| Poisson | mean | jackknife Vc (patched mgcv) | 0.373 | 0.562 | 0.294 | 20 | |
| Poisson | mean | jackknife Vp (jackknife = TRUE) | 0.331 | 0.499 | 0.259 | 20 |
5 Small G, a null effect and Satterthwaite critical values
Three sets of runs, 200 replicates per cell. (i) Per-point Satterthwaite critical values for REML + CL2 (exact CL2, Bayesian form; as in the Satterthwaite subsection above) on every synthetic cell outside the core block and on the plasmode cells with curve flips and attached residuals (nine datasets, G = 40 and all subjects; satterthwaite/). (ii) The core factorial at G = 20 (cells 124–144): the first 20 curves of each replicate, so the cells share their draws with the G = 40 and G = 100 core cells. (iii) β = 0 (cells 145–152): the smooth-truth setting at the default signal with β set to zero, Gaussian and binary, independent and smooth errors, G = 40 and 100 (small-g-null/). Critical values: z = z0.975, \(t_{G-1}\), and “Satt”, the per-point Satterthwaite df of refund’s coef.pffr(crit = "satterthwaite") (full Gram matrix).
5.1 Satterthwaite critical values beyond the core cells
Checks. Task files: 22,000 synthetic and 24,000 plasmode, 0 failed. The REML estimates, CL2 SEs, model-based SEs and truths reproduce the study’s stored records of the same replicates (15,400 synthetic and 24,000 plasmode task records; largest relative difference 9.4e-06 and 0.0e+00). Scored with z, coverage equals the study’s REML + CL2 summary in all 771 cell × estimand rows compared (largest absolute difference 0.0e+00, the same replicate count in every row). The CL2 covariance rebuilt from the df computation equals refund’s (largest relative difference 0.0e+00), the df use the cell’s G in every replicate, and 0 per-point df are undefined. Software: refund 79a346fb, mgcv 1.9-5.
Synthetic extensions. Satterthwaite never gives lower coverage than z: the smallest paired difference over all synthetic and plasmode cell × estimand rows is 0.01 pp. Over the extension blocks it adds 2.16 pp at G = 40 and 0.75 pp at G = 100 on average (mean width ratio 1.100 and 1.037; Table 26). Coverage under Satterthwaite is below 0.93 in 24 cell × estimand rows (Table 25): misregistration (Poisson, binary; the mean, α, γ); dense grid (binary; α); rough truth (binary; α, γ); low-rank covariate (binary; α); basis size (binary; α). The lowest is Poisson α in the misregistration block at G = 100 (0.865; z 0.858).
Plasmode. Satterthwaite adds 2.29 pp at G = 40 and 1.00 pp with all subjects on average (width ratio 1.115 and 1.047). Averaged over truth and residual sources, coverage stays below 0.93 for gait α (0.925 at G = 40); weather β (0.863 at G = 40, 0.899 at G = 73); weather γ (0.921 at G = 40, 0.927 at G = 73); electricity α (0.879 at G = 40, 0.912 at G = 102) (Table 27). The curve sign flips make the plasmode curves independent, so these shortfalls do not come from dependence between curves.
Degrees of freedom. For β (f), γ and the mean, the median per-point df of a cell is 0.22–0.47 times G − 1 over all synthetic cells outside the core block (including G = 20 and β = 0; median over cells 0.34) and 0.18–0.83 on the plasmode cells (median 0.34); for α it is 0.44–0.97 (plasmode 0.24–0.59; Table 28). For these terms the Satterthwaite reference thus amounts to a t distribution with far fewer df than \(t_{G-1}\).
| block | G | cells | mean z | mean Satt | beta z | beta Satt | alpha z | alpha Satt | gamma z | gamma Satt |
|---|---|---|---|---|---|---|---|---|---|---|
| misregistration | 40 | 6 | 0.924 | 0.948 | 0.942 | 0.964 | 0.930 | 0.943 | 0.893 | 0.924 |
| misregistration | 100 | 6 | 0.930 | 0.940 | 0.943 | 0.953 | 0.919 | 0.923 | 0.906 | 0.922 |
| misregistration (AR(1) cell) | 100 | 1 | 0.937 | 0.946 | 0.944 | 0.953 | 0.949 | 0.951 | 0.928 | 0.940 |
| dense grid | 100 | 9 | 0.947 | 0.955 | 0.954 | 0.962 | 0.943 | 0.947 | 0.944 | 0.953 |
| rough truth | 100 | 14 | 0.944 | 0.953 | 0.948 | 0.957 | 0.945 | 0.948 | 0.936 | 0.946 |
| term type f(x,t) | 100 | 9 | 0.945 | 0.960 | 0.955 | 0.964 | 0.941 | 0.951 | ||
| extended families | 100 | 8 | 0.950 | 0.958 | 0.962 | 0.968 | 0.948 | 0.951 | 0.948 | 0.957 |
| AR(1) home turf | 100 | 1 | 0.942 | 0.950 | 0.940 | 0.949 | 0.955 | 0.957 | 0.937 | 0.947 |
| sign-changing | 100 | 3 | 0.942 | 0.950 | 0.947 | 0.956 | 0.944 | 0.948 | 0.940 | 0.950 |
| heteroskedastic | 40 | 3 | 0.934 | 0.955 | 0.935 | 0.958 | 0.935 | 0.946 | 0.932 | 0.956 |
| heteroskedastic | 100 | 3 | 0.944 | 0.952 | 0.948 | 0.956 | 0.948 | 0.951 | 0.942 | 0.952 |
| low-rank covariate | 100 | 6 | 0.949 | 0.956 | 0.970 | 0.976 | 0.942 | 0.945 | 0.943 | 0.953 |
| basis size | 100 | 12 | 0.962 | 0.968 | 0.976 | 0.980 | 0.950 | 0.953 | 0.954 | 0.962 |
| block | family | error | G | signal | truth | estimand | z | t(G−1) | Satt | q05 Satt |
|---|---|---|---|---|---|---|---|---|---|---|
| misregistration | Poisson | misreg. | 40 | high | wiggly | E(Y | X) | 0.899 | 0.909 | 0.930 | 0.865 |
| misregistration | Poisson | misreg. | 100 | high | smooth | E(Y | X) | 0.910 | 0.914 | 0.924 | 0.850 |
| misregistration | Poisson | misreg. | 100 | high | wiggly | E(Y | X) | 0.906 | 0.909 | 0.919 | 0.850 |
| misregistration | binary | misreg. | 40 | high | wiggly | alpha(t) | 0.911 | 0.921 | 0.925 | 0.873 |
| misregistration | binary | misreg. | 100 | high | wiggly | alpha(t) | 0.913 | 0.917 | 0.917 | 0.818 |
| misregistration | Poisson | misreg. | 100 | high | smooth | alpha(t) | 0.905 | 0.909 | 0.910 | 0.848 |
| misregistration | Poisson | misreg. | 100 | high | wiggly | alpha(t) | 0.858 | 0.864 | 0.865 | 0.713 |
| misregistration | binary | misreg. | 40 | high | smooth | gamma(t) | 0.897 | 0.907 | 0.928 | 0.885 |
| misregistration | binary | misreg. | 40 | high | wiggly | gamma(t) | 0.867 | 0.877 | 0.900 | 0.740 |
| misregistration | binary | misreg. | 100 | high | wiggly | gamma(t) | 0.881 | 0.885 | 0.896 | 0.743 |
| misregistration | Poisson | misreg. | 40 | high | smooth | gamma(t) | 0.879 | 0.890 | 0.918 | 0.855 |
| misregistration | Poisson | misreg. | 40 | high | wiggly | gamma(t) | 0.873 | 0.882 | 0.909 | 0.810 |
| misregistration | Poisson | misreg. | 100 | high | smooth | gamma(t) | 0.894 | 0.899 | 0.915 | 0.840 |
| misregistration | Poisson | misreg. | 100 | high | wiggly | gamma(t) | 0.885 | 0.889 | 0.906 | 0.802 |
| dense grid | binary | OU | 100 | mid | smooth | alpha(t) | 0.919 | 0.924 | 0.925 | 0.887 |
| dense grid | binary | smooth | 100 | mid | smooth | alpha(t) | 0.920 | 0.923 | 0.925 | 0.890 |
| rough truth | binary | iid | 100 | mid | wiggly | alpha(t) | 0.921 | 0.924 | 0.924 | 0.775 |
| rough truth | binary | smooth | 100 | mid | wiggly | alpha(t) | 0.920 | 0.924 | 0.925 | 0.875 |
| rough truth | binary | iid | 100 | mid | wiggly | gamma(t) | 0.877 | 0.882 | 0.890 | 0.710 |
| rough truth | binary | iid | 100 | high | wiggly | gamma(t) | 0.916 | 0.920 | 0.928 | 0.868 |
| low-rank covariate | binary | iid | 100 | mid | smooth | alpha(t) | 0.923 | 0.927 | 0.927 | 0.785 |
| low-rank covariate | binary | smooth | 100 | mid | smooth | alpha(t) | 0.922 | 0.926 | 0.926 | 0.885 |
| basis size | binary | smooth | 100 | mid | smooth | alpha(t) | 0.915 | 0.919 | 0.920 | 0.865 |
| basis size | binary | smooth | 100 | mid | smooth | alpha(t) | 0.899 | 0.902 | 0.904 | 0.853 |
| setting | contrast | G | rows | diff (pp) | diff min | diff max | width ratio | width min | width max |
|---|---|---|---|---|---|---|---|---|---|
| plasmode | Satt - t | 40 | 240 | 1.525 | 0.177 | 3.484 | 1.08 | 1.019 | 1.18 |
| plasmode | Satt - t | all | 240 | 0.686 | 0.000 | 2.032 | 1.03 | 1.002 | 1.09 |
| plasmode | Satt - z | 40 | 240 | 2.287 | 0.839 | 4.274 | 1.11 | 1.052 | 1.22 |
| plasmode | Satt - z | all | 240 | 1.003 | 0.129 | 2.758 | 1.05 | 1.013 | 1.11 |
| plasmode | Satt B2 - t | 40 | 240 | 1.357 | 0.097 | 3.290 | 1.07 | 1.013 | 1.17 |
| plasmode | Satt B2 - t | all | 240 | 0.613 | -0.016 | 1.952 | 1.03 | 1.001 | 1.08 |
| plasmode | Satt diag - Satt | 40 | 240 | 0.050 | -0.355 | 2.242 | 1.01 | 0.991 | 1.10 |
| plasmode | Satt diag - Satt | all | 240 | 0.044 | -0.129 | 1.177 | 1.00 | 0.996 | 1.05 |
| synthetic extensions | Satt - t | 100 | 279 | 0.479 | 0.000 | 1.710 | 1.02 | 1.002 | 1.10 |
| synthetic extensions | Satt - t | 40 | 36 | 1.351 | 0.339 | 2.855 | 1.06 | 1.013 | 1.15 |
| synthetic extensions | Satt - z | 100 | 279 | 0.747 | 0.012 | 2.129 | 1.04 | 1.014 | 1.11 |
| synthetic extensions | Satt - z | 40 | 36 | 2.158 | 0.737 | 3.984 | 1.10 | 1.045 | 1.19 |
| synthetic extensions | Satt B2 - t | 100 | 279 | 0.332 | -0.145 | 1.581 | 1.02 | 0.990 | 1.08 |
| synthetic extensions | Satt B2 - t | 40 | 36 | 0.940 | -0.226 | 2.403 | 1.05 | 0.994 | 1.14 |
| synthetic extensions | Satt diag - Satt | 100 | 279 | 0.019 | -0.081 | 0.661 | 1.00 | 0.998 | 1.04 |
| synthetic extensions | Satt diag - Satt | 40 | 36 | 0.077 | -0.145 | 0.806 | 1.01 | 0.995 | 1.12 |
| app | G | cells | mean z | mean Satt | beta z | beta Satt | alpha z | alpha Satt | gamma z | gamma Satt |
|---|---|---|---|---|---|---|---|---|---|---|
| ECG strain | 40 | 12 | 0.936 | 0.964 | 0.934 | 0.959 | 0.949 | 0.967 | 0.943 | 0.977 |
| ECG strain | 78 | 12 | 0.944 | 0.960 | 0.944 | 0.957 | 0.949 | 0.960 | 0.951 | 0.967 |
| AF trial | 40 | 6 | 0.934 | 0.955 | 0.920 | 0.947 | 0.923 | 0.938 | 0.936 | 0.949 |
| AF trial | 118 | 6 | 0.946 | 0.953 | 0.938 | 0.947 | 0.943 | 0.948 | 0.946 | 0.949 |
| running | 40 | 6 | 0.922 | 0.948 | 0.902 | 0.931 | 0.927 | 0.950 | 0.926 | 0.951 |
| running | 90 | 6 | 0.933 | 0.944 | 0.919 | 0.932 | 0.939 | 0.950 | 0.933 | 0.943 |
| DTI | 40 | 6 | 0.918 | 0.944 | 0.917 | 0.943 | 0.936 | 0.956 | 0.933 | 0.951 |
| DTI | 92 | 6 | 0.932 | 0.943 | 0.927 | 0.938 | 0.937 | 0.947 | 0.932 | 0.937 |
| gait | 40 | 6 | 0.936 | 0.958 | 0.935 | 0.958 | 0.892 | 0.925 | 0.946 | 0.959 |
| gait | 138 | 6 | 0.943 | 0.950 | 0.939 | 0.946 | 0.930 | 0.939 | 0.932 | 0.938 |
| ECG 8-lead | 40 | 6 | 0.944 | 0.960 | 0.936 | 0.963 | 0.956 | 0.967 | 0.951 | 0.961 |
| ECG 8-lead | 100 | 6 | 0.952 | 0.959 | 0.942 | 0.955 | 0.956 | 0.960 | 0.960 | 0.963 |
| ocean | 40 | 6 | 0.932 | 0.958 | 0.928 | 0.956 | 0.931 | 0.954 | 0.933 | 0.954 |
| ocean | 116 | 6 | 0.935 | 0.943 | 0.925 | 0.936 | 0.943 | 0.947 | 0.947 | 0.952 |
| weather | 40 | 6 | 0.916 | 0.940 | 0.835 | 0.863 | 0.939 | 0.963 | 0.890 | 0.921 |
| weather | 73 | 6 | 0.924 | 0.938 | 0.882 | 0.899 | 0.927 | 0.943 | 0.907 | 0.927 |
| electricity | 40 | 6 | 0.922 | 0.943 | 0.914 | 0.941 | 0.859 | 0.879 | 0.966 | 0.977 |
| electricity | 102 | 6 | 0.927 | 0.936 | 0.927 | 0.941 | 0.904 | 0.912 | 0.950 | 0.953 |
| setting | G | estimand | cells | df min | df max | df/(G−1) min | df/(G−1) median | df/(G−1) max | undefined |
|---|---|---|---|---|---|---|---|---|---|
| plasmode | 40 | mean | 60 | 11.43 | 19.11 | 0.29 | 0.35 | 0.49 | 0 |
| plasmode | 40 | beta | 60 | 10.65 | 13.52 | 0.27 | 0.31 | 0.35 | 0 |
| plasmode | 40 | alpha | 60 | 12.36 | 20.92 | 0.32 | 0.35 | 0.54 | 0 |
| plasmode | 40 | gamma | 60 | 7.03 | 25.35 | 0.18 | 0.41 | 0.65 | 0 |
| plasmode | all | mean | 60 | 21.45 | 50.84 | 0.28 | 0.35 | 0.51 | 0 |
| plasmode | all | beta | 60 | 19.81 | 47.94 | 0.20 | 0.30 | 0.35 | 0 |
| plasmode | all | alpha | 60 | 22.62 | 57.98 | 0.24 | 0.34 | 0.59 | 0 |
| plasmode | all | gamma | 60 | 13.56 | 96.92 | 0.18 | 0.46 | 0.83 | 0 |
| synthetic | 20 | mean | 21 | 6.40 | 7.52 | 0.34 | 0.35 | 0.40 | 0 |
| synthetic | 20 | beta | 21 | 6.00 | 6.94 | 0.32 | 0.33 | 0.37 | 0 |
| synthetic | 20 | alpha | 21 | 8.44 | 12.20 | 0.44 | 0.52 | 0.64 | 0 |
| synthetic | 20 | gamma | 21 | 6.10 | 6.58 | 0.32 | 0.33 | 0.35 | 0 |
| synthetic | 40 | mean | 13 | 12.68 | 18.25 | 0.33 | 0.36 | 0.47 | 0 |
| synthetic | 40 | beta | 13 | 11.36 | 13.55 | 0.29 | 0.34 | 0.35 | 0 |
| synthetic | 40 | alpha | 13 | 24.53 | 35.97 | 0.63 | 0.65 | 0.92 | 0 |
| synthetic | 40 | gamma | 13 | 11.35 | 13.45 | 0.29 | 0.33 | 0.34 | 0 |
| synthetic | 100 | mean | 76 | 21.85 | 46.26 | 0.22 | 0.36 | 0.47 | 0 |
| synthetic | 100 | beta | 76 | 24.32 | 44.31 | 0.25 | 0.34 | 0.45 | 0 |
| synthetic | 100 | alpha | 67 | 71.99 | 96.05 | 0.73 | 0.83 | 0.97 | 0 |
| synthetic | 100 | gamma | 76 | 23.97 | 34.01 | 0.24 | 0.33 | 0.34 | 0 |
5.2 The core factorial at G = 20
Coverage. With dependent errors, REML + CL2 covers β, γ and the mean 0.89–0.92 at G = 20 (means over cells per family and estimand), against 0.92–0.94 at G = 40 and 0.94–0.95 at G = 100; for α the values are 0.85–0.91, 0.90–0.94 and 0.93–0.95, lowest for binary responses (Figure 12, Table 29). NCV + CL2 drops to 0.78–0.90, with 5% pointwise quantiles down to 0.50; it should not be used at this G. The bias-aware NCV interval covers 0.91–0.97 and the model-based REML interval 0.58–0.77. With independent errors, binary α and γ undercover for every arm, including the model-based interval (0.85–0.93; model-based 0.885 for α and 0.928 for γ), so this shortfall is a small-sample problem of the binary fit, not of the sandwich. REML + CL2 intervals at G = 20 are 1.14–2.10 times as wide as at G = 40 (per cell; geometric mean 1.50; Table 30).
Satterthwaite at G = 20 (mid signal only; the Satterthwaite runs cover only the mid-signal cells at this G). With dependent errors, REML + CL2 with the Satterthwaite critical value covers 0.94–0.96 for β, γ and the mean, against 0.89–0.92 with z and 0.91–0.93 with \(t_{G-1}\) (Table 31). Binary α stays at 0.897 (dependent) and 0.844 (independent). With independent Gaussian and Poisson errors Satterthwaite overcovers β, γ and the mean (0.972–0.998).
Accuracy and detection. At G = 20 with dependent errors, the per-replicate median relative error of β is Gaussian 0.30 vs 1.19, Poisson 0.29 vs 0.86, binary 0.74 vs 2.43 (NCV vs REML; medians over cells), and the cell-level relative RMSE is also lower for NCV in every family (Table 32). Detection of clearly non-zero β is 0.17–0.46 for REML + CL2 and 0.11–0.52 for the bias-aware NCV interval (G = 100: 0.42–0.83 and 0.47–0.87).
| family | dependence | estimand | REML, model-based | REML + CL2 | NCV + CL2 | NCV + CL2, bias-aware |
|---|---|---|---|---|---|---|
| Gaussian | independent | E(Y | X) | 0.966 (0.93) | 0.949 (0.91) | 0.941 (0.90) | 0.949 (0.91) |
| Gaussian | independent | beta(s,t) | 0.992 (0.97) | 0.987 (0.96) | 0.983 (0.95) | 0.988 (0.96) |
| Gaussian | independent | alpha(t) | 0.952 (0.89) | 0.934 (0.86) | 0.932 (0.86) | 0.937 (0.87) |
| Gaussian | independent | gamma(t) | 0.956 (0.93) | 0.932 (0.91) | 0.927 (0.90) | 0.932 (0.91) |
| Gaussian | dependent | E(Y | X) | 0.576 (0.50) | 0.909 (0.87) | 0.875 (0.79) | 0.943 (0.90) |
| Gaussian | dependent | beta(s,t) | 0.597 (0.51) | 0.912 (0.88) | 0.882 (0.80) | 0.975 (0.95) |
| Gaussian | dependent | alpha(t) | 0.580 (0.51) | 0.904 (0.86) | 0.864 (0.71) | 0.920 (0.83) |
| Gaussian | dependent | gamma(t) | 0.589 (0.53) | 0.917 (0.89) | 0.872 (0.81) | 0.927 (0.89) |
| Poisson | independent | E(Y | X) | 0.968 (0.94) | 0.950 (0.92) | 0.934 (0.89) | 0.945 (0.91) |
| Poisson | independent | beta(s,t) | 0.993 (0.98) | 0.989 (0.96) | 0.982 (0.95) | 0.989 (0.96) |
| Poisson | independent | alpha(t) | 0.956 (0.91) | 0.939 (0.90) | 0.920 (0.85) | 0.935 (0.89) |
| Poisson | independent | gamma(t) | 0.955 (0.93) | 0.931 (0.91) | 0.918 (0.90) | 0.928 (0.91) |
| Poisson | dependent | E(Y | X) | 0.650 (0.57) | 0.907 (0.87) | 0.874 (0.80) | 0.934 (0.89) |
| Poisson | dependent | beta(s,t) | 0.685 (0.59) | 0.912 (0.88) | 0.903 (0.84) | 0.969 (0.94) |
| Poisson | dependent | alpha(t) | 0.645 (0.56) | 0.906 (0.85) | 0.864 (0.76) | 0.923 (0.85) |
| Poisson | dependent | gamma(t) | 0.662 (0.60) | 0.914 (0.88) | 0.862 (0.80) | 0.919 (0.88) |
| binary | independent | E(Y | X) | 0.939 (0.85) | 0.903 (0.79) | 0.907 (0.82) | 0.926 (0.84) |
| binary | independent | beta(s,t) | 0.957 (0.86) | 0.935 (0.82) | 0.947 (0.87) | 0.959 (0.88) |
| binary | independent | alpha(t) | 0.885 (0.76) | 0.847 (0.66) | 0.850 (0.66) | 0.872 (0.71) |
| binary | independent | gamma(t) | 0.928 (0.87) | 0.875 (0.78) | 0.874 (0.79) | 0.893 (0.81) |
| binary | dependent | E(Y | X) | 0.722 (0.64) | 0.895 (0.85) | 0.831 (0.69) | 0.936 (0.87) |
| binary | dependent | beta(s,t) | 0.774 (0.70) | 0.912 (0.88) | 0.783 (0.62) | 0.954 (0.90) |
| binary | dependent | alpha(t) | 0.680 (0.57) | 0.852 (0.77) | 0.832 (0.67) | 0.932 (0.80) |
| binary | dependent | gamma(t) | 0.721 (0.66) | 0.895 (0.85) | 0.815 (0.71) | 0.913 (0.85) |
| family | dependence | estimand | REML 20 | REML 40 | REML 100 | NCV 20 | NCV 40 | NCV 100 | width |
|---|---|---|---|---|---|---|---|---|---|
| Gaussian | indep. | alpha | 0.934 | 0.945 | 0.953 | 0.932 | 0.939 | 0.945 | 1.42 |
| Gaussian | indep. | beta | 0.987 | 0.987 | 0.985 | 0.983 | 0.982 | 0.981 | 1.32 |
| Gaussian | indep. | gamma | 0.932 | 0.946 | 0.950 | 0.927 | 0.937 | 0.944 | 1.44 |
| Gaussian | indep. | mean | 0.949 | 0.957 | 0.960 | 0.941 | 0.947 | 0.950 | 1.41 |
| Gaussian | dep. | alpha | 0.904 | 0.937 | 0.950 | 0.864 | 0.904 | 0.937 | 1.51 |
| Gaussian | dep. | beta | 0.912 | 0.926 | 0.942 | 0.882 | 0.918 | 0.932 | 1.65 |
| Gaussian | dep. | gamma | 0.917 | 0.942 | 0.943 | 0.872 | 0.912 | 0.925 | 1.54 |
| Gaussian | dep. | mean | 0.909 | 0.931 | 0.943 | 0.875 | 0.912 | 0.928 | 1.56 |
| Poisson | indep. | alpha | 0.939 | 0.952 | 0.950 | 0.920 | 0.933 | 0.931 | 1.41 |
| Poisson | indep. | beta | 0.989 | 0.988 | 0.982 | 0.982 | 0.983 | 0.979 | 1.35 |
| Poisson | indep. | gamma | 0.931 | 0.945 | 0.949 | 0.918 | 0.933 | 0.942 | 1.48 |
| Poisson | indep. | mean | 0.950 | 0.956 | 0.958 | 0.934 | 0.943 | 0.945 | 1.50 |
| Poisson | dep. | alpha | 0.906 | 0.938 | 0.949 | 0.864 | 0.903 | 0.924 | 1.47 |
| Poisson | dep. | beta | 0.912 | 0.928 | 0.943 | 0.903 | 0.923 | 0.933 | 1.53 |
| Poisson | dep. | gamma | 0.914 | 0.937 | 0.941 | 0.862 | 0.904 | 0.924 | 1.55 |
| Poisson | dep. | mean | 0.907 | 0.932 | 0.943 | 0.874 | 0.908 | 0.925 | 1.83 |
| binary | indep. | alpha | 0.847 | 0.899 | 0.936 | 0.850 | 0.896 | 0.922 | 1.26 |
| binary | indep. | beta | 0.935 | 0.979 | 0.990 | 0.947 | 0.975 | 0.981 | 1.20 |
| binary | indep. | gamma | 0.875 | 0.906 | 0.929 | 0.874 | 0.893 | 0.917 | 1.34 |
| binary | indep. | mean | 0.903 | 0.937 | 0.954 | 0.907 | 0.930 | 0.941 | 1.31 |
| binary | dep. | alpha | 0.852 | 0.900 | 0.928 | 0.832 | 0.853 | 0.873 | 1.54 |
| binary | dep. | beta | 0.912 | 0.934 | 0.951 | 0.783 | 0.844 | 0.913 | 1.63 |
| binary | dep. | gamma | 0.895 | 0.922 | 0.936 | 0.815 | 0.822 | 0.867 | 1.59 |
| binary | dep. | mean | 0.895 | 0.925 | 0.942 | 0.831 | 0.861 | 0.895 | 1.38 |
| family | dependence | estimand | z 20 | t 20 | Satt 20 | q05 20 | z 40 | Satt 40 | z 100 | Satt 100 |
|---|---|---|---|---|---|---|---|---|---|---|
| Gaussian | indep. | alpha | 0.939 | 0.952 | 0.963 | 0.915 | 0.947 | 0.956 | 0.956 | 0.959 |
| Gaussian | indep. | beta | 0.989 | 0.993 | 0.998 | 0.990 | 0.989 | 0.995 | 0.986 | 0.989 |
| Gaussian | indep. | gamma | 0.937 | 0.950 | 0.974 | 0.955 | 0.951 | 0.969 | 0.950 | 0.959 |
| Gaussian | indep. | mean | 0.952 | 0.964 | 0.982 | 0.965 | 0.958 | 0.973 | 0.961 | 0.967 |
| Gaussian | dep. | alpha | 0.905 | 0.922 | 0.945 | 0.911 | 0.938 | 0.952 | 0.951 | 0.953 |
| Gaussian | dep. | beta | 0.913 | 0.930 | 0.961 | 0.938 | 0.926 | 0.950 | 0.942 | 0.951 |
| Gaussian | dep. | gamma | 0.918 | 0.934 | 0.964 | 0.948 | 0.942 | 0.961 | 0.943 | 0.953 |
| Gaussian | dep. | mean | 0.909 | 0.927 | 0.958 | 0.935 | 0.931 | 0.953 | 0.943 | 0.951 |
| Poisson | indep. | alpha | 0.937 | 0.953 | 0.963 | 0.912 | 0.953 | 0.961 | 0.951 | 0.954 |
| Poisson | indep. | beta | 0.990 | 0.993 | 0.998 | 0.990 | 0.989 | 0.995 | 0.986 | 0.989 |
| Poisson | indep. | gamma | 0.930 | 0.943 | 0.972 | 0.953 | 0.948 | 0.967 | 0.955 | 0.965 |
| Poisson | indep. | mean | 0.949 | 0.961 | 0.981 | 0.960 | 0.958 | 0.974 | 0.960 | 0.967 |
| Poisson | dep. | alpha | 0.904 | 0.923 | 0.941 | 0.891 | 0.934 | 0.945 | 0.949 | 0.952 |
| Poisson | dep. | beta | 0.912 | 0.929 | 0.960 | 0.935 | 0.928 | 0.953 | 0.943 | 0.953 |
| Poisson | dep. | gamma | 0.915 | 0.931 | 0.962 | 0.946 | 0.940 | 0.963 | 0.941 | 0.951 |
| Poisson | dep. | mean | 0.908 | 0.925 | 0.956 | 0.930 | 0.932 | 0.954 | 0.944 | 0.953 |
| binary | indep. | alpha | 0.804 | 0.825 | 0.844 | 0.613 | 0.859 | 0.872 | 0.924 | 0.927 |
| binary | indep. | beta | 0.898 | 0.911 | 0.933 | 0.800 | 0.973 | 0.982 | 0.990 | 0.993 |
| binary | indep. | gamma | 0.850 | 0.872 | 0.914 | 0.833 | 0.890 | 0.918 | 0.923 | 0.932 |
| binary | indep. | mean | 0.879 | 0.898 | 0.927 | 0.810 | 0.925 | 0.946 | 0.949 | 0.957 |
| binary | dep. | alpha | 0.856 | 0.879 | 0.897 | 0.821 | 0.894 | 0.909 | 0.923 | 0.928 |
| binary | dep. | beta | 0.910 | 0.928 | 0.958 | 0.935 | 0.932 | 0.954 | 0.951 | 0.960 |
| binary | dep. | gamma | 0.895 | 0.914 | 0.950 | 0.919 | 0.921 | 0.945 | 0.931 | 0.941 |
| binary | dep. | mean | 0.895 | 0.915 | 0.945 | 0.913 | 0.924 | 0.946 | 0.940 | 0.949 |
| estimand | family | dependence | G | cells | REML cell | NCV cell | REML med. | NCV med. | det. REML | det. NCV-b |
|---|---|---|---|---|---|---|---|---|---|---|
| mean | Gaussian | indep. | 20 | 3 | 0.197 | 0.197 | 0.193 | 0.193 | ||
| mean | Gaussian | indep. | 40 | 3 | 0.139 | 0.138 | 0.135 | 0.133 | ||
| mean | Gaussian | indep. | 100 | 3 | 0.091 | 0.089 | 0.091 | 0.088 | ||
| mean | Gaussian | dep. | 20 | 6 | 0.595 | 0.427 | 0.570 | 0.402 | ||
| mean | Gaussian | dep. | 40 | 6 | 0.365 | 0.283 | 0.358 | 0.270 | ||
| mean | Gaussian | dep. | 100 | 6 | 0.217 | 0.181 | 0.215 | 0.178 | ||
| mean | Poisson | indep. | 20 | 2 | 0.177 | 0.185 | 0.156 | 0.160 | ||
| mean | Poisson | indep. | 40 | 2 | 0.114 | 0.114 | 0.103 | 0.103 | ||
| mean | Poisson | indep. | 100 | 2 | 0.070 | 0.069 | 0.067 | 0.065 | ||
| mean | Poisson | dep. | 20 | 4 | 0.674 | 0.405 | 0.470 | 0.351 | ||
| mean | Poisson | dep. | 40 | 4 | 0.328 | 0.244 | 0.280 | 0.217 | ||
| mean | Poisson | dep. | 100 | 4 | 0.176 | 0.143 | 0.160 | 0.130 | ||
| mean | binary | indep. | 20 | 2 | 0.572 | 0.604 | 0.539 | 0.566 | ||
| mean | binary | indep. | 40 | 2 | 0.398 | 0.412 | 0.386 | 0.393 | ||
| mean | binary | indep. | 100 | 2 | 0.262 | 0.268 | 0.258 | 0.260 | ||
| mean | binary | dep. | 20 | 4 | 1.452 | 1.031 | 1.352 | 0.971 | ||
| mean | binary | dep. | 40 | 4 | 0.905 | 0.734 | 0.868 | 0.704 | ||
| mean | binary | dep. | 100 | 4 | 0.549 | 0.482 | 0.530 | 0.468 | ||
| beta | Gaussian | indep. | 20 | 3 | 0.193 | 0.182 | 0.185 | 0.159 | 0.903 | 0.912 |
| beta | Gaussian | indep. | 40 | 3 | 0.150 | 0.139 | 0.146 | 0.116 | 0.948 | 0.959 |
| beta | Gaussian | indep. | 100 | 3 | 0.112 | 0.090 | 0.109 | 0.083 | 0.977 | 0.982 |
| beta | Gaussian | dep. | 20 | 6 | 1.491 | 0.484 | 1.188 | 0.301 | 0.361 | 0.419 |
| beta | Gaussian | dep. | 40 | 6 | 0.865 | 0.287 | 0.680 | 0.216 | 0.533 | 0.601 |
| beta | Gaussian | dep. | 100 | 6 | 0.461 | 0.187 | 0.353 | 0.152 | 0.737 | 0.796 |
| beta | Poisson | indep. | 20 | 2 | 0.195 | 0.191 | 0.189 | 0.162 | 0.921 | 0.930 |
| beta | Poisson | indep. | 40 | 2 | 0.149 | 0.126 | 0.145 | 0.118 | 0.956 | 0.964 |
| beta | Poisson | indep. | 100 | 2 | 0.110 | 0.090 | 0.108 | 0.082 | 0.980 | 0.986 |
| beta | Poisson | dep. | 20 | 4 | 1.217 | 0.471 | 0.865 | 0.289 | 0.456 | 0.523 |
| beta | Poisson | dep. | 40 | 4 | 0.728 | 0.264 | 0.571 | 0.206 | 0.628 | 0.703 |
| beta | Poisson | dep. | 100 | 4 | 0.403 | 0.178 | 0.322 | 0.141 | 0.833 | 0.874 |
| beta | binary | indep. | 20 | 2 | 0.494 | 0.610 | 0.449 | 0.440 | 0.595 | 0.548 |
| beta | binary | indep. | 40 | 2 | 0.358 | 0.376 | 0.340 | 0.317 | 0.744 | 0.747 |
| beta | binary | indep. | 100 | 2 | 0.257 | 0.261 | 0.246 | 0.229 | 0.862 | 0.879 |
| beta | binary | dep. | 20 | 4 | 3.586 | 1.133 | 2.428 | 0.743 | 0.169 | 0.108 |
| beta | binary | dep. | 40 | 4 | 1.837 | 0.795 | 1.479 | 0.577 | 0.245 | 0.241 |
| beta | binary | dep. | 100 | 4 | 0.989 | 0.441 | 0.853 | 0.359 | 0.415 | 0.475 |
5.3 A null effect: β = 0
Pointwise false-positive rate (share of grid points whose interval excludes zero; nominal 5%; Table 33). With smooth errors, REML + CL2 has 5.4–7.6 pp, with \(t_{G-1}\) 5.1–6.8 pp and with Satterthwaite 4.6–5.2 pp. The model-based intervals reach 22.8–39.9 pp (REML) and 46.2–57.3 pp (NCV), NCV + CL2 11.1–15.1 pp. With independent errors REML + CL2 has 4.2–5.0 pp at G = 100 and 5.6 pp for binary at G = 40.
The exception is Gaussian, independent errors, G = 40: REML + CL2 has 9.5 pp (MC SE 1.8 pp) and Satterthwaite 7.5 pp, against 5.0 pp at G = 100. The excluded points are concentrated in a minority of replicates: 0.18 of the replicates have more than 5% of the grid excluding zero, and the 90% quantile of the per-replicate share is 0.24. Under β = 0 the REML fit shrinks β close to the null space of its penalty in most replicates, and in the remaining ones a large part of the grid excludes zero. This is the known weakness of Bayesian smoother intervals for terms shrunk to the penalty null space (Marra and Wood, 2012). It does not carry over to the non-null core cells, where the REML ff term is far from the null space (mean EDF 28.0 in the matching core cell).
The bias-aware NCV interval has the lowest rate under smooth errors (1.7–1.8 pp). Its variance is the NCV + CL2 variance plus (NCV − REML)², and under β = 0 the REML estimate is noisy (RMS 0.70–2.04 against 0.12–0.41 for NCV), so the low rate reflects inflated widths, not calibration.
Replicates with excluded points (descriptive; Table 34). Under smooth errors, 0.77–0.92 of the replicates have at least one grid point where the REML + CL2 interval excludes zero (0.72–0.83 with Satterthwaite) and 0.42–0.62 have more than 5% of the grid excluding zero. These shares are not error rates: the intervals are pointwise, and a statement that β is zero everywhere needs a simultaneous band or a global test, which this study does not evaluate.
Size of the null estimate (Table 35; RMS over the grid, mean over replicates). The NCV estimate is 0.15–0.21 times the REML estimate’s RMS under smooth errors and 1.56–1.68 times under independent errors.
Other estimands (Table 36). REML + CL2 covers the mean, α and γ at 0.90–0.95 under smooth errors and 0.85–0.95 under independent errors; for α and γ the difference to the matching non-null core cell is -0.7 to 1.6 pp. Its lowest value is α in the binary cell with independent errors at G = 40 (0.852). NCV + CL2 covers 0.84–0.94 under smooth errors, the bias-aware NCV interval 0.91–0.97.
| family | error | G | REML, model-based | REML + CL2 | REML + CL2, t(G−1) | REML + CL2, Satt | NCV, model-based | NCV + CL2 | NCV + CL2, bias-aware |
|---|---|---|---|---|---|---|---|---|---|
| Gaussian | iid | 40 | 2.4 (0.9) | 9.5 (1.8) | 8.6 (1.7) | 7.5 (1.7) | 6.6 (1.4) | 9.6 (1.7) | 7.9 (1.7) |
| Gaussian | iid | 100 | 0.9 (0.4) | 5.0 (1.3) | 4.9 (1.3) | 4.6 (1.3) | 4.0 (0.9) | 4.7 (1.0) | 3.5 (1.0) |
| Gaussian | smooth | 40 | 39.9 (1.1) | 7.6 (0.5) | 6.8 (0.4) | 5.2 (0.4) | 51.6 (2.6) | 13.0 (1.5) | 1.7 (0.3) |
| Gaussian | smooth | 100 | 36.1 (1.0) | 5.8 (0.4) | 5.5 (0.4) | 4.9 (0.3) | 57.3 (2.6) | 15.1 (1.9) | 1.7 (0.3) |
| binary | iid | 40 | 0.9 (0.4) | 5.6 (1.3) | 4.8 (1.2) | 2.9 (0.9) | 4.0 (0.9) | 6.0 (1.2) | 3.5 (1.0) |
| binary | iid | 100 | 1.5 (0.7) | 4.2 (1.2) | 3.4 (1.1) | 3.1 (1.0) | 3.3 (0.9) | 4.8 (1.1) | 3.7 (1.1) |
| binary | smooth | 40 | 25.0 (1.1) | 7.1 (0.5) | 6.3 (0.5) | 4.8 (0.4) | 46.2 (2.5) | 11.1 (1.4) | 1.8 (0.3) |
| binary | smooth | 100 | 22.8 (1.1) | 5.4 (0.4) | 5.1 (0.4) | 4.6 (0.4) | 47.6 (2.7) | 11.8 (1.6) | 1.8 (0.4) |
| family | error | G | REML, model-based | REML + CL2 | REML + CL2, t(G−1) | REML + CL2, Satt | NCV, model-based | NCV + CL2 | NCV + CL2, bias-aware |
|---|---|---|---|---|---|---|---|---|---|
| Gaussian | iid | 40 | 0.07 / 0.06 | 0.20 / 0.18 | 0.18 / 0.17 | 0.15 / 0.14 | 0.24 / 0.18 | 0.29 / 0.22 | 0.16 / 0.14 |
| Gaussian | iid | 100 | 0.04 / 0.03 | 0.14 / 0.11 | 0.12 / 0.11 | 0.12 / 0.10 | 0.23 / 0.16 | 0.24 / 0.17 | 0.10 / 0.09 |
| Gaussian | smooth | 40 | 1.00 / 0.98 | 0.92 / 0.62 | 0.88 / 0.56 | 0.83 / 0.41 | 0.73 / 0.73 | 0.41 / 0.41 | 0.40 / 0.10 |
| Gaussian | smooth | 100 | 0.99 / 0.97 | 0.86 / 0.48 | 0.86 / 0.45 | 0.81 / 0.41 | 0.80 / 0.80 | 0.41 / 0.40 | 0.38 / 0.12 |
| binary | iid | 40 | 0.04 / 0.04 | 0.14 / 0.14 | 0.14 / 0.12 | 0.11 / 0.09 | 0.23 / 0.17 | 0.27 / 0.20 | 0.12 / 0.10 |
| binary | iid | 100 | 0.04 / 0.04 | 0.13 / 0.10 | 0.12 / 0.09 | 0.10 / 0.08 | 0.22 / 0.14 | 0.27 / 0.16 | 0.12 / 0.09 |
| binary | smooth | 40 | 0.94 / 0.89 | 0.83 / 0.54 | 0.81 / 0.48 | 0.76 / 0.33 | 0.73 / 0.73 | 0.37 / 0.35 | 0.34 / 0.09 |
| binary | smooth | 100 | 0.94 / 0.87 | 0.77 / 0.42 | 0.75 / 0.40 | 0.72 / 0.34 | 0.73 / 0.73 | 0.34 / 0.33 | 0.32 / 0.12 |
| family | error | G | RMS REML | RMS NCV | NCV / REML |
|---|---|---|---|---|---|
| Gaussian | iid | 40 | 0.036 | 0.059 | 1.623 |
| Gaussian | iid | 100 | 0.021 | 0.034 | 1.629 |
| Gaussian | smooth | 40 | 1.398 | 0.205 | 0.146 |
| Gaussian | smooth | 100 | 0.702 | 0.117 | 0.166 |
| binary | iid | 40 | 0.084 | 0.142 | 1.685 |
| binary | iid | 100 | 0.053 | 0.083 | 1.558 |
| binary | smooth | 40 | 2.043 | 0.410 | 0.201 |
| binary | smooth | 100 | 1.005 | 0.208 | 0.207 |
| family | error | G | estimand | REML, model-based | REML + CL2 | NCV + CL2 | NCV + CL2, bias-aware | REML + CL2, core cell |
|---|---|---|---|---|---|---|---|---|
| Gaussian | iid | 40 | E(Y | X) | 0.961 | 0.944 | 0.935 | 0.944 | |
| Gaussian | iid | 40 | alpha(t) | 0.951 | 0.944 | 0.934 | 0.939 | 0.947 |
| Gaussian | iid | 40 | gamma(t) | 0.961 | 0.949 | 0.944 | 0.947 | 0.951 |
| Gaussian | iid | 100 | E(Y | X) | 0.961 | 0.948 | 0.941 | 0.949 | |
| Gaussian | iid | 100 | alpha(t) | 0.954 | 0.952 | 0.939 | 0.945 | 0.956 |
| Gaussian | iid | 100 | gamma(t) | 0.961 | 0.951 | 0.946 | 0.951 | 0.950 |
| Gaussian | smooth | 40 | E(Y | X) | 0.600 | 0.930 | 0.918 | 0.965 | |
| Gaussian | smooth | 40 | alpha(t) | 0.616 | 0.936 | 0.922 | 0.940 | 0.937 |
| Gaussian | smooth | 40 | gamma(t) | 0.621 | 0.945 | 0.928 | 0.947 | 0.946 |
| Gaussian | smooth | 100 | E(Y | X) | 0.624 | 0.943 | 0.926 | 0.967 | |
| Gaussian | smooth | 100 | alpha(t) | 0.641 | 0.950 | 0.939 | 0.950 | 0.950 |
| Gaussian | smooth | 100 | gamma(t) | 0.654 | 0.949 | 0.936 | 0.948 | 0.948 |
| binary | iid | 40 | E(Y | X) | 0.941 | 0.893 | 0.898 | 0.918 | |
| binary | iid | 40 | alpha(t) | 0.900 | 0.852 | 0.870 | 0.884 | 0.859 |
| binary | iid | 40 | gamma(t) | 0.938 | 0.888 | 0.878 | 0.897 | 0.890 |
| binary | iid | 100 | E(Y | X) | 0.953 | 0.925 | 0.926 | 0.936 | |
| binary | iid | 100 | alpha(t) | 0.943 | 0.923 | 0.923 | 0.930 | 0.924 |
| binary | iid | 100 | gamma(t) | 0.944 | 0.918 | 0.916 | 0.920 | 0.923 |
| binary | smooth | 40 | E(Y | X) | 0.705 | 0.923 | 0.872 | 0.953 | |
| binary | smooth | 40 | alpha(t) | 0.680 | 0.901 | 0.869 | 0.925 | 0.885 |
| binary | smooth | 40 | gamma(t) | 0.699 | 0.930 | 0.845 | 0.912 | 0.922 |
| binary | smooth | 100 | E(Y | X) | 0.716 | 0.938 | 0.873 | 0.943 | |
| binary | smooth | 100 | alpha(t) | 0.714 | 0.925 | 0.860 | 0.917 | 0.920 |
| binary | smooth | 100 | gamma(t) | 0.715 | 0.937 | 0.854 | 0.905 | 0.939 |
6 Misspecified truth
Does REML + CL2 cover the true surface when β(s,t) is not in the spline space, and does a larger basis repair it? (misspec/, Gaussian, R² 0.5, D = 61, M1, R = 200.) Three truths, each scored against the exact functions rather than their basis projection: T0 the study’s smooth β (control); T1 = T0 + a 2-D Gaussian bump (sd 0.05, centre (0.3, 0.6)); T2 = T0 + Σ_{j=12..16} √2 cos(jπs) sin(πt), directions the functional covariates barely excite. Each feature carries 20% of the β-term signal, then β is rescaled to the study’s signal; α and γ are the study’s. Cells: truth × {iid G = 100, smooth G = 100, smooth G = 40} × basis {default: ff 8 × 10, t 12; xlarge: ff 23 × 31, t 39}. Arms: REML model-based, REML + CL2 (Bayesian and frequentist), NCV + CL2 (Bayesian); z intervals. The NCV fits in the xlarge cells use replicates 1–100 only; the T0 xlarge G = 100 cells are the study’s cells (replicates 1–4 recomputed). Common random numbers against the study’s records of the T0 cells: REML estimates and SEs agree to about 1e-12, NCV estimates to 2.3e-06 absolute. Bump region: β grid points within 2 sd (elliptical) of the bump centre.
Control (T0). REML + CL2 covers β at 0.93–1.00 and the mean at 0.93–0.98; model-based intervals under smooth errors cover β at 0.60–0.67, as in the main study.
Default basis. Approximation bias lowers the coverage of β and the mean sharply under T2 and inside T1’s bump. T1: REML + CL2 covers β at 0.88–0.91 on average but only 0.17–0.58 inside the bump region, where bias-eliminated coverage is 0.92–0.95 (Table 39): the loss is bias, not variance. T2: REML + CL2 covers β at 0.30–0.56 and the mean at 0.61–0.85; mean |bias|/SD for β is 1.9–7.9. NCV + CL2 is no better (β 0.27–0.33).
xlarge basis. Under smooth errors REML + CL2 recovers the control’s (T0) coverage of the exact truth: β 0.93–0.95, mean 0.94–0.95, T1 bump region 0.93–0.94; the paired gain for T2 β at G = 100 is 49 pp (Table 41). Under iid errors it does not: T1 bump region 0.61, T2 β 0.86 (mean |bias|/SD for β 0.29 and 1.03, against 0.05–0.25 under smooth errors). NCV + CL2 stays biased even with the xlarge basis: T1 bump region 0.20–0.50, T2 β under smooth errors 0.41–0.79 (NCV in the xlarge cells: 100 replicates, MC SE up to 0.016). The xlarge REML coverage is bought with very variable estimates: REML’s β MSE under smooth errors is 11.8–66.9 against 0.12–39.47 for NCV (default basis REML: 0.8–40.5; Table 42).
α and γ are unaffected: REML + CL2 covers them at 0.93–0.98 in every cell.
| truth | amplitude | share ∫β² | rel. L2 err. default | resid. signal default | rel. L2 err. xlarge | resid. signal xlarge |
|---|---|---|---|---|---|---|
| T0 smooth | 0.0 | 0.00 | 0.00 | 0.000 | 0.000 | 0.000 |
| T1 bump | 9.3 | 0.21 | 0.26 | 0.023 | 0.004 | 0.000 |
| T2 weak | 3.8 | 0.93 | 0.89 | 0.220 | 0.097 | 0.001 |
| estimand | truth | setting | basis | REML model-based | REML + CL2 | REML + CL2 (freq.) | NCV + CL2 |
|---|---|---|---|---|---|---|---|
| alpha | T0 smooth | iid, G = 100 | default | 0.956 (0.004) | 0.956 (0.004) | 0.941 (0.005) | 0.945 (0.005) |
| alpha | T0 smooth | iid, G = 100 | xlarge | 0.978 (0.002) | 0.975 (0.002) | 0.940 (0.004) | 0.967 (0.004) |
| alpha | T0 smooth | smooth, G = 100 | default | 0.641 (0.011) | 0.950 (0.005) | 0.948 (0.005) | 0.941 (0.006) |
| alpha | T0 smooth | smooth, G = 100 | xlarge | 0.739 (0.010) | 0.954 (0.004) | 0.942 (0.005) | 0.944 (0.007) |
| alpha | T0 smooth | smooth, G = 40 | default | 0.614 (0.013) | 0.937 (0.007) | 0.933 (0.007) | 0.919 (0.008) |
| alpha | T0 smooth | smooth, G = 40 | xlarge | 0.640 (0.012) | 0.938 (0.006) | 0.926 (0.007) | 0.932 (0.009) |
| alpha | T1 bump | iid, G = 100 | default | 0.953 (0.004) | 0.954 (0.004) | 0.941 (0.005) | 0.947 (0.005) |
| alpha | T1 bump | iid, G = 100 | xlarge | 0.978 (0.002) | 0.974 (0.003) | 0.942 (0.004) | 0.966 (0.004) |
| alpha | T1 bump | smooth, G = 100 | default | 0.645 (0.011) | 0.949 (0.005) | 0.947 (0.005) | 0.937 (0.006) |
| alpha | T1 bump | smooth, G = 100 | xlarge | 0.739 (0.010) | 0.955 (0.004) | 0.942 (0.005) | 0.943 (0.007) |
| alpha | T1 bump | smooth, G = 40 | default | 0.616 (0.013) | 0.935 (0.007) | 0.932 (0.007) | 0.919 (0.008) |
| alpha | T1 bump | smooth, G = 40 | xlarge | 0.641 (0.012) | 0.938 (0.006) | 0.926 (0.007) | 0.927 (0.009) |
| alpha | T2 weak | iid, G = 100 | default | 0.842 (0.012) | 0.945 (0.007) | 0.937 (0.007) | 0.941 (0.007) |
| alpha | T2 weak | iid, G = 100 | xlarge | 0.975 (0.003) | 0.973 (0.003) | 0.939 (0.004) | 0.968 (0.004) |
| alpha | T2 weak | smooth, G = 100 | default | 0.634 (0.013) | 0.946 (0.006) | 0.944 (0.006) | 0.936 (0.006) |
| alpha | T2 weak | smooth, G = 100 | xlarge | 0.732 (0.010) | 0.955 (0.004) | 0.941 (0.005) | 0.941 (0.007) |
| alpha | T2 weak | smooth, G = 40 | default | 0.597 (0.013) | 0.936 (0.007) | 0.932 (0.008) | 0.929 (0.008) |
| alpha | T2 weak | smooth, G = 40 | xlarge | 0.629 (0.013) | 0.936 (0.006) | 0.926 (0.007) | 0.939 (0.009) |
| beta | T0 smooth | iid, G = 100 | default | 0.989 (0.001) | 0.986 (0.001) | 0.940 (0.003) | 0.982 (0.001) |
| beta | T0 smooth | iid, G = 100 | xlarge | 1.000 (0.000) | 1.000 (0.000) | 0.942 (0.002) | 0.999 (0.000) |
| beta | T0 smooth | smooth, G = 100 | default | 0.657 (0.008) | 0.944 (0.003) | 0.935 (0.003) | 0.934 (0.004) |
| beta | T0 smooth | smooth, G = 100 | xlarge | 0.673 (0.006) | 0.954 (0.002) | 0.939 (0.002) | 0.972 (0.003) |
| beta | T0 smooth | smooth, G = 40 | default | 0.615 (0.009) | 0.926 (0.004) | 0.915 (0.004) | 0.924 (0.005) |
| beta | T0 smooth | smooth, G = 40 | xlarge | 0.604 (0.006) | 0.938 (0.003) | 0.923 (0.003) | 0.963 (0.003) |
| beta | T1 bump | iid, G = 100 | default | 0.882 (0.001) | 0.878 (0.001) | 0.823 (0.002) | 0.875 (0.002) |
| beta | T1 bump | iid, G = 100 | xlarge | 0.983 (0.000) | 0.982 (0.000) | 0.907 (0.001) | 0.974 (0.001) |
| beta | T1 bump | smooth, G = 100 | default | 0.620 (0.006) | 0.908 (0.003) | 0.899 (0.003) | 0.866 (0.003) |
| beta | T1 bump | smooth, G = 100 | xlarge | 0.681 (0.005) | 0.955 (0.002) | 0.939 (0.002) | 0.932 (0.002) |
| beta | T1 bump | smooth, G = 40 | default | 0.604 (0.008) | 0.909 (0.003) | 0.898 (0.004) | 0.875 (0.004) |
| beta | T1 bump | smooth, G = 40 | xlarge | 0.607 (0.006) | 0.938 (0.003) | 0.924 (0.003) | 0.930 (0.003) |
| beta | T2 weak | iid, G = 100 | default | 0.265 (0.002) | 0.299 (0.002) | 0.273 (0.002) | 0.275 (0.005) |
| beta | T2 weak | iid, G = 100 | xlarge | 0.856 (0.002) | 0.860 (0.002) | 0.769 (0.003) | 0.887 (0.004) |
| beta | T2 weak | smooth, G = 100 | default | 0.251 (0.002) | 0.441 (0.003) | 0.433 (0.004) | 0.275 (0.004) |
| beta | T2 weak | smooth, G = 100 | xlarge | 0.621 (0.005) | 0.935 (0.002) | 0.920 (0.002) | 0.791 (0.016) |
| beta | T2 weak | smooth, G = 40 | default | 0.293 (0.003) | 0.559 (0.004) | 0.548 (0.004) | 0.333 (0.004) |
| beta | T2 weak | smooth, G = 40 | xlarge | 0.579 (0.005) | 0.932 (0.003) | 0.918 (0.003) | 0.409 (0.014) |
| gamma | T0 smooth | iid, G = 100 | default | 0.960 (0.004) | 0.950 (0.005) | 0.937 (0.005) | 0.946 (0.005) |
| gamma | T0 smooth | iid, G = 100 | xlarge | 0.976 (0.002) | 0.973 (0.003) | 0.936 (0.004) | 0.966 (0.004) |
| gamma | T0 smooth | smooth, G = 100 | default | 0.655 (0.012) | 0.948 (0.005) | 0.945 (0.006) | 0.935 (0.006) |
| gamma | T0 smooth | smooth, G = 100 | xlarge | 0.734 (0.011) | 0.950 (0.005) | 0.939 (0.005) | 0.935 (0.007) |
| gamma | T0 smooth | smooth, G = 40 | default | 0.621 (0.012) | 0.946 (0.006) | 0.941 (0.006) | 0.927 (0.007) |
| gamma | T0 smooth | smooth, G = 40 | xlarge | 0.643 (0.013) | 0.932 (0.007) | 0.921 (0.007) | 0.929 (0.010) |
| gamma | T1 bump | iid, G = 100 | default | 0.956 (0.004) | 0.949 (0.004) | 0.937 (0.005) | 0.944 (0.005) |
| gamma | T1 bump | iid, G = 100 | xlarge | 0.976 (0.002) | 0.975 (0.002) | 0.936 (0.004) | 0.968 (0.004) |
| gamma | T1 bump | smooth, G = 100 | default | 0.657 (0.012) | 0.950 (0.005) | 0.947 (0.005) | 0.936 (0.006) |
| gamma | T1 bump | smooth, G = 100 | xlarge | 0.736 (0.011) | 0.950 (0.005) | 0.937 (0.005) | 0.939 (0.007) |
| gamma | T1 bump | smooth, G = 40 | default | 0.630 (0.013) | 0.947 (0.006) | 0.942 (0.006) | 0.927 (0.007) |
| gamma | T1 bump | smooth, G = 40 | xlarge | 0.640 (0.013) | 0.932 (0.007) | 0.922 (0.007) | 0.924 (0.010) |
| gamma | T2 weak | iid, G = 100 | default | 0.839 (0.012) | 0.946 (0.008) | 0.935 (0.009) | 0.930 (0.009) |
| gamma | T2 weak | iid, G = 100 | xlarge | 0.975 (0.003) | 0.974 (0.003) | 0.935 (0.004) | 0.966 (0.004) |
| gamma | T2 weak | smooth, G = 100 | default | 0.622 (0.013) | 0.944 (0.007) | 0.940 (0.007) | 0.933 (0.007) |
| gamma | T2 weak | smooth, G = 100 | xlarge | 0.729 (0.011) | 0.950 (0.005) | 0.937 (0.005) | 0.931 (0.008) |
| gamma | T2 weak | smooth, G = 40 | default | 0.604 (0.013) | 0.945 (0.007) | 0.940 (0.007) | 0.936 (0.007) |
| gamma | T2 weak | smooth, G = 40 | xlarge | 0.635 (0.013) | 0.934 (0.007) | 0.922 (0.007) | 0.931 (0.010) |
| mean | T0 smooth | iid, G = 100 | default | 0.967 (0.001) | 0.961 (0.002) | 0.939 (0.002) | 0.950 (0.002) |
| mean | T0 smooth | iid, G = 100 | xlarge | 0.985 (0.001) | 0.982 (0.001) | 0.938 (0.001) | 0.974 (0.001) |
| mean | T0 smooth | smooth, G = 100 | default | 0.634 (0.005) | 0.944 (0.002) | 0.940 (0.002) | 0.929 (0.003) |
| mean | T0 smooth | smooth, G = 100 | xlarge | 0.706 (0.003) | 0.951 (0.002) | 0.939 (0.002) | 0.943 (0.003) |
| mean | T0 smooth | smooth, G = 40 | default | 0.606 (0.005) | 0.931 (0.003) | 0.926 (0.003) | 0.922 (0.004) |
| mean | T0 smooth | smooth, G = 40 | xlarge | 0.633 (0.004) | 0.938 (0.002) | 0.926 (0.002) | 0.935 (0.004) |
| mean | T1 bump | iid, G = 100 | default | 0.815 (0.001) | 0.810 (0.002) | 0.781 (0.002) | 0.807 (0.002) |
| mean | T1 bump | iid, G = 100 | xlarge | 0.966 (0.001) | 0.964 (0.001) | 0.914 (0.001) | 0.953 (0.001) |
| mean | T1 bump | smooth, G = 100 | default | 0.582 (0.004) | 0.898 (0.002) | 0.894 (0.002) | 0.868 (0.002) |
| mean | T1 bump | smooth, G = 100 | xlarge | 0.708 (0.003) | 0.950 (0.001) | 0.938 (0.002) | 0.922 (0.003) |
| mean | T1 bump | smooth, G = 40 | default | 0.584 (0.005) | 0.912 (0.003) | 0.906 (0.003) | 0.886 (0.004) |
| mean | T1 bump | smooth, G = 40 | xlarge | 0.633 (0.004) | 0.938 (0.002) | 0.926 (0.002) | 0.916 (0.004) |
| mean | T2 weak | iid, G = 100 | default | 0.506 (0.002) | 0.614 (0.003) | 0.591 (0.003) | 0.597 (0.004) |
| mean | T2 weak | iid, G = 100 | xlarge | 0.948 (0.001) | 0.949 (0.001) | 0.896 (0.002) | 0.957 (0.002) |
| mean | T2 weak | smooth, G = 100 | default | 0.439 (0.003) | 0.771 (0.003) | 0.765 (0.003) | 0.686 (0.003) |
| mean | T2 weak | smooth, G = 100 | xlarge | 0.697 (0.003) | 0.948 (0.002) | 0.937 (0.002) | 0.895 (0.007) |
| mean | T2 weak | smooth, G = 40 | default | 0.478 (0.004) | 0.851 (0.003) | 0.844 (0.004) | 0.775 (0.004) |
| mean | T2 weak | smooth, G = 40 | xlarge | 0.622 (0.004) | 0.937 (0.002) | 0.926 (0.002) | 0.801 (0.005) |
| setting | basis | arm | bump_region | elsewhere | bump_region_bias_elim |
|---|---|---|---|---|---|
| iid, G = 100 | default | REML model-based | 0.172 | 0.901 | 0.955 |
| iid, G = 100 | default | REML + CL2 | 0.171 | 0.897 | 0.950 |
| iid, G = 100 | default | REML + CL2 (freq.) | 0.145 | 0.841 | 0.901 |
| iid, G = 100 | default | NCV + CL2 | 0.200 | 0.893 | 0.830 |
| iid, G = 100 | xlarge | REML model-based | 0.619 | 0.993 | 1.000 |
| iid, G = 100 | xlarge | REML + CL2 | 0.608 | 0.992 | 1.000 |
| iid, G = 100 | xlarge | REML + CL2 (freq.) | 0.419 | 0.920 | 0.923 |
| iid, G = 100 | xlarge | NCV + CL2 | 0.502 | 0.986 | 0.989 |
| smooth, G = 100 | default | REML model-based | 0.223 | 0.630 | 0.557 |
| smooth, G = 100 | default | REML + CL2 | 0.468 | 0.920 | 0.924 |
| smooth, G = 100 | default | REML + CL2 (freq.) | 0.457 | 0.910 | 0.916 |
| smooth, G = 100 | default | NCV + CL2 | 0.173 | 0.885 | 0.730 |
| smooth, G = 100 | xlarge | REML model-based | 0.641 | 0.682 | 0.650 |
| smooth, G = 100 | xlarge | REML + CL2 | 0.942 | 0.955 | 0.949 |
| smooth, G = 100 | xlarge | REML + CL2 (freq.) | 0.926 | 0.939 | 0.934 |
| smooth, G = 100 | xlarge | NCV + CL2 | 0.224 | 0.951 | 0.890 |
| smooth, G = 40 | default | REML model-based | 0.280 | 0.612 | 0.526 |
| smooth, G = 40 | default | REML + CL2 | 0.583 | 0.918 | 0.921 |
| smooth, G = 40 | default | REML + CL2 (freq.) | 0.574 | 0.907 | 0.911 |
| smooth, G = 40 | default | NCV + CL2 | 0.160 | 0.894 | 0.876 |
| smooth, G = 40 | xlarge | REML model-based | 0.554 | 0.608 | 0.564 |
| smooth, G = 40 | xlarge | REML + CL2 | 0.928 | 0.939 | 0.930 |
| smooth, G = 40 | xlarge | REML + CL2 (freq.) | 0.916 | 0.924 | 0.920 |
| smooth, G = 40 | xlarge | NCV + CL2 | 0.203 | 0.949 | 0.956 |
| estimand | truth | setting | basis | arm | coverage | coverage_bias_elim | pointwise_q05 | mean_absbias_sd | se_sd_ratio |
|---|---|---|---|---|---|---|---|---|---|
| beta | T1 bump | iid, G = 100 | default | REML + CL2 | 0.878 | 0.981 | 0.013 | 0.869 | 1.246 |
| beta | T1 bump | iid, G = 100 | default | NCV + CL2 | 0.875 | 0.961 | 0.117 | 0.582 | 1.096 |
| beta | T1 bump | iid, G = 100 | xlarge | REML + CL2 | 0.982 | 1.000 | 0.986 | 0.290 | 1.848 |
| beta | T1 bump | iid, G = 100 | xlarge | NCV + CL2 | 0.974 | 0.999 | 0.912 | 0.327 | 1.844 |
| beta | T1 bump | smooth, G = 100 | default | REML + CL2 | 0.908 | 0.940 | 0.686 | 0.267 | 0.958 |
| beta | T1 bump | smooth, G = 100 | default | NCV + CL2 | 0.866 | 0.930 | 0.275 | 0.415 | 0.927 |
| beta | T1 bump | smooth, G = 100 | xlarge | REML + CL2 | 0.955 | 0.956 | 0.930 | 0.064 | 1.029 |
| beta | T1 bump | smooth, G = 100 | xlarge | NCV + CL2 | 0.932 | 0.978 | 0.560 | 0.355 | 1.184 |
| beta | T1 bump | smooth, G = 40 | default | REML + CL2 | 0.909 | 0.925 | 0.836 | 0.182 | 0.927 |
| beta | T1 bump | smooth, G = 40 | default | NCV + CL2 | 0.875 | 0.934 | 0.501 | 0.412 | 0.921 |
| beta | T1 bump | smooth, G = 40 | xlarge | REML + CL2 | 0.938 | 0.939 | 0.905 | 0.048 | 1.002 |
| beta | T1 bump | smooth, G = 40 | xlarge | NCV + CL2 | 0.930 | 0.975 | 0.700 | 0.369 | 1.094 |
| beta | T2 weak | iid, G = 100 | default | REML + CL2 | 0.299 | 0.961 | 0.000 | 7.931 | 1.003 |
| beta | T2 weak | iid, G = 100 | default | NCV + CL2 | 0.275 | 0.880 | 0.000 | 6.367 | 0.896 |
| beta | T2 weak | iid, G = 100 | xlarge | REML + CL2 | 0.860 | 0.988 | 0.101 | 1.025 | 1.301 |
| beta | T2 weak | iid, G = 100 | xlarge | NCV + CL2 | 0.887 | 0.975 | 0.322 | 0.659 | 1.186 |
| beta | T2 weak | smooth, G = 100 | default | REML + CL2 | 0.441 | 0.922 | 0.000 | 3.279 | 0.948 |
| beta | T2 weak | smooth, G = 100 | default | NCV + CL2 | 0.275 | 0.928 | 0.000 | 6.619 | 0.849 |
| beta | T2 weak | smooth, G = 100 | xlarge | REML + CL2 | 0.935 | 0.953 | 0.850 | 0.247 | 1.035 |
| beta | T2 weak | smooth, G = 100 | xlarge | NCV + CL2 | 0.791 | 0.871 | 0.300 | 0.557 | 0.904 |
| beta | T2 weak | smooth, G = 40 | default | REML + CL2 | 0.559 | 0.911 | 0.000 | 1.901 | 0.916 |
| beta | T2 weak | smooth, G = 40 | default | NCV + CL2 | 0.333 | 0.918 | 0.000 | 4.522 | 0.802 |
| beta | T2 weak | smooth, G = 40 | xlarge | REML + CL2 | 0.932 | 0.937 | 0.890 | 0.131 | 1.013 |
| beta | T2 weak | smooth, G = 40 | xlarge | NCV + CL2 | 0.409 | 0.885 | 0.040 | 1.655 | 0.787 |
| mean | T1 bump | iid, G = 100 | default | REML + CL2 | 0.810 | 0.961 | 0.057 | 0.940 | 1.087 |
| mean | T1 bump | iid, G = 100 | default | NCV + CL2 | 0.807 | 0.954 | 0.080 | 0.881 | 1.045 |
| mean | T1 bump | iid, G = 100 | xlarge | REML + CL2 | 0.964 | 0.984 | 0.917 | 0.266 | 1.252 |
| mean | T1 bump | iid, G = 100 | xlarge | NCV + CL2 | 0.953 | 0.985 | 0.840 | 0.398 | 1.275 |
| mean | T1 bump | smooth, G = 100 | default | REML + CL2 | 0.898 | 0.944 | 0.690 | 0.374 | 1.001 |
| mean | T1 bump | smooth, G = 100 | default | NCV + CL2 | 0.868 | 0.940 | 0.482 | 0.507 | 0.980 |
| mean | T1 bump | smooth, G = 100 | xlarge | REML + CL2 | 0.950 | 0.952 | 0.925 | 0.086 | 1.036 |
| mean | T1 bump | smooth, G = 100 | xlarge | NCV + CL2 | 0.922 | 0.959 | 0.780 | 0.360 | 1.070 |
| mean | T1 bump | smooth, G = 40 | default | REML + CL2 | 0.912 | 0.932 | 0.820 | 0.240 | 0.995 |
| mean | T1 bump | smooth, G = 40 | default | NCV + CL2 | 0.886 | 0.933 | 0.642 | 0.408 | 0.992 |
| mean | T1 bump | smooth, G = 40 | xlarge | REML + CL2 | 0.938 | 0.938 | 0.910 | 0.065 | 1.023 |
| mean | T1 bump | smooth, G = 40 | xlarge | NCV + CL2 | 0.916 | 0.951 | 0.790 | 0.366 | 1.063 |
| mean | T2 weak | iid, G = 100 | default | REML + CL2 | 0.614 | 0.952 | 0.000 | 1.773 | 1.031 |
| mean | T2 weak | iid, G = 100 | default | NCV + CL2 | 0.597 | 0.943 | 0.005 | 1.727 | 0.985 |
| mean | T2 weak | iid, G = 100 | xlarge | REML + CL2 | 0.949 | 0.978 | 0.840 | 0.454 | 1.195 |
| mean | T2 weak | iid, G = 100 | xlarge | NCV + CL2 | 0.957 | 0.975 | 0.880 | 0.347 | 1.169 |
| mean | T2 weak | smooth, G = 100 | default | REML + CL2 | 0.771 | 0.943 | 0.165 | 0.935 | 0.998 |
| mean | T2 weak | smooth, G = 100 | default | NCV + CL2 | 0.686 | 0.939 | 0.030 | 1.294 | 0.975 |
| mean | T2 weak | smooth, G = 100 | xlarge | REML + CL2 | 0.948 | 0.951 | 0.920 | 0.130 | 1.036 |
| mean | T2 weak | smooth, G = 100 | xlarge | NCV + CL2 | 0.895 | 0.936 | 0.725 | 0.432 | 1.006 |
| mean | T2 weak | smooth, G = 40 | default | REML + CL2 | 0.851 | 0.928 | 0.535 | 0.553 | 0.981 |
| mean | T2 weak | smooth, G = 40 | default | NCV + CL2 | 0.775 | 0.929 | 0.207 | 0.862 | 0.975 |
| mean | T2 weak | smooth, G = 40 | xlarge | REML + CL2 | 0.937 | 0.938 | 0.910 | 0.090 | 1.028 |
| mean | T2 weak | smooth, G = 40 | xlarge | NCV + CL2 | 0.801 | 0.939 | 0.274 | 0.749 | 0.981 |
| estimand | truth | setting | arm | n_pairs | diff | diff_se |
|---|---|---|---|---|---|---|
| beta | T0 smooth | iid, G = 100 | NCV + CL2 | 100 | 0.016 | 0.002 |
| beta | T0 smooth | iid, G = 100 | REML + CL2 | 200 | 0.014 | 0.001 |
| beta | T0 smooth | smooth, G = 100 | NCV + CL2 | 100 | 0.032 | 0.003 |
| beta | T0 smooth | smooth, G = 100 | REML + CL2 | 200 | 0.011 | 0.004 |
| beta | T0 smooth | smooth, G = 40 | NCV + CL2 | 100 | 0.039 | 0.004 |
| beta | T0 smooth | smooth, G = 40 | REML + CL2 | 200 | 0.012 | 0.005 |
| beta | T1 bump | iid, G = 100 | NCV + CL2 | 100 | 0.099 | 0.002 |
| beta | T1 bump | iid, G = 100 | REML + CL2 | 200 | 0.105 | 0.001 |
| beta | T1 bump | smooth, G = 100 | NCV + CL2 | 100 | 0.060 | 0.003 |
| beta | T1 bump | smooth, G = 100 | REML + CL2 | 200 | 0.047 | 0.003 |
| beta | T1 bump | smooth, G = 40 | NCV + CL2 | 100 | 0.053 | 0.003 |
| beta | T1 bump | smooth, G = 40 | REML + CL2 | 200 | 0.029 | 0.004 |
| beta | T2 weak | iid, G = 100 | NCV + CL2 | 100 | 0.610 | 0.008 |
| beta | T2 weak | iid, G = 100 | REML + CL2 | 200 | 0.561 | 0.003 |
| beta | T2 weak | smooth, G = 100 | NCV + CL2 | 100 | 0.515 | 0.017 |
| beta | T2 weak | smooth, G = 100 | REML + CL2 | 200 | 0.493 | 0.004 |
| beta | T2 weak | smooth, G = 40 | NCV + CL2 | 100 | 0.080 | 0.012 |
| beta | T2 weak | smooth, G = 40 | REML + CL2 | 200 | 0.373 | 0.005 |
| mean | T0 smooth | iid, G = 100 | NCV + CL2 | 100 | 0.018 | 0.002 |
| mean | T0 smooth | iid, G = 100 | REML + CL2 | 200 | 0.022 | 0.001 |
| mean | T0 smooth | smooth, G = 100 | NCV + CL2 | 100 | 0.007 | 0.001 |
| mean | T0 smooth | smooth, G = 100 | REML + CL2 | 200 | 0.007 | 0.002 |
| mean | T0 smooth | smooth, G = 40 | NCV + CL2 | 100 | 0.007 | 0.002 |
| mean | T0 smooth | smooth, G = 40 | REML + CL2 | 200 | 0.006 | 0.003 |
| mean | T1 bump | iid, G = 100 | NCV + CL2 | 100 | 0.143 | 0.002 |
| mean | T1 bump | iid, G = 100 | REML + CL2 | 200 | 0.154 | 0.002 |
| mean | T1 bump | smooth, G = 100 | NCV + CL2 | 100 | 0.049 | 0.001 |
| mean | T1 bump | smooth, G = 100 | REML + CL2 | 200 | 0.052 | 0.002 |
| mean | T1 bump | smooth, G = 40 | NCV + CL2 | 100 | 0.027 | 0.001 |
| mean | T1 bump | smooth, G = 40 | REML + CL2 | 200 | 0.025 | 0.003 |
| mean | T2 weak | iid, G = 100 | NCV + CL2 | 100 | 0.362 | 0.006 |
| mean | T2 weak | iid, G = 100 | REML + CL2 | 200 | 0.336 | 0.003 |
| mean | T2 weak | smooth, G = 100 | NCV + CL2 | 100 | 0.205 | 0.007 |
| mean | T2 weak | smooth, G = 100 | REML + CL2 | 200 | 0.177 | 0.003 |
| mean | T2 weak | smooth, G = 40 | NCV + CL2 | 100 | 0.025 | 0.002 |
| mean | T2 weak | smooth, G = 40 | REML + CL2 | 200 | 0.086 | 0.004 |
| estimand | truth | setting | basis | mse_REML | mse_NCV | bias_share_REML | bias_share_NCV |
|---|---|---|---|---|---|---|---|
| beta | T0 smooth | iid, G = 100 | default | 0.041 | 0.027 | 0.044 | 0.213 |
| beta | T0 smooth | iid, G = 100 | xlarge | 0.090 | 0.041 | 0.043 | 0.260 |
| beta | T0 smooth | smooth, G = 100 | default | 0.752 | 0.118 | 0.007 | 0.128 |
| beta | T0 smooth | smooth, G = 100 | xlarge | 11.772 | 0.118 | 0.003 | 0.172 |
| beta | T0 smooth | smooth, G = 40 | default | 2.750 | 0.263 | 0.009 | 0.111 |
| beta | T0 smooth | smooth, G = 40 | xlarge | 52.770 | 0.255 | -0.001 | 0.169 |
| beta | T1 bump | iid, G = 100 | default | 0.331 | 0.364 | 0.836 | 0.694 |
| beta | T1 bump | iid, G = 100 | xlarge | 0.234 | 0.237 | 0.485 | 0.581 |
| beta | T1 bump | smooth, G = 100 | default | 1.111 | 0.512 | 0.215 | 0.627 |
| beta | T1 bump | smooth, G = 100 | xlarge | 12.527 | 0.423 | 0.002 | 0.603 |
| beta | T1 bump | smooth, G = 40 | default | 3.136 | 0.656 | 0.082 | 0.580 |
| beta | T1 bump | smooth, G = 40 | xlarge | 54.334 | 0.655 | -0.001 | 0.513 |
| beta | T2 weak | iid, G = 100 | default | 39.416 | 38.276 | 0.994 | 0.987 |
| beta | T2 weak | iid, G = 100 | xlarge | 7.724 | 9.856 | 0.807 | 0.526 |
| beta | T2 weak | smooth, G = 100 | default | 38.119 | 39.378 | 0.959 | 0.990 |
| beta | T2 weak | smooth, G = 100 | xlarge | 20.566 | 18.944 | 0.166 | 0.470 |
| beta | T2 weak | smooth, G = 40 | default | 40.543 | 39.877 | 0.882 | 0.984 |
| beta | T2 weak | smooth, G = 40 | xlarge | 66.856 | 39.467 | 0.045 | 0.876 |
| mean | T0 smooth | iid, G = 100 | default | 0.008 | 0.008 | 0.043 | 0.134 |
| mean | T0 smooth | iid, G = 100 | xlarge | 0.014 | 0.011 | 0.046 | 0.212 |
| mean | T0 smooth | smooth, G = 100 | default | 0.049 | 0.034 | 0.005 | 0.102 |
| mean | T0 smooth | smooth, G = 100 | xlarge | 0.121 | 0.037 | 0.003 | 0.146 |
| mean | T0 smooth | smooth, G = 40 | default | 0.142 | 0.083 | 0.005 | 0.096 |
| mean | T0 smooth | smooth, G = 40 | xlarge | 0.471 | 0.090 | 0.001 | 0.146 |
| mean | T1 bump | iid, G = 100 | default | 0.031 | 0.031 | 0.703 | 0.673 |
| mean | T1 bump | iid, G = 100 | xlarge | 0.018 | 0.018 | 0.196 | 0.317 |
| mean | T1 bump | smooth, G = 100 | default | 0.072 | 0.063 | 0.275 | 0.433 |
| mean | T1 bump | smooth, G = 100 | xlarge | 0.127 | 0.050 | 0.009 | 0.268 |
| mean | T1 bump | smooth, G = 40 | default | 0.167 | 0.115 | 0.126 | 0.304 |
| mean | T1 bump | smooth, G = 40 | xlarge | 0.482 | 0.107 | 0.002 | 0.224 |
| mean | T2 weak | iid, G = 100 | default | 0.141 | 0.135 | 0.854 | 0.846 |
| mean | T2 weak | iid, G = 100 | xlarge | 0.035 | 0.034 | 0.289 | 0.185 |
| mean | T2 weak | smooth, G = 100 | default | 0.174 | 0.166 | 0.622 | 0.748 |
| mean | T2 weak | smooth, G = 100 | xlarge | 0.147 | 0.112 | 0.025 | 0.266 |
| mean | T2 weak | smooth, G = 40 | default | 0.300 | 0.235 | 0.359 | 0.563 |
| mean | T2 weak | smooth, G = 40 | xlarge | 0.529 | 0.246 | 0.009 | 0.509 |
7 Same-package comparators
refund’s own remedies for dependent residuals against REML + CL2 (comparators/, Gaussian M1; R = 200 with common random numbers, REML estimates and SEs equal to the study’s records). Cells: the six Gaussian core cells (iid / OU / smooth errors × G = 40, 100, default signal) and twelve plasmode cells (the six datasets with independent curves, log-midpoint truth, REML residuals, curve flips, all subjects and G = 40). Arms:
- REML model-based (V_c) and REML + CL2 (the study’s arms);
- pcre: REML fit with a
pcre()curve effect on the eigenfunctions of the REML residual curves (fpca.sc, pve 0.95), model-based V_c; - GLS raw / GLS FPCA: the pre-deprecation
pffr_gls()algorithm (refund’spffrGLS()only errors in the study build) with the residual covariance estimated by the raw covariance of the REML residual curves (refund’s documented use) or by the FPCA covariance plus white noise; model-based V_c; - curve bootstrap of the REML fit (resampling and refit as
pffr_coefboot(), B = 199, replicates 1–100): percentile intervals and bootstrap-SE Wald intervals around the REML estimate.
Point estimates: pcre and both GLS arms have their own; the bootstrap arms use the REML estimate. NCV’s estimates for the same cells and replicates come from the study (summaries/final/cells.csv, summaries/plasmode/coverage.csv). Relative error as in Section 3.3.4, at the cell level: √(MSE / grid mean of the squared truth), the truth centred for the mean and α(t).
Coverage. Under dependent synthetic errors pcre and GLS with the FPCA covariance cover β at 0.93–0.97 and 0.93–0.97, REML + CL2 at 0.93–0.94. On the plasmode cells, with real residual curves, they fall to 0.59–0.86 and 0.60–0.86 while REML + CL2 stays at 0.91–0.94 (Figure 14, Table 43). GLS with the raw residual covariance, refund’s documented use, undercovers everywhere (β 0.59–0.89 even under independent errors). pcre’s intervals for α(t) cover at 0.51–0.59 under dependent synthetic errors. The curve bootstrap covers β at 0.93–0.97 (percentile, plasmode) with intervals 1.42 times as wide as REML + CL2’s, at a median of 507 s per replicate against 2.4 s for the REML fit.
Point estimates. Relative to the NCV fit, the β MSE of pcre and of GLS (FPCA covariance) is 1.41–2.50 and 1.41–2.53 times NCV’s under dependent synthetic errors, REML’s 5.76–10.45 times (Figure 15, Table 44): both remedies remove most of REML’s excess error but stay above NCV’s. On the plasmode cells the ratios are 0.23–1.25 (pcre) and 0.21–1.20 (GLS FPCA), against 1.08–8.54 for REML; pcre’s MSE is below NCV’s in 8 of 12 cells. For the mean the ratios of pcre to NCV are 1.13–1.26 (synthetic dependent) and 0.86–1.30 (plasmode); for γ(t) 1.01–1.09 and 0.66–2.21; for α(t) 1.04–1.06 and 0.71–1.37. Under independent errors pcre and GLS (FPCA) estimate β like the REML fit (MSE 0.97–0.98 and 0.99 times REML’s), and NCV 0.65–0.85 times. Relative errors of β per plasmode cell are in Table 45.
| setting | estimand | arm | cells | coverage | coverage_min | q05 | width_rel | score_rel |
|---|---|---|---|---|---|---|---|---|
| plasmode | E(Y | X) | REML model-based | 12 | 0.62 | 0.50 | 0.32 | 0.42 | 2.70 |
| plasmode | E(Y | X) | REML + CL2 | 12 | 0.94 | 0.92 | 0.88 | 1.00 | 1.00 |
| plasmode | E(Y | X) | pcre | 12 | 0.78 | 0.73 | 0.54 | 0.63 | 1.46 |
| plasmode | E(Y | X) | GLS (FPCA cov.) | 12 | 0.80 | 0.66 | 0.58 | 0.63 | 1.35 |
| plasmode | E(Y | X) | GLS (raw cov.) | 12 | 0.58 | 0.46 | 0.43 | 0.38 | 1.99 |
| plasmode | E(Y | X) | bootstrap percentile | 12 | 0.95 | 0.94 | 0.90 | 1.14 | 1.05 |
| plasmode | E(Y | X) | bootstrap Wald | 12 | 0.96 | 0.94 | 0.91 | 1.12 | 1.00 |
| plasmode | beta(s,t) | REML model-based | 12 | 0.58 | 0.47 | 0.32 | 0.39 | 2.90 |
| plasmode | beta(s,t) | REML + CL2 | 12 | 0.93 | 0.91 | 0.88 | 1.00 | 1.00 |
| plasmode | beta(s,t) | pcre | 12 | 0.74 | 0.59 | 0.42 | 0.40 | 1.05 |
| plasmode | beta(s,t) | GLS (FPCA cov.) | 12 | 0.73 | 0.60 | 0.45 | 0.38 | 1.03 |
| plasmode | beta(s,t) | GLS (raw cov.) | 12 | 0.56 | 0.45 | 0.38 | 0.30 | 1.62 |
| plasmode | beta(s,t) | bootstrap percentile | 12 | 0.95 | 0.93 | 0.91 | 1.42 | 1.22 |
| plasmode | beta(s,t) | bootstrap Wald | 12 | 0.97 | 0.94 | 0.93 | 1.39 | 1.15 |
| plasmode | alpha(t) | REML model-based | 12 | 0.58 | 0.42 | 0.33 | 0.39 | 2.83 |
| plasmode | alpha(t) | REML + CL2 | 12 | 0.93 | 0.89 | 0.89 | 1.00 | 1.00 |
| plasmode | alpha(t) | pcre | 12 | 0.76 | 0.59 | 0.63 | 0.62 | 1.52 |
| plasmode | alpha(t) | GLS (FPCA cov.) | 12 | 0.75 | 0.57 | 0.63 | 0.58 | 1.50 |
| plasmode | alpha(t) | GLS (raw cov.) | 12 | 0.57 | 0.40 | 0.50 | 0.35 | 1.95 |
| plasmode | alpha(t) | bootstrap percentile | 12 | 0.95 | 0.93 | 0.92 | 1.07 | 0.97 |
| plasmode | alpha(t) | bootstrap Wald | 12 | 0.95 | 0.92 | 0.91 | 1.06 | 0.96 |
| plasmode | gamma(t) | REML model-based | 12 | 0.62 | 0.50 | 0.37 | 0.43 | 2.67 |
| plasmode | gamma(t) | REML + CL2 | 12 | 0.94 | 0.92 | 0.90 | 1.00 | 1.00 |
| plasmode | gamma(t) | pcre | 12 | 0.80 | 0.44 | 0.64 | 0.71 | 1.33 |
| plasmode | gamma(t) | GLS (FPCA cov.) | 12 | 0.79 | 0.50 | 0.64 | 0.65 | 1.33 |
| plasmode | gamma(t) | GLS (raw cov.) | 12 | 0.60 | 0.43 | 0.48 | 0.40 | 1.89 |
| plasmode | gamma(t) | bootstrap percentile | 12 | 0.95 | 0.94 | 0.92 | 1.13 | 1.05 |
| plasmode | gamma(t) | bootstrap Wald | 12 | 0.96 | 0.93 | 0.91 | 1.11 | 1.02 |
| syn. dep. | E(Y | X) | REML model-based | 4 | 0.63 | 0.61 | 0.54 | 0.47 | 2.05 |
| syn. dep. | E(Y | X) | REML + CL2 | 4 | 0.94 | 0.93 | 0.91 | 1.00 | 1.00 |
| syn. dep. | E(Y | X) | pcre | 4 | 0.90 | 0.88 | 0.84 | 0.76 | 0.91 |
| syn. dep. | E(Y | X) | GLS (FPCA cov.) | 4 | 0.92 | 0.90 | 0.88 | 0.81 | 0.88 |
| syn. dep. | E(Y | X) | GLS (raw cov.) | 4 | 0.61 | 0.38 | 0.55 | 0.43 | 2.03 |
| syn. dep. | E(Y | X) | bootstrap percentile | 4 | 0.95 | 0.94 | 0.91 | 1.17 | 1.10 |
| syn. dep. | E(Y | X) | bootstrap Wald | 4 | 0.97 | 0.96 | 0.93 | 1.16 | 1.02 |
| syn. dep. | beta(s,t) | REML model-based | 4 | 0.65 | 0.61 | 0.56 | 0.49 | 2.06 |
| syn. dep. | beta(s,t) | REML + CL2 | 4 | 0.93 | 0.93 | 0.91 | 1.00 | 1.00 |
| syn. dep. | beta(s,t) | pcre | 4 | 0.96 | 0.93 | 0.92 | 0.59 | 0.55 |
| syn. dep. | beta(s,t) | GLS (FPCA cov.) | 4 | 0.95 | 0.93 | 0.91 | 0.58 | 0.55 |
| syn. dep. | beta(s,t) | GLS (raw cov.) | 4 | 0.69 | 0.46 | 0.63 | 0.40 | 1.33 |
| syn. dep. | beta(s,t) | bootstrap percentile | 4 | 0.95 | 0.94 | 0.91 | 1.60 | 1.41 |
| syn. dep. | beta(s,t) | bootstrap Wald | 4 | 0.98 | 0.97 | 0.94 | 1.57 | 1.28 |
| syn. dep. | alpha(t) | REML model-based | 4 | 0.64 | 0.61 | 0.56 | 0.47 | 2.04 |
| syn. dep. | alpha(t) | REML + CL2 | 4 | 0.94 | 0.94 | 0.92 | 1.00 | 1.00 |
| syn. dep. | alpha(t) | pcre | 4 | 0.56 | 0.51 | 0.50 | 0.40 | 2.41 |
| syn. dep. | alpha(t) | GLS (FPCA cov.) | 4 | 0.92 | 0.90 | 0.87 | 0.90 | 1.02 |
| syn. dep. | alpha(t) | GLS (raw cov.) | 4 | 0.54 | 0.28 | 0.50 | 0.46 | 3.07 |
| syn. dep. | alpha(t) | bootstrap percentile | 4 | 0.95 | 0.94 | 0.92 | 1.05 | 1.02 |
| syn. dep. | alpha(t) | bootstrap Wald | 4 | 0.95 | 0.95 | 0.92 | 1.04 | 1.00 |
| syn. dep. | gamma(t) | REML model-based | 4 | 0.65 | 0.62 | 0.58 | 0.47 | 1.96 |
| syn. dep. | gamma(t) | REML + CL2 | 4 | 0.94 | 0.94 | 0.92 | 1.00 | 1.00 |
| syn. dep. | gamma(t) | pcre | 4 | 0.93 | 0.92 | 0.90 | 0.92 | 0.97 |
| syn. dep. | gamma(t) | GLS (FPCA cov.) | 4 | 0.92 | 0.90 | 0.89 | 0.90 | 0.98 |
| syn. dep. | gamma(t) | GLS (raw cov.) | 4 | 0.62 | 0.42 | 0.58 | 0.45 | 2.00 |
| syn. dep. | gamma(t) | bootstrap percentile | 4 | 0.96 | 0.94 | 0.92 | 1.07 | 1.01 |
| syn. dep. | gamma(t) | bootstrap Wald | 4 | 0.96 | 0.95 | 0.93 | 1.05 | 0.97 |
| syn. iid | E(Y | X) | REML model-based | 2 | 0.97 | 0.97 | 0.94 | 1.01 | 0.98 |
| syn. iid | E(Y | X) | REML + CL2 | 2 | 0.96 | 0.96 | 0.93 | 1.00 | 1.00 |
| syn. iid | E(Y | X) | pcre | 2 | 0.97 | 0.97 | 0.94 | 1.03 | 0.99 |
| syn. iid | E(Y | X) | GLS (FPCA cov.) | 2 | 0.97 | 0.97 | 0.94 | 1.02 | 0.98 |
| syn. iid | E(Y | X) | GLS (raw cov.) | 2 | 0.53 | 0.29 | 0.47 | 0.42 | 3.05 |
| syn. iid | E(Y | X) | bootstrap percentile | 2 | 0.94 | 0.93 | 0.90 | 0.93 | 1.01 |
| syn. iid | E(Y | X) | bootstrap Wald | 2 | 0.94 | 0.94 | 0.90 | 0.92 | 0.98 |
| syn. iid | beta(s,t) | REML model-based | 2 | 0.99 | 0.99 | 0.97 | 1.02 | 1.01 |
| syn. iid | beta(s,t) | REML + CL2 | 2 | 0.99 | 0.99 | 0.97 | 1.00 | 1.00 |
| syn. iid | beta(s,t) | pcre | 2 | 0.99 | 0.99 | 0.98 | 1.04 | 1.02 |
| syn. iid | beta(s,t) | GLS (FPCA cov.) | 2 | 0.99 | 0.99 | 0.98 | 1.02 | 1.01 |
| syn. iid | beta(s,t) | GLS (raw cov.) | 2 | 0.74 | 0.59 | 0.65 | 0.49 | 1.50 |
| syn. iid | beta(s,t) | bootstrap percentile | 2 | 0.94 | 0.94 | 0.90 | 0.82 | 0.98 |
| syn. iid | beta(s,t) | bootstrap Wald | 2 | 0.96 | 0.96 | 0.92 | 0.81 | 0.90 |
| syn. iid | alpha(t) | REML model-based | 2 | 0.96 | 0.96 | 0.93 | 1.01 | 0.98 |
| syn. iid | alpha(t) | REML + CL2 | 2 | 0.95 | 0.95 | 0.92 | 1.00 | 1.00 |
| syn. iid | alpha(t) | pcre | 2 | 0.96 | 0.96 | 0.93 | 1.00 | 0.98 |
| syn. iid | alpha(t) | GLS (FPCA cov.) | 2 | 0.96 | 0.96 | 0.94 | 1.01 | 0.98 |
| syn. iid | alpha(t) | GLS (raw cov.) | 2 | 0.41 | 0.17 | 0.36 | 0.42 | 5.72 |
| syn. iid | alpha(t) | bootstrap percentile | 2 | 0.94 | 0.93 | 0.90 | 0.95 | 1.01 |
| syn. iid | alpha(t) | bootstrap Wald | 2 | 0.94 | 0.93 | 0.91 | 0.95 | 0.99 |
| syn. iid | gamma(t) | REML model-based | 2 | 0.96 | 0.96 | 0.93 | 1.01 | 0.97 |
| syn. iid | gamma(t) | REML + CL2 | 2 | 0.95 | 0.95 | 0.92 | 1.00 | 1.00 |
| syn. iid | gamma(t) | pcre | 2 | 0.96 | 0.96 | 0.94 | 1.04 | 0.98 |
| syn. iid | gamma(t) | GLS (FPCA cov.) | 2 | 0.96 | 0.96 | 0.94 | 1.02 | 0.97 |
| syn. iid | gamma(t) | GLS (raw cov.) | 2 | 0.58 | 0.36 | 0.51 | 0.41 | 2.40 |
| syn. iid | gamma(t) | bootstrap percentile | 2 | 0.94 | 0.94 | 0.89 | 0.95 | 1.00 |
| syn. iid | gamma(t) | bootstrap Wald | 2 | 0.94 | 0.94 | 0.90 | 0.94 | 0.97 |
| setting | estimand | fit | cells | rel_error | mse_vs_REML | vs_NCV_min | vs_NCV_max | mse_vs_NCV |
|---|---|---|---|---|---|---|---|---|
| plasmode | E(Y | X) | REML | 12 | 0.21 | 1.00 | 1.02 | 1.48 | 1.14 |
| plasmode | E(Y | X) | NCV | 12 | 0.19 | 0.87 | 1.00 | 1.00 | 1.00 |
| plasmode | E(Y | X) | pcre | 12 | 0.21 | 0.97 | 0.86 | 1.30 | 1.11 |
| plasmode | E(Y | X) | GLS (FPCA cov.) | 12 | 0.19 | 0.89 | 0.77 | 1.27 | 1.01 |
| plasmode | E(Y | X) | GLS (raw cov.) | 12 | 0.18 | 0.79 | 0.67 | 1.21 | 0.91 |
| plasmode | beta(s,t) | REML | 12 | 1.49 | 1.00 | 1.08 | 8.54 | 2.73 |
| plasmode | beta(s,t) | NCV | 12 | 0.96 | 0.37 | 1.00 | 1.00 | 1.00 |
| plasmode | beta(s,t) | pcre | 12 | 0.78 | 0.26 | 0.23 | 1.25 | 0.72 |
| plasmode | beta(s,t) | GLS (FPCA cov.) | 12 | 0.76 | 0.25 | 0.21 | 1.20 | 0.68 |
| plasmode | beta(s,t) | GLS (raw cov.) | 12 | 0.89 | 0.35 | 0.47 | 2.06 | 0.94 |
| plasmode | alpha(t) | REML | 12 | 0.64 | 1.00 | 0.86 | 1.47 | 1.07 |
| plasmode | alpha(t) | NCV | 12 | 0.63 | 0.93 | 1.00 | 1.00 | 1.00 |
| plasmode | alpha(t) | pcre | 12 | 0.65 | 0.95 | 0.71 | 1.37 | 1.01 |
| plasmode | alpha(t) | GLS (FPCA cov.) | 12 | 0.66 | 0.86 | 0.58 | 1.24 | 0.92 |
| plasmode | alpha(t) | GLS (raw cov.) | 12 | 0.61 | 0.73 | 0.34 | 1.16 | 0.78 |
| plasmode | gamma(t) | REML | 12 | 0.85 | 1.00 | 0.91 | 1.71 | 1.17 |
| plasmode | gamma(t) | NCV | 12 | 0.82 | 0.85 | 1.00 | 1.00 | 1.00 |
| plasmode | gamma(t) | pcre | 12 | 0.89 | 0.86 | 0.66 | 2.21 | 1.01 |
| plasmode | gamma(t) | GLS (FPCA cov.) | 12 | 0.87 | 0.80 | 0.64 | 1.54 | 0.94 |
| plasmode | gamma(t) | GLS (raw cov.) | 12 | 0.75 | 0.79 | 0.63 | 1.34 | 0.93 |
| syn. dep. | E(Y | X) | REML | 4 | 0.29 | 1.00 | 1.43 | 1.70 | 1.55 |
| syn. dep. | E(Y | X) | NCV | 4 | 0.23 | 0.65 | 1.00 | 1.00 | 1.00 |
| syn. dep. | E(Y | X) | pcre | 4 | 0.25 | 0.77 | 1.13 | 1.26 | 1.19 |
| syn. dep. | E(Y | X) | GLS (FPCA cov.) | 4 | 0.25 | 0.77 | 1.14 | 1.25 | 1.19 |
| syn. dep. | E(Y | X) | GLS (raw cov.) | 4 | 0.29 | 0.96 | 1.04 | 1.89 | 1.48 |
| syn. dep. | beta(s,t) | REML | 4 | 0.65 | 1.00 | 5.76 | 10.45 | 7.41 |
| syn. dep. | beta(s,t) | NCV | 4 | 0.24 | 0.13 | 1.00 | 1.00 | 1.00 |
| syn. dep. | beta(s,t) | pcre | 4 | 0.31 | 0.24 | 1.41 | 2.50 | 1.75 |
| syn. dep. | beta(s,t) | GLS (FPCA cov.) | 4 | 0.31 | 0.24 | 1.41 | 2.53 | 1.79 |
| syn. dep. | beta(s,t) | GLS (raw cov.) | 4 | 0.52 | 0.51 | 1.61 | 7.25 | 3.75 |
| syn. dep. | alpha(t) | REML | 4 | 0.32 | 1.00 | 1.03 | 1.10 | 1.06 |
| syn. dep. | alpha(t) | NCV | 4 | 0.31 | 0.94 | 1.00 | 1.00 | 1.00 |
| syn. dep. | alpha(t) | pcre | 4 | 0.32 | 0.99 | 1.04 | 1.06 | 1.05 |
| syn. dep. | alpha(t) | GLS (FPCA cov.) | 4 | 0.32 | 0.99 | 1.04 | 1.06 | 1.05 |
| syn. dep. | alpha(t) | GLS (raw cov.) | 4 | 0.41 | 1.58 | 1.04 | 2.69 | 1.68 |
| syn. dep. | gamma(t) | REML | 4 | 0.24 | 1.00 | 1.04 | 1.12 | 1.07 |
| syn. dep. | gamma(t) | NCV | 4 | 0.24 | 0.93 | 1.00 | 1.00 | 1.00 |
| syn. dep. | gamma(t) | pcre | 4 | 0.24 | 0.97 | 1.01 | 1.09 | 1.04 |
| syn. dep. | gamma(t) | GLS (FPCA cov.) | 4 | 0.24 | 0.96 | 1.01 | 1.07 | 1.03 |
| syn. dep. | gamma(t) | GLS (raw cov.) | 4 | 0.25 | 0.99 | 0.94 | 1.24 | 1.06 |
| syn. iid | E(Y | X) | REML | 2 | 0.11 | 1.00 | 1.02 | 1.04 | 1.03 |
| syn. iid | E(Y | X) | NCV | 2 | 0.11 | 0.97 | 1.00 | 1.00 | 1.00 |
| syn. iid | E(Y | X) | pcre | 2 | 0.11 | 1.00 | 1.02 | 1.04 | 1.03 |
| syn. iid | E(Y | X) | GLS (FPCA cov.) | 2 | 0.11 | 1.00 | 1.02 | 1.04 | 1.03 |
| syn. iid | E(Y | X) | GLS (raw cov.) | 2 | 0.15 | 1.61 | 1.51 | 1.83 | 1.66 |
| syn. iid | beta(s,t) | REML | 2 | 0.13 | 1.00 | 1.17 | 1.53 | 1.34 |
| syn. iid | beta(s,t) | NCV | 2 | 0.11 | 0.75 | 1.00 | 1.00 | 1.00 |
| syn. iid | beta(s,t) | pcre | 2 | 0.13 | 0.98 | 1.14 | 1.51 | 1.31 |
| syn. iid | beta(s,t) | GLS (FPCA cov.) | 2 | 0.13 | 0.99 | 1.16 | 1.52 | 1.33 |
| syn. iid | beta(s,t) | GLS (raw cov.) | 2 | 0.14 | 1.13 | 1.12 | 2.06 | 1.52 |
| syn. iid | alpha(t) | REML | 2 | 0.15 | 1.00 | 0.96 | 0.96 | 0.96 |
| syn. iid | alpha(t) | NCV | 2 | 0.16 | 1.04 | 1.00 | 1.00 | 1.00 |
| syn. iid | alpha(t) | pcre | 2 | 0.15 | 1.00 | 0.96 | 0.96 | 0.96 |
| syn. iid | alpha(t) | GLS (FPCA cov.) | 2 | 0.15 | 1.00 | 0.96 | 0.96 | 0.96 |
| syn. iid | alpha(t) | GLS (raw cov.) | 2 | 0.30 | 3.37 | 2.14 | 4.90 | 3.24 |
| syn. iid | gamma(t) | REML | 2 | 0.12 | 1.00 | 0.96 | 0.96 | 0.96 |
| syn. iid | gamma(t) | NCV | 2 | 0.12 | 1.04 | 1.00 | 1.00 | 1.00 |
| syn. iid | gamma(t) | pcre | 2 | 0.12 | 1.00 | 0.96 | 0.97 | 0.96 |
| syn. iid | gamma(t) | GLS (FPCA cov.) | 2 | 0.12 | 1.00 | 0.96 | 0.96 | 0.96 |
| syn. iid | gamma(t) | GLS (raw cov.) | 2 | 0.12 | 1.11 | 0.97 | 1.18 | 1.07 |
| app | G | REML | pcre | GLS (raw cov.) | GLS (FPCA cov.) | NCV |
|---|---|---|---|---|---|---|
| ECG strain | 78 | 0.74 | 0.74 | 0.66 | 0.71 | 0.71 |
| ECG strain | 40 | 1.10 | 0.97 | 0.95 | 0.94 | 0.97 |
| AF trial | 118 | 1.61 | 0.80 | 0.69 | 0.79 | 0.87 |
| AF trial | 40 | 3.43 | 1.19 | 1.08 | 1.16 | 1.49 |
| running | 90 | 1.05 | 0.70 | 0.68 | 0.66 | 0.75 |
| running | 40 | 1.75 | 0.97 | 1.01 | 0.91 | 0.95 |
| DTI | 92 | 0.63 | 0.55 | 0.52 | 0.54 | 0.49 |
| DTI | 40 | 0.93 | 0.75 | 0.83 | 0.74 | 0.67 |
| gait | 138 | 1.36 | 0.70 | 0.69 | 0.69 | 1.01 |
| gait | 40 | 2.42 | 1.14 | 1.40 | 1.09 | 1.22 |
| ECG 8-lead | 100 | 2.77 | 0.57 | 1.12 | 0.55 | 1.19 |
| ECG 8-lead | 40 | 4.82 | 0.80 | 2.37 | 0.81 | 1.65 |
Comparator run details (bootstrap on replicates 1–100; the GLS algorithm rebuilt from refund’s internals, with the documented nearPD repair because refund’s compute_sqrt_sigma_inv() returns NaN for a singular residual covariance; two failed bootstrap rows in DTI at G = 40, kept; time per arm in comparators-time.csv): comparators/summaries/SUMMARY.md.
7.1 CL2 on the GLS and pcre fits
GLS whitens each curve with Σ̂^{-1/2}; the transform acts within curves, so curves stay independent clusters and CL2 (Bayesian form, curves as clusters) applies to the whitened fit, valid even if Σ̂ is wrong. pcre + CL2 is CL2 on the pcre fit. Same cells and replicates as above; shown here for replicates 1–100, where all arms exist (comparators/summaries/cl2-arms-cells.csv, cl2-arms-SUMMARY.md: CL2 on the whitened fit reproduces refund’s pffr_vcov() construction and a direct dense computation to ~1e-10; one pcre + CL2 replicate failed in the SVD and is excluded for that arm only).
Under dependent synthetic errors GLS (FPCA covariance) + CL2 covers β at 0.959 at 0.61 times REML + CL2’s width (interval score 0.55 times). On the plasmode cells it covers β at 0.823 (per dataset 0.65–0.90; plain GLS 0.731, REML + CL2 0.931) at 0.46 times the width; its SE matches the spread of the GLS estimate there (cl2-arms-SUMMARY.md), so the remaining shortfall is bias of the GLS estimate on real residual curves. pcre + CL2 behaves alike (β 0.831) and repairs pcre’s α intervals on synthetic data (0.56 → 0.95). GLS with the raw residual covariance + CL2 fails (0.65 under dependent synthetic errors): with more grid points than curves, Σ̂ absorbs the residuals themselves.
| setting | estimand | arm | coverage | cov_min | q05 | width_rel | score_rel | mse_vs_NCV | detection |
|---|---|---|---|---|---|---|---|---|---|
| plasmode | E(Y | X) | REML + CL2 | 0.94 | 0.92 | 0.87 | 1.00 | 1.00 | 1.15 | 0.97 |
| plasmode | E(Y | X) | NCV + CL2 | 0.89 | 0.85 | 0.74 | 0.81 | 1.04 | 1.00 | 0.99 |
| plasmode | E(Y | X) | GLS (FPCA cov.) | 0.80 | 0.66 | 0.57 | 0.63 | 1.34 | 1.01 | 0.99 |
| plasmode | E(Y | X) | GLS (FPCA cov.) + CL2 | 0.89 | 0.83 | 0.74 | 0.79 | 1.02 | 1.01 | 0.99 |
| plasmode | E(Y | X) | GLS (raw cov.) + CL2 | 0.84 | 0.51 | 0.73 | 0.66 | 1.10 | 0.90 | 0.99 |
| plasmode | E(Y | X) | pcre | 0.78 | 0.73 | 0.54 | 0.63 | 1.45 | 1.10 | 0.99 |
| plasmode | E(Y | X) | pcre + CL2 | 0.91 | 0.86 | 0.74 | 0.86 | 1.04 | 1.10 | 0.99 |
| plasmode | beta(s,t) | REML + CL2 | 0.93 | 0.91 | 0.87 | 1.00 | 1.00 | 2.75 | 0.38 |
| plasmode | beta(s,t) | NCV + CL2 | 0.76 | 0.60 | 0.46 | 0.46 | 1.13 | 1.00 | 0.54 |
| plasmode | beta(s,t) | GLS (FPCA cov.) | 0.73 | 0.60 | 0.45 | 0.38 | 1.02 | 0.67 | 0.55 |
| plasmode | beta(s,t) | GLS (FPCA cov.) + CL2 | 0.82 | 0.63 | 0.56 | 0.46 | 0.83 | 0.67 | 0.48 |
| plasmode | beta(s,t) | GLS (raw cov.) + CL2 | 0.79 | 0.52 | 0.63 | 0.48 | 0.95 | 0.90 | 0.57 |
| plasmode | beta(s,t) | pcre | 0.74 | 0.59 | 0.42 | 0.40 | 1.03 | 0.71 | 0.49 |
| plasmode | beta(s,t) | pcre + CL2 | 0.83 | 0.66 | 0.53 | 0.49 | 0.85 | 0.71 | 0.42 |
| plasmode | alpha(t) | REML + CL2 | 0.94 | 0.91 | 0.88 | 1.00 | 1.00 | 1.06 | 0.76 |
| plasmode | alpha(t) | NCV + CL2 | 0.89 | 0.79 | 0.77 | 0.85 | 1.12 | 1.00 | 0.79 |
| plasmode | alpha(t) | GLS (FPCA cov.) | 0.75 | 0.57 | 0.62 | 0.58 | 1.52 | 0.93 | 0.86 |
| plasmode | alpha(t) | GLS (FPCA cov.) + CL2 | 0.87 | 0.77 | 0.77 | 0.74 | 1.15 | 0.93 | 0.80 |
| plasmode | alpha(t) | GLS (raw cov.) + CL2 | 0.83 | 0.56 | 0.74 | 0.61 | 1.10 | 0.77 | 0.86 |
| plasmode | alpha(t) | pcre | 0.77 | 0.59 | 0.63 | 0.62 | 1.54 | 1.02 | 0.85 |
| plasmode | alpha(t) | pcre + CL2 | 0.89 | 0.80 | 0.77 | 0.83 | 1.15 | 1.02 | 0.78 |
| plasmode | gamma(t) | REML + CL2 | 0.94 | 0.92 | 0.89 | 1.00 | 1.00 | 1.19 | 0.41 |
| plasmode | gamma(t) | NCV + CL2 | 0.90 | 0.86 | 0.79 | 0.84 | 1.01 | 1.00 | 0.45 |
| plasmode | gamma(t) | GLS (FPCA cov.) | 0.79 | 0.50 | 0.63 | 0.65 | 1.34 | 0.96 | 0.52 |
| plasmode | gamma(t) | GLS (FPCA cov.) + CL2 | 0.85 | 0.65 | 0.74 | 0.73 | 1.10 | 0.96 | 0.46 |
| plasmode | gamma(t) | GLS (raw cov.) + CL2 | 0.82 | 0.52 | 0.73 | 0.64 | 1.16 | 0.94 | 0.55 |
| plasmode | gamma(t) | pcre | 0.80 | 0.45 | 0.65 | 0.71 | 1.33 | 1.02 | 0.49 |
| plasmode | gamma(t) | pcre + CL2 | 0.87 | 0.54 | 0.74 | 0.83 | 1.13 | 1.02 | 0.40 |
| syn. dep. | E(Y | X) | REML + CL2 | 0.94 | 0.93 | 0.90 | 1.00 | 1.00 | 1.59 | 0.98 |
| syn. dep. | E(Y | X) | NCV + CL2 | 0.93 | 0.92 | 0.87 | 0.76 | 0.79 | 1.00 | 0.99 |
| syn. dep. | E(Y | X) | GLS (FPCA cov.) | 0.93 | 0.91 | 0.87 | 0.81 | 0.87 | 1.19 | 0.99 |
| syn. dep. | E(Y | X) | GLS (FPCA cov.) + CL2 | 0.94 | 0.93 | 0.90 | 0.86 | 0.87 | 1.19 | 0.99 |
| syn. dep. | E(Y | X) | GLS (raw cov.) + CL2 | 0.59 | 0.28 | 0.51 | 0.39 | 2.11 | 1.50 | 1.00 |
| syn. dep. | E(Y | X) | pcre | 0.90 | 0.88 | 0.84 | 0.76 | 0.89 | 1.19 | 0.99 |
| syn. dep. | E(Y | X) | pcre + CL2 | 0.98 | 0.97 | 0.95 | 1.05 | 0.91 | 1.19 | 0.98 |
| syn. dep. | beta(s,t) | REML + CL2 | 0.94 | 0.93 | 0.89 | 1.00 | 1.00 | 7.72 | 0.66 |
| syn. dep. | beta(s,t) | NCV + CL2 | 0.93 | 0.92 | 0.87 | 0.36 | 0.37 | 1.00 | 0.95 |
| syn. dep. | beta(s,t) | GLS (FPCA cov.) | 0.95 | 0.93 | 0.91 | 0.58 | 0.55 | 1.84 | 0.87 |
| syn. dep. | beta(s,t) | GLS (FPCA cov.) + CL2 | 0.96 | 0.94 | 0.92 | 0.61 | 0.55 | 1.84 | 0.86 |
| syn. dep. | beta(s,t) | GLS (raw cov.) + CL2 | 0.65 | 0.36 | 0.57 | 0.36 | 1.43 | 3.87 | 0.91 |
| syn. dep. | beta(s,t) | pcre | 0.96 | 0.93 | 0.92 | 0.59 | 0.55 | 1.81 | 0.87 |
| syn. dep. | beta(s,t) | pcre + CL2 | 0.98 | 0.96 | 0.96 | 0.71 | 0.58 | 1.81 | 0.82 |
| syn. dep. | alpha(t) | REML + CL2 | 0.95 | 0.94 | 0.91 | 1.00 | 1.00 | 1.07 | 0.98 |
| syn. dep. | alpha(t) | NCV + CL2 | 0.93 | 0.93 | 0.86 | 0.93 | 0.98 | 1.00 | 0.99 |
| syn. dep. | alpha(t) | GLS (FPCA cov.) | 0.92 | 0.89 | 0.84 | 0.90 | 1.03 | 1.06 | 1.00 |
| syn. dep. | alpha(t) | GLS (FPCA cov.) + CL2 | 0.93 | 0.92 | 0.84 | 0.95 | 1.02 | 1.06 | 0.99 |
| syn. dep. | alpha(t) | GLS (raw cov.) + CL2 | 0.52 | 0.21 | 0.46 | 0.42 | 3.08 | 1.65 | 1.00 |
| syn. dep. | alpha(t) | pcre | 0.56 | 0.51 | 0.47 | 0.40 | 2.37 | 1.05 | 1.00 |
| syn. dep. | alpha(t) | pcre + CL2 | 0.95 | 0.94 | 0.92 | 1.00 | 0.99 | 1.05 | 0.99 |
| syn. dep. | gamma(t) | REML + CL2 | 0.95 | 0.95 | 0.92 | 1.00 | 1.00 | 1.05 | 0.92 |
| syn. dep. | gamma(t) | NCV + CL2 | 0.93 | 0.92 | 0.87 | 0.91 | 0.99 | 1.00 | 0.94 |
| syn. dep. | gamma(t) | GLS (FPCA cov.) | 0.93 | 0.91 | 0.89 | 0.90 | 0.98 | 1.02 | 0.94 |
| syn. dep. | gamma(t) | GLS (FPCA cov.) + CL2 | 0.94 | 0.93 | 0.90 | 0.95 | 0.99 | 1.02 | 0.93 |
| syn. dep. | gamma(t) | GLS (raw cov.) + CL2 | 0.61 | 0.33 | 0.54 | 0.41 | 2.06 | 1.04 | 0.99 |
| syn. dep. | gamma(t) | pcre | 0.94 | 0.93 | 0.89 | 0.92 | 0.97 | 1.03 | 0.93 |
| syn. dep. | gamma(t) | pcre + CL2 | 0.98 | 0.98 | 0.96 | 1.22 | 1.08 | 1.03 | 0.87 |
| syn. iid | E(Y | X) | REML + CL2 | 0.96 | 0.96 | 0.92 | 1.00 | 1.00 | 1.03 | 1.00 |
| syn. iid | E(Y | X) | NCV + CL2 | 0.95 | 0.95 | 0.90 | 0.92 | 0.97 | 1.00 | 1.00 |
| syn. iid | E(Y | X) | GLS (FPCA cov.) | 0.97 | 0.97 | 0.94 | 1.02 | 0.98 | 1.03 | 1.00 |
| syn. iid | E(Y | X) | GLS (FPCA cov.) + CL2 | 0.96 | 0.96 | 0.92 | 1.00 | 1.00 | 1.03 | 1.00 |
| syn. iid | E(Y | X) | GLS (raw cov.) + CL2 | 0.46 | 0.14 | 0.38 | 0.28 | 3.45 | 1.66 | 1.00 |
| syn. iid | E(Y | X) | pcre | 0.97 | 0.97 | 0.94 | 1.03 | 0.99 | 1.03 | 1.00 |
| syn. iid | E(Y | X) | pcre + CL2 | 0.97 | 0.97 | 0.94 | 1.02 | 1.01 | 1.03 | 1.00 |
| syn. iid | beta(s,t) | REML + CL2 | 0.99 | 0.99 | 0.97 | 1.00 | 1.00 | 1.27 | 0.97 |
| syn. iid | beta(s,t) | NCV + CL2 | 0.98 | 0.98 | 0.94 | 0.75 | 0.78 | 1.00 | 0.98 |
| syn. iid | beta(s,t) | GLS (FPCA cov.) | 0.99 | 0.99 | 0.97 | 1.02 | 1.01 | 1.26 | 0.97 |
| syn. iid | beta(s,t) | GLS (FPCA cov.) + CL2 | 0.99 | 0.99 | 0.97 | 1.00 | 1.00 | 1.26 | 0.97 |
| syn. iid | beta(s,t) | GLS (raw cov.) + CL2 | 0.64 | 0.38 | 0.54 | 0.37 | 1.84 | 1.44 | 0.99 |
| syn. iid | beta(s,t) | pcre | 0.99 | 0.99 | 0.98 | 1.03 | 1.02 | 1.25 | 0.97 |
| syn. iid | beta(s,t) | pcre + CL2 | 0.99 | 0.99 | 0.97 | 1.01 | 1.01 | 1.25 | 0.97 |
| syn. iid | alpha(t) | REML + CL2 | 0.95 | 0.95 | 0.92 | 1.00 | 1.00 | 0.98 | 1.00 |
| syn. iid | alpha(t) | NCV + CL2 | 0.95 | 0.94 | 0.90 | 0.98 | 1.01 | 1.00 | 1.00 |
| syn. iid | alpha(t) | GLS (FPCA cov.) | 0.96 | 0.96 | 0.92 | 1.01 | 0.98 | 0.98 | 1.00 |
| syn. iid | alpha(t) | GLS (FPCA cov.) + CL2 | 0.95 | 0.95 | 0.92 | 1.00 | 1.00 | 0.98 | 1.00 |
| syn. iid | alpha(t) | GLS (raw cov.) + CL2 | 0.36 | 0.06 | 0.30 | 0.25 | 6.22 | 3.36 | 1.00 |
| syn. iid | alpha(t) | pcre | 0.96 | 0.96 | 0.92 | 1.00 | 0.98 | 0.98 | 1.00 |
| syn. iid | alpha(t) | pcre + CL2 | 0.96 | 0.95 | 0.92 | 1.00 | 1.00 | 0.98 | 1.00 |
| syn. iid | gamma(t) | REML + CL2 | 0.96 | 0.95 | 0.92 | 1.00 | 1.00 | 0.96 | 1.00 |
| syn. iid | gamma(t) | NCV + CL2 | 0.95 | 0.95 | 0.88 | 0.96 | 1.01 | 1.00 | 1.00 |
| syn. iid | gamma(t) | GLS (FPCA cov.) | 0.97 | 0.96 | 0.93 | 1.02 | 0.98 | 0.96 | 1.00 |
| syn. iid | gamma(t) | GLS (FPCA cov.) + CL2 | 0.96 | 0.96 | 0.92 | 1.00 | 1.00 | 0.96 | 1.00 |
| syn. iid | gamma(t) | GLS (raw cov.) + CL2 | 0.48 | 0.16 | 0.42 | 0.26 | 2.87 | 1.06 | 1.00 |
| syn. iid | gamma(t) | pcre | 0.97 | 0.97 | 0.94 | 1.04 | 0.99 | 0.96 | 1.00 |
| syn. iid | gamma(t) | pcre + CL2 | 0.96 | 0.96 | 0.92 | 1.03 | 1.01 | 0.96 | 1.00 |
8 Intervals centred at the NCV fit
NCV + CL2 under-covers (Section 3.5). This section collects the diagnosis and every correction evaluated (ncv-rbc/, lambda-correction/): a one-step bias correction of the NCV estimate (θ̃ = (2I − A)θ̂ with A = V_e V_p⁻¹, covariance propagated through the same linear map; frequentist CL2 inside, plus the Bayesian allowance propagated as (I − A)(V_p − V_e)(I − A)’), two steps of it, a cluster-robust term for the variability of the NCV smoothing parameters (infinitesimal jackknife over curves, with or without the cross term between coefficients and smoothing parameters; Gaussian only), their combinations, and the NCV estimate with REML + CL2’s half-width. Replicates: synthetic Gaussian/Poisson/binary core and rough-truth cells, fresh replicates 201–400 (combinations: Gaussian, 201–300); plasmode cells with all subjects and curve flips, replicates 1–100 (combinations 1–50); misspecified truths with the default basis, replicates 1–100. SE/SD: median over grid points of the root-mean-square estimated SE over the replicate SD of the estimate.
Why NCV + CL2 misses. In the synthetic Gaussian/Poisson cells its SE matches the spread of the NCV estimate and the shortfall is smoothing bias (mean |bias|/SD of β 0.24–0.36 with the smooth truth, 0.46–0.74 with the rough truth; Section 3.5). On the plasmode datasets two things go wrong. Its SE is short (SE/SD 0.66–0.96 across datasets), and refitting with the smoothing parameters fixed at each cell’s median NCV value restores the SE (1.03–1.18), so the shortfall comes from the variability of the selected smoothing parameters. Even then coverage of β stays at 0.64–0.92: smoothing bias at the amount of smoothing NCV selects remains (Table 49).
Corrections. The one-step bias correction removes part of the bias on synthetic data (Table 47) but not enough on rough truths or for binary responses, and adds little on the plasmode datasets. The λ term, alone or combined with the bias correction, raises coverage further but overshoots the variance (SE/SD above 1) on synthetic data (Table 48) and still leaves β at 0.65–0.94 (one step + λ term) on the plasmode datasets. With truths outside the spline space every NCV-centred interval fails (Table 50). The NCV estimate with REML + CL2’s half-width has REML + CL2’s width by construction; its better interval score reflects only the more accurate centre, its detection of clearly non-zero points is lower, and it under-covers under the REML-fitted truth (Table 51; Section 10.10).
| group | arm | cells | cov_min | coverage | q05 | width | se_sd | detect |
|---|---|---|---|---|---|---|---|---|
| rough truth, Gaussian/Poisson | REML + CL2 | 5 | 0.937 | 0.940 | 0.914 | 1.000 | 0.973 | 0.746 |
| rough truth, Gaussian/Poisson | NCV + CL2 | 5 | 0.817 | 0.849 | 0.609 | 0.515 | 0.933 | 0.922 |
| rough truth, Gaussian/Poisson | bias-corrected (1 step) | 5 | 0.873 | 0.899 | 0.784 | 0.623 | 0.917 | 0.902 |
| rough truth, Gaussian/Poisson | bias-corrected (2 steps, freq.) | 5 | 0.888 | 0.911 | 0.835 | 0.679 | 0.917 | 0.885 |
| rough truth, binary | REML + CL2 | 2 | 0.943 | 0.944 | 0.918 | 1.000 | 0.947 | 0.349 |
| rough truth, binary | NCV + CL2 | 2 | 0.810 | 0.816 | 0.483 | 0.406 | 0.881 | 0.678 |
| rough truth, binary | bias-corrected (1 step) | 2 | 0.864 | 0.867 | 0.690 | 0.493 | 0.837 | 0.654 |
| rough truth, binary | bias-corrected (2 steps, freq.) | 2 | 0.874 | 0.876 | 0.752 | 0.538 | 0.821 | 0.628 |
| smooth truth, Gaussian/Poisson | REML + CL2 | 8 | 0.929 | 0.937 | 0.911 | 1.000 | 0.952 | 0.656 |
| smooth truth, Gaussian/Poisson | NCV + CL2 | 8 | 0.919 | 0.930 | 0.872 | 0.381 | 0.920 | 0.933 |
| smooth truth, Gaussian/Poisson | bias-corrected (1 step) | 8 | 0.926 | 0.932 | 0.901 | 0.468 | 0.894 | 0.910 |
| smooth truth, Gaussian/Poisson | bias-corrected (2 steps, freq.) | 8 | 0.919 | 0.926 | 0.895 | 0.515 | 0.883 | 0.891 |
| smooth truth, binary | REML + CL2 | 4 | 0.929 | 0.940 | 0.913 | 1.000 | 0.923 | 0.225 |
| smooth truth, binary | NCV + CL2 | 4 | 0.800 | 0.864 | 0.746 | 0.341 | 0.819 | 0.558 |
| smooth truth, binary | bias-corrected (1 step) | 4 | 0.815 | 0.876 | 0.800 | 0.413 | 0.780 | 0.553 |
| smooth truth, binary | bias-corrected (2 steps, freq.) | 4 | 0.814 | 0.871 | 0.797 | 0.450 | 0.766 | 0.535 |
| group | arm | coverage | q05 | width | se_sd |
|---|---|---|---|---|---|
| rough truth | REML + CL2 | 0.940 | 0.915 | 1.000 | 0.971 |
| rough truth | NCV + CL2 | 0.846 | 0.591 | 0.509 | 0.915 |
| rough truth | bias-corrected (1 step) | 0.894 | 0.767 | 0.614 | 0.898 |
| rough truth | NCV + λ term | 0.876 | 0.653 | 0.566 | 1.046 |
| rough truth | NCV + λ term (cross) | 0.889 | 0.677 | 0.597 | 1.113 |
| rough truth | 1 step + λ term | 0.929 | 0.820 | 0.717 | 1.100 |
| rough truth | 1 step + λ term (cross) | 0.941 | 0.838 | 0.765 | 1.185 |
| rough truth | 2 steps + λ term (cross) | 0.958 | 0.887 | 0.869 | 1.250 |
| smooth truth | REML + CL2 | 0.938 | 0.911 | 1.000 | 0.953 |
| smooth truth | NCV + CL2 | 0.925 | 0.867 | 0.383 | 0.898 |
| smooth truth | bias-corrected (1 step) | 0.928 | 0.886 | 0.471 | 0.872 |
| smooth truth | NCV + λ term | 0.948 | 0.890 | 0.436 | 1.108 |
| smooth truth | NCV + λ term (cross) | 0.957 | 0.900 | 0.457 | 1.176 |
| smooth truth | 1 step + λ term | 0.961 | 0.920 | 0.564 | 1.186 |
| smooth truth | 1 step + λ term (cross) | 0.971 | 0.938 | 0.597 | 1.271 |
| smooth truth | 2 steps + λ term (cross) | 0.975 | 0.942 | 0.691 | 1.358 |
| app | NCV + CL2 | NCV + λ term | NCV, λ fixed at median (oracle) | bias-corrected (1 step) | 1 step + λ term | 1 step + λ term (cross) | REML + CL2 |
|---|---|---|---|---|---|---|---|
| ECG strain | 0.89 (0.76) 0.95 | 0.92 (0.80) 1.02 | 0.92 (0.85) 1.03 | 0.91 (0.82) 0.96 | 0.94 (0.86) 1.06 | 0.94 (0.87) 1.10 | 0.94 (0.89) 1.01 |
| AF trial | 0.81 (0.53) 0.84 | 0.88 (0.65) 1.12 | 0.87 (0.57) 1.07 | 0.84 (0.66) 0.83 | 0.92 (0.79) 1.27 | 0.94 (0.81) 1.44 | 0.94 (0.86) 0.96 |
| running | 0.72 (0.42) 0.76 | 0.77 (0.47) 0.95 | 0.69 (0.29) 1.04 | 0.76 (0.48) 0.77 | 0.81 (0.53) 1.07 | 0.83 (0.55) 1.20 | 0.91 (0.84) 0.90 |
| DTI | 0.84 (0.54) 0.96 | 0.87 (0.61) 1.09 | 0.86 (0.57) 1.07 | 0.88 (0.68) 0.94 | 0.92 (0.75) 1.16 | 0.93 (0.77) 1.24 | 0.93 (0.86) 0.99 |
| gait | 0.81 (0.53) 0.89 | 0.87 (0.64) 1.07 | 0.88 (0.62) 1.07 | 0.84 (0.61) 0.89 | 0.91 (0.74) 1.21 | 0.92 (0.75) 1.31 | 0.94 (0.89) 0.99 |
| ECG 8-lead | 0.76 (0.47) 0.81 | 0.86 (0.62) 1.29 | 0.85 (0.46) 1.16 | 0.80 (0.56) 0.78 | 0.90 (0.73) 1.52 | 0.92 (0.75) 1.64 | 0.94 (0.88) 0.94 |
| ocean | 0.72 (0.30) 0.86 | 0.76 (0.36) 1.03 | 0.71 (0.24) 1.03 | 0.78 (0.39) 0.88 | 0.83 (0.45) 1.11 | 0.84 (0.47) 1.20 | 0.93 (0.87) 0.94 |
| weather | 0.53 (0.30) 0.70 | 0.59 (0.41) 0.94 | 0.69 (0.29) 1.18 | 0.60 (0.43) 0.72 | 0.65 (0.50) 1.07 | 0.66 (0.53) 1.23 | 0.88 (0.82) 0.93 |
| electricity | 0.62 (0.37) 0.66 | 0.68 (0.44) 0.86 | 0.64 (0.30) 1.07 | 0.63 (0.38) 0.64 | 0.71 (0.47) 0.96 | 0.73 (0.50) 1.12 | 0.93 (0.88) 0.90 |
| setting | NCV + CL2 | 2 steps + λ term (cross) | bias-corrected (1 step) | 1 step + λ term | REML + CL2 |
|---|---|---|---|---|---|
| T0 independent G = 100 | 0.98 (0.94) | 0.99 (0.96) | 0.97 (0.94) | 0.98 (0.96) | 0.99 (0.97) |
| T0 smooth errors G = 100 | 0.94 (0.86) | 0.98 (0.96) | 0.94 (0.91) | 0.97 (0.94) | 0.94 (0.92) |
| T0 smooth errors G = 40 | 0.92 (0.87) | 0.98 (0.95) | 0.93 (0.90) | 0.97 (0.94) | 0.93 (0.90) |
| T1 independent G = 100 | 0.88 (0.11) | 0.90 (0.25) | 0.85 (0.13) | 0.88 (0.16) | 0.88 (0.01) |
| T1 smooth errors G = 100 | 0.87 (0.26) | 0.94 (0.64) | 0.87 (0.33) | 0.92 (0.49) | 0.91 (0.69) |
| T1 smooth errors G = 40 | 0.88 (0.48) | 0.95 (0.74) | 0.88 (0.53) | 0.93 (0.66) | 0.91 (0.84) |
| T2 independent G = 100 | 0.28 (0.00) | 0.33 (0.00) | 0.29 (0.00) | 0.31 (0.00) | 0.30 (0.00) |
| T2 smooth errors G = 100 | 0.28 (0.00) | 0.36 (0.00) | 0.30 (0.00) | 0.32 (0.00) | 0.44 (0.00) |
| T2 smooth errors G = 40 | 0.33 (0.00) | 0.45 (0.00) | 0.36 (0.00) | 0.40 (0.00) | 0.56 (0.00) |
| setting | arm | coverage | q05 | score_vs_REML | detection |
|---|---|---|---|---|---|
| plasmode, MID-fitted truth | REML + CL2 | 0.929 | 0.858 | 1.000 | 0.405 |
| plasmode, MID-fitted truth | NCV + CL2 | 0.769 | 0.450 | 1.123 | 0.585 |
| plasmode, MID-fitted truth | 1 step + λ term (cross) | 0.877 | 0.659 | 0.997 | 0.380 |
| plasmode, MID-fitted truth | NCV estimate, REML + CL2 width | 0.955 | 0.853 | 0.868 | 0.296 |
| plasmode, NCV-fitted truth | REML + CL2 | 0.917 | 0.846 | 1.000 | 0.394 |
| plasmode, NCV-fitted truth | NCV + CL2 | 0.827 | 0.651 | 0.724 | 0.649 |
| plasmode, NCV-fitted truth | 1 step + λ term (cross) | 0.910 | 0.835 | 0.717 | 0.441 |
| plasmode, NCV-fitted truth | NCV estimate, REML + CL2 width | 0.957 | 0.894 | 0.828 | 0.296 |
| plasmode, REML-fitted truth | REML + CL2 | 0.923 | 0.840 | 1.000 | 0.496 |
| plasmode, REML-fitted truth | NCV + CL2 | 0.680 | 0.393 | 2.079 | 0.532 |
| plasmode, REML-fitted truth | 1 step + λ term (cross) | 0.807 | 0.567 | 1.734 | 0.333 |
| plasmode, REML-fitted truth | NCV estimate, REML + CL2 width | 0.906 | 0.673 | 1.043 | 0.298 |
| synthetic, rough truth | REML + CL2 | 0.938 | 0.900 | 1.000 | 0.693 |
| synthetic, rough truth | NCV + CL2 | 0.846 | 0.591 | 0.731 | 0.905 |
| synthetic, rough truth | 1 step + λ term (cross) | 0.941 | 0.838 | 0.695 | 0.817 |
| synthetic, rough truth | NCV estimate, REML + CL2 width | 0.986 | 0.953 | 0.817 | 0.650 |
| synthetic, smooth truth | REML + CL2 | 0.934 | 0.897 | 1.000 | 0.661 |
| synthetic, smooth truth | NCV + CL2 | 0.925 | 0.867 | 0.409 | 0.940 |
| synthetic, smooth truth | 1 step + λ term (cross) | 0.971 | 0.938 | 0.485 | 0.878 |
| synthetic, smooth truth | NCV estimate, REML + CL2 width | 0.995 | 0.982 | 0.772 | 0.637 |
9 Smoothing-parameter uncertainty in CL2
CL2 treats the smoothing parameters as fixed. This section reports a curve-robust term for their variability, added to CL2 for the REML fit and for the curve-blocked NCV fit (lambda-correction/).
Construction. With \(\boldsymbol\rho = \log\boldsymbol\lambda\) (for REML also \(\log\phi\)), give each curve \(g\) a weight \(w_g\) in the fitting criterion \(C\). The implicit-function theorem gives the influence of curve \(g\) on \(\hat{\boldsymbol\rho}\), \(\mathbf d_g = -\mathbf H^{-1}(\mathbf s_g - \bar{\mathbf s})\), from the curve scores \(\mathbf s_g = \partial^2 C/\partial\boldsymbol\rho\,\partial w_g\) and the Hessian \(\mathbf H\) of \(C\) in \(\boldsymbol\rho\) (REML criterion with analytic curve scores; NCV criterion with curve blocks, analytic scores for Gaussian responses and central differences in \(\boldsymbol\rho\) for Poisson and binary responses, where a \(\rho_k\) with curvature below \(10^{-3}\) of the largest is treated as fixed). The infinitesimal jackknife (IJ) over curves gives \(\hat{\mathbf V}_\rho = \sum_g \mathbf d_g\mathbf d_g^\top\), which enters the coefficient covariance as \(\mathbf J\hat{\mathbf V}_\rho\mathbf J^\top\) with \(\mathbf J = \partial\hat{\boldsymbol\theta}/\partial\boldsymbol\rho\), on top of CL2 (Bayesian form). Variants:
- “+ IJ”: the term above; “+ IJ ×”: with the first-order cross term between the coefficient and the smoothing-parameter influences of each curve (total curve influence);
- “+ OS”: \(\hat{\mathbf V}_\rho\) from a one-step leave-curve-out update of ρ̂ instead of the IJ; “+ JK”: from the exact curve jackknife (refits with zero prior weights for one curve, warm start; REML only, reference);
- “jackknife refit”: the exact curve jackknife of \(\hat{\boldsymbol\theta}\) itself plus \(\mathbf V_p - \mathbf V_e\) (REML only, reference);
- “+ mgcv J”: mgcv’s own first-order term \(\mathbf J[\mathbf H^{-1}]_\rho\mathbf J^\top\) (model-based, not curve-robust); “+ mgcv Vc − Vp”: mgcv’s full smoothing-parameter correction \(\mathbf V_c - \mathbf V_p\) (REML only);
- “λ fixed”: CL2 of a refit with every smoothing parameter fixed at the cell’s geometric-mean selected value, computed from the same replicates (a diagnostic, not an interval one can compute from one dataset).
Runs. REML: the 18 core cells at the default signal (family × error process × G), the study’s 200 replicates, and all 130 plasmode cells with 200 replicates. NCV: the 6 Gaussian and 12 Poisson/binary core cells at the default signal with 200 replicates, and the 130 plasmode cells with replicates 1–100. Every run reproduces the study’s estimates and CL2 standard errors on the same replicates (common random numbers; lambda-correction/summaries*/ *-crn.csv). z intervals throughout. For the REML runs SE/SD is the root-mean-square estimated SE over the replicate SD; for the NCV runs, where the IJ inflation is heavy-tailed over replicates, the median-based SE/SD is reported.
REML. In the dependent core cells the IJ term raises REML + CL2’s coverage of β from 0.929 to 0.945 at G = 40 and from 0.946 to 0.958 at G = 100, at 1.09 and 1.07 times the width; the interval score changes by a factor 0.996 and 1.008 (Table 52, Table 53). mgcv’s own terms add less (0.935 and 0.936 at G = 40); the exact curve jackknife of the smoothing parameters gives 0.954, and the full jackknife refit overcovers (0.977). Fixing λ at the cell’s geometric-mean REML value covers 0.945 at 0.99 times CL2’s width (interval score 0.880 times): CL2’s shortfall comes from which λ REML selects in a replicate. The added width does not go where CL2 misses: within cells, the rank correlation between a replicate’s SE inflation and CL2’s share of missed grid points in that replicate is -0.24 (G = 40) and -0.11 (G = 100) under dependent errors (0.34 and 0.33 under independent errors; lambda-correction/summaries/lc-targeting.csv). On the plasmode cells the term raises β coverage by 0.0–1.8 pp per cell at interval-score ratios 0.96–1.11 (Table 54).
NCV. For the curve-blocked NCV fit the term brings β coverage in the dependent synthetic cells to nominal: Gaussian 0.92–0.93 → 0.95, Poisson 0.92–0.94 → 0.95, the same as the λ-fixed diagnostic (0.95), at 1.09–1.21 times the width (Table 55). The median SE/SD moves only from 0.80–0.86 to 0.83–0.89: the inflation is concentrated in a few replicates, and it is larger where CL2 misses more (within-cell rank correlation of SE inflation and missed share under dependent errors 0.17–0.46). mgcv’s term gives 0.93–0.94. For binary responses at G = 40 the term reaches 0.859 (0.889 with the cross term) from 0.787, and the λ-fixed refit covers only 0.704: the selected smoothing parameters jump between interior and boundary solutions (lambda-correction/summaries-ncvg/ncv-boundary.csv). On the counted plasmode datasets (G = all) the term raises β coverage from 0.71–0.91 to 0.76–0.93 per dataset (stress tests 0.59–0.71 → 0.66–0.75), median SE/SD 0.55–0.99 → 0.72–1.06 (Table 56); the smoothing bias of the NCV fit on these data remains (Section 8, Table 49). Computing times are in Section 14.
| estimand | dependence | G | CL2 | + mgcv J | + mgcv Vc − Vp | + IJ | + IJ × | + OS | + JK | jackknife refit | λ fixed |
|---|---|---|---|---|---|---|---|---|---|---|---|
| E(Y | X) | dependent | 40 | 0.929 | 0.932 | 0.933 | 0.938 | 0.944 | 0.940 | 0.946 | 0.967 | 0.941 |
| E(Y | X) | dependent | 100 | 0.943 | 0.944 | 0.945 | 0.947 | 0.953 | 0.948 | 0.951 | 0.963 | 0.949 |
| E(Y | X) | independent | 40 | 0.947 | 0.954 | 0.958 | 0.952 | 0.957 | 0.953 | 0.956 | 0.964 | 0.955 |
| E(Y | X) | independent | 100 | 0.957 | 0.960 | 0.962 | 0.958 | 0.961 | 0.958 | 0.958 | 0.964 | 0.961 |
| beta(s,t) | dependent | 40 | 0.929 | 0.935 | 0.936 | 0.945 | 0.956 | 0.949 | 0.954 | 0.977 | 0.945 |
| beta(s,t) | dependent | 100 | 0.946 | 0.950 | 0.951 | 0.958 | 0.968 | 0.959 | 0.963 | 0.977 | 0.955 |
| beta(s,t) | independent | 40 | 0.984 | 0.987 | 0.989 | 0.986 | 0.987 | 0.986 | 0.986 | 0.989 | 0.990 |
| beta(s,t) | independent | 100 | 0.987 | 0.989 | 0.990 | 0.988 | 0.989 | 0.988 | 0.988 | 0.990 | 0.988 |
| alpha(t) | dependent | 40 | 0.922 | 0.926 | 0.927 | 0.929 | 0.937 | 0.930 | 0.935 | 0.960 | 0.936 |
| alpha(t) | dependent | 100 | 0.941 | 0.943 | 0.944 | 0.945 | 0.949 | 0.945 | 0.946 | 0.957 | 0.947 |
| alpha(t) | independent | 40 | 0.920 | 0.933 | 0.936 | 0.931 | 0.937 | 0.932 | 0.936 | 0.946 | 0.925 |
| alpha(t) | independent | 100 | 0.944 | 0.950 | 0.951 | 0.947 | 0.953 | 0.947 | 0.947 | 0.955 | 0.950 |
| gamma(t) | dependent | 40 | 0.934 | 0.938 | 0.939 | 0.941 | 0.946 | 0.942 | 0.946 | 0.968 | 0.948 |
| gamma(t) | dependent | 100 | 0.938 | 0.941 | 0.941 | 0.942 | 0.946 | 0.942 | 0.943 | 0.955 | 0.946 |
| gamma(t) | independent | 40 | 0.930 | 0.941 | 0.945 | 0.937 | 0.944 | 0.938 | 0.944 | 0.953 | 0.944 |
| gamma(t) | independent | 100 | 0.943 | 0.948 | 0.951 | 0.945 | 0.950 | 0.945 | 0.945 | 0.954 | 0.952 |
| estimand | dependence | G | CL2 | + mgcv J | + mgcv Vc − Vp | + IJ | + IJ × | + OS | + JK | jackknife refit | λ fixed |
|---|---|---|---|---|---|---|---|---|---|---|---|
| E(Y | X) | dependent | 40 | 1.00 / 1.000 / 0.98 | 1.01 / 0.993 / 0.99 | 1.01 / 0.991 / 0.99 | 1.04 / 0.992 / 1.03 | 1.06 / 0.988 / 1.06 | 1.05 / 0.991 / 1.04 | 1.10 / 1.015 / 1.16 | 1.23 / 1.037 / 1.22 | 1.01 / 0.950 / 1.04 |
| E(Y | X) | dependent | 100 | 1.00 / 1.000 / 1.00 | 1.01 / 0.997 / 1.01 | 1.01 / 0.997 / 1.01 | 1.02 / 1.000 / 1.03 | 1.05 / 0.999 / 1.05 | 1.03 / 1.000 / 1.03 | 1.06 / 1.015 / 1.08 | 1.13 / 1.031 / 1.15 | 1.01 / 0.978 / 1.04 |
| E(Y | X) | independent | 40 | 1.00 / 1.000 / 1.11 | 1.02 / 0.991 / 1.14 | 1.04 / 0.990 / 1.16 | 1.02 / 0.992 / 1.13 | 1.04 / 0.995 / 1.16 | 1.02 / 0.992 / 1.13 | 1.10 / 1.047 / 1.43 | 1.10 / 1.011 / 1.22 | 0.99 / 0.952 / 1.18 |
| E(Y | X) | independent | 100 | 1.00 / 1.000 / 1.10 | 1.01 / 0.999 / 1.12 | 1.02 / 1.000 / 1.13 | 1.00 / 0.999 / 1.11 | 1.02 / 1.000 / 1.13 | 1.00 / 0.999 / 1.11 | 1.01 / 0.999 / 1.11 | 1.04 / 1.006 / 1.15 | 1.00 / 0.981 / 1.13 |
| beta(s,t) | dependent | 40 | 1.00 / 1.000 / 0.92 | 1.02 / 0.984 / 0.93 | 1.02 / 0.984 / 0.94 | 1.09 / 0.996 / 1.02 | 1.15 / 0.991 / 1.08 | 1.11 / 0.998 / 1.05 | 1.20 / 1.047 / 1.16 | 1.44 / 1.132 / 1.37 | 0.99 / 0.880 / 1.05 |
| beta(s,t) | dependent | 100 | 1.00 / 1.000 / 0.97 | 1.01 / 0.991 / 0.98 | 1.02 / 0.993 / 0.98 | 1.07 / 1.008 / 1.07 | 1.13 / 1.007 / 1.13 | 1.08 / 1.010 / 1.08 | 1.17 / 1.068 / 1.20 | 1.34 / 1.147 / 1.43 | 1.00 / 0.936 / 1.06 |
| beta(s,t) | independent | 40 | 1.00 / 1.000 / 1.39 | 1.02 / 1.006 / 1.41 | 1.05 / 1.024 / 1.45 | 1.01 / 1.001 / 1.40 | 1.02 / 1.009 / 1.42 | 1.01 / 1.001 / 1.40 | 1.08 / 1.063 / 1.71 | 1.06 / 1.037 / 1.48 | 1.00 / 0.961 / 1.46 |
| beta(s,t) | independent | 100 | 1.00 / 1.000 / 1.35 | 1.01 / 1.005 / 1.37 | 1.03 / 1.019 / 1.39 | 1.00 / 1.001 / 1.36 | 1.01 / 1.005 / 1.37 | 1.00 / 1.001 / 1.36 | 1.00 / 1.001 / 1.36 | 1.03 / 1.017 / 1.39 | 1.00 / 0.995 / 1.38 |
| dataset | G | CL2 | + mgcv J | + mgcv Vc − Vp | + IJ | + IJ × |
|---|---|---|---|---|---|---|
| ECG strain | all | 0.944 | 0.944 (1.00) | 0.945 (1.00) | 0.945 (1.00) | 0.946 (1.00) |
| ECG strain | 40 | 0.934 | 0.934 (1.00) | 0.934 (1.00) | 0.935 (1.00) | 0.938 (0.99) |
| AF trial | all | 0.938 | 0.941 (0.99) | 0.942 (0.99) | 0.950 (1.03) | 0.963 (1.02) |
| AF trial | 40 | 0.920 | 0.925 (1.00) | 0.926 (1.00) | 0.936 (1.04) | 0.950 (1.04) |
| running | all | 0.919 | 0.921 (1.00) | 0.921 (1.00) | 0.928 (1.02) | 0.939 (1.02) |
| running | 40 | 0.902 | 0.904 (1.00) | 0.905 (1.00) | 0.911 (1.00) | 0.922 (1.00) |
| DTI | all | 0.927 | 0.929 (0.99) | 0.929 (0.99) | 0.935 (0.99) | 0.943 (0.98) |
| DTI | 40 | 0.917 | 0.920 (0.99) | 0.921 (0.99) | 0.928 (0.98) | 0.938 (0.97) |
| gait | all | 0.939 | 0.940 (1.00) | 0.940 (1.00) | 0.942 (1.00) | 0.949 (1.00) |
| gait | 40 | 0.935 | 0.936 (1.00) | 0.936 (1.00) | 0.941 (1.01) | 0.949 (1.02) |
| ECG 8-lead | all | 0.942 | 0.944 (0.99) | 0.944 (1.00) | 0.956 (1.00) | 0.965 (1.01) |
| ECG 8-lead | 40 | 0.936 | 0.938 (1.00) | 0.939 (1.00) | 0.951 (1.02) | 0.959 (1.03) |
| ocean | all | 0.925 | 0.926 (1.00) | 0.926 (1.00) | 0.929 (1.02) | 0.938 (1.02) |
| ocean | 40 | 0.928 | 0.929 (1.00) | 0.930 (1.00) | 0.937 (1.00) | 0.947 (1.00) |
| weather | all | 0.882 | 0.884 (1.00) | 0.884 (1.00) | 0.892 (1.00) | 0.907 (1.00) |
| weather | 40 | 0.835 | 0.837 (1.00) | 0.838 (0.99) | 0.846 (1.02) | 0.860 (1.02) |
| electricity | all | 0.927 | 0.928 (1.00) | 0.929 (1.00) | 0.933 (1.01) | 0.944 (1.01) |
| electricity | 40 | 0.914 | 0.915 (1.00) | 0.915 (1.00) | 0.921 (1.01) | 0.930 (1.02) |
| family | G | CL2 | + mgcv J | + IJ | + IJ × | λ fixed |
|---|---|---|---|---|---|---|
| Gaussian | 40 | 0.923 (0.86) 1.00 / 0.80 | 0.925 (0.87) 1.01 / 0.80 | 0.948 (0.90) 1.13 / 0.83 | 0.957 (0.91) 1.19 / 0.86 | 0.949 (0.91) 1.00 / 1.07 |
| Gaussian | 100 | 0.934 (0.86) 1.00 / 0.86 | 0.936 (0.86) 1.01 / 0.87 | 0.951 (0.89) 1.10 / 0.89 | 0.958 (0.90) 1.14 / 0.91 | 0.950 (0.90) 1.00 / 1.07 |
| Poisson | 40 | 0.924 (0.88) 1.00 / 0.80 | 0.927 (0.88) 1.01 / 0.80 | 0.951 (0.91) 1.21 / 0.84 | 0.959 (0.92) 1.28 / 0.87 | 0.952 (0.92) 1.01 / 1.08 |
| Poisson | 100 | 0.935 (0.88) 1.00 / 0.84 | 0.938 (0.88) 1.01 / 0.85 | 0.953 (0.90) 1.09 / 0.87 | 0.961 (0.91) 1.14 / 0.89 | 0.953 (0.91) 1.00 / 1.09 |
| binary | 40 | 0.787 (0.61) 1.00 / 0.57 | 0.796 (0.62) 1.02 / 0.58 | 0.859 (0.72) 1.27 / 0.66 | 0.889 (0.77) 1.40 / 0.72 | 0.704 (0.23) 0.70 / 1.11 |
| binary | 100 | 0.899 (0.78) 1.00 / 0.78 | 0.905 (0.79) 1.02 / 0.79 | 0.933 (0.85) 1.16 / 0.83 | 0.948 (0.89) 1.26 / 0.89 | 0.949 (0.88) 0.96 / 1.14 |
| dataset | CL2 | + mgcv J | + IJ | + IJ × |
|---|---|---|---|---|
| ECG strain | 0.906 (0.80) 0.99 | 0.906 (0.80) 0.99 | 0.926 (0.84) 1.06 | 0.930 (0.85) 1.07 |
| AF trial | 0.810 (0.52) 0.80 | 0.812 (0.53) 0.80 | 0.879 (0.65) 0.99 | 0.895 (0.69) 1.05 |
| running | 0.712 (0.40) 0.65 | 0.738 (0.42) 0.70 | 0.759 (0.45) 0.72 | 0.777 (0.47) 0.76 |
| DTI | 0.847 (0.58) 0.91 | 0.961 (0.84) 1.62 | 0.876 (0.65) 0.98 | 0.888 (0.67) 1.02 |
| gait | 0.808 (0.53) 0.83 | 0.857 (0.64) 0.95 | 0.867 (0.65) 0.93 | 0.873 (0.65) 0.95 |
| ECG 8-lead | 0.763 (0.48) 0.55 | 0.763 (0.48) 0.55 | 0.854 (0.62) 0.74 | 0.868 (0.64) 0.78 |
| ocean | 0.712 (0.29) 0.74 | 0.712 (0.29) 0.74 | 0.752 (0.35) 0.81 | 0.769 (0.36) 0.86 |
| weather | 0.592 (0.39) 0.62 | 0.594 (0.39) 0.62 | 0.658 (0.48) 0.77 | 0.685 (0.52) 0.85 |
| electricity | 0.619 (0.40) 0.34 | 0.619 (0.40) 0.34 | 0.675 (0.45) 0.44 | 0.696 (0.47) 0.49 |
10 Plasmode study
10.1 Datasets and models
Nine datasets, each fitted with the study’s model Y(t) ~ α(t) + ∫X(s)β(s,t)ds + z γ(t) (Gaussian, identity link; ff basis 8 × 10, intercept and γ(t) bases k = 12, except intercept k = 40 and γ k = 30 for ECG 8-lead because with k = 12 neither the functional intercept nor the LBBB effect can follow the QRS complex (22% of the REML residual sum of squares was mean-curve misfit; 0.2% with the larger bases), so that mean structure does not leak into the frozen truth’s residuals). Six datasets with independent curves are counted in the consistency criterion; three whose curves are serially or spatially dependent units are stress tests. The cardiology datasets are patient or volunteer data of N. Bouchahda’s group: they are described here in aggregate only and never enter the repository. Table 57 lists the data, Table 58 the fitted models and their frozen fits (EDF per term in Table 85), Table 59 the residual dependence and covariate diagnostics, Table 60 the block construction of the stress tests; the test-cohort balance is tabulated at the end of the section on the construction of the plasmode cells.
ECG strain (ECG-to-strain study (consulting project with N. Bouchahda’s group, 2025–26); first beat per subject; local only (patient and volunteer data; de-identified arrays on LRZ, aggregates here)). Counted dataset: 78 curves; response: global longitudinal strain, 4-chamber view (%), one cardiac cycle normalised to [0, 1], native grid about 52 frames per beat (29–83) (fitted grid: interpolated to 61 points); functional covariate: simultaneously recorded ECG of the same beat (digitised pixel intensity, shape usable, amplitude up to scale), 51 points; scalar covariate z: age (standardised). Preprocessing: cycle time normalised to the R–R interval; subjects with at least two beats (the second beat supplies the beat-to-beat difference curves). Fit: R² 0.47, ff EDF 59.6 (REML) vs 44.0 (NCV); residual design effect 19.9 (DE/D 0.33), lag-1 correlation 0.96, 15% negative correlations (minimum r -0.35), SD ratio along t 25, 4 FPCs for 90% of the residual variance, covariate rank 15, phase share 63%. Why it is here: real cardiac misregistration (largest phase share), strongest dependence among the cardiology sets with sign-changing correlations and near-zero variance at both ends of the cycle; the only model-free error source.
AF trial (atrial-fibrillation trial (N. Bouchahda’s group): patients randomised to digoxin or beta-blocker; local only (patient data; de-identified arrays on LRZ, aggregates here)). Counted dataset: 118 curves; response: left-atrial strain, 4-chamber view, at day 30 (%), one cycle on [0, 1], native grid 30 points per cycle (fitted grid: used as is); functional covariate: the same strain curve at day 0, 51 points; scalar covariate z: randomised treatment (digoxin = 1). Preprocessing: none beyond the cycle normalisation; a baseline-adjusted functional ANCOVA. Fit: R² 0.19, ff EDF 33.2 (REML) vs 25.3 (NCV); residual design effect 10.9 (DE/D 0.36), lag-1 correlation 0.89, 20% negative correlations (minimum r -0.27), SD ratio along t 30, 3 FPCs for 90% of the residual variance, covariate rank 8, phase share 43%. Why it is here: a randomised binary covariate, so γ(t) is a model-defined target informed by a trial; the coarsest grid; large phase variability with a broad, noisy peak.
running (Fukuchi, Fukuchi & Duarte 2017 (PeerJ 5:e3298) via tidyfundata::running; one row per person, first condition; public (tidyfundata, MIT; original data public)). Counted dataset: 90 curves; response: sagittal knee moment over normalised stance, native grid 101 points (fitted grid: interpolated to 61 points (as in the original 16 cells)); functional covariate: sagittal knee angle over the same stance, 51 points; scalar covariate z: body mass (standardised). Preprocessing: one row per person (post-intervention rows dropped); speed and backpack conditions differ between persons and stay in the residuals. Fit: R² 0.46, ff EDF 53.7 (REML) vs 54.8 (NCV); residual design effect 33.3 (DE/D 0.55), lag-1 correlation 0.99, 12% negative correlations (minimum r -0.10), SD ratio along t 5, 2 FPCs for 90% of the residual variance, covariate rank 7, phase share 30%. Why it is here: the lowest-rank functional covariate together with gait, very smooth residuals (strongest dependence of the counted datasets).
DTI (refund::DTI, baseline visit (Goldsmith et al. 2011): multiple-sclerosis patients and controls; public (refund, GPL ≥ 2)). Counted dataset: 92 curves; response: fractional anisotropy along the right corticospinal tract, native grid 55 points (fitted grid: used as is); functional covariate: fractional anisotropy along the corpus callosum, 51 points; scalar covariate z: case status (MS = 1). Preprocessing: complete cases of both tracts. Fit: R² 0.21, ff EDF 53.8 (REML) vs 35.1 (NCV); residual design effect 9.9 (DE/D 0.18), lag-1 correlation 0.81, 19% negative correlations (minimum r -0.34), SD ratio along t 2, 16 FPCs for 90% of the residual variance, covariate rank 24, phase share 22%. Why it is here: the study’s application and the source of the synthetic smooth error process; no phase variation (registration control), richest covariate rank.
gait (Van Criekinge et al. 2023 (Sci Data 10:852; figshare 24192489), able-bodied adults walking barefoot at preferred speed; public (CC0)). Counted dataset: 138 curves; response: sagittal knee moment over one stride, one mean stride per subject, native grid 1001 points (0.1% of the stride) (fitted grid: every 10th point (101)); functional covariate: sagittal knee angle over the stride, 51 points; scalar covariate z: age (standardised; from the subject-characteristics file). Preprocessing: the authors’ stride normalisation and averaging over strides (within-subject stride variability is averaged out). Fit: R² 0.40, ff EDF 68.7 (REML) vs 60.8 (NCV); residual design effect 37.0 (DE/D 0.37), lag-1 correlation 0.98, 33% negative correlations (minimum r -0.54), SD ratio along t 6, 6 FPCs for 90% of the residual variance, covariate rank 8, phase share 3%. Why it is here: the largest G; the running data’s model on an independent cohort with a walking task; a third of the residual correlations negative; no phase variation.
ECG 8-lead (roahd::mfD_healthy and mfD_LBBB (Ieva et al.), 50 healthy and 50 left-bundle-branch-block subjects; public (roahd, GPL-3)). Counted dataset: 100 curves; response: ECG lead V5 (µV) over one beat, native grid 1024 points at 1 kHz (fitted grid: interpolated to 101 points); functional covariate: ECG lead I (presumed from the documented lead order), 51 points; scalar covariate z: LBBB indicator. Preprocessing: registered and smoothed by the package authors; lead identities resolved from the R-wave progression. Fit: R² 0.21, ff EDF 39.4 (REML) vs 11.2 (NCV); residual design effect 6.2 (DE/D 0.06), lag-1 correlation 0.94, 32% negative correlations (minimum r -0.68), SD ratio along t 42, 4 FPCs for 90% of the residual variance, covariate rank 15, phase share 33%. Why it is here: the strongest sign-changing residual correlation and the most extreme variance profile along the beat; residual phase (QRS width, LBBB morphology) survives the registration; larger intercept and γ bases.
ocean (Hawaii Ocean Time-series profiles (FRegSigCom::ocean), monthly casts; public (FRegSigCom, GPL ≥ 2; HOT data public)). Stress test: 116 curves; response: dissolved oxygen over depth, native grid 101 points (fitted grid: used as is); functional covariate: temperature over depth, 51 points; scalar covariate z: cast order (standardised). Preprocessing: none; casts kept in time order. Fit: R² 0.36, ff EDF 59.2 (REML) vs 49.8 (NCV); residual design effect 39.7 (DE/D 0.39), lag-1 correlation 0.99, 22% negative correlations (minimum r -0.27), SD ratio along t 3, 5 FPCs for 90% of the residual variance, covariate rank 6, phase share 0%; 39 blocks of 2–3 curves, residual-score correlation 0.04 within and -0.10 between blocks. Why it is here: stress test: consecutive monthly casts (no residual-score autocorrelation found, so the block arm is a pure cost check); very smooth residuals.
weather (AEMET open data via fda.usc::aemet, 73 Spanish stations, daily means 1980–2009; public (fda.usc, GPL-2)). Stress test: 73 curves; response: log precipitation over the year, native grid 365 days (fitted grid: every 5th day (73)); functional covariate: daily mean temperature over the year, 51 points; scalar covariate z: altitude (standardised). Preprocessing: none beyond the day thinning. Fit: R² 0.41, ff EDF 29.5 (REML) vs 38.6 (NCV); residual design effect 29.1 (DE/D 0.40), lag-1 correlation 0.50, 1% negative correlations (minimum r -0.10), SD ratio along t 2, 19 FPCs for 90% of the residual variance, covariate rank 4, phase share 6%; 16 blocks of 1–8 curves, residual-score correlation 0.55 within and 0.36 between blocks. Why it is here: stress test: spatially correlated stations (residual scores correlated up to about 250 km); the lowest covariate rank of all datasets; rough residuals with almost no negative correlations; NCV rougher than REML.
electricity (Adelaide electricity demand (fds::mondaydemand, fds::mondaytempairport; Magnano, Boland & Hyndman 2008), every 5th Monday 1997–2007; public (fds, GPL-3)). Stress test: 102 curves; response: half-hourly electricity demand over the day, native grid 48 half-hours (fitted grid: used as is); functional covariate: airport temperature over the same day, 51 points; scalar covariate z: week index (standardised; absorbs the linear trend). Preprocessing: every 5th Monday (thins, not removes, the week-to-week dependence; the annual cycle stays). Fit: R² 0.31, ff EDF 33.2 (REML) vs 10.0 (NCV); residual design effect 35.8 (DE/D 0.75), lag-1 correlation 0.98, 0% negative correlations (minimum r 0.31), SD ratio along t 3, 2 FPCs for 90% of the residual variance, covariate rank 11, phase share 0%; 51 blocks of 2 curves, residual-score correlation 0.21 within and 0.01 between blocks. Why it is here: stress test: a weekly time series with dependence to about five weeks; strongly positively correlated residuals (2 FPCs for 90%), the largest REML–NCV EDF gap.
| dataset | role | access | G | native | fitted D | grid used | z |
|---|---|---|---|---|---|---|---|
| ECG strain | counted | local only | 78 | about 52 frames per beat (29–83) | 61 | interpolated to 61 points | age (std.) |
| AF trial | counted | local only | 118 | 30 points per cycle | 30 | used as is | randomised treatment (digoxin = 1) |
| running | counted | public | 90 | 101 points | 61 | interpolated to 61 points (as in the original 16 cells) | body mass (std.) |
| DTI | counted | public | 92 | 55 points | 55 | used as is | case status (MS = 1) |
| gait | counted | public | 138 | 1001 points (0.1% of the stride) | 101 | every 10th point (101) | age (std.) |
| ECG 8-lead | counted | public | 100 | 1024 points at 1 kHz | 101 | interpolated to 101 points | LBBB indicator |
| ocean | stress | public | 116 | 101 points | 101 | used as is | cast order (std.) |
| weather | stress | public | 73 | 365 days | 73 | every 5th day (73) | altitude (std.) |
| electricity | stress | public | 102 | 48 half-hours | 48 | used as is | week index (std.) |
| dataset | intercept basis k | gamma(t) basis k | ff basis (s × t) | R² | ff EDF REML / NCV / MID | total EDF REML / NCV / MID | MID interior |
|---|---|---|---|---|---|---|---|
| ECG strain | 12 | 12 | 8 × 10 | 0.47 | 59.6 / 44.0 / 52.9 | 77.1 / 54.1 / 63.4 | FALSE |
| AF trial | 12 | 12 | 8 × 10 | 0.19 | 33.2 / 25.3 / 29.5 | 44.7 / 32.8 / 38.9 | TRUE |
| running | 12 | 12 | 8 × 10 | 0.46 | 53.7 / 54.8 / 60.4 | 68.0 / 65.9 / 73.1 | FALSE |
| DTI | 12 | 12 | 8 × 10 | 0.21 | 53.8 / 35.1 / 47.0 | 68.5 / 47.9 / 60.7 | TRUE |
| gait | 12 | 12 | 8 × 10 | 0.40 | 68.7 / 60.8 / 65.3 | 89.0 / 80.1 / 85.2 | FALSE |
| ECG 8-lead | 40 | 30 | 8 × 10 | 0.21 | 39.4 / 11.2 / 23.9 | 103.2 / 77.0 / 88.9 | TRUE |
| ocean | 12 | 12 | 8 × 10 | 0.36 | 59.2 / 49.8 / 60.3 | 74.3 / 58.7 / 72.1 | FALSE |
| weather | 12 | 12 | 8 × 10 | 0.41 | 29.5 / 38.6 / 41.9 | 45.4 / 60.8 / 61.5 | FALSE |
| electricity | 12 | 12 | 8 × 10 | 0.31 | 33.2 / 10.0 / 23.9 | 45.8 / 22.0 / 35.7 | FALSE |
| dataset | DE | DE/D | lag 1 step | lag 10% | lag 25% | neg. corr. share | min r | sd ratio | FPC90 | rank X | phase share |
|---|---|---|---|---|---|---|---|---|---|---|---|
| ECG strain | 19.88 | 0.33 | 0.96 | 0.62 | 0.21 | 0.15 | -0.35 | 24.53 | 4 | 15 | 0.63 |
| AF trial | 10.93 | 0.36 | 0.89 | 0.68 | 0.36 | 0.20 | -0.27 | 29.78 | 3 | 8 | 0.43 |
| running | 33.32 | 0.55 | 0.99 | 0.87 | 0.59 | 0.12 | -0.10 | 4.63 | 2 | 7 | 0.30 |
| DTI | 9.91 | 0.18 | 0.81 | 0.23 | 0.11 | 0.19 | -0.34 | 2.31 | 16 | 24 | 0.22 |
| gait | 36.96 | 0.37 | 0.98 | 0.48 | 0.26 | 0.33 | -0.54 | 6.15 | 6 | 8 | 0.03 |
| ECG 8-lead | 6.19 | 0.06 | 0.94 | 0.36 | 0.08 | 0.32 | -0.68 | 42.26 | 4 | 15 | 0.33 |
| ocean | 39.74 | 0.39 | 0.99 | 0.74 | 0.43 | 0.22 | -0.27 | 2.86 | 5 | 6 | 0.00 |
| weather | 29.12 | 0.40 | 0.50 | 0.49 | 0.39 | 0.01 | -0.10 | 2.22 | 19 | 4 | 0.06 |
| electricity | 35.77 | 0.75 | 0.98 | 0.89 | 0.82 | 0.00 | 0.31 | 3.07 | 2 | 11 | 0.00 |
| dataset | block rule | blocks | block size | alt. blocks | score corr. within | score corr. between | between pairs |
|---|---|---|---|---|---|---|---|
| ocean | 3 consecutive monthly casts (cost check; the autocorrelation rule gives single casts) | 39 | 2–3 | 0.04 | -0.10 | 38 | |
| weather | complete-linkage clusters of station distance cut at 250 km (mismatch arm: 150 km) | 16 | 1–8 | 25 | 0.55 | 0.36 | 276 |
| electricity | 2 consecutive frame curves (10 weeks) | 51 | 2 | 0.21 | 0.01 | 50 |
MID is a third smoothing level, not a midpoint in fit: its ff EDF lies strictly between the two fits for every term on 3 of 9 datasets (AF trial, DTI, ECG 8-lead) and exceeds both fits’ on running, ocean, weather.
10.2 Construction of the plasmode cells
Each dataset’s model is fitted to the real data by REML and by curve-blocked NCV; each fit’s fitted means are frozen as a truth, and the MID truth refits with every smoothing parameter fixed at the log-midpoint of the REML and NCV values. Replicates add each subject’s real residual curve (from the REML or the NCV fit) with a random sign, attached to its own covariates; the ECG strain data add beat-to-beat difference curves as a model-free error source at two scales. Cells cross truth source × residual source × G (all subjects or 40); the three stress tests add block-flip cells with an imposed within-block dependence and a block-NCV arm. 130 cells, 26000 tasks, NCV converged in 99.9% of them. The mean estimand is the conditional mean of a fixed test cohort of 50 subjects (a seeded sample on 7 datasets, the first 50 rows on 2; the choice was made by a permutation test of the cohort’s balance in the scalar covariate and the first three covariate scores; table below), evaluated at the native grid points and scored conditional on that cohort. Nothing per subject leaves the analysis: all tables are aggregates.
Replicate data sets are \(Y^*_i(t) = \mu_i(t) + w_i\,e_{\pi(i)}(t)\): \(\mu_i\) is subject i’s fitted mean under the frozen truth, \(e_j\) the residual curve of subject j from the REML or the NCV fit to the real data (or a beat-to-beat difference curve), \(w_i = \pm 1\) an independent random sign per curve (curve flips) or one sign per block of curves (block flips, stress tests only, all subjects), and \(\pi\) the identity (attached: each subject keeps its own residual curve) or a random permutation (detached: residual curves reassigned across subjects; running data only, Table 61). At G = 40 every replicate draws its own random subset of 40 subjects. Each cell has 200 replicates; the replicate’s random draws (signs, subset, permutation) depend only on the replicate number, so cells of a dataset are paired. Every replicate is fitted and scored exactly like a synthetic data set (same bases except ECG 8-lead, same arms).
| truth | residual | G | estimand | arm | attached | detached | difference |
|---|---|---|---|---|---|---|---|
| NCV | NCV | 90 | E(Y | X) | NCV + CL2 | 0.919 | 0.917 | -0.001 |
| NCV | NCV | 90 | E(Y | X) | NCV + CL2, bias-aware | 0.943 | 0.947 | 0.004 |
| NCV | NCV | 90 | E(Y | X) | REML + CL2 | 0.930 | 0.936 | 0.006 |
| NCV | NCV | 90 | E(Y | X) | REML, model-based | 0.565 | 0.577 | 0.013 |
| NCV | NCV | 90 | beta(s,t) | NCV + CL2 | 0.875 | 0.844 | -0.031 |
| NCV | NCV | 90 | beta(s,t) | NCV + CL2, bias-aware | 0.948 | 0.947 | -0.001 |
| NCV | NCV | 90 | beta(s,t) | REML + CL2 | 0.922 | 0.926 | 0.004 |
| NCV | NCV | 90 | beta(s,t) | REML, model-based | 0.528 | 0.506 | -0.021 |
| NCV | NCV | 40 | E(Y | X) | NCV + CL2 | 0.885 | 0.888 | 0.003 |
| NCV | NCV | 40 | E(Y | X) | NCV + CL2, bias-aware | 0.937 | 0.941 | 0.004 |
| NCV | NCV | 40 | E(Y | X) | REML + CL2 | 0.921 | 0.926 | 0.005 |
| NCV | NCV | 40 | E(Y | X) | REML, model-based | 0.522 | 0.533 | 0.010 |
| NCV | NCV | 40 | beta(s,t) | NCV + CL2 | 0.760 | 0.747 | -0.013 |
| NCV | NCV | 40 | beta(s,t) | NCV + CL2, bias-aware | 0.927 | 0.929 | 0.003 |
| NCV | NCV | 40 | beta(s,t) | REML + CL2 | 0.910 | 0.926 | 0.016 |
| NCV | NCV | 40 | beta(s,t) | REML, model-based | 0.463 | 0.453 | -0.010 |
| REML | REML | 90 | E(Y | X) | NCV + CL2 | 0.861 | 0.841 | -0.020 |
| REML | REML | 90 | E(Y | X) | NCV + CL2, bias-aware | 0.925 | 0.932 | 0.007 |
| REML | REML | 90 | E(Y | X) | REML + CL2 | 0.935 | 0.939 | 0.004 |
| REML | REML | 90 | E(Y | X) | REML, model-based | 0.570 | 0.582 | 0.012 |
| REML | REML | 90 | beta(s,t) | NCV + CL2 | 0.586 | 0.505 | -0.081 |
| REML | REML | 90 | beta(s,t) | NCV + CL2, bias-aware | 0.841 | 0.850 | 0.008 |
| REML | REML | 90 | beta(s,t) | REML + CL2 | 0.916 | 0.919 | 0.003 |
| REML | REML | 90 | beta(s,t) | REML, model-based | 0.481 | 0.468 | -0.013 |
| REML | REML | 40 | E(Y | X) | NCV + CL2 | 0.837 | 0.839 | 0.001 |
| REML | REML | 40 | E(Y | X) | NCV + CL2, bias-aware | 0.925 | 0.929 | 0.004 |
| REML | REML | 40 | E(Y | X) | REML + CL2 | 0.921 | 0.928 | 0.007 |
| REML | REML | 40 | E(Y | X) | REML, model-based | 0.522 | 0.536 | 0.014 |
| REML | REML | 40 | beta(s,t) | NCV + CL2 | 0.427 | 0.412 | -0.015 |
| REML | REML | 40 | beta(s,t) | NCV + CL2, bias-aware | 0.829 | 0.836 | 0.008 |
| REML | REML | 40 | beta(s,t) | REML + CL2 | 0.887 | 0.909 | 0.022 |
| REML | REML | 40 | beta(s,t) | REML, model-based | 0.421 | 0.422 | 0.001 |
| dataset | G | cohort | one-sided at 5% | permutation p | SMD z | SMD PC1 | SMD PC2 | SMD PC3 |
|---|---|---|---|---|---|---|---|---|
| ECG strain | 78 | first 50 rows | FALSE | 0.302 | 0.313 | 0.021 | -0.180 | -0.402 |
| AF trial | 118 | seeded sample of 50 | TRUE | 0.000 | 1.788 | 0.046 | -0.231 | 0.317 |
| running | 90 | seeded sample of 50 | TRUE | 0.000 | -0.331 | -0.398 | 0.916 | -0.008 |
| DTI | 92 | seeded sample of 50 | TRUE | 0.000 | -1.149 | -0.408 | -0.258 | -0.036 |
| gait | 138 | seeded sample of 50 | TRUE | 0.000 | 1.668 | 0.118 | 0.299 | 0.377 |
| ECG 8-lead | 100 | seeded sample of 50 | TRUE | 0.000 | -1.990 | -1.344 | 0.235 | 0.274 |
| ocean | 116 | seeded sample of 50 | TRUE | 0.000 | -1.725 | -0.255 | -0.346 | 0.109 |
| weather | 73 | first 50 rows | FALSE | 0.411 | -0.173 | 0.130 | 0.225 | 0.378 |
| electricity | 102 | seeded sample of 50 | TRUE | 0.000 | -1.724 | 0.029 | -0.091 | -0.297 |
10.3 Coverage per arm, truth source, residual source and G
Across the pooled cells of the six counted datasets (Figure 17, Table 63) model-based intervals cover the mean at 0.54–0.75 and β at 0.51–0.67; REML + CL2 covers them at 0.93–0.95 and 0.92–0.94. The bias-aware NCV interval covers the mean at 0.93–0.96 and β at 0.89–0.94, NCV + CL2 alone 0.73–0.91 for β. On the three stress tests every arm is lower: REML + CL2 0.88–0.93 for β and 0.92–0.93 for the mean; CL2 improves on the model-based intervals on every dataset but is not calibrated on the stress tests. Their curve-flip replicates have independent errors across curves, so these shortfalls are properties of the datasets’ covariates and residual shapes, not of between-curve dependence (Section 10.9 isolates that). REML + CL2’s coverage is nearly flat across truth source and residual source (largest range within a counted dataset: 1.2 pp; within a stress test up to 7.3 pp on the weather data), whereas the bias-aware NCV interval’s β coverage moves with the truth source: on the counted datasets 0.84–0.93 under the REML-fitted truth, 0.89–0.94 under MID and 0.93–0.97 under the NCV-fitted truth.
| app | role | estimand | NCV + CL2 | NCV + CL2, bias-aware | REML + CL2 | REML, model-based | AR(1) working model |
|---|---|---|---|---|---|---|---|
| ECG strain | counted | beta(s,t) | 0.912 | 0.938 | 0.944 | 0.545 | 0.874 |
| ECG strain | counted | E(Y | X) | 0.921 | 0.943 | 0.944 | 0.576 | 0.869 |
| AF trial | counted | beta(s,t) | 0.789 | 0.916 | 0.938 | 0.666 | 0.838 |
| AF trial | counted | E(Y | X) | 0.897 | 0.936 | 0.946 | 0.680 | 0.870 |
| running | counted | beta(s,t) | 0.734 | 0.892 | 0.919 | 0.507 | 0.728 |
| running | counted | E(Y | X) | 0.896 | 0.934 | 0.933 | 0.569 | 0.787 |
| DTI | counted | beta(s,t) | 0.846 | 0.926 | 0.927 | 0.632 | 0.858 |
| DTI | counted | E(Y | X) | 0.907 | 0.938 | 0.932 | 0.646 | 0.902 |
| gait | counted | beta(s,t) | 0.816 | 0.925 | 0.939 | 0.517 | 0.914 |
| gait | counted | E(Y | X) | 0.922 | 0.944 | 0.943 | 0.538 | 0.927 |
| ECG 8-lead | counted | beta(s,t) | 0.739 | 0.937 | 0.942 | 0.638 | 0.720 |
| ECG 8-lead | counted | E(Y | X) | 0.896 | 0.957 | 0.952 | 0.753 | 0.924 |
| ocean | stress test | beta(s,t) | 0.708 | 0.916 | 0.925 | 0.412 | 0.854 |
| ocean | stress test | E(Y | X) | 0.902 | 0.939 | 0.935 | 0.456 | 0.932 |
| weather | stress test | beta(s,t) | 0.608 | 0.828 | 0.882 | 0.341 | 0.438 |
| weather | stress test | E(Y | X) | 0.869 | 0.920 | 0.924 | 0.540 | 0.675 |
| electricity | stress test | beta(s,t) | 0.611 | 0.911 | 0.927 | 0.432 | 0.677 |
| electricity | stress test | E(Y | X) | 0.881 | 0.947 | 0.927 | 0.498 | 0.844 |
10.3.1 Lower tails, by truth source
REML + CL2’s 5% pointwise quantile is 0.87–0.90 for β and 0.89–0.91 for the mean on the counted datasets (mean over pool cells; worst cell 0.84–0.90 for β); the bias-aware NCV interval’s is 0.77–0.86 for β and NCV + CL2’s 0.44–0.81 (Table 86). REML + CL2’s own tail depends on the truth source (Table 64): under the REML-fitted truth its β quantile on the counted datasets is 0.85–0.90 (stress tests 0.83–0.89), under the NCV-fitted truth 0.88–0.90; on the running data under the REML truth 30% of β’s grid points have pointwise coverage below 0.90 (DTI 19%).
| app | role | cov REML truth | cov MID | cov NCV truth | q05 REML truth | q05 MID | q05 NCV truth |
|---|---|---|---|---|---|---|---|
| ECG strain | counted | 0.943 | 0.943 | 0.944 | 0.901 | 0.900 | 0.903 |
| AF trial | counted | 0.933 | 0.938 | 0.942 | 0.873 | 0.878 | 0.885 |
| running | counted | 0.915 | 0.921 | 0.922 | 0.860 | 0.880 | 0.880 |
| DTI | counted | 0.920 | 0.930 | 0.931 | 0.847 | 0.889 | 0.891 |
| gait | counted | 0.939 | 0.940 | 0.940 | 0.903 | 0.905 | 0.900 |
| ECG 8-lead | counted | 0.938 | 0.944 | 0.944 | 0.870 | 0.873 | 0.876 |
| ocean | stress test | 0.927 | 0.924 | 0.924 | 0.888 | 0.890 | 0.883 |
| weather | stress test | 0.890 | 0.909 | 0.847 | 0.830 | 0.860 | 0.808 |
| electricity | stress test | 0.909 | 0.937 | 0.936 | 0.845 | 0.897 | 0.901 |
The pooled studentised error of the REML fit (descriptive, see the synthetic section for its limits) reads differently on the two groups of datasets (Table 95, means over the two model-residual sources). On the counted datasets the SD of z for β at G = all is 1.01–1.14 with 5.6–8.5% of |z| > 1.96 (mean of z -0.00 to 0.03): a mild scale error, consistent with the 1–3 pp shortfall. On the stress tests the SD is 1.07–2.86 (per-cell maximum 3.05 on weather) with only 6.3–15.3% of |z| > 1.96, where a normal pivot with SD 2.86 would give 49%: a scale mixture, i.e. SE failure concentrated on a subset of grid points or replicates rather than a uniform underestimation. With the model-based SE the SD of z is 2.20–3.95 on the counted datasets. Under the REML-fitted truth the SD is 1.02–1.14 against 1.01–1.12 under the NCV-fitted truth (counted datasets). A region-wise view is not available from the stored aggregates.
10.3.2 G = 40
With 40 subjects instead of all, REML + CL2 loses 0.4–1.7 pp on the counted datasets (paired over truth × residual cells; Table 87), the bias-aware NCV interval 0.2–3.1 pp and NCV + CL2 alone 1.9–18.6 pp. The rule’s point decisions on the G = 40 cells (no bootstrap flags; Table 88) are ECG strain P, AF trial N, running N, DTI N, gait F, ECG 8-lead F, ocean N, weather N, electricity N for β and ECG strain E, AF trial F, running P, DTI N, gait E, ECG 8-lead P, ocean E, weather N, electricity N for the mean.
10.4 Estimation by truth source
The NCV/REML MSE ratio depends on which fit supplied the truth (Figure 18, Table 65). For β it is 0.04–0.78 under the NCV-fitted truth, 0.05–0.90 under MID and 0.32–2.24 under the REML-fitted truth; NCV loses under the REML truth on 4 datasets (electricity, DTI, ocean, running), and the ordering NCV truth < MID < REML truth holds on 9 of 9. For the mean the ratios are 0.52–0.93, 0.58–0.99 and 0.86–1.19. The gains are not confined to the surface in the plasmode study: for γ(t) the ratio is 0.28–1.05 under the NCV truth and 0.88–1.30 under the REML truth, for α(t) 0.38–1.11 and 0.95–1.20, with resolved univariate gains (bootstrap interval below 0.9, NCV truth, REML residuals) on ECG strain gamma(t) 0.60 [0.56, 0.64]; DTI gamma(t) 0.66 [0.61, 0.70]; ECG 8-lead alpha(t) 0.63 [0.57, 0.68]; weather alpha(t) 0.42 [0.36, 0.47]; weather gamma(t) 0.34 [0.28, 0.40]; electricity alpha(t) 0.73 [0.69, 0.77]. Changing the truth source changes the truth’s shape and signal as well as its smoothness, and an NCV-derived truth aligns the target with the NCV fit, so this is a truth-source sensitivity and not evidence for a pure smoothness mechanism: on the weather data NCV is rougher than REML (ff EDF 38.6 vs 29.5) yet wins for β under all three truths (REML truth 0.67), and on the running data the two fits have the same ff EDF (ratio 0.98) yet the MID-truth ratio is 0.53.
| role | app | estimand | REML | MID | NCV |
|---|---|---|---|---|---|
| counted | ECG strain | alpha(t) | 1.071 | 1.029 | 0.980 |
| counted | AF trial | alpha(t) | 1.035 | 1.013 | 0.977 |
| counted | running | alpha(t) | 1.201 | 1.160 | 1.108 |
| counted | DTI | alpha(t) | 0.984 | 0.921 | 0.881 |
| counted | gait | alpha(t) | 0.952 | 0.938 | 0.919 |
| counted | ECG 8-lead | alpha(t) | 0.977 | 0.751 | 0.625 |
| stress test | ocean | alpha(t) | 1.174 | 1.102 | 1.026 |
| stress test | weather | alpha(t) | 0.971 | 0.656 | 0.383 |
| stress test | electricity | alpha(t) | 0.955 | 0.766 | 0.736 |
| counted | ECG strain | beta(s,t) | 0.952 | 0.904 | 0.780 |
| counted | AF trial | beta(s,t) | 0.321 | 0.268 | 0.233 |
| counted | running | beta(s,t) | 1.523 | 0.534 | 0.241 |
| counted | DTI | beta(s,t) | 1.167 | 0.573 | 0.396 |
| counted | gait | beta(s,t) | 0.625 | 0.552 | 0.427 |
| counted | ECG 8-lead | beta(s,t) | 0.317 | 0.179 | 0.113 |
| stress test | ocean | beta(s,t) | 1.806 | 0.347 | 0.068 |
| stress test | weather | beta(s,t) | 0.672 | 0.440 | 0.333 |
| stress test | electricity | beta(s,t) | 2.237 | 0.052 | 0.039 |
| counted | ECG strain | gamma(t) | 1.184 | 0.634 | 0.628 |
| counted | AF trial | gamma(t) | 1.004 | 0.968 | 0.919 |
| counted | running | gamma(t) | 1.106 | 1.088 | 1.046 |
| counted | DTI | gamma(t) | 0.883 | 0.746 | 0.662 |
| counted | gait | gamma(t) | 1.028 | 1.018 | 1.017 |
| counted | ECG 8-lead | gamma(t) | 1.042 | 1.046 | 1.016 |
| stress test | ocean | gamma(t) | 1.290 | 1.082 | 0.941 |
| stress test | weather | gamma(t) | 1.301 | 0.673 | 0.284 |
| stress test | electricity | gamma(t) | 0.972 | 0.979 | 0.983 |
| counted | ECG strain | E(Y | X) | 1.016 | 0.966 | 0.907 |
| counted | AF trial | E(Y | X) | 0.964 | 0.943 | 0.910 |
| counted | running | E(Y | X) | 1.154 | 0.988 | 0.904 |
| counted | DTI | E(Y | X) | 1.002 | 0.868 | 0.748 |
| counted | gait | E(Y | X) | 0.991 | 0.971 | 0.928 |
| counted | ECG 8-lead | E(Y | X) | 0.865 | 0.767 | 0.681 |
| stress test | ocean | E(Y | X) | 1.136 | 0.884 | 0.796 |
| stress test | weather | E(Y | X) | 1.084 | 0.970 | 0.833 |
| stress test | electricity | E(Y | X) | 1.188 | 0.583 | 0.517 |
10.4.1 Relative error and informativeness of the intervals
As in the synthetic study (Section 3.3.4; summaries/plasmode/relative-error.csv, computed from the task records, aggregates only). Cells with all subjects and curve flips; values per dataset are medians over its truth × residual cells (relative error, half-width, detection) or means (coverage).
On the real-data truths β is weakly determined for both fits: the median relative error of the REML estimate is 0.63–3.42 across the datasets, of the NCV estimate 0.47–0.81 (Table 66). REML + CL2 covers at 0.88–0.94 with half-widths of 1.17–5.72 times the size of the effect and detects 0.08–0.64 of the clearly non-zero grid points; NCV + CL2 detects 0.46–0.77 but covers at 0.61–0.91. For the mean the relative errors are 0.08–0.29 (REML) and 0.08–0.29 (NCV); for γ(t) 0.17–0.78 and 0.16–0.82.
| role | app | cov NCV+CL2 | cov bias-aware | cov REML+CL2 | rel NCV+CL2 | rel REML+CL2 | hw NCV+CL2 | hw bias-aware | hw REML+CL2 | det NCV+CL2 | det bias-aware | det REML+CL2 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| counted | ECG strain | 0.91 | 0.94 | 0.94 | 0.68 | 0.70 | 1.18 | 1.30 | 1.42 | 0.46 | 0.40 | 0.44 |
| counted | AF trial | 0.79 | 0.92 | 0.94 | 0.71 | 0.87 | 0.96 | 1.53 | 1.82 | 0.53 | 0.30 | 0.37 |
| counted | running | 0.73 | 0.89 | 0.92 | 0.66 | 0.82 | 0.68 | 1.52 | 1.85 | 0.73 | 0.58 | 0.64 |
| counted | DTI | 0.85 | 0.93 | 0.93 | 0.47 | 0.63 | 0.76 | 1.08 | 1.17 | 0.77 | 0.56 | 0.64 |
| counted | gait | 0.82 | 0.92 | 0.94 | 0.77 | 1.01 | 1.02 | 1.93 | 2.39 | 0.65 | 0.49 | 0.55 |
| counted | ECG 8-lead | 0.74 | 0.94 | 0.94 | 0.81 | 2.43 | 0.85 | 4.37 | 4.88 | 0.57 | 0.21 | 0.22 |
| stress test | ocean | 0.71 | 0.92 | 0.92 | 0.67 | 1.03 | 0.53 | 2.11 | 2.25 | 0.53 | 0.18 | 0.45 |
| stress test | weather | 0.61 | 0.83 | 0.88 | 0.68 | 1.01 | 0.77 | 1.79 | 1.98 | 0.54 | 0.23 | 0.32 |
| stress test | electricity | 0.61 | 0.91 | 0.93 | 0.81 | 3.42 | 0.56 | 5.62 | 5.72 | 0.51 | 0.09 | 0.08 |
10.5 The per-dataset rule and cross-dataset consistency
The rule was applied per dataset on its pooled cells (Table 67). Question 1 is consistent: on every counted dataset CL2 reduces the calibration error of the REML fit’s intervals by 0.20–0.42 (stress tests 0.38–0.54; class C everywhere for both fits; consistency label “consistent for P”). The recipe decision is not: for β the classes are ECG strain:E AF trial:U running:N DTI:N gait:U ECG 8-lead:F and the label “inconsistent (N)”; for the mean ECG strain:E AF trial:E running:U DTI:U gait:E ECG 8-lead:U, “unresolved”. The proposal is the point decision on 1 of the 12 counted dataset-estimand pairs, the fallback on 3, neither on 2, equivalent on 6.
The N labels are fragile. They arise because the fallback’s calibration error for β exceeds 2 pp on running and DTI (running: 0.030 [0.020, 0.041]; DTI: 0.023 [0.018, 0.029]), i.e. by 0.3–1.0 pp with bootstrap intervals that include 2 pp, while the proposal’s is 0.04–0.07. Under the symmetric criterion (point decisions on the same nine-dataset pools, Table 68) the decision differs from the undercoverage-only one on 0 of 18 dataset-estimand pairs; these are point classifications without bootstrap flags, and the agreement is not a robustness result: the proposal already fails the undercoverage bound for β on 8 of 9 datasets, so adding overcoverage to the criterion cannot change its class there.
| dataset | role | estimand | n_cells | CE proposal | CE fallback | score ratio | decision | class | Q1 class (REML) | Q1 class (NCV) |
|---|---|---|---|---|---|---|---|---|---|---|
| ECG strain | counted | beta | 12 | 0.013 [0.009, 0.018] | 0.005 [0.000, 0.010] | 0.97 [0.96, 0.99] | E | E | C | C |
| AF trial | counted | beta | 6 | 0.034 [0.024, 0.045] | 0.012 [0.004, 0.021] | 0.94 [0.89, 1.01] | F | U | C | C |
| running | counted | beta | 6 | 0.072 [0.061, 0.084] | 0.030 [0.020, 0.041] | 1.10 [1.05, 1.14] | N | N | C | C |
| DTI | counted | beta | 6 | 0.039 [0.033, 0.044] | 0.023 [0.018, 0.029] | 0.94 [0.92, 0.97] | N | N | C | C |
| gait | counted | beta | 6 | 0.027 [0.019, 0.035] | 0.010 [0.000, 0.019] | 0.95 [0.91, 0.99] | F | U | C | C |
| ECG 8-lead | counted | beta | 6 | 0.031 [0.024, 0.037] | 0.008 [0.002, 0.014] | 0.80 [0.77, 0.83] | F | F | C | C |
| AF trial | counted | gamma | 6 | 0.038 [0.019, 0.055] | 0.000 [0.000, 0.018] | 1.08 [1.03, 1.12] | F | U | C | C |
| DTI | counted | gamma | 6 | 0.019 [0.006, 0.037] | 0.016 [0.000, 0.034] | 0.93 [0.89, 0.97] | P | U | C | C |
| ECG 8-lead | counted | gamma | 6 | 0.000 [0.000, 0.002] | 0.000 [0.000, 0.000] | 1.04 [1.01, 1.07] | E | U | C | C |
| ECG strain | counted | mean | 12 | 0.009 [0.006, 0.012] | 0.008 [0.004, 0.011] | 1.00 [1.00, 1.01] | E | E | C | C |
| AF trial | counted | mean | 6 | 0.014 [0.009, 0.019] | 0.003 [0.000, 0.009] | 1.03 [1.01, 1.05] | E | E | C | C |
| running | counted | mean | 6 | 0.018 [0.013, 0.023] | 0.017 [0.011, 0.023] | 1.02 [1.01, 1.03] | E | U | C | C |
| DTI | counted | mean | 6 | 0.015 [0.012, 0.019] | 0.018 [0.013, 0.022] | 0.95 [0.94, 0.96] | E | U | C | C |
| gait | counted | mean | 6 | 0.006 [0.001, 0.010] | 0.006 [0.000, 0.011] | 0.99 [0.98, 1.00] | E | E | C | C |
| ECG 8-lead | counted | mean | 6 | 0.000 [0.000, 0.003] | 0.000 [0.000, 0.000] | 0.94 [0.93, 0.96] | P | U | C | C |
| ocean | stress test | beta | 6 | 0.050 [0.043, 0.057] | 0.025 [0.014, 0.036] | 1.03 [0.98, 1.08] | N | N | C | C |
| weather | stress test | beta | 6 | 0.124 [0.098, 0.154] | 0.071 [0.049, 0.096] | 1.14 [1.08, 1.21] | N | N | C | C |
| electricity | stress test | beta | 6 | 0.082 [0.070, 0.094] | 0.026 [0.016, 0.038] | 0.98 [0.95, 1.02] | N | N | C | C |
| ocean | stress test | mean | 6 | 0.012 [0.007, 0.017] | 0.015 [0.009, 0.020] | 0.98 [0.97, 1.00] | E | U | C | C |
| weather | stress test | mean | 6 | 0.030 [0.019, 0.042] | 0.027 [0.019, 0.037] | 1.05 [1.02, 1.08] | N | N | C | C |
| electricity | stress test | mean | 6 | 0.021 [0.015, 0.027] | 0.022 [0.015, 0.031] | 0.93 [0.91, 0.94] | N | N | C | C |
| app | role | estimand | n_cells | ce_proposal_under | ce_fallback_under | ce_proposal_sym | ce_fallback_sym | score_ratio | decision_under | decision_sym |
|---|---|---|---|---|---|---|---|---|---|---|
| ECG strain | counted | beta | 12 | 0.013 | 0.005 | 0.013 | 0.005 | 0.974 | E | E |
| AF trial | counted | beta | 6 | 0.034 | 0.012 | 0.034 | 0.012 | 0.944 | F | F |
| running | counted | beta | 6 | 0.072 | 0.030 | 0.072 | 0.030 | 1.099 | N | N |
| DTI | counted | beta | 6 | 0.039 | 0.023 | 0.039 | 0.023 | 0.945 | N | N |
| gait | counted | beta | 6 | 0.027 | 0.010 | 0.027 | 0.010 | 0.949 | F | F |
| ECG 8-lead | counted | beta | 6 | 0.031 | 0.008 | 0.033 | 0.008 | 0.797 | F | F |
| ocean | stress test | beta | 6 | 0.050 | 0.025 | 0.051 | 0.025 | 1.028 | N | N |
| weather | stress test | beta | 6 | 0.124 | 0.071 | 0.124 | 0.071 | 1.142 | N | N |
| electricity | stress test | beta | 6 | 0.082 | 0.026 | 0.084 | 0.026 | 0.981 | N | N |
| ECG strain | counted | mean | 12 | 0.009 | 0.008 | 0.009 | 0.008 | 1.004 | E | E |
| AF trial | counted | mean | 6 | 0.014 | 0.003 | 0.014 | 0.003 | 1.030 | E | E |
| running | counted | mean | 6 | 0.018 | 0.017 | 0.018 | 0.017 | 1.016 | E | E |
| DTI | counted | mean | 6 | 0.015 | 0.018 | 0.015 | 0.018 | 0.952 | E | E |
| gait | counted | mean | 6 | 0.006 | 0.006 | 0.006 | 0.006 | 0.992 | E | E |
| ECG 8-lead | counted | mean | 6 | 0.000 | 0.000 | 0.010 | 0.001 | 0.941 | P | P |
| ocean | stress test | mean | 6 | 0.012 | 0.015 | 0.012 | 0.015 | 0.981 | E | E |
| weather | stress test | mean | 6 | 0.030 | 0.027 | 0.030 | 0.027 | 1.051 | N | N |
| electricity | stress test | mean | 6 | 0.021 | 0.022 | 0.024 | 0.022 | 0.927 | N | N |
10.6 Attribution: truth source versus residual source
On the 2 × 2 of REML/NCV truths × REML/NCV residuals (G = all), the truth-source main effect on log(MSE NCV / MSE REML) for β is -4.05 to -0.24 (NCV truth minus REML truth; the ratio is multiplied by 0.02–0.78), the residual-source main effect -0.27 to 0.08 and the interaction -0.15 to 0.29 (Table 90). The residual-source effect on the log ratio is “small” (inside log[0.95, 1.05] with its interval) on 1 dataset, “not small” on 2 (DTI -0.086, electricity -0.267) and undetermined on the remaining 6. Of the 90 residual-source effects on coverage, 84 are inside ±2 pp with their interval; the exceptions are ocean beta(s,t) REML, model-based (-1.4 pp); ocean beta(s,t) NCV + CL2 (-0.8 pp); weather beta(s,t) NCV, model-based (-1.4 pp); weather beta(s,t) NCV + CL2 (-3.2 pp); electricity beta(s,t) REML, model-based (-0.8 pp); electricity beta(s,t) REML + CL2 (-1.3 pp). The rule’s class on the REML-residual cells alone differs from that on the NCV-residual cells alone on 3 of 21 dataset-estimand pairs (AF trial beta: F vs U; ocean mean: E vs U; electricity beta: U vs N); 0 flips are between P and F. Truth source is the dominant factor for estimation; the residual source matters on two datasets.
10.7 Beat-to-beat errors (ECG strain)
The beat-to-beat difference curves are the one error source not shaped by any fit (design effect 21.7 against 19.9 for the REML residuals). With them, REML + CL2’s coverage changes by 0.1–1.2 pp (variance-matched) and 0.1–1.2 pp (own scale) relative to the REML-residual cells with the same truth, the bias-aware NCV interval’s by 0.3–1.1 pp and 0.7–1.5 pp, and the model-based intervals’ by -3.9 to -1.6 pp and -3.0 to -0.8 pp (Table 69). NCV’s β gain is larger with the variance-matched beat errors than with the REML residuals under every truth (ratios 0.73–0.91 vs 0.78–0.98) and smaller at the beats’ own scale (0.89–0.97).
| truth | residual | estimand | NCV + CL2 | NCV + CL2, bias-aware | REML + CL2 | REML, model-based | AR(1) working model | mse_ratio_ncv_reml |
|---|---|---|---|---|---|---|---|---|
| REML | beta(s,t) | 0.899 | 0.929 | 0.943 | 0.559 | 0.951 | ||
| REML | beta(s,t) | 0.904 | 0.931 | 0.943 | 0.563 | 0.982 | ||
| REML | beta(s,t) | 0.932 | 0.944 | 0.945 | 0.536 | 0.970 | ||
| REML | beta(s,t) | 0.910 | 0.939 | 0.944 | 0.526 | 0.907 | ||
| MID | beta(s,t) | 0.900 | 0.931 | 0.943 | 0.558 | 0.890 | ||
| MID | beta(s,t) | 0.904 | 0.932 | 0.944 | 0.561 | 0.874 | 0.927 | |
| MID | beta(s,t) | 0.933 | 0.946 | 0.945 | 0.534 | 0.943 | ||
| MID | beta(s,t) | 0.910 | 0.940 | 0.945 | 0.524 | 0.837 | 0.857 | |
| NCV | beta(s,t) | 0.909 | 0.941 | 0.944 | 0.559 | 0.732 | ||
| NCV | beta(s,t) | 0.909 | 0.939 | 0.944 | 0.562 | 0.782 | ||
| NCV | beta(s,t) | 0.928 | 0.946 | 0.945 | 0.532 | 0.887 | ||
| NCV | beta(s,t) | 0.906 | 0.942 | 0.945 | 0.523 | 0.728 | ||
| REML | E(Y | X) | 0.904 | 0.933 | 0.939 | 0.583 | 1.036 | ||
| REML | E(Y | X) | 0.906 | 0.933 | 0.938 | 0.586 | 1.043 | ||
| REML | E(Y | X) | 0.940 | 0.948 | 0.950 | 0.579 | 0.995 | ||
| REML | E(Y | X) | 0.921 | 0.945 | 0.949 | 0.571 | 0.992 | ||
| MID | E(Y | X) | 0.916 | 0.938 | 0.940 | 0.581 | 0.959 | ||
| MID | E(Y | X) | 0.916 | 0.937 | 0.938 | 0.583 | 0.869 | 0.977 | |
| MID | E(Y | X) | 0.940 | 0.951 | 0.950 | 0.570 | 0.974 | ||
| MID | E(Y | X) | 0.923 | 0.947 | 0.950 | 0.564 | 0.847 | 0.953 | |
| NCV | E(Y | X) | 0.918 | 0.942 | 0.940 | 0.582 | 0.875 | ||
| NCV | E(Y | X) | 0.916 | 0.939 | 0.938 | 0.583 | 0.905 | ||
| NCV | E(Y | X) | 0.935 | 0.950 | 0.950 | 0.569 | 0.962 | ||
| NCV | E(Y | X) | 0.921 | 0.948 | 0.950 | 0.564 | 0.888 |
10.8 AR(1) in the plasmode study
In the common (MID truth, REML residuals) cell the AR(1) working model covers the mean at 0.67–0.93 and β at 0.44–0.91 across the nine datasets (Table 91), with ρ̂ at 0.45–0.99 (at its 0.99 cap on the running data) and median fit times of 28–162 s. It reaches 0.93 for both primary estimands on 0 datasets. The sensitivity cells (running under the REML- and NCV-fitted truths, ECG strain with beat errors) give 0.55–0.85.
10.9 Block arm on the dependent-curve datasets
The stress tests’ curve-flip cells generate independent errors across curves, so the only evidence about between-curve dependence is the contrast of block-flip against curve-flip cells. Under imposed block dependence REML + curve-clustered CL2 changes by -1.8 to 2.1 pp relative to the curve-flip cells (bias-aware NCV -2.1 to 0.9 pp). Leaving out and clustering on blocks does not help estimation: the paired MSE ratio block-NCV / NCV is 0.99–1.11 (Table 70), and under independent curves block clustering costs 0.1–2.9 pp of coverage. Verdicts: ocean mean: cost only; ocean beta: inconclusive; weather mean: hurts; weather beta: inconclusive; electricity mean: cost only; electricity beta: inconclusive. Block-clustered CL2 does not recover the shortfall on the weather data (REML + block CL2 0.89–0.90 against REML + curve CL2 0.90–0.91 under block flips); its 16 blocks are below the ≥ 40 clusters of the study’s operating range, and the mechanism is untested there. The partition-mismatch arm (150 km blocks against 250 km generating blocks) gives an MSE ratio of 1.00–1.04.
| dataset | estimand | verdict | cells | mse_ratio_block | coverage_loss | coverage_loss_recipe | mse_ratio_mismatch |
|---|---|---|---|---|---|---|---|
| ocean | beta | inconclusive | block_flips | 1.105 [0.979, 1.278] | 0.006 [0.005, 0.007] | 0.006 [0.003, 0.009] | |
| ocean | beta | inconclusive | curve_flips | 1.168 [1.029, 1.290] | 0.001 [-0.000, 0.002] | 0.004 [0.001, 0.007] | |
| ocean | mean | cost only | block_flips | 0.995 [0.989, 1.001] | -0.006 [-0.007, -0.005] | -0.005 [-0.007, -0.004] | |
| ocean | mean | cost only | curve_flips | 1.003 [0.997, 1.010] | 0.003 [0.001, 0.004] | 0.003 [0.002, 0.005] | |
| weather | beta | inconclusive | block_flips | 1.017 [0.955, 1.090] | 0.011 [0.009, 0.014] | 0.014 [0.008, 0.019] | 1.039 [0.957, 1.128] |
| weather | beta | inconclusive | curve_flips | 1.061 [1.002, 1.132] | 0.010 [0.007, 0.014] | -0.007 [-0.014, -0.001] | |
| weather | mean | hurts | block_flips | 1.014 [0.995, 1.032] | -0.006 [-0.010, -0.002] | -0.003 [-0.007, 0.002] | 0.999 [0.982, 1.015] |
| weather | mean | hurts | curve_flips | 1.009 [0.986, 1.030] | 0.029 [0.023, 0.033] | 0.026 [0.021, 0.031] | |
| electricity | beta | inconclusive | block_flips | 1.021 [1.003, 1.045] | 0.000 [-0.000, 0.001] | -0.002 [-0.003, -0.000] | |
| electricity | beta | inconclusive | curve_flips | 1.055 [0.977, 1.161] | 0.001 [-0.000, 0.001] | 0.000 [-0.001, 0.002] | |
| electricity | mean | cost only | block_flips | 1.001 [0.995, 1.008] | -0.008 [-0.009, -0.006] | -0.008 [-0.009, -0.007] | |
| electricity | mean | cost only | curve_flips | 1.004 [0.995, 1.013] | 0.001 [-0.000, 0.003] | 0.001 [-0.001, 0.002] |
10.10 The hybrid interval and coverage maps in the plasmode study
Averaged over all nine datasets, the hybrid (NCV estimate, REML’s CL2 standard error) is worse than REML + CL2 on every metric under REML-fitted truths and better only under the NCV- and MID-derived truths, partly by overcovering (Table 71; β at G = all, model residuals, equal-weighted means over the nine datasets and two residual sources, not over truth sources): under the REML truth it covers 0.906 against REML + CL2’s 0.924 with 5% quantiles 0.667 against 0.869 and interval score 1.04 times REML + CL2’s; under MID 0.962 against 0.932 (quantiles 0.877 vs 0.886); under the NCV truth 0.970 against 0.925 (quantiles 0.918 vs 0.881). Its width equals REML + CL2’s by construction; the bias-aware interval is 0.91–1.02 times as wide across truth sources. The coverage maps (Figure 19) show where the intervals built on the NCV estimate fall short under a REML-fitted truth: on the running data NCV + CL2 has 87% of grid points below 0.90 coverage, the bias-aware interval 70% and the hybrid 49%, against REML + CL2’s 30%; the pointwise shortfall of the bias-aware interval correlates with the truth’s curvature (Spearman 0.46 running, 0.57 DTI) and with (β̂_NCV − β̂_REML)² / se²_CL2,NCV (0.68, 0.58). Under the NCV-fitted truth the same maps are near nominal for every construction. Per-dataset values are in Table 93. On the six counted datasets alone its average coverage under the REML-fitted truth is close to REML + CL2’s and its interval score slightly lower; its 5% pointwise quantile is far lower (paper, §6.4).
| truth | estimand | n_cells | cov REML+CL2 | cov bias-aware | cov hybrid | q05 REML+CL2 | q05 bias-aware | q05 hybrid | width bias-aware/REML+CL2 | width hybrid/REML+CL2 | IS bias-aware/REML+CL2 | IS hybrid/REML+CL2 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| REML | E(Y | X) | 18 | 0.939 | 0.930 | 0.927 | 0.895 | 0.856 | 0.825 | 1.013 | 1 | 1.056 | 1.038 |
| MID | E(Y | X) | 18 | 0.937 | 0.941 | 0.947 | 0.894 | 0.878 | 0.884 | 0.979 | 1 | 0.973 | 0.955 |
| NCV | E(Y | X) | 18 | 0.935 | 0.946 | 0.954 | 0.887 | 0.880 | 0.897 | 0.972 | 1 | 0.936 | 0.918 |
| REML | beta(s,t) | 18 | 0.924 | 0.873 | 0.906 | 0.869 | 0.741 | 0.667 | 1.020 | 1 | 1.230 | 1.042 |
| MID | beta(s,t) | 18 | 0.932 | 0.918 | 0.962 | 0.886 | 0.801 | 0.877 | 0.909 | 1 | 0.926 | 0.861 |
| NCV | beta(s,t) | 18 | 0.925 | 0.938 | 0.970 | 0.881 | 0.860 | 0.918 | 0.906 | 1 | 0.830 | 0.823 |
11 DTI application
The paper’s application (analysis/dti-application-case.R → summaries/dti-case/): right corticospinal tract FA on corpus-callosum FA and MS status, baseline visit, complete cases — the model of the DTI plasmode dataset — fitted with refund 79a346fb by REML and by curve-blocked NCV.
| estimand | fit | se | median_halfwidth | share_excluding_zero | median_se_ratio_cl2_model |
|---|---|---|---|---|---|
| beta | REML | se_model | 0.914 | 0.526 | 2.28 |
| beta | REML | se_cl2 | 2.049 | 0.258 | 2.28 |
| gamma | REML | se_model | 0.010 | 0.382 | 1.88 |
| gamma | REML | se_cl2 | 0.018 | 0.236 | 1.88 |
| fit | G | D | total_edf | ff_edf | gamma_edf | r2 | median_abs_diff_in_se |
|---|---|---|---|---|---|---|---|
| REML | 92 | 55 | 69.5 | 53.8 | 4.93 | 0.726 | |
| NCV | 92 | 55 | 48.9 | 35.1 | 1.96 | 0.717 | 0.641 |
12 The main recipes on every metric
One view of the four main interval constructions on every reported metric: grid-average coverage, the 5% quantile of pointwise coverage (q05), width and interval score relative to REML + CL2 (geometric means of per-cell ratios; the interval score is a proper scoring rule, < 1 is better), the median relative error of the centre, the median relative half-width and the median detection rate (definitions in the reading guide and Section 3.3.4). Synthetic: the core cells (three families, all signal levels), by dependence and G. Plasmode: the attached curve-flip cells with all subjects, all truth × residual sources. REML + CL2 and the model-based interval share the REML centre, NCV + CL2 and the bias-aware interval the NCV centre, so their relative errors coincide. AR(1) results on the same metrics are in Table 17 and Table 91, the comparators in Table 43 and Table 46, the NCV-centred corrections in Table 51.
In the dependent synthetic cells at G = 100 the four constructions cover β at 0.712 (model-based), 0.945 (REML + CL2), 0.927 (NCV + CL2) and 0.981 (bias-aware), with interval scores 1.75, 1, 0.45 and 0.69 times REML + CL2’s; the NCV centre’s median relative error is 0.17 against 0.40 for REML, and the detection rates are 0.76 (REML + CL2), 0.96 (NCV + CL2) and 0.82 (bias-aware) (Table 74). In the dependent settings the model-based interval’s score is 1.75–1.85 times REML + CL2’s: the score’s penalty term prices its undercoverage. On the counted plasmode datasets the pooled β coverage is 0.578, 0.936, 0.821 and 0.925 and the interval scores 2.94, 1, 1.09 and 0.95 times REML + CL2’s (Table 75); per dataset the bias-aware interval’s score ratio is 0.80–1.14 and NCV + CL2’s 0.75–1.84 (Table 76). The interval score rewards the NCV centre’s smaller error even where the interval undercovers (NCV + CL2 in the dependent synthetic cells: coverage 0.927, score 0.45 times REML + CL2’s), so a better score is not evidence of calibration (Figure 21).
| errors | G | estimand | arm | cells | coverage | cov_min | q05 | width rel | IS rel | IS rel max | rel err | half-w | detect |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| dep. | 100 | E(Y | X) | REML, model-based | 14 | 0.676 | 0.631 | 0.593 | 0.508 | 1.846 | 2.080 | 0.215 | 0.202 | |
| dep. | 100 | E(Y | X) | REML + CL2 | 14 | 0.943 | 0.940 | 0.913 | 1.000 | 1.000 | 1.000 | 0.215 | 0.431 | |
| dep. | 100 | E(Y | X) | NCV + CL2 | 14 | 0.918 | 0.876 | 0.845 | 0.769 | 0.859 | 0.993 | 0.178 | 0.334 | |
| dep. | 100 | E(Y | X) | NCV + CL2, bias-aware | 14 | 0.952 | 0.933 | 0.909 | 0.928 | 0.884 | 0.939 | 0.178 | 0.403 | |
| dep. | 100 | beta(s,t) | REML, model-based | 14 | 0.712 | 0.653 | 0.626 | 0.544 | 1.755 | 2.106 | 0.404 | 0.442 | 0.896 |
| dep. | 100 | beta(s,t) | REML + CL2 | 14 | 0.945 | 0.940 | 0.919 | 1.000 | 1.000 | 1.000 | 0.404 | 0.823 | 0.756 |
| dep. | 100 | beta(s,t) | NCV + CL2 | 14 | 0.927 | 0.896 | 0.847 | 0.399 | 0.448 | 0.542 | 0.166 | 0.318 | 0.958 |
| dep. | 100 | beta(s,t) | NCV + CL2, bias-aware | 14 | 0.981 | 0.971 | 0.944 | 0.816 | 0.690 | 0.716 | 0.166 | 0.720 | 0.820 |
| dep. | 40 | E(Y | X) | REML, model-based | 14 | 0.660 | 0.601 | 0.580 | 0.498 | 1.842 | 2.151 | 0.361 | 0.340 | |
| dep. | 40 | E(Y | X) | REML + CL2 | 14 | 0.930 | 0.921 | 0.898 | 1.000 | 1.000 | 1.000 | 0.361 | 0.740 | |
| dep. | 40 | E(Y | X) | NCV + CL2 | 14 | 0.896 | 0.842 | 0.813 | 0.706 | 0.833 | 0.989 | 0.272 | 0.527 | |
| dep. | 40 | E(Y | X) | NCV + CL2, bias-aware | 14 | 0.948 | 0.930 | 0.904 | 0.933 | 0.856 | 0.922 | 0.272 | 0.687 | |
| dep. | 40 | beta(s,t) | REML, model-based | 14 | 0.686 | 0.609 | 0.607 | 0.535 | 1.833 | 2.288 | 0.731 | 0.760 | 0.748 |
| dep. | 40 | beta(s,t) | REML + CL2 | 14 | 0.929 | 0.923 | 0.899 | 1.000 | 1.000 | 1.000 | 0.731 | 1.420 | 0.497 |
| dep. | 40 | beta(s,t) | NCV + CL2 | 14 | 0.898 | 0.785 | 0.817 | 0.337 | 0.409 | 0.698 | 0.242 | 0.462 | 0.895 |
| dep. | 40 | beta(s,t) | NCV + CL2, bias-aware | 14 | 0.976 | 0.948 | 0.942 | 0.847 | 0.671 | 0.753 | 0.242 | 1.270 | 0.593 |
| iid | 100 | E(Y | X) | REML, model-based | 7 | 0.965 | 0.962 | 0.934 | 1.020 | 0.987 | 0.992 | 0.091 | 0.191 | |
| iid | 100 | E(Y | X) | REML + CL2 | 7 | 0.958 | 0.949 | 0.921 | 1.000 | 1.000 | 1.000 | 0.091 | 0.191 | |
| iid | 100 | E(Y | X) | NCV + CL2 | 7 | 0.946 | 0.939 | 0.889 | 0.930 | 0.977 | 1.034 | 0.088 | 0.174 | |
| iid | 100 | E(Y | X) | NCV + CL2, bias-aware | 7 | 0.954 | 0.945 | 0.905 | 0.965 | 0.978 | 1.022 | 0.088 | 0.182 | |
| iid | 100 | beta(s,t) | REML, model-based | 7 | 0.989 | 0.983 | 0.971 | 1.024 | 1.013 | 1.038 | 0.123 | 0.327 | 0.972 |
| iid | 100 | beta(s,t) | REML + CL2 | 7 | 0.985 | 0.979 | 0.964 | 1.000 | 1.000 | 1.000 | 0.123 | 0.321 | 0.973 |
| iid | 100 | beta(s,t) | NCV + CL2 | 7 | 0.981 | 0.976 | 0.936 | 0.775 | 0.794 | 0.922 | 0.095 | 0.229 | 0.984 |
| iid | 100 | beta(s,t) | NCV + CL2, bias-aware | 7 | 0.988 | 0.986 | 0.951 | 0.845 | 0.841 | 0.934 | 0.095 | 0.255 | 0.979 |
| iid | 40 | E(Y | X) | REML, model-based | 7 | 0.964 | 0.952 | 0.927 | 1.027 | 0.969 | 0.975 | 0.135 | 0.295 | |
| iid | 40 | E(Y | X) | REML + CL2 | 7 | 0.951 | 0.925 | 0.906 | 1.000 | 1.000 | 1.000 | 0.135 | 0.295 | |
| iid | 40 | E(Y | X) | NCV + CL2 | 7 | 0.941 | 0.923 | 0.884 | 0.947 | 0.990 | 1.034 | 0.133 | 0.273 | |
| iid | 40 | E(Y | X) | NCV + CL2, bias-aware | 7 | 0.949 | 0.935 | 0.898 | 0.980 | 0.984 | 1.007 | 0.133 | 0.280 | |
| iid | 40 | beta(s,t) | REML, model-based | 7 | 0.990 | 0.984 | 0.969 | 1.032 | 1.014 | 1.032 | 0.164 | 0.455 | 0.944 |
| iid | 40 | beta(s,t) | REML + CL2 | 7 | 0.985 | 0.973 | 0.957 | 1.000 | 1.000 | 1.000 | 0.164 | 0.449 | 0.945 |
| iid | 40 | beta(s,t) | NCV + CL2 | 7 | 0.981 | 0.973 | 0.942 | 0.817 | 0.834 | 0.981 | 0.135 | 0.338 | 0.960 |
| iid | 40 | beta(s,t) | NCV + CL2, bias-aware | 7 | 0.987 | 0.979 | 0.954 | 0.874 | 0.865 | 0.986 | 0.135 | 0.362 | 0.955 |
| role | estimand | arm | cells | coverage | cov_min | q05 | width rel | IS rel | IS rel max | rel err | half-w | detect |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| counted | E(Y | X) | REML, model-based | 42 | 0.620 | 0.538 | 0.310 | 0.427 | 2.768 | 3.36 | 0.136 | 0.091 | |
| counted | E(Y | X) | REML + CL2 | 42 | 0.942 | 0.930 | 0.895 | 1.000 | 1.000 | 1.00 | 0.136 | 0.270 | |
| counted | E(Y | X) | NCV + CL2 | 42 | 0.909 | 0.861 | 0.789 | 0.866 | 1.033 | 1.25 | 0.134 | 0.242 | |
| counted | E(Y | X) | NCV + CL2, bias-aware | 42 | 0.942 | 0.925 | 0.877 | 0.981 | 0.991 | 1.09 | 0.134 | 0.264 | |
| counted | beta(s,t) | REML, model-based | 42 | 0.578 | 0.479 | 0.294 | 0.395 | 2.941 | 3.64 | 0.836 | 0.604 | 0.825 |
| counted | beta(s,t) | REML + CL2 | 42 | 0.936 | 0.915 | 0.886 | 1.000 | 1.000 | 1.00 | 0.836 | 1.752 | 0.507 |
| counted | beta(s,t) | NCV + CL2 | 42 | 0.821 | 0.586 | 0.590 | 0.568 | 1.087 | 3.68 | 0.684 | 0.886 | 0.626 |
| counted | beta(s,t) | NCV + CL2, bias-aware | 42 | 0.925 | 0.838 | 0.820 | 0.924 | 0.951 | 1.58 | 0.684 | 1.495 | 0.441 |
| counted | alpha(t) | REML, model-based | 42 | 0.593 | 0.412 | 0.323 | 0.392 | 2.899 | 4.39 | 0.411 | 0.226 | |
| counted | alpha(t) | REML + CL2 | 42 | 0.943 | 0.928 | 0.912 | 1.000 | 1.000 | 1.00 | 0.411 | 0.922 | |
| counted | alpha(t) | NCV + CL2 | 42 | 0.914 | 0.858 | 0.839 | 0.899 | 1.091 | 1.49 | 0.385 | 0.874 | |
| counted | alpha(t) | NCV + CL2, bias-aware | 42 | 0.938 | 0.919 | 0.881 | 0.984 | 1.050 | 1.25 | 0.385 | 0.892 | |
| counted | gamma(t) | REML, model-based | 42 | 0.628 | 0.517 | 0.375 | 0.441 | 2.639 | 3.52 | 0.602 | 0.483 | 0.729 |
| counted | gamma(t) | REML + CL2 | 42 | 0.944 | 0.928 | 0.906 | 1.000 | 1.000 | 1.00 | 0.602 | 1.427 | 0.379 |
| counted | gamma(t) | NCV + CL2 | 42 | 0.901 | 0.810 | 0.812 | 0.873 | 1.065 | 1.70 | 0.577 | 1.139 | 0.477 |
| counted | gamma(t) | NCV + CL2, bias-aware | 42 | 0.934 | 0.900 | 0.868 | 0.978 | 1.024 | 1.23 | 0.577 | 1.387 | 0.380 |
| stress test | E(Y | X) | REML, model-based | 18 | 0.498 | 0.452 | 0.291 | 0.343 | 3.031 | 3.45 | 0.230 | 0.153 | |
| stress test | E(Y | X) | REML + CL2 | 18 | 0.929 | 0.912 | 0.884 | 1.000 | 1.000 | 1.00 | 0.230 | 0.450 | |
| stress test | E(Y | X) | NCV + CL2 | 18 | 0.884 | 0.784 | 0.766 | 0.808 | 1.035 | 1.54 | 0.192 | 0.321 | |
| stress test | E(Y | X) | NCV + CL2, bias-aware | 18 | 0.935 | 0.912 | 0.866 | 1.004 | 0.985 | 1.17 | 0.192 | 0.447 | |
| stress test | beta(s,t) | REML, model-based | 18 | 0.395 | 0.315 | 0.241 | 0.274 | 3.597 | 4.18 | 1.063 | 0.561 | 0.766 |
| stress test | beta(s,t) | REML + CL2 | 18 | 0.911 | 0.840 | 0.867 | 1.000 | 1.000 | 1.00 | 1.063 | 2.248 | 0.321 |
| stress test | beta(s,t) | NCV + CL2 | 18 | 0.642 | 0.316 | 0.365 | 0.300 | 1.314 | 6.18 | 0.678 | 0.627 | 0.528 |
| stress test | beta(s,t) | NCV + CL2, bias-aware | 18 | 0.885 | 0.800 | 0.781 | 0.990 | 1.048 | 1.73 | 0.678 | 2.109 | 0.141 |
| stress test | alpha(t) | REML, model-based | 18 | 0.444 | 0.397 | 0.314 | 0.308 | 3.301 | 3.74 | 0.443 | 0.264 | |
| stress test | alpha(t) | REML + CL2 | 18 | 0.924 | 0.884 | 0.874 | 1.000 | 1.000 | 1.00 | 0.443 | 1.070 | |
| stress test | alpha(t) | NCV + CL2 | 18 | 0.885 | 0.724 | 0.825 | 0.807 | 0.969 | 1.31 | 0.461 | 0.974 | |
| stress test | alpha(t) | NCV + CL2, bias-aware | 18 | 0.933 | 0.876 | 0.894 | 0.978 | 0.960 | 1.13 | 0.461 | 1.038 | |
| stress test | gamma(t) | REML, model-based | 18 | 0.409 | 0.357 | 0.296 | 0.280 | 3.287 | 3.92 | 0.222 | 0.172 | 0.974 |
| stress test | gamma(t) | REML + CL2 | 18 | 0.934 | 0.882 | 0.914 | 1.000 | 1.000 | 1.00 | 0.222 | 0.597 | 0.809 |
| stress test | gamma(t) | NCV + CL2 | 18 | 0.814 | 0.329 | 0.712 | 0.733 | 1.231 | 2.86 | 0.225 | 0.468 | 0.964 |
| stress test | gamma(t) | NCV + CL2, bias-aware | 18 | 0.914 | 0.745 | 0.861 | 0.986 | 1.057 | 1.52 | 0.225 | 0.504 | 0.724 |
| dataset | arm | cells | coverage | q05 | width rel | IS rel | rel err | half-w | detect |
|---|---|---|---|---|---|---|---|---|---|
| ECG strain | REML, model-based | 12 | 0.545 | 0.219 | 0.351 | 3.241 | 0.698 | 0.466 | 0.836 |
| ECG strain | REML + CL2 | 12 | 0.944 | 0.901 | 1.000 | 1.000 | 0.698 | 1.417 | 0.442 |
| ECG strain | NCV + CL2 | 12 | 0.912 | 0.809 | 0.844 | 1.009 | 0.676 | 1.177 | 0.461 |
| ECG strain | NCV + CL2, bias-aware | 12 | 0.938 | 0.862 | 0.930 | 0.974 | 0.676 | 1.300 | 0.404 |
| AF trial | REML, model-based | 6 | 0.666 | 0.363 | 0.498 | 2.565 | 0.865 | 0.945 | 0.689 |
| AF trial | REML + CL2 | 6 | 0.938 | 0.879 | 1.000 | 1.000 | 0.865 | 1.824 | 0.372 |
| AF trial | NCV + CL2 | 6 | 0.789 | 0.485 | 0.493 | 1.190 | 0.714 | 0.963 | 0.533 |
| AF trial | NCV + CL2, bias-aware | 6 | 0.916 | 0.772 | 0.922 | 0.944 | 0.714 | 1.534 | 0.303 |
| running | REML, model-based | 6 | 0.507 | 0.268 | 0.344 | 3.240 | 0.825 | 0.600 | 0.865 |
| running | REML + CL2 | 6 | 0.919 | 0.873 | 1.000 | 1.000 | 0.825 | 1.851 | 0.637 |
| running | NCV + CL2 | 6 | 0.734 | 0.441 | 0.527 | 1.497 | 0.661 | 0.678 | 0.735 |
| running | NCV + CL2, bias-aware | 6 | 0.892 | 0.766 | 0.979 | 1.099 | 0.661 | 1.516 | 0.578 |
| DTI | REML, model-based | 6 | 0.632 | 0.475 | 0.491 | 2.103 | 0.633 | 0.596 | 0.872 |
| DTI | REML + CL2 | 6 | 0.927 | 0.876 | 1.000 | 1.000 | 0.633 | 1.169 | 0.638 |
| DTI | NCV + CL2 | 6 | 0.846 | 0.586 | 0.647 | 1.034 | 0.469 | 0.760 | 0.772 |
| DTI | NCV + CL2, bias-aware | 6 | 0.926 | 0.806 | 0.947 | 0.945 | 0.469 | 1.083 | 0.562 |
| gait | REML, model-based | 6 | 0.517 | 0.231 | 0.308 | 3.596 | 1.007 | 0.653 | 0.825 |
| gait | REML + CL2 | 6 | 0.939 | 0.903 | 1.000 | 1.000 | 1.007 | 2.388 | 0.546 |
| gait | NCV + CL2 | 6 | 0.816 | 0.547 | 0.562 | 1.283 | 0.765 | 1.024 | 0.650 |
| gait | NCV + CL2, bias-aware | 6 | 0.925 | 0.833 | 0.895 | 0.949 | 0.765 | 1.931 | 0.486 |
| ECG 8-lead | REML, model-based | 6 | 0.638 | 0.282 | 0.468 | 2.880 | 2.427 | 2.197 | 0.567 |
| ECG 8-lead | REML + CL2 | 6 | 0.942 | 0.873 | 1.000 | 1.000 | 2.427 | 4.878 | 0.216 |
| ECG 8-lead | NCV + CL2 | 6 | 0.739 | 0.449 | 0.284 | 0.747 | 0.806 | 0.851 | 0.568 |
| ECG 8-lead | NCV + CL2, bias-aware | 6 | 0.937 | 0.840 | 0.868 | 0.797 | 0.806 | 4.371 | 0.210 |
| ocean | REML, model-based | 6 | 0.412 | 0.224 | 0.266 | 3.707 | 1.035 | 0.542 | 0.791 |
| ocean | REML + CL2 | 6 | 0.925 | 0.887 | 1.000 | 1.000 | 1.035 | 2.248 | 0.450 |
| ocean | NCV + CL2 | 6 | 0.708 | 0.281 | 0.340 | 1.430 | 0.667 | 0.527 | 0.528 |
| ocean | NCV + CL2, bias-aware | 6 | 0.916 | 0.794 | 1.003 | 1.028 | 0.667 | 2.109 | 0.178 |
| weather | REML, model-based | 6 | 0.341 | 0.233 | 0.279 | 3.321 | 1.007 | 0.561 | 0.766 |
| weather | REML + CL2 | 6 | 0.882 | 0.833 | 1.000 | 1.000 | 1.007 | 1.979 | 0.321 |
| weather | NCV + CL2 | 6 | 0.608 | 0.415 | 0.414 | 1.838 | 0.678 | 0.771 | 0.538 |
| weather | NCV + CL2, bias-aware | 6 | 0.828 | 0.715 | 0.936 | 1.142 | 0.678 | 1.786 | 0.228 |
| electricity | REML, model-based | 6 | 0.432 | 0.266 | 0.276 | 3.781 | 3.418 | 2.224 | 0.478 |
| electricity | REML + CL2 | 6 | 0.927 | 0.881 | 1.000 | 1.000 | 3.418 | 5.722 | 0.076 |
| electricity | NCV + CL2 | 6 | 0.611 | 0.398 | 0.192 | 0.863 | 0.815 | 0.564 | 0.512 |
| electricity | NCV + CL2, bias-aware | 6 | 0.911 | 0.835 | 1.035 | 0.981 | 0.815 | 5.623 | 0.094 |
13 Cross-study assessment
Table 77 summarises, property by property, what the synthetic study found and what the nine plasmode datasets can say about it; the evidence behind each row is in the list that follows. Associations across datasets are Spearman rank correlations over n = 9 (or the 6 counted) datasets with two-sided permutation p-values; at n = 9 a |rho| below about 0.6 is uninformative, and the datasets’ properties are confounded with each other and with the data source. Each correlation uses, per dataset, the pool mean over its attached curve-flip cells at G = all subjects (all truth × residual combinations) for coverages, the geometric mean over residual sources of the MID-truth cell ratios for the NCV gain, the (MID truth, REML residual) cell for AR(1), and the diagnostics of the REML residual curves (cross-study-dataset-outcomes.csv). The synthetic study spans design effects 5–12 in its dependent core cells (D = 61; realized values; realized values are means over 20 replicates without an uncertainty estimate and are missing for 2 cells, the negative-binomial data fitted as Poisson, where the design values stand in); the plasmode datasets span 6–40 with grid sizes 30–101. Because 1’Σ1/trΣ grows with the grid size, the normalisation DE/D = (1’Σ1/trΣ)/D (0.06–0.75) is reported beside it. The design effect is the grid-mean variance inflation and does not capture sign-changing or misregistration dependence: the sign-changing cells have realized DE 1.7–2.3 and the misregistration cells 1.5–1.9 while their model-based coverage is 0.70–0.88.
| property | synthetic: inference | synthetic: estimation | plasmode | verdict |
|---|---|---|---|---|
| Dependence strength (design effect DE) | model-based 0.96–0.99 (iid) → 0.63–0.82 (dependent); REML + CL2 0.94–0.99 | β ratio 0.55–1.06 (iid) vs 0.12–0.23 (dependent) | DE 6–40; model-based shortfall vs DE rho = 0.73, p = 0.03, n = 9; CL2 improves coverage on every dataset and meets the point adequacy bound for β on 4 of 6 counted datasets | reinforces the direction; DE is a grid-mean variance inflation that misses sign-changing and misregistration dependence and is confounded with D |
| Dependence shape (sign changes; variance along t or by covariate) | sign-changing: model-based 0.65–0.88, REML + CL2 0.94–0.96; covariate-dependent variance breaks model-based γ (0.49–0.51) and AR(1) (0.88–0.89), not CL2 (0.93–0.94) | one cell per family; the realized DE of the sign-changing (1.7–2.3) and misregistration (1.5–1.9) cells is near 1 although model-based coverage is 0.70–0.88 | negative-correlation share 0–33%, SD ratio 2–42; shortfall vs SD ratio rho = -0.73, p = 0.03, n = 9; CL2 β coverage vs SD ratio rho = 0.80, p = 0.01, n = 9 (counted only rho = 0.66, p = 0.18, n = 6) | not separately testable (shape, DE and dataset role are confounded) |
| Misregistration (phase variability) | model-based 0.71–0.73, REML + CL2 0.93–0.94, AR(1) mean 0.84; Poisson mean undercovers for every recipe (0.90–0.91) | β ratio 0.44–0.68 | phase share 0–63%; AR(1) β vs phase rho = 0.17, p = 0.67, n = 9; CL2 β vs phase rho = 0.49, p = 0.18, n = 9 | partly (CL2 near nominal on the two cardiac datasets with real misregistration; AR(1)’s real-data shortfall has no single attributable cause) |
| Number of curves G | REML + CL2 loses 1.1–1.8 pp at G = 40; NCV + CL2 1.0–6.9 pp | β ratio 0.21–0.33 (G = 40) vs 0.26–0.34 (G = 100) | within-dataset G = all vs 40: REML + CL2 loses -0.4 to 4.7 pp, NCV + CL2 1.6–21.3 pp; across datasets (G 73–138) CL2 β vs G rho = 0.30, p = 0.46, n = 9 | reinforces (paired within-dataset contrast of the same size) |
| Grid density D | D 61 → 241: model-based β −30.7–36.9 pp; REML + CL2 -1.3 to 0.0 pp | NCV β advantage × 3.2–6.1 | not varied within a dataset (D 30–101); DE/D = (1’S1/trS)/D 0.06–0.75; shortfall vs DE/D rho = 0.77, p = 0.02, n = 9 | not testable |
| Covariate rank (identifiability of β) | low-rank X: REML + CL2 β 0.94–0.95 | β ratio 0.09–0.12 (low-rank) vs 0.16–0.17 (rich), smooth process | rank 4–24; MID-truth β gain vs rank rho = 0.19, p = 0.63, n = 9; CL2 β vs rank rho = 0.67, p = 0.05, n = 9; the two lowest-rank counted datasets (running, AF trial) rank 1 and 3 of 6 in CL2 β coverage | partly (estimation direction consistent; coverage association uninformative at n = 9) |
| Truth roughness / EDF gap between REML and NCV | rough truth: NCV + CL2 β 0.80–0.90, bias-aware 0.93–0.95, REML + CL2 0.94–0.95 | β ratio 0.21–0.43 (rough) vs 0.12–0.22 (smooth); EDF ratio 1.92–2.31 | β ratio ordered NCV < MID < REML truth on 9 of 9; NCV loses under REML truth on 4; MID gain vs EDF ratio rho = -0.33, p = 0.39, n = 9; weather: NCV rougher than REML yet wins under all truths | partly (strong truth-source sensitivity; the EDF-gap mechanism is not identified in the plasmode study) |
| Signal strength | REML + CL2 within 1.6 pp of nominal across signal levels | β ratio 0.12 (low) to 0.17 (high), Gaussian smooth process | R² 0.19–0.47; shortfall vs R² rho = 0.55, p = 0.12, n = 9; MID gain vs R² rho = 0.50, p = 0.17, n = 9 | uninformative at n = 9 (confounded) |
| Term type and estimand | same pattern for α, γ, β, mean, f(x,t); REML + CL2 differs by -0.7 to 0.5 pp between f(x,t) and ff | gain on the surface: γ(t) ratio 0.93–1.04, α(t) 0.83–0.98 (dependent, G = 100) | ff model only; γ ratio 0.28–1.05 / 0.88–1.30 (NCV / REML truth), α 0.38–1.11 / 0.95–1.20; α gains < 0.9 on 4 datasets under the NCV truth | partly (gains most consistent for the surface; univariate gains on several datasets; f(x,t) not testable) |
| Basis size | bias-aware β 0.97–0.98 → 1.00; REML + CL2 0.94–0.95 → 0.95–0.96 (default → xlarge) | β ratio 0.16–0.17 → 0.01 | default bases except ECG 8-lead (k = 40/30 by design), whose MID-truth β gain ranks 2 of 9 (1 = largest) | not testable (one dataset, two changes) |
| Response family | six families: model-based 0.63–0.78, REML + CL2 0.94–0.95 (smooth process, G = 100) | β ratio 0.12–0.22 across families | Gaussian only | not testable |
| Between-curve dependence | not in the design (independent curves) | — | block-flip minus curve-flip coverage: REML + curve CL2 -1.8 to 2.1 pp; block NCV/CL2 MSE ratio 0.99–1.11 | plasmode only (imposed block dependence; curve-flip shortfalls are not evidence about it) |
| Other remedies for dependent residuals (pcre, GLS, curve bootstrap) | dependent, Gaussian: pcre β 0.93–0.97, GLS (FPCA) 0.93–0.97, REML + CL2 0.93–0.94 | β MSE pcre and GLS (FPCA) 1.41–2.53 times NCV’s | pcre β 0.59–0.86, GLS (FPCA) 0.60–0.86; with CL2 0.66–0.94 and 0.63–0.92; bootstrap percentile 0.93–0.97; REML + CL2 0.91–0.94 | pcre and GLS: synthetic coverage does not transfer (on real residual curves their estimates keep a bias that CL2 on the same fit does not remove); the curve bootstrap covers on both, at about 156 times the time of REML + CL2 |
| Intervals centred at the NCV fit with corrections | smoothing-parameter term: NCV + CL2 β 0.92–0.94 → 0.95 (Gaussian, Poisson, dependent); binary G = 40 0.86 | — | one-step bias correction + smoothing-parameter term: β 0.65–0.94 per dataset | does not transfer (smoothing bias of the NCV fit on real data remains) |
Evidence per row:
- Dependence strength. Model-based shortfall for β against DE: rho = 0.73, p = 0.03, n = 9; against DE/D: rho = 0.77, p = 0.02, n = 9. The dataset with the smallest DE, ECG 8-lead (DE 6.2, DE/D 0.06), still has a shortfall of 31.2 pp; the dataset furthest above the shortfall-on-log-DE trend is weather. REML + CL2 β coverage against DE: rho = -0.33, p = 0.37, n = 9 (counted only rho = -0.14, p = 0.79, n = 6).
- Dependence shape. ECG 8-lead’s residuals have the strongest sign-changing correlations of all datasets (minimum r -0.68) and the most extreme variance profile (SD ratio 42). The design effect 1’Σ1/trΣ is a grid-mean variance inflation that does not capture such features (see the sign-changing and misregistration cells above); REML + CL2 β coverage against the SD ratio is 0.80 (p = 0.01) over all nine datasets and 0.66 (p = 0.18) over the counted six, i.e. driven by the stress tests.
- Misregistration. The two cardiac datasets with real misregistration (ECG strain 63%, AF 43%) have REML + CL2 β coverage 0.944 and 0.938 and AR(1) coverage 0.874 and 0.838; AR(1)’s lowest real-data coverages are on weather and electricity (phase shares 6% and 0%).
- Number of curves. Paired within-dataset losses at G = 40 are in Table 87 and the synthetic ones in Table 80.
- Covariate rank. MID-truth β gain against rank: rho = 0.19, p = 0.63, n = 9; REML + CL2 β coverage against rank: rho = 0.67, p = 0.05, n = 9 (counted only rho = 0.38, p = 0.48, n = 6). Counted datasets ordered by REML + CL2 β coverage: running 0.919, DTI 0.927, AF trial 0.938, gait 0.939, ECG 8-lead 0.942, ECG strain 0.944.
- Truth roughness / EDF gap. MID-truth β gain against the EDF ratio: -0.33 (p = 0.39, n = 9). Datasets ordered by the MID-truth ratio (largest gain last): ECG strain 0.90 (EDF ratio 1.35); DTI 0.57 (EDF ratio 1.53); gait 0.55 (EDF ratio 1.13); running 0.53 (EDF ratio 0.98); weather 0.44 (EDF ratio 0.77); ocean 0.35 (EDF ratio 1.19); AF trial 0.27 (EDF ratio 1.31); ECG 8-lead 0.18 (EDF ratio 3.53); electricity 0.05 (EDF ratio 3.33). In Figure 22 (c) 5 of 9 datasets lie below the synthetic cells’ log-log trend.
- Signal. Shortfall against R²: rho = 0.55, p = 0.12, n = 9; MID gain against R²: rho = 0.50, p = 0.17, n = 9.
- Term type. Datasets with resolved α(t) or γ(t) gains under the NCV truth are listed in Section 10.4; the surface gain is the only one present on every dataset under the NCV and MID truths (9 of 9).
- Basis size. MID-truth β gains ordered: electricity 0.05, ECG 8-lead 0.18, AF trial 0.27, ocean 0.35, weather 0.44, running 0.53, gait 0.55, DTI 0.57, ECG strain 0.90; among the counted datasets ECG 8-lead ranks 1 of 6 (1 = largest gain).
- Other remedies. Per-cell results and CL2 on the GLS and pcre fits: Section 7, Section 7.1.
- NCV-centred corrections. Per-dataset results: Section 8, Section 9; all metrics of the main recipes side by side: Section 12.
- Between-curve dependence. Block-flip minus curve-flip coverages per recipe and dataset: ocean E(Y | X) bias-aware NCV (curve) -0.5 pp; ocean E(Y | X) REML + curve CL2 -0.3 pp; ocean beta(s,t) bias-aware NCV (curve) 0.9 pp; ocean beta(s,t) REML + curve CL2 1.0 pp; weather E(Y | X) bias-aware NCV (curve) -2.1 pp; weather E(Y | X) REML + curve CL2 -1.8 pp; weather beta(s,t) bias-aware NCV (curve) -0.6 pp; weather beta(s,t) REML + curve CL2 2.1 pp; electricity E(Y | X) bias-aware NCV (curve) -0.9 pp; electricity E(Y | X) REML + curve CL2 0.3 pp; electricity beta(s,t) bias-aware NCV (curve) -1.7 pp; electricity beta(s,t) REML + curve CL2 -0.7 pp.
14 Computing time
All times are elapsed seconds on one core with single-threaded BLAS (OpenBLAS, OPENBLAS_NUM_THREADS=1). The study runs used LRZ CoolMUC-4: the serial cluster (serial_std, one core per task) for the synthetic and plasmode studies, the comparators and the REML smoothing-parameter runs, and partly its cm4 nodes, which are slower per core, for the NCV smoothing-parameter runs. The recipe benchmark (analysis/timing-benchmark.R) fits REML, NCV and AR(1) to the same data sets, Gaussian, G = 100, D = 61, 10 replicates for each of the nine error processes of the AR(1) cells; its machine is not recorded (per core, the LRZ serial nodes and the development laptop, an Intel Core i7-8665U, run at about the same speed). “Per fit” and “per replicate” rows summarise individual fits; “cell median” rows summarise the per-cell median times stored with the study summaries, so their spread is across cells (families, G, D and datasets differ). The REML fit time excludes CL2; the CL2 rows are the covariance computation alone. No times were recorded for the one- and two-step bias corrections of the NCV fit (ncv-rbc/); each step is a linear map of the fit’s own matrices (Section 8). The hybrid and the bias-aware interval need both fits and their CL2 covariances, nothing else.
On identical data (Gaussian, G = 100) the median REML fit takes 2.0 s, the curve-blocked NCV fit 8.6 s and the AR(1) fit with ρ profiled 39 s (12 grid fits plus a refinement); a CL2 covariance adds 1.0 s. REML + CL2 therefore costs about 3 s per data set, NCV + CL2 about 10 s and the bias-aware interval (both fits, both covariances) about 12 s. Among the comparators the pcre fit has a median of 20 s on the plasmode cells with a long tail (90% quantile 265 s), the GLS fits take 2.9–3.7 s against 2.2–2.6 s for the REML fit in the same cells, and the curve bootstrap with 199 refits takes a median of 535 s per data set; CL2 on a GLS or pcre fit adds a median of 0.12–1.45 s. The curve-robust smoothing-parameter term adds a median of 1.2 s to a REML fit and 2.1–2.5 s to a Gaussian NCV fit; the exact curve jackknife it approximates takes 65 s (Table 78, Figure 23).
| component | setting | unit | hardware | n | median | q10 | q90 |
|---|---|---|---|---|---|---|---|
| REML fit | synthetic, Gaussian, G = 100, D = 61, 9 error processes | per fit | 1 core | 90 | 2.02 | 1.66 | 4.04 |
| NCV fit (curve blocks) | synthetic, Gaussian, G = 100, D = 61, 9 error processes | per fit | 1 core | 90 | 8.56 | 5.92 | 13.62 |
| AR(1) fit (ρ profiled) | synthetic, Gaussian, G = 100, D = 61, 9 error processes | per fit | 1 core | 90 | 39.05 | 32.06 | 51.54 |
| CL2 covariance of the REML fit | synthetic, Gaussian, G = 100, D = 61, 9 error processes | per fit | 1 core | 90 | 0.95 | 0.78 | 1.27 |
| CL2 covariance of the NCV fit | synthetic, Gaussian, G = 100, D = 61, 9 error processes | per fit | 1 core | 90 | 0.95 | 0.78 | 1.29 |
| REML fit | synthetic study, all cells | cell median | LRZ serial node, 1 core | 123 | 3.06 | 1.39 | 18.30 |
| NCV fit (curve blocks) | synthetic study, all cells | cell median | LRZ serial node, 1 core | 123 | 11.24 | 4.48 | 58.04 |
| NCV fit (pointwise) | synthetic study, Gaussian G = 100 | cell median | LRZ serial node, 1 core | 3 | 15.39 | 13.81 | 18.12 |
| AR(1) fit (ρ profiled) | synthetic study, AR(1) cells | cell median | LRZ serial node, 1 core | 9 | 49.49 | 45.80 | 51.83 |
| REML fit | plasmode study, all cells | cell median | LRZ serial node, 1 core | 130 | 3.27 | 1.23 | 9.18 |
| NCV fit (curve blocks) | plasmode study, all cells | cell median | LRZ serial node, 1 core | 130 | 10.34 | 5.25 | 24.09 |
| NCV fit (block neighbourhoods) | plasmode stress tests, block arm | cell median | LRZ serial node, 1 core | 26 | 13.60 | 12.28 | 24.29 |
| AR(1) fit (ρ profiled) | plasmode study, AR(1) cells | cell median | LRZ serial node, 1 core | 12 | 48.49 | 39.63 | 153.30 |
| CL2 covariance of the REML fit | plasmode study, all cells | cell median | LRZ serial node, 1 core | 130 | 0.89 | 0.44 | 3.17 |
| CL2 covariance of the NCV fit | plasmode study, all cells | cell median | LRZ serial node, 1 core | 130 | 0.89 | 0.44 | 3.17 |
| block CL2 covariance of the block-NCV fit | plasmode stress tests, block arm | cell median | LRZ serial node, 1 core | 26 | 4.26 | 3.72 | 8.87 |
| REML fit | synthetic comparator cells (Gaussian core, G = 40, 100) | per replicate | LRZ serial node, 1 core | 1200 | 2.20 | 0.97 | 4.60 |
| pcre fit (incl. FPCA of the residuals) | synthetic comparator cells (Gaussian core, G = 40, 100) | per replicate | LRZ serial node, 1 core | 1200 | 13.47 | 2.82 | 229.44 |
| GLS fit (FPCA covariance) | synthetic comparator cells (Gaussian core, G = 40, 100) | per replicate | LRZ serial node, 1 core | 1200 | 2.97 | 1.34 | 4.73 |
| GLS fit (raw covariance) | synthetic comparator cells (Gaussian core, G = 40, 100) | per replicate | LRZ serial node, 1 core | 1200 | 2.87 | 1.37 | 4.17 |
| curve bootstrap (199 REML refits) | synthetic comparator cells (Gaussian core, G = 40, 100) | per replicate | LRZ serial node, 1 core | 600 | 450.27 | 240.81 | 867.99 |
| CL2 covariance of the REML fit | synthetic comparator cells (Gaussian core, G = 40, 100) | per replicate | LRZ serial node, 1 core | 1200 | 0.99 | 0.43 | 1.38 |
| CL2 on the GLS fit (FPCA covariance) | synthetic comparator cells (Gaussian core, G = 40, 100) | per replicate | LRZ serial node, 1 core | 1200 | 0.13 | 0.06 | 0.20 |
| CL2 on the GLS fit (raw covariance) | synthetic comparator cells (Gaussian core, G = 40, 100) | per replicate | LRZ serial node, 1 core | 1200 | 0.14 | 0.06 | 0.20 |
| CL2 on the pcre fit | synthetic comparator cells (Gaussian core, G = 40, 100) | per replicate | LRZ serial node, 1 core | 600 | 1.29 | 0.60 | 4.12 |
| REML fit | plasmode comparator cells (6 datasets, G = all, 40) | per replicate | LRZ serial node, 1 core | 2400 | 2.56 | 1.00 | 9.05 |
| pcre fit (incl. FPCA of the residuals) | plasmode comparator cells (6 datasets, G = all, 40) | per replicate | LRZ serial node, 1 core | 2400 | 20.22 | 4.30 | 264.64 |
| GLS fit (FPCA covariance) | plasmode comparator cells (6 datasets, G = all, 40) | per replicate | LRZ serial node, 1 core | 2400 | 3.40 | 1.54 | 10.42 |
| GLS fit (raw covariance) | plasmode comparator cells (6 datasets, G = all, 40) | per replicate | LRZ serial node, 1 core | 2400 | 3.66 | 1.59 | 11.79 |
| curve bootstrap (199 REML refits) | plasmode comparator cells (6 datasets, G = all, 40) | per replicate | LRZ serial node, 1 core | 1198 | 534.53 | 225.66 | 1919.57 |
| CL2 covariance of the REML fit | plasmode comparator cells (6 datasets, G = all, 40) | per replicate | LRZ serial node, 1 core | 2400 | 0.85 | 0.40 | 2.39 |
| CL2 on the GLS fit (FPCA covariance) | plasmode comparator cells (6 datasets, G = all, 40) | per replicate | LRZ serial node, 1 core | 2400 | 0.12 | 0.06 | 0.40 |
| CL2 on the GLS fit (raw covariance) | plasmode comparator cells (6 datasets, G = all, 40) | per replicate | LRZ serial node, 1 core | 2400 | 0.12 | 0.06 | 0.40 |
| CL2 on the pcre fit | plasmode comparator cells (6 datasets, G = all, 40) | per replicate | LRZ serial node, 1 core | 1199 | 1.45 | 0.56 | 4.59 |
| λ term for REML (derivatives + infinitesimal jackknife) | synthetic core cells, default signal (G = 40, 100) | per replicate | LRZ serial node, 1 core | 3600 | 1.16 | 0.46 | 1.53 |
| λ term for REML, one-step leave-curve-out variant | synthetic core cells, default signal (G = 40, 100) | per replicate | LRZ serial node, 1 core | 3600 | 2.50 | 0.71 | 3.18 |
| exact curve jackknife of the REML smoothing parameters (reference) | synthetic core cells, default signal (G = 40, 100) | per replicate | LRZ serial node, 1 core | 3600 | 64.84 | 15.09 | 130.84 |
| λ term for REML (derivatives + infinitesimal jackknife) | plasmode study, all cells | per replicate | LRZ serial or cm4 node, 1 core | 26000 | 1.17 | 0.50 | 3.48 |
| λ term for NCV (derivatives, Hessian, curve scores) | synthetic core cells, Gaussian | per replicate | LRZ serial or cm4 node, 1 core | 1200 | 2.10 | 0.98 | 3.95 |
| λ term for NCV (derivatives, Hessian, curve scores) | synthetic core cells, Poisson and binary | per replicate | LRZ cm4 node, 1 core | 2400 | 5.32 | 2.17 | 12.27 |
| λ term for NCV (derivatives, Hessian, curve scores) | plasmode study, all cells | per replicate | LRZ serial or cm4 node, 1 core | 13000 | 2.48 | 1.00 | 8.61 |
15 Limitations
Error law of the plasmode. Sign-flipped frozen residual curves reproduce each subject’s empirical within-curve covariance exactly and nothing else: the error law is a two-point mixture, symmetric by construction, with no between-subject variation in the residual shapes beyond the observed ones. Residuals are attached to the subject’s own covariates; on the two datasets run both attached and detached, the two versions gave the same decisions.
Frozen truths favour their own selector. A truth taken from the REML fit is REML-shaped, one from the NCV fit is NCV-shaped, and MID (log-midpoint smoothing parameters) is interior for every term only on 3 of 9 datasets. Changing the truth source changes the truth’s shape and signal as well as its smoothness, so the plasmode estimation results are stated per truth source; the synthetic study is the only place where the truth is independent of both selectors.
Representable truths. All synthetic truths of the main design are projected onto the fitted bases, so approximation bias is absent by design; the rough-truth cells add roughness that the bases can represent but the penalties shrink. Truths outside the spline space are studied only in the misspecification experiment (Section 6: Gaussian, three truths, two basis sizes).
Precision. 200 replicates give stored MC SEs of the grid-average coverage of 0.21–0.33 pp (REML + CL2) to 0.42–0.86 pp (model-based) in the dependent synthetic cells and 0.17–1.95 pp for REML + CL2 in the plasmode cells; a single grid point’s coverage has a binomial SE of 1.5 pp near 0.95. A 2 pp undercoverage in one cell is therefore resolved for grid averages but the rule’s calibration error pools cells and subtracts the MC variance, and several plasmode classes are “unresolved” because their bootstrap intervals straddle the 2 pp threshold.
The calibration error. The undercoverage-only CE selects the undercovering cells by their estimated coverage before subtracting the MC variance, so the variance correction is approximate (cells that undercover by chance are counted, cells that overcover by chance are not).
Two calibration criteria. The undercoverage-only and the symmetric criterion are both reported. Under both, the plasmode decision is inconsistent across datasets for β.
Selected datasets. The nine datasets were chosen to spread design effect, covariate rank and phase variability, not sampled; three have dependent curves and are stress tests; all are Gaussian; the ECG 8-lead model uses larger bases than the others; the running data keep one condition per person with the other conditions’ variation in the residuals; the mean estimand is scored on a fixed cohort of 50 subjects. The cardiology datasets are not public.
Diagnostics. The realized synthetic design effects are means over 20 replicates per cell (analysis/describe-cells.R), without an uncertainty estimate. The studentised-error diagnostics (writeup/lrz-zstats.R) cover all 123 synthetic cells and all plasmode cells, for β everywhere and for the mean in Gaussian cells.
Scope. Simultaneous bands, G < 40, non-Gaussian real data, native-grid density on real data, dependence models beyond AR(1), pcre and GLS (Section 7) and the finite-sample calibration error of the sandwich pivot itself are outside both studies.
16 Supplementary tables
All tables are also written as CSV files to writeup/tables/; the per-cell tables (S1–S5) are the full study, the tables in the main text are views of them. Column abbreviations: cov = grid-average coverage, q05 = 5% quantile of pointwise coverage, IS = interval score, CE = calibration error.
16.1 Detail tables referenced in the main text
| dependence | G | estimand | arm | q05_min | q05_median | coverage_min |
|---|---|---|---|---|---|---|
| dependent | 40 | beta(s,t) | NCV + CL2 | 0.600 | 0.861 | 0.785 |
| dependent | 40 | beta(s,t) | NCV + CL2, bias-aware | 0.885 | 0.950 | 0.948 |
| dependent | 40 | beta(s,t) | REML + CL2 | 0.891 | 0.897 | 0.923 |
| dependent | 40 | beta(s,t) | REML, model-based | 0.520 | 0.580 | 0.609 |
| dependent | 40 | E(Y | X) | NCV + CL2 | 0.685 | 0.827 | 0.842 |
| dependent | 40 | E(Y | X) | NCV + CL2, bias-aware | 0.860 | 0.907 | 0.930 |
| dependent | 40 | E(Y | X) | REML + CL2 | 0.887 | 0.900 | 0.921 |
| dependent | 40 | E(Y | X) | REML, model-based | 0.515 | 0.560 | 0.601 |
| dependent | 100 | beta(s,t) | NCV + CL2 | 0.775 | 0.861 | 0.896 |
| dependent | 100 | beta(s,t) | NCV + CL2, bias-aware | 0.920 | 0.950 | 0.971 |
| dependent | 100 | beta(s,t) | REML + CL2 | 0.911 | 0.915 | 0.940 |
| dependent | 100 | beta(s,t) | REML, model-based | 0.540 | 0.596 | 0.653 |
| dependent | 100 | E(Y | X) | NCV + CL2 | 0.740 | 0.863 | 0.876 |
| dependent | 100 | E(Y | X) | NCV + CL2, bias-aware | 0.865 | 0.920 | 0.933 |
| dependent | 100 | E(Y | X) | REML + CL2 | 0.910 | 0.915 | 0.940 |
| dependent | 100 | E(Y | X) | REML, model-based | 0.535 | 0.570 | 0.631 |
| independent | 40 | beta(s,t) | NCV + CL2 | 0.925 | 0.945 | 0.973 |
| independent | 40 | beta(s,t) | NCV + CL2, bias-aware | 0.930 | 0.960 | 0.979 |
| independent | 40 | beta(s,t) | REML + CL2 | 0.915 | 0.965 | 0.973 |
| independent | 40 | beta(s,t) | REML, model-based | 0.945 | 0.975 | 0.984 |
| independent | 40 | E(Y | X) | NCV + CL2 | 0.835 | 0.900 | 0.923 |
| independent | 40 | E(Y | X) | NCV + CL2, bias-aware | 0.850 | 0.905 | 0.935 |
| independent | 40 | E(Y | X) | REML + CL2 | 0.825 | 0.925 | 0.925 |
| independent | 40 | E(Y | X) | REML, model-based | 0.875 | 0.940 | 0.952 |
| independent | 100 | beta(s,t) | NCV + CL2 | 0.925 | 0.940 | 0.976 |
| independent | 100 | beta(s,t) | NCV + CL2, bias-aware | 0.945 | 0.951 | 0.986 |
| independent | 100 | beta(s,t) | REML + CL2 | 0.955 | 0.965 | 0.979 |
| independent | 100 | beta(s,t) | REML, model-based | 0.965 | 0.970 | 0.983 |
| independent | 100 | E(Y | X) | NCV + CL2 | 0.860 | 0.900 | 0.939 |
| independent | 100 | E(Y | X) | NCV + CL2, bias-aware | 0.870 | 0.910 | 0.945 |
| independent | 100 | E(Y | X) | REML + CL2 | 0.885 | 0.930 | 0.949 |
| independent | 100 | E(Y | X) | REML, model-based | 0.910 | 0.935 | 0.962 |
| dependence | family | estimand | arm | G = 40 | G = 100 | loss at G = 40 |
|---|---|---|---|---|---|---|
| dependent | Gaussian | E(Y | X) | REML, model-based | 0.615 | 0.640 | 0.025 |
| dependent | Gaussian | E(Y | X) | REML + CL2 | 0.931 | 0.943 | 0.012 |
| dependent | Gaussian | E(Y | X) | NCV + CL2 | 0.912 | 0.928 | 0.016 |
| dependent | Gaussian | E(Y | X) | NCV + CL2, bias-aware | 0.954 | 0.957 | 0.003 |
| dependent | Gaussian | E(Y | X) | NCV + CL2 (freq.), bias-aware | 0.949 | 0.953 | 0.004 |
| dependent | Gaussian | beta(s,t) | REML, model-based | 0.625 | 0.663 | 0.038 |
| dependent | Gaussian | beta(s,t) | REML + CL2 | 0.926 | 0.942 | 0.016 |
| dependent | Gaussian | beta(s,t) | NCV + CL2 | 0.918 | 0.932 | 0.014 |
| dependent | Gaussian | beta(s,t) | NCV + CL2, bias-aware | 0.982 | 0.983 | 0.001 |
| dependent | Gaussian | beta(s,t) | NCV + CL2 (freq.), bias-aware | 0.976 | 0.977 | 0.001 |
| dependent | Poisson | E(Y | X) | REML, model-based | 0.657 | 0.663 | 0.005 |
| dependent | Poisson | E(Y | X) | REML + CL2 | 0.932 | 0.943 | 0.011 |
| dependent | Poisson | E(Y | X) | NCV + CL2 | 0.908 | 0.925 | 0.017 |
| dependent | Poisson | E(Y | X) | NCV + CL2, bias-aware | 0.950 | 0.955 | 0.005 |
| dependent | Poisson | E(Y | X) | NCV + CL2 (freq.), bias-aware | 0.943 | 0.950 | 0.006 |
| dependent | Poisson | beta(s,t) | REML, model-based | 0.682 | 0.695 | 0.013 |
| dependent | Poisson | beta(s,t) | REML + CL2 | 0.928 | 0.943 | 0.014 |
| dependent | Poisson | beta(s,t) | NCV + CL2 | 0.923 | 0.933 | 0.010 |
| dependent | Poisson | beta(s,t) | NCV + CL2, bias-aware | 0.980 | 0.982 | 0.002 |
| dependent | Poisson | beta(s,t) | NCV + CL2 (freq.), bias-aware | 0.973 | 0.975 | 0.002 |
| dependent | binary | E(Y | X) | REML, model-based | 0.731 | 0.742 | 0.011 |
| dependent | binary | E(Y | X) | REML + CL2 | 0.925 | 0.942 | 0.017 |
| dependent | binary | E(Y | X) | NCV + CL2 | 0.861 | 0.895 | 0.034 |
| dependent | binary | E(Y | X) | NCV + CL2, bias-aware | 0.937 | 0.942 | 0.005 |
| dependent | binary | E(Y | X) | NCV + CL2 (freq.), bias-aware | 0.927 | 0.932 | 0.005 |
| dependent | binary | beta(s,t) | REML, model-based | 0.782 | 0.800 | 0.018 |
| dependent | binary | beta(s,t) | REML + CL2 | 0.934 | 0.951 | 0.018 |
| dependent | binary | beta(s,t) | NCV + CL2 | 0.844 | 0.913 | 0.069 |
| dependent | binary | beta(s,t) | NCV + CL2, bias-aware | 0.962 | 0.976 | 0.014 |
| dependent | binary | beta(s,t) | NCV + CL2 (freq.), bias-aware | 0.950 | 0.965 | 0.014 |
| cell | family | error | signal | truth | estimand | NCV + CL2 | NCV + CL2, bias-aware | NCV + CL2 (freq.), bias-aware | REML + CL2 |
|---|---|---|---|---|---|---|---|---|---|
| 42 | binary | smooth | high | smooth | beta(s,t) | 0.925 | 0.980 | 0.970 | 0.952 |
| 77 | binary | smooth | high | wiggly | beta(s,t) | 0.803 | 0.931 | 0.912 | 0.947 |
| 36 | binary | smooth | mid | smooth | beta(s,t) | 0.896 | 0.972 | 0.962 | 0.952 |
| 75 | binary | smooth | mid | wiggly | beta(s,t) | 0.799 | 0.940 | 0.923 | 0.948 |
| 38 | binary | iid | high | smooth | beta(s,t) | 0.980 | 0.987 | 0.928 | 0.989 |
| 76 | binary | iid | high | wiggly | beta(s,t) | 0.919 | 0.929 | 0.820 | 0.944 |
| 32 | binary | iid | mid | smooth | beta(s,t) | 0.982 | 0.987 | 0.911 | 0.990 |
| 74 | binary | iid | mid | wiggly | beta(s,t) | 0.911 | 0.921 | 0.784 | 0.935 |
| 18 | Gaussian | smooth | high | smooth | beta(s,t) | 0.938 | 0.984 | 0.978 | 0.943 |
| 69 | Gaussian | smooth | high | wiggly | beta(s,t) | 0.900 | 0.950 | 0.940 | 0.942 |
| 6 | Gaussian | smooth | low | smooth | beta(s,t) | 0.925 | 0.983 | 0.977 | 0.944 |
| 65 | Gaussian | smooth | low | wiggly | beta(s,t) | 0.803 | 0.945 | 0.934 | 0.942 |
| 12 | Gaussian | smooth | mid | smooth | beta(s,t) | 0.934 | 0.983 | 0.978 | 0.944 |
| 67 | Gaussian | smooth | mid | wiggly | beta(s,t) | 0.824 | 0.935 | 0.924 | 0.941 |
| 14 | Gaussian | iid | high | smooth | beta(s,t) | 0.979 | 0.990 | 0.959 | 0.979 |
| 68 | Gaussian | iid | high | wiggly | beta(s,t) | 0.954 | 0.959 | 0.916 | 0.963 |
| 2 | Gaussian | iid | low | smooth | beta(s,t) | 0.982 | 0.989 | 0.939 | 0.989 |
| 64 | Gaussian | iid | low | wiggly | beta(s,t) | 0.933 | 0.938 | 0.843 | 0.948 |
| 8 | Gaussian | iid | mid | smooth | beta(s,t) | 0.982 | 0.989 | 0.951 | 0.986 |
| 66 | Gaussian | iid | mid | wiggly | beta(s,t) | 0.953 | 0.955 | 0.894 | 0.961 |
| 30 | Poisson | smooth | high | smooth | beta(s,t) | 0.929 | 0.980 | 0.973 | 0.944 |
| 73 | Poisson | smooth | high | wiggly | beta(s,t) | 0.860 | 0.935 | 0.924 | 0.940 |
| 24 | Poisson | smooth | mid | smooth | beta(s,t) | 0.935 | 0.984 | 0.978 | 0.945 |
| 71 | Poisson | smooth | mid | wiggly | beta(s,t) | 0.811 | 0.932 | 0.920 | 0.943 |
| 26 | Poisson | iid | high | smooth | beta(s,t) | 0.976 | 0.986 | 0.948 | 0.979 |
| 72 | Poisson | iid | high | wiggly | beta(s,t) | 0.950 | 0.955 | 0.903 | 0.961 |
| 20 | Poisson | iid | mid | smooth | beta(s,t) | 0.982 | 0.990 | 0.949 | 0.986 |
| 70 | Poisson | iid | mid | wiggly | beta(s,t) | 0.952 | 0.955 | 0.889 | 0.960 |
| cell | error | G | estimand | NCV + CL2 | NCV + CL2, bias-aware | REML + CL2 | REML, model-based | AR(1) working model |
|---|---|---|---|---|---|---|---|---|
| 11 | smooth | 40 | beta(s,t) | 0.924 | 0.983 | 0.926 | 0.615 | |
| 11 | smooth | 40 | gamma(t) | 0.927 | 0.946 | 0.946 | 0.621 | |
| 11 | smooth | 40 | E(Y | X) | 0.922 | 0.959 | 0.931 | 0.606 | |
| 12 | smooth | 100 | beta(s,t) | 0.934 | 0.983 | 0.944 | 0.657 | 0.993 |
| 12 | smooth | 100 | gamma(t) | 0.935 | 0.946 | 0.948 | 0.655 | 0.967 |
| 12 | smooth | 100 | E(Y | X) | 0.929 | 0.959 | 0.944 | 0.634 | 0.972 |
| 102 | var(z) | 40 | beta(s,t) | 0.925 | 0.985 | 0.939 | 0.619 | |
| 102 | var(z) | 40 | gamma(t) | 0.898 | 0.931 | 0.926 | 0.490 | |
| 102 | var(z) | 40 | E(Y | X) | 0.912 | 0.955 | 0.934 | 0.588 | |
| 103 | var(z) | 100 | beta(s,t) | 0.935 | 0.984 | 0.949 | 0.660 | 0.992 |
| 103 | var(z) | 100 | gamma(t) | 0.920 | 0.945 | 0.938 | 0.494 | 0.879 |
| 103 | var(z) | 100 | E(Y | X) | 0.923 | 0.955 | 0.944 | 0.606 | 0.949 |
| 100 | var(t) | 40 | beta(s,t) | 0.918 | 0.979 | 0.926 | 0.619 | |
| 100 | var(t) | 40 | gamma(t) | 0.925 | 0.945 | 0.944 | 0.632 | |
| 100 | var(t) | 40 | E(Y | X) | 0.919 | 0.957 | 0.932 | 0.614 | |
| 101 | var(t) | 100 | beta(s,t) | 0.933 | 0.983 | 0.944 | 0.662 | 0.991 |
| 101 | var(t) | 100 | gamma(t) | 0.932 | 0.943 | 0.947 | 0.661 | 0.967 |
| 101 | var(t) | 100 | E(Y | X) | 0.928 | 0.959 | 0.944 | 0.642 | 0.972 |
| 104 | var(t,z) | 40 | beta(s,t) | 0.924 | 0.983 | 0.940 | 0.629 | |
| 104 | var(t,z) | 40 | gamma(t) | 0.893 | 0.930 | 0.925 | 0.502 | |
| 104 | var(t,z) | 40 | E(Y | X) | 0.911 | 0.955 | 0.936 | 0.596 | |
| 105 | var(t,z) | 100 | beta(s,t) | 0.934 | 0.984 | 0.950 | 0.666 | 0.990 |
| 105 | var(t,z) | 100 | gamma(t) | 0.920 | 0.944 | 0.940 | 0.510 | 0.889 |
| 105 | var(t,z) | 100 | E(Y | X) | 0.922 | 0.955 | 0.944 | 0.615 | 0.951 |
| cell | family | error | estimand | AR(1) working model | NCV + CL2 | NCV + CL2, bias-aware | REML + CL2 | REML, model-based |
|---|---|---|---|---|---|---|---|---|
| 98 | binary | sign-chg. | beta(s,t) | 0.927 | 0.977 | 0.959 | 0.885 | |
| 98 | binary | sign-chg. | E(Y | X) | 0.882 | 0.945 | 0.943 | 0.794 | |
| 96 | Gaussian | sign-chg. | beta(s,t) | 0.992 | 0.935 | 0.985 | 0.942 | 0.700 |
| 96 | Gaussian | sign-chg. | E(Y | X) | 0.963 | 0.915 | 0.957 | 0.942 | 0.646 |
| 97 | Poisson | sign-chg. | beta(s,t) | 0.935 | 0.986 | 0.942 | 0.728 | |
| 97 | Poisson | sign-chg. | E(Y | X) | 0.905 | 0.951 | 0.941 | 0.670 |
| family | error | estimand | rich | low-rank |
|---|---|---|---|---|
| binary | smooth | alpha(t) | 0.88 [0.84, 0.93] | 0.87 [0.82, 0.91] |
| binary | smooth | beta(s,t) | 0.17 [0.14, 0.20] | 0.12 [0.08, 0.17] |
| binary | smooth | gamma(t) | 0.96 [0.91, 1.03] | 0.95 [0.90, 1.01] |
| binary | smooth | E(Y | X) | 0.74 [0.71, 0.77] | 0.77 [0.74, 0.80] |
| binary | iid | alpha(t) | 1.10 [1.05, 1.17] | 1.06 [1.03, 1.11] |
| binary | iid | beta(s,t) | 1.06 [0.95, 1.19] | 1.26 [1.03, 1.56] |
| binary | iid | gamma(t) | 1.05 [1.03, 1.09] | 1.07 [1.04, 1.10] |
| binary | iid | E(Y | X) | 1.06 [1.04, 1.08] | 1.07 [1.05, 1.09] |
| Gaussian | smooth | alpha(t) | 0.97 [0.94, 0.99] | 0.97 [0.94, 0.99] |
| Gaussian | smooth | beta(s,t) | 0.16 [0.13, 0.19] | 0.09 [0.06, 0.11] |
| Gaussian | smooth | gamma(t) | 0.95 [0.92, 0.97] | 0.95 [0.93, 0.97] |
| Gaussian | smooth | E(Y | X) | 0.69 [0.67, 0.71] | 0.70 [0.68, 0.72] |
| Gaussian | iid | alpha(t) | 1.04 [1.02, 1.07] | 1.04 [1.02, 1.07] |
| Gaussian | iid | beta(s,t) | 0.65 [0.62, 0.68] | 0.71 [0.62, 0.82] |
| Gaussian | iid | gamma(t) | 1.04 [1.02, 1.07] | 1.04 [1.02, 1.07] |
| Gaussian | iid | E(Y | X) | 0.96 [0.95, 0.97] | 0.97 [0.96, 0.99] |
| Poisson | smooth | alpha(t) | 0.94 [0.90, 0.97] | 0.93 [0.90, 0.97] |
| Poisson | smooth | beta(s,t) | 0.16 [0.13, 0.19] | 0.09 [0.06, 0.12] |
| Poisson | smooth | gamma(t) | 0.98 [0.95, 1.01] | 0.98 [0.95, 1.01] |
| Poisson | smooth | E(Y | X) | 0.68 [0.64, 0.71] | 0.69 [0.65, 0.72] |
| Poisson | iid | alpha(t) | 1.04 [1.01, 1.07] | 1.03 [1.00, 1.06] |
| Poisson | iid | beta(s,t) | 0.71 [0.65, 0.78] | 0.73 [0.63, 0.84] |
| Poisson | iid | gamma(t) | 1.07 [1.04, 1.10] | 1.07 [1.04, 1.10] |
| Poisson | iid | E(Y | X) | 0.98 [0.96, 1.00] | 0.99 [0.97, 1.01] |
| dataset | alpha(t): REML / NCV / MID | ff REML / NCV / MID | gamma(t): REML / NCV / MID | total REML / NCV / MID |
|---|---|---|---|---|
| ECG strain | 9.5 / 9.0 / 9.2 | 59.6 / 44.0 / 52.9 | 8.0 / 1.0 / 1.2 | 77.1 / 54.1 / 63.4 |
| AF trial | 5.8 / 3.6 / 4.7 | 33.2 / 25.3 / 29.5 | 5.7 / 3.9 / 4.8 | 44.7 / 32.8 / 38.9 |
| running | 8.0 / 7.0 / 7.5 | 53.7 / 54.8 / 60.4 | 6.4 / 4.1 / 5.2 | 68.0 / 65.9 / 73.1 |
| DTI | 9.7 / 10.9 / 10.5 | 53.8 / 35.1 / 47.0 | 4.9 / 2.0 / 3.2 | 68.5 / 47.9 / 60.7 |
| gait | 10.9 / 11.0 / 11.0 | 68.7 / 60.8 / 65.3 | 9.4 / 8.3 / 8.9 | 89.0 / 80.1 / 85.2 |
| ECG 8-lead | 36.6 / 37.9 / 37.3 | 39.4 / 11.2 / 23.9 | 27.2 / 28.0 / 27.7 | 103.2 / 77.0 / 88.9 |
| ocean | 8.6 / 6.5 / 7.7 | 59.2 / 49.8 / 60.3 | 6.6 / 2.5 / 4.2 | 74.3 / 58.7 / 72.1 |
| weather | 7.9 / 10.6 / 9.4 | 29.5 / 38.6 / 41.9 | 8.0 / 11.6 / 10.2 | 45.4 / 60.8 / 61.5 |
| electricity | 10.1 / 11.0 / 10.8 | 33.2 / 10.0 / 23.9 | 2.5 / 1.0 / 1.0 | 45.8 / 22.0 / 35.7 |
| app | role | estimand | q05 mean: ncv_cl2 | q05 mean: ncv_cl2_bias | q05 mean: reml_cl2 | q05 mean: reml_model | q05 min: ncv_cl2 | q05 min: ncv_cl2_bias | q05 min: reml_cl2 | q05 min: reml_model |
|---|---|---|---|---|---|---|---|---|---|---|
| ECG strain | counted | beta(s,t) | 0.809 | 0.862 | 0.901 | 0.219 | 0.775 | 0.840 | 0.895 | 0.151 |
| ECG strain | counted | E(Y | X) | 0.820 | 0.876 | 0.888 | 0.238 | 0.765 | 0.850 | 0.880 | 0.190 |
| AF trial | counted | beta(s,t) | 0.485 | 0.772 | 0.879 | 0.363 | 0.455 | 0.745 | 0.870 | 0.360 |
| AF trial | counted | E(Y | X) | 0.782 | 0.856 | 0.896 | 0.385 | 0.760 | 0.845 | 0.895 | 0.385 |
| running | counted | beta(s,t) | 0.441 | 0.766 | 0.873 | 0.268 | 0.300 | 0.690 | 0.860 | 0.255 |
| running | counted | E(Y | X) | 0.770 | 0.875 | 0.887 | 0.268 | 0.655 | 0.860 | 0.880 | 0.260 |
| DTI | counted | beta(s,t) | 0.586 | 0.806 | 0.876 | 0.475 | 0.316 | 0.715 | 0.845 | 0.445 |
| DTI | counted | E(Y | X) | 0.789 | 0.871 | 0.892 | 0.469 | 0.705 | 0.840 | 0.890 | 0.465 |
| gait | counted | beta(s,t) | 0.547 | 0.833 | 0.903 | 0.231 | 0.512 | 0.810 | 0.900 | 0.225 |
| gait | counted | E(Y | X) | 0.833 | 0.895 | 0.910 | 0.243 | 0.830 | 0.895 | 0.910 | 0.240 |
| ECG 8-lead | counted | beta(s,t) | 0.449 | 0.840 | 0.873 | 0.282 | 0.301 | 0.740 | 0.870 | 0.265 |
| ECG 8-lead | counted | E(Y | X) | 0.714 | 0.887 | 0.907 | 0.329 | 0.635 | 0.860 | 0.905 | 0.325 |
| ocean | stress test | beta(s,t) | 0.281 | 0.794 | 0.887 | 0.224 | 0.055 | 0.720 | 0.880 | 0.210 |
| ocean | stress test | E(Y | X) | 0.815 | 0.891 | 0.899 | 0.234 | 0.755 | 0.880 | 0.895 | 0.230 |
| weather | stress test | beta(s,t) | 0.415 | 0.715 | 0.833 | 0.233 | 0.252 | 0.670 | 0.796 | 0.200 |
| weather | stress test | E(Y | X) | 0.728 | 0.809 | 0.865 | 0.342 | 0.662 | 0.785 | 0.835 | 0.335 |
| electricity | stress test | beta(s,t) | 0.398 | 0.835 | 0.881 | 0.266 | 0.075 | 0.695 | 0.820 | 0.230 |
| electricity | stress test | E(Y | X) | 0.756 | 0.899 | 0.887 | 0.295 | 0.505 | 0.830 | 0.880 | 0.285 |
| app | role | estimand | NCV + CL2 | NCV + CL2, bias-aware | REML + CL2 | REML, model-based |
|---|---|---|---|---|---|---|
| ECG strain | counted | beta | 0.035 | 0.006 | 0.010 | 0.019 |
| ECG strain | counted | mean | 0.019 | 0.003 | 0.008 | 0.016 |
| AF trial | counted | beta | 0.186 | 0.031 | 0.017 | 0.045 |
| AF trial | counted | mean | 0.047 | 0.009 | 0.012 | 0.034 |
| running | counted | beta | 0.124 | 0.013 | 0.017 | 0.060 |
| running | counted | mean | 0.030 | 0.002 | 0.011 | 0.044 |
| DTI | counted | beta | 0.089 | 0.018 | 0.010 | 0.013 |
| DTI | counted | mean | 0.044 | 0.013 | 0.014 | 0.019 |
| gait | counted | beta | 0.185 | 0.020 | 0.004 | 0.048 |
| gait | counted | mean | 0.058 | 0.005 | 0.007 | 0.035 |
| ECG 8-lead | counted | beta | 0.083 | 0.003 | 0.006 | 0.002 |
| ECG 8-lead | counted | mean | 0.031 | 0.006 | 0.008 | 0.022 |
| ocean | stress test | beta | 0.141 | 0.010 | -0.004 | 0.032 |
| ocean | stress test | mean | 0.024 | -0.003 | 0.003 | 0.028 |
| weather | stress test | beta | 0.213 | 0.059 | 0.047 | 0.038 |
| weather | stress test | mean | 0.041 | 0.007 | 0.008 | 0.038 |
| electricity | stress test | beta | 0.016 | 0.008 | 0.014 | 0.072 |
| electricity | stress test | mean | 0.004 | 0.006 | 0.005 | 0.052 |
| app | role | estimand | n_cells | ce_proposal_under | ce_fallback_under | ce_proposal_sym | ce_fallback_sym | score_ratio | decision_under | decision_sym |
|---|---|---|---|---|---|---|---|---|---|---|
| ECG strain | counted | beta | 12 | 0.019 | 0.016 | 0.019 | 0.016 | 0.943 | P | P |
| AF trial | counted | beta | 6 | 0.067 | 0.029 | 0.067 | 0.029 | 0.906 | N | N |
| running | counted | beta | 6 | 0.082 | 0.048 | 0.082 | 0.048 | 1.014 | N | N |
| DTI | counted | beta | 6 | 0.048 | 0.033 | 0.048 | 0.033 | 0.962 | N | N |
| gait | counted | beta | 6 | 0.046 | 0.014 | 0.046 | 0.014 | 0.983 | F | F |
| ECG 8-lead | counted | beta | 6 | 0.031 | 0.014 | 0.032 | 0.014 | 0.769 | F | F |
| ocean | stress test | beta | 6 | 0.056 | 0.021 | 0.056 | 0.021 | 0.953 | N | N |
| weather | stress test | beta | 6 | 0.182 | 0.120 | 0.182 | 0.120 | 1.172 | N | N |
| electricity | stress test | beta | 6 | 0.107 | 0.048 | 0.109 | 0.048 | 0.901 | N | N |
| ECG strain | counted | mean | 12 | 0.011 | 0.014 | 0.011 | 0.014 | 0.990 | E | E |
| AF trial | counted | mean | 6 | 0.023 | 0.016 | 0.023 | 0.016 | 0.995 | F | F |
| running | counted | mean | 6 | 0.019 | 0.028 | 0.019 | 0.028 | 0.986 | P | P |
| DTI | counted | mean | 6 | 0.025 | 0.032 | 0.025 | 0.032 | 0.938 | N | N |
| gait | counted | mean | 6 | 0.011 | 0.014 | 0.011 | 0.014 | 1.001 | E | E |
| ECG 8-lead | counted | mean | 6 | 0.003 | 0.006 | 0.005 | 0.006 | 0.904 | P | P |
| ocean | stress test | mean | 6 | 0.008 | 0.018 | 0.008 | 0.018 | 0.951 | E | E |
| weather | stress test | mean | 6 | 0.037 | 0.034 | 0.037 | 0.034 | 1.042 | N | N |
| electricity | stress test | mean | 6 | 0.021 | 0.027 | 0.021 | 0.027 | 0.866 | N | N |
| dataset | estimand | criterion | n_cells | ce_proposal | ce_fallback | score_ratio | decision |
|---|---|---|---|---|---|---|---|
| ECG strain (79-subject frame) | alpha(t) | symmetric | 8 | 0.015 | 0.001 | 1.043 | fallback (equivalent) |
| ECG strain (79-subject frame) | alpha(t) | under only | 8 | 0.015 | 0.003 | 1.043 | fallback (equivalent) |
| ECG strain (79-subject frame) | beta(s,t) | symmetric | 8 | 0.017 | 0.009 | 0.954 | fallback (equivalent) |
| ECG strain (79-subject frame) | beta(s,t) | under only | 8 | 0.017 | 0.009 | 0.954 | fallback (equivalent) |
| ECG strain (79-subject frame) | gamma(t) | symmetric | 8 | 0.021 | 0.006 | 0.963 | fallback |
| ECG strain (79-subject frame) | gamma(t) | under only | 8 | 0.015 | 0.007 | 0.963 | fallback (equivalent) |
| ECG strain (79-subject frame) | E(Y | X) | symmetric | 8 | 0.013 | 0.015 | 0.984 | fallback (equivalent) |
| ECG strain (79-subject frame) | E(Y | X) | under only | 8 | 0.013 | 0.015 | 0.984 | fallback (equivalent) |
| running (first-50 cohort) | alpha(t) | symmetric | 8 | 0.033 | 0.015 | 1.189 | fallback |
| running (first-50 cohort) | alpha(t) | under only | 8 | 0.033 | 0.015 | 1.189 | fallback |
| running (first-50 cohort) | beta(s,t) | symmetric | 8 | 0.079 | 0.037 | 1.126 | neither adequate; less bad: fallback |
| running (first-50 cohort) | beta(s,t) | under only | 8 | 0.079 | 0.037 | 1.126 | neither adequate; less bad: fallback |
| running (first-50 cohort) | gamma(t) | symmetric | 8 | 0.034 | 0.023 | 1.098 | neither adequate; less bad: fallback |
| running (first-50 cohort) | gamma(t) | under only | 8 | 0.034 | 0.023 | 1.098 | neither adequate; less bad: fallback |
| running (first-50 cohort) | E(Y | X) | symmetric | 8 | 0.017 | 0.021 | 1.013 | proposal |
| running (first-50 cohort) | E(Y | X) | under only | 8 | 0.017 | 0.021 | 1.013 | proposal |
| dataset | estimand | quantity | truth | residual | interaction | residual verdict |
|---|---|---|---|---|---|---|
| ECG strain | beta | coverage:ncv_cl2_bias | 0.010 [0.007, 0.013] | -0.000 [-0.003, 0.002] | 0.004 [0.002, 0.006] | small |
| AF trial | beta | coverage:ncv_cl2_bias | 0.018 [0.009, 0.026] | -0.001 [-0.004, 0.002] | 0.001 [-0.004, 0.005] | small |
| running | beta | coverage:ncv_cl2_bias | 0.106 [0.090, 0.123] | -0.000 [-0.006, 0.005] | 0.006 [-0.006, 0.017] | small |
| DTI | beta | coverage:ncv_cl2_bias | 0.073 [0.063, 0.082] | 0.003 [0.000, 0.006] | -0.003 [-0.008, 0.002] | small |
| gait | beta | coverage:ncv_cl2_bias | 0.023 [0.017, 0.028] | 0.000 [-0.002, 0.002] | 0.001 [-0.002, 0.003] | small |
| ECG 8-lead | beta | coverage:ncv_cl2_bias | 0.075 [0.064, 0.085] | 0.005 [0.001, 0.008] | -0.000 [-0.006, 0.006] | small |
| ocean | beta | coverage:ncv_cl2_bias | 0.094 [0.081, 0.106] | -0.003 [-0.007, 0.001] | 0.002 [-0.007, 0.011] | small |
| weather | beta | coverage:ncv_cl2_bias | 0.011 [-0.015, 0.036] | -0.008 [-0.019, 0.004] | -0.009 [-0.029, 0.010] | small |
| electricity | beta | coverage:ncv_cl2_bias | 0.174 [0.153, 0.197] | -0.006 [-0.014, 0.002] | 0.021 [0.005, 0.037] | small |
| ECG strain | mean | coverage:ncv_cl2_bias | 0.007 [0.006, 0.009] | 0.001 [-0.001, 0.003] | 0.003 [0.001, 0.004] | small |
| AF trial | mean | coverage:ncv_cl2_bias | 0.001 [-0.002, 0.003] | -0.001 [-0.002, 0.000] | 0.000 [-0.001, 0.002] | small |
| running | mean | coverage:ncv_cl2_bias | 0.018 [0.014, 0.023] | 0.000 [-0.002, 0.002] | 0.000 [-0.003, 0.003] | small |
| DTI | mean | coverage:ncv_cl2_bias | 0.024 [0.021, 0.027] | 0.002 [0.001, 0.004] | 0.001 [-0.001, 0.003] | small |
| gait | mean | coverage:ncv_cl2_bias | 0.005 [0.004, 0.006] | 0.000 [-0.000, 0.001] | 0.000 [-0.001, 0.001] | small |
| ECG 8-lead | mean | coverage:ncv_cl2_bias | 0.017 [0.013, 0.020] | 0.001 [-0.000, 0.003] | 0.001 [-0.001, 0.003] | small |
| ocean | mean | coverage:ncv_cl2_bias | 0.012 [0.008, 0.016] | 0.001 [-0.001, 0.003] | 0.002 [-0.001, 0.004] | small |
| weather | mean | coverage:ncv_cl2_bias | 0.012 [0.006, 0.017] | 0.001 [-0.002, 0.005] | 0.002 [-0.003, 0.006] | small |
| electricity | mean | coverage:ncv_cl2_bias | 0.054 [0.046, 0.062] | 0.000 [-0.004, 0.004] | 0.003 [-0.003, 0.009] | small |
| ECG strain | beta | coverage:reml_cl2 | 0.000 [-0.000, 0.001] | -0.000 [-0.002, 0.002] | -0.000 [-0.001, 0.001] | small |
| AF trial | beta | coverage:reml_cl2 | 0.008 [0.005, 0.012] | -0.001 [-0.004, 0.001] | 0.000 [-0.002, 0.002] | small |
| running | beta | coverage:reml_cl2 | 0.006 [-0.002, 0.014] | -0.000 [-0.007, 0.006] | 0.001 [-0.007, 0.008] | small |
| DTI | beta | coverage:reml_cl2 | 0.010 [0.005, 0.015] | 0.002 [-0.001, 0.005] | -0.000 [-0.003, 0.003] | small |
| gait | beta | coverage:reml_cl2 | 0.001 [-0.000, 0.002] | 0.000 [-0.001, 0.001] | 0.000 [-0.001, 0.001] | small |
| ECG 8-lead | beta | coverage:reml_cl2 | 0.006 [0.003, 0.010] | -0.001 [-0.004, 0.002] | -0.001 [-0.004, 0.003] | small |
| ocean | beta | coverage:reml_cl2 | -0.003 [-0.006, -0.000] | -0.006 [-0.011, -0.002] | -0.000 [-0.003, 0.003] | small |
| weather | beta | coverage:reml_cl2 | -0.043 [-0.071, -0.017] | -0.004 [-0.016, 0.007] | -0.022 [-0.043, -0.003] | small |
| electricity | beta | coverage:reml_cl2 | 0.028 [0.013, 0.044] | -0.013 [-0.023, -0.003] | 0.022 [0.005, 0.038] | undetermined |
| ECG strain | mean | coverage:reml_cl2 | 0.000 [-0.000, 0.001] | 0.002 [-0.000, 0.003] | 0.000 [-0.001, 0.001] | small |
| AF trial | mean | coverage:reml_cl2 | 0.001 [-0.000, 0.002] | -0.002 [-0.003, -0.001] | -0.000 [-0.001, 0.001] | small |
| running | mean | coverage:reml_cl2 | -0.005 [-0.006, -0.003] | -0.000 [-0.003, 0.002] | 0.000 [-0.001, 0.001] | small |
| DTI | mean | coverage:reml_cl2 | -0.001 [-0.002, 0.000] | 0.002 [0.000, 0.003] | -0.000 [-0.001, 0.001] | small |
| gait | mean | coverage:reml_cl2 | -0.000 [-0.001, 0.000] | 0.000 [-0.000, 0.001] | -0.000 [-0.000, 0.000] | small |
| ECG 8-lead | mean | coverage:reml_cl2 | 0.001 [0.000, 0.002] | 0.000 [-0.001, 0.002] | 0.000 [-0.001, 0.001] | small |
| ocean | mean | coverage:reml_cl2 | -0.003 [-0.004, -0.003] | -0.001 [-0.002, 0.001] | 0.000 [-0.001, 0.001] | small |
| weather | mean | coverage:reml_cl2 | -0.021 [-0.027, -0.015] | 0.005 [0.001, 0.009] | -0.006 [-0.011, -0.002] | small |
| electricity | mean | coverage:reml_cl2 | -0.007 [-0.011, -0.002] | -0.002 [-0.006, 0.003] | 0.006 [0.001, 0.010] | small |
| ECG strain | beta | coverage:reml_model | -0.001 [-0.002, 0.000] | -0.003 [-0.007, 0.000] | 0.001 [-0.001, 0.002] | small |
| AF trial | beta | coverage:reml_model | 0.018 [0.010, 0.026] | -0.007 [-0.012, -0.002] | 0.001 [-0.007, 0.008] | small |
| running | beta | coverage:reml_model | 0.044 [0.036, 0.053] | 0.002 [-0.007, 0.013] | 0.008 [-0.001, 0.017] | small |
| DTI | beta | coverage:reml_model | 0.028 [0.022, 0.034] | -0.004 [-0.009, 0.001] | -0.004 [-0.010, 0.001] | small |
| gait | beta | coverage:reml_model | 0.007 [0.005, 0.010] | -0.002 [-0.003, 0.000] | 0.000 [-0.001, 0.001] | small |
| ECG 8-lead | beta | coverage:reml_model | 0.017 [0.012, 0.023] | -0.011 [-0.015, -0.006] | -0.003 [-0.010, 0.003] | small |
| ocean | beta | coverage:reml_model | 0.004 [-0.002, 0.009] | -0.014 [-0.021, -0.007] | -0.003 [-0.010, 0.003] | undetermined |
| weather | beta | coverage:reml_model | 0.032 [0.019, 0.047] | -0.001 [-0.013, 0.014] | -0.009 [-0.024, 0.006] | small |
| electricity | beta | coverage:reml_model | 0.078 [0.060, 0.099] | -0.008 [-0.022, 0.006] | 0.020 [-0.001, 0.041] | undetermined |
| ECG strain | mean | coverage:reml_model | -0.002 [-0.003, -0.001] | -0.002 [-0.004, 0.000] | 0.002 [0.000, 0.003] | small |
| AF trial | mean | coverage:reml_model | -0.001 [-0.002, 0.001] | -0.002 [-0.004, -0.000] | 0.001 [-0.001, 0.002] | small |
| running | mean | coverage:reml_model | 0.001 [-0.001, 0.003] | -0.006 [-0.010, -0.002] | -0.000 [-0.002, 0.001] | small |
| DTI | mean | coverage:reml_model | 0.006 [0.003, 0.008] | -0.006 [-0.009, -0.002] | -0.003 [-0.006, -0.001] | small |
| gait | mean | coverage:reml_model | 0.000 [-0.000, 0.001] | -0.001 [-0.002, -0.000] | 0.000 [-0.000, 0.001] | small |
| ECG 8-lead | mean | coverage:reml_model | 0.008 [0.006, 0.009] | -0.007 [-0.009, -0.005] | -0.001 [-0.002, -0.000] | small |
| ocean | mean | coverage:reml_model | -0.006 [-0.008, -0.005] | -0.003 [-0.005, -0.000] | 0.001 [-0.001, 0.002] | small |
| weather | mean | coverage:reml_model | 0.007 [0.003, 0.012] | 0.000 [-0.005, 0.006] | -0.000 [-0.004, 0.004] | small |
| electricity | mean | coverage:reml_model | 0.010 [0.005, 0.015] | -0.001 [-0.008, 0.007] | 0.006 [-0.000, 0.013] | small |
| ECG strain | beta | log_mse_ratio | -0.245 [-0.267, -0.225] | -0.049 [-0.059, -0.038] | -0.032 [-0.043, -0.022] | undetermined |
| AF trial | beta | log_mse_ratio | -0.323 [-0.566, -0.089] | -0.125 [-0.208, -0.046] | -0.027 [-0.153, 0.105] | undetermined |
| running | beta | log_mse_ratio | -1.844 [-2.042, -1.623] | 0.017 [-0.060, 0.101] | -0.024 [-0.159, 0.113] | undetermined |
| DTI | beta | log_mse_ratio | -1.082 [-1.142, -1.017] | -0.086 [-0.118, -0.059] | -0.048 [-0.092, -0.009] | not small |
| gait | beta | log_mse_ratio | -0.381 [-0.503, -0.287] | -0.001 [-0.021, 0.018] | -0.004 [-0.041, 0.033] | small |
| ECG 8-lead | beta | log_mse_ratio | -1.029 [-1.469, -0.781] | 0.078 [-0.113, 0.259] | 0.186 [-0.173, 0.559] | undetermined |
| ocean | beta | log_mse_ratio | -3.273 [-3.480, -3.060] | 0.038 [-0.135, 0.240] | 0.287 [-0.013, 0.678] | undetermined |
| weather | beta | log_mse_ratio | -0.703 [-0.786, -0.625] | 0.011 [-0.050, 0.072] | -0.027 [-0.104, 0.048] | undetermined |
| electricity | beta | log_mse_ratio | -4.051 [-4.404, -3.753] | -0.267 [-0.403, -0.125] | -0.151 [-0.386, 0.066] | not small |
| ECG strain | mean | log_mse_ratio | -0.155 [-0.168, -0.143] | -0.020 [-0.029, -0.012] | -0.027 [-0.038, -0.017] | small |
| AF trial | mean | log_mse_ratio | -0.057 [-0.077, -0.035] | -0.005 [-0.013, 0.004] | -0.006 [-0.019, 0.007] | small |
| running | mean | log_mse_ratio | -0.243 [-0.263, -0.223] | 0.002 [-0.008, 0.013] | 0.012 [-0.003, 0.025] | small |
| DTI | mean | log_mse_ratio | -0.293 [-0.312, -0.273] | -0.016 [-0.025, -0.007] | -0.008 [-0.020, 0.003] | small |
| gait | mean | log_mse_ratio | -0.066 [-0.075, -0.056] | 0.000 [-0.002, 0.003] | -0.000 [-0.005, 0.004] | small |
| ECG 8-lead | mean | log_mse_ratio | -0.240 [-0.267, -0.212] | -0.029 [-0.044, -0.015] | -0.001 [-0.021, 0.020] | small |
| ocean | mean | log_mse_ratio | -0.355 [-0.375, -0.333] | -0.018 [-0.029, -0.008] | 0.009 [-0.003, 0.022] | small |
| weather | mean | log_mse_ratio | -0.264 [-0.301, -0.229] | 0.013 [-0.007, 0.033] | -0.043 [-0.073, -0.013] | small |
| electricity | mean | log_mse_ratio | -0.832 [-0.893, -0.767] | -0.051 [-0.078, -0.024] | 0.030 [-0.015, 0.077] | undetermined |
| dataset | role | cell_role | truth | residual | estimand | AR(1) | AR(1) q05 | REML + CL2 | rho | fit time s |
|---|---|---|---|---|---|---|---|---|---|---|
| ECG strain | counted | principal | MID | REML residuals | beta(s,t) | 0.874 | 0.650 | 0.944 | 0.976 | 44.5 |
| ECG strain | counted | principal | MID | REML residuals | gamma(t) | 0.926 | 0.725 | 0.948 | 0.976 | 44.5 |
| ECG strain | counted | principal | MID | REML residuals | E(Y | X) | 0.869 | 0.620 | 0.938 | 0.976 | 44.5 |
| ECG strain | counted | sensitivity | MID | beat differences (variance-matched) | beta(s,t) | 0.837 | 0.475 | 0.945 | 0.979 | 39.5 |
| ECG strain | counted | sensitivity | MID | beat differences (variance-matched) | gamma(t) | 0.943 | 0.782 | 0.957 | 0.979 | 39.5 |
| ECG strain | counted | sensitivity | MID | beat differences (variance-matched) | E(Y | X) | 0.847 | 0.497 | 0.950 | 0.979 | 39.5 |
| AF trial | counted | principal | MID | REML residuals | beta(s,t) | 0.838 | 0.520 | 0.938 | 0.940 | 27.6 |
| AF trial | counted | principal | MID | REML residuals | gamma(t) | 0.877 | 0.685 | 0.946 | 0.940 | 27.6 |
| AF trial | counted | principal | MID | REML residuals | E(Y | X) | 0.870 | 0.630 | 0.947 | 0.940 | 27.6 |
| running | counted | principal | MID | REML residuals | beta(s,t) | 0.728 | 0.445 | 0.922 | 0.990 | 52.8 |
| running | counted | principal | MID | REML residuals | gamma(t) | 0.806 | 0.540 | 0.934 | 0.990 | 52.8 |
| running | counted | principal | MID | REML residuals | E(Y | X) | 0.787 | 0.500 | 0.934 | 0.990 | 52.8 |
| running | counted | sensitivity | REML | REML residuals | beta(s,t) | 0.548 | 0.165 | 0.916 | 0.990 | 52.5 |
| running | counted | sensitivity | NCV | NCV residuals | beta(s,t) | 0.811 | 0.545 | 0.922 | 0.990 | 52.7 |
| running | counted | sensitivity | REML | REML residuals | gamma(t) | 0.805 | 0.538 | 0.937 | 0.990 | 52.5 |
| running | counted | sensitivity | NCV | NCV residuals | gamma(t) | 0.798 | 0.545 | 0.929 | 0.990 | 52.7 |
| running | counted | sensitivity | REML | REML residuals | E(Y | X) | 0.754 | 0.470 | 0.935 | 0.990 | 52.5 |
| running | counted | sensitivity | NCV | NCV residuals | E(Y | X) | 0.793 | 0.495 | 0.930 | 0.990 | 52.7 |
| DTI | counted | principal | MID | REML residuals | beta(s,t) | 0.858 | 0.470 | 0.929 | 0.817 | 42.7 |
| DTI | counted | principal | MID | REML residuals | gamma(t) | 0.900 | 0.855 | 0.934 | 0.817 | 42.7 |
| DTI | counted | principal | MID | REML residuals | E(Y | X) | 0.902 | 0.667 | 0.932 | 0.817 | 42.7 |
| gait | counted | principal | MID | REML residuals | beta(s,t) | 0.914 | 0.730 | 0.940 | 0.987 | 161.6 |
| gait | counted | principal | MID | REML residuals | gamma(t) | 0.919 | 0.770 | 0.933 | 0.987 | 161.6 |
| gait | counted | principal | MID | REML residuals | E(Y | X) | 0.927 | 0.760 | 0.943 | 0.987 | 161.6 |
| ECG 8-lead | counted | principal | MID | REML residuals | beta(s,t) | 0.720 | 0.170 | 0.944 | 0.910 | 120.4 |
| ECG 8-lead | counted | principal | MID | REML residuals | gamma(t) | 0.940 | 0.620 | 0.961 | 0.910 | 120.4 |
| ECG 8-lead | counted | principal | MID | REML residuals | E(Y | X) | 0.924 | 0.555 | 0.953 | 0.910 | 120.4 |
| ocean | stress test | principal | MID | REML residuals | beta(s,t) | 0.854 | 0.299 | 0.927 | 0.990 | 157.0 |
| ocean | stress test | principal | MID | REML residuals | gamma(t) | 0.934 | 0.815 | 0.945 | 0.990 | 157.0 |
| ocean | stress test | principal | MID | REML residuals | E(Y | X) | 0.932 | 0.775 | 0.935 | 0.990 | 157.0 |
| weather | stress test | principal | MID | REML residuals | beta(s,t) | 0.438 | 0.320 | 0.905 | 0.445 | 44.3 |
| weather | stress test | principal | MID | REML residuals | gamma(t) | 0.567 | 0.500 | 0.909 | 0.445 | 44.3 |
| weather | stress test | principal | MID | REML residuals | E(Y | X) | 0.675 | 0.440 | 0.922 | 0.445 | 44.3 |
| electricity | stress test | principal | MID | REML residuals | beta(s,t) | 0.677 | 0.220 | 0.939 | 0.979 | 40.7 |
| electricity | stress test | principal | MID | REML residuals | gamma(t) | 0.908 | 0.823 | 0.956 | 0.979 | 40.7 |
| electricity | stress test | principal | MID | REML residuals | E(Y | X) | 0.844 | 0.640 | 0.927 | 0.979 | 40.7 |
| dataset | estimand | cells | standard | block | ncv_blockcl2 | blockncv_curvecl2 | reml_curvecl2 | reml_blockcl2 | mismatch | reml_altcl2 |
|---|---|---|---|---|---|---|---|---|---|---|
| ocean | beta | block_flips | 0.925 | 0.919 | 0.920 | 0.925 | 0.934 | 0.929 | ||
| ocean | beta | curve_flips | 0.916 | 0.912 | 0.916 | 0.913 | 0.925 | 0.922 | ||
| ocean | mean | block_flips | 0.933 | 0.939 | 0.940 | 0.933 | 0.932 | 0.936 | ||
| ocean | mean | curve_flips | 0.939 | 0.936 | 0.937 | 0.938 | 0.935 | 0.930 | ||
| weather | beta | block_flips | 0.823 | 0.809 | 0.813 | 0.821 | 0.903 | 0.889 | 0.806 | 0.883 |
| weather | beta | curve_flips | 0.828 | 0.836 | 0.824 | 0.846 | 0.882 | 0.867 | ||
| weather | mean | block_flips | 0.898 | 0.901 | 0.908 | 0.895 | 0.906 | 0.902 | 0.900 | 0.904 |
| weather | mean | curve_flips | 0.920 | 0.894 | 0.898 | 0.922 | 0.924 | 0.892 | ||
| electricity | beta | block_flips | 0.895 | 0.896 | 0.894 | 0.897 | 0.920 | 0.921 | ||
| electricity | beta | curve_flips | 0.911 | 0.911 | 0.911 | 0.912 | 0.927 | 0.925 | ||
| electricity | mean | block_flips | 0.938 | 0.946 | 0.946 | 0.939 | 0.931 | 0.938 | ||
| electricity | mean | curve_flips | 0.947 | 0.946 | 0.946 | 0.947 | 0.927 | 0.926 |
| dataset | estimand | n_cells | incl | cov_reml_cl2 | cov_ncv_cl2_bias | cov_hybrid | q05_reml_cl2 | q05_ncv_cl2_bias | q05_hybrid | IS_ncv_cl2_bias/reml_cl2 | IS_hybrid/reml_cl2 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| ECG strain | beta | 24 | 0.996 | 0.939 | 0.935 | 0.949 | 0.893 | 0.856 | 0.878 | 0.959 | 0.931 |
| AF trial | beta | 12 | 0.971 | 0.929 | 0.900 | 0.955 | 0.861 | 0.747 | 0.817 | 0.925 | 0.806 |
| running | beta | 16 | 0.937 | 0.913 | 0.887 | 0.924 | 0.857 | 0.758 | 0.739 | 1.083 | 0.905 |
| DTI | beta | 12 | 0.974 | 0.922 | 0.917 | 0.957 | 0.866 | 0.784 | 0.841 | 0.953 | 0.858 |
| gait | beta | 12 | 0.967 | 0.937 | 0.915 | 0.964 | 0.898 | 0.806 | 0.869 | 0.966 | 0.879 |
| ECG 8-lead | beta | 12 | 0.960 | 0.939 | 0.935 | 0.980 | 0.861 | 0.837 | 0.933 | 0.783 | 0.805 |
| ocean | beta | 14 | 0.936 | 0.928 | 0.913 | 0.963 | 0.879 | 0.785 | 0.838 | 1.001 | 0.845 |
| weather | beta | 14 | 0.963 | 0.865 | 0.802 | 0.890 | 0.813 | 0.688 | 0.794 | 1.176 | 0.932 |
| electricity | beta | 14 | 0.933 | 0.920 | 0.906 | 0.941 | 0.862 | 0.829 | 0.779 | 0.957 | 0.925 |
| ECG strain | mean | 24 | 0.994 | 0.940 | 0.941 | 0.935 | 0.880 | 0.873 | 0.849 | 0.997 | 0.995 |
| AF trial | mean | 12 | 0.989 | 0.940 | 0.932 | 0.938 | 0.885 | 0.837 | 0.855 | 1.012 | 0.972 |
| running | mean | 16 | 0.983 | 0.929 | 0.934 | 0.926 | 0.880 | 0.866 | 0.823 | 1.006 | 0.995 |
| DTI | mean | 12 | 0.994 | 0.925 | 0.932 | 0.946 | 0.880 | 0.856 | 0.876 | 0.945 | 0.919 |
| gait | mean | 12 | 0.988 | 0.940 | 0.942 | 0.943 | 0.902 | 0.884 | 0.872 | 0.997 | 0.993 |
| ECG 8-lead | mean | 12 | 0.975 | 0.948 | 0.954 | 0.963 | 0.892 | 0.879 | 0.896 | 0.922 | 0.917 |
| ocean | mean | 14 | 0.988 | 0.933 | 0.940 | 0.941 | 0.886 | 0.883 | 0.871 | 0.972 | 0.959 |
| weather | mean | 14 | 0.988 | 0.918 | 0.914 | 0.925 | 0.844 | 0.793 | 0.823 | 1.049 | 0.981 |
| electricity | mean | 14 | 0.980 | 0.926 | 0.943 | 0.953 | 0.877 | 0.890 | 0.883 | 0.908 | 0.896 |
| block | family | error | G | basis | covariate | estimand | z_mean_cl2 | z_sd_cl2 | share_big_cl2 | z_mean_model | z_sd_model | share_big_model |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ar1_home | Gaussian | AR(1) | 100 | default | rich | alpha(t) | -0.039 | 0.997 | 0.045 | -0.084 | 2.155 | 0.364 |
| ar1_home | Gaussian | AR(1) | 100 | default | rich | beta(s,t) | 0.003 | 1.032 | 0.060 | 0.010 | 2.130 | 0.349 |
| ar1_home | Gaussian | AR(1) | 100 | default | rich | gamma(t) | -0.042 | 1.022 | 0.063 | -0.098 | 2.167 | 0.355 |
| ar1_home | Gaussian | AR(1) | 100 | default | rich | E(Y | X) | -0.017 | 1.031 | 0.058 | -0.039 | 2.219 | 0.375 |
| basis_size | binary | smooth | 100 | large | rich | alpha(t) | -0.382 | 1.062 | 0.085 | -0.561 | 1.737 | 0.272 |
| basis_size | binary | smooth | 100 | large | rich | beta(s,t) | 0.003 | 0.962 | 0.042 | 0.005 | 1.582 | 0.211 |
| basis_size | binary | smooth | 100 | large | rich | gamma(t) | -0.016 | 1.014 | 0.058 | -0.017 | 1.693 | 0.238 |
| basis_size | binary | smooth | 100 | xlarge | rich | alpha(t) | -0.594 | 1.043 | 0.101 | -0.869 | 1.603 | 0.287 |
| basis_size | binary | smooth | 100 | xlarge | rich | beta(s,t) | 0.001 | 0.959 | 0.040 | 0.004 | 1.555 | 0.205 |
| basis_size | binary | smooth | 100 | xlarge | rich | gamma(t) | 0.003 | 1.007 | 0.054 | 0.017 | 1.643 | 0.221 |
| basis_size | binary | iid | 100 | large | rich | alpha(t) | 0.025 | 1.014 | 0.056 | 0.044 | 0.937 | 0.039 |
| basis_size | binary | iid | 100 | large | rich | beta(s,t) | 0.010 | 0.626 | 0.003 | 0.009 | 0.592 | 0.002 |
| basis_size | binary | iid | 100 | large | rich | gamma(t) | -0.047 | 1.056 | 0.064 | -0.049 | 0.985 | 0.042 |
| basis_size | binary | iid | 100 | xlarge | rich | alpha(t) | 0.026 | 0.962 | 0.044 | 0.043 | 0.892 | 0.031 |
| basis_size | binary | iid | 100 | xlarge | rich | beta(s,t) | 0.007 | 0.525 | 0.001 | 0.007 | 0.498 | 0.000 |
| basis_size | binary | iid | 100 | xlarge | rich | gamma(t) | -0.045 | 1.011 | 0.051 | -0.046 | 0.945 | 0.034 |
| basis_size | Gaussian | smooth | 100 | large | rich | alpha(t) | -0.023 | 0.999 | 0.050 | -0.044 | 1.918 | 0.303 |
| basis_size | Gaussian | smooth | 100 | large | rich | beta(s,t) | 0.000 | 1.015 | 0.055 | 0.002 | 2.060 | 0.329 |
| basis_size | Gaussian | smooth | 100 | large | rich | gamma(t) | -0.022 | 0.999 | 0.050 | -0.046 | 1.896 | 0.297 |
| basis_size | Gaussian | smooth | 100 | large | rich | E(Y | X) | -0.013 | 1.010 | 0.053 | -0.027 | 2.018 | 0.327 |
| basis_size | Gaussian | smooth | 100 | xlarge | rich | alpha(t) | -0.031 | 0.982 | 0.046 | -0.056 | 1.756 | 0.261 |
| basis_size | Gaussian | smooth | 100 | xlarge | rich | beta(s,t) | 0.001 | 0.983 | 0.046 | 0.004 | 2.005 | 0.327 |
| basis_size | Gaussian | smooth | 100 | xlarge | rich | gamma(t) | -0.022 | 0.998 | 0.050 | -0.040 | 1.762 | 0.266 |
| basis_size | Gaussian | smooth | 100 | xlarge | rich | E(Y | X) | -0.014 | 0.991 | 0.049 | -0.028 | 1.872 | 0.294 |
| basis_size | Gaussian | iid | 100 | large | rich | alpha(t) | -0.007 | 0.936 | 0.035 | -0.006 | 0.925 | 0.032 |
| basis_size | Gaussian | iid | 100 | large | rich | beta(s,t) | 0.002 | 0.650 | 0.003 | 0.002 | 0.637 | 0.002 |
| basis_size | Gaussian | iid | 100 | large | rich | gamma(t) | -0.025 | 0.934 | 0.037 | -0.026 | 0.908 | 0.029 |
| basis_size | Gaussian | iid | 100 | large | rich | E(Y | X) | -0.001 | 0.878 | 0.027 | -0.002 | 0.861 | 0.022 |
| basis_size | Gaussian | iid | 100 | xlarge | rich | alpha(t) | -0.006 | 0.884 | 0.025 | -0.005 | 0.875 | 0.022 |
| basis_size | Gaussian | iid | 100 | xlarge | rich | beta(s,t) | 0.001 | 0.535 | 0.000 | 0.001 | 0.526 | 0.000 |
| basis_size | Gaussian | iid | 100 | xlarge | rich | gamma(t) | -0.022 | 0.884 | 0.027 | -0.022 | 0.864 | 0.024 |
| basis_size | Gaussian | iid | 100 | xlarge | rich | E(Y | X) | -0.001 | 0.822 | 0.018 | -0.001 | 0.810 | 0.015 |
| basis_size | Poisson | smooth | 100 | large | rich | alpha(t) | -0.169 | 0.986 | 0.051 | -0.304 | 1.810 | 0.284 |
| basis_size | Poisson | smooth | 100 | large | rich | beta(s,t) | 0.001 | 1.004 | 0.052 | 0.000 | 1.929 | 0.298 |
| basis_size | Poisson | smooth | 100 | large | rich | gamma(t) | -0.027 | 0.995 | 0.050 | -0.046 | 1.804 | 0.277 |
| basis_size | Poisson | smooth | 100 | xlarge | rich | alpha(t) | -0.241 | 0.967 | 0.050 | -0.404 | 1.639 | 0.248 |
| basis_size | Poisson | smooth | 100 | xlarge | rich | beta(s,t) | 0.000 | 0.978 | 0.045 | 0.003 | 1.843 | 0.285 |
| basis_size | Poisson | smooth | 100 | xlarge | rich | gamma(t) | -0.019 | 0.983 | 0.047 | -0.023 | 1.646 | 0.231 |
| basis_size | Poisson | iid | 100 | large | rich | alpha(t) | 0.034 | 0.932 | 0.036 | 0.036 | 0.917 | 0.032 |
| basis_size | Poisson | iid | 100 | large | rich | beta(s,t) | 0.005 | 0.645 | 0.003 | 0.005 | 0.630 | 0.002 |
| basis_size | Poisson | iid | 100 | large | rich | gamma(t) | -0.035 | 0.931 | 0.036 | -0.036 | 0.903 | 0.029 |
| basis_size | Poisson | iid | 100 | xlarge | rich | alpha(t) | 0.037 | 0.878 | 0.025 | 0.039 | 0.864 | 0.022 |
| basis_size | Poisson | iid | 100 | xlarge | rich | beta(s,t) | 0.003 | 0.529 | 0.000 | 0.003 | 0.518 | 0.000 |
| basis_size | Poisson | iid | 100 | xlarge | rich | gamma(t) | -0.030 | 0.886 | 0.027 | -0.032 | 0.862 | 0.022 |
| core | binary | smooth | 40 | default | rich | alpha(t) | -0.454 | 1.134 | 0.106 | -0.657 | 1.902 | 0.323 |
| core | binary | smooth | 40 | default | rich | beta(s,t) | 0.009 | 1.081 | 0.071 | 0.015 | 1.716 | 0.243 |
| core | binary | smooth | 40 | default | rich | gamma(t) | -0.045 | 1.119 | 0.080 | -0.073 | 1.939 | 0.296 |
| core | binary | smooth | 100 | default | rich | alpha(t) | -0.241 | 1.063 | 0.075 | -0.355 | 1.844 | 0.284 |
| core | binary | smooth | 100 | default | rich | beta(s,t) | 0.001 | 0.987 | 0.048 | 0.004 | 1.608 | 0.219 |
| core | binary | smooth | 100 | default | rich | gamma(t) | -0.019 | 1.029 | 0.061 | -0.025 | 1.818 | 0.266 |
| core | binary | iid | 40 | default | rich | alpha(t) | 0.044 | 1.276 | 0.101 | 0.064 | 1.149 | 0.070 |
| core | binary | iid | 40 | default | rich | beta(s,t) | 0.011 | 0.835 | 0.021 | 0.010 | 0.767 | 0.013 |
| core | binary | iid | 40 | default | rich | gamma(t) | -0.068 | 1.203 | 0.094 | -0.065 | 1.055 | 0.063 |
| core | binary | iid | 100 | default | rich | alpha(t) | 0.033 | 1.060 | 0.064 | 0.049 | 0.993 | 0.050 |
| core | binary | iid | 100 | default | rich | beta(s,t) | 0.011 | 0.751 | 0.010 | 0.011 | 0.713 | 0.007 |
| core | binary | iid | 100 | default | rich | gamma(t) | -0.041 | 1.082 | 0.071 | -0.043 | 1.018 | 0.054 |
| core | binary | OU | 40 | default | rich | alpha(t) | -0.393 | 1.130 | 0.095 | -0.521 | 1.778 | 0.271 |
| core | binary | OU | 40 | default | rich | beta(s,t) | 0.012 | 1.048 | 0.061 | 0.019 | 1.511 | 0.194 |
| core | binary | OU | 40 | default | rich | gamma(t) | -0.062 | 1.113 | 0.076 | -0.111 | 1.728 | 0.265 |
| core | binary | OU | 100 | default | rich | alpha(t) | -0.232 | 1.058 | 0.068 | -0.317 | 1.712 | 0.246 |
| core | binary | OU | 100 | default | rich | beta(s,t) | 0.006 | 0.987 | 0.049 | 0.010 | 1.479 | 0.181 |
| core | binary | OU | 100 | default | rich | gamma(t) | -0.037 | 1.053 | 0.068 | -0.059 | 1.714 | 0.251 |
| core | Gaussian | smooth | 40 | default | rich | alpha(t) | -0.041 | 1.055 | 0.064 | -0.080 | 2.315 | 0.385 |
| core | Gaussian | smooth | 40 | default | rich | beta(s,t) | 0.003 | 1.094 | 0.075 | 0.006 | 2.336 | 0.387 |
| core | Gaussian | smooth | 40 | default | rich | gamma(t) | -0.050 | 1.029 | 0.055 | -0.103 | 2.219 | 0.377 |
| core | Gaussian | smooth | 40 | default | rich | E(Y | X) | -0.020 | 1.073 | 0.069 | -0.040 | 2.334 | 0.394 |
| core | Gaussian | smooth | 100 | default | rich | alpha(t) | -0.032 | 1.003 | 0.051 | -0.062 | 2.144 | 0.359 |
| core | Gaussian | smooth | 100 | default | rich | beta(s,t) | 0.003 | 1.021 | 0.056 | 0.009 | 2.123 | 0.344 |
| core | Gaussian | smooth | 100 | default | rich | gamma(t) | -0.026 | 1.002 | 0.052 | -0.062 | 2.111 | 0.347 |
| core | Gaussian | smooth | 100 | default | rich | E(Y | X) | -0.016 | 1.026 | 0.056 | -0.034 | 2.201 | 0.366 |
| core | Gaussian | iid | 40 | default | rich | alpha(t) | -0.027 | 1.014 | 0.055 | -0.018 | 0.981 | 0.045 |
| core | Gaussian | iid | 40 | default | rich | beta(s,t) | 0.003 | 0.769 | 0.013 | 0.003 | 0.743 | 0.009 |
| core | Gaussian | iid | 40 | default | rich | gamma(t) | -0.026 | 1.007 | 0.054 | -0.030 | 0.956 | 0.041 |
| core | Gaussian | iid | 40 | default | rich | E(Y | X) | -0.010 | 0.959 | 0.043 | -0.008 | 0.919 | 0.033 |
| core | Gaussian | iid | 100 | default | rich | alpha(t) | -0.011 | 1.008 | 0.047 | -0.008 | 0.993 | 0.044 |
| core | Gaussian | iid | 100 | default | rich | beta(s,t) | 0.005 | 0.794 | 0.015 | 0.005 | 0.773 | 0.012 |
| core | Gaussian | iid | 100 | default | rich | gamma(t) | -0.031 | 0.990 | 0.050 | -0.033 | 0.959 | 0.039 |
| core | Gaussian | iid | 100 | default | rich | E(Y | X) | -0.002 | 0.952 | 0.040 | -0.001 | 0.927 | 0.034 |
| core | Gaussian | OU | 40 | default | rich | alpha(t) | -0.042 | 1.048 | 0.062 | -0.081 | 2.214 | 0.369 |
| core | Gaussian | OU | 40 | default | rich | beta(s,t) | 0.006 | 1.093 | 0.074 | 0.012 | 2.182 | 0.363 |
| core | Gaussian | OU | 40 | default | rich | gamma(t) | -0.060 | 1.035 | 0.062 | -0.129 | 2.125 | 0.350 |
| core | Gaussian | OU | 40 | default | rich | E(Y | X) | -0.017 | 1.075 | 0.069 | -0.035 | 2.227 | 0.377 |
| core | Gaussian | OU | 100 | default | rich | alpha(t) | -0.035 | 1.003 | 0.049 | -0.075 | 2.062 | 0.341 |
| core | Gaussian | OU | 100 | default | rich | beta(s,t) | 0.004 | 1.029 | 0.059 | 0.010 | 2.049 | 0.329 |
| core | Gaussian | OU | 100 | default | rich | gamma(t) | -0.046 | 1.022 | 0.062 | -0.100 | 2.066 | 0.339 |
| core | Gaussian | OU | 100 | default | rich | E(Y | X) | -0.016 | 1.032 | 0.058 | -0.036 | 2.127 | 0.354 |
| core | Poisson | smooth | 40 | default | rich | alpha(t) | -0.202 | 1.031 | 0.063 | -0.383 | 2.080 | 0.353 |
| core | Poisson | smooth | 40 | default | rich | beta(s,t) | 0.006 | 1.079 | 0.071 | 0.010 | 2.063 | 0.330 |
| core | Poisson | smooth | 40 | default | rich | gamma(t) | -0.050 | 1.038 | 0.060 | -0.095 | 2.056 | 0.348 |
| core | Poisson | smooth | 100 | default | rich | alpha(t) | -0.112 | 0.997 | 0.052 | -0.215 | 2.037 | 0.338 |
| core | Poisson | smooth | 100 | default | rich | beta(s,t) | 0.001 | 1.015 | 0.055 | 0.004 | 1.977 | 0.311 |
| core | Poisson | smooth | 100 | default | rich | gamma(t) | -0.038 | 1.012 | 0.056 | -0.077 | 2.025 | 0.334 |
| core | Poisson | iid | 40 | default | rich | alpha(t) | 0.030 | 0.991 | 0.048 | 0.036 | 0.960 | 0.039 |
| core | Poisson | iid | 40 | default | rich | beta(s,t) | 0.009 | 0.769 | 0.012 | 0.008 | 0.741 | 0.008 |
| core | Poisson | iid | 40 | default | rich | gamma(t) | -0.026 | 1.026 | 0.055 | -0.027 | 0.969 | 0.045 |
| core | Poisson | iid | 100 | default | rich | alpha(t) | 0.033 | 1.008 | 0.050 | 0.035 | 0.991 | 0.048 |
| core | Poisson | iid | 100 | default | rich | beta(s,t) | 0.008 | 0.810 | 0.018 | 0.009 | 0.785 | 0.013 |
| core | Poisson | iid | 100 | default | rich | gamma(t) | -0.043 | 0.997 | 0.051 | -0.044 | 0.963 | 0.041 |
| core | Poisson | OU | 40 | default | rich | alpha(t) | -0.193 | 1.022 | 0.060 | -0.353 | 1.982 | 0.334 |
| core | Poisson | OU | 40 | default | rich | beta(s,t) | 0.008 | 1.081 | 0.072 | 0.014 | 1.938 | 0.306 |
| core | Poisson | OU | 40 | default | rich | gamma(t) | -0.071 | 1.066 | 0.066 | -0.145 | 1.994 | 0.332 |
| core | Poisson | OU | 100 | default | rich | alpha(t) | -0.115 | 1.006 | 0.051 | -0.219 | 1.978 | 0.331 |
| core | Poisson | OU | 100 | default | rich | beta(s,t) | 0.004 | 1.029 | 0.059 | 0.010 | 1.925 | 0.299 |
| core | Poisson | OU | 100 | default | rich | gamma(t) | -0.065 | 1.028 | 0.063 | -0.124 | 1.962 | 0.318 |
| dense_grid | binary | smooth | 100 | default | rich | alpha(t) | -0.421 | 1.034 | 0.080 | -1.282 | 3.386 | 0.590 |
| dense_grid | binary | smooth | 100 | default | rich | beta(s,t) | 0.003 | 1.026 | 0.057 | 0.013 | 3.578 | 0.580 |
| dense_grid | binary | smooth | 100 | default | rich | gamma(t) | -0.005 | 1.013 | 0.057 | -0.028 | 3.410 | 0.555 |
| dense_grid | binary | iid | 100 | default | rich | alpha(t) | -0.007 | 1.004 | 0.049 | -0.001 | 0.970 | 0.043 |
| dense_grid | binary | iid | 100 | default | rich | beta(s,t) | 0.005 | 0.739 | 0.010 | 0.004 | 0.713 | 0.007 |
| dense_grid | binary | iid | 100 | default | rich | gamma(t) | -0.063 | 1.001 | 0.049 | -0.063 | 0.962 | 0.040 |
| dense_grid | binary | OU | 100 | default | rich | alpha(t) | -0.393 | 1.043 | 0.081 | -1.064 | 3.076 | 0.553 |
| dense_grid | binary | OU | 100 | default | rich | beta(s,t) | 0.004 | 1.047 | 0.062 | 0.013 | 3.278 | 0.550 |
| dense_grid | binary | OU | 100 | default | rich | gamma(t) | -0.029 | 1.036 | 0.059 | -0.087 | 3.111 | 0.526 |
| dense_grid | Gaussian | smooth | 100 | default | rich | alpha(t) | -0.024 | 1.007 | 0.053 | -0.098 | 4.203 | 0.627 |
| dense_grid | Gaussian | smooth | 100 | default | rich | beta(s,t) | 0.000 | 1.024 | 0.056 | 0.007 | 4.468 | 0.650 |
| dense_grid | Gaussian | smooth | 100 | default | rich | gamma(t) | -0.016 | 1.010 | 0.053 | -0.085 | 4.168 | 0.631 |
| dense_grid | Gaussian | smooth | 100 | default | rich | E(Y | X) | -0.011 | 1.028 | 0.057 | -0.051 | 4.408 | 0.647 |
| dense_grid | Gaussian | iid | 100 | default | rich | alpha(t) | -0.023 | 0.997 | 0.049 | -0.022 | 0.990 | 0.048 |
| dense_grid | Gaussian | iid | 100 | default | rich | beta(s,t) | 0.002 | 0.854 | 0.024 | 0.002 | 0.837 | 0.020 |
| dense_grid | Gaussian | iid | 100 | default | rich | gamma(t) | -0.027 | 1.009 | 0.057 | -0.030 | 0.979 | 0.049 |
| dense_grid | Gaussian | iid | 100 | default | rich | E(Y | X) | -0.011 | 0.970 | 0.044 | -0.010 | 0.948 | 0.039 |
| dense_grid | Gaussian | OU | 100 | default | rich | alpha(t) | -0.031 | 0.996 | 0.047 | -0.135 | 3.932 | 0.618 |
| dense_grid | Gaussian | OU | 100 | default | rich | beta(s,t) | 0.002 | 1.038 | 0.060 | 0.013 | 4.293 | 0.641 |
| dense_grid | Gaussian | OU | 100 | default | rich | gamma(t) | -0.036 | 1.027 | 0.064 | -0.162 | 3.984 | 0.603 |
| dense_grid | Gaussian | OU | 100 | default | rich | E(Y | X) | -0.012 | 1.035 | 0.059 | -0.055 | 4.186 | 0.633 |
| dense_grid | Poisson | smooth | 100 | default | rich | alpha(t) | -0.178 | 0.997 | 0.053 | -0.690 | 3.958 | 0.620 |
| dense_grid | Poisson | smooth | 100 | default | rich | beta(s,t) | 0.000 | 1.029 | 0.057 | 0.004 | 4.226 | 0.631 |
| dense_grid | Poisson | smooth | 100 | default | rich | gamma(t) | -0.020 | 1.007 | 0.053 | -0.084 | 3.943 | 0.613 |
| dense_grid | Poisson | iid | 100 | default | rich | alpha(t) | 0.001 | 0.993 | 0.049 | 0.001 | 0.986 | 0.046 |
| dense_grid | Poisson | iid | 100 | default | rich | beta(s,t) | 0.003 | 0.840 | 0.022 | 0.004 | 0.821 | 0.017 |
| dense_grid | Poisson | iid | 100 | default | rich | gamma(t) | -0.041 | 1.000 | 0.052 | -0.042 | 0.965 | 0.043 |
| dense_grid | Poisson | OU | 100 | default | rich | alpha(t) | -0.178 | 0.993 | 0.048 | -0.670 | 3.761 | 0.603 |
| dense_grid | Poisson | OU | 100 | default | rich | beta(s,t) | 0.002 | 1.045 | 0.062 | 0.010 | 4.092 | 0.626 |
| dense_grid | Poisson | OU | 100 | default | rich | gamma(t) | -0.046 | 1.024 | 0.064 | -0.178 | 3.783 | 0.589 |
| families | beta | smooth | 100 | default | rich | alpha(t) | -0.063 | 1.020 | 0.055 | -0.124 | 2.141 | 0.358 |
| families | beta | smooth | 100 | default | rich | beta(s,t) | 0.002 | 1.031 | 0.059 | 0.007 | 2.112 | 0.341 |
| families | beta | smooth | 100 | default | rich | gamma(t) | -0.029 | 1.012 | 0.054 | -0.070 | 2.096 | 0.344 |
| families | beta | iid | 100 | default | rich | alpha(t) | 0.021 | 1.023 | 0.050 | 0.024 | 0.995 | 0.042 |
| families | beta | iid | 100 | default | rich | beta(s,t) | 0.004 | 0.801 | 0.016 | 0.005 | 0.774 | 0.011 |
| families | beta | iid | 100 | default | rich | gamma(t) | -0.032 | 1.000 | 0.050 | -0.034 | 0.959 | 0.041 |
| families | negative binomial | smooth | 100 | default | rich | alpha(t) | -0.201 | 1.010 | 0.058 | -0.440 | 2.324 | 0.399 |
| families | negative binomial | smooth | 100 | default | rich | beta(s,t) | 0.002 | 1.036 | 0.060 | 0.005 | 2.436 | 0.400 |
| families | negative binomial | smooth | 100 | default | rich | gamma(t) | -0.029 | 1.020 | 0.057 | -0.074 | 2.401 | 0.400 |
| families | negative binomial | iid | 100 | default | rich | alpha(t) | -0.010 | 1.010 | 0.050 | -0.005 | 1.100 | 0.075 |
| families | negative binomial | iid | 100 | default | rich | beta(s,t) | 0.008 | 0.825 | 0.020 | 0.009 | 0.882 | 0.032 |
| families | negative binomial | iid | 100 | default | rich | gamma(t) | -0.041 | 0.992 | 0.048 | -0.050 | 1.097 | 0.077 |
| families | scaled t | smooth | 100 | default | rich | alpha(t) | -0.027 | 1.000 | 0.049 | -0.053 | 2.090 | 0.349 |
| families | scaled t | smooth | 100 | default | rich | beta(s,t) | 0.003 | 1.017 | 0.056 | 0.009 | 2.062 | 0.329 |
| families | scaled t | smooth | 100 | default | rich | gamma(t) | -0.012 | 0.996 | 0.050 | -0.027 | 2.073 | 0.332 |
| families | scaled t | iid | 100 | default | rich | alpha(t) | 0.004 | 1.000 | 0.046 | 0.006 | 0.992 | 0.045 |
| families | scaled t | iid | 100 | default | rich | beta(s,t) | 0.003 | 0.802 | 0.015 | 0.003 | 0.784 | 0.012 |
| families | scaled t | iid | 100 | default | rich | gamma(t) | -0.030 | 0.990 | 0.050 | -0.031 | 0.966 | 0.044 |
| heteroskedastic | Gaussian | var(z) | 40 | default | rich | alpha(t) | -0.042 | 1.066 | 0.070 | -0.084 | 2.331 | 0.382 |
| heteroskedastic | Gaussian | var(z) | 40 | default | rich | beta(s,t) | 0.000 | 1.054 | 0.061 | 0.002 | 2.313 | 0.381 |
| heteroskedastic | Gaussian | var(z) | 40 | default | rich | gamma(t) | -0.087 | 1.126 | 0.074 | -0.242 | 3.068 | 0.510 |
| heteroskedastic | Gaussian | var(z) | 40 | default | rich | E(Y | X) | -0.017 | 1.072 | 0.066 | -0.046 | 2.515 | 0.412 |
| heteroskedastic | Gaussian | var(z) | 100 | default | rich | alpha(t) | -0.045 | 1.015 | 0.055 | -0.105 | 2.154 | 0.353 |
| heteroskedastic | Gaussian | var(z) | 100 | default | rich | beta(s,t) | 0.002 | 1.007 | 0.051 | 0.003 | 2.125 | 0.340 |
| heteroskedastic | Gaussian | var(z) | 100 | default | rich | gamma(t) | -0.056 | 1.068 | 0.062 | -0.184 | 2.965 | 0.506 |
| heteroskedastic | Gaussian | var(z) | 100 | default | rich | E(Y | X) | -0.021 | 1.035 | 0.056 | -0.056 | 2.393 | 0.394 |
| heteroskedastic | Gaussian | var(t) | 40 | default | rich | alpha(t) | -0.039 | 1.043 | 0.061 | -0.070 | 2.292 | 0.370 |
| heteroskedastic | Gaussian | var(t) | 40 | default | rich | beta(s,t) | 0.002 | 1.091 | 0.074 | 0.007 | 2.328 | 0.381 |
| heteroskedastic | Gaussian | var(t) | 40 | default | rich | gamma(t) | -0.058 | 1.025 | 0.056 | -0.119 | 2.212 | 0.368 |
| heteroskedastic | Gaussian | var(t) | 40 | default | rich | E(Y | X) | -0.019 | 1.069 | 0.068 | -0.035 | 2.331 | 0.386 |
| heteroskedastic | Gaussian | var(t) | 100 | default | rich | alpha(t) | -0.028 | 0.995 | 0.049 | -0.046 | 2.133 | 0.350 |
| heteroskedastic | Gaussian | var(t) | 100 | default | rich | beta(s,t) | 0.003 | 1.018 | 0.056 | 0.009 | 2.122 | 0.338 |
| heteroskedastic | Gaussian | var(t) | 100 | default | rich | gamma(t) | -0.029 | 1.001 | 0.053 | -0.067 | 2.114 | 0.339 |
| heteroskedastic | Gaussian | var(t) | 100 | default | rich | E(Y | X) | -0.015 | 1.023 | 0.056 | -0.028 | 2.203 | 0.358 |
| heteroskedastic | Gaussian | var(t,z) | 40 | default | rich | alpha(t) | -0.038 | 1.057 | 0.065 | -0.070 | 2.304 | 0.371 |
| heteroskedastic | Gaussian | var(t,z) | 40 | default | rich | beta(s,t) | -0.001 | 1.048 | 0.060 | 0.003 | 2.293 | 0.371 |
| heteroskedastic | Gaussian | var(t,z) | 40 | default | rich | gamma(t) | -0.090 | 1.122 | 0.075 | -0.246 | 3.056 | 0.498 |
| heteroskedastic | Gaussian | var(t,z) | 40 | default | rich | E(Y | X) | -0.017 | 1.068 | 0.064 | -0.043 | 2.506 | 0.404 |
| heteroskedastic | Gaussian | var(t,z) | 100 | default | rich | alpha(t) | -0.043 | 1.008 | 0.053 | -0.099 | 2.135 | 0.345 |
| heteroskedastic | Gaussian | var(t,z) | 100 | default | rich | beta(s,t) | 0.002 | 1.003 | 0.050 | 0.005 | 2.128 | 0.334 |
| heteroskedastic | Gaussian | var(t,z) | 100 | default | rich | gamma(t) | -0.058 | 1.064 | 0.060 | -0.187 | 2.953 | 0.490 |
| heteroskedastic | Gaussian | var(t,z) | 100 | default | rich | E(Y | X) | -0.020 | 1.032 | 0.056 | -0.051 | 2.392 | 0.385 |
| lowrank_covariate | binary | smooth | 100 | default | lowrank | alpha(t) | -0.230 | 1.082 | 0.078 | -0.342 | 1.921 | 0.298 |
| lowrank_covariate | binary | smooth | 100 | default | lowrank | beta(s,t) | 0.000 | 0.992 | 0.046 | 0.001 | 1.592 | 0.211 |
| lowrank_covariate | binary | smooth | 100 | default | lowrank | gamma(t) | -0.018 | 1.032 | 0.060 | -0.027 | 1.822 | 0.269 |
| lowrank_covariate | binary | iid | 100 | default | lowrank | alpha(t) | 0.022 | 1.109 | 0.077 | 0.047 | 1.016 | 0.058 |
| lowrank_covariate | binary | iid | 100 | default | lowrank | beta(s,t) | 0.009 | 0.702 | 0.007 | 0.009 | 0.653 | 0.004 |
| lowrank_covariate | binary | iid | 100 | default | lowrank | gamma(t) | -0.051 | 1.118 | 0.078 | -0.053 | 1.038 | 0.056 |
| lowrank_covariate | Gaussian | smooth | 100 | default | lowrank | alpha(t) | -0.032 | 1.002 | 0.051 | -0.062 | 2.143 | 0.358 |
| lowrank_covariate | Gaussian | smooth | 100 | default | lowrank | beta(s,t) | 0.001 | 1.018 | 0.055 | 0.005 | 2.089 | 0.334 |
| lowrank_covariate | Gaussian | smooth | 100 | default | lowrank | gamma(t) | -0.027 | 1.000 | 0.051 | -0.065 | 2.108 | 0.345 |
| lowrank_covariate | Gaussian | smooth | 100 | default | lowrank | E(Y | X) | -0.016 | 1.026 | 0.056 | -0.034 | 2.203 | 0.367 |
| lowrank_covariate | Gaussian | iid | 100 | default | lowrank | alpha(t) | -0.011 | 1.005 | 0.044 | -0.008 | 0.993 | 0.044 |
| lowrank_covariate | Gaussian | iid | 100 | default | lowrank | beta(s,t) | 0.003 | 0.724 | 0.009 | 0.003 | 0.702 | 0.006 |
| lowrank_covariate | Gaussian | iid | 100 | default | lowrank | gamma(t) | -0.031 | 0.987 | 0.050 | -0.034 | 0.957 | 0.040 |
| lowrank_covariate | Gaussian | iid | 100 | default | lowrank | E(Y | X) | -0.002 | 0.953 | 0.040 | -0.001 | 0.928 | 0.034 |
| lowrank_covariate | Poisson | smooth | 100 | default | lowrank | alpha(t) | -0.113 | 0.997 | 0.053 | -0.221 | 2.051 | 0.333 |
| lowrank_covariate | Poisson | smooth | 100 | default | lowrank | beta(s,t) | 0.001 | 1.012 | 0.054 | 0.004 | 1.970 | 0.307 |
| lowrank_covariate | Poisson | smooth | 100 | default | lowrank | gamma(t) | -0.033 | 1.004 | 0.055 | -0.068 | 2.029 | 0.337 |
| lowrank_covariate | Poisson | iid | 100 | default | lowrank | alpha(t) | 0.023 | 1.004 | 0.048 | 0.027 | 0.988 | 0.048 |
| lowrank_covariate | Poisson | iid | 100 | default | lowrank | beta(s,t) | 0.006 | 0.720 | 0.008 | 0.006 | 0.694 | 0.006 |
| lowrank_covariate | Poisson | iid | 100 | default | lowrank | gamma(t) | -0.044 | 0.981 | 0.045 | -0.045 | 0.952 | 0.037 |
| oscillating | binary | sign-chg. | 100 | default | rich | alpha(t) | -0.231 | 1.052 | 0.066 | -0.296 | 1.587 | 0.232 |
| oscillating | binary | sign-chg. | 100 | default | rich | beta(s,t) | -0.002 | 0.951 | 0.041 | -0.003 | 1.252 | 0.115 |
| oscillating | binary | sign-chg. | 100 | default | rich | gamma(t) | -0.018 | 1.050 | 0.063 | -0.026 | 1.576 | 0.210 |
| oscillating | Gaussian | sign-chg. | 100 | default | rich | alpha(t) | -0.012 | 1.007 | 0.049 | -0.030 | 2.160 | 0.367 |
| oscillating | Gaussian | sign-chg. | 100 | default | rich | beta(s,t) | 0.002 | 1.029 | 0.058 | 0.006 | 1.917 | 0.300 |
| oscillating | Gaussian | sign-chg. | 100 | default | rich | gamma(t) | -0.023 | 1.016 | 0.056 | -0.052 | 2.129 | 0.355 |
| oscillating | Gaussian | sign-chg. | 100 | default | rich | E(Y | X) | -0.004 | 1.031 | 0.058 | -0.013 | 2.123 | 0.354 |
| oscillating | Poisson | sign-chg. | 100 | default | rich | alpha(t) | -0.117 | 1.015 | 0.053 | -0.226 | 2.058 | 0.344 |
| oscillating | Poisson | sign-chg. | 100 | default | rich | beta(s,t) | 0.001 | 1.027 | 0.058 | 0.004 | 1.801 | 0.272 |
| oscillating | Poisson | sign-chg. | 100 | default | rich | gamma(t) | -0.023 | 1.023 | 0.061 | -0.050 | 2.013 | 0.327 |
| rough_truth | binary | smooth | 100 | default | rich | alpha(t) | -0.246 | 1.057 | 0.074 | -0.362 | 1.833 | 0.282 |
| rough_truth | binary | smooth | 100 | default | rich | beta(s,t) | 0.000 | 1.007 | 0.052 | 0.002 | 1.654 | 0.234 |
| rough_truth | binary | smooth | 100 | default | rich | gamma(t) | -0.005 | 1.045 | 0.065 | -0.026 | 1.847 | 0.268 |
| rough_truth | binary | iid | 100 | default | rich | alpha(t) | 0.070 | 1.070 | 0.066 | 0.084 | 0.995 | 0.051 |
| rough_truth | binary | iid | 100 | default | rich | beta(s,t) | 0.002 | 1.044 | 0.060 | 0.001 | 0.989 | 0.048 |
| rough_truth | binary | iid | 100 | default | rich | gamma(t) | -0.023 | 1.205 | 0.103 | -0.035 | 1.120 | 0.082 |
| rough_truth | Gaussian | smooth | 100 | default | rich | alpha(t) | -0.030 | 1.004 | 0.051 | -0.058 | 2.149 | 0.359 |
| rough_truth | Gaussian | smooth | 100 | default | rich | beta(s,t) | 0.002 | 1.031 | 0.058 | 0.007 | 2.164 | 0.355 |
| rough_truth | Gaussian | smooth | 100 | default | rich | gamma(t) | -0.021 | 1.010 | 0.054 | -0.065 | 2.117 | 0.350 |
| rough_truth | Gaussian | smooth | 100 | default | rich | E(Y | X) | -0.016 | 1.029 | 0.057 | -0.033 | 2.213 | 0.369 |
| rough_truth | Gaussian | iid | 100 | default | rich | alpha(t) | -0.011 | 1.008 | 0.046 | -0.008 | 0.994 | 0.044 |
| rough_truth | Gaussian | iid | 100 | default | rich | beta(s,t) | 0.002 | 0.958 | 0.042 | 0.002 | 0.930 | 0.036 |
| rough_truth | Gaussian | iid | 100 | default | rich | gamma(t) | -0.031 | 1.019 | 0.056 | -0.034 | 0.988 | 0.047 |
| rough_truth | Gaussian | iid | 100 | default | rich | E(Y | X) | -0.003 | 0.995 | 0.050 | -0.003 | 0.969 | 0.043 |
| rough_truth | Poisson | smooth | 100 | default | rich | alpha(t) | -0.114 | 0.994 | 0.051 | -0.214 | 2.024 | 0.333 |
| rough_truth | Poisson | smooth | 100 | default | rich | beta(s,t) | 0.001 | 1.032 | 0.059 | 0.004 | 2.029 | 0.326 |
| rough_truth | Poisson | smooth | 100 | default | rich | gamma(t) | -0.026 | 1.016 | 0.056 | -0.069 | 2.017 | 0.326 |
| rough_truth | Poisson | iid | 100 | default | rich | alpha(t) | 0.043 | 1.013 | 0.049 | 0.046 | 0.999 | 0.048 |
| rough_truth | Poisson | iid | 100 | default | rich | beta(s,t) | 0.001 | 0.941 | 0.040 | 0.001 | 0.912 | 0.033 |
| rough_truth | Poisson | iid | 100 | default | rich | gamma(t) | -0.042 | 1.025 | 0.056 | -0.044 | 0.990 | 0.046 |
| term_type | binary | smooth | 100 | default | rich | f(x,t) | 0.002 | 1.022 | 0.055 | 0.059 | 1.689 | 0.246 |
| term_type | binary | smooth | 100 | default | rich | gamma(t) | 0.041 | 1.051 | 0.057 | 0.076 | 1.807 | 0.277 |
| term_type | binary | iid | 100 | default | rich | f(x,t) | -0.059 | 0.815 | 0.018 | -0.057 | 0.778 | 0.012 |
| term_type | binary | iid | 100 | default | rich | gamma(t) | -0.017 | 1.080 | 0.070 | -0.016 | 0.998 | 0.048 |
| term_type | binary | OU | 100 | default | rich | f(x,t) | -0.010 | 1.005 | 0.051 | 0.040 | 1.556 | 0.209 |
| term_type | binary | OU | 100 | default | rich | gamma(t) | 0.011 | 1.108 | 0.075 | 0.018 | 1.810 | 0.271 |
| term_type | Gaussian | smooth | 100 | default | rich | f(x,t) | -0.006 | 1.035 | 0.058 | -0.013 | 2.123 | 0.350 |
| term_type | Gaussian | smooth | 100 | default | rich | gamma(t) | 0.007 | 1.008 | 0.050 | 0.006 | 2.115 | 0.352 |
| term_type | Gaussian | smooth | 100 | default | rich | E(Y | X) | -0.020 | 1.054 | 0.063 | -0.041 | 2.133 | 0.350 |
| term_type | Gaussian | iid | 100 | default | rich | f(x,t) | -0.021 | 0.897 | 0.029 | -0.022 | 0.878 | 0.024 |
| term_type | Gaussian | iid | 100 | default | rich | gamma(t) | -0.016 | 0.977 | 0.048 | -0.019 | 0.951 | 0.038 |
| term_type | Gaussian | iid | 100 | default | rich | E(Y | X) | 0.000 | 0.944 | 0.039 | -0.001 | 0.915 | 0.032 |
| term_type | Gaussian | OU | 100 | default | rich | f(x,t) | 0.000 | 1.016 | 0.055 | 0.000 | 2.000 | 0.323 |
| term_type | Gaussian | OU | 100 | default | rich | gamma(t) | -0.024 | 1.023 | 0.062 | -0.057 | 2.062 | 0.339 |
| term_type | Gaussian | OU | 100 | default | rich | E(Y | X) | -0.017 | 1.047 | 0.062 | -0.036 | 2.031 | 0.332 |
| term_type | Poisson | smooth | 100 | default | rich | f(x,t) | 0.009 | 1.031 | 0.057 | 0.027 | 2.027 | 0.329 |
| term_type | Poisson | smooth | 100 | default | rich | gamma(t) | 0.015 | 1.017 | 0.054 | 0.025 | 2.035 | 0.336 |
| term_type | Poisson | iid | 100 | default | rich | f(x,t) | -0.048 | 0.884 | 0.027 | -0.047 | 0.863 | 0.022 |
| term_type | Poisson | iid | 100 | default | rich | gamma(t) | -0.004 | 0.994 | 0.051 | -0.008 | 0.963 | 0.041 |
| term_type | Poisson | OU | 100 | default | rich | f(x,t) | 0.010 | 1.014 | 0.055 | 0.029 | 1.918 | 0.303 |
| term_type | Poisson | OU | 100 | default | rich | gamma(t) | -0.012 | 1.024 | 0.062 | -0.031 | 1.979 | 0.327 |
| warp | Gaussian | misreg. | 40 | default | rich | alpha(t) | 0.009 | 1.001 | 0.054 | -0.005 | 1.847 | 0.240 |
| warp | Gaussian | misreg. | 40 | default | rich | beta(s,t) | 0.001 | 1.011 | 0.053 | 0.002 | 1.842 | 0.250 |
| warp | Gaussian | misreg. | 40 | default | rich | gamma(t) | -0.013 | 1.123 | 0.078 | -0.043 | 2.211 | 0.302 |
| warp | Gaussian | misreg. | 40 | default | rich | E(Y | X) | 0.009 | 1.059 | 0.064 | -0.001 | 2.032 | 0.282 |
| warp | Gaussian | misreg. | 100 | default | rich | alpha(t) | 0.015 | 1.012 | 0.049 | 0.021 | 1.847 | 0.251 |
| warp | Gaussian | misreg. | 100 | default | rich | beta(s,t) | 0.001 | 1.009 | 0.052 | 0.005 | 1.887 | 0.259 |
| warp | Gaussian | misreg. | 100 | default | rich | gamma(t) | -0.021 | 1.074 | 0.070 | -0.051 | 2.170 | 0.294 |
| warp | Gaussian | misreg. | 100 | default | rich | E(Y | X) | 0.009 | 1.040 | 0.060 | 0.008 | 2.040 | 0.282 |
| warp_ar1 | Gaussian | misreg. | 100 | default | rich | alpha(t) | 0.012 | 1.024 | 0.051 | 0.017 | 1.943 | 0.268 |
| warp_ar1 | Gaussian | misreg. | 100 | default | rich | beta(s,t) | 0.001 | 1.026 | 0.056 | 0.004 | 1.961 | 0.271 |
| warp_ar1 | Gaussian | misreg. | 100 | default | rich | gamma(t) | -0.020 | 1.087 | 0.072 | -0.048 | 2.264 | 0.308 |
| warp_ar1 | Gaussian | misreg. | 100 | default | rich | E(Y | X) | 0.007 | 1.051 | 0.063 | 0.006 | 2.122 | 0.294 |
| dataset | role | truth | estimand | z_mean_cl2 | z_sd_cl2 | share_big_cl2 | z_mean_model | z_sd_model | share_big_model |
|---|---|---|---|---|---|---|---|---|---|
| ECG strain | counted | REML | beta(s,t) | 0.000 | 1.024 | 0.057 | -0.001 | 3.24 | 0.439 |
| ECG strain | counted | MID | beta(s,t) | -0.003 | 1.025 | 0.057 | 0.000 | 3.25 | 0.440 |
| ECG strain | counted | NCV | beta(s,t) | -0.003 | 1.022 | 0.056 | 0.001 | 3.25 | 0.440 |
| AF trial | counted | REML | beta(s,t) | 0.032 | 1.067 | 0.067 | 0.077 | 2.46 | 0.343 |
| AF trial | counted | MID | beta(s,t) | 0.022 | 1.044 | 0.062 | 0.060 | 2.42 | 0.333 |
| AF trial | counted | NCV | beta(s,t) | 0.021 | 1.028 | 0.058 | 0.063 | 2.37 | 0.326 |
| running | counted | REML | beta(s,t) | 0.003 | 1.143 | 0.085 | -0.003 | 3.65 | 0.520 |
| running | counted | MID | beta(s,t) | 0.002 | 1.115 | 0.079 | -0.004 | 3.40 | 0.484 |
| running | counted | NCV | beta(s,t) | 0.001 | 1.115 | 0.078 | -0.001 | 3.38 | 0.475 |
| DTI | counted | REML | beta(s,t) | 0.006 | 1.118 | 0.080 | 0.014 | 2.36 | 0.386 |
| DTI | counted | MID | beta(s,t) | 0.009 | 1.079 | 0.070 | 0.018 | 2.22 | 0.361 |
| DTI | counted | NCV | beta(s,t) | 0.012 | 1.078 | 0.069 | 0.024 | 2.20 | 0.358 |
| gait | counted | REML | beta(s,t) | 0.002 | 1.048 | 0.061 | -0.006 | 3.95 | 0.487 |
| gait | counted | MID | beta(s,t) | 0.001 | 1.044 | 0.060 | -0.006 | 3.94 | 0.484 |
| gait | counted | NCV | beta(s,t) | 0.000 | 1.045 | 0.060 | -0.007 | 3.93 | 0.480 |
| ECG 8-lead | counted | REML | beta(s,t) | 0.011 | 1.046 | 0.062 | 0.025 | 2.87 | 0.371 |
| ECG 8-lead | counted | MID | beta(s,t) | 0.008 | 1.015 | 0.056 | 0.017 | 2.79 | 0.360 |
| ECG 8-lead | counted | NCV | beta(s,t) | 0.009 | 1.011 | 0.056 | 0.018 | 2.75 | 0.354 |
| ocean | stress test | REML | beta(s,t) | 0.001 | 1.095 | 0.073 | 0.004 | 4.46 | 0.589 |
| ocean | stress test | MID | beta(s,t) | 0.002 | 1.108 | 0.076 | 0.006 | 4.35 | 0.588 |
| ocean | stress test | NCV | beta(s,t) | 0.001 | 1.111 | 0.076 | 0.005 | 4.37 | 0.586 |
| weather | stress test | REML | beta(s,t) | -0.040 | 2.809 | 0.110 | -0.047 | 5.83 | 0.683 |
| weather | stress test | MID | beta(s,t) | -0.018 | 1.866 | 0.091 | -0.016 | 4.60 | 0.642 |
| weather | stress test | NCV | beta(s,t) | -0.087 | 2.861 | 0.153 | -0.083 | 5.47 | 0.651 |
| electricity | stress test | REML | beta(s,t) | 0.001 | 1.392 | 0.091 | -0.016 | 4.70 | 0.621 |
| electricity | stress test | MID | beta(s,t) | -0.005 | 1.077 | 0.063 | -0.023 | 3.74 | 0.541 |
| electricity | stress test | NCV | beta(s,t) | -0.002 | 1.068 | 0.064 | -0.002 | 3.77 | 0.543 |
| ECG strain | counted | REML | E(Y | X) | -0.006 | 1.040 | 0.061 | -0.016 | 3.07 | 0.415 |
| ECG strain | counted | MID | E(Y | X) | -0.006 | 1.040 | 0.061 | -0.018 | 3.09 | 0.418 |
| ECG strain | counted | NCV | E(Y | X) | -0.006 | 1.039 | 0.061 | -0.017 | 3.08 | 0.417 |
| AF trial | counted | REML | E(Y | X) | -0.027 | 1.015 | 0.055 | -0.097 | 2.34 | 0.320 |
| AF trial | counted | MID | E(Y | X) | -0.028 | 1.012 | 0.054 | -0.098 | 2.34 | 0.320 |
| AF trial | counted | NCV | E(Y | X) | -0.028 | 1.011 | 0.054 | -0.094 | 2.34 | 0.320 |
| running | counted | REML | E(Y | X) | -0.011 | 1.062 | 0.065 | -0.028 | 3.30 | 0.433 |
| running | counted | MID | E(Y | X) | -0.010 | 1.068 | 0.067 | -0.026 | 3.27 | 0.428 |
| running | counted | NCV | E(Y | X) | -0.011 | 1.079 | 0.070 | -0.028 | 3.29 | 0.432 |
| DTI | counted | REML | E(Y | X) | 0.010 | 1.067 | 0.067 | 0.013 | 2.20 | 0.358 |
| DTI | counted | MID | E(Y | X) | 0.012 | 1.069 | 0.067 | 0.019 | 2.17 | 0.353 |
| DTI | counted | NCV | E(Y | X) | 0.012 | 1.074 | 0.068 | 0.019 | 2.17 | 0.352 |
| gait | counted | REML | E(Y | X) | -0.015 | 1.028 | 0.057 | -0.069 | 3.62 | 0.462 |
| gait | counted | MID | E(Y | X) | -0.014 | 1.027 | 0.057 | -0.068 | 3.62 | 0.462 |
| gait | counted | NCV | E(Y | X) | -0.014 | 1.029 | 0.057 | -0.066 | 3.62 | 0.462 |
| ECG 8-lead | counted | REML | E(Y | X) | -0.007 | 0.986 | 0.049 | -0.006 | 2.21 | 0.251 |
| ECG 8-lead | counted | MID | E(Y | X) | -0.009 | 0.979 | 0.047 | -0.006 | 2.19 | 0.246 |
| ECG 8-lead | counted | NCV | E(Y | X) | -0.011 | 0.980 | 0.047 | -0.007 | 2.17 | 0.244 |
| ocean | stress test | REML | E(Y | X) | 0.002 | 1.059 | 0.063 | 0.003 | 3.86 | 0.541 |
| ocean | stress test | MID | E(Y | X) | 0.003 | 1.066 | 0.065 | 0.004 | 3.86 | 0.543 |
| ocean | stress test | NCV | E(Y | X) | 0.003 | 1.071 | 0.067 | 0.002 | 3.88 | 0.547 |
| weather | stress test | REML | E(Y | X) | 0.021 | 1.082 | 0.067 | 0.061 | 2.86 | 0.464 |
| weather | stress test | MID | E(Y | X) | 0.022 | 1.102 | 0.075 | 0.056 | 2.81 | 0.458 |
| weather | stress test | NCV | E(Y | X) | 0.006 | 1.148 | 0.088 | 0.050 | 2.80 | 0.457 |
| electricity | stress test | REML | E(Y | X) | 0.016 | 1.073 | 0.069 | 0.032 | 3.43 | 0.511 |
| electricity | stress test | MID | E(Y | X) | 0.004 | 1.089 | 0.073 | -0.004 | 3.32 | 0.495 |
| electricity | stress test | NCV | E(Y | X) | 0.003 | 1.101 | 0.076 | -0.001 | 3.34 | 0.501 |
16.2 S1 Synthetic coverage per cell (mean and β), all arms
| cell | block | family | error | G | signal | truth | D | basis | estimand | NCV + CL2 | NCV + CL2, bias-aware | NCV + CL2 (freq.), bias-aware | NCV, model-based | REML + CL2 | REML, model-based | AR(1) working model |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | core | Gaussian | iid | 40 | low | smooth | 61 | default | beta(s,t) | 0.984 (0.002) | 0.988 (0.001) | 0.924 (0.004) | 0.984 (0.002) | 0.990 (0.001) | 0.994 (0.001) | |
| 1 | core | Gaussian | iid | 40 | low | smooth | 61 | default | E(Y | X) | 0.946 (0.003) | 0.952 (0.003) | 0.910 (0.004) | 0.952 (0.002) | 0.956 (0.002) | 0.967 (0.002) | |
| 2 | core | Gaussian | iid | 100 | low | smooth | 61 | default | beta(s,t) | 0.982 (0.001) | 0.989 (0.001) | 0.939 (0.003) | 0.983 (0.001) | 0.989 (0.001) | 0.992 (0.001) | |
| 2 | core | Gaussian | iid | 100 | low | smooth | 61 | default | E(Y | X) | 0.950 (0.002) | 0.955 (0.002) | 0.922 (0.003) | 0.952 (0.002) | 0.961 (0.002) | 0.968 (0.002) | |
| 3 | core | Gaussian | OU | 40 | low | smooth | 61 | default | beta(s,t) | 0.898 (0.007) | 0.978 (0.003) | 0.972 (0.003) | 0.648 (0.010) | 0.925 (0.004) | 0.635 (0.009) | |
| 3 | core | Gaussian | OU | 40 | low | smooth | 61 | default | E(Y | X) | 0.883 (0.005) | 0.943 (0.003) | 0.937 (0.003) | 0.546 (0.007) | 0.929 (0.003) | 0.617 (0.005) | |
| 4 | core | Gaussian | OU | 100 | low | smooth | 61 | default | beta(s,t) | 0.924 (0.004) | 0.982 (0.002) | 0.976 (0.002) | 0.682 (0.009) | 0.941 (0.003) | 0.671 (0.008) | |
| 4 | core | Gaussian | OU | 100 | low | smooth | 61 | default | E(Y | X) | 0.920 (0.003) | 0.954 (0.002) | 0.948 (0.002) | 0.595 (0.006) | 0.942 (0.002) | 0.641 (0.005) | |
| 5 | core | Gaussian | smooth | 40 | low | smooth | 61 | default | beta(s,t) | 0.902 (0.007) | 0.979 (0.002) | 0.974 (0.003) | 0.633 (0.010) | 0.923 (0.004) | 0.609 (0.010) | |
| 5 | core | Gaussian | smooth | 40 | low | smooth | 61 | default | E(Y | X) | 0.895 (0.005) | 0.949 (0.003) | 0.944 (0.003) | 0.545 (0.007) | 0.930 (0.003) | 0.601 (0.006) | |
| 6 | core | Gaussian | smooth | 100 | low | smooth | 61 | default | beta(s,t) | 0.925 (0.004) | 0.983 (0.002) | 0.977 (0.002) | 0.667 (0.009) | 0.944 (0.003) | 0.658 (0.009) | |
| 6 | core | Gaussian | smooth | 100 | low | smooth | 61 | default | E(Y | X) | 0.921 (0.003) | 0.956 (0.002) | 0.951 (0.002) | 0.586 (0.006) | 0.944 (0.002) | 0.631 (0.005) | |
| 7 | core | Gaussian | iid | 40 | mid | smooth | 61 | default | beta(s,t) | 0.982 (0.002) | 0.989 (0.001) | 0.937 (0.003) | 0.983 (0.002) | 0.989 (0.001) | 0.992 (0.001) | |
| 7 | core | Gaussian | iid | 40 | mid | smooth | 61 | default | E(Y | X) | 0.948 (0.002) | 0.954 (0.002) | 0.921 (0.003) | 0.954 (0.002) | 0.958 (0.002) | 0.968 (0.002) | |
| 8 | core | Gaussian | iid | 100 | mid | smooth | 61 | default | beta(s,t) | 0.982 (0.001) | 0.989 (0.001) | 0.951 (0.003) | 0.983 (0.001) | 0.986 (0.001) | 0.989 (0.001) | |
| 8 | core | Gaussian | iid | 100 | mid | smooth | 61 | default | E(Y | X) | 0.950 (0.002) | 0.958 (0.002) | 0.933 (0.002) | 0.953 (0.002) | 0.961 (0.002) | 0.967 (0.001) | |
| 9 | core | Gaussian | OU | 40 | mid | smooth | 61 | default | beta(s,t) | 0.922 (0.005) | 0.982 (0.002) | 0.976 (0.002) | 0.682 (0.009) | 0.927 (0.004) | 0.638 (0.008) | |
| 9 | core | Gaussian | OU | 40 | mid | smooth | 61 | default | E(Y | X) | 0.920 (0.003) | 0.956 (0.002) | 0.951 (0.002) | 0.603 (0.005) | 0.931 (0.003) | 0.624 (0.005) | |
| 10 | core | Gaussian | OU | 100 | mid | smooth | 61 | default | beta(s,t) | 0.934 (0.004) | 0.983 (0.001) | 0.977 (0.002) | 0.700 (0.007) | 0.940 (0.003) | 0.672 (0.007) | 0.972 (0.002) |
| 10 | core | Gaussian | OU | 100 | mid | smooth | 61 | default | E(Y | X) | 0.931 (0.003) | 0.958 (0.002) | 0.953 (0.002) | 0.623 (0.005) | 0.942 (0.002) | 0.646 (0.005) | 0.934 (0.003) |
| 11 | core | Gaussian | smooth | 40 | mid | smooth | 61 | default | beta(s,t) | 0.924 (0.005) | 0.983 (0.002) | 0.978 (0.002) | 0.670 (0.009) | 0.926 (0.004) | 0.615 (0.009) | |
| 11 | core | Gaussian | smooth | 40 | mid | smooth | 61 | default | E(Y | X) | 0.922 (0.004) | 0.959 (0.003) | 0.954 (0.003) | 0.591 (0.006) | 0.931 (0.003) | 0.606 (0.005) | |
| 12 | core | Gaussian | smooth | 100 | mid | smooth | 61 | default | beta(s,t) | 0.934 (0.004) | 0.983 (0.002) | 0.978 (0.002) | 0.684 (0.008) | 0.944 (0.003) | 0.657 (0.008) | 0.993 (0.001) |
| 12 | core | Gaussian | smooth | 100 | mid | smooth | 61 | default | E(Y | X) | 0.929 (0.003) | 0.959 (0.002) | 0.954 (0.002) | 0.607 (0.005) | 0.944 (0.002) | 0.634 (0.005) | 0.972 (0.002) |
| 13 | core | Gaussian | iid | 40 | high | smooth | 61 | default | beta(s,t) | 0.981 (0.001) | 0.989 (0.001) | 0.948 (0.003) | 0.983 (0.001) | 0.984 (0.001) | 0.988 (0.001) | |
| 13 | core | Gaussian | iid | 40 | high | smooth | 61 | default | E(Y | X) | 0.948 (0.002) | 0.956 (0.002) | 0.930 (0.002) | 0.954 (0.002) | 0.957 (0.002) | 0.966 (0.002) | |
| 14 | core | Gaussian | iid | 100 | high | smooth | 61 | default | beta(s,t) | 0.979 (0.001) | 0.990 (0.001) | 0.959 (0.002) | 0.980 (0.001) | 0.979 (0.001) | 0.983 (0.001) | |
| 14 | core | Gaussian | iid | 100 | high | smooth | 61 | default | E(Y | X) | 0.950 (0.002) | 0.960 (0.002) | 0.941 (0.002) | 0.954 (0.002) | 0.958 (0.002) | 0.963 (0.001) | |
| 15 | core | Gaussian | OU | 40 | high | smooth | 61 | default | beta(s,t) | 0.932 (0.003) | 0.984 (0.001) | 0.978 (0.002) | 0.695 (0.007) | 0.927 (0.004) | 0.637 (0.008) | |
| 15 | core | Gaussian | OU | 40 | high | smooth | 61 | default | E(Y | X) | 0.927 (0.003) | 0.959 (0.002) | 0.955 (0.002) | 0.626 (0.005) | 0.932 (0.002) | 0.628 (0.005) | |
| 16 | core | Gaussian | OU | 100 | high | smooth | 61 | default | beta(s,t) | 0.936 (0.003) | 0.982 (0.001) | 0.976 (0.002) | 0.703 (0.007) | 0.941 (0.003) | 0.670 (0.006) | |
| 16 | core | Gaussian | OU | 100 | high | smooth | 61 | default | E(Y | X) | 0.934 (0.002) | 0.958 (0.002) | 0.954 (0.002) | 0.638 (0.005) | 0.942 (0.002) | 0.651 (0.004) | |
| 17 | core | Gaussian | smooth | 40 | high | smooth | 61 | default | beta(s,t) | 0.931 (0.004) | 0.984 (0.002) | 0.978 (0.002) | 0.684 (0.008) | 0.926 (0.004) | 0.617 (0.008) | |
| 17 | core | Gaussian | smooth | 40 | high | smooth | 61 | default | E(Y | X) | 0.928 (0.003) | 0.960 (0.002) | 0.956 (0.003) | 0.610 (0.005) | 0.932 (0.003) | 0.611 (0.005) | |
| 18 | core | Gaussian | smooth | 100 | high | smooth | 61 | default | beta(s,t) | 0.938 (0.003) | 0.984 (0.001) | 0.978 (0.002) | 0.693 (0.007) | 0.943 (0.003) | 0.653 (0.007) | |
| 18 | core | Gaussian | smooth | 100 | high | smooth | 61 | default | E(Y | X) | 0.933 (0.003) | 0.959 (0.002) | 0.955 (0.002) | 0.621 (0.005) | 0.944 (0.002) | 0.636 (0.004) | |
| 19 | core | Poisson | iid | 40 | mid | smooth | 61 | default | beta(s,t) | 0.985 (0.001) | 0.990 (0.001) | 0.936 (0.003) | 0.986 (0.001) | 0.989 (0.001) | 0.993 (0.001) | |
| 19 | core | Poisson | iid | 40 | mid | smooth | 61 | default | E(Y | X) | 0.947 (0.002) | 0.953 (0.002) | 0.916 (0.003) | 0.954 (0.002) | 0.958 (0.002) | 0.968 (0.002) | |
| 20 | core | Poisson | iid | 100 | mid | smooth | 61 | default | beta(s,t) | 0.982 (0.001) | 0.990 (0.001) | 0.949 (0.002) | 0.983 (0.001) | 0.986 (0.001) | 0.989 (0.001) | |
| 20 | core | Poisson | iid | 100 | mid | smooth | 61 | default | E(Y | X) | 0.948 (0.002) | 0.956 (0.002) | 0.929 (0.002) | 0.952 (0.002) | 0.960 (0.002) | 0.966 (0.002) | |
| 21 | core | Poisson | OU | 40 | mid | smooth | 61 | default | beta(s,t) | 0.921 (0.005) | 0.980 (0.002) | 0.972 (0.003) | 0.705 (0.009) | 0.927 (0.004) | 0.684 (0.008) | |
| 21 | core | Poisson | OU | 40 | mid | smooth | 61 | default | E(Y | X) | 0.910 (0.004) | 0.949 (0.003) | 0.943 (0.003) | 0.612 (0.006) | 0.930 (0.003) | 0.657 (0.005) | |
| 22 | core | Poisson | OU | 100 | mid | smooth | 61 | default | beta(s,t) | 0.936 (0.003) | 0.982 (0.002) | 0.976 (0.002) | 0.715 (0.007) | 0.942 (0.003) | 0.698 (0.007) | |
| 22 | core | Poisson | OU | 100 | mid | smooth | 61 | default | E(Y | X) | 0.929 (0.003) | 0.956 (0.002) | 0.951 (0.002) | 0.632 (0.005) | 0.943 (0.002) | 0.664 (0.004) | |
| 23 | core | Poisson | smooth | 40 | mid | smooth | 61 | default | beta(s,t) | 0.926 (0.005) | 0.983 (0.002) | 0.976 (0.003) | 0.691 (0.009) | 0.928 (0.004) | 0.658 (0.009) | |
| 23 | core | Poisson | smooth | 40 | mid | smooth | 61 | default | E(Y | X) | 0.914 (0.004) | 0.954 (0.003) | 0.948 (0.003) | 0.603 (0.007) | 0.934 (0.003) | 0.642 (0.005) | |
| 24 | core | Poisson | smooth | 100 | mid | smooth | 61 | default | beta(s,t) | 0.935 (0.004) | 0.984 (0.002) | 0.978 (0.002) | 0.706 (0.008) | 0.945 (0.003) | 0.682 (0.008) | |
| 24 | core | Poisson | smooth | 100 | mid | smooth | 61 | default | E(Y | X) | 0.927 (0.003) | 0.957 (0.002) | 0.952 (0.002) | 0.617 (0.005) | 0.945 (0.002) | 0.653 (0.005) | |
| 25 | core | Poisson | iid | 40 | high | smooth | 61 | default | beta(s,t) | 0.981 (0.001) | 0.988 (0.001) | 0.936 (0.003) | 0.983 (0.001) | 0.986 (0.001) | 0.990 (0.001) | |
| 25 | core | Poisson | iid | 40 | high | smooth | 61 | default | E(Y | X) | 0.939 (0.003) | 0.950 (0.002) | 0.918 (0.003) | 0.948 (0.002) | 0.955 (0.002) | 0.966 (0.002) | |
| 26 | core | Poisson | iid | 100 | high | smooth | 61 | default | beta(s,t) | 0.976 (0.002) | 0.986 (0.001) | 0.948 (0.002) | 0.979 (0.002) | 0.979 (0.001) | 0.985 (0.001) | |
| 26 | core | Poisson | iid | 100 | high | smooth | 61 | default | E(Y | X) | 0.941 (0.002) | 0.954 (0.002) | 0.930 (0.003) | 0.947 (0.002) | 0.955 (0.002) | 0.962 (0.002) | |
| 27 | core | Poisson | OU | 40 | high | smooth | 61 | default | beta(s,t) | 0.922 (0.004) | 0.979 (0.002) | 0.970 (0.002) | 0.715 (0.008) | 0.929 (0.004) | 0.703 (0.007) | |
| 27 | core | Poisson | OU | 40 | high | smooth | 61 | default | E(Y | X) | 0.902 (0.004) | 0.947 (0.003) | 0.940 (0.003) | 0.627 (0.006) | 0.930 (0.003) | 0.674 (0.005) | |
| 28 | core | Poisson | OU | 100 | high | smooth | 61 | default | beta(s,t) | 0.933 (0.003) | 0.981 (0.001) | 0.974 (0.002) | 0.723 (0.006) | 0.940 (0.003) | 0.705 (0.007) | |
| 28 | core | Poisson | OU | 100 | high | smooth | 61 | default | E(Y | X) | 0.922 (0.003) | 0.953 (0.002) | 0.947 (0.002) | 0.643 (0.005) | 0.940 (0.002) | 0.672 (0.005) | |
| 29 | core | Poisson | smooth | 40 | high | smooth | 61 | default | beta(s,t) | 0.923 (0.004) | 0.979 (0.002) | 0.972 (0.002) | 0.699 (0.008) | 0.930 (0.004) | 0.683 (0.008) | |
| 29 | core | Poisson | smooth | 40 | high | smooth | 61 | default | E(Y | X) | 0.905 (0.004) | 0.949 (0.003) | 0.943 (0.003) | 0.610 (0.006) | 0.934 (0.003) | 0.656 (0.005) | |
| 30 | core | Poisson | smooth | 100 | high | smooth | 61 | default | beta(s,t) | 0.929 (0.003) | 0.980 (0.002) | 0.973 (0.002) | 0.711 (0.007) | 0.944 (0.003) | 0.696 (0.007) | |
| 30 | core | Poisson | smooth | 100 | high | smooth | 61 | default | E(Y | X) | 0.923 (0.003) | 0.954 (0.002) | 0.949 (0.002) | 0.625 (0.006) | 0.945 (0.002) | 0.662 (0.004) | |
| 31 | core | binary | iid | 40 | mid | smooth | 61 | default | beta(s,t) | 0.973 (0.005) | 0.979 (0.004) | 0.893 (0.007) | 0.974 (0.004) | 0.973 (0.005) | 0.984 (0.004) | |
| 31 | core | binary | iid | 40 | mid | smooth | 61 | default | E(Y | X) | 0.923 (0.004) | 0.935 (0.004) | 0.879 (0.005) | 0.927 (0.004) | 0.925 (0.004) | 0.952 (0.003) | |
| 32 | core | binary | iid | 100 | mid | smooth | 61 | default | beta(s,t) | 0.982 (0.002) | 0.987 (0.001) | 0.911 (0.005) | 0.982 (0.002) | 0.990 (0.001) | 0.994 (0.001) | |
| 32 | core | binary | iid | 100 | mid | smooth | 61 | default | E(Y | X) | 0.939 (0.003) | 0.945 (0.003) | 0.896 (0.004) | 0.942 (0.003) | 0.949 (0.003) | 0.963 (0.002) | |
| 33 | core | binary | OU | 40 | mid | smooth | 61 | default | beta(s,t) | 0.790 (0.014) | 0.948 (0.005) | 0.935 (0.005) | 0.627 (0.017) | 0.937 (0.004) | 0.800 (0.008) | |
| 33 | core | binary | OU | 40 | mid | smooth | 61 | default | E(Y | X) | 0.842 (0.007) | 0.930 (0.003) | 0.920 (0.004) | 0.587 (0.008) | 0.927 (0.003) | 0.747 (0.006) | |
| 34 | core | binary | OU | 100 | mid | smooth | 61 | default | beta(s,t) | 0.901 (0.008) | 0.971 (0.003) | 0.958 (0.004) | 0.738 (0.011) | 0.951 (0.003) | 0.819 (0.007) | |
| 34 | core | binary | OU | 100 | mid | smooth | 61 | default | E(Y | X) | 0.876 (0.005) | 0.933 (0.003) | 0.922 (0.003) | 0.628 (0.007) | 0.940 (0.003) | 0.752 (0.005) | |
| 35 | core | binary | smooth | 40 | mid | smooth | 61 | default | beta(s,t) | 0.785 (0.013) | 0.953 (0.005) | 0.944 (0.005) | 0.569 (0.016) | 0.926 (0.005) | 0.749 (0.010) | |
| 35 | core | binary | smooth | 40 | mid | smooth | 61 | default | E(Y | X) | 0.857 (0.006) | 0.942 (0.003) | 0.935 (0.003) | 0.550 (0.008) | 0.921 (0.004) | 0.704 (0.006) | |
| 36 | core | binary | smooth | 100 | mid | smooth | 61 | default | beta(s,t) | 0.896 (0.008) | 0.972 (0.003) | 0.962 (0.003) | 0.708 (0.012) | 0.952 (0.003) | 0.779 (0.008) | |
| 36 | core | binary | smooth | 100 | mid | smooth | 61 | default | E(Y | X) | 0.883 (0.004) | 0.939 (0.003) | 0.930 (0.003) | 0.601 (0.007) | 0.941 (0.003) | 0.722 (0.005) | |
| 37 | core | binary | iid | 40 | high | smooth | 61 | default | beta(s,t) | 0.978 (0.002) | 0.985 (0.002) | 0.912 (0.005) | 0.980 (0.002) | 0.986 (0.003) | 0.991 (0.003) | |
| 37 | core | binary | iid | 40 | high | smooth | 61 | default | E(Y | X) | 0.937 (0.003) | 0.945 (0.003) | 0.895 (0.004) | 0.943 (0.003) | 0.949 (0.003) | 0.964 (0.003) | |
| 38 | core | binary | iid | 100 | high | smooth | 61 | default | beta(s,t) | 0.980 (0.002) | 0.987 (0.001) | 0.928 (0.004) | 0.982 (0.002) | 0.989 (0.001) | 0.993 (0.001) | |
| 38 | core | binary | iid | 100 | high | smooth | 61 | default | E(Y | X) | 0.943 (0.002) | 0.951 (0.002) | 0.911 (0.003) | 0.948 (0.002) | 0.959 (0.002) | 0.967 (0.002) | |
| 39 | core | binary | OU | 40 | high | smooth | 61 | default | beta(s,t) | 0.905 (0.008) | 0.974 (0.003) | 0.959 (0.004) | 0.755 (0.010) | 0.940 (0.004) | 0.813 (0.008) | |
| 39 | core | binary | OU | 40 | high | smooth | 61 | default | E(Y | X) | 0.871 (0.005) | 0.935 (0.003) | 0.922 (0.003) | 0.651 (0.007) | 0.929 (0.003) | 0.761 (0.005) | |
| 40 | core | binary | OU | 100 | high | smooth | 61 | default | beta(s,t) | 0.930 (0.004) | 0.980 (0.002) | 0.968 (0.002) | 0.780 (0.008) | 0.951 (0.003) | 0.820 (0.007) | |
| 40 | core | binary | OU | 100 | high | smooth | 61 | default | E(Y | X) | 0.913 (0.003) | 0.949 (0.002) | 0.938 (0.002) | 0.699 (0.005) | 0.944 (0.002) | 0.766 (0.004) | |
| 41 | core | binary | smooth | 40 | high | smooth | 61 | default | beta(s,t) | 0.897 (0.008) | 0.974 (0.003) | 0.962 (0.004) | 0.722 (0.011) | 0.932 (0.004) | 0.765 (0.009) | |
| 41 | core | binary | smooth | 40 | high | smooth | 61 | default | E(Y | X) | 0.874 (0.005) | 0.942 (0.003) | 0.931 (0.003) | 0.624 (0.007) | 0.923 (0.003) | 0.713 (0.006) | |
| 42 | core | binary | smooth | 100 | high | smooth | 61 | default | beta(s,t) | 0.925 (0.005) | 0.980 (0.002) | 0.970 (0.002) | 0.759 (0.008) | 0.952 (0.003) | 0.783 (0.007) | |
| 42 | core | binary | smooth | 100 | high | smooth | 61 | default | E(Y | X) | 0.907 (0.004) | 0.948 (0.003) | 0.939 (0.003) | 0.663 (0.006) | 0.943 (0.002) | 0.729 (0.005) | |
| 43 | warp | Gaussian | misreg. | 40 | high | smooth | 61 | default | beta(s,t) | 0.937 (0.004) | 0.974 (0.003) | 0.956 (0.004) | 0.752 (0.008) | 0.949 (0.004) | 0.761 (0.007) | |
| 43 | warp | Gaussian | misreg. | 40 | high | smooth | 61 | default | E(Y | X) | 0.919 (0.004) | 0.946 (0.003) | 0.934 (0.004) | 0.696 (0.007) | 0.938 (0.003) | 0.729 (0.005) | |
| 44 | warp | Gaussian | misreg. | 40 | high | wiggly | 61 | default | beta(s,t) | 0.820 (0.008) | 0.928 (0.003) | 0.901 (0.004) | 0.595 (0.009) | 0.945 (0.003) | 0.739 (0.006) | |
| 44 | warp | Gaussian | misreg. | 40 | high | wiggly | 61 | default | E(Y | X) | 0.885 (0.004) | 0.932 (0.003) | 0.918 (0.003) | 0.635 (0.006) | 0.934 (0.003) | 0.706 (0.005) | |
| 45 | warp | Gaussian | misreg. | 100 | high | smooth | 61 | default | beta(s,t) | 0.937 (0.004) | 0.976 (0.002) | 0.959 (0.003) | 0.739 (0.008) | 0.951 (0.003) | 0.753 (0.007) | 0.875 (0.005) |
| 45 | warp | Gaussian | misreg. | 100 | high | smooth | 61 | default | E(Y | X) | 0.930 (0.004) | 0.952 (0.003) | 0.941 (0.003) | 0.706 (0.006) | 0.942 (0.003) | 0.730 (0.005) | 0.844 (0.005) |
| 46 | warp | Gaussian | misreg. | 100 | high | wiggly | 61 | default | beta(s,t) | 0.863 (0.006) | 0.934 (0.003) | 0.908 (0.004) | 0.628 (0.007) | 0.944 (0.003) | 0.729 (0.006) | |
| 46 | warp | Gaussian | misreg. | 100 | high | wiggly | 61 | default | E(Y | X) | 0.911 (0.003) | 0.938 (0.003) | 0.926 (0.003) | 0.665 (0.005) | 0.937 (0.003) | 0.706 (0.005) | |
| 47 | warp | Poisson | misreg. | 40 | high | smooth | 61 | default | E(Y | X) | 0.792 (0.007) | 0.910 (0.004) | 0.900 (0.004) | 0.486 (0.010) | 0.902 (0.004) | 0.634 (0.005) | |
| 48 | warp | Poisson | misreg. | 40 | high | wiggly | 61 | default | E(Y | X) | 0.773 (0.007) | 0.910 (0.003) | 0.902 (0.003) | 0.454 (0.009) | 0.899 (0.003) | 0.614 (0.005) | |
| 49 | warp | Poisson | misreg. | 100 | high | smooth | 61 | default | E(Y | X) | 0.801 (0.007) | 0.922 (0.003) | 0.915 (0.003) | 0.438 (0.009) | 0.910 (0.003) | 0.571 (0.004) | |
| 50 | warp | Poisson | misreg. | 100 | high | wiggly | 61 | default | E(Y | X) | 0.780 (0.006) | 0.918 (0.003) | 0.912 (0.003) | 0.409 (0.007) | 0.906 (0.003) | 0.558 (0.004) | |
| 51 | warp | binary | misreg. | 40 | high | smooth | 61 | default | E(Y | X) | 0.919 (0.004) | 0.932 (0.003) | 0.884 (0.004) | 0.901 (0.004) | 0.944 (0.003) | 0.945 (0.003) | |
| 52 | warp | binary | misreg. | 40 | high | wiggly | 61 | default | E(Y | X) | 0.892 (0.004) | 0.908 (0.004) | 0.854 (0.005) | 0.871 (0.005) | 0.925 (0.003) | 0.924 (0.003) | |
| 53 | warp | binary | misreg. | 100 | high | smooth | 61 | default | E(Y | X) | 0.931 (0.003) | 0.941 (0.003) | 0.902 (0.003) | 0.909 (0.004) | 0.954 (0.002) | 0.943 (0.003) | |
| 54 | warp | binary | misreg. | 100 | high | wiggly | 61 | default | E(Y | X) | 0.909 (0.003) | 0.919 (0.003) | 0.878 (0.004) | 0.882 (0.004) | 0.932 (0.003) | 0.918 (0.003) | |
| 55 | dense_grid | Gaussian | iid | 100 | mid | smooth | 241 | default | beta(s,t) | 0.976 (0.001) | 0.988 (0.001) | 0.956 (0.002) | 0.978 (0.001) | 0.976 (0.001) | 0.980 (0.001) | |
| 55 | dense_grid | Gaussian | iid | 100 | mid | smooth | 241 | default | E(Y | X) | 0.947 (0.002) | 0.957 (0.002) | 0.938 (0.002) | 0.951 (0.002) | 0.956 (0.002) | 0.961 (0.001) | |
| 56 | dense_grid | Gaussian | OU | 100 | mid | smooth | 241 | default | beta(s,t) | 0.917 (0.004) | 0.988 (0.001) | 0.987 (0.001) | 0.406 (0.006) | 0.940 (0.002) | 0.359 (0.004) | |
| 56 | dense_grid | Gaussian | OU | 100 | mid | smooth | 241 | default | E(Y | X) | 0.926 (0.003) | 0.966 (0.002) | 0.965 (0.002) | 0.348 (0.004) | 0.941 (0.002) | 0.367 (0.003) | |
| 57 | dense_grid | Gaussian | smooth | 100 | mid | smooth | 241 | default | beta(s,t) | 0.916 (0.004) | 0.989 (0.001) | 0.988 (0.001) | 0.391 (0.007) | 0.944 (0.002) | 0.350 (0.004) | |
| 57 | dense_grid | Gaussian | smooth | 100 | mid | smooth | 241 | default | E(Y | X) | 0.923 (0.003) | 0.967 (0.002) | 0.966 (0.002) | 0.333 (0.004) | 0.943 (0.002) | 0.353 (0.003) | |
| 58 | dense_grid | Poisson | iid | 100 | mid | smooth | 241 | default | beta(s,t) | 0.978 (0.001) | 0.988 (0.001) | 0.955 (0.002) | 0.979 (0.001) | 0.978 (0.001) | 0.983 (0.001) | |
| 58 | dense_grid | Poisson | iid | 100 | mid | smooth | 241 | default | E(Y | X) | 0.947 (0.002) | 0.958 (0.002) | 0.936 (0.002) | 0.951 (0.002) | 0.957 (0.002) | 0.963 (0.001) | |
| 59 | dense_grid | Poisson | OU | 100 | mid | smooth | 241 | default | beta(s,t) | 0.915 (0.004) | 0.988 (0.001) | 0.986 (0.001) | 0.418 (0.006) | 0.938 (0.002) | 0.374 (0.004) | |
| 59 | dense_grid | Poisson | OU | 100 | mid | smooth | 241 | default | E(Y | X) | 0.922 (0.003) | 0.965 (0.002) | 0.964 (0.002) | 0.357 (0.004) | 0.941 (0.002) | 0.380 (0.003) | |
| 60 | dense_grid | Poisson | smooth | 100 | mid | smooth | 241 | default | beta(s,t) | 0.915 (0.004) | 0.988 (0.001) | 0.987 (0.001) | 0.406 (0.007) | 0.943 (0.003) | 0.369 (0.004) | |
| 60 | dense_grid | Poisson | smooth | 100 | mid | smooth | 241 | default | E(Y | X) | 0.921 (0.003) | 0.966 (0.002) | 0.965 (0.002) | 0.342 (0.004) | 0.943 (0.002) | 0.370 (0.003) | |
| 61 | dense_grid | binary | iid | 100 | mid | smooth | 241 | default | beta(s,t) | 0.984 (0.001) | 0.990 (0.001) | 0.937 (0.003) | 0.985 (0.001) | 0.990 (0.001) | 0.993 (0.001) | |
| 61 | dense_grid | binary | iid | 100 | mid | smooth | 241 | default | E(Y | X) | 0.951 (0.002) | 0.956 (0.002) | 0.921 (0.003) | 0.955 (0.002) | 0.962 (0.002) | 0.970 (0.002) | |
| 62 | dense_grid | binary | OU | 100 | mid | smooth | 241 | default | beta(s,t) | 0.868 (0.010) | 0.986 (0.001) | 0.984 (0.001) | 0.464 (0.010) | 0.938 (0.002) | 0.450 (0.005) | |
| 62 | dense_grid | binary | OU | 100 | mid | smooth | 241 | default | E(Y | X) | 0.861 (0.006) | 0.953 (0.002) | 0.951 (0.002) | 0.365 (0.005) | 0.938 (0.002) | 0.452 (0.003) | |
| 63 | dense_grid | binary | smooth | 100 | mid | smooth | 241 | default | beta(s,t) | 0.866 (0.009) | 0.986 (0.001) | 0.984 (0.001) | 0.422 (0.010) | 0.943 (0.002) | 0.420 (0.005) | |
| 63 | dense_grid | binary | smooth | 100 | mid | smooth | 241 | default | E(Y | X) | 0.872 (0.005) | 0.959 (0.002) | 0.958 (0.002) | 0.337 (0.005) | 0.941 (0.002) | 0.418 (0.003) | |
| 64 | rough_truth | Gaussian | iid | 100 | low | wiggly | 61 | default | beta(s,t) | 0.933 (0.004) | 0.938 (0.004) | 0.843 (0.007) | 0.933 (0.004) | 0.948 (0.003) | 0.957 (0.002) | |
| 64 | rough_truth | Gaussian | iid | 100 | low | wiggly | 61 | default | E(Y | X) | 0.942 (0.002) | 0.945 (0.002) | 0.913 (0.003) | 0.945 (0.002) | 0.948 (0.002) | 0.956 (0.002) | |
| 65 | rough_truth | Gaussian | smooth | 100 | low | wiggly | 61 | default | beta(s,t) | 0.803 (0.006) | 0.945 (0.002) | 0.934 (0.003) | 0.515 (0.006) | 0.942 (0.003) | 0.645 (0.007) | |
| 65 | rough_truth | Gaussian | smooth | 100 | low | wiggly | 61 | default | E(Y | X) | 0.912 (0.003) | 0.948 (0.002) | 0.943 (0.002) | 0.579 (0.005) | 0.943 (0.002) | 0.627 (0.005) | |
| 66 | rough_truth | Gaussian | iid | 100 | mid | wiggly | 61 | default | beta(s,t) | 0.953 (0.003) | 0.955 (0.003) | 0.894 (0.004) | 0.955 (0.003) | 0.961 (0.002) | 0.967 (0.002) | |
| 66 | rough_truth | Gaussian | iid | 100 | mid | wiggly | 61 | default | E(Y | X) | 0.946 (0.002) | 0.949 (0.002) | 0.926 (0.002) | 0.951 (0.002) | 0.952 (0.002) | 0.958 (0.002) | |
| 67 | rough_truth | Gaussian | smooth | 100 | mid | wiggly | 61 | default | beta(s,t) | 0.824 (0.007) | 0.935 (0.003) | 0.924 (0.003) | 0.528 (0.007) | 0.941 (0.003) | 0.642 (0.007) | |
| 67 | rough_truth | Gaussian | smooth | 100 | mid | wiggly | 61 | default | E(Y | X) | 0.920 (0.002) | 0.948 (0.002) | 0.944 (0.002) | 0.597 (0.004) | 0.944 (0.002) | 0.631 (0.004) | |
| 68 | rough_truth | Gaussian | iid | 100 | high | wiggly | 61 | default | beta(s,t) | 0.954 (0.002) | 0.959 (0.002) | 0.916 (0.003) | 0.959 (0.002) | 0.963 (0.002) | 0.969 (0.002) | |
| 68 | rough_truth | Gaussian | iid | 100 | high | wiggly | 61 | default | E(Y | X) | 0.946 (0.002) | 0.949 (0.002) | 0.933 (0.002) | 0.951 (0.002) | 0.952 (0.002) | 0.958 (0.001) | |
| 69 | rough_truth | Gaussian | smooth | 100 | high | wiggly | 61 | default | beta(s,t) | 0.900 (0.005) | 0.950 (0.003) | 0.940 (0.003) | 0.587 (0.007) | 0.942 (0.003) | 0.649 (0.006) | |
| 69 | rough_truth | Gaussian | smooth | 100 | high | wiggly | 61 | default | E(Y | X) | 0.932 (0.002) | 0.950 (0.002) | 0.946 (0.002) | 0.614 (0.004) | 0.944 (0.002) | 0.635 (0.004) | |
| 70 | rough_truth | Poisson | iid | 100 | mid | wiggly | 61 | default | beta(s,t) | 0.952 (0.003) | 0.955 (0.002) | 0.889 (0.004) | 0.955 (0.002) | 0.960 (0.002) | 0.966 (0.002) | |
| 70 | rough_truth | Poisson | iid | 100 | mid | wiggly | 61 | default | E(Y | X) | 0.944 (0.002) | 0.947 (0.002) | 0.922 (0.002) | 0.948 (0.002) | 0.950 (0.002) | 0.957 (0.002) | |
| 71 | rough_truth | Poisson | smooth | 100 | mid | wiggly | 61 | default | beta(s,t) | 0.811 (0.007) | 0.932 (0.003) | 0.920 (0.003) | 0.530 (0.007) | 0.943 (0.003) | 0.670 (0.007) | |
| 71 | rough_truth | Poisson | smooth | 100 | mid | wiggly | 61 | default | E(Y | X) | 0.918 (0.003) | 0.947 (0.002) | 0.942 (0.002) | 0.607 (0.004) | 0.944 (0.002) | 0.652 (0.004) | |
| 72 | rough_truth | Poisson | iid | 100 | high | wiggly | 61 | default | beta(s,t) | 0.950 (0.002) | 0.955 (0.002) | 0.903 (0.004) | 0.955 (0.002) | 0.961 (0.002) | 0.968 (0.002) | |
| 72 | rough_truth | Poisson | iid | 100 | high | wiggly | 61 | default | E(Y | X) | 0.940 (0.002) | 0.946 (0.002) | 0.925 (0.002) | 0.945 (0.002) | 0.949 (0.002) | 0.957 (0.002) | |
| 73 | rough_truth | Poisson | smooth | 100 | high | wiggly | 61 | default | beta(s,t) | 0.860 (0.007) | 0.935 (0.003) | 0.924 (0.004) | 0.578 (0.007) | 0.940 (0.003) | 0.677 (0.006) | |
| 73 | rough_truth | Poisson | smooth | 100 | high | wiggly | 61 | default | E(Y | X) | 0.919 (0.003) | 0.945 (0.002) | 0.940 (0.002) | 0.617 (0.005) | 0.943 (0.002) | 0.661 (0.004) | |
| 74 | rough_truth | binary | iid | 100 | mid | wiggly | 61 | default | beta(s,t) | 0.911 (0.004) | 0.921 (0.004) | 0.784 (0.008) | 0.912 (0.004) | 0.935 (0.003) | 0.950 (0.002) | |
| 74 | rough_truth | binary | iid | 100 | mid | wiggly | 61 | default | E(Y | X) | 0.926 (0.003) | 0.934 (0.003) | 0.886 (0.004) | 0.929 (0.003) | 0.927 (0.003) | 0.944 (0.002) | |
| 75 | rough_truth | binary | smooth | 100 | mid | wiggly | 61 | default | beta(s,t) | 0.799 (0.009) | 0.940 (0.004) | 0.923 (0.004) | 0.587 (0.010) | 0.948 (0.003) | 0.766 (0.008) | |
| 75 | rough_truth | binary | smooth | 100 | mid | wiggly | 61 | default | E(Y | X) | 0.872 (0.005) | 0.933 (0.003) | 0.924 (0.003) | 0.596 (0.007) | 0.940 (0.002) | 0.718 (0.005) | |
| 76 | rough_truth | binary | iid | 100 | high | wiggly | 61 | default | beta(s,t) | 0.919 (0.005) | 0.929 (0.005) | 0.820 (0.008) | 0.921 (0.005) | 0.944 (0.003) | 0.954 (0.003) | |
| 76 | rough_truth | binary | iid | 100 | high | wiggly | 61 | default | E(Y | X) | 0.934 (0.002) | 0.938 (0.002) | 0.898 (0.003) | 0.939 (0.002) | 0.942 (0.002) | 0.952 (0.002) | |
| 77 | rough_truth | binary | smooth | 100 | high | wiggly | 61 | default | beta(s,t) | 0.803 (0.006) | 0.931 (0.003) | 0.912 (0.004) | 0.598 (0.006) | 0.947 (0.003) | 0.767 (0.007) | |
| 77 | rough_truth | binary | smooth | 100 | high | wiggly | 61 | default | E(Y | X) | 0.899 (0.003) | 0.941 (0.002) | 0.931 (0.003) | 0.655 (0.005) | 0.943 (0.002) | 0.729 (0.004) | |
| 78 | term_type | Gaussian | iid | 100 | mid | smooth | 61 | default | E(Y | X) | 0.956 (0.002) | 0.965 (0.001) | 0.933 (0.002) | 0.959 (0.002) | 0.961 (0.001) | 0.968 (0.001) | |
| 79 | term_type | Gaussian | OU | 100 | mid | smooth | 61 | default | E(Y | X) | 0.925 (0.003) | 0.957 (0.002) | 0.949 (0.002) | 0.628 (0.005) | 0.938 (0.002) | 0.668 (0.004) | |
| 80 | term_type | Gaussian | smooth | 100 | mid | smooth | 61 | default | E(Y | X) | 0.925 (0.003) | 0.957 (0.002) | 0.950 (0.002) | 0.614 (0.005) | 0.937 (0.002) | 0.650 (0.004) | |
| 81 | term_type | Poisson | iid | 100 | mid | smooth | 61 | default | E(Y | X) | 0.947 (0.002) | 0.958 (0.002) | 0.921 (0.002) | 0.950 (0.002) | 0.961 (0.001) | 0.969 (0.001) | |
| 82 | term_type | Poisson | OU | 100 | mid | smooth | 61 | default | E(Y | X) | 0.910 (0.003) | 0.951 (0.002) | 0.943 (0.002) | 0.632 (0.005) | 0.939 (0.002) | 0.689 (0.004) | |
| 83 | term_type | Poisson | smooth | 100 | mid | smooth | 61 | default | E(Y | X) | 0.911 (0.003) | 0.952 (0.002) | 0.945 (0.002) | 0.620 (0.005) | 0.936 (0.002) | 0.668 (0.004) | |
| 84 | term_type | binary | iid | 100 | mid | smooth | 61 | default | E(Y | X) | 0.945 (0.002) | 0.954 (0.002) | 0.887 (0.004) | 0.949 (0.003) | 0.963 (0.002) | 0.974 (0.002) | |
| 85 | term_type | binary | OU | 100 | mid | smooth | 61 | default | E(Y | X) | 0.864 (0.006) | 0.934 (0.004) | 0.917 (0.004) | 0.648 (0.007) | 0.936 (0.002) | 0.772 (0.004) | |
| 86 | term_type | binary | smooth | 100 | mid | smooth | 61 | default | E(Y | X) | 0.877 (0.005) | 0.945 (0.003) | 0.933 (0.003) | 0.629 (0.007) | 0.936 (0.002) | 0.743 (0.005) | |
| 87 | families | scaled t | iid | 100 | mid | smooth | 61 | default | beta(s,t) | 0.982 (0.001) | 0.990 (0.001) | 0.954 (0.002) | 0.983 (0.001) | 0.985 (0.001) | 0.988 (0.001) | |
| 87 | families | scaled t | iid | 100 | mid | smooth | 61 | default | E(Y | X) | 0.950 (0.002) | 0.959 (0.002) | 0.936 (0.002) | 0.953 (0.002) | 0.960 (0.001) | 0.965 (0.001) | |
| 88 | families | scaled t | smooth | 100 | mid | smooth | 61 | default | beta(s,t) | 0.940 (0.004) | 0.985 (0.001) | 0.979 (0.002) | 0.703 (0.008) | 0.944 (0.003) | 0.671 (0.007) | |
| 88 | families | scaled t | smooth | 100 | mid | smooth | 61 | default | E(Y | X) | 0.938 (0.003) | 0.962 (0.002) | 0.958 (0.002) | 0.636 (0.005) | 0.947 (0.002) | 0.644 (0.005) | |
| 89 | families | beta | iid | 100 | mid | smooth | 61 | default | beta(s,t) | 0.979 (0.002) | 0.988 (0.001) | 0.947 (0.003) | 0.981 (0.002) | 0.984 (0.001) | 0.989 (0.001) | |
| 89 | families | beta | iid | 100 | mid | smooth | 61 | default | E(Y | X) | 0.946 (0.002) | 0.956 (0.002) | 0.930 (0.002) | 0.953 (0.002) | 0.958 (0.002) | 0.966 (0.001) | |
| 90 | families | beta | smooth | 100 | mid | smooth | 61 | default | beta(s,t) | 0.929 (0.004) | 0.982 (0.002) | 0.976 (0.002) | 0.691 (0.008) | 0.941 (0.003) | 0.659 (0.007) | |
| 90 | families | beta | smooth | 100 | mid | smooth | 61 | default | E(Y | X) | 0.924 (0.003) | 0.956 (0.002) | 0.951 (0.002) | 0.623 (0.005) | 0.941 (0.002) | 0.635 (0.004) | |
| 91 | families | negative binomial | iid | 100 | mid | smooth | 61 | default | beta(s,t) | 0.982 (0.002) | 0.990 (0.001) | 0.944 (0.003) | 0.983 (0.001) | 0.988 (0.001) | 0.991 (0.001) | |
| 91 | families | negative binomial | iid | 100 | mid | smooth | 61 | default | E(Y | X) | 0.950 (0.002) | 0.958 (0.002) | 0.930 (0.003) | 0.954 (0.002) | 0.962 (0.002) | 0.968 (0.002) | |
| 92 | families | negative binomial | smooth | 100 | mid | smooth | 61 | default | beta(s,t) | 0.932 (0.004) | 0.983 (0.002) | 0.977 (0.002) | 0.699 (0.009) | 0.943 (0.003) | 0.669 (0.008) | |
| 92 | families | negative binomial | smooth | 100 | mid | smooth | 61 | default | E(Y | X) | 0.925 (0.003) | 0.956 (0.002) | 0.951 (0.002) | 0.622 (0.005) | 0.943 (0.002) | 0.641 (0.005) | |
| 93 | families | negative binomial | iid | 100 | mid | smooth | 61 | default | beta(s,t) | 0.962 (0.002) | 0.982 (0.001) | 0.955 (0.002) | 0.929 (0.003) | 0.972 (0.002) | 0.945 (0.002) | |
| 93 | families | negative binomial | iid | 100 | mid | smooth | 61 | default | E(Y | X) | 0.934 (0.002) | 0.950 (0.002) | 0.932 (0.002) | 0.868 (0.004) | 0.952 (0.002) | 0.898 (0.003) | |
| 94 | families | negative binomial | smooth | 100 | mid | smooth | 61 | default | beta(s,t) | 0.913 (0.005) | 0.984 (0.001) | 0.981 (0.002) | 0.558 (0.008) | 0.938 (0.003) | 0.531 (0.006) | |
| 94 | families | negative binomial | smooth | 100 | mid | smooth | 61 | default | E(Y | X) | 0.913 (0.003) | 0.957 (0.002) | 0.954 (0.002) | 0.486 (0.005) | 0.939 (0.002) | 0.529 (0.004) | |
| 95 | ar1_home | Gaussian | AR(1) | 100 | mid | smooth | 61 | default | beta(s,t) | 0.933 (0.003) | 0.984 (0.001) | 0.979 (0.002) | 0.684 (0.007) | 0.940 (0.003) | 0.651 (0.007) | 0.989 (0.001) |
| 95 | ar1_home | Gaussian | AR(1) | 100 | mid | smooth | 61 | default | E(Y | X) | 0.930 (0.003) | 0.959 (0.002) | 0.955 (0.002) | 0.603 (0.005) | 0.942 (0.002) | 0.625 (0.004) | 0.961 (0.002) |
| 96 | oscillating | Gaussian | sign-chg. | 100 | mid | smooth | 61 | default | beta(s,t) | 0.935 (0.003) | 0.985 (0.001) | 0.977 (0.001) | 0.782 (0.005) | 0.942 (0.003) | 0.700 (0.006) | 0.992 (0.001) |
| 96 | oscillating | Gaussian | sign-chg. | 100 | mid | smooth | 61 | default | E(Y | X) | 0.915 (0.003) | 0.957 (0.002) | 0.951 (0.002) | 0.642 (0.005) | 0.942 (0.002) | 0.646 (0.004) | 0.963 (0.002) |
| 97 | oscillating | Poisson | sign-chg. | 100 | mid | smooth | 61 | default | beta(s,t) | 0.935 (0.003) | 0.986 (0.001) | 0.977 (0.001) | 0.791 (0.005) | 0.942 (0.003) | 0.728 (0.006) | |
| 97 | oscillating | Poisson | sign-chg. | 100 | mid | smooth | 61 | default | E(Y | X) | 0.905 (0.003) | 0.951 (0.002) | 0.944 (0.002) | 0.653 (0.005) | 0.941 (0.002) | 0.670 (0.005) | |
| 98 | oscillating | binary | sign-chg. | 100 | mid | smooth | 61 | default | beta(s,t) | 0.927 (0.005) | 0.977 (0.002) | 0.956 (0.003) | 0.841 (0.007) | 0.959 (0.002) | 0.885 (0.005) | |
| 98 | oscillating | binary | sign-chg. | 100 | mid | smooth | 61 | default | E(Y | X) | 0.882 (0.005) | 0.945 (0.003) | 0.930 (0.003) | 0.724 (0.006) | 0.943 (0.002) | 0.794 (0.005) | |
| 99 | warp_ar1 | Gaussian | misreg. | 100 | high | wiggly | 61 | default | beta(s,t) | 0.863 (0.006) | 0.934 (0.003) | 0.908 (0.004) | 0.628 (0.007) | 0.944 (0.003) | 0.729 (0.006) | 0.860 (0.005) |
| 99 | warp_ar1 | Gaussian | misreg. | 100 | high | wiggly | 61 | default | E(Y | X) | 0.911 (0.003) | 0.938 (0.003) | 0.926 (0.003) | 0.665 (0.005) | 0.937 (0.003) | 0.706 (0.005) | 0.838 (0.004) |
| 100 | heteroskedastic | Gaussian | var(t) | 40 | mid | smooth | 61 | default | beta(s,t) | 0.918 (0.005) | 0.979 (0.003) | 0.973 (0.003) | 0.662 (0.009) | 0.926 (0.004) | 0.619 (0.009) | |
| 100 | heteroskedastic | Gaussian | var(t) | 40 | mid | smooth | 61 | default | E(Y | X) | 0.919 (0.004) | 0.957 (0.003) | 0.951 (0.003) | 0.593 (0.006) | 0.932 (0.003) | 0.614 (0.005) | |
| 101 | heteroskedastic | Gaussian | var(t) | 100 | mid | smooth | 61 | default | beta(s,t) | 0.933 (0.004) | 0.983 (0.002) | 0.977 (0.002) | 0.686 (0.008) | 0.944 (0.003) | 0.662 (0.007) | 0.991 (0.001) |
| 101 | heteroskedastic | Gaussian | var(t) | 100 | mid | smooth | 61 | default | E(Y | X) | 0.928 (0.003) | 0.959 (0.002) | 0.954 (0.002) | 0.614 (0.005) | 0.944 (0.002) | 0.642 (0.005) | 0.972 (0.002) |
| 102 | heteroskedastic | Gaussian | var(z) | 40 | mid | smooth | 61 | default | beta(s,t) | 0.925 (0.004) | 0.985 (0.002) | 0.979 (0.002) | 0.668 (0.009) | 0.939 (0.003) | 0.619 (0.009) | |
| 102 | heteroskedastic | Gaussian | var(z) | 40 | mid | smooth | 61 | default | E(Y | X) | 0.912 (0.004) | 0.955 (0.003) | 0.950 (0.003) | 0.560 (0.007) | 0.934 (0.003) | 0.588 (0.006) | |
| 103 | heteroskedastic | Gaussian | var(z) | 100 | mid | smooth | 61 | default | beta(s,t) | 0.935 (0.004) | 0.984 (0.001) | 0.978 (0.002) | 0.683 (0.009) | 0.949 (0.003) | 0.660 (0.008) | 0.992 (0.001) |
| 103 | heteroskedastic | Gaussian | var(z) | 100 | mid | smooth | 61 | default | E(Y | X) | 0.923 (0.003) | 0.955 (0.002) | 0.950 (0.002) | 0.572 (0.006) | 0.944 (0.002) | 0.606 (0.005) | 0.949 (0.003) |
| 104 | heteroskedastic | Gaussian | var(t,z) | 40 | mid | smooth | 61 | default | beta(s,t) | 0.924 (0.005) | 0.983 (0.002) | 0.977 (0.002) | 0.667 (0.009) | 0.940 (0.003) | 0.629 (0.009) | |
| 104 | heteroskedastic | Gaussian | var(t,z) | 40 | mid | smooth | 61 | default | E(Y | X) | 0.911 (0.004) | 0.955 (0.003) | 0.949 (0.003) | 0.563 (0.007) | 0.936 (0.003) | 0.596 (0.005) | |
| 105 | heteroskedastic | Gaussian | var(t,z) | 100 | mid | smooth | 61 | default | beta(s,t) | 0.934 (0.004) | 0.984 (0.001) | 0.978 (0.002) | 0.688 (0.009) | 0.950 (0.003) | 0.666 (0.008) | 0.990 (0.001) |
| 105 | heteroskedastic | Gaussian | var(t,z) | 100 | mid | smooth | 61 | default | E(Y | X) | 0.922 (0.003) | 0.955 (0.002) | 0.950 (0.002) | 0.580 (0.006) | 0.944 (0.002) | 0.615 (0.005) | 0.951 (0.002) |
| 106 | lowrank_covariate | Gaussian | iid | 100 | mid | smooth | 61 | default | beta(s,t) | 0.986 (0.001) | 0.994 (0.001) | 0.953 (0.003) | 0.986 (0.001) | 0.991 (0.001) | 0.994 (0.001) | |
| 106 | lowrank_covariate | Gaussian | iid | 100 | mid | smooth | 61 | default | E(Y | X) | 0.948 (0.002) | 0.956 (0.002) | 0.933 (0.002) | 0.952 (0.002) | 0.960 (0.002) | 0.966 (0.001) | |
| 107 | lowrank_covariate | Gaussian | smooth | 100 | mid | smooth | 61 | default | beta(s,t) | 0.936 (0.004) | 0.987 (0.001) | 0.982 (0.002) | 0.712 (0.009) | 0.945 (0.003) | 0.666 (0.010) | |
| 107 | lowrank_covariate | Gaussian | smooth | 100 | mid | smooth | 61 | default | E(Y | X) | 0.930 (0.003) | 0.958 (0.002) | 0.954 (0.002) | 0.607 (0.005) | 0.944 (0.002) | 0.633 (0.005) | |
| 108 | lowrank_covariate | Poisson | iid | 100 | mid | smooth | 61 | default | beta(s,t) | 0.987 (0.001) | 0.994 (0.001) | 0.953 (0.003) | 0.987 (0.001) | 0.992 (0.001) | 0.994 (0.001) | |
| 108 | lowrank_covariate | Poisson | iid | 100 | mid | smooth | 61 | default | E(Y | X) | 0.946 (0.002) | 0.954 (0.002) | 0.928 (0.003) | 0.950 (0.002) | 0.959 (0.002) | 0.965 (0.002) | |
| 109 | lowrank_covariate | Poisson | smooth | 100 | mid | smooth | 61 | default | beta(s,t) | 0.939 (0.004) | 0.989 (0.001) | 0.984 (0.002) | 0.735 (0.009) | 0.946 (0.004) | 0.693 (0.009) | |
| 109 | lowrank_covariate | Poisson | smooth | 100 | mid | smooth | 61 | default | E(Y | X) | 0.928 (0.003) | 0.957 (0.002) | 0.952 (0.002) | 0.617 (0.006) | 0.945 (0.002) | 0.652 (0.005) | |
| 110 | lowrank_covariate | binary | iid | 100 | mid | smooth | 61 | default | beta(s,t) | 0.987 (0.002) | 0.992 (0.001) | 0.915 (0.005) | 0.987 (0.002) | 0.993 (0.001) | 0.996 (0.001) | |
| 110 | lowrank_covariate | binary | iid | 100 | mid | smooth | 61 | default | E(Y | X) | 0.937 (0.003) | 0.944 (0.003) | 0.899 (0.004) | 0.941 (0.003) | 0.945 (0.003) | 0.960 (0.002) | |
| 111 | lowrank_covariate | binary | smooth | 100 | mid | smooth | 61 | default | beta(s,t) | 0.904 (0.008) | 0.977 (0.003) | 0.966 (0.003) | 0.734 (0.012) | 0.954 (0.004) | 0.789 (0.010) | |
| 111 | lowrank_covariate | binary | smooth | 100 | mid | smooth | 61 | default | E(Y | X) | 0.890 (0.004) | 0.939 (0.003) | 0.931 (0.003) | 0.609 (0.007) | 0.940 (0.003) | 0.718 (0.005) | |
| 112 | basis_size | Gaussian | iid | 100 | mid | smooth | 61 | large | beta(s,t) | 0.994 (0.001) | 0.997 (0.000) | 0.959 (0.002) | 0.994 (0.001) | 0.997 (0.000) | 0.998 (0.000) | |
| 112 | basis_size | Gaussian | iid | 100 | mid | smooth | 61 | large | E(Y | X) | 0.962 (0.001) | 0.970 (0.001) | 0.934 (0.002) | 0.964 (0.001) | 0.973 (0.001) | 0.978 (0.001) | |
| 113 | basis_size | Gaussian | iid | 100 | mid | smooth | 61 | xlarge | beta(s,t) | 0.999 (0.000) | 0.999 (0.000) | 0.964 (0.002) | 0.999 (0.000) | 1.000 (0.000) | 1.000 (0.000) | |
| 113 | basis_size | Gaussian | iid | 100 | mid | smooth | 61 | xlarge | E(Y | X) | 0.972 (0.001) | 0.979 (0.001) | 0.934 (0.002) | 0.973 (0.001) | 0.982 (0.001) | 0.985 (0.001) | |
| 114 | basis_size | Gaussian | smooth | 100 | mid | smooth | 61 | large | beta(s,t) | 0.952 (0.003) | 0.994 (0.001) | 0.990 (0.001) | 0.789 (0.006) | 0.945 (0.003) | 0.671 (0.007) | |
| 114 | basis_size | Gaussian | smooth | 100 | mid | smooth | 61 | large | E(Y | X) | 0.933 (0.003) | 0.971 (0.001) | 0.965 (0.002) | 0.660 (0.005) | 0.947 (0.002) | 0.673 (0.004) | |
| 115 | basis_size | Gaussian | smooth | 100 | mid | smooth | 61 | xlarge | beta(s,t) | 0.967 (0.002) | 0.998 (0.000) | 0.995 (0.000) | 0.871 (0.005) | 0.954 (0.002) | 0.673 (0.006) | |
| 115 | basis_size | Gaussian | smooth | 100 | mid | smooth | 61 | xlarge | E(Y | X) | 0.937 (0.002) | 0.979 (0.001) | 0.973 (0.001) | 0.707 (0.005) | 0.951 (0.002) | 0.706 (0.003) | |
| 116 | basis_size | Poisson | iid | 100 | mid | smooth | 61 | large | beta(s,t) | 0.994 (0.001) | 0.997 (0.000) | 0.956 (0.002) | 0.995 (0.001) | 0.997 (0.000) | 0.998 (0.000) | |
| 116 | basis_size | Poisson | iid | 100 | mid | smooth | 61 | large | E(Y | X) | 0.961 (0.002) | 0.969 (0.001) | 0.929 (0.002) | 0.963 (0.002) | 0.974 (0.001) | 0.978 (0.001) | |
| 117 | basis_size | Poisson | iid | 100 | mid | smooth | 61 | xlarge | beta(s,t) | 0.999 (0.000) | 0.999 (0.000) | 0.961 (0.002) | 0.999 (0.000) | 1.000 (0.000) | 1.000 (0.000) | |
| 117 | basis_size | Poisson | iid | 100 | mid | smooth | 61 | xlarge | E(Y | X) | 0.970 (0.001) | 0.977 (0.001) | 0.927 (0.002) | 0.971 (0.001) | 0.982 (0.001) | 0.985 (0.001) | |
| 118 | basis_size | Poisson | smooth | 100 | mid | smooth | 61 | large | beta(s,t) | 0.953 (0.003) | 0.995 (0.001) | 0.990 (0.001) | 0.805 (0.006) | 0.948 (0.003) | 0.702 (0.007) | |
| 118 | basis_size | Poisson | smooth | 100 | mid | smooth | 61 | large | E(Y | X) | 0.930 (0.003) | 0.970 (0.001) | 0.963 (0.002) | 0.666 (0.005) | 0.949 (0.002) | 0.697 (0.004) | |
| 119 | basis_size | Poisson | smooth | 100 | mid | smooth | 61 | xlarge | beta(s,t) | 0.970 (0.002) | 0.998 (0.000) | 0.995 (0.000) | 0.886 (0.005) | 0.955 (0.002) | 0.715 (0.006) | |
| 119 | basis_size | Poisson | smooth | 100 | mid | smooth | 61 | xlarge | E(Y | X) | 0.935 (0.003) | 0.977 (0.001) | 0.970 (0.001) | 0.714 (0.005) | 0.952 (0.002) | 0.738 (0.003) | |
| 120 | basis_size | binary | iid | 100 | mid | smooth | 61 | large | beta(s,t) | 0.994 (0.001) | 0.996 (0.001) | 0.909 (0.004) | 0.994 (0.001) | 0.997 (0.000) | 0.998 (0.000) | |
| 120 | basis_size | binary | iid | 100 | mid | smooth | 61 | large | E(Y | X) | 0.954 (0.002) | 0.959 (0.002) | 0.895 (0.004) | 0.956 (0.002) | 0.965 (0.002) | 0.975 (0.001) | |
| 121 | basis_size | binary | iid | 100 | mid | smooth | 61 | xlarge | beta(s,t) | 0.999 (0.000) | 0.999 (0.000) | 0.908 (0.004) | 0.999 (0.000) | 0.999 (0.000) | 1.000 (0.000) | |
| 121 | basis_size | binary | iid | 100 | mid | smooth | 61 | xlarge | E(Y | X) | 0.963 (0.002) | 0.967 (0.002) | 0.891 (0.004) | 0.965 (0.002) | 0.974 (0.001) | 0.982 (0.001) | |
| 122 | basis_size | binary | smooth | 100 | mid | smooth | 61 | large | beta(s,t) | 0.917 (0.007) | 0.990 (0.001) | 0.983 (0.001) | 0.785 (0.011) | 0.958 (0.003) | 0.789 (0.007) | |
| 122 | basis_size | binary | smooth | 100 | mid | smooth | 61 | large | E(Y | X) | 0.892 (0.004) | 0.959 (0.002) | 0.950 (0.002) | 0.641 (0.007) | 0.946 (0.002) | 0.753 (0.004) | |
| 123 | basis_size | binary | smooth | 100 | mid | smooth | 61 | xlarge | beta(s,t) | 0.933 (0.007) | 0.997 (0.000) | 0.993 (0.001) | 0.845 (0.010) | 0.960 (0.002) | 0.795 (0.006) | |
| 123 | basis_size | binary | smooth | 100 | mid | smooth | 61 | xlarge | E(Y | X) | 0.899 (0.004) | 0.973 (0.002) | 0.964 (0.002) | 0.671 (0.007) | 0.950 (0.002) | 0.781 (0.003) |
16.3 S2 Synthetic coverage of α(t), γ(t) and f(x,t)
| cell | block | family | error | G | signal | truth | D | basis | estimand | NCV + CL2 | NCV + CL2, bias-aware | REML + CL2 | REML, model-based |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | core | Gaussian | iid | 40 | low | smooth | 61 | default | alpha(t) | 0.937 (0.005) | 0.941 (0.005) | 0.940 (0.005) | 0.954 (0.005) |
| 1 | core | Gaussian | iid | 40 | low | smooth | 61 | default | gamma(t) | 0.935 (0.007) | 0.938 (0.007) | 0.942 (0.006) | 0.958 (0.005) |
| 2 | core | Gaussian | iid | 100 | low | smooth | 61 | default | alpha(t) | 0.941 (0.005) | 0.944 (0.005) | 0.950 (0.005) | 0.955 (0.005) |
| 2 | core | Gaussian | iid | 100 | low | smooth | 61 | default | gamma(t) | 0.948 (0.005) | 0.952 (0.005) | 0.957 (0.005) | 0.968 (0.004) |
| 3 | core | Gaussian | OU | 40 | low | smooth | 61 | default | alpha(t) | 0.847 (0.014) | 0.912 (0.008) | 0.933 (0.007) | 0.621 (0.013) |
| 3 | core | Gaussian | OU | 40 | low | smooth | 61 | default | gamma(t) | 0.867 (0.011) | 0.915 (0.008) | 0.937 (0.007) | 0.640 (0.012) |
| 4 | core | Gaussian | OU | 100 | low | smooth | 61 | default | alpha(t) | 0.924 (0.007) | 0.939 (0.006) | 0.948 (0.005) | 0.653 (0.011) |
| 4 | core | Gaussian | OU | 100 | low | smooth | 61 | default | gamma(t) | 0.904 (0.008) | 0.923 (0.007) | 0.937 (0.006) | 0.656 (0.012) |
| 5 | core | Gaussian | smooth | 40 | low | smooth | 61 | default | alpha(t) | 0.871 (0.013) | 0.919 (0.009) | 0.930 (0.007) | 0.608 (0.014) |
| 5 | core | Gaussian | smooth | 40 | low | smooth | 61 | default | gamma(t) | 0.889 (0.010) | 0.926 (0.008) | 0.945 (0.006) | 0.620 (0.014) |
| 6 | core | Gaussian | smooth | 100 | low | smooth | 61 | default | alpha(t) | 0.925 (0.007) | 0.941 (0.006) | 0.948 (0.006) | 0.635 (0.012) |
| 6 | core | Gaussian | smooth | 100 | low | smooth | 61 | default | gamma(t) | 0.925 (0.007) | 0.942 (0.006) | 0.948 (0.006) | 0.649 (0.013) |
| 7 | core | Gaussian | iid | 40 | mid | smooth | 61 | default | alpha(t) | 0.940 (0.005) | 0.943 (0.005) | 0.947 (0.004) | 0.956 (0.004) |
| 7 | core | Gaussian | iid | 40 | mid | smooth | 61 | default | gamma(t) | 0.941 (0.005) | 0.944 (0.005) | 0.951 (0.005) | 0.959 (0.004) |
| 8 | core | Gaussian | iid | 100 | mid | smooth | 61 | default | alpha(t) | 0.945 (0.005) | 0.949 (0.005) | 0.956 (0.004) | 0.956 (0.004) |
| 8 | core | Gaussian | iid | 100 | mid | smooth | 61 | default | gamma(t) | 0.946 (0.005) | 0.951 (0.004) | 0.950 (0.005) | 0.960 (0.004) |
| 9 | core | Gaussian | OU | 40 | mid | smooth | 61 | default | alpha(t) | 0.924 (0.006) | 0.936 (0.006) | 0.940 (0.006) | 0.632 (0.012) |
| 9 | core | Gaussian | OU | 40 | mid | smooth | 61 | default | gamma(t) | 0.920 (0.007) | 0.940 (0.006) | 0.938 (0.006) | 0.653 (0.011) |
| 10 | core | Gaussian | OU | 100 | mid | smooth | 61 | default | alpha(t) | 0.938 (0.006) | 0.945 (0.005) | 0.951 (0.005) | 0.659 (0.010) |
| 10 | core | Gaussian | OU | 100 | mid | smooth | 61 | default | gamma(t) | 0.924 (0.006) | 0.936 (0.006) | 0.938 (0.006) | 0.663 (0.012) |
| 11 | core | Gaussian | smooth | 40 | mid | smooth | 61 | default | alpha(t) | 0.919 (0.008) | 0.936 (0.007) | 0.937 (0.007) | 0.614 (0.013) |
| 11 | core | Gaussian | smooth | 40 | mid | smooth | 61 | default | gamma(t) | 0.927 (0.007) | 0.946 (0.006) | 0.946 (0.006) | 0.621 (0.012) |
| 12 | core | Gaussian | smooth | 100 | mid | smooth | 61 | default | alpha(t) | 0.941 (0.006) | 0.949 (0.005) | 0.950 (0.005) | 0.641 (0.011) |
| 12 | core | Gaussian | smooth | 100 | mid | smooth | 61 | default | gamma(t) | 0.935 (0.006) | 0.946 (0.006) | 0.948 (0.005) | 0.655 (0.012) |
| 13 | core | Gaussian | iid | 40 | high | smooth | 61 | default | alpha(t) | 0.939 (0.004) | 0.943 (0.004) | 0.947 (0.004) | 0.954 (0.004) |
| 13 | core | Gaussian | iid | 40 | high | smooth | 61 | default | gamma(t) | 0.935 (0.005) | 0.943 (0.005) | 0.945 (0.005) | 0.959 (0.004) |
| 14 | core | Gaussian | iid | 100 | high | smooth | 61 | default | alpha(t) | 0.950 (0.004) | 0.953 (0.004) | 0.954 (0.004) | 0.956 (0.004) |
| 14 | core | Gaussian | iid | 100 | high | smooth | 61 | default | gamma(t) | 0.938 (0.005) | 0.947 (0.005) | 0.944 (0.005) | 0.955 (0.004) |
| 15 | core | Gaussian | OU | 40 | high | smooth | 61 | default | alpha(t) | 0.933 (0.006) | 0.941 (0.006) | 0.941 (0.005) | 0.638 (0.011) |
| 15 | core | Gaussian | OU | 40 | high | smooth | 61 | default | gamma(t) | 0.931 (0.006) | 0.943 (0.006) | 0.940 (0.006) | 0.657 (0.010) |
| 16 | core | Gaussian | OU | 100 | high | smooth | 61 | default | alpha(t) | 0.946 (0.005) | 0.949 (0.005) | 0.953 (0.005) | 0.665 (0.010) |
| 16 | core | Gaussian | OU | 100 | high | smooth | 61 | default | gamma(t) | 0.928 (0.006) | 0.936 (0.005) | 0.939 (0.005) | 0.666 (0.011) |
| 17 | core | Gaussian | smooth | 40 | high | smooth | 61 | default | alpha(t) | 0.930 (0.007) | 0.943 (0.007) | 0.940 (0.007) | 0.622 (0.012) |
| 17 | core | Gaussian | smooth | 40 | high | smooth | 61 | default | gamma(t) | 0.935 (0.006) | 0.949 (0.006) | 0.945 (0.006) | 0.629 (0.012) |
| 18 | core | Gaussian | smooth | 100 | high | smooth | 61 | default | alpha(t) | 0.946 (0.005) | 0.954 (0.005) | 0.949 (0.005) | 0.646 (0.011) |
| 18 | core | Gaussian | smooth | 100 | high | smooth | 61 | default | gamma(t) | 0.936 (0.006) | 0.946 (0.005) | 0.947 (0.005) | 0.655 (0.011) |
| 19 | core | Poisson | iid | 40 | mid | smooth | 61 | default | alpha(t) | 0.940 (0.005) | 0.946 (0.005) | 0.953 (0.004) | 0.960 (0.004) |
| 19 | core | Poisson | iid | 40 | mid | smooth | 61 | default | gamma(t) | 0.939 (0.005) | 0.943 (0.005) | 0.948 (0.005) | 0.955 (0.005) |
| 20 | core | Poisson | iid | 100 | mid | smooth | 61 | default | alpha(t) | 0.936 (0.005) | 0.943 (0.005) | 0.951 (0.005) | 0.954 (0.005) |
| 20 | core | Poisson | iid | 100 | mid | smooth | 61 | default | gamma(t) | 0.947 (0.005) | 0.951 (0.005) | 0.955 (0.005) | 0.963 (0.004) |
| 21 | core | Poisson | OU | 40 | mid | smooth | 61 | default | alpha(t) | 0.909 (0.007) | 0.937 (0.006) | 0.935 (0.006) | 0.655 (0.011) |
| 21 | core | Poisson | OU | 40 | mid | smooth | 61 | default | gamma(t) | 0.905 (0.009) | 0.932 (0.007) | 0.937 (0.006) | 0.667 (0.012) |
| 22 | core | Poisson | OU | 100 | mid | smooth | 61 | default | alpha(t) | 0.934 (0.006) | 0.948 (0.006) | 0.950 (0.005) | 0.668 (0.011) |
| 22 | core | Poisson | OU | 100 | mid | smooth | 61 | default | gamma(t) | 0.923 (0.007) | 0.933 (0.006) | 0.937 (0.006) | 0.677 (0.011) |
| 23 | core | Poisson | smooth | 40 | mid | smooth | 61 | default | alpha(t) | 0.913 (0.009) | 0.941 (0.007) | 0.933 (0.007) | 0.641 (0.013) |
| 23 | core | Poisson | smooth | 40 | mid | smooth | 61 | default | gamma(t) | 0.910 (0.008) | 0.937 (0.007) | 0.943 (0.006) | 0.647 (0.012) |
| 24 | core | Poisson | smooth | 100 | mid | smooth | 61 | default | alpha(t) | 0.927 (0.007) | 0.948 (0.006) | 0.948 (0.005) | 0.665 (0.012) |
| 24 | core | Poisson | smooth | 100 | mid | smooth | 61 | default | gamma(t) | 0.931 (0.006) | 0.944 (0.006) | 0.945 (0.005) | 0.662 (0.012) |
| 25 | core | Poisson | iid | 40 | high | smooth | 61 | default | alpha(t) | 0.927 (0.006) | 0.946 (0.005) | 0.950 (0.004) | 0.961 (0.004) |
| 25 | core | Poisson | iid | 40 | high | smooth | 61 | default | gamma(t) | 0.927 (0.006) | 0.936 (0.005) | 0.941 (0.005) | 0.956 (0.005) |
| 26 | core | Poisson | iid | 100 | high | smooth | 61 | default | alpha(t) | 0.926 (0.006) | 0.941 (0.005) | 0.948 (0.005) | 0.950 (0.005) |
| 26 | core | Poisson | iid | 100 | high | smooth | 61 | default | gamma(t) | 0.937 (0.005) | 0.943 (0.005) | 0.943 (0.005) | 0.954 (0.004) |
| 27 | core | Poisson | OU | 40 | high | smooth | 61 | default | alpha(t) | 0.898 (0.009) | 0.946 (0.006) | 0.944 (0.006) | 0.677 (0.011) |
| 27 | core | Poisson | OU | 40 | high | smooth | 61 | default | gamma(t) | 0.899 (0.008) | 0.928 (0.006) | 0.930 (0.006) | 0.669 (0.011) |
| 28 | core | Poisson | OU | 100 | high | smooth | 61 | default | alpha(t) | 0.918 (0.007) | 0.947 (0.005) | 0.947 (0.005) | 0.671 (0.011) |
| 28 | core | Poisson | OU | 100 | high | smooth | 61 | default | gamma(t) | 0.921 (0.007) | 0.932 (0.006) | 0.938 (0.006) | 0.688 (0.010) |
| 29 | core | Poisson | smooth | 40 | high | smooth | 61 | default | alpha(t) | 0.891 (0.009) | 0.946 (0.006) | 0.940 (0.006) | 0.653 (0.012) |
| 29 | core | Poisson | smooth | 40 | high | smooth | 61 | default | gamma(t) | 0.902 (0.007) | 0.934 (0.006) | 0.938 (0.006) | 0.658 (0.011) |
| 30 | core | Poisson | smooth | 100 | high | smooth | 61 | default | alpha(t) | 0.918 (0.007) | 0.949 (0.005) | 0.949 (0.005) | 0.659 (0.011) |
| 30 | core | Poisson | smooth | 100 | high | smooth | 61 | default | gamma(t) | 0.921 (0.006) | 0.937 (0.006) | 0.943 (0.005) | 0.669 (0.011) |
| 31 | core | binary | iid | 40 | mid | smooth | 61 | default | alpha(t) | 0.873 (0.012) | 0.889 (0.012) | 0.859 (0.014) | 0.903 (0.013) |
| 31 | core | binary | iid | 40 | mid | smooth | 61 | default | gamma(t) | 0.876 (0.012) | 0.896 (0.012) | 0.890 (0.011) | 0.931 (0.009) |
| 32 | core | binary | iid | 100 | mid | smooth | 61 | default | alpha(t) | 0.912 (0.008) | 0.920 (0.007) | 0.924 (0.007) | 0.943 (0.006) |
| 32 | core | binary | iid | 100 | mid | smooth | 61 | default | gamma(t) | 0.912 (0.009) | 0.917 (0.009) | 0.923 (0.008) | 0.945 (0.007) |
| 33 | core | binary | OU | 40 | mid | smooth | 61 | default | alpha(t) | 0.844 (0.012) | 0.908 (0.009) | 0.904 (0.008) | 0.712 (0.014) |
| 33 | core | binary | OU | 40 | mid | smooth | 61 | default | gamma(t) | 0.799 (0.014) | 0.885 (0.010) | 0.920 (0.009) | 0.724 (0.013) |
| 34 | core | binary | OU | 100 | mid | smooth | 61 | default | alpha(t) | 0.836 (0.013) | 0.902 (0.008) | 0.926 (0.007) | 0.749 (0.011) |
| 34 | core | binary | OU | 100 | mid | smooth | 61 | default | gamma(t) | 0.826 (0.014) | 0.888 (0.011) | 0.924 (0.008) | 0.740 (0.012) |
| 35 | core | binary | smooth | 40 | mid | smooth | 61 | default | alpha(t) | 0.874 (0.011) | 0.931 (0.008) | 0.885 (0.010) | 0.668 (0.016) |
| 35 | core | binary | smooth | 40 | mid | smooth | 61 | default | gamma(t) | 0.838 (0.012) | 0.915 (0.009) | 0.922 (0.008) | 0.691 (0.015) |
| 36 | core | binary | smooth | 100 | mid | smooth | 61 | default | alpha(t) | 0.856 (0.012) | 0.908 (0.008) | 0.920 (0.008) | 0.702 (0.013) |
| 36 | core | binary | smooth | 100 | mid | smooth | 61 | default | gamma(t) | 0.854 (0.012) | 0.914 (0.009) | 0.939 (0.007) | 0.730 (0.013) |
| 37 | core | binary | iid | 40 | high | smooth | 61 | default | alpha(t) | 0.920 (0.007) | 0.929 (0.007) | 0.938 (0.006) | 0.956 (0.005) |
| 37 | core | binary | iid | 40 | high | smooth | 61 | default | gamma(t) | 0.910 (0.009) | 0.918 (0.008) | 0.921 (0.008) | 0.942 (0.007) |
| 38 | core | binary | iid | 100 | high | smooth | 61 | default | alpha(t) | 0.933 (0.006) | 0.938 (0.005) | 0.949 (0.005) | 0.958 (0.004) |
| 38 | core | binary | iid | 100 | high | smooth | 61 | default | gamma(t) | 0.922 (0.007) | 0.926 (0.007) | 0.935 (0.006) | 0.948 (0.005) |
| 39 | core | binary | OU | 40 | high | smooth | 61 | default | alpha(t) | 0.835 (0.015) | 0.910 (0.009) | 0.907 (0.008) | 0.747 (0.011) |
| 39 | core | binary | OU | 40 | high | smooth | 61 | default | gamma(t) | 0.817 (0.014) | 0.892 (0.009) | 0.929 (0.008) | 0.746 (0.012) |
| 40 | core | binary | OU | 100 | high | smooth | 61 | default | alpha(t) | 0.909 (0.007) | 0.934 (0.006) | 0.937 (0.006) | 0.759 (0.011) |
| 40 | core | binary | OU | 100 | high | smooth | 61 | default | gamma(t) | 0.893 (0.009) | 0.917 (0.007) | 0.941 (0.005) | 0.758 (0.010) |
| 41 | core | binary | smooth | 40 | high | smooth | 61 | default | alpha(t) | 0.859 (0.011) | 0.929 (0.007) | 0.903 (0.008) | 0.686 (0.013) |
| 41 | core | binary | smooth | 40 | high | smooth | 61 | default | gamma(t) | 0.833 (0.012) | 0.902 (0.009) | 0.919 (0.008) | 0.716 (0.013) |
| 42 | core | binary | smooth | 100 | high | smooth | 61 | default | alpha(t) | 0.891 (0.008) | 0.932 (0.006) | 0.929 (0.007) | 0.730 (0.011) |
| 42 | core | binary | smooth | 100 | high | smooth | 61 | default | gamma(t) | 0.895 (0.009) | 0.926 (0.007) | 0.940 (0.006) | 0.737 (0.011) |
| 43 | warp | Gaussian | misreg. | 40 | high | smooth | 61 | default | alpha(t) | 0.932 (0.008) | 0.940 (0.008) | 0.943 (0.007) | 0.766 (0.013) |
| 43 | warp | Gaussian | misreg. | 40 | high | smooth | 61 | default | gamma(t) | 0.903 (0.009) | 0.927 (0.008) | 0.929 (0.007) | 0.718 (0.013) |
| 44 | warp | Gaussian | misreg. | 40 | high | wiggly | 61 | default | alpha(t) | 0.924 (0.008) | 0.941 (0.007) | 0.949 (0.006) | 0.754 (0.012) |
| 44 | warp | Gaussian | misreg. | 40 | high | wiggly | 61 | default | gamma(t) | 0.872 (0.009) | 0.912 (0.007) | 0.915 (0.007) | 0.678 (0.011) |
| 45 | warp | Gaussian | misreg. | 100 | high | smooth | 61 | default | alpha(t) | 0.948 (0.007) | 0.952 (0.007) | 0.953 (0.007) | 0.766 (0.012) |
| 45 | warp | Gaussian | misreg. | 100 | high | smooth | 61 | default | gamma(t) | 0.920 (0.007) | 0.930 (0.007) | 0.933 (0.007) | 0.719 (0.013) |
| 46 | warp | Gaussian | misreg. | 100 | high | wiggly | 61 | default | alpha(t) | 0.943 (0.007) | 0.950 (0.006) | 0.949 (0.006) | 0.732 (0.011) |
| 46 | warp | Gaussian | misreg. | 100 | high | wiggly | 61 | default | gamma(t) | 0.923 (0.006) | 0.930 (0.006) | 0.928 (0.006) | 0.692 (0.013) |
| 55 | dense_grid | Gaussian | iid | 100 | mid | smooth | 241 | default | alpha(t) | 0.943 (0.004) | 0.949 (0.004) | 0.951 (0.004) | 0.952 (0.004) |
| 55 | dense_grid | Gaussian | iid | 100 | mid | smooth | 241 | default | gamma(t) | 0.933 (0.004) | 0.939 (0.004) | 0.943 (0.004) | 0.951 (0.004) |
| 56 | dense_grid | Gaussian | OU | 100 | mid | smooth | 241 | default | alpha(t) | 0.941 (0.005) | 0.953 (0.004) | 0.953 (0.005) | 0.382 (0.009) |
| 56 | dense_grid | Gaussian | OU | 100 | mid | smooth | 241 | default | gamma(t) | 0.919 (0.006) | 0.936 (0.006) | 0.936 (0.006) | 0.397 (0.011) |
| 57 | dense_grid | Gaussian | smooth | 100 | mid | smooth | 241 | default | alpha(t) | 0.938 (0.006) | 0.954 (0.005) | 0.947 (0.005) | 0.373 (0.010) |
| 57 | dense_grid | Gaussian | smooth | 100 | mid | smooth | 241 | default | gamma(t) | 0.930 (0.006) | 0.950 (0.006) | 0.947 (0.005) | 0.369 (0.010) |
| 58 | dense_grid | Poisson | iid | 100 | mid | smooth | 241 | default | alpha(t) | 0.943 (0.005) | 0.951 (0.004) | 0.951 (0.004) | 0.954 (0.004) |
| 58 | dense_grid | Poisson | iid | 100 | mid | smooth | 241 | default | gamma(t) | 0.935 (0.005) | 0.942 (0.004) | 0.948 (0.004) | 0.957 (0.004) |
| 59 | dense_grid | Poisson | OU | 100 | mid | smooth | 241 | default | alpha(t) | 0.932 (0.006) | 0.956 (0.005) | 0.952 (0.005) | 0.397 (0.010) |
| 59 | dense_grid | Poisson | OU | 100 | mid | smooth | 241 | default | gamma(t) | 0.915 (0.007) | 0.934 (0.006) | 0.936 (0.006) | 0.411 (0.010) |
| 60 | dense_grid | Poisson | smooth | 100 | mid | smooth | 241 | default | alpha(t) | 0.925 (0.007) | 0.953 (0.005) | 0.947 (0.005) | 0.380 (0.010) |
| 60 | dense_grid | Poisson | smooth | 100 | mid | smooth | 241 | default | gamma(t) | 0.923 (0.006) | 0.948 (0.005) | 0.947 (0.005) | 0.387 (0.010) |
| 61 | dense_grid | binary | iid | 100 | mid | smooth | 241 | default | alpha(t) | 0.940 (0.005) | 0.945 (0.005) | 0.951 (0.005) | 0.957 (0.004) |
| 61 | dense_grid | binary | iid | 100 | mid | smooth | 241 | default | gamma(t) | 0.942 (0.005) | 0.944 (0.005) | 0.951 (0.005) | 0.960 (0.004) |
| 62 | dense_grid | binary | OU | 100 | mid | smooth | 241 | default | alpha(t) | 0.831 (0.013) | 0.928 (0.007) | 0.919 (0.007) | 0.447 (0.012) |
| 62 | dense_grid | binary | OU | 100 | mid | smooth | 241 | default | gamma(t) | 0.813 (0.014) | 0.913 (0.008) | 0.941 (0.006) | 0.474 (0.011) |
| 63 | dense_grid | binary | smooth | 100 | mid | smooth | 241 | default | alpha(t) | 0.843 (0.012) | 0.938 (0.005) | 0.920 (0.007) | 0.410 (0.011) |
| 63 | dense_grid | binary | smooth | 100 | mid | smooth | 241 | default | gamma(t) | 0.843 (0.012) | 0.932 (0.007) | 0.943 (0.005) | 0.445 (0.011) |
| 64 | rough_truth | Gaussian | iid | 100 | low | wiggly | 61 | default | alpha(t) | 0.941 (0.005) | 0.944 (0.005) | 0.952 (0.005) | 0.955 (0.004) |
| 64 | rough_truth | Gaussian | iid | 100 | low | wiggly | 61 | default | gamma(t) | 0.939 (0.005) | 0.943 (0.004) | 0.942 (0.005) | 0.952 (0.004) |
| 65 | rough_truth | Gaussian | smooth | 100 | low | wiggly | 61 | default | alpha(t) | 0.925 (0.007) | 0.940 (0.006) | 0.947 (0.006) | 0.634 (0.012) |
| 65 | rough_truth | Gaussian | smooth | 100 | low | wiggly | 61 | default | gamma(t) | 0.928 (0.007) | 0.938 (0.006) | 0.942 (0.006) | 0.642 (0.012) |
| 66 | rough_truth | Gaussian | iid | 100 | mid | wiggly | 61 | default | alpha(t) | 0.945 (0.005) | 0.950 (0.004) | 0.956 (0.004) | 0.956 (0.004) |
| 66 | rough_truth | Gaussian | iid | 100 | mid | wiggly | 61 | default | gamma(t) | 0.940 (0.005) | 0.944 (0.004) | 0.945 (0.004) | 0.954 (0.004) |
| 67 | rough_truth | Gaussian | smooth | 100 | mid | wiggly | 61 | default | alpha(t) | 0.937 (0.006) | 0.947 (0.005) | 0.950 (0.005) | 0.642 (0.012) |
| 67 | rough_truth | Gaussian | smooth | 100 | mid | wiggly | 61 | default | gamma(t) | 0.937 (0.006) | 0.943 (0.006) | 0.947 (0.006) | 0.652 (0.012) |
| 68 | rough_truth | Gaussian | iid | 100 | high | wiggly | 61 | default | alpha(t) | 0.948 (0.004) | 0.952 (0.004) | 0.955 (0.004) | 0.957 (0.004) |
| 68 | rough_truth | Gaussian | iid | 100 | high | wiggly | 61 | default | gamma(t) | 0.938 (0.005) | 0.941 (0.005) | 0.945 (0.004) | 0.954 (0.004) |
| 69 | rough_truth | Gaussian | smooth | 100 | high | wiggly | 61 | default | alpha(t) | 0.945 (0.005) | 0.950 (0.005) | 0.950 (0.005) | 0.648 (0.011) |
| 69 | rough_truth | Gaussian | smooth | 100 | high | wiggly | 61 | default | gamma(t) | 0.944 (0.005) | 0.947 (0.005) | 0.947 (0.005) | 0.657 (0.011) |
| 70 | rough_truth | Poisson | iid | 100 | mid | wiggly | 61 | default | alpha(t) | 0.941 (0.005) | 0.946 (0.005) | 0.952 (0.005) | 0.952 (0.004) |
| 70 | rough_truth | Poisson | iid | 100 | mid | wiggly | 61 | default | gamma(t) | 0.936 (0.005) | 0.941 (0.005) | 0.942 (0.005) | 0.952 (0.005) |
| 71 | rough_truth | Poisson | smooth | 100 | mid | wiggly | 61 | default | alpha(t) | 0.936 (0.006) | 0.944 (0.006) | 0.950 (0.005) | 0.666 (0.012) |
| 71 | rough_truth | Poisson | smooth | 100 | mid | wiggly | 61 | default | gamma(t) | 0.930 (0.006) | 0.938 (0.006) | 0.945 (0.005) | 0.670 (0.011) |
| 72 | rough_truth | Poisson | iid | 100 | high | wiggly | 61 | default | alpha(t) | 0.936 (0.006) | 0.950 (0.005) | 0.950 (0.005) | 0.952 (0.005) |
| 72 | rough_truth | Poisson | iid | 100 | high | wiggly | 61 | default | gamma(t) | 0.937 (0.005) | 0.945 (0.005) | 0.945 (0.004) | 0.955 (0.004) |
| 73 | rough_truth | Poisson | smooth | 100 | high | wiggly | 61 | default | alpha(t) | 0.939 (0.006) | 0.952 (0.005) | 0.948 (0.005) | 0.668 (0.012) |
| 73 | rough_truth | Poisson | smooth | 100 | high | wiggly | 61 | default | gamma(t) | 0.924 (0.006) | 0.937 (0.005) | 0.943 (0.005) | 0.678 (0.011) |
| 74 | rough_truth | binary | iid | 100 | mid | wiggly | 61 | default | alpha(t) | 0.920 (0.008) | 0.926 (0.008) | 0.921 (0.007) | 0.942 (0.006) |
| 74 | rough_truth | binary | iid | 100 | mid | wiggly | 61 | default | gamma(t) | 0.897 (0.008) | 0.906 (0.008) | 0.877 (0.008) | 0.907 (0.007) |
| 75 | rough_truth | binary | smooth | 100 | mid | wiggly | 61 | default | alpha(t) | 0.862 (0.012) | 0.914 (0.008) | 0.920 (0.007) | 0.708 (0.013) |
| 75 | rough_truth | binary | smooth | 100 | mid | wiggly | 61 | default | gamma(t) | 0.839 (0.011) | 0.904 (0.007) | 0.931 (0.006) | 0.720 (0.012) |
| 76 | rough_truth | binary | iid | 100 | high | wiggly | 61 | default | alpha(t) | 0.939 (0.005) | 0.942 (0.005) | 0.947 (0.005) | 0.957 (0.004) |
| 76 | rough_truth | binary | iid | 100 | high | wiggly | 61 | default | gamma(t) | 0.918 (0.006) | 0.923 (0.006) | 0.916 (0.006) | 0.928 (0.006) |
| 77 | rough_truth | binary | smooth | 100 | high | wiggly | 61 | default | alpha(t) | 0.892 (0.008) | 0.935 (0.006) | 0.932 (0.006) | 0.729 (0.011) |
| 77 | rough_truth | binary | smooth | 100 | high | wiggly | 61 | default | gamma(t) | 0.902 (0.008) | 0.927 (0.006) | 0.938 (0.006) | 0.745 (0.011) |
| 78 | term_type | Gaussian | iid | 100 | mid | smooth | 61 | default | f(x,t) | 0.975 (0.001) | 0.983 (0.001) | 0.971 (0.002) | 0.976 (0.001) |
| 78 | term_type | Gaussian | iid | 100 | mid | smooth | 61 | default | gamma(t) | 0.946 (0.005) | 0.951 (0.005) | 0.952 (0.004) | 0.962 (0.004) |
| 79 | term_type | Gaussian | OU | 100 | mid | smooth | 61 | default | f(x,t) | 0.940 (0.003) | 0.975 (0.002) | 0.945 (0.002) | 0.677 (0.005) |
| 79 | term_type | Gaussian | OU | 100 | mid | smooth | 61 | default | gamma(t) | 0.925 (0.007) | 0.935 (0.006) | 0.938 (0.006) | 0.661 (0.012) |
| 80 | term_type | Gaussian | smooth | 100 | mid | smooth | 61 | default | f(x,t) | 0.939 (0.003) | 0.973 (0.002) | 0.942 (0.003) | 0.650 (0.005) |
| 80 | term_type | Gaussian | smooth | 100 | mid | smooth | 61 | default | gamma(t) | 0.935 (0.007) | 0.947 (0.006) | 0.950 (0.005) | 0.648 (0.012) |
| 81 | term_type | Poisson | iid | 100 | mid | smooth | 61 | default | f(x,t) | 0.967 (0.002) | 0.976 (0.001) | 0.973 (0.001) | 0.978 (0.001) |
| 81 | term_type | Poisson | iid | 100 | mid | smooth | 61 | default | gamma(t) | 0.942 (0.005) | 0.946 (0.005) | 0.949 (0.005) | 0.959 (0.004) |
| 82 | term_type | Poisson | OU | 100 | mid | smooth | 61 | default | f(x,t) | 0.920 (0.004) | 0.964 (0.002) | 0.945 (0.002) | 0.697 (0.005) |
| 82 | term_type | Poisson | OU | 100 | mid | smooth | 61 | default | gamma(t) | 0.920 (0.007) | 0.932 (0.006) | 0.938 (0.006) | 0.673 (0.011) |
| 83 | term_type | Poisson | smooth | 100 | mid | smooth | 61 | default | f(x,t) | 0.922 (0.004) | 0.965 (0.002) | 0.943 (0.003) | 0.671 (0.005) |
| 83 | term_type | Poisson | smooth | 100 | mid | smooth | 61 | default | gamma(t) | 0.931 (0.006) | 0.942 (0.006) | 0.946 (0.005) | 0.664 (0.012) |
| 84 | term_type | binary | iid | 100 | mid | smooth | 61 | default | f(x,t) | 0.969 (0.002) | 0.976 (0.002) | 0.982 (0.001) | 0.988 (0.001) |
| 84 | term_type | binary | iid | 100 | mid | smooth | 61 | default | gamma(t) | 0.922 (0.008) | 0.929 (0.008) | 0.930 (0.007) | 0.952 (0.006) |
| 85 | term_type | binary | OU | 100 | mid | smooth | 61 | default | f(x,t) | 0.885 (0.008) | 0.958 (0.004) | 0.949 (0.003) | 0.791 (0.005) |
| 85 | term_type | binary | OU | 100 | mid | smooth | 61 | default | gamma(t) | 0.834 (0.014) | 0.889 (0.011) | 0.925 (0.008) | 0.729 (0.013) |
| 86 | term_type | binary | smooth | 100 | mid | smooth | 61 | default | f(x,t) | 0.895 (0.008) | 0.966 (0.003) | 0.945 (0.003) | 0.754 (0.006) |
| 86 | term_type | binary | smooth | 100 | mid | smooth | 61 | default | gamma(t) | 0.850 (0.012) | 0.910 (0.009) | 0.943 (0.006) | 0.723 (0.013) |
| 87 | families | scaled t | iid | 100 | mid | smooth | 61 | default | alpha(t) | 0.947 (0.004) | 0.950 (0.004) | 0.954 (0.004) | 0.955 (0.004) |
| 87 | families | scaled t | iid | 100 | mid | smooth | 61 | default | gamma(t) | 0.946 (0.005) | 0.950 (0.004) | 0.950 (0.004) | 0.956 (0.004) |
| 88 | families | scaled t | smooth | 100 | mid | smooth | 61 | default | alpha(t) | 0.947 (0.005) | 0.955 (0.005) | 0.951 (0.005) | 0.651 (0.012) |
| 88 | families | scaled t | smooth | 100 | mid | smooth | 61 | default | gamma(t) | 0.944 (0.006) | 0.952 (0.005) | 0.950 (0.005) | 0.668 (0.012) |
| 89 | families | beta | iid | 100 | mid | smooth | 61 | default | alpha(t) | 0.940 (0.005) | 0.946 (0.005) | 0.950 (0.004) | 0.958 (0.004) |
| 89 | families | beta | iid | 100 | mid | smooth | 61 | default | gamma(t) | 0.939 (0.005) | 0.946 (0.004) | 0.950 (0.004) | 0.959 (0.004) |
| 90 | families | beta | smooth | 100 | mid | smooth | 61 | default | alpha(t) | 0.935 (0.006) | 0.948 (0.005) | 0.945 (0.005) | 0.642 (0.012) |
| 90 | families | beta | smooth | 100 | mid | smooth | 61 | default | gamma(t) | 0.931 (0.006) | 0.943 (0.006) | 0.946 (0.005) | 0.656 (0.012) |
| 91 | families | negative binomial | iid | 100 | mid | smooth | 61 | default | alpha(t) | 0.935 (0.005) | 0.943 (0.005) | 0.950 (0.005) | 0.954 (0.004) |
| 91 | families | negative binomial | iid | 100 | mid | smooth | 61 | default | gamma(t) | 0.948 (0.005) | 0.953 (0.005) | 0.957 (0.005) | 0.965 (0.004) |
| 92 | families | negative binomial | smooth | 100 | mid | smooth | 61 | default | alpha(t) | 0.924 (0.007) | 0.943 (0.006) | 0.942 (0.006) | 0.647 (0.012) |
| 92 | families | negative binomial | smooth | 100 | mid | smooth | 61 | default | gamma(t) | 0.926 (0.007) | 0.940 (0.006) | 0.943 (0.005) | 0.658 (0.012) |
| 93 | families | negative binomial | iid | 100 | mid | smooth | 61 | default | alpha(t) | 0.923 (0.006) | 0.941 (0.005) | 0.950 (0.005) | 0.896 (0.007) |
| 93 | families | negative binomial | iid | 100 | mid | smooth | 61 | default | gamma(t) | 0.927 (0.006) | 0.939 (0.005) | 0.948 (0.005) | 0.881 (0.007) |
| 94 | families | negative binomial | smooth | 100 | mid | smooth | 61 | default | alpha(t) | 0.915 (0.007) | 0.948 (0.006) | 0.942 (0.006) | 0.554 (0.011) |
| 94 | families | negative binomial | smooth | 100 | mid | smooth | 61 | default | gamma(t) | 0.909 (0.008) | 0.935 (0.006) | 0.942 (0.005) | 0.541 (0.012) |
| 95 | ar1_home | Gaussian | AR(1) | 100 | mid | smooth | 61 | default | alpha(t) | 0.939 (0.005) | 0.949 (0.005) | 0.955 (0.005) | 0.636 (0.011) |
| 95 | ar1_home | Gaussian | AR(1) | 100 | mid | smooth | 61 | default | gamma(t) | 0.921 (0.006) | 0.933 (0.006) | 0.937 (0.006) | 0.645 (0.012) |
| 96 | oscillating | Gaussian | sign-chg. | 100 | mid | smooth | 61 | default | alpha(t) | 0.905 (0.007) | 0.941 (0.006) | 0.951 (0.005) | 0.633 (0.011) |
| 96 | oscillating | Gaussian | sign-chg. | 100 | mid | smooth | 61 | default | gamma(t) | 0.902 (0.007) | 0.935 (0.006) | 0.944 (0.005) | 0.645 (0.012) |
| 97 | oscillating | Poisson | sign-chg. | 100 | mid | smooth | 61 | default | alpha(t) | 0.885 (0.009) | 0.937 (0.006) | 0.947 (0.005) | 0.656 (0.011) |
| 97 | oscillating | Poisson | sign-chg. | 100 | mid | smooth | 61 | default | gamma(t) | 0.882 (0.008) | 0.918 (0.007) | 0.939 (0.006) | 0.673 (0.011) |
| 98 | oscillating | binary | sign-chg. | 100 | mid | smooth | 61 | default | alpha(t) | 0.844 (0.012) | 0.918 (0.007) | 0.934 (0.006) | 0.768 (0.010) |
| 98 | oscillating | binary | sign-chg. | 100 | mid | smooth | 61 | default | gamma(t) | 0.810 (0.012) | 0.904 (0.008) | 0.937 (0.006) | 0.790 (0.010) |
| 99 | warp_ar1 | Gaussian | misreg. | 100 | high | wiggly | 61 | default | alpha(t) | 0.943 (0.007) | 0.950 (0.006) | 0.949 (0.006) | 0.732 (0.011) |
| 99 | warp_ar1 | Gaussian | misreg. | 100 | high | wiggly | 61 | default | gamma(t) | 0.923 (0.006) | 0.930 (0.006) | 0.928 (0.006) | 0.692 (0.013) |
| 100 | heteroskedastic | Gaussian | var(t) | 40 | mid | smooth | 61 | default | alpha(t) | 0.922 (0.008) | 0.937 (0.007) | 0.939 (0.006) | 0.630 (0.013) |
| 100 | heteroskedastic | Gaussian | var(t) | 40 | mid | smooth | 61 | default | gamma(t) | 0.925 (0.007) | 0.945 (0.006) | 0.944 (0.006) | 0.632 (0.012) |
| 101 | heteroskedastic | Gaussian | var(t) | 100 | mid | smooth | 61 | default | alpha(t) | 0.943 (0.005) | 0.952 (0.005) | 0.951 (0.005) | 0.650 (0.011) |
| 101 | heteroskedastic | Gaussian | var(t) | 100 | mid | smooth | 61 | default | gamma(t) | 0.932 (0.006) | 0.943 (0.006) | 0.947 (0.005) | 0.661 (0.012) |
| 102 | heteroskedastic | Gaussian | var(z) | 40 | mid | smooth | 61 | default | alpha(t) | 0.913 (0.008) | 0.935 (0.007) | 0.930 (0.007) | 0.618 (0.014) |
| 102 | heteroskedastic | Gaussian | var(z) | 40 | mid | smooth | 61 | default | gamma(t) | 0.898 (0.009) | 0.931 (0.007) | 0.926 (0.007) | 0.490 (0.014) |
| 103 | heteroskedastic | Gaussian | var(z) | 100 | mid | smooth | 61 | default | alpha(t) | 0.936 (0.006) | 0.946 (0.005) | 0.945 (0.005) | 0.647 (0.012) |
| 103 | heteroskedastic | Gaussian | var(z) | 100 | mid | smooth | 61 | default | gamma(t) | 0.920 (0.007) | 0.945 (0.005) | 0.938 (0.006) | 0.494 (0.013) |
| 104 | heteroskedastic | Gaussian | var(t,z) | 40 | mid | smooth | 61 | default | alpha(t) | 0.917 (0.007) | 0.936 (0.007) | 0.935 (0.006) | 0.629 (0.014) |
| 104 | heteroskedastic | Gaussian | var(t,z) | 40 | mid | smooth | 61 | default | gamma(t) | 0.893 (0.009) | 0.930 (0.007) | 0.925 (0.007) | 0.502 (0.014) |
| 105 | heteroskedastic | Gaussian | var(t,z) | 100 | mid | smooth | 61 | default | alpha(t) | 0.939 (0.006) | 0.949 (0.005) | 0.947 (0.005) | 0.655 (0.012) |
| 105 | heteroskedastic | Gaussian | var(t,z) | 100 | mid | smooth | 61 | default | gamma(t) | 0.920 (0.006) | 0.944 (0.006) | 0.940 (0.005) | 0.510 (0.013) |
| 106 | lowrank_covariate | Gaussian | iid | 100 | mid | smooth | 61 | default | alpha(t) | 0.945 (0.005) | 0.950 (0.005) | 0.956 (0.004) | 0.956 (0.004) |
| 106 | lowrank_covariate | Gaussian | iid | 100 | mid | smooth | 61 | default | gamma(t) | 0.946 (0.005) | 0.951 (0.004) | 0.950 (0.005) | 0.960 (0.004) |
| 107 | lowrank_covariate | Gaussian | smooth | 100 | mid | smooth | 61 | default | alpha(t) | 0.939 (0.006) | 0.950 (0.005) | 0.949 (0.005) | 0.642 (0.011) |
| 107 | lowrank_covariate | Gaussian | smooth | 100 | mid | smooth | 61 | default | gamma(t) | 0.934 (0.006) | 0.946 (0.006) | 0.949 (0.005) | 0.655 (0.012) |
| 108 | lowrank_covariate | Poisson | iid | 100 | mid | smooth | 61 | default | alpha(t) | 0.937 (0.005) | 0.942 (0.005) | 0.952 (0.005) | 0.952 (0.005) |
| 108 | lowrank_covariate | Poisson | iid | 100 | mid | smooth | 61 | default | gamma(t) | 0.948 (0.005) | 0.950 (0.005) | 0.955 (0.005) | 0.963 (0.004) |
| 109 | lowrank_covariate | Poisson | smooth | 100 | mid | smooth | 61 | default | alpha(t) | 0.928 (0.007) | 0.948 (0.006) | 0.947 (0.006) | 0.667 (0.012) |
| 109 | lowrank_covariate | Poisson | smooth | 100 | mid | smooth | 61 | default | gamma(t) | 0.932 (0.006) | 0.944 (0.006) | 0.945 (0.005) | 0.663 (0.012) |
| 110 | lowrank_covariate | binary | iid | 100 | mid | smooth | 61 | default | alpha(t) | 0.917 (0.007) | 0.921 (0.007) | 0.923 (0.007) | 0.942 (0.006) |
| 110 | lowrank_covariate | binary | iid | 100 | mid | smooth | 61 | default | gamma(t) | 0.910 (0.009) | 0.917 (0.009) | 0.922 (0.008) | 0.944 (0.007) |
| 111 | lowrank_covariate | binary | smooth | 100 | mid | smooth | 61 | default | alpha(t) | 0.868 (0.011) | 0.913 (0.008) | 0.922 (0.007) | 0.702 (0.013) |
| 111 | lowrank_covariate | binary | smooth | 100 | mid | smooth | 61 | default | gamma(t) | 0.867 (0.011) | 0.915 (0.008) | 0.940 (0.007) | 0.731 (0.013) |
| 112 | basis_size | Gaussian | iid | 100 | mid | smooth | 61 | large | alpha(t) | 0.955 (0.004) | 0.961 (0.004) | 0.965 (0.003) | 0.968 (0.003) |
| 112 | basis_size | Gaussian | iid | 100 | mid | smooth | 61 | large | gamma(t) | 0.952 (0.004) | 0.960 (0.004) | 0.963 (0.003) | 0.971 (0.003) |
| 113 | basis_size | Gaussian | iid | 100 | mid | smooth | 61 | xlarge | alpha(t) | 0.966 (0.003) | 0.971 (0.003) | 0.975 (0.002) | 0.978 (0.002) |
| 113 | basis_size | Gaussian | iid | 100 | mid | smooth | 61 | xlarge | gamma(t) | 0.960 (0.003) | 0.968 (0.003) | 0.973 (0.003) | 0.976 (0.002) |
| 114 | basis_size | Gaussian | smooth | 100 | mid | smooth | 61 | large | alpha(t) | 0.941 (0.005) | 0.956 (0.005) | 0.950 (0.005) | 0.697 (0.010) |
| 114 | basis_size | Gaussian | smooth | 100 | mid | smooth | 61 | large | gamma(t) | 0.932 (0.006) | 0.949 (0.005) | 0.950 (0.005) | 0.703 (0.011) |
| 115 | basis_size | Gaussian | smooth | 100 | mid | smooth | 61 | xlarge | alpha(t) | 0.944 (0.005) | 0.960 (0.004) | 0.954 (0.004) | 0.739 (0.010) |
| 115 | basis_size | Gaussian | smooth | 100 | mid | smooth | 61 | xlarge | gamma(t) | 0.934 (0.006) | 0.955 (0.005) | 0.950 (0.005) | 0.734 (0.011) |
| 116 | basis_size | Poisson | iid | 100 | mid | smooth | 61 | large | alpha(t) | 0.946 (0.005) | 0.955 (0.004) | 0.964 (0.004) | 0.968 (0.003) |
| 116 | basis_size | Poisson | iid | 100 | mid | smooth | 61 | large | gamma(t) | 0.951 (0.004) | 0.955 (0.004) | 0.964 (0.003) | 0.971 (0.003) |
| 117 | basis_size | Poisson | iid | 100 | mid | smooth | 61 | xlarge | alpha(t) | 0.958 (0.004) | 0.968 (0.003) | 0.975 (0.003) | 0.978 (0.002) |
| 117 | basis_size | Poisson | iid | 100 | mid | smooth | 61 | xlarge | gamma(t) | 0.959 (0.004) | 0.964 (0.003) | 0.973 (0.003) | 0.978 (0.002) |
| 118 | basis_size | Poisson | smooth | 100 | mid | smooth | 61 | large | alpha(t) | 0.927 (0.007) | 0.956 (0.005) | 0.949 (0.005) | 0.716 (0.010) |
| 118 | basis_size | Poisson | smooth | 100 | mid | smooth | 61 | large | gamma(t) | 0.928 (0.006) | 0.943 (0.005) | 0.950 (0.005) | 0.723 (0.010) |
| 119 | basis_size | Poisson | smooth | 100 | mid | smooth | 61 | xlarge | alpha(t) | 0.931 (0.007) | 0.963 (0.005) | 0.950 (0.005) | 0.752 (0.010) |
| 119 | basis_size | Poisson | smooth | 100 | mid | smooth | 61 | xlarge | gamma(t) | 0.929 (0.006) | 0.947 (0.005) | 0.953 (0.005) | 0.769 (0.010) |
| 120 | basis_size | binary | iid | 100 | mid | smooth | 61 | large | alpha(t) | 0.933 (0.006) | 0.938 (0.006) | 0.944 (0.006) | 0.961 (0.005) |
| 120 | basis_size | binary | iid | 100 | mid | smooth | 61 | large | gamma(t) | 0.921 (0.008) | 0.926 (0.008) | 0.936 (0.007) | 0.958 (0.006) |
| 121 | basis_size | binary | iid | 100 | mid | smooth | 61 | xlarge | alpha(t) | 0.937 (0.007) | 0.945 (0.006) | 0.956 (0.005) | 0.969 (0.004) |
| 121 | basis_size | binary | iid | 100 | mid | smooth | 61 | xlarge | gamma(t) | 0.928 (0.008) | 0.934 (0.007) | 0.949 (0.006) | 0.966 (0.005) |
| 122 | basis_size | binary | smooth | 100 | mid | smooth | 61 | large | alpha(t) | 0.866 (0.011) | 0.921 (0.008) | 0.915 (0.007) | 0.728 (0.013) |
| 122 | basis_size | binary | smooth | 100 | mid | smooth | 61 | large | gamma(t) | 0.865 (0.011) | 0.923 (0.009) | 0.942 (0.006) | 0.762 (0.012) |
| 123 | basis_size | binary | smooth | 100 | mid | smooth | 61 | xlarge | alpha(t) | 0.877 (0.011) | 0.936 (0.006) | 0.899 (0.008) | 0.713 (0.014) |
| 123 | basis_size | binary | smooth | 100 | mid | smooth | 61 | xlarge | gamma(t) | 0.872 (0.011) | 0.929 (0.008) | 0.946 (0.006) | 0.779 (0.013) |
16.4 S3 Synthetic MSE per cell
| cell | block | family | error | G | signal | truth | basis | estimand | MSE NCV | MSE REML | ratio NCV/REML |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | core | Gaussian | iid | 40 | low | smooth | default | alpha(t) | 0.014 | 0.013 | 1.06 [1.03, 1.09] |
| 1 | core | Gaussian | iid | 40 | low | smooth | default | beta(s,t) | 0.158 | 0.168 | 0.94 [0.86, 1.04] |
| 1 | core | Gaussian | iid | 40 | low | smooth | default | gamma(t) | 0.013 | 0.013 | 1.06 [1.04, 1.09] |
| 1 | core | Gaussian | iid | 40 | low | smooth | default | E(Y | X) | 0.064 | 0.062 | 1.03 [1.01, 1.04] |
| 2 | core | Gaussian | iid | 100 | low | smooth | default | alpha(t) | 0.006 | 0.006 | 1.05 [1.02, 1.08] |
| 2 | core | Gaussian | iid | 100 | low | smooth | default | beta(s,t) | 0.078 | 0.093 | 0.84 [0.77, 0.91] |
| 2 | core | Gaussian | iid | 100 | low | smooth | default | gamma(t) | 0.006 | 0.005 | 1.06 [1.03, 1.09] |
| 2 | core | Gaussian | iid | 100 | low | smooth | default | E(Y | X) | 0.027 | 0.027 | 1.00 [0.98, 1.01] |
| 3 | core | Gaussian | OU | 40 | low | smooth | default | alpha(t) | 0.067 | 0.066 | 1.02 [0.95, 1.08] |
| 3 | core | Gaussian | OU | 40 | low | smooth | default | beta(s,t) | 0.961 | 8.149 | 0.12 [0.09, 0.15] |
| 3 | core | Gaussian | OU | 40 | low | smooth | default | gamma(t) | 0.061 | 0.062 | 0.98 [0.92, 1.04] |
| 3 | core | Gaussian | OU | 40 | low | smooth | default | E(Y | X) | 0.296 | 0.480 | 0.62 [0.59, 0.64] |
| 4 | core | Gaussian | OU | 100 | low | smooth | default | alpha(t) | 0.024 | 0.024 | 0.97 [0.94, 1.00] |
| 4 | core | Gaussian | OU | 100 | low | smooth | default | beta(s,t) | 0.359 | 2.396 | 0.15 [0.13, 0.17] |
| 4 | core | Gaussian | OU | 100 | low | smooth | default | gamma(t) | 0.024 | 0.025 | 0.97 [0.94, 1.00] |
| 4 | core | Gaussian | OU | 100 | low | smooth | default | E(Y | X) | 0.116 | 0.172 | 0.68 [0.66, 0.70] |
| 5 | core | Gaussian | smooth | 40 | low | smooth | default | alpha(t) | 0.069 | 0.073 | 0.95 [0.90, 1.01] |
| 5 | core | Gaussian | smooth | 40 | low | smooth | default | beta(s,t) | 0.942 | 10.481 | 0.09 [0.07, 0.11] |
| 5 | core | Gaussian | smooth | 40 | low | smooth | default | gamma(t) | 0.062 | 0.067 | 0.92 [0.87, 0.98] |
| 5 | core | Gaussian | smooth | 40 | low | smooth | default | E(Y | X) | 0.306 | 0.527 | 0.58 [0.56, 0.61] |
| 6 | core | Gaussian | smooth | 100 | low | smooth | default | alpha(t) | 0.025 | 0.026 | 0.98 [0.94, 1.03] |
| 6 | core | Gaussian | smooth | 100 | low | smooth | default | beta(s,t) | 0.350 | 2.814 | 0.12 [0.10, 0.15] |
| 6 | core | Gaussian | smooth | 100 | low | smooth | default | gamma(t) | 0.024 | 0.025 | 0.93 [0.91, 0.96] |
| 6 | core | Gaussian | smooth | 100 | low | smooth | default | E(Y | X) | 0.121 | 0.183 | 0.66 [0.64, 0.68] |
| 7 | core | Gaussian | iid | 40 | mid | smooth | default | alpha(t) | 0.004 | 0.004 | 1.04 [1.01, 1.06] |
| 7 | core | Gaussian | iid | 40 | mid | smooth | default | beta(s,t) | 0.063 | 0.074 | 0.85 [0.70, 1.13] |
| 7 | core | Gaussian | iid | 40 | mid | smooth | default | gamma(t) | 0.004 | 0.004 | 1.04 [1.02, 1.06] |
| 7 | core | Gaussian | iid | 40 | mid | smooth | default | E(Y | X) | 0.019 | 0.020 | 0.98 [0.96, 1.00] |
| 8 | core | Gaussian | iid | 100 | mid | smooth | default | alpha(t) | 0.002 | 0.002 | 1.04 [1.02, 1.07] |
| 8 | core | Gaussian | iid | 100 | mid | smooth | default | beta(s,t) | 0.027 | 0.041 | 0.65 [0.62, 0.68] |
| 8 | core | Gaussian | iid | 100 | mid | smooth | default | gamma(t) | 0.002 | 0.002 | 1.04 [1.02, 1.07] |
| 8 | core | Gaussian | iid | 100 | mid | smooth | default | E(Y | X) | 0.008 | 0.008 | 0.96 [0.95, 0.97] |
| 9 | core | Gaussian | OU | 40 | mid | smooth | default | alpha(t) | 0.017 | 0.018 | 0.93 [0.90, 0.96] |
| 9 | core | Gaussian | OU | 40 | mid | smooth | default | beta(s,t) | 0.279 | 2.186 | 0.13 [0.10, 0.16] |
| 9 | core | Gaussian | OU | 40 | mid | smooth | default | gamma(t) | 0.016 | 0.017 | 0.93 [0.90, 0.97] |
| 9 | core | Gaussian | OU | 40 | mid | smooth | default | E(Y | X) | 0.080 | 0.130 | 0.62 [0.59, 0.64] |
| 10 | core | Gaussian | OU | 100 | mid | smooth | default | alpha(t) | 0.006 | 0.007 | 0.96 [0.95, 0.98] |
| 10 | core | Gaussian | OU | 100 | mid | smooth | default | beta(s,t) | 0.112 | 0.647 | 0.17 [0.15, 0.20] |
| 10 | core | Gaussian | OU | 100 | mid | smooth | default | gamma(t) | 0.006 | 0.007 | 0.97 [0.94, 0.99] |
| 10 | core | Gaussian | OU | 100 | mid | smooth | default | E(Y | X) | 0.032 | 0.046 | 0.70 [0.68, 0.72] |
| 11 | core | Gaussian | smooth | 40 | mid | smooth | default | alpha(t) | 0.018 | 0.020 | 0.91 [0.87, 0.95] |
| 11 | core | Gaussian | smooth | 40 | mid | smooth | default | beta(s,t) | 0.263 | 2.750 | 0.10 [0.08, 0.12] |
| 11 | core | Gaussian | smooth | 40 | mid | smooth | default | gamma(t) | 0.016 | 0.018 | 0.89 [0.85, 0.93] |
| 11 | core | Gaussian | smooth | 40 | mid | smooth | default | E(Y | X) | 0.083 | 0.142 | 0.59 [0.57, 0.61] |
| 12 | core | Gaussian | smooth | 100 | mid | smooth | default | alpha(t) | 0.007 | 0.007 | 0.97 [0.94, 0.99] |
| 12 | core | Gaussian | smooth | 100 | mid | smooth | default | beta(s,t) | 0.118 | 0.752 | 0.16 [0.13, 0.19] |
| 12 | core | Gaussian | smooth | 100 | mid | smooth | default | gamma(t) | 0.007 | 0.007 | 0.95 [0.92, 0.97] |
| 12 | core | Gaussian | smooth | 100 | mid | smooth | default | E(Y | X) | 0.034 | 0.049 | 0.69 [0.67, 0.71] |
| 13 | core | Gaussian | iid | 40 | high | smooth | default | alpha(t) | 0.001 | 0.001 | 1.03 [1.01, 1.05] |
| 13 | core | Gaussian | iid | 40 | high | smooth | default | beta(s,t) | 0.020 | 0.033 | 0.62 [0.58, 0.67] |
| 13 | core | Gaussian | iid | 40 | high | smooth | default | gamma(t) | 0.001 | 0.001 | 1.04 [1.01, 1.06] |
| 13 | core | Gaussian | iid | 40 | high | smooth | default | E(Y | X) | 0.006 | 0.006 | 0.94 [0.92, 0.95] |
| 14 | core | Gaussian | iid | 100 | high | smooth | default | alpha(t) | 0.000 | 0.000 | 1.03 [1.01, 1.05] |
| 14 | core | Gaussian | iid | 100 | high | smooth | default | beta(s,t) | 0.010 | 0.018 | 0.55 [0.53, 0.57] |
| 14 | core | Gaussian | iid | 100 | high | smooth | default | gamma(t) | 0.000 | 0.000 | 1.07 [1.04, 1.10] |
| 14 | core | Gaussian | iid | 100 | high | smooth | default | E(Y | X) | 0.002 | 0.003 | 0.93 [0.92, 0.94] |
| 15 | core | Gaussian | OU | 40 | high | smooth | default | alpha(t) | 0.004 | 0.005 | 0.93 [0.90, 0.96] |
| 15 | core | Gaussian | OU | 40 | high | smooth | default | beta(s,t) | 0.074 | 0.618 | 0.12 [0.10, 0.14] |
| 15 | core | Gaussian | OU | 40 | high | smooth | default | gamma(t) | 0.004 | 0.005 | 0.93 [0.90, 0.96] |
| 15 | core | Gaussian | OU | 40 | high | smooth | default | E(Y | X) | 0.022 | 0.035 | 0.63 [0.61, 0.65] |
| 16 | core | Gaussian | OU | 100 | high | smooth | default | alpha(t) | 0.002 | 0.002 | 0.98 [0.96, 1.00] |
| 16 | core | Gaussian | OU | 100 | high | smooth | default | beta(s,t) | 0.038 | 0.190 | 0.20 [0.17, 0.23] |
| 16 | core | Gaussian | OU | 100 | high | smooth | default | gamma(t) | 0.002 | 0.002 | 0.99 [0.97, 1.01] |
| 16 | core | Gaussian | OU | 100 | high | smooth | default | E(Y | X) | 0.009 | 0.013 | 0.72 [0.71, 0.74] |
| 17 | core | Gaussian | smooth | 40 | high | smooth | default | alpha(t) | 0.005 | 0.005 | 0.91 [0.88, 0.95] |
| 17 | core | Gaussian | smooth | 40 | high | smooth | default | beta(s,t) | 0.080 | 0.753 | 0.11 [0.09, 0.13] |
| 17 | core | Gaussian | smooth | 40 | high | smooth | default | gamma(t) | 0.004 | 0.005 | 0.91 [0.87, 0.94] |
| 17 | core | Gaussian | smooth | 40 | high | smooth | default | E(Y | X) | 0.023 | 0.038 | 0.61 [0.59, 0.63] |
| 18 | core | Gaussian | smooth | 100 | high | smooth | default | alpha(t) | 0.002 | 0.002 | 0.97 [0.95, 1.00] |
| 18 | core | Gaussian | smooth | 100 | high | smooth | default | beta(s,t) | 0.035 | 0.214 | 0.17 [0.14, 0.19] |
| 18 | core | Gaussian | smooth | 100 | high | smooth | default | gamma(t) | 0.002 | 0.002 | 0.98 [0.96, 1.00] |
| 18 | core | Gaussian | smooth | 100 | high | smooth | default | E(Y | X) | 0.010 | 0.013 | 0.71 [0.69, 0.73] |
| 19 | core | Poisson | iid | 40 | mid | smooth | default | alpha(t) | 0.002 | 0.001 | 1.04 [1.02, 1.06] |
| 19 | core | Poisson | iid | 40 | mid | smooth | default | beta(s,t) | 0.017 | 0.024 | 0.72 [0.68, 0.76] |
| 19 | core | Poisson | iid | 40 | mid | smooth | default | gamma(t) | 0.002 | 0.001 | 1.04 [1.02, 1.06] |
| 19 | core | Poisson | iid | 40 | mid | smooth | default | E(Y | X) | 0.082 | 0.082 | 1.00 [0.98, 1.03] |
| 20 | core | Poisson | iid | 100 | mid | smooth | default | alpha(t) | 0.001 | 0.001 | 1.04 [1.01, 1.07] |
| 20 | core | Poisson | iid | 100 | mid | smooth | default | beta(s,t) | 0.009 | 0.013 | 0.71 [0.65, 0.78] |
| 20 | core | Poisson | iid | 100 | mid | smooth | default | gamma(t) | 0.001 | 0.001 | 1.07 [1.04, 1.10] |
| 20 | core | Poisson | iid | 100 | mid | smooth | default | E(Y | X) | 0.033 | 0.034 | 0.98 [0.96, 1.00] |
| 21 | core | Poisson | OU | 40 | mid | smooth | default | alpha(t) | 0.006 | 0.007 | 0.86 [0.81, 0.90] |
| 21 | core | Poisson | OU | 40 | mid | smooth | default | beta(s,t) | 0.085 | 0.618 | 0.14 [0.11, 0.17] |
| 21 | core | Poisson | OU | 40 | mid | smooth | default | gamma(t) | 0.006 | 0.006 | 0.97 [0.92, 1.03] |
| 21 | core | Poisson | OU | 40 | mid | smooth | default | E(Y | X) | 0.362 | 0.599 | 0.60 [0.55, 0.65] |
| 22 | core | Poisson | OU | 100 | mid | smooth | default | alpha(t) | 0.002 | 0.003 | 0.92 [0.88, 0.95] |
| 22 | core | Poisson | OU | 100 | mid | smooth | default | beta(s,t) | 0.038 | 0.199 | 0.19 [0.15, 0.23] |
| 22 | core | Poisson | OU | 100 | mid | smooth | default | gamma(t) | 0.002 | 0.002 | 1.00 [0.97, 1.02] |
| 22 | core | Poisson | OU | 100 | mid | smooth | default | E(Y | X) | 0.138 | 0.195 | 0.71 [0.67, 0.74] |
| 23 | core | Poisson | smooth | 40 | mid | smooth | default | alpha(t) | 0.006 | 0.008 | 0.83 [0.78, 0.89] |
| 23 | core | Poisson | smooth | 40 | mid | smooth | default | beta(s,t) | 0.079 | 0.814 | 0.10 [0.08, 0.12] |
| 23 | core | Poisson | smooth | 40 | mid | smooth | default | gamma(t) | 0.006 | 0.006 | 0.96 [0.92, 1.01] |
| 23 | core | Poisson | smooth | 40 | mid | smooth | default | E(Y | X) | 0.382 | 0.668 | 0.57 [0.53, 0.61] |
| 24 | core | Poisson | smooth | 100 | mid | smooth | default | alpha(t) | 0.003 | 0.003 | 0.94 [0.90, 0.97] |
| 24 | core | Poisson | smooth | 100 | mid | smooth | default | beta(s,t) | 0.036 | 0.228 | 0.16 [0.13, 0.19] |
| 24 | core | Poisson | smooth | 100 | mid | smooth | default | gamma(t) | 0.002 | 0.002 | 0.98 [0.95, 1.01] |
| 24 | core | Poisson | smooth | 100 | mid | smooth | default | E(Y | X) | 0.148 | 0.219 | 0.68 [0.64, 0.71] |
| 25 | core | Poisson | iid | 40 | high | smooth | default | alpha(t) | 0.003 | 0.003 | 1.08 [1.05, 1.12] |
| 25 | core | Poisson | iid | 40 | high | smooth | default | beta(s,t) | 0.038 | 0.054 | 0.71 [0.66, 0.79] |
| 25 | core | Poisson | iid | 40 | high | smooth | default | gamma(t) | 0.003 | 0.003 | 1.09 [1.06, 1.14] |
| 25 | core | Poisson | iid | 40 | high | smooth | default | E(Y | X) | 0.512 | 0.513 | 1.00 [0.92, 1.08] |
| 26 | core | Poisson | iid | 100 | high | smooth | default | alpha(t) | 0.001 | 0.001 | 1.06 [1.03, 1.10] |
| 26 | core | Poisson | iid | 100 | high | smooth | default | beta(s,t) | 0.018 | 0.029 | 0.62 [0.59, 0.65] |
| 26 | core | Poisson | iid | 100 | high | smooth | default | gamma(t) | 0.001 | 0.001 | 1.11 [1.07, 1.15] |
| 26 | core | Poisson | iid | 100 | high | smooth | default | E(Y | X) | 0.159 | 0.177 | 0.90 [0.85, 0.95] |
| 27 | core | Poisson | OU | 40 | high | smooth | default | alpha(t) | 0.013 | 0.014 | 0.93 [0.87, 1.01] |
| 27 | core | Poisson | OU | 40 | high | smooth | default | beta(s,t) | 0.139 | 0.908 | 0.15 [0.13, 0.18] |
| 27 | core | Poisson | OU | 40 | high | smooth | default | gamma(t) | 0.010 | 0.011 | 0.97 [0.91, 1.03] |
| 27 | core | Poisson | OU | 40 | high | smooth | default | E(Y | X) | 2.173 | 4.381 | 0.50 [0.37, 0.64] |
| 28 | core | Poisson | OU | 100 | high | smooth | default | alpha(t) | 0.005 | 0.005 | 0.96 [0.92, 1.00] |
| 28 | core | Poisson | OU | 100 | high | smooth | default | beta(s,t) | 0.059 | 0.303 | 0.19 [0.16, 0.24] |
| 28 | core | Poisson | OU | 100 | high | smooth | default | gamma(t) | 0.004 | 0.004 | 1.04 [1.01, 1.07] |
| 28 | core | Poisson | OU | 100 | high | smooth | default | E(Y | X) | 0.641 | 1.083 | 0.59 [0.52, 0.67] |
| 29 | core | Poisson | smooth | 40 | high | smooth | default | alpha(t) | 0.013 | 0.015 | 0.88 [0.81, 0.96] |
| 29 | core | Poisson | smooth | 40 | high | smooth | default | beta(s,t) | 0.155 | 1.136 | 0.14 [0.11, 0.17] |
| 29 | core | Poisson | smooth | 40 | high | smooth | default | gamma(t) | 0.011 | 0.011 | 0.99 [0.93, 1.04] |
| 29 | core | Poisson | smooth | 40 | high | smooth | default | E(Y | X) | 2.477 | 5.048 | 0.49 [0.36, 0.63] |
| 30 | core | Poisson | smooth | 100 | high | smooth | default | alpha(t) | 0.005 | 0.005 | 0.96 [0.92, 1.01] |
| 30 | core | Poisson | smooth | 100 | high | smooth | default | beta(s,t) | 0.072 | 0.324 | 0.22 [0.18, 0.28] |
| 30 | core | Poisson | smooth | 100 | high | smooth | default | gamma(t) | 0.004 | 0.004 | 1.03 [0.99, 1.06] |
| 30 | core | Poisson | smooth | 100 | high | smooth | default | E(Y | X) | 0.732 | 1.244 | 0.59 [0.51, 0.68] |
| 31 | core | binary | iid | 40 | mid | smooth | default | alpha(t) | 0.019 | 0.017 | 1.07 [1.01, 1.13] |
| 31 | core | binary | iid | 40 | mid | smooth | default | beta(s,t) | 0.173 | 0.151 | 1.15 [0.99, 1.36] |
| 31 | core | binary | iid | 40 | mid | smooth | default | gamma(t) | 0.020 | 0.017 | 1.17 [1.10, 1.25] |
| 31 | core | binary | iid | 40 | mid | smooth | default | E(Y | X) | 0.002 | 0.002 | 1.08 [1.05, 1.12] |
| 32 | core | binary | iid | 100 | mid | smooth | default | alpha(t) | 0.008 | 0.007 | 1.10 [1.05, 1.17] |
| 32 | core | binary | iid | 100 | mid | smooth | default | beta(s,t) | 0.080 | 0.075 | 1.06 [0.95, 1.19] |
| 32 | core | binary | iid | 100 | mid | smooth | default | gamma(t) | 0.008 | 0.007 | 1.05 [1.03, 1.09] |
| 32 | core | binary | iid | 100 | mid | smooth | default | E(Y | X) | 0.001 | 0.001 | 1.06 [1.04, 1.08] |
| 33 | core | binary | OU | 40 | mid | smooth | default | alpha(t) | 0.048 | 0.071 | 0.68 [0.63, 0.73] |
| 33 | core | binary | OU | 40 | mid | smooth | default | beta(s,t) | 0.750 | 3.762 | 0.20 [0.15, 0.25] |
| 33 | core | binary | OU | 40 | mid | smooth | default | gamma(t) | 0.052 | 0.065 | 0.81 [0.75, 0.87] |
| 33 | core | binary | OU | 40 | mid | smooth | default | E(Y | X) | 0.006 | 0.009 | 0.66 [0.62, 0.69] |
| 34 | core | binary | OU | 100 | mid | smooth | default | alpha(t) | 0.024 | 0.026 | 0.91 [0.87, 0.97] |
| 34 | core | binary | OU | 100 | mid | smooth | default | beta(s,t) | 0.249 | 1.135 | 0.22 [0.19, 0.26] |
| 34 | core | binary | OU | 100 | mid | smooth | default | gamma(t) | 0.024 | 0.024 | 1.00 [0.95, 1.06] |
| 34 | core | binary | OU | 100 | mid | smooth | default | E(Y | X) | 0.003 | 0.004 | 0.80 [0.77, 0.83] |
| 35 | core | binary | smooth | 40 | mid | smooth | default | alpha(t) | 0.054 | 0.096 | 0.56 [0.52, 0.61] |
| 35 | core | binary | smooth | 40 | mid | smooth | default | beta(s,t) | 1.013 | 6.579 | 0.15 [0.11, 0.20] |
| 35 | core | binary | smooth | 40 | mid | smooth | default | gamma(t) | 0.054 | 0.079 | 0.69 [0.64, 0.74] |
| 35 | core | binary | smooth | 40 | mid | smooth | default | E(Y | X) | 0.007 | 0.012 | 0.59 [0.56, 0.62] |
| 36 | core | binary | smooth | 100 | mid | smooth | default | alpha(t) | 0.027 | 0.031 | 0.88 [0.84, 0.93] |
| 36 | core | binary | smooth | 100 | mid | smooth | default | beta(s,t) | 0.277 | 1.661 | 0.17 [0.14, 0.20] |
| 36 | core | binary | smooth | 100 | mid | smooth | default | gamma(t) | 0.026 | 0.027 | 0.96 [0.91, 1.03] |
| 36 | core | binary | smooth | 100 | mid | smooth | default | E(Y | X) | 0.003 | 0.004 | 0.74 [0.71, 0.77] |
| 37 | core | binary | iid | 40 | high | smooth | default | alpha(t) | 0.026 | 0.024 | 1.09 [1.05, 1.13] |
| 37 | core | binary | iid | 40 | high | smooth | default | beta(s,t) | 0.280 | 0.272 | 1.03 [0.90, 1.21] |
| 37 | core | binary | iid | 40 | high | smooth | default | gamma(t) | 0.027 | 0.025 | 1.06 [1.03, 1.11] |
| 37 | core | binary | iid | 40 | high | smooth | default | E(Y | X) | 0.002 | 0.002 | 1.06 [1.03, 1.09] |
| 38 | core | binary | iid | 100 | high | smooth | default | alpha(t) | 0.011 | 0.011 | 1.08 [1.05, 1.12] |
| 38 | core | binary | iid | 100 | high | smooth | default | beta(s,t) | 0.145 | 0.148 | 0.98 [0.87, 1.12] |
| 38 | core | binary | iid | 100 | high | smooth | default | gamma(t) | 0.012 | 0.011 | 1.06 [1.04, 1.09] |
| 38 | core | binary | iid | 100 | high | smooth | default | E(Y | X) | 0.001 | 0.001 | 1.03 [1.01, 1.05] |
| 39 | core | binary | OU | 40 | high | smooth | default | alpha(t) | 0.076 | 0.104 | 0.73 [0.68, 0.79] |
| 39 | core | binary | OU | 40 | high | smooth | default | beta(s,t) | 0.870 | 4.415 | 0.20 [0.16, 0.24] |
| 39 | core | binary | OU | 40 | high | smooth | default | gamma(t) | 0.078 | 0.086 | 0.91 [0.84, 0.99] |
| 39 | core | binary | OU | 40 | high | smooth | default | E(Y | X) | 0.007 | 0.010 | 0.74 [0.71, 0.77] |
| 40 | core | binary | OU | 100 | high | smooth | default | alpha(t) | 0.031 | 0.036 | 0.86 [0.82, 0.90] |
| 40 | core | binary | OU | 100 | high | smooth | default | beta(s,t) | 0.324 | 1.419 | 0.23 [0.20, 0.26] |
| 40 | core | binary | OU | 100 | high | smooth | default | gamma(t) | 0.031 | 0.031 | 0.99 [0.95, 1.04] |
| 40 | core | binary | OU | 100 | high | smooth | default | E(Y | X) | 0.003 | 0.004 | 0.77 [0.75, 0.80] |
| 41 | core | binary | smooth | 40 | high | smooth | default | alpha(t) | 0.084 | 0.143 | 0.59 [0.54, 0.64] |
| 41 | core | binary | smooth | 40 | high | smooth | default | beta(s,t) | 1.316 | 7.727 | 0.17 [0.12, 0.23] |
| 41 | core | binary | smooth | 40 | high | smooth | default | gamma(t) | 0.089 | 0.110 | 0.81 [0.75, 0.88] |
| 41 | core | binary | smooth | 40 | high | smooth | default | E(Y | X) | 0.008 | 0.012 | 0.66 [0.63, 0.70] |
| 42 | core | binary | smooth | 100 | high | smooth | default | alpha(t) | 0.036 | 0.044 | 0.83 [0.79, 0.88] |
| 42 | core | binary | smooth | 100 | high | smooth | default | beta(s,t) | 0.360 | 2.115 | 0.17 [0.14, 0.20] |
| 42 | core | binary | smooth | 100 | high | smooth | default | gamma(t) | 0.034 | 0.036 | 0.94 [0.89, 0.99] |
| 42 | core | binary | smooth | 100 | high | smooth | default | E(Y | X) | 0.003 | 0.004 | 0.73 [0.70, 0.75] |
| 43 | warp | Gaussian | misreg. | 40 | high | smooth | default | alpha(t) | 0.004 | 0.004 | 1.02 [0.97, 1.06] |
| 43 | warp | Gaussian | misreg. | 40 | high | smooth | default | beta(s,t) | 0.171 | 0.389 | 0.44 [0.33, 0.56] |
| 43 | warp | Gaussian | misreg. | 40 | high | smooth | default | gamma(t) | 0.007 | 0.006 | 1.06 [1.00, 1.13] |
| 43 | warp | Gaussian | misreg. | 40 | high | smooth | default | E(Y | X) | 0.029 | 0.035 | 0.82 [0.79, 0.85] |
| 44 | warp | Gaussian | misreg. | 40 | high | wiggly | default | alpha(t) | 0.005 | 0.005 | 1.01 [0.96, 1.06] |
| 44 | warp | Gaussian | misreg. | 40 | high | wiggly | default | beta(s,t) | 0.333 | 0.516 | 0.65 [0.54, 0.78] |
| 44 | warp | Gaussian | misreg. | 40 | high | wiggly | default | gamma(t) | 0.010 | 0.009 | 1.12 [1.06, 1.18] |
| 44 | warp | Gaussian | misreg. | 40 | high | wiggly | default | E(Y | X) | 0.043 | 0.048 | 0.91 [0.88, 0.93] |
| 45 | warp | Gaussian | misreg. | 100 | high | smooth | default | alpha(t) | 0.002 | 0.002 | 0.98 [0.96, 1.01] |
| 45 | warp | Gaussian | misreg. | 100 | high | smooth | default | beta(s,t) | 0.071 | 0.152 | 0.47 [0.39, 0.55] |
| 45 | warp | Gaussian | misreg. | 100 | high | smooth | default | gamma(t) | 0.002 | 0.002 | 1.02 [1.00, 1.04] |
| 45 | warp | Gaussian | misreg. | 100 | high | smooth | default | E(Y | X) | 0.012 | 0.014 | 0.83 [0.82, 0.85] |
| 46 | warp | Gaussian | misreg. | 100 | high | wiggly | default | alpha(t) | 0.002 | 0.002 | 0.98 [0.95, 1.01] |
| 46 | warp | Gaussian | misreg. | 100 | high | wiggly | default | beta(s,t) | 0.173 | 0.254 | 0.68 [0.62, 0.74] |
| 46 | warp | Gaussian | misreg. | 100 | high | wiggly | default | gamma(t) | 0.004 | 0.003 | 1.02 [1.01, 1.03] |
| 46 | warp | Gaussian | misreg. | 100 | high | wiggly | default | E(Y | X) | 0.018 | 0.019 | 0.91 [0.89, 0.92] |
| 47 | warp | Poisson | misreg. | 40 | high | smooth | default | E(Y | X) | 8.251 | 12.928 | 0.64 [0.54, 0.74] |
| 48 | warp | Poisson | misreg. | 40 | high | wiggly | default | E(Y | X) | 7.802 | 12.900 | 0.60 [0.46, 0.80] |
| 49 | warp | Poisson | misreg. | 100 | high | smooth | default | E(Y | X) | 4.625 | 6.419 | 0.72 [0.64, 0.81] |
| 50 | warp | Poisson | misreg. | 100 | high | wiggly | default | E(Y | X) | 4.130 | 5.709 | 0.72 [0.64, 0.82] |
| 51 | warp | binary | misreg. | 40 | high | smooth | default | E(Y | X) | 0.003 | 0.003 | 1.09 [1.06, 1.11] |
| 52 | warp | binary | misreg. | 40 | high | wiggly | default | E(Y | X) | 0.003 | 0.003 | 1.08 [1.06, 1.11] |
| 53 | warp | binary | misreg. | 100 | high | smooth | default | E(Y | X) | 0.001 | 0.001 | 1.01 [1.00, 1.03] |
| 54 | warp | binary | misreg. | 100 | high | wiggly | default | E(Y | X) | 0.002 | 0.001 | 1.04 [1.03, 1.06] |
| 55 | dense_grid | Gaussian | iid | 100 | mid | smooth | default | alpha(t) | 0.000 | 0.000 | 1.04 [1.01, 1.07] |
| 55 | dense_grid | Gaussian | iid | 100 | mid | smooth | default | beta(s,t) | 0.011 | 0.018 | 0.58 [0.54, 0.64] |
| 55 | dense_grid | Gaussian | iid | 100 | mid | smooth | default | gamma(t) | 0.001 | 0.000 | 1.12 [1.09, 1.15] |
| 55 | dense_grid | Gaussian | iid | 100 | mid | smooth | default | E(Y | X) | 0.003 | 0.003 | 0.94 [0.93, 0.96] |
| 56 | dense_grid | Gaussian | OU | 100 | mid | smooth | default | alpha(t) | 0.006 | 0.007 | 0.91 [0.89, 0.94] |
| 56 | dense_grid | Gaussian | OU | 100 | mid | smooth | default | beta(s,t) | 0.111 | 2.028 | 0.05 [0.05, 0.07] |
| 56 | dense_grid | Gaussian | OU | 100 | mid | smooth | default | gamma(t) | 0.006 | 0.007 | 0.92 [0.89, 0.94] |
| 56 | dense_grid | Gaussian | OU | 100 | mid | smooth | default | E(Y | X) | 0.032 | 0.060 | 0.52 [0.51, 0.54] |
| 57 | dense_grid | Gaussian | smooth | 100 | mid | smooth | default | alpha(t) | 0.007 | 0.008 | 0.87 [0.84, 0.90] |
| 57 | dense_grid | Gaussian | smooth | 100 | mid | smooth | default | beta(s,t) | 0.115 | 2.325 | 0.05 [0.04, 0.06] |
| 57 | dense_grid | Gaussian | smooth | 100 | mid | smooth | default | gamma(t) | 0.007 | 0.008 | 0.87 [0.84, 0.90] |
| 57 | dense_grid | Gaussian | smooth | 100 | mid | smooth | default | E(Y | X) | 0.034 | 0.069 | 0.50 [0.48, 0.51] |
| 58 | dense_grid | Poisson | iid | 100 | mid | smooth | default | alpha(t) | 0.000 | 0.000 | 1.05 [1.02, 1.08] |
| 58 | dense_grid | Poisson | iid | 100 | mid | smooth | default | beta(s,t) | 0.003 | 0.006 | 0.57 [0.55, 0.59] |
| 58 | dense_grid | Poisson | iid | 100 | mid | smooth | default | gamma(t) | 0.000 | 0.000 | 1.13 [1.09, 1.17] |
| 58 | dense_grid | Poisson | iid | 100 | mid | smooth | default | E(Y | X) | 0.010 | 0.011 | 0.94 [0.92, 0.96] |
| 59 | dense_grid | Poisson | OU | 100 | mid | smooth | default | alpha(t) | 0.002 | 0.003 | 0.83 [0.80, 0.87] |
| 59 | dense_grid | Poisson | OU | 100 | mid | smooth | default | beta(s,t) | 0.038 | 0.665 | 0.06 [0.05, 0.07] |
| 59 | dense_grid | Poisson | OU | 100 | mid | smooth | default | gamma(t) | 0.002 | 0.002 | 0.94 [0.91, 0.97] |
| 59 | dense_grid | Poisson | OU | 100 | mid | smooth | default | E(Y | X) | 0.135 | 0.272 | 0.49 [0.47, 0.52] |
| 60 | dense_grid | Poisson | smooth | 100 | mid | smooth | default | alpha(t) | 0.002 | 0.003 | 0.80 [0.76, 0.85] |
| 60 | dense_grid | Poisson | smooth | 100 | mid | smooth | default | beta(s,t) | 0.036 | 0.764 | 0.05 [0.04, 0.06] |
| 60 | dense_grid | Poisson | smooth | 100 | mid | smooth | default | gamma(t) | 0.002 | 0.003 | 0.88 [0.85, 0.92] |
| 60 | dense_grid | Poisson | smooth | 100 | mid | smooth | default | E(Y | X) | 0.147 | 0.320 | 0.46 [0.43, 0.49] |
| 61 | dense_grid | binary | iid | 100 | mid | smooth | default | alpha(t) | 0.002 | 0.002 | 1.07 [1.04, 1.11] |
| 61 | dense_grid | binary | iid | 100 | mid | smooth | default | beta(s,t) | 0.027 | 0.032 | 0.84 [0.78, 0.93] |
| 61 | dense_grid | binary | iid | 100 | mid | smooth | default | gamma(t) | 0.002 | 0.002 | 1.06 [1.04, 1.07] |
| 61 | dense_grid | binary | iid | 100 | mid | smooth | default | E(Y | X) | 0.000 | 0.000 | 1.01 [1.00, 1.03] |
| 62 | dense_grid | binary | OU | 100 | mid | smooth | default | alpha(t) | 0.022 | 0.034 | 0.66 [0.61, 0.71] |
| 62 | dense_grid | binary | OU | 100 | mid | smooth | default | beta(s,t) | 0.232 | 6.414 | 0.04 [0.03, 0.04] |
| 62 | dense_grid | binary | OU | 100 | mid | smooth | default | gamma(t) | 0.023 | 0.029 | 0.79 [0.74, 0.85] |
| 62 | dense_grid | binary | OU | 100 | mid | smooth | default | E(Y | X) | 0.003 | 0.005 | 0.47 [0.45, 0.49] |
| 63 | dense_grid | binary | smooth | 100 | mid | smooth | default | alpha(t) | 0.026 | 0.047 | 0.56 [0.52, 0.60] |
| 63 | dense_grid | binary | smooth | 100 | mid | smooth | default | beta(s,t) | 0.299 | 9.570 | 0.03 [0.03, 0.04] |
| 63 | dense_grid | binary | smooth | 100 | mid | smooth | default | gamma(t) | 0.025 | 0.037 | 0.68 [0.63, 0.73] |
| 63 | dense_grid | binary | smooth | 100 | mid | smooth | default | E(Y | X) | 0.003 | 0.007 | 0.41 [0.39, 0.42] |
| 64 | rough_truth | Gaussian | iid | 100 | low | wiggly | default | alpha(t) | 0.006 | 0.006 | 1.05 [1.03, 1.09] |
| 64 | rough_truth | Gaussian | iid | 100 | low | wiggly | default | beta(s,t) | 0.225 | 0.207 | 1.09 [1.07, 1.11] |
| 64 | rough_truth | Gaussian | iid | 100 | low | wiggly | default | gamma(t) | 0.007 | 0.007 | 1.05 [1.02, 1.08] |
| 64 | rough_truth | Gaussian | iid | 100 | low | wiggly | default | E(Y | X) | 0.034 | 0.033 | 1.04 [1.03, 1.05] |
| 65 | rough_truth | Gaussian | smooth | 100 | low | wiggly | default | alpha(t) | 0.026 | 0.026 | 0.98 [0.94, 1.02] |
| 65 | rough_truth | Gaussian | smooth | 100 | low | wiggly | default | beta(s,t) | 0.666 | 3.131 | 0.21 [0.19, 0.25] |
| 65 | rough_truth | Gaussian | smooth | 100 | low | wiggly | default | gamma(t) | 0.028 | 0.028 | 1.01 [0.98, 1.03] |
| 65 | rough_truth | Gaussian | smooth | 100 | low | wiggly | default | E(Y | X) | 0.139 | 0.193 | 0.72 [0.70, 0.74] |
| 66 | rough_truth | Gaussian | iid | 100 | mid | wiggly | default | alpha(t) | 0.002 | 0.002 | 1.05 [1.02, 1.07] |
| 66 | rough_truth | Gaussian | iid | 100 | mid | wiggly | default | beta(s,t) | 0.085 | 0.084 | 1.02 [1.01, 1.03] |
| 66 | rough_truth | Gaussian | iid | 100 | mid | wiggly | default | gamma(t) | 0.002 | 0.002 | 1.06 [1.04, 1.09] |
| 66 | rough_truth | Gaussian | iid | 100 | mid | wiggly | default | E(Y | X) | 0.010 | 0.010 | 1.02 [1.01, 1.03] |
| 67 | rough_truth | Gaussian | smooth | 100 | mid | wiggly | default | alpha(t) | 0.007 | 0.007 | 0.98 [0.96, 1.01] |
| 67 | rough_truth | Gaussian | smooth | 100 | mid | wiggly | default | beta(s,t) | 0.312 | 0.930 | 0.34 [0.30, 0.38] |
| 67 | rough_truth | Gaussian | smooth | 100 | mid | wiggly | default | gamma(t) | 0.007 | 0.007 | 1.02 [1.00, 1.03] |
| 67 | rough_truth | Gaussian | smooth | 100 | mid | wiggly | default | E(Y | X) | 0.042 | 0.053 | 0.79 [0.77, 0.81] |
| 68 | rough_truth | Gaussian | iid | 100 | high | wiggly | default | alpha(t) | 0.000 | 0.000 | 1.04 [1.02, 1.06] |
| 68 | rough_truth | Gaussian | iid | 100 | high | wiggly | default | beta(s,t) | 0.032 | 0.031 | 1.06 [1.02, 1.12] |
| 68 | rough_truth | Gaussian | iid | 100 | high | wiggly | default | gamma(t) | 0.000 | 0.000 | 1.05 [1.04, 1.07] |
| 68 | rough_truth | Gaussian | iid | 100 | high | wiggly | default | E(Y | X) | 0.003 | 0.003 | 1.01 [1.00, 1.02] |
| 69 | rough_truth | Gaussian | smooth | 100 | high | wiggly | default | alpha(t) | 0.002 | 0.002 | 0.99 [0.97, 1.01] |
| 69 | rough_truth | Gaussian | smooth | 100 | high | wiggly | default | beta(s,t) | 0.117 | 0.293 | 0.40 [0.36, 0.44] |
| 69 | rough_truth | Gaussian | smooth | 100 | high | wiggly | default | gamma(t) | 0.002 | 0.002 | 1.01 [0.99, 1.02] |
| 69 | rough_truth | Gaussian | smooth | 100 | high | wiggly | default | E(Y | X) | 0.012 | 0.015 | 0.83 [0.81, 0.84] |
| 70 | rough_truth | Poisson | iid | 100 | mid | wiggly | default | alpha(t) | 0.001 | 0.001 | 1.02 [0.99, 1.05] |
| 70 | rough_truth | Poisson | iid | 100 | mid | wiggly | default | beta(s,t) | 0.028 | 0.027 | 1.05 [1.02, 1.07] |
| 70 | rough_truth | Poisson | iid | 100 | mid | wiggly | default | gamma(t) | 0.001 | 0.001 | 1.07 [1.04, 1.10] |
| 70 | rough_truth | Poisson | iid | 100 | mid | wiggly | default | E(Y | X) | 0.040 | 0.039 | 1.02 [1.01, 1.03] |
| 71 | rough_truth | Poisson | smooth | 100 | mid | wiggly | default | alpha(t) | 0.003 | 0.003 | 0.93 [0.90, 0.96] |
| 71 | rough_truth | Poisson | smooth | 100 | mid | wiggly | default | beta(s,t) | 0.093 | 0.271 | 0.34 [0.31, 0.39] |
| 71 | rough_truth | Poisson | smooth | 100 | mid | wiggly | default | gamma(t) | 0.003 | 0.002 | 1.03 [1.01, 1.05] |
| 71 | rough_truth | Poisson | smooth | 100 | mid | wiggly | default | E(Y | X) | 0.170 | 0.227 | 0.75 [0.72, 0.78] |
| 72 | rough_truth | Poisson | iid | 100 | high | wiggly | default | alpha(t) | 0.001 | 0.001 | 1.05 [1.01, 1.09] |
| 72 | rough_truth | Poisson | iid | 100 | high | wiggly | default | beta(s,t) | 0.057 | 0.055 | 1.04 [1.01, 1.06] |
| 72 | rough_truth | Poisson | iid | 100 | high | wiggly | default | gamma(t) | 0.001 | 0.001 | 1.11 [1.07, 1.15] |
| 72 | rough_truth | Poisson | iid | 100 | high | wiggly | default | E(Y | X) | 0.162 | 0.163 | 0.99 [0.97, 1.02] |
| 73 | rough_truth | Poisson | smooth | 100 | high | wiggly | default | alpha(t) | 0.005 | 0.005 | 0.92 [0.89, 0.96] |
| 73 | rough_truth | Poisson | smooth | 100 | high | wiggly | default | beta(s,t) | 0.194 | 0.457 | 0.43 [0.39, 0.47] |
| 73 | rough_truth | Poisson | smooth | 100 | high | wiggly | default | gamma(t) | 0.004 | 0.004 | 1.09 [1.06, 1.13] |
| 73 | rough_truth | Poisson | smooth | 100 | high | wiggly | default | E(Y | X) | 0.730 | 1.096 | 0.67 [0.60, 0.73] |
| 74 | rough_truth | binary | iid | 100 | mid | wiggly | default | alpha(t) | 0.008 | 0.007 | 1.08 [1.03, 1.14] |
| 74 | rough_truth | binary | iid | 100 | mid | wiggly | default | beta(s,t) | 0.166 | 0.142 | 1.17 [1.11, 1.24] |
| 74 | rough_truth | binary | iid | 100 | mid | wiggly | default | gamma(t) | 0.010 | 0.010 | 1.00 [0.97, 1.03] |
| 74 | rough_truth | binary | iid | 100 | mid | wiggly | default | E(Y | X) | 0.001 | 0.001 | 1.05 [1.03, 1.07] |
| 75 | rough_truth | binary | smooth | 100 | mid | wiggly | default | alpha(t) | 0.027 | 0.031 | 0.87 [0.82, 0.92] |
| 75 | rough_truth | binary | smooth | 100 | mid | wiggly | default | beta(s,t) | 0.388 | 1.783 | 0.22 [0.18, 0.26] |
| 75 | rough_truth | binary | smooth | 100 | mid | wiggly | default | gamma(t) | 0.030 | 0.029 | 1.03 [0.98, 1.09] |
| 75 | rough_truth | binary | smooth | 100 | mid | wiggly | default | E(Y | X) | 0.003 | 0.004 | 0.77 [0.74, 0.80] |
| 76 | rough_truth | binary | iid | 100 | high | wiggly | default | alpha(t) | 0.011 | 0.011 | 1.05 [1.03, 1.08] |
| 76 | rough_truth | binary | iid | 100 | high | wiggly | default | beta(s,t) | 0.354 | 0.310 | 1.14 [1.10, 1.19] |
| 76 | rough_truth | binary | iid | 100 | high | wiggly | default | gamma(t) | 0.014 | 0.014 | 1.02 [1.00, 1.04] |
| 76 | rough_truth | binary | iid | 100 | high | wiggly | default | E(Y | X) | 0.001 | 0.001 | 1.05 [1.03, 1.06] |
| 77 | rough_truth | binary | smooth | 100 | high | wiggly | default | alpha(t) | 0.037 | 0.044 | 0.85 [0.80, 0.90] |
| 77 | rough_truth | binary | smooth | 100 | high | wiggly | default | beta(s,t) | 0.717 | 2.324 | 0.31 [0.27, 0.35] |
| 77 | rough_truth | binary | smooth | 100 | high | wiggly | default | gamma(t) | 0.039 | 0.040 | 0.98 [0.94, 1.02] |
| 77 | rough_truth | binary | smooth | 100 | high | wiggly | default | E(Y | X) | 0.004 | 0.005 | 0.79 [0.76, 0.81] |
| 78 | term_type | Gaussian | iid | 100 | mid | smooth | default | f(x,t) | 0.004 | 0.005 | 0.73 [0.72, 0.75] |
| 78 | term_type | Gaussian | iid | 100 | mid | smooth | default | gamma(t) | 0.002 | 0.002 | 1.05 [1.02, 1.08] |
| 78 | term_type | Gaussian | iid | 100 | mid | smooth | default | E(Y | X) | 0.010 | 0.010 | 0.93 [0.91, 0.95] |
| 79 | term_type | Gaussian | OU | 100 | mid | smooth | default | f(x,t) | 0.016 | 0.030 | 0.51 [0.49, 0.53] |
| 79 | term_type | Gaussian | OU | 100 | mid | smooth | default | gamma(t) | 0.007 | 0.007 | 0.97 [0.95, 0.99] |
| 79 | term_type | Gaussian | OU | 100 | mid | smooth | default | E(Y | X) | 0.038 | 0.056 | 0.67 [0.64, 0.70] |
| 80 | term_type | Gaussian | smooth | 100 | mid | smooth | default | f(x,t) | 0.018 | 0.035 | 0.51 [0.49, 0.53] |
| 80 | term_type | Gaussian | smooth | 100 | mid | smooth | default | gamma(t) | 0.007 | 0.007 | 0.95 [0.93, 0.98] |
| 80 | term_type | Gaussian | smooth | 100 | mid | smooth | default | E(Y | X) | 0.042 | 0.064 | 0.66 [0.63, 0.68] |
| 81 | term_type | Poisson | iid | 100 | mid | smooth | default | f(x,t) | 0.002 | 0.002 | 0.91 [0.89, 0.94] |
| 81 | term_type | Poisson | iid | 100 | mid | smooth | default | gamma(t) | 0.001 | 0.001 | 1.05 [1.03, 1.08] |
| 81 | term_type | Poisson | iid | 100 | mid | smooth | default | E(Y | X) | 0.029 | 0.031 | 0.93 [0.92, 0.95] |
| 82 | term_type | Poisson | OU | 100 | mid | smooth | default | f(x,t) | 0.007 | 0.011 | 0.60 [0.57, 0.63] |
| 82 | term_type | Poisson | OU | 100 | mid | smooth | default | gamma(t) | 0.002 | 0.002 | 0.99 [0.97, 1.02] |
| 82 | term_type | Poisson | OU | 100 | mid | smooth | default | E(Y | X) | 0.111 | 0.162 | 0.68 [0.66, 0.70] |
| 83 | term_type | Poisson | smooth | 100 | mid | smooth | default | f(x,t) | 0.007 | 0.013 | 0.57 [0.54, 0.60] |
| 83 | term_type | Poisson | smooth | 100 | mid | smooth | default | gamma(t) | 0.002 | 0.002 | 0.98 [0.96, 1.01] |
| 83 | term_type | Poisson | smooth | 100 | mid | smooth | default | E(Y | X) | 0.120 | 0.182 | 0.66 [0.64, 0.68] |
| 84 | term_type | binary | iid | 100 | mid | smooth | default | f(x,t) | 0.015 | 0.014 | 1.07 [1.04, 1.11] |
| 84 | term_type | binary | iid | 100 | mid | smooth | default | gamma(t) | 0.008 | 0.007 | 1.10 [1.06, 1.14] |
| 84 | term_type | binary | iid | 100 | mid | smooth | default | E(Y | X) | 0.001 | 0.001 | 1.07 [1.05, 1.10] |
| 85 | term_type | binary | OU | 100 | mid | smooth | default | f(x,t) | 0.044 | 0.093 | 0.47 [0.44, 0.51] |
| 85 | term_type | binary | OU | 100 | mid | smooth | default | gamma(t) | 0.024 | 0.024 | 0.99 [0.93, 1.05] |
| 85 | term_type | binary | OU | 100 | mid | smooth | default | E(Y | X) | 0.002 | 0.004 | 0.66 [0.63, 0.68] |
| 86 | term_type | binary | smooth | 100 | mid | smooth | default | f(x,t) | 0.052 | 0.126 | 0.41 [0.38, 0.44] |
| 86 | term_type | binary | smooth | 100 | mid | smooth | default | gamma(t) | 0.026 | 0.028 | 0.94 [0.89, 1.01] |
| 86 | term_type | binary | smooth | 100 | mid | smooth | default | E(Y | X) | 0.003 | 0.005 | 0.59 [0.56, 0.62] |
| 87 | families | scaled t | iid | 100 | mid | smooth | default | alpha(t) | 0.001 | 0.001 | 1.04 [1.02, 1.07] |
| 87 | families | scaled t | iid | 100 | mid | smooth | default | beta(s,t) | 0.020 | 0.034 | 0.60 [0.58, 0.63] |
| 87 | families | scaled t | iid | 100 | mid | smooth | default | gamma(t) | 0.001 | 0.001 | 1.05 [1.02, 1.09] |
| 87 | families | scaled t | iid | 100 | mid | smooth | default | E(Y | X) | 0.006 | 0.006 | 0.95 [0.94, 0.97] |
| 88 | families | scaled t | smooth | 100 | mid | smooth | default | alpha(t) | 0.005 | 0.005 | 0.97 [0.95, 1.00] |
| 88 | families | scaled t | smooth | 100 | mid | smooth | default | beta(s,t) | 0.103 | 0.473 | 0.22 [0.18, 0.26] |
| 88 | families | scaled t | smooth | 100 | mid | smooth | default | gamma(t) | 0.005 | 0.005 | 0.95 [0.93, 0.98] |
| 88 | families | scaled t | smooth | 100 | mid | smooth | default | E(Y | X) | 0.024 | 0.034 | 0.71 [0.70, 0.74] |
| 89 | families | beta | iid | 100 | mid | smooth | default | alpha(t) | 0.000 | 0.000 | 1.06 [1.03, 1.09] |
| 89 | families | beta | iid | 100 | mid | smooth | default | beta(s,t) | 0.007 | 0.010 | 0.75 [0.65, 0.91] |
| 89 | families | beta | iid | 100 | mid | smooth | default | gamma(t) | 0.000 | 0.000 | 1.08 [1.05, 1.12] |
| 89 | families | beta | iid | 100 | mid | smooth | default | E(Y | X) | 0.000 | 0.000 | 0.98 [0.96, 0.99] |
| 90 | families | beta | smooth | 100 | mid | smooth | default | alpha(t) | 0.002 | 0.002 | 0.97 [0.94, 1.00] |
| 90 | families | beta | smooth | 100 | mid | smooth | default | beta(s,t) | 0.032 | 0.176 | 0.18 [0.15, 0.22] |
| 90 | families | beta | smooth | 100 | mid | smooth | default | gamma(t) | 0.002 | 0.002 | 0.96 [0.94, 0.99] |
| 90 | families | beta | smooth | 100 | mid | smooth | default | E(Y | X) | 0.000 | 0.000 | 0.70 [0.68, 0.72] |
| 91 | families | negative binomial | iid | 100 | mid | smooth | default | alpha(t) | 0.001 | 0.001 | 1.06 [1.03, 1.10] |
| 91 | families | negative binomial | iid | 100 | mid | smooth | default | beta(s,t) | 0.015 | 0.017 | 0.89 [0.75, 1.05] |
| 91 | families | negative binomial | iid | 100 | mid | smooth | default | gamma(t) | 0.001 | 0.001 | 1.10 [1.06, 1.14] |
| 91 | families | negative binomial | iid | 100 | mid | smooth | default | E(Y | X) | 0.060 | 0.057 | 1.06 [1.03, 1.09] |
| 92 | families | negative binomial | smooth | 100 | mid | smooth | default | alpha(t) | 0.004 | 0.004 | 0.92 [0.88, 0.96] |
| 92 | families | negative binomial | smooth | 100 | mid | smooth | default | beta(s,t) | 0.068 | 0.391 | 0.18 [0.14, 0.21] |
| 92 | families | negative binomial | smooth | 100 | mid | smooth | default | gamma(t) | 0.004 | 0.004 | 0.97 [0.94, 1.00] |
| 92 | families | negative binomial | smooth | 100 | mid | smooth | default | E(Y | X) | 0.280 | 0.407 | 0.69 [0.64, 0.74] |
| 93 | families | negative binomial | iid | 100 | mid | smooth | default | alpha(t) | 0.001 | 0.001 | 1.00 [0.97, 1.04] |
| 93 | families | negative binomial | iid | 100 | mid | smooth | default | beta(s,t) | 0.013 | 0.025 | 0.53 [0.49, 0.59] |
| 93 | families | negative binomial | iid | 100 | mid | smooth | default | gamma(t) | 0.001 | 0.001 | 1.04 [1.01, 1.08] |
| 93 | families | negative binomial | iid | 100 | mid | smooth | default | E(Y | X) | 0.062 | 0.068 | 0.90 [0.87, 0.93] |
| 94 | families | negative binomial | smooth | 100 | mid | smooth | default | alpha(t) | 0.004 | 0.004 | 0.87 [0.82, 0.92] |
| 94 | families | negative binomial | smooth | 100 | mid | smooth | default | beta(s,t) | 0.065 | 0.749 | 0.09 [0.07, 0.11] |
| 94 | families | negative binomial | smooth | 100 | mid | smooth | default | gamma(t) | 0.004 | 0.004 | 0.96 [0.92, 0.99] |
| 94 | families | negative binomial | smooth | 100 | mid | smooth | default | E(Y | X) | 0.296 | 0.534 | 0.55 [0.52, 0.59] |
| 95 | ar1_home | Gaussian | AR(1) | 100 | mid | smooth | default | alpha(t) | 0.007 | 0.007 | 0.95 [0.93, 0.97] |
| 95 | ar1_home | Gaussian | AR(1) | 100 | mid | smooth | default | beta(s,t) | 0.112 | 0.761 | 0.15 [0.13, 0.17] |
| 95 | ar1_home | Gaussian | AR(1) | 100 | mid | smooth | default | gamma(t) | 0.007 | 0.007 | 0.95 [0.92, 0.97] |
| 95 | ar1_home | Gaussian | AR(1) | 100 | mid | smooth | default | E(Y | X) | 0.035 | 0.052 | 0.67 [0.66, 0.69] |
| 96 | oscillating | Gaussian | sign-chg. | 100 | mid | smooth | default | alpha(t) | 0.006 | 0.007 | 0.89 [0.84, 0.93] |
| 96 | oscillating | Gaussian | sign-chg. | 100 | mid | smooth | default | beta(s,t) | 0.088 | 0.478 | 0.18 [0.17, 0.20] |
| 96 | oscillating | Gaussian | sign-chg. | 100 | mid | smooth | default | gamma(t) | 0.006 | 0.007 | 0.86 [0.82, 0.90] |
| 96 | oscillating | Gaussian | sign-chg. | 100 | mid | smooth | default | E(Y | X) | 0.029 | 0.048 | 0.61 [0.59, 0.62] |
| 97 | oscillating | Poisson | sign-chg. | 100 | mid | smooth | default | alpha(t) | 0.002 | 0.003 | 0.79 [0.75, 0.82] |
| 97 | oscillating | Poisson | sign-chg. | 100 | mid | smooth | default | beta(s,t) | 0.029 | 0.149 | 0.19 [0.18, 0.21] |
| 97 | oscillating | Poisson | sign-chg. | 100 | mid | smooth | default | gamma(t) | 0.002 | 0.002 | 0.92 [0.88, 0.97] |
| 97 | oscillating | Poisson | sign-chg. | 100 | mid | smooth | default | E(Y | X) | 0.120 | 0.203 | 0.59 [0.56, 0.62] |
| 98 | oscillating | binary | sign-chg. | 100 | mid | smooth | default | alpha(t) | 0.017 | 0.027 | 0.64 [0.60, 0.68] |
| 98 | oscillating | binary | sign-chg. | 100 | mid | smooth | default | beta(s,t) | 0.202 | 0.782 | 0.26 [0.21, 0.32] |
| 98 | oscillating | binary | sign-chg. | 100 | mid | smooth | default | gamma(t) | 0.018 | 0.024 | 0.73 [0.69, 0.77] |
| 98 | oscillating | binary | sign-chg. | 100 | mid | smooth | default | E(Y | X) | 0.002 | 0.003 | 0.61 [0.58, 0.63] |
| 99 | warp_ar1 | Gaussian | misreg. | 100 | high | wiggly | default | alpha(t) | 0.002 | 0.002 | 0.98 [0.95, 1.01] |
| 99 | warp_ar1 | Gaussian | misreg. | 100 | high | wiggly | default | beta(s,t) | 0.173 | 0.254 | 0.68 [0.62, 0.74] |
| 99 | warp_ar1 | Gaussian | misreg. | 100 | high | wiggly | default | gamma(t) | 0.004 | 0.003 | 1.02 [1.01, 1.03] |
| 99 | warp_ar1 | Gaussian | misreg. | 100 | high | wiggly | default | E(Y | X) | 0.018 | 0.019 | 0.91 [0.89, 0.92] |
| 100 | heteroskedastic | Gaussian | var(t) | 40 | mid | smooth | default | alpha(t) | 0.018 | 0.020 | 0.91 [0.88, 0.95] |
| 100 | heteroskedastic | Gaussian | var(t) | 40 | mid | smooth | default | beta(s,t) | 0.369 | 2.812 | 0.13 [0.10, 0.17] |
| 100 | heteroskedastic | Gaussian | var(t) | 40 | mid | smooth | default | gamma(t) | 0.017 | 0.018 | 0.90 [0.86, 0.94] |
| 100 | heteroskedastic | Gaussian | var(t) | 40 | mid | smooth | default | E(Y | X) | 0.086 | 0.144 | 0.60 [0.58, 0.62] |
| 101 | heteroskedastic | Gaussian | var(t) | 100 | mid | smooth | default | alpha(t) | 0.007 | 0.007 | 0.96 [0.94, 0.99] |
| 101 | heteroskedastic | Gaussian | var(t) | 100 | mid | smooth | default | beta(s,t) | 0.127 | 0.768 | 0.17 [0.14, 0.20] |
| 101 | heteroskedastic | Gaussian | var(t) | 100 | mid | smooth | default | gamma(t) | 0.007 | 0.007 | 0.94 [0.92, 0.96] |
| 101 | heteroskedastic | Gaussian | var(t) | 100 | mid | smooth | default | E(Y | X) | 0.034 | 0.050 | 0.68 [0.66, 0.70] |
| 102 | heteroskedastic | Gaussian | var(z) | 40 | mid | smooth | default | alpha(t) | 0.018 | 0.019 | 0.93 [0.89, 0.97] |
| 102 | heteroskedastic | Gaussian | var(z) | 40 | mid | smooth | default | beta(s,t) | 0.267 | 2.620 | 0.10 [0.08, 0.13] |
| 102 | heteroskedastic | Gaussian | var(z) | 40 | mid | smooth | default | gamma(t) | 0.031 | 0.037 | 0.83 [0.79, 0.88] |
| 102 | heteroskedastic | Gaussian | var(z) | 40 | mid | smooth | default | E(Y | X) | 0.104 | 0.164 | 0.63 [0.61, 0.66] |
| 103 | heteroskedastic | Gaussian | var(z) | 100 | mid | smooth | default | alpha(t) | 0.007 | 0.007 | 0.97 [0.94, 0.99] |
| 103 | heteroskedastic | Gaussian | var(z) | 100 | mid | smooth | default | beta(s,t) | 0.124 | 0.761 | 0.16 [0.14, 0.19] |
| 103 | heteroskedastic | Gaussian | var(z) | 100 | mid | smooth | default | gamma(t) | 0.013 | 0.014 | 0.90 [0.87, 0.92] |
| 103 | heteroskedastic | Gaussian | var(z) | 100 | mid | smooth | default | E(Y | X) | 0.042 | 0.059 | 0.72 [0.70, 0.74] |
| 104 | heteroskedastic | Gaussian | var(t,z) | 40 | mid | smooth | default | alpha(t) | 0.017 | 0.019 | 0.93 [0.89, 0.97] |
| 104 | heteroskedastic | Gaussian | var(t,z) | 40 | mid | smooth | default | beta(s,t) | 0.274 | 2.565 | 0.11 [0.09, 0.13] |
| 104 | heteroskedastic | Gaussian | var(t,z) | 40 | mid | smooth | default | gamma(t) | 0.031 | 0.038 | 0.83 [0.79, 0.88] |
| 104 | heteroskedastic | Gaussian | var(t,z) | 40 | mid | smooth | default | E(Y | X) | 0.104 | 0.165 | 0.63 [0.61, 0.66] |
| 105 | heteroskedastic | Gaussian | var(t,z) | 100 | mid | smooth | default | alpha(t) | 0.007 | 0.007 | 0.96 [0.93, 0.98] |
| 105 | heteroskedastic | Gaussian | var(t,z) | 100 | mid | smooth | default | beta(s,t) | 0.120 | 0.793 | 0.15 [0.13, 0.18] |
| 105 | heteroskedastic | Gaussian | var(t,z) | 100 | mid | smooth | default | gamma(t) | 0.013 | 0.014 | 0.89 [0.85, 0.92] |
| 105 | heteroskedastic | Gaussian | var(t,z) | 100 | mid | smooth | default | E(Y | X) | 0.042 | 0.060 | 0.71 [0.69, 0.73] |
| 106 | lowrank_covariate | Gaussian | iid | 100 | mid | smooth | default | alpha(t) | 0.002 | 0.002 | 1.04 [1.02, 1.07] |
| 106 | lowrank_covariate | Gaussian | iid | 100 | mid | smooth | default | beta(s,t) | 0.072 | 0.102 | 0.71 [0.62, 0.82] |
| 106 | lowrank_covariate | Gaussian | iid | 100 | mid | smooth | default | gamma(t) | 0.002 | 0.002 | 1.04 [1.02, 1.07] |
| 106 | lowrank_covariate | Gaussian | iid | 100 | mid | smooth | default | E(Y | X) | 0.008 | 0.008 | 0.97 [0.96, 0.99] |
| 107 | lowrank_covariate | Gaussian | smooth | 100 | mid | smooth | default | alpha(t) | 0.007 | 0.007 | 0.97 [0.94, 0.99] |
| 107 | lowrank_covariate | Gaussian | smooth | 100 | mid | smooth | default | beta(s,t) | 0.319 | 3.747 | 0.09 [0.06, 0.11] |
| 107 | lowrank_covariate | Gaussian | smooth | 100 | mid | smooth | default | gamma(t) | 0.007 | 0.007 | 0.95 [0.93, 0.97] |
| 107 | lowrank_covariate | Gaussian | smooth | 100 | mid | smooth | default | E(Y | X) | 0.033 | 0.048 | 0.70 [0.68, 0.72] |
| 108 | lowrank_covariate | Poisson | iid | 100 | mid | smooth | default | alpha(t) | 0.001 | 0.001 | 1.03 [1.00, 1.06] |
| 108 | lowrank_covariate | Poisson | iid | 100 | mid | smooth | default | beta(s,t) | 0.023 | 0.031 | 0.73 [0.63, 0.84] |
| 108 | lowrank_covariate | Poisson | iid | 100 | mid | smooth | default | gamma(t) | 0.001 | 0.001 | 1.07 [1.04, 1.10] |
| 108 | lowrank_covariate | Poisson | iid | 100 | mid | smooth | default | E(Y | X) | 0.032 | 0.032 | 0.99 [0.97, 1.01] |
| 109 | lowrank_covariate | Poisson | smooth | 100 | mid | smooth | default | alpha(t) | 0.003 | 0.003 | 0.93 [0.90, 0.97] |
| 109 | lowrank_covariate | Poisson | smooth | 100 | mid | smooth | default | beta(s,t) | 0.098 | 1.107 | 0.09 [0.06, 0.12] |
| 109 | lowrank_covariate | Poisson | smooth | 100 | mid | smooth | default | gamma(t) | 0.002 | 0.002 | 0.98 [0.95, 1.01] |
| 109 | lowrank_covariate | Poisson | smooth | 100 | mid | smooth | default | E(Y | X) | 0.144 | 0.210 | 0.69 [0.65, 0.72] |
| 110 | lowrank_covariate | binary | iid | 100 | mid | smooth | default | alpha(t) | 0.008 | 0.007 | 1.06 [1.03, 1.11] |
| 110 | lowrank_covariate | binary | iid | 100 | mid | smooth | default | beta(s,t) | 0.198 | 0.157 | 1.26 [1.03, 1.56] |
| 110 | lowrank_covariate | binary | iid | 100 | mid | smooth | default | gamma(t) | 0.008 | 0.007 | 1.07 [1.04, 1.10] |
| 110 | lowrank_covariate | binary | iid | 100 | mid | smooth | default | E(Y | X) | 0.001 | 0.001 | 1.07 [1.05, 1.09] |
| 111 | lowrank_covariate | binary | smooth | 100 | mid | smooth | default | alpha(t) | 0.026 | 0.030 | 0.87 [0.82, 0.91] |
| 111 | lowrank_covariate | binary | smooth | 100 | mid | smooth | default | beta(s,t) | 0.848 | 7.015 | 0.12 [0.08, 0.17] |
| 111 | lowrank_covariate | binary | smooth | 100 | mid | smooth | default | gamma(t) | 0.025 | 0.027 | 0.95 [0.90, 1.01] |
| 111 | lowrank_covariate | binary | smooth | 100 | mid | smooth | default | E(Y | X) | 0.003 | 0.004 | 0.77 [0.74, 0.80] |
| 112 | basis_size | Gaussian | iid | 100 | mid | smooth | large | alpha(t) | 0.002 | 0.002 | 0.94 [0.93, 0.96] |
| 112 | basis_size | Gaussian | iid | 100 | mid | smooth | large | beta(s,t) | 0.038 | 0.068 | 0.56 [0.53, 0.60] |
| 112 | basis_size | Gaussian | iid | 100 | mid | smooth | large | gamma(t) | 0.002 | 0.002 | 1.01 [0.98, 1.03] |
| 112 | basis_size | Gaussian | iid | 100 | mid | smooth | large | E(Y | X) | 0.010 | 0.011 | 0.89 [0.88, 0.90] |
| 113 | basis_size | Gaussian | iid | 100 | mid | smooth | xlarge | alpha(t) | 0.002 | 0.002 | 0.89 [0.87, 0.91] |
| 113 | basis_size | Gaussian | iid | 100 | mid | smooth | xlarge | beta(s,t) | 0.044 | 0.090 | 0.49 [0.47, 0.51] |
| 113 | basis_size | Gaussian | iid | 100 | mid | smooth | xlarge | gamma(t) | 0.002 | 0.002 | 0.96 [0.94, 0.98] |
| 113 | basis_size | Gaussian | iid | 100 | mid | smooth | xlarge | E(Y | X) | 0.012 | 0.014 | 0.85 [0.83, 0.86] |
| 114 | basis_size | Gaussian | smooth | 100 | mid | smooth | large | alpha(t) | 0.007 | 0.008 | 0.85 [0.82, 0.88] |
| 114 | basis_size | Gaussian | smooth | 100 | mid | smooth | large | beta(s,t) | 0.120 | 3.619 | 0.03 [0.03, 0.04] |
| 114 | basis_size | Gaussian | smooth | 100 | mid | smooth | large | gamma(t) | 0.007 | 0.008 | 0.87 [0.83, 0.90] |
| 114 | basis_size | Gaussian | smooth | 100 | mid | smooth | large | E(Y | X) | 0.038 | 0.080 | 0.47 [0.45, 0.49] |
| 115 | basis_size | Gaussian | smooth | 100 | mid | smooth | xlarge | alpha(t) | 0.007 | 0.009 | 0.81 [0.77, 0.85] |
| 115 | basis_size | Gaussian | smooth | 100 | mid | smooth | xlarge | beta(s,t) | 0.148 | 11.772 | 0.01 [0.01, 0.02] |
| 115 | basis_size | Gaussian | smooth | 100 | mid | smooth | xlarge | gamma(t) | 0.007 | 0.009 | 0.82 [0.79, 0.85] |
| 115 | basis_size | Gaussian | smooth | 100 | mid | smooth | xlarge | E(Y | X) | 0.040 | 0.121 | 0.33 [0.32, 0.34] |
| 116 | basis_size | Poisson | iid | 100 | mid | smooth | large | alpha(t) | 0.001 | 0.001 | 0.96 [0.94, 0.99] |
| 116 | basis_size | Poisson | iid | 100 | mid | smooth | large | beta(s,t) | 0.011 | 0.020 | 0.55 [0.53, 0.57] |
| 116 | basis_size | Poisson | iid | 100 | mid | smooth | large | gamma(t) | 0.001 | 0.001 | 1.05 [1.03, 1.07] |
| 116 | basis_size | Poisson | iid | 100 | mid | smooth | large | E(Y | X) | 0.042 | 0.046 | 0.91 [0.89, 0.93] |
| 117 | basis_size | Poisson | iid | 100 | mid | smooth | xlarge | alpha(t) | 0.001 | 0.001 | 0.92 [0.90, 0.95] |
| 117 | basis_size | Poisson | iid | 100 | mid | smooth | xlarge | beta(s,t) | 0.013 | 0.026 | 0.52 [0.49, 0.54] |
| 117 | basis_size | Poisson | iid | 100 | mid | smooth | xlarge | gamma(t) | 0.001 | 0.001 | 1.03 [1.01, 1.06] |
| 117 | basis_size | Poisson | iid | 100 | mid | smooth | xlarge | E(Y | X) | 0.049 | 0.056 | 0.87 [0.86, 0.89] |
| 118 | basis_size | Poisson | smooth | 100 | mid | smooth | large | alpha(t) | 0.003 | 0.003 | 0.82 [0.77, 0.86] |
| 118 | basis_size | Poisson | smooth | 100 | mid | smooth | large | beta(s,t) | 0.035 | 1.014 | 0.03 [0.03, 0.04] |
| 118 | basis_size | Poisson | smooth | 100 | mid | smooth | large | gamma(t) | 0.002 | 0.003 | 0.92 [0.88, 0.95] |
| 118 | basis_size | Poisson | smooth | 100 | mid | smooth | large | E(Y | X) | 0.164 | 0.382 | 0.43 [0.40, 0.46] |
| 119 | basis_size | Poisson | smooth | 100 | mid | smooth | xlarge | alpha(t) | 0.003 | 0.004 | 0.75 [0.71, 0.81] |
| 119 | basis_size | Poisson | smooth | 100 | mid | smooth | xlarge | beta(s,t) | 0.044 | 3.286 | 0.01 [0.01, 0.02] |
| 119 | basis_size | Poisson | smooth | 100 | mid | smooth | xlarge | gamma(t) | 0.003 | 0.003 | 0.89 [0.85, 0.93] |
| 119 | basis_size | Poisson | smooth | 100 | mid | smooth | xlarge | E(Y | X) | 0.175 | 0.554 | 0.32 [0.29, 0.34] |
| 120 | basis_size | binary | iid | 100 | mid | smooth | large | alpha(t) | 0.008 | 0.007 | 1.06 [1.02, 1.10] |
| 120 | basis_size | binary | iid | 100 | mid | smooth | large | beta(s,t) | 0.098 | 0.094 | 1.05 [0.91, 1.22] |
| 120 | basis_size | binary | iid | 100 | mid | smooth | large | gamma(t) | 0.009 | 0.008 | 1.10 [1.05, 1.16] |
| 120 | basis_size | binary | iid | 100 | mid | smooth | large | E(Y | X) | 0.001 | 0.001 | 1.06 [1.04, 1.09] |
| 121 | basis_size | binary | iid | 100 | mid | smooth | xlarge | alpha(t) | 0.008 | 0.007 | 1.05 [1.00, 1.11] |
| 121 | basis_size | binary | iid | 100 | mid | smooth | xlarge | beta(s,t) | 0.107 | 0.104 | 1.03 [0.92, 1.16] |
| 121 | basis_size | binary | iid | 100 | mid | smooth | xlarge | gamma(t) | 0.009 | 0.008 | 1.10 [1.05, 1.18] |
| 121 | basis_size | binary | iid | 100 | mid | smooth | xlarge | E(Y | X) | 0.001 | 0.001 | 1.05 [1.03, 1.08] |
| 122 | basis_size | binary | smooth | 100 | mid | smooth | large | alpha(t) | 0.026 | 0.038 | 0.68 [0.64, 0.73] |
| 122 | basis_size | binary | smooth | 100 | mid | smooth | large | beta(s,t) | 0.294 | 7.326 | 0.04 [0.03, 0.05] |
| 122 | basis_size | binary | smooth | 100 | mid | smooth | large | gamma(t) | 0.026 | 0.031 | 0.83 [0.78, 0.88] |
| 122 | basis_size | binary | smooth | 100 | mid | smooth | large | E(Y | X) | 0.003 | 0.006 | 0.49 [0.46, 0.51] |
| 123 | basis_size | binary | smooth | 100 | mid | smooth | xlarge | alpha(t) | 0.026 | 0.050 | 0.51 [0.47, 0.56] |
| 123 | basis_size | binary | smooth | 100 | mid | smooth | xlarge | beta(s,t) | 0.326 | 32.696 | 0.01 [0.01, 0.01] |
| 123 | basis_size | binary | smooth | 100 | mid | smooth | xlarge | gamma(t) | 0.026 | 0.035 | 0.74 [0.69, 0.78] |
| 123 | basis_size | binary | smooth | 100 | mid | smooth | xlarge | E(Y | X) | 0.003 | 0.010 | 0.31 [0.30, 0.33] |
16.5 S4 Plasmode coverage per cell
| dataset | truth | residual | flip | error | G | estimand | AR(1) working model | NCV + CL2 | NCV + CL2, bias-aware | REML + CL2 | REML, model-based |
|---|---|---|---|---|---|---|---|---|---|---|---|
| ECG strain | MID | NCV residuals | curve | attached | 78 | beta(s,t) | 0.900 (0.005) | 0.931 (0.004) | 0.943 (0.003) | 0.558 (0.006) | |
| ECG strain | MID | REML residuals | curve | attached | 78 | beta(s,t) | 0.874 (0.005) | 0.904 (0.005) | 0.932 (0.004) | 0.944 (0.003) | 0.561 (0.006) |
| ECG strain | MID | beat differences (own scale) | curve | attached | 78 | beta(s,t) | 0.933 (0.004) | 0.946 (0.003) | 0.945 (0.004) | 0.534 (0.005) | |
| ECG strain | MID | beat differences (variance-matched) | curve | attached | 78 | beta(s,t) | 0.837 (0.004) | 0.910 (0.005) | 0.940 (0.003) | 0.945 (0.004) | 0.524 (0.005) |
| ECG strain | NCV | NCV residuals | curve | attached | 78 | beta(s,t) | 0.909 (0.005) | 0.941 (0.003) | 0.944 (0.003) | 0.559 (0.006) | |
| ECG strain | NCV | REML residuals | curve | attached | 78 | beta(s,t) | 0.909 (0.005) | 0.939 (0.003) | 0.944 (0.003) | 0.562 (0.006) | |
| ECG strain | NCV | beat differences (own scale) | curve | attached | 78 | beta(s,t) | 0.928 (0.004) | 0.946 (0.003) | 0.945 (0.004) | 0.532 (0.005) | |
| ECG strain | NCV | beat differences (variance-matched) | curve | attached | 78 | beta(s,t) | 0.906 (0.005) | 0.942 (0.003) | 0.945 (0.004) | 0.523 (0.005) | |
| ECG strain | REML | NCV residuals | curve | attached | 78 | beta(s,t) | 0.899 (0.005) | 0.929 (0.004) | 0.943 (0.003) | 0.559 (0.006) | |
| ECG strain | REML | REML residuals | curve | attached | 78 | beta(s,t) | 0.904 (0.005) | 0.931 (0.004) | 0.943 (0.003) | 0.563 (0.006) | |
| ECG strain | REML | beat differences (own scale) | curve | attached | 78 | beta(s,t) | 0.932 (0.004) | 0.944 (0.003) | 0.945 (0.003) | 0.536 (0.005) | |
| ECG strain | REML | beat differences (variance-matched) | curve | attached | 78 | beta(s,t) | 0.910 (0.004) | 0.939 (0.003) | 0.944 (0.004) | 0.526 (0.005) | |
| ECG strain | MID | NCV residuals | curve | attached | 40 | beta(s,t) | 0.873 (0.006) | 0.928 (0.004) | 0.935 (0.004) | 0.529 (0.006) | |
| ECG strain | MID | REML residuals | curve | attached | 40 | beta(s,t) | 0.873 (0.006) | 0.925 (0.004) | 0.932 (0.004) | 0.532 (0.006) | |
| ECG strain | MID | beat differences (own scale) | curve | attached | 40 | beta(s,t) | 0.913 (0.005) | 0.936 (0.004) | 0.934 (0.004) | 0.524 (0.006) | |
| ECG strain | MID | beat differences (variance-matched) | curve | attached | 40 | beta(s,t) | 0.845 (0.011) | 0.930 (0.004) | 0.935 (0.004) | 0.516 (0.006) | |
| ECG strain | NCV | NCV residuals | curve | attached | 40 | beta(s,t) | 0.887 (0.006) | 0.942 (0.004) | 0.936 (0.004) | 0.530 (0.006) | |
| ECG strain | NCV | REML residuals | curve | attached | 40 | beta(s,t) | 0.887 (0.006) | 0.941 (0.004) | 0.933 (0.004) | 0.533 (0.006) | |
| ECG strain | NCV | beat differences (own scale) | curve | attached | 40 | beta(s,t) | 0.910 (0.005) | 0.939 (0.004) | 0.935 (0.004) | 0.522 (0.006) | |
| ECG strain | NCV | beat differences (variance-matched) | curve | attached | 40 | beta(s,t) | 0.854 (0.011) | 0.938 (0.004) | 0.935 (0.004) | 0.515 (0.006) | |
| ECG strain | REML | NCV residuals | curve | attached | 40 | beta(s,t) | 0.867 (0.006) | 0.922 (0.004) | 0.934 (0.004) | 0.530 (0.006) | |
| ECG strain | REML | REML residuals | curve | attached | 40 | beta(s,t) | 0.868 (0.006) | 0.921 (0.004) | 0.932 (0.004) | 0.533 (0.006) | |
| ECG strain | REML | beat differences (own scale) | curve | attached | 40 | beta(s,t) | 0.911 (0.004) | 0.933 (0.004) | 0.934 (0.004) | 0.526 (0.006) | |
| ECG strain | REML | beat differences (variance-matched) | curve | attached | 40 | beta(s,t) | 0.837 (0.012) | 0.927 (0.004) | 0.934 (0.004) | 0.517 (0.006) | |
| ECG strain | MID | NCV residuals | curve | attached | 78 | E(Y | X) | 0.916 (0.003) | 0.938 (0.002) | 0.940 (0.002) | 0.581 (0.004) | |
| ECG strain | MID | REML residuals | curve | attached | 78 | E(Y | X) | 0.869 (0.003) | 0.916 (0.003) | 0.937 (0.002) | 0.938 (0.002) | 0.583 (0.004) |
| ECG strain | MID | beat differences (own scale) | curve | attached | 78 | E(Y | X) | 0.940 (0.002) | 0.951 (0.002) | 0.950 (0.002) | 0.570 (0.003) | |
| ECG strain | MID | beat differences (variance-matched) | curve | attached | 78 | E(Y | X) | 0.847 (0.003) | 0.923 (0.002) | 0.947 (0.002) | 0.950 (0.002) | 0.564 (0.003) |
| ECG strain | NCV | NCV residuals | curve | attached | 78 | E(Y | X) | 0.918 (0.003) | 0.942 (0.002) | 0.940 (0.002) | 0.582 (0.004) | |
| ECG strain | NCV | REML residuals | curve | attached | 78 | E(Y | X) | 0.916 (0.003) | 0.939 (0.002) | 0.938 (0.003) | 0.583 (0.004) | |
| ECG strain | NCV | beat differences (own scale) | curve | attached | 78 | E(Y | X) | 0.935 (0.002) | 0.950 (0.002) | 0.950 (0.002) | 0.569 (0.003) | |
| ECG strain | NCV | beat differences (variance-matched) | curve | attached | 78 | E(Y | X) | 0.921 (0.002) | 0.948 (0.002) | 0.950 (0.002) | 0.564 (0.003) | |
| ECG strain | REML | NCV residuals | curve | attached | 78 | E(Y | X) | 0.904 (0.003) | 0.933 (0.002) | 0.939 (0.002) | 0.583 (0.004) | |
| ECG strain | REML | REML residuals | curve | attached | 78 | E(Y | X) | 0.906 (0.003) | 0.933 (0.002) | 0.938 (0.002) | 0.586 (0.004) | |
| ECG strain | REML | beat differences (own scale) | curve | attached | 78 | E(Y | X) | 0.940 (0.002) | 0.948 (0.002) | 0.950 (0.002) | 0.579 (0.003) | |
| ECG strain | REML | beat differences (variance-matched) | curve | attached | 78 | E(Y | X) | 0.921 (0.002) | 0.945 (0.002) | 0.949 (0.002) | 0.571 (0.003) | |
| ECG strain | MID | NCV residuals | curve | attached | 40 | E(Y | X) | 0.901 (0.003) | 0.937 (0.002) | 0.932 (0.002) | 0.562 (0.004) | |
| ECG strain | MID | REML residuals | curve | attached | 40 | E(Y | X) | 0.902 (0.003) | 0.936 (0.003) | 0.932 (0.003) | 0.564 (0.004) | |
| ECG strain | MID | beat differences (own scale) | curve | attached | 40 | E(Y | X) | 0.926 (0.003) | 0.945 (0.002) | 0.941 (0.002) | 0.559 (0.004) | |
| ECG strain | MID | beat differences (variance-matched) | curve | attached | 40 | E(Y | X) | 0.890 (0.006) | 0.943 (0.002) | 0.941 (0.003) | 0.554 (0.004) | |
| ECG strain | NCV | NCV residuals | curve | attached | 40 | E(Y | X) | 0.908 (0.003) | 0.943 (0.002) | 0.933 (0.002) | 0.562 (0.004) | |
| ECG strain | NCV | REML residuals | curve | attached | 40 | E(Y | X) | 0.908 (0.003) | 0.942 (0.003) | 0.932 (0.003) | 0.564 (0.004) | |
| ECG strain | NCV | beat differences (own scale) | curve | attached | 40 | E(Y | X) | 0.924 (0.003) | 0.946 (0.002) | 0.941 (0.002) | 0.559 (0.004) | |
| ECG strain | NCV | beat differences (variance-matched) | curve | attached | 40 | E(Y | X) | 0.896 (0.005) | 0.947 (0.002) | 0.941 (0.003) | 0.554 (0.004) | |
| ECG strain | REML | NCV residuals | curve | attached | 40 | E(Y | X) | 0.887 (0.003) | 0.930 (0.002) | 0.931 (0.002) | 0.562 (0.004) | |
| ECG strain | REML | REML residuals | curve | attached | 40 | E(Y | X) | 0.885 (0.003) | 0.930 (0.003) | 0.931 (0.003) | 0.565 (0.004) | |
| ECG strain | REML | beat differences (own scale) | curve | attached | 40 | E(Y | X) | 0.924 (0.003) | 0.941 (0.002) | 0.940 (0.002) | 0.566 (0.004) | |
| ECG strain | REML | beat differences (variance-matched) | curve | attached | 40 | E(Y | X) | 0.876 (0.006) | 0.938 (0.002) | 0.940 (0.002) | 0.558 (0.004) | |
| AF trial | MID | NCV residuals | curve | attached | 118 | beta(s,t) | 0.788 (0.012) | 0.915 (0.006) | 0.939 (0.005) | 0.665 (0.008) | |
| AF trial | MID | REML residuals | curve | attached | 118 | beta(s,t) | 0.838 (0.008) | 0.792 (0.012) | 0.915 (0.006) | 0.938 (0.004) | 0.669 (0.008) |
| AF trial | NCV | NCV residuals | curve | attached | 118 | beta(s,t) | 0.789 (0.013) | 0.925 (0.005) | 0.941 (0.004) | 0.671 (0.009) | |
| AF trial | NCV | REML residuals | curve | attached | 118 | beta(s,t) | 0.792 (0.013) | 0.926 (0.005) | 0.942 (0.004) | 0.678 (0.008) | |
| AF trial | REML | NCV residuals | curve | attached | 118 | beta(s,t) | 0.783 (0.012) | 0.907 (0.006) | 0.933 (0.005) | 0.653 (0.008) | |
| AF trial | REML | REML residuals | curve | attached | 118 | beta(s,t) | 0.790 (0.011) | 0.908 (0.006) | 0.934 (0.004) | 0.660 (0.008) | |
| AF trial | MID | NCV residuals | curve | attached | 40 | beta(s,t) | 0.596 (0.018) | 0.883 (0.007) | 0.920 (0.006) | 0.620 (0.010) | |
| AF trial | MID | REML residuals | curve | attached | 40 | beta(s,t) | 0.589 (0.018) | 0.883 (0.007) | 0.922 (0.006) | 0.625 (0.010) | |
| AF trial | NCV | NCV residuals | curve | attached | 40 | beta(s,t) | 0.640 (0.016) | 0.904 (0.006) | 0.925 (0.006) | 0.628 (0.011) | |
| AF trial | NCV | REML residuals | curve | attached | 40 | beta(s,t) | 0.649 (0.016) | 0.906 (0.006) | 0.926 (0.006) | 0.631 (0.011) | |
| AF trial | REML | NCV residuals | curve | attached | 40 | beta(s,t) | 0.570 (0.019) | 0.868 (0.008) | 0.913 (0.006) | 0.609 (0.010) | |
| AF trial | REML | REML residuals | curve | attached | 40 | beta(s,t) | 0.572 (0.018) | 0.865 (0.008) | 0.916 (0.006) | 0.614 (0.010) | |
| AF trial | MID | NCV residuals | curve | attached | 118 | E(Y | X) | 0.898 (0.004) | 0.936 (0.003) | 0.945 (0.002) | 0.679 (0.004) | |
| AF trial | MID | REML residuals | curve | attached | 118 | E(Y | X) | 0.870 (0.004) | 0.899 (0.005) | 0.937 (0.003) | 0.947 (0.002) | 0.681 (0.004) |
| AF trial | NCV | NCV residuals | curve | attached | 118 | E(Y | X) | 0.891 (0.005) | 0.936 (0.003) | 0.945 (0.003) | 0.678 (0.004) | |
| AF trial | NCV | REML residuals | curve | attached | 118 | E(Y | X) | 0.893 (0.005) | 0.937 (0.003) | 0.947 (0.002) | 0.681 (0.004) | |
| AF trial | REML | NCV residuals | curve | attached | 118 | E(Y | X) | 0.900 (0.004) | 0.935 (0.003) | 0.945 (0.002) | 0.679 (0.004) | |
| AF trial | REML | REML residuals | curve | attached | 118 | E(Y | X) | 0.903 (0.004) | 0.936 (0.003) | 0.946 (0.002) | 0.681 (0.004) | |
| AF trial | MID | NCV residuals | curve | attached | 40 | E(Y | X) | 0.849 (0.006) | 0.926 (0.004) | 0.933 (0.004) | 0.644 (0.006) | |
| AF trial | MID | REML residuals | curve | attached | 40 | E(Y | X) | 0.849 (0.006) | 0.927 (0.004) | 0.934 (0.004) | 0.648 (0.006) | |
| AF trial | NCV | NCV residuals | curve | attached | 40 | E(Y | X) | 0.853 (0.006) | 0.930 (0.004) | 0.934 (0.004) | 0.645 (0.006) | |
| AF trial | NCV | REML residuals | curve | attached | 40 | E(Y | X) | 0.856 (0.005) | 0.931 (0.003) | 0.935 (0.004) | 0.647 (0.006) | |
| AF trial | REML | NCV residuals | curve | attached | 40 | E(Y | X) | 0.845 (0.006) | 0.924 (0.004) | 0.932 (0.004) | 0.644 (0.006) | |
| AF trial | REML | REML residuals | curve | attached | 40 | E(Y | X) | 0.847 (0.006) | 0.925 (0.004) | 0.934 (0.004) | 0.648 (0.005) | |
| running | MID | NCV residuals | curve | attached | 90 | beta(s,t) | 0.747 (0.010) | 0.888 (0.006) | 0.919 (0.006) | 0.518 (0.009) | |
| running | MID | REML residuals | curve | attached | 90 | beta(s,t) | 0.728 (0.006) | 0.738 (0.011) | 0.893 (0.006) | 0.922 (0.005) | 0.515 (0.009) |
| running | NCV | NCV residuals | curve | attached | 90 | beta(s,t) | 0.811 (0.005) | 0.875 (0.006) | 0.948 (0.004) | 0.922 (0.005) | 0.528 (0.010) |
| running | NCV | NCV residuals | curve | detached | 90 | beta(s,t) | 0.844 (0.010) | 0.947 (0.005) | 0.926 (0.006) | 0.506 (0.011) | |
| running | NCV | REML residuals | curve | attached | 90 | beta(s,t) | 0.870 (0.006) | 0.945 (0.004) | 0.922 (0.005) | 0.521 (0.010) | |
| running | REML | NCV residuals | curve | attached | 90 | beta(s,t) | 0.589 (0.019) | 0.838 (0.009) | 0.915 (0.007) | 0.479 (0.010) | |
| running | REML | REML residuals | curve | attached | 90 | beta(s,t) | 0.548 (0.011) | 0.586 (0.020) | 0.841 (0.010) | 0.916 (0.007) | 0.481 (0.009) |
| running | REML | REML residuals | curve | detached | 90 | beta(s,t) | 0.505 (0.018) | 0.850 (0.009) | 0.919 (0.006) | 0.468 (0.008) | |
| running | MID | NCV residuals | curve | attached | 40 | beta(s,t) | 0.636 (0.015) | 0.880 (0.007) | 0.903 (0.007) | 0.452 (0.010) | |
| running | MID | REML residuals | curve | attached | 40 | beta(s,t) | 0.638 (0.014) | 0.889 (0.006) | 0.912 (0.007) | 0.467 (0.010) | |
| running | NCV | NCV residuals | curve | attached | 40 | beta(s,t) | 0.760 (0.014) | 0.927 (0.006) | 0.910 (0.007) | 0.463 (0.011) | |
| running | NCV | NCV residuals | curve | detached | 40 | beta(s,t) | 0.747 (0.013) | 0.929 (0.005) | 0.926 (0.006) | 0.453 (0.011) | |
| running | NCV | REML residuals | curve | attached | 40 | beta(s,t) | 0.767 (0.013) | 0.928 (0.005) | 0.914 (0.006) | 0.463 (0.011) | |
| running | REML | NCV residuals | curve | attached | 40 | beta(s,t) | 0.432 (0.015) | 0.823 (0.010) | 0.888 (0.009) | 0.414 (0.010) | |
| running | REML | REML residuals | curve | attached | 40 | beta(s,t) | 0.427 (0.015) | 0.829 (0.011) | 0.887 (0.009) | 0.421 (0.010) | |
| running | REML | REML residuals | curve | detached | 40 | beta(s,t) | 0.412 (0.014) | 0.836 (0.009) | 0.909 (0.007) | 0.422 (0.009) | |
| running | MID | NCV residuals | curve | attached | 90 | E(Y | X) | 0.907 (0.003) | 0.933 (0.003) | 0.933 (0.003) | 0.569 (0.005) | |
| running | MID | REML residuals | curve | attached | 90 | E(Y | X) | 0.787 (0.004) | 0.902 (0.003) | 0.934 (0.003) | 0.934 (0.003) | 0.575 (0.004) |
| running | NCV | NCV residuals | curve | attached | 90 | E(Y | X) | 0.793 (0.004) | 0.919 (0.003) | 0.943 (0.003) | 0.930 (0.003) | 0.565 (0.005) |
| running | NCV | NCV residuals | curve | detached | 90 | E(Y | X) | 0.917 (0.004) | 0.947 (0.003) | 0.936 (0.003) | 0.577 (0.005) | |
| running | NCV | REML residuals | curve | attached | 90 | E(Y | X) | 0.917 (0.003) | 0.943 (0.003) | 0.931 (0.003) | 0.571 (0.005) | |
| running | REML | NCV residuals | curve | attached | 90 | E(Y | X) | 0.867 (0.005) | 0.925 (0.003) | 0.935 (0.003) | 0.564 (0.005) | |
| running | REML | REML residuals | curve | attached | 90 | E(Y | X) | 0.754 (0.004) | 0.861 (0.005) | 0.925 (0.003) | 0.935 (0.003) | 0.570 (0.004) |
| running | REML | REML residuals | curve | detached | 90 | E(Y | X) | 0.841 (0.006) | 0.932 (0.003) | 0.939 (0.003) | 0.582 (0.005) | |
| running | MID | NCV residuals | curve | attached | 40 | E(Y | X) | 0.871 (0.005) | 0.930 (0.003) | 0.922 (0.004) | 0.527 (0.006) | |
| running | MID | REML residuals | curve | attached | 40 | E(Y | X) | 0.871 (0.005) | 0.932 (0.003) | 0.923 (0.004) | 0.531 (0.006) | |
| running | NCV | NCV residuals | curve | attached | 40 | E(Y | X) | 0.885 (0.005) | 0.937 (0.003) | 0.921 (0.004) | 0.522 (0.006) | |
| running | NCV | NCV residuals | curve | detached | 40 | E(Y | X) | 0.888 (0.005) | 0.941 (0.004) | 0.926 (0.004) | 0.533 (0.006) | |
| running | NCV | REML residuals | curve | attached | 40 | E(Y | X) | 0.886 (0.005) | 0.938 (0.003) | 0.921 (0.004) | 0.527 (0.006) | |
| running | REML | NCV residuals | curve | attached | 40 | E(Y | X) | 0.841 (0.005) | 0.924 (0.004) | 0.922 (0.004) | 0.520 (0.006) | |
| running | REML | REML residuals | curve | attached | 40 | E(Y | X) | 0.837 (0.006) | 0.925 (0.004) | 0.921 (0.004) | 0.522 (0.005) | |
| running | REML | REML residuals | curve | detached | 40 | E(Y | X) | 0.839 (0.005) | 0.929 (0.004) | 0.928 (0.004) | 0.536 (0.005) | |
| DTI | MID | NCV residuals | curve | attached | 92 | beta(s,t) | 0.885 (0.005) | 0.940 (0.003) | 0.931 (0.003) | 0.636 (0.005) | |
| DTI | MID | REML residuals | curve | attached | 92 | beta(s,t) | 0.858 (0.007) | 0.881 (0.005) | 0.936 (0.003) | 0.929 (0.003) | 0.643 (0.005) |
| DTI | NCV | NCV residuals | curve | attached | 92 | beta(s,t) | 0.902 (0.005) | 0.957 (0.003) | 0.931 (0.003) | 0.639 (0.006) | |
| DTI | NCV | REML residuals | curve | attached | 92 | beta(s,t) | 0.900 (0.005) | 0.956 (0.003) | 0.930 (0.003) | 0.645 (0.006) | |
| DTI | REML | NCV residuals | curve | attached | 92 | beta(s,t) | 0.754 (0.008) | 0.886 (0.005) | 0.922 (0.004) | 0.613 (0.005) | |
| DTI | REML | REML residuals | curve | attached | 92 | beta(s,t) | 0.752 (0.008) | 0.882 (0.005) | 0.919 (0.004) | 0.615 (0.005) | |
| DTI | MID | NCV residuals | curve | attached | 40 | beta(s,t) | 0.787 (0.010) | 0.919 (0.004) | 0.919 (0.003) | 0.620 (0.006) | |
| DTI | MID | REML residuals | curve | attached | 40 | beta(s,t) | 0.799 (0.009) | 0.921 (0.004) | 0.922 (0.003) | 0.631 (0.006) | |
| DTI | NCV | NCV residuals | curve | attached | 40 | beta(s,t) | 0.803 (0.012) | 0.933 (0.005) | 0.921 (0.003) | 0.626 (0.007) | |
| DTI | NCV | REML residuals | curve | attached | 40 | beta(s,t) | 0.801 (0.013) | 0.929 (0.005) | 0.922 (0.004) | 0.635 (0.007) | |
| DTI | REML | NCV residuals | curve | attached | 40 | beta(s,t) | 0.671 (0.011) | 0.874 (0.006) | 0.909 (0.004) | 0.599 (0.006) | |
| DTI | REML | REML residuals | curve | attached | 40 | beta(s,t) | 0.679 (0.010) | 0.873 (0.005) | 0.909 (0.004) | 0.604 (0.006) | |
| DTI | MID | NCV residuals | curve | attached | 92 | E(Y | X) | 0.914 (0.003) | 0.941 (0.002) | 0.933 (0.002) | 0.644 (0.004) | |
| DTI | MID | REML residuals | curve | attached | 92 | E(Y | X) | 0.902 (0.004) | 0.911 (0.003) | 0.937 (0.002) | 0.932 (0.002) | 0.651 (0.004) |
| DTI | NCV | NCV residuals | curve | attached | 92 | E(Y | X) | 0.921 (0.003) | 0.951 (0.002) | 0.933 (0.002) | 0.644 (0.004) | |
| DTI | NCV | REML residuals | curve | attached | 92 | E(Y | X) | 0.919 (0.003) | 0.948 (0.002) | 0.931 (0.002) | 0.652 (0.004) | |
| DTI | REML | NCV residuals | curve | attached | 92 | E(Y | X) | 0.889 (0.003) | 0.927 (0.002) | 0.934 (0.002) | 0.640 (0.004) | |
| DTI | REML | REML residuals | curve | attached | 92 | E(Y | X) | 0.886 (0.003) | 0.925 (0.002) | 0.932 (0.002) | 0.644 (0.004) | |
| DTI | MID | NCV residuals | curve | attached | 40 | E(Y | X) | 0.867 (0.004) | 0.926 (0.003) | 0.918 (0.003) | 0.623 (0.004) | |
| DTI | MID | REML residuals | curve | attached | 40 | E(Y | X) | 0.870 (0.004) | 0.925 (0.003) | 0.919 (0.003) | 0.633 (0.004) | |
| DTI | NCV | NCV residuals | curve | attached | 40 | E(Y | X) | 0.877 (0.005) | 0.934 (0.003) | 0.918 (0.003) | 0.625 (0.004) | |
| DTI | NCV | REML residuals | curve | attached | 40 | E(Y | X) | 0.874 (0.005) | 0.931 (0.003) | 0.917 (0.003) | 0.634 (0.004) | |
| DTI | REML | NCV residuals | curve | attached | 40 | E(Y | X) | 0.844 (0.005) | 0.919 (0.003) | 0.919 (0.003) | 0.620 (0.004) | |
| DTI | REML | REML residuals | curve | attached | 40 | E(Y | X) | 0.847 (0.004) | 0.918 (0.003) | 0.919 (0.003) | 0.628 (0.004) | |
| gait | MID | NCV residuals | curve | attached | 138 | beta(s,t) | 0.812 (0.011) | 0.923 (0.004) | 0.940 (0.004) | 0.516 (0.007) | |
| gait | MID | REML residuals | curve | attached | 138 | beta(s,t) | 0.914 (0.004) | 0.810 (0.011) | 0.922 (0.004) | 0.940 (0.004) | 0.517 (0.007) |
| gait | NCV | NCV residuals | curve | attached | 138 | beta(s,t) | 0.832 (0.010) | 0.937 (0.004) | 0.940 (0.004) | 0.520 (0.007) | |
| gait | NCV | REML residuals | curve | attached | 138 | beta(s,t) | 0.831 (0.010) | 0.937 (0.004) | 0.940 (0.005) | 0.521 (0.007) | |
| gait | REML | NCV residuals | curve | attached | 138 | beta(s,t) | 0.804 (0.011) | 0.914 (0.004) | 0.939 (0.004) | 0.512 (0.006) | |
| gait | REML | REML residuals | curve | attached | 138 | beta(s,t) | 0.804 (0.011) | 0.914 (0.004) | 0.938 (0.004) | 0.514 (0.006) | |
| gait | MID | NCV residuals | curve | attached | 40 | beta(s,t) | 0.626 (0.014) | 0.904 (0.004) | 0.936 (0.004) | 0.469 (0.006) | |
| gait | MID | REML residuals | curve | attached | 40 | beta(s,t) | 0.625 (0.013) | 0.905 (0.004) | 0.936 (0.004) | 0.470 (0.006) | |
| gait | NCV | NCV residuals | curve | attached | 40 | beta(s,t) | 0.672 (0.014) | 0.915 (0.004) | 0.935 (0.004) | 0.470 (0.007) | |
| gait | NCV | REML residuals | curve | attached | 40 | beta(s,t) | 0.673 (0.014) | 0.917 (0.004) | 0.936 (0.004) | 0.471 (0.007) | |
| gait | REML | NCV residuals | curve | attached | 40 | beta(s,t) | 0.595 (0.013) | 0.893 (0.004) | 0.934 (0.004) | 0.465 (0.006) | |
| gait | REML | REML residuals | curve | attached | 40 | beta(s,t) | 0.595 (0.013) | 0.893 (0.004) | 0.935 (0.004) | 0.466 (0.006) | |
| gait | MID | NCV residuals | curve | attached | 138 | E(Y | X) | 0.922 (0.003) | 0.944 (0.002) | 0.943 (0.003) | 0.538 (0.004) | |
| gait | MID | REML residuals | curve | attached | 138 | E(Y | X) | 0.927 (0.003) | 0.922 (0.003) | 0.944 (0.002) | 0.943 (0.003) | 0.539 (0.004) |
| gait | NCV | NCV residuals | curve | attached | 138 | E(Y | X) | 0.924 (0.003) | 0.947 (0.002) | 0.943 (0.003) | 0.538 (0.004) | |
| gait | NCV | REML residuals | curve | attached | 138 | E(Y | X) | 0.924 (0.003) | 0.947 (0.002) | 0.943 (0.003) | 0.539 (0.004) | |
| gait | REML | NCV residuals | curve | attached | 138 | E(Y | X) | 0.919 (0.003) | 0.942 (0.002) | 0.943 (0.003) | 0.538 (0.004) | |
| gait | REML | REML residuals | curve | attached | 138 | E(Y | X) | 0.919 (0.003) | 0.942 (0.002) | 0.943 (0.003) | 0.539 (0.004) | |
| gait | MID | NCV residuals | curve | attached | 40 | E(Y | X) | 0.863 (0.006) | 0.939 (0.003) | 0.936 (0.003) | 0.503 (0.004) | |
| gait | MID | REML residuals | curve | attached | 40 | E(Y | X) | 0.864 (0.006) | 0.940 (0.003) | 0.936 (0.003) | 0.504 (0.004) | |
| gait | NCV | NCV residuals | curve | attached | 40 | E(Y | X) | 0.868 (0.006) | 0.940 (0.003) | 0.936 (0.003) | 0.503 (0.004) | |
| gait | NCV | REML residuals | curve | attached | 40 | E(Y | X) | 0.869 (0.006) | 0.941 (0.003) | 0.936 (0.003) | 0.503 (0.004) | |
| gait | REML | NCV residuals | curve | attached | 40 | E(Y | X) | 0.860 (0.006) | 0.937 (0.003) | 0.936 (0.003) | 0.503 (0.004) | |
| gait | REML | REML residuals | curve | attached | 40 | E(Y | X) | 0.860 (0.006) | 0.938 (0.003) | 0.936 (0.003) | 0.504 (0.004) | |
| ECG 8-lead | MID | NCV residuals | curve | attached | 100 | beta(s,t) | 0.744 (0.012) | 0.945 (0.003) | 0.944 (0.003) | 0.636 (0.006) | |
| ECG 8-lead | MID | REML residuals | curve | attached | 100 | beta(s,t) | 0.720 (0.009) | 0.722 (0.013) | 0.937 (0.004) | 0.944 (0.003) | 0.644 (0.006) |
| ECG 8-lead | NCV | NCV residuals | curve | attached | 100 | beta(s,t) | 0.831 (0.010) | 0.974 (0.002) | 0.944 (0.003) | 0.640 (0.006) | |
| ECG 8-lead | NCV | REML residuals | curve | attached | 100 | beta(s,t) | 0.824 (0.010) | 0.970 (0.003) | 0.945 (0.003) | 0.652 (0.006) | |
| ECG 8-lead | REML | NCV residuals | curve | attached | 100 | beta(s,t) | 0.658 (0.014) | 0.900 (0.006) | 0.938 (0.003) | 0.624 (0.005) | |
| ECG 8-lead | REML | REML residuals | curve | attached | 100 | beta(s,t) | 0.653 (0.015) | 0.895 (0.006) | 0.938 (0.004) | 0.633 (0.005) | |
| ECG 8-lead | MID | NCV residuals | curve | attached | 40 | beta(s,t) | 0.637 (0.014) | 0.940 (0.003) | 0.937 (0.004) | 0.639 (0.006) | |
| ECG 8-lead | MID | REML residuals | curve | attached | 40 | beta(s,t) | 0.646 (0.014) | 0.936 (0.004) | 0.937 (0.004) | 0.641 (0.006) | |
| ECG 8-lead | NCV | NCV residuals | curve | attached | 40 | beta(s,t) | 0.770 (0.012) | 0.965 (0.003) | 0.936 (0.004) | 0.640 (0.007) | |
| ECG 8-lead | NCV | REML residuals | curve | attached | 40 | beta(s,t) | 0.774 (0.012) | 0.965 (0.003) | 0.937 (0.004) | 0.645 (0.007) | |
| ECG 8-lead | REML | NCV residuals | curve | attached | 40 | beta(s,t) | 0.550 (0.015) | 0.900 (0.004) | 0.934 (0.004) | 0.625 (0.005) | |
| ECG 8-lead | REML | REML residuals | curve | attached | 40 | beta(s,t) | 0.562 (0.016) | 0.895 (0.005) | 0.934 (0.004) | 0.631 (0.005) | |
| ECG 8-lead | MID | NCV residuals | curve | attached | 100 | E(Y | X) | 0.905 (0.004) | 0.960 (0.002) | 0.953 (0.002) | 0.751 (0.004) | |
| ECG 8-lead | MID | REML residuals | curve | attached | 100 | E(Y | X) | 0.924 (0.003) | 0.898 (0.005) | 0.957 (0.002) | 0.953 (0.002) | 0.757 (0.004) |
| ECG 8-lead | NCV | NCV residuals | curve | attached | 100 | E(Y | X) | 0.907 (0.005) | 0.965 (0.002) | 0.953 (0.002) | 0.753 (0.004) | |
| ECG 8-lead | NCV | REML residuals | curve | attached | 100 | E(Y | X) | 0.906 (0.005) | 0.964 (0.002) | 0.952 (0.002) | 0.760 (0.004) | |
| ECG 8-lead | REML | NCV residuals | curve | attached | 100 | E(Y | X) | 0.882 (0.005) | 0.948 (0.002) | 0.951 (0.002) | 0.746 (0.004) | |
| ECG 8-lead | REML | REML residuals | curve | attached | 100 | E(Y | X) | 0.878 (0.005) | 0.948 (0.002) | 0.951 (0.002) | 0.752 (0.004) | |
| ECG 8-lead | MID | NCV residuals | curve | attached | 40 | E(Y | X) | 0.863 (0.005) | 0.951 (0.002) | 0.944 (0.002) | 0.729 (0.004) | |
| ECG 8-lead | MID | REML residuals | curve | attached | 40 | E(Y | X) | 0.867 (0.005) | 0.950 (0.003) | 0.944 (0.003) | 0.734 (0.005) | |
| ECG 8-lead | NCV | NCV residuals | curve | attached | 40 | E(Y | X) | 0.881 (0.005) | 0.959 (0.002) | 0.944 (0.003) | 0.730 (0.005) | |
| ECG 8-lead | NCV | REML residuals | curve | attached | 40 | E(Y | X) | 0.884 (0.005) | 0.957 (0.002) | 0.944 (0.003) | 0.736 (0.005) | |
| ECG 8-lead | REML | NCV residuals | curve | attached | 40 | E(Y | X) | 0.846 (0.006) | 0.945 (0.002) | 0.944 (0.003) | 0.724 (0.004) | |
| ECG 8-lead | REML | REML residuals | curve | attached | 40 | E(Y | X) | 0.849 (0.006) | 0.943 (0.003) | 0.944 (0.003) | 0.730 (0.004) | |
| ocean | MID | NCV residuals | curve | attached | 116 | beta(s,t) | 0.715 (0.006) | 0.915 (0.004) | 0.921 (0.006) | 0.407 (0.008) | |
| ocean | MID | REML residuals | curve | attached | 116 | beta(s,t) | 0.854 (0.003) | 0.717 (0.005) | 0.912 (0.004) | 0.927 (0.006) | 0.418 (0.008) |
| ocean | NCV | NCV residuals | curve | attached | 116 | beta(s,t) | 0.846 (0.008) | 0.963 (0.003) | 0.920 (0.006) | 0.406 (0.008) | |
| ocean | NCV | NCV residuals | block | attached | 116 | beta(s,t) | 0.855 (0.007) | 0.966 (0.002) | 0.934 (0.006) | 0.420 (0.009) | |
| ocean | NCV | REML residuals | curve | attached | 116 | beta(s,t) | 0.856 (0.007) | 0.965 (0.002) | 0.927 (0.006) | 0.422 (0.009) | |
| ocean | REML | NCV residuals | curve | attached | 116 | beta(s,t) | 0.554 (0.012) | 0.869 (0.007) | 0.924 (0.006) | 0.404 (0.008) | |
| ocean | REML | REML residuals | curve | attached | 116 | beta(s,t) | 0.561 (0.013) | 0.872 (0.006) | 0.930 (0.006) | 0.417 (0.008) | |
| ocean | REML | REML residuals | block | attached | 116 | beta(s,t) | 0.564 (0.014) | 0.883 (0.006) | 0.935 (0.006) | 0.417 (0.007) | |
| ocean | MID | NCV residuals | curve | attached | 40 | beta(s,t) | 0.585 (0.014) | 0.914 (0.004) | 0.927 (0.005) | 0.376 (0.007) | |
| ocean | MID | REML residuals | curve | attached | 40 | beta(s,t) | 0.589 (0.014) | 0.914 (0.004) | 0.933 (0.005) | 0.387 (0.007) | |
| ocean | NCV | NCV residuals | curve | attached | 40 | beta(s,t) | 0.689 (0.012) | 0.944 (0.003) | 0.929 (0.005) | 0.375 (0.008) | |
| ocean | NCV | REML residuals | curve | attached | 40 | beta(s,t) | 0.700 (0.012) | 0.946 (0.003) | 0.934 (0.005) | 0.388 (0.008) | |
| ocean | REML | NCV residuals | curve | attached | 40 | beta(s,t) | 0.416 (0.012) | 0.860 (0.005) | 0.922 (0.005) | 0.375 (0.007) | |
| ocean | REML | REML residuals | curve | attached | 40 | beta(s,t) | 0.420 (0.012) | 0.858 (0.005) | 0.926 (0.005) | 0.383 (0.007) | |
| ocean | MID | NCV residuals | curve | attached | 116 | E(Y | X) | 0.911 (0.004) | 0.941 (0.003) | 0.934 (0.003) | 0.456 (0.004) | |
| ocean | MID | REML residuals | curve | attached | 116 | E(Y | X) | 0.932 (0.003) | 0.910 (0.004) | 0.938 (0.003) | 0.935 (0.003) | 0.459 (0.004) |
| ocean | NCV | NCV residuals | curve | attached | 116 | E(Y | X) | 0.912 (0.004) | 0.945 (0.003) | 0.933 (0.003) | 0.452 (0.004) | |
| ocean | NCV | NCV residuals | block | attached | 116 | E(Y | X) | 0.902 (0.004) | 0.937 (0.003) | 0.931 (0.003) | 0.448 (0.004) | |
| ocean | NCV | REML residuals | curve | attached | 116 | E(Y | X) | 0.911 (0.004) | 0.944 (0.003) | 0.934 (0.003) | 0.454 (0.004) | |
| ocean | REML | NCV residuals | curve | attached | 116 | E(Y | X) | 0.883 (0.004) | 0.933 (0.003) | 0.936 (0.003) | 0.458 (0.004) | |
| ocean | REML | REML residuals | curve | attached | 116 | E(Y | X) | 0.883 (0.004) | 0.932 (0.003) | 0.937 (0.003) | 0.461 (0.004) | |
| ocean | REML | REML residuals | block | attached | 116 | E(Y | X) | 0.875 (0.004) | 0.929 (0.003) | 0.933 (0.003) | 0.456 (0.004) | |
| ocean | MID | NCV residuals | curve | attached | 40 | E(Y | X) | 0.882 (0.004) | 0.942 (0.002) | 0.931 (0.003) | 0.427 (0.004) | |
| ocean | MID | REML residuals | curve | attached | 40 | E(Y | X) | 0.882 (0.004) | 0.942 (0.002) | 0.933 (0.003) | 0.432 (0.004) | |
| ocean | NCV | NCV residuals | curve | attached | 40 | E(Y | X) | 0.885 (0.004) | 0.946 (0.003) | 0.931 (0.003) | 0.425 (0.004) | |
| ocean | NCV | REML residuals | curve | attached | 40 | E(Y | X) | 0.886 (0.004) | 0.945 (0.003) | 0.933 (0.003) | 0.430 (0.004) | |
| ocean | REML | NCV residuals | curve | attached | 40 | E(Y | X) | 0.867 (0.004) | 0.940 (0.002) | 0.932 (0.003) | 0.428 (0.004) | |
| ocean | REML | REML residuals | curve | attached | 40 | E(Y | X) | 0.868 (0.004) | 0.939 (0.002) | 0.934 (0.003) | 0.432 (0.004) | |
| weather | MID | NCV residuals | curve | attached | 73 | beta(s,t) | 0.697 (0.022) | 0.867 (0.013) | 0.913 (0.009) | 0.357 (0.010) | |
| weather | MID | REML residuals | curve | attached | 73 | beta(s,t) | 0.438 (0.013) | 0.712 (0.020) | 0.871 (0.012) | 0.905 (0.010) | 0.360 (0.010) |
| weather | NCV | NCV residuals | curve | attached | 73 | beta(s,t) | 0.536 (0.030) | 0.808 (0.020) | 0.840 (0.020) | 0.347 (0.013) | |
| weather | NCV | NCV residuals | block | attached | 73 | beta(s,t) | 0.540 (0.031) | 0.831 (0.017) | 0.896 (0.016) | 0.369 (0.011) | |
| weather | NCV | REML residuals | curve | attached | 73 | beta(s,t) | 0.582 (0.029) | 0.820 (0.019) | 0.855 (0.018) | 0.352 (0.012) | |
| weather | REML | NCV residuals | curve | attached | 73 | beta(s,t) | 0.551 (0.020) | 0.801 (0.015) | 0.893 (0.011) | 0.319 (0.010) | |
| weather | REML | REML residuals | curve | attached | 73 | beta(s,t) | 0.569 (0.018) | 0.804 (0.014) | 0.887 (0.012) | 0.315 (0.010) | |
| weather | REML | REML residuals | block | attached | 73 | beta(s,t) | 0.605 (0.018) | 0.814 (0.014) | 0.911 (0.009) | 0.350 (0.013) | |
| weather | MID | NCV residuals | curve | attached | 40 | beta(s,t) | 0.454 (0.030) | 0.795 (0.019) | 0.866 (0.018) | 0.310 (0.012) | |
| weather | MID | REML residuals | curve | attached | 40 | beta(s,t) | 0.453 (0.030) | 0.806 (0.018) | 0.863 (0.017) | 0.327 (0.012) | |
| weather | NCV | NCV residuals | curve | attached | 40 | beta(s,t) | 0.357 (0.028) | 0.741 (0.023) | 0.798 (0.022) | 0.298 (0.013) | |
| weather | NCV | REML residuals | curve | attached | 40 | beta(s,t) | 0.351 (0.028) | 0.730 (0.023) | 0.769 (0.023) | 0.305 (0.014) | |
| weather | REML | NCV residuals | curve | attached | 40 | beta(s,t) | 0.386 (0.026) | 0.769 (0.019) | 0.861 (0.018) | 0.288 (0.011) | |
| weather | REML | REML residuals | curve | attached | 40 | beta(s,t) | 0.366 (0.025) | 0.776 (0.018) | 0.853 (0.017) | 0.295 (0.011) | |
| weather | MID | NCV residuals | curve | attached | 73 | E(Y | X) | 0.875 (0.009) | 0.923 (0.006) | 0.929 (0.005) | 0.542 (0.008) | |
| weather | MID | REML residuals | curve | attached | 73 | E(Y | X) | 0.675 (0.009) | 0.880 (0.008) | 0.923 (0.006) | 0.922 (0.005) | 0.542 (0.008) |
| weather | NCV | NCV residuals | curve | attached | 73 | E(Y | X) | 0.867 (0.007) | 0.925 (0.005) | 0.913 (0.006) | 0.543 (0.008) | |
| weather | NCV | NCV residuals | block | attached | 73 | E(Y | X) | 0.820 (0.008) | 0.899 (0.005) | 0.897 (0.004) | 0.489 (0.006) | |
| weather | NCV | REML residuals | curve | attached | 73 | E(Y | X) | 0.870 (0.007) | 0.923 (0.005) | 0.912 (0.005) | 0.543 (0.008) | |
| weather | REML | NCV residuals | curve | attached | 73 | E(Y | X) | 0.856 (0.010) | 0.912 (0.007) | 0.937 (0.005) | 0.536 (0.009) | |
| weather | REML | REML residuals | curve | attached | 73 | E(Y | X) | 0.865 (0.009) | 0.912 (0.006) | 0.929 (0.006) | 0.536 (0.008) | |
| weather | REML | REML residuals | block | attached | 73 | E(Y | X) | 0.842 (0.008) | 0.898 (0.005) | 0.914 (0.004) | 0.522 (0.007) | |
| weather | MID | NCV residuals | curve | attached | 40 | E(Y | X) | 0.818 (0.009) | 0.909 (0.006) | 0.918 (0.006) | 0.496 (0.008) | |
| weather | MID | REML residuals | curve | attached | 40 | E(Y | X) | 0.820 (0.009) | 0.909 (0.007) | 0.912 (0.006) | 0.504 (0.008) | |
| weather | NCV | NCV residuals | curve | attached | 40 | E(Y | X) | 0.859 (0.007) | 0.923 (0.005) | 0.915 (0.005) | 0.505 (0.008) | |
| weather | NCV | REML residuals | curve | attached | 40 | E(Y | X) | 0.855 (0.007) | 0.919 (0.006) | 0.907 (0.006) | 0.511 (0.008) | |
| weather | REML | NCV residuals | curve | attached | 40 | E(Y | X) | 0.809 (0.009) | 0.908 (0.007) | 0.925 (0.006) | 0.496 (0.008) | |
| weather | REML | REML residuals | curve | attached | 40 | E(Y | X) | 0.806 (0.009) | 0.910 (0.006) | 0.921 (0.006) | 0.502 (0.008) | |
| electricity | MID | NCV residuals | curve | attached | 102 | beta(s,t) | 0.604 (0.015) | 0.945 (0.006) | 0.934 (0.006) | 0.462 (0.013) | |
| electricity | MID | REML residuals | curve | attached | 102 | beta(s,t) | 0.677 (0.013) | 0.598 (0.016) | 0.942 (0.005) | 0.939 (0.005) | 0.457 (0.013) |
| electricity | NCV | NCV residuals | curve | attached | 102 | beta(s,t) | 0.912 (0.011) | 0.985 (0.004) | 0.935 (0.007) | 0.458 (0.014) | |
| electricity | NCV | NCV residuals | block | attached | 102 | beta(s,t) | 0.915 (0.011) | 0.991 (0.002) | 0.935 (0.006) | 0.452 (0.013) | |
| electricity | NCV | REML residuals | curve | attached | 102 | beta(s,t) | 0.901 (0.012) | 0.980 (0.004) | 0.937 (0.006) | 0.456 (0.013) | |
| electricity | REML | NCV residuals | curve | attached | 102 | beta(s,t) | 0.316 (0.018) | 0.800 (0.013) | 0.897 (0.010) | 0.370 (0.010) | |
| electricity | REML | REML residuals | curve | attached | 102 | beta(s,t) | 0.332 (0.019) | 0.817 (0.011) | 0.920 (0.009) | 0.387 (0.011) | |
| electricity | REML | REML residuals | block | attached | 102 | beta(s,t) | 0.308 (0.020) | 0.799 (0.014) | 0.905 (0.011) | 0.390 (0.010) | |
| electricity | MID | NCV residuals | curve | attached | 40 | beta(s,t) | 0.672 (0.011) | 0.967 (0.004) | 0.936 (0.007) | 0.385 (0.012) | |
| electricity | MID | REML residuals | curve | attached | 40 | beta(s,t) | 0.656 (0.012) | 0.959 (0.005) | 0.932 (0.007) | 0.378 (0.011) | |
| electricity | NCV | NCV residuals | curve | attached | 40 | beta(s,t) | 0.869 (0.016) | 0.982 (0.004) | 0.936 (0.006) | 0.381 (0.013) | |
| electricity | NCV | REML residuals | curve | attached | 40 | beta(s,t) | 0.880 (0.014) | 0.982 (0.004) | 0.938 (0.006) | 0.386 (0.012) | |
| electricity | REML | NCV residuals | curve | attached | 40 | beta(s,t) | 0.252 (0.016) | 0.752 (0.014) | 0.861 (0.012) | 0.312 (0.009) | |
| electricity | REML | REML residuals | curve | attached | 40 | beta(s,t) | 0.239 (0.015) | 0.778 (0.012) | 0.878 (0.010) | 0.319 (0.009) | |
| electricity | MID | NCV residuals | curve | attached | 102 | E(Y | X) | 0.921 (0.006) | 0.961 (0.004) | 0.927 (0.005) | 0.506 (0.007) | |
| electricity | MID | REML residuals | curve | attached | 102 | E(Y | X) | 0.844 (0.007) | 0.918 (0.006) | 0.959 (0.004) | 0.927 (0.005) | 0.504 (0.006) |
| electricity | NCV | NCV residuals | curve | attached | 102 | E(Y | X) | 0.935 (0.006) | 0.968 (0.004) | 0.925 (0.005) | 0.501 (0.007) | |
| electricity | NCV | NCV residuals | block | attached | 102 | E(Y | X) | 0.925 (0.007) | 0.966 (0.004) | 0.926 (0.004) | 0.490 (0.007) | |
| electricity | NCV | REML residuals | curve | attached | 102 | E(Y | X) | 0.936 (0.006) | 0.967 (0.004) | 0.923 (0.005) | 0.498 (0.006) | |
| electricity | REML | NCV residuals | curve | attached | 102 | E(Y | X) | 0.794 (0.008) | 0.913 (0.006) | 0.929 (0.005) | 0.487 (0.007) | |
| electricity | REML | REML residuals | curve | attached | 102 | E(Y | X) | 0.784 (0.007) | 0.914 (0.005) | 0.933 (0.005) | 0.491 (0.006) | |
| electricity | REML | REML residuals | block | attached | 102 | E(Y | X) | 0.773 (0.008) | 0.911 (0.005) | 0.935 (0.005) | 0.493 (0.007) | |
| electricity | MID | NCV residuals | curve | attached | 40 | E(Y | X) | 0.905 (0.007) | 0.953 (0.005) | 0.924 (0.005) | 0.451 (0.007) | |
| electricity | MID | REML residuals | curve | attached | 40 | E(Y | X) | 0.908 (0.007) | 0.953 (0.005) | 0.923 (0.005) | 0.455 (0.008) | |
| electricity | NCV | NCV residuals | curve | attached | 40 | E(Y | X) | 0.910 (0.008) | 0.957 (0.005) | 0.923 (0.005) | 0.446 (0.007) | |
| electricity | NCV | REML residuals | curve | attached | 40 | E(Y | X) | 0.915 (0.007) | 0.957 (0.005) | 0.923 (0.005) | 0.452 (0.008) | |
| electricity | REML | NCV residuals | curve | attached | 40 | E(Y | X) | 0.816 (0.007) | 0.911 (0.006) | 0.917 (0.005) | 0.433 (0.006) | |
| electricity | REML | REML residuals | curve | attached | 40 | E(Y | X) | 0.808 (0.007) | 0.915 (0.005) | 0.924 (0.005) | 0.438 (0.007) |
16.6 S5 Plasmode MSE ratios per cell
| dataset | truth | residual | flip | error | G | alpha | beta | gamma | mean |
|---|---|---|---|---|---|---|---|---|---|
| ECG strain | MID | NCV residuals | curve | attached | 78 | 1.04 [1.00, 1.09] | 0.89 [0.86, 0.92] | 0.53 [0.48, 0.58] | 0.96 [0.94, 0.98] |
| ECG strain | MID | REML residuals | curve | attached | 78 | 1.04 [1.00, 1.09] | 0.93 [0.90, 0.96] | 0.60 [0.56, 0.65] | 0.98 [0.96, 1.00] |
| ECG strain | MID | beat differences (own scale) | curve | attached | 78 | 1.01 [0.99, 1.04] | 0.94 [0.93, 0.96] | 0.72 [0.68, 0.75] | 0.97 [0.97, 0.98] |
| ECG strain | MID | beat differences (variance-matched) | curve | attached | 78 | 1.02 [0.98, 1.07] | 0.86 [0.83, 0.88] | 0.70 [0.66, 0.73] | 0.95 [0.94, 0.97] |
| ECG strain | NCV | NCV residuals | curve | attached | 78 | 0.97 [0.93, 1.02] | 0.73 [0.71, 0.76] | 0.53 [0.48, 0.58] | 0.87 [0.86, 0.89] |
| ECG strain | NCV | REML residuals | curve | attached | 78 | 1.00 [0.95, 1.04] | 0.78 [0.76, 0.80] | 0.60 [0.56, 0.64] | 0.90 [0.89, 0.92] |
| ECG strain | NCV | beat differences (own scale) | curve | attached | 78 | 1.00 [0.97, 1.04] | 0.89 [0.87, 0.91] | 0.70 [0.67, 0.73] | 0.96 [0.95, 0.98] |
| ECG strain | NCV | beat differences (variance-matched) | curve | attached | 78 | 0.95 [0.90, 1.00] | 0.73 [0.71, 0.75] | 0.69 [0.65, 0.72] | 0.89 [0.87, 0.90] |
| ECG strain | REML | NCV residuals | curve | attached | 78 | 1.11 [1.07, 1.16] | 0.95 [0.92, 0.98] | 1.27 [1.19, 1.36] | 1.04 [1.01, 1.06] |
| ECG strain | REML | REML residuals | curve | attached | 78 | 1.10 [1.06, 1.14] | 0.98 [0.95, 1.01] | 1.24 [1.17, 1.33] | 1.04 [1.02, 1.06] |
| ECG strain | REML | beat differences (own scale) | curve | attached | 78 | 1.01 [0.99, 1.03] | 0.97 [0.96, 0.98] | 1.05 [1.04, 1.06] | 0.99 [0.99, 1.00] |
| ECG strain | REML | beat differences (variance-matched) | curve | attached | 78 | 1.07 [1.02, 1.12] | 0.91 [0.88, 0.93] | 1.20 [1.13, 1.27] | 0.99 [0.98, 1.01] |
| ECG strain | MID | NCV residuals | curve | attached | 40 | 0.95 [0.90, 1.01] | 0.74 [0.71, 0.77] | 0.54 [0.49, 0.59] | 0.87 [0.85, 0.90] |
| ECG strain | MID | REML residuals | curve | attached | 40 | 0.96 [0.91, 1.01] | 0.78 [0.75, 0.81] | 0.58 [0.53, 0.64] | 0.89 [0.87, 0.92] |
| ECG strain | MID | beat differences (own scale) | curve | attached | 40 | 0.99 [0.94, 1.05] | 0.89 [0.86, 0.91] | 0.69 [0.64, 0.73] | 0.95 [0.93, 0.97] |
| ECG strain | MID | beat differences (variance-matched) | curve | attached | 40 | 0.94 [0.86, 1.03] | 0.69 [0.66, 0.73] | 0.68 [0.63, 0.73] | 0.93 [0.88, 0.97] |
| ECG strain | NCV | NCV residuals | curve | attached | 40 | 0.83 [0.79, 0.89] | 0.59 [0.56, 0.62] | 0.53 [0.47, 0.58] | 0.77 [0.75, 0.80] |
| ECG strain | NCV | REML residuals | curve | attached | 40 | 0.85 [0.80, 0.91] | 0.62 [0.59, 0.65] | 0.59 [0.53, 0.64] | 0.80 [0.77, 0.82] |
| ECG strain | NCV | beat differences (own scale) | curve | attached | 40 | 0.93 [0.87, 0.99] | 0.77 [0.75, 0.80] | 0.68 [0.64, 0.72] | 0.89 [0.87, 0.91] |
| ECG strain | NCV | beat differences (variance-matched) | curve | attached | 40 | 0.82 [0.75, 0.90] | 0.56 [0.53, 0.59] | 0.67 [0.62, 0.71] | 0.81 [0.77, 0.85] |
| ECG strain | REML | NCV residuals | curve | attached | 40 | 1.05 [1.00, 1.11] | 0.81 [0.78, 0.84] | 1.05 [0.99, 1.11] | 0.95 [0.92, 0.97] |
| ECG strain | REML | REML residuals | curve | attached | 40 | 1.07 [1.02, 1.14] | 0.85 [0.82, 0.88] | 1.13 [1.07, 1.20] | 0.97 [0.95, 1.00] |
| ECG strain | REML | beat differences (own scale) | curve | attached | 40 | 1.03 [0.98, 1.09] | 0.94 [0.91, 0.97] | 1.14 [1.09, 1.21] | 1.00 [0.97, 1.02] |
| ECG strain | REML | beat differences (variance-matched) | curve | attached | 40 | 1.07 [0.98, 1.17] | 0.75 [0.71, 0.79] | 1.27 [1.20, 1.36] | 1.01 [0.97, 1.06] |
| AF trial | MID | NCV residuals | curve | attached | 118 | 1.01 [0.97, 1.06] | 0.25 [0.19, 0.33] | 0.96 [0.93, 1.00] | 0.94 [0.91, 0.97] |
| AF trial | MID | REML residuals | curve | attached | 118 | 1.01 [0.97, 1.06] | 0.29 [0.22, 0.38] | 0.97 [0.93, 1.01] | 0.95 [0.92, 0.98] |
| AF trial | NCV | NCV residuals | curve | attached | 118 | 0.98 [0.94, 1.02] | 0.22 [0.16, 0.30] | 0.92 [0.88, 0.95] | 0.91 [0.88, 0.94] |
| AF trial | NCV | REML residuals | curve | attached | 118 | 0.98 [0.94, 1.02] | 0.25 [0.19, 0.34] | 0.92 [0.89, 0.96] | 0.91 [0.88, 0.94] |
| AF trial | REML | NCV residuals | curve | attached | 118 | 1.04 [1.00, 1.09] | 0.30 [0.24, 0.39] | 1.00 [0.97, 1.04] | 0.96 [0.93, 0.99] |
| AF trial | REML | REML residuals | curve | attached | 118 | 1.03 [0.99, 1.07] | 0.34 [0.27, 0.43] | 1.01 [0.97, 1.05] | 0.96 [0.94, 0.99] |
| AF trial | MID | NCV residuals | curve | attached | 40 | 0.97 [0.90, 1.04] | 0.17 [0.10, 0.28] | 0.84 [0.78, 0.89] | 0.78 [0.74, 0.82] |
| AF trial | MID | REML residuals | curve | attached | 40 | 0.97 [0.91, 1.05] | 0.19 [0.11, 0.31] | 0.84 [0.79, 0.90] | 0.79 [0.75, 0.83] |
| AF trial | NCV | NCV residuals | curve | attached | 40 | 0.93 [0.87, 1.01] | 0.12 [0.07, 0.21] | 0.81 [0.76, 0.86] | 0.72 [0.68, 0.76] |
| AF trial | NCV | REML residuals | curve | attached | 40 | 0.93 [0.88, 1.00] | 0.13 [0.08, 0.23] | 0.81 [0.76, 0.87] | 0.73 [0.70, 0.77] |
| AF trial | REML | NCV residuals | curve | attached | 40 | 0.99 [0.92, 1.06] | 0.19 [0.12, 0.29] | 0.86 [0.81, 0.92] | 0.81 [0.77, 0.85] |
| AF trial | REML | REML residuals | curve | attached | 40 | 1.01 [0.95, 1.10] | 0.20 [0.13, 0.31] | 0.87 [0.82, 0.93] | 0.82 [0.79, 0.86] |
| running | MID | NCV residuals | curve | attached | 90 | 1.16 [1.10, 1.23] | 0.55 [0.50, 0.61] | 1.08 [1.04, 1.12] | 0.99 [0.98, 1.01] |
| running | MID | REML residuals | curve | attached | 90 | 1.16 [1.09, 1.23] | 0.52 [0.46, 0.58] | 1.10 [1.06, 1.15] | 0.98 [0.97, 1.00] |
| running | NCV | NCV residuals | curve | attached | 90 | 1.10 [1.04, 1.17] | 0.24 [0.19, 0.30] | 1.03 [0.99, 1.08] | 0.91 [0.89, 0.92] |
| running | NCV | NCV residuals | curve | detached | 90 | 1.06 [1.00, 1.13] | 0.15 [0.12, 0.20] | 1.09 [1.03, 1.17] | 0.87 [0.85, 0.90] |
| running | NCV | REML residuals | curve | attached | 90 | 1.11 [1.05, 1.18] | 0.24 [0.19, 0.30] | 1.06 [1.02, 1.12] | 0.90 [0.88, 0.92] |
| running | REML | NCV residuals | curve | attached | 90 | 1.19 [1.12, 1.26] | 1.55 [1.40, 1.71] | 1.09 [1.05, 1.13] | 1.15 [1.12, 1.18] |
| running | REML | REML residuals | curve | attached | 90 | 1.21 [1.13, 1.29] | 1.50 [1.37, 1.65] | 1.12 [1.07, 1.17] | 1.16 [1.13, 1.19] |
| running | REML | REML residuals | curve | detached | 90 | 1.14 [1.07, 1.23] | 1.35 [1.22, 1.51] | 1.17 [1.09, 1.25] | 1.20 [1.17, 1.24] |
| running | MID | NCV residuals | curve | attached | 40 | 1.09 [1.02, 1.18] | 0.31 [0.25, 0.38] | 0.99 [0.92, 1.05] | 0.92 [0.88, 0.96] |
| running | MID | REML residuals | curve | attached | 40 | 1.11 [1.03, 1.21] | 0.29 [0.24, 0.36] | 0.99 [0.92, 1.06] | 0.91 [0.87, 0.95] |
| running | NCV | NCV residuals | curve | attached | 40 | 1.00 [0.93, 1.09] | 0.15 [0.11, 0.20] | 0.92 [0.87, 0.99] | 0.83 [0.79, 0.86] |
| running | NCV | NCV residuals | curve | detached | 40 | 1.00 [0.92, 1.08] | 0.14 [0.10, 0.19] | 0.82 [0.77, 0.89] | 0.77 [0.74, 0.81] |
| running | NCV | REML residuals | curve | attached | 40 | 1.02 [0.95, 1.10] | 0.15 [0.11, 0.20] | 0.93 [0.87, 0.99] | 0.82 [0.79, 0.86] |
| running | REML | NCV residuals | curve | attached | 40 | 1.13 [1.05, 1.24] | 0.82 [0.72, 0.93] | 0.99 [0.93, 1.06] | 1.01 [0.97, 1.06] |
| running | REML | REML residuals | curve | attached | 40 | 1.15 [1.06, 1.27] | 0.80 [0.71, 0.91] | 1.00 [0.93, 1.08] | 1.01 [0.97, 1.05] |
| running | REML | REML residuals | curve | detached | 40 | 1.10 [1.02, 1.20] | 0.73 [0.65, 0.83] | 0.97 [0.90, 1.06] | 0.97 [0.94, 1.01] |
| DTI | MID | NCV residuals | curve | attached | 92 | 0.91 [0.88, 0.95] | 0.54 [0.51, 0.57] | 0.75 [0.70, 0.79] | 0.86 [0.85, 0.87] |
| DTI | MID | REML residuals | curve | attached | 92 | 0.93 [0.89, 0.97] | 0.61 [0.57, 0.64] | 0.74 [0.70, 0.78] | 0.88 [0.86, 0.89] |
| DTI | NCV | NCV residuals | curve | attached | 92 | 0.87 [0.83, 0.91] | 0.37 [0.35, 0.41] | 0.66 [0.61, 0.71] | 0.74 [0.72, 0.76] |
| DTI | NCV | REML residuals | curve | attached | 92 | 0.89 [0.85, 0.93] | 0.42 [0.39, 0.45] | 0.66 [0.61, 0.70] | 0.76 [0.74, 0.77] |
| DTI | REML | NCV residuals | curve | attached | 92 | 0.98 [0.94, 1.01] | 1.13 [1.08, 1.18] | 0.88 [0.83, 0.93] | 1.00 [0.98, 1.01] |
| DTI | REML | REML residuals | curve | attached | 92 | 0.99 [0.96, 1.03] | 1.20 [1.16, 1.25] | 0.89 [0.84, 0.93] | 1.01 [0.99, 1.02] |
| DTI | MID | NCV residuals | curve | attached | 40 | 0.91 [0.86, 0.97] | 0.49 [0.46, 0.53] | 0.68 [0.62, 0.74] | 0.81 [0.78, 0.84] |
| DTI | MID | REML residuals | curve | attached | 40 | 0.91 [0.86, 0.95] | 0.53 [0.49, 0.57] | 0.68 [0.63, 0.73] | 0.82 [0.80, 0.85] |
| DTI | NCV | NCV residuals | curve | attached | 40 | 0.92 [0.86, 0.98] | 0.37 [0.33, 0.40] | 0.63 [0.57, 0.68] | 0.71 [0.68, 0.73] |
| DTI | NCV | REML residuals | curve | attached | 40 | 0.91 [0.85, 0.97] | 0.40 [0.37, 0.44] | 0.64 [0.59, 0.70] | 0.73 [0.71, 0.76] |
| DTI | REML | NCV residuals | curve | attached | 40 | 0.91 [0.86, 0.97] | 0.84 [0.80, 0.89] | 0.75 [0.70, 0.81] | 0.87 [0.85, 0.90] |
| DTI | REML | REML residuals | curve | attached | 40 | 0.92 [0.86, 0.97] | 0.90 [0.85, 0.94] | 0.75 [0.70, 0.81] | 0.88 [0.86, 0.91] |
| gait | MID | NCV residuals | curve | attached | 138 | 0.94 [0.91, 0.97] | 0.55 [0.49, 0.62] | 1.02 [0.99, 1.06] | 0.97 [0.95, 0.99] |
| gait | MID | REML residuals | curve | attached | 138 | 0.94 [0.91, 0.97] | 0.55 [0.49, 0.62] | 1.01 [0.98, 1.05] | 0.97 [0.95, 0.99] |
| gait | NCV | NCV residuals | curve | attached | 138 | 0.92 [0.88, 0.96] | 0.43 [0.35, 0.50] | 1.02 [0.98, 1.06] | 0.93 [0.91, 0.95] |
| gait | NCV | REML residuals | curve | attached | 138 | 0.92 [0.88, 0.95] | 0.43 [0.35, 0.50] | 1.01 [0.98, 1.05] | 0.93 [0.91, 0.95] |
| gait | REML | NCV residuals | curve | attached | 138 | 0.95 [0.92, 0.98] | 0.63 [0.56, 0.69] | 1.03 [1.00, 1.07] | 0.99 [0.97, 1.01] |
| gait | REML | REML residuals | curve | attached | 138 | 0.95 [0.93, 0.98] | 0.62 [0.56, 0.69] | 1.02 [0.99, 1.05] | 0.99 [0.97, 1.01] |
| gait | MID | NCV residuals | curve | attached | 40 | 0.86 [0.80, 0.92] | 0.26 [0.20, 0.33] | 0.88 [0.83, 0.95] | 0.91 [0.87, 0.94] |
| gait | MID | REML residuals | curve | attached | 40 | 0.86 [0.80, 0.92] | 0.25 [0.19, 0.32] | 0.89 [0.83, 0.96] | 0.91 [0.87, 0.94] |
| gait | NCV | NCV residuals | curve | attached | 40 | 0.85 [0.79, 0.91] | 0.22 [0.16, 0.29] | 0.87 [0.81, 0.93] | 0.88 [0.85, 0.92] |
| gait | NCV | REML residuals | curve | attached | 40 | 0.85 [0.79, 0.91] | 0.22 [0.15, 0.29] | 0.87 [0.81, 0.94] | 0.88 [0.85, 0.92] |
| gait | REML | NCV residuals | curve | attached | 40 | 0.86 [0.80, 0.92] | 0.29 [0.23, 0.36] | 0.89 [0.83, 0.96] | 0.92 [0.89, 0.95] |
| gait | REML | REML residuals | curve | attached | 40 | 0.87 [0.81, 0.93] | 0.29 [0.23, 0.36] | 0.89 [0.84, 0.97] | 0.92 [0.89, 0.96] |
| ECG 8-lead | MID | NCV residuals | curve | attached | 100 | 0.74 [0.70, 0.80] | 0.17 [0.11, 0.24] | 1.03 [0.98, 1.08] | 0.75 [0.73, 0.78] |
| ECG 8-lead | MID | REML residuals | curve | attached | 100 | 0.76 [0.71, 0.81] | 0.19 [0.13, 0.25] | 1.06 [1.02, 1.12] | 0.78 [0.75, 0.81] |
| ECG 8-lead | NCV | NCV residuals | curve | attached | 100 | 0.62 [0.56, 0.68] | 0.12 [0.06, 0.20] | 1.01 [0.95, 1.07] | 0.67 [0.64, 0.70] |
| ECG 8-lead | NCV | REML residuals | curve | attached | 100 | 0.63 [0.57, 0.68] | 0.10 [0.07, 0.15] | 1.02 [0.97, 1.09] | 0.69 [0.66, 0.72] |
| ECG 8-lead | REML | NCV residuals | curve | attached | 100 | 0.97 [0.91, 1.04] | 0.31 [0.26, 0.38] | 1.03 [0.99, 1.08] | 0.85 [0.83, 0.88] |
| ECG 8-lead | REML | REML residuals | curve | attached | 100 | 0.98 [0.92, 1.06] | 0.32 [0.27, 0.38] | 1.05 [1.01, 1.10] | 0.88 [0.85, 0.91] |
| ECG 8-lead | MID | NCV residuals | curve | attached | 40 | 0.67 [0.61, 0.74] | 0.10 [0.07, 0.13] | 1.00 [0.93, 1.07] | 0.66 [0.62, 0.69] |
| ECG 8-lead | MID | REML residuals | curve | attached | 40 | 0.68 [0.62, 0.75] | 0.12 [0.08, 0.16] | 0.98 [0.91, 1.05] | 0.68 [0.64, 0.71] |
| ECG 8-lead | NCV | NCV residuals | curve | attached | 40 | 0.57 [0.51, 0.64] | 0.08 [0.05, 0.12] | 0.93 [0.87, 1.01] | 0.58 [0.55, 0.61] |
| ECG 8-lead | NCV | REML residuals | curve | attached | 40 | 0.58 [0.52, 0.64] | 0.09 [0.06, 0.13] | 0.93 [0.86, 1.00] | 0.60 [0.56, 0.63] |
| ECG 8-lead | REML | NCV residuals | curve | attached | 40 | 0.83 [0.76, 0.92] | 0.14 [0.11, 0.18] | 0.99 [0.92, 1.05] | 0.71 [0.68, 0.74] |
| ECG 8-lead | REML | REML residuals | curve | attached | 40 | 0.83 [0.76, 0.91] | 0.18 [0.14, 0.23] | 1.00 [0.93, 1.07] | 0.74 [0.71, 0.77] |
| ocean | MID | NCV residuals | curve | attached | 116 | 1.09 [1.04, 1.13] | 0.33 [0.28, 0.38] | 1.08 [1.00, 1.18] | 0.87 [0.86, 0.89] |
| ocean | MID | REML residuals | curve | attached | 116 | 1.12 [1.07, 1.17] | 0.37 [0.31, 0.43] | 1.08 [1.00, 1.18] | 0.89 [0.88, 0.91] |
| ocean | NCV | NCV residuals | curve | attached | 116 | 1.01 [0.95, 1.07] | 0.07 [0.05, 0.11] | 0.94 [0.88, 1.02] | 0.79 [0.77, 0.81] |
| ocean | NCV | NCV residuals | block | attached | 116 | 1.00 [0.95, 1.05] | 0.05 [0.04, 0.06] | 0.84 [0.79, 0.89] | 0.81 [0.79, 0.83] |
| ocean | NCV | REML residuals | curve | attached | 116 | 1.04 [0.98, 1.09] | 0.06 [0.05, 0.07] | 0.94 [0.88, 1.02] | 0.80 [0.78, 0.82] |
| ocean | REML | NCV residuals | curve | attached | 116 | 1.16 [1.10, 1.23] | 1.71 [1.55, 1.90] | 1.29 [1.18, 1.45] | 1.12 [1.10, 1.15] |
| ocean | REML | REML residuals | curve | attached | 116 | 1.18 [1.12, 1.25] | 1.90 [1.72, 2.13] | 1.29 [1.17, 1.42] | 1.15 [1.12, 1.18] |
| ocean | REML | REML residuals | block | attached | 116 | 1.16 [1.09, 1.24] | 1.82 [1.62, 2.07] | 1.22 [1.11, 1.33] | 1.15 [1.12, 1.18] |
| ocean | MID | NCV residuals | curve | attached | 40 | 0.90 [0.83, 0.97] | 0.13 [0.11, 0.15] | 0.76 [0.70, 0.83] | 0.76 [0.74, 0.79] |
| ocean | MID | REML residuals | curve | attached | 40 | 0.93 [0.86, 1.00] | 0.13 [0.11, 0.15] | 0.76 [0.70, 0.82] | 0.77 [0.75, 0.80] |
| ocean | NCV | NCV residuals | curve | attached | 40 | 0.85 [0.79, 0.92] | 0.04 [0.03, 0.06] | 0.64 [0.59, 0.69] | 0.70 [0.67, 0.73] |
| ocean | NCV | REML residuals | curve | attached | 40 | 0.88 [0.81, 0.95] | 0.04 [0.03, 0.05] | 0.65 [0.60, 0.69] | 0.71 [0.68, 0.74] |
| ocean | REML | NCV residuals | curve | attached | 40 | 0.94 [0.87, 1.02] | 0.64 [0.57, 0.72] | 0.86 [0.78, 0.95] | 0.85 [0.82, 0.88] |
| ocean | REML | REML residuals | curve | attached | 40 | 0.97 [0.90, 1.05] | 0.69 [0.61, 0.78] | 0.86 [0.79, 0.95] | 0.87 [0.83, 0.90] |
| weather | MID | NCV residuals | curve | attached | 73 | 0.63 [0.57, 0.71] | 0.44 [0.39, 0.49] | 0.64 [0.56, 0.75] | 0.98 [0.93, 1.03] |
| weather | MID | REML residuals | curve | attached | 73 | 0.68 [0.61, 0.76] | 0.44 [0.39, 0.49] | 0.71 [0.62, 0.83] | 0.96 [0.92, 1.01] |
| weather | NCV | NCV residuals | curve | attached | 73 | 0.35 [0.31, 0.40] | 0.33 [0.29, 0.38] | 0.24 [0.20, 0.28] | 0.83 [0.80, 0.87] |
| weather | NCV | NCV residuals | block | attached | 73 | 0.32 [0.28, 0.37] | 0.41 [0.36, 0.48] | 0.20 [0.17, 0.24] | 0.86 [0.83, 0.90] |
| weather | NCV | REML residuals | curve | attached | 73 | 0.42 [0.36, 0.47] | 0.33 [0.29, 0.38] | 0.34 [0.28, 0.40] | 0.84 [0.80, 0.87] |
| weather | REML | NCV residuals | curve | attached | 73 | 0.95 [0.84, 1.08] | 0.68 [0.62, 0.75] | 1.28 [1.13, 1.50] | 1.10 [1.05, 1.16] |
| weather | REML | REML residuals | curve | attached | 73 | 1.00 [0.88, 1.13] | 0.66 [0.60, 0.73] | 1.32 [1.16, 1.57] | 1.07 [1.02, 1.11] |
| weather | REML | REML residuals | block | attached | 73 | 0.82 [0.71, 0.94] | 0.83 [0.75, 0.90] | 1.16 [1.02, 1.31] | 1.10 [1.06, 1.14] |
| weather | MID | NCV residuals | curve | attached | 40 | 0.50 [0.41, 0.61] | 0.32 [0.28, 0.38] | 0.61 [0.49, 0.76] | 0.90 [0.85, 0.95] |
| weather | MID | REML residuals | curve | attached | 40 | 0.59 [0.49, 0.69] | 0.35 [0.31, 0.41] | 0.75 [0.61, 0.94] | 0.93 [0.88, 0.98] |
| weather | NCV | NCV residuals | curve | attached | 40 | 0.30 [0.24, 0.38] | 0.21 [0.17, 0.25] | 0.32 [0.25, 0.40] | 0.71 [0.67, 0.75] |
| weather | NCV | REML residuals | curve | attached | 40 | 0.38 [0.31, 0.46] | 0.22 [0.19, 0.27] | 0.39 [0.31, 0.49] | 0.74 [0.70, 0.78] |
| weather | REML | NCV residuals | curve | attached | 40 | 0.64 [0.52, 0.78] | 0.44 [0.39, 0.51] | 1.00 [0.80, 1.23] | 0.97 [0.91, 1.02] |
| weather | REML | REML residuals | curve | attached | 40 | 0.77 [0.63, 0.91] | 0.45 [0.39, 0.51] | 1.18 [0.97, 1.44] | 0.99 [0.94, 1.05] |
| electricity | MID | NCV residuals | curve | attached | 102 | 0.77 [0.73, 0.82] | 0.05 [0.03, 0.06] | 0.96 [0.92, 1.01] | 0.57 [0.54, 0.60] |
| electricity | MID | REML residuals | curve | attached | 102 | 0.76 [0.72, 0.80] | 0.06 [0.04, 0.08] | 0.99 [0.94, 1.04] | 0.60 [0.57, 0.62] |
| electricity | NCV | NCV residuals | curve | attached | 102 | 0.74 [0.70, 0.78] | 0.03 [0.02, 0.05] | 0.97 [0.92, 1.02] | 0.51 [0.48, 0.54] |
| electricity | NCV | NCV residuals | block | attached | 102 | 0.74 [0.70, 0.78] | 0.03 [0.02, 0.04] | 0.95 [0.90, 0.99] | 0.52 [0.49, 0.55] |
| electricity | NCV | REML residuals | curve | attached | 102 | 0.73 [0.69, 0.77] | 0.05 [0.03, 0.06] | 1.00 [0.95, 1.05] | 0.53 [0.50, 0.56] |
| electricity | REML | NCV residuals | curve | attached | 102 | 0.95 [0.88, 1.01] | 2.03 [1.81, 2.29] | 0.99 [0.95, 1.02] | 1.15 [1.10, 1.19] |
| electricity | REML | REML residuals | curve | attached | 102 | 0.96 [0.90, 1.02] | 2.46 [2.21, 2.76] | 0.96 [0.93, 0.99] | 1.23 [1.18, 1.28] |
| electricity | REML | REML residuals | block | attached | 102 | 0.94 [0.89, 1.00] | 2.38 [2.12, 2.71] | 0.97 [0.94, 1.01] | 1.26 [1.21, 1.32] |
| electricity | MID | NCV residuals | curve | attached | 40 | 0.76 [0.71, 0.82] | 0.03 [0.02, 0.04] | 0.88 [0.81, 0.95] | 0.49 [0.45, 0.53] |
| electricity | MID | REML residuals | curve | attached | 40 | 0.78 [0.73, 0.85] | 0.04 [0.02, 0.06] | 0.89 [0.82, 0.96] | 0.49 [0.45, 0.53] |
| electricity | NCV | NCV residuals | curve | attached | 40 | 0.75 [0.69, 0.81] | 0.02 [0.02, 0.04] | 0.89 [0.82, 0.96] | 0.46 [0.43, 0.50] |
| electricity | NCV | REML residuals | curve | attached | 40 | 0.76 [0.71, 0.82] | 0.04 [0.02, 0.06] | 0.89 [0.82, 0.96] | 0.46 [0.43, 0.50] |
| electricity | REML | NCV residuals | curve | attached | 40 | 0.87 [0.81, 0.94] | 0.84 [0.75, 0.94] | 0.87 [0.81, 0.94] | 0.76 [0.72, 0.80] |
| electricity | REML | REML residuals | curve | attached | 40 | 0.82 [0.76, 0.89] | 0.98 [0.88, 1.10] | 0.86 [0.79, 0.93] | 0.78 [0.74, 0.81] |
16.7 S7 Relative error and interval informativeness per cell
| cell | block | family | signal | error | G | rel_N | rel_R | hw_N | hw_R | det_N | det_R |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | core | gaussian | low | iid | 40 | 0.19 | 0.22 | 0.49 | 0.60 | 0.92 | 0.90 |
| 2 | core | gaussian | low | iid | 100 | 0.13 | 0.16 | 0.34 | 0.44 | 0.96 | 0.95 |
| 3 | core | gaussian | low | ou | 40 | 0.37 | 1.37 | 0.66 | 2.40 | 0.77 | 0.27 |
| 4 | core | gaussian | low | ou | 100 | 0.26 | 0.68 | 0.47 | 1.36 | 0.89 | 0.49 |
| 5 | core | gaussian | low | fpc | 40 | 0.38 | 1.39 | 0.67 | 2.69 | 0.76 | 0.25 |
| 6 | core | gaussian | low | fpc | 100 | 0.26 | 0.70 | 0.47 | 1.44 | 0.90 | 0.47 |
| 7 | core | gaussian | mid | iid | 40 | 0.12 | 0.15 | 0.29 | 0.40 | 0.97 | 0.96 |
| 8 | core | gaussian | mid | iid | 100 | 0.08 | 0.11 | 0.20 | 0.29 | 0.99 | 0.98 |
| 9 | core | gaussian | mid | ou | 40 | 0.21 | 0.66 | 0.42 | 1.24 | 0.92 | 0.54 |
| 10 | core | gaussian | mid | ou | 100 | 0.15 | 0.35 | 0.29 | 0.71 | 0.97 | 0.80 |
| 11 | core | gaussian | mid | fpc | 40 | 0.22 | 0.70 | 0.41 | 1.35 | 0.92 | 0.51 |
| 12 | core | gaussian | mid | fpc | 100 | 0.15 | 0.36 | 0.29 | 0.75 | 0.97 | 0.78 |
| 13 | core | gaussian | high | iid | 40 | 0.07 | 0.10 | 0.18 | 0.25 | 1.00 | 0.99 |
| 14 | core | gaussian | high | iid | 100 | 0.05 | 0.07 | 0.12 | 0.18 | 1.00 | 1.00 |
| 15 | core | gaussian | high | ou | 40 | 0.13 | 0.33 | 0.25 | 0.64 | 0.99 | 0.83 |
| 16 | core | gaussian | high | ou | 100 | 0.09 | 0.19 | 0.17 | 0.39 | 1.00 | 0.95 |
| 17 | core | gaussian | high | fpc | 40 | 0.13 | 0.35 | 0.26 | 0.68 | 0.99 | 0.80 |
| 18 | core | gaussian | high | fpc | 100 | 0.09 | 0.20 | 0.17 | 0.40 | 1.00 | 0.94 |
| 19 | core | poisson | mid | iid | 40 | 0.13 | 0.16 | 0.34 | 0.45 | 0.96 | 0.95 |
| 20 | core | poisson | mid | iid | 100 | 0.09 | 0.12 | 0.23 | 0.32 | 0.98 | 0.97 |
| 21 | core | poisson | mid | ou | 40 | 0.24 | 0.69 | 0.46 | 1.36 | 0.89 | 0.53 |
| 22 | core | poisson | mid | ou | 100 | 0.17 | 0.40 | 0.32 | 0.80 | 0.96 | 0.77 |
| 23 | core | poisson | mid | fpc | 40 | 0.24 | 0.76 | 0.46 | 1.48 | 0.90 | 0.49 |
| 24 | core | poisson | mid | fpc | 100 | 0.17 | 0.41 | 0.32 | 0.84 | 0.96 | 0.75 |
| 25 | core | poisson | high | iid | 40 | 0.10 | 0.13 | 0.24 | 0.33 | 0.98 | 0.97 |
| 26 | core | poisson | high | iid | 100 | 0.07 | 0.09 | 0.16 | 0.23 | 1.00 | 0.99 |
| 27 | core | poisson | high | ou | 40 | 0.17 | 0.42 | 0.32 | 0.85 | 0.95 | 0.77 |
| 28 | core | poisson | high | ou | 100 | 0.12 | 0.25 | 0.22 | 0.51 | 0.99 | 0.91 |
| 29 | core | poisson | high | fpc | 40 | 0.17 | 0.45 | 0.33 | 0.89 | 0.95 | 0.73 |
| 30 | core | poisson | high | fpc | 100 | 0.12 | 0.25 | 0.22 | 0.52 | 0.98 | 0.91 |
| 31 | core | binomial | mid | iid | 40 | 0.38 | 0.40 | 0.93 | 0.99 | 0.66 | 0.66 |
| 32 | core | binomial | mid | iid | 100 | 0.27 | 0.29 | 0.66 | 0.78 | 0.84 | 0.81 |
| 33 | core | binomial | mid | ou | 40 | 0.73 | 1.75 | 1.02 | 3.51 | 0.46 | 0.17 |
| 34 | core | binomial | mid | ou | 100 | 0.43 | 1.04 | 0.77 | 2.16 | 0.68 | 0.31 |
| 35 | core | binomial | mid | fpc | 40 | 0.74 | 2.22 | 1.04 | 4.16 | 0.40 | 0.15 |
| 36 | core | binomial | mid | fpc | 100 | 0.45 | 1.18 | 0.79 | 2.41 | 0.66 | 0.27 |
| 37 | core | binomial | high | iid | 40 | 0.25 | 0.28 | 0.61 | 0.74 | 0.86 | 0.83 |
| 38 | core | binomial | high | iid | 100 | 0.18 | 0.20 | 0.44 | 0.55 | 0.93 | 0.92 |
| 39 | core | binomial | high | ou | 40 | 0.40 | 0.97 | 0.71 | 1.99 | 0.75 | 0.36 |
| 40 | core | binomial | high | ou | 100 | 0.28 | 0.60 | 0.51 | 1.25 | 0.88 | 0.57 |
| 41 | core | binomial | high | fpc | 40 | 0.43 | 1.21 | 0.74 | 2.37 | 0.72 | 0.30 |
| 42 | core | binomial | high | fpc | 100 | 0.29 | 0.67 | 0.52 | 1.38 | 0.88 | 0.52 |
| 43 | warp | gaussian | high | warp | 40 | 0.15 | 0.28 | 0.30 | 0.59 | 0.96 | 0.84 |
| 44 | warp | gaussian | high | warp | 40 | 0.28 | 0.35 | 0.42 | 0.72 | 0.90 | 0.77 |
| 45 | warp | gaussian | high | warp | 100 | 0.11 | 0.19 | 0.21 | 0.40 | 0.99 | 0.95 |
| 46 | warp | gaussian | high | warp | 100 | 0.21 | 0.24 | 0.34 | 0.49 | 0.97 | 0.91 |
| 55 | dense_grid | gaussian | mid | iid | 100 | 0.05 | 0.07 | 0.12 | 0.18 | 1.00 | 1.00 |
| 56 | dense_grid | gaussian | mid | ou | 100 | 0.15 | 0.77 | 0.27 | 1.51 | 0.98 | 0.57 |
| 57 | dense_grid | gaussian | mid | fpc | 100 | 0.15 | 0.81 | 0.27 | 1.65 | 0.98 | 0.52 |
| 58 | dense_grid | poisson | mid | iid | 100 | 0.06 | 0.08 | 0.14 | 0.20 | 1.00 | 0.99 |
| 59 | dense_grid | poisson | mid | ou | 100 | 0.17 | 0.86 | 0.29 | 1.74 | 0.96 | 0.54 |
| 60 | dense_grid | poisson | mid | fpc | 100 | 0.17 | 0.92 | 0.30 | 1.88 | 0.96 | 0.49 |
| 61 | dense_grid | binomial | mid | iid | 100 | 0.17 | 0.19 | 0.41 | 0.52 | 0.94 | 0.93 |
| 62 | dense_grid | binomial | mid | ou | 100 | 0.42 | 2.64 | 0.68 | 5.32 | 0.75 | 0.11 |
| 63 | dense_grid | binomial | mid | fpc | 100 | 0.44 | 3.27 | 0.72 | 6.64 | 0.71 | 0.09 |
| 64 | rough_truth | gaussian | low | iid | 100 | 0.25 | 0.24 | 0.50 | 0.50 | 0.95 | 0.95 |
| 65 | rough_truth | gaussian | low | fpc | 100 | 0.39 | 0.74 | 0.54 | 1.47 | 0.81 | 0.42 |
| 66 | rough_truth | gaussian | mid | iid | 100 | 0.15 | 0.15 | 0.32 | 0.33 | 0.99 | 0.99 |
| 67 | rough_truth | gaussian | mid | fpc | 100 | 0.29 | 0.42 | 0.42 | 0.80 | 0.94 | 0.75 |
| 68 | rough_truth | gaussian | high | iid | 100 | 0.09 | 0.09 | 0.19 | 0.21 | 1.00 | 1.00 |
| 69 | rough_truth | gaussian | high | fpc | 100 | 0.18 | 0.26 | 0.31 | 0.48 | 0.99 | 0.95 |
| 70 | rough_truth | poisson | mid | iid | 100 | 0.18 | 0.17 | 0.37 | 0.37 | 0.99 | 0.99 |
| 71 | rough_truth | poisson | mid | fpc | 100 | 0.31 | 0.44 | 0.43 | 0.90 | 0.91 | 0.72 |
| 72 | rough_truth | poisson | high | iid | 100 | 0.12 | 0.12 | 0.26 | 0.27 | 0.99 | 0.99 |
| 73 | rough_truth | poisson | high | fpc | 100 | 0.23 | 0.30 | 0.36 | 0.61 | 0.97 | 0.90 |
| 74 | rough_truth | binomial | mid | iid | 100 | 0.41 | 0.40 | 0.76 | 0.80 | 0.73 | 0.74 |
| 75 | rough_truth | binomial | mid | fpc | 100 | 0.56 | 1.17 | 0.80 | 2.50 | 0.58 | 0.24 |
| 76 | rough_truth | binomial | high | iid | 100 | 0.31 | 0.30 | 0.61 | 0.60 | 0.88 | 0.89 |
| 77 | rough_truth | binomial | high | fpc | 100 | 0.42 | 0.70 | 0.57 | 1.46 | 0.78 | 0.47 |
| 87 | families | scat | mid | iid | 100 | 0.07 | 0.10 | 0.18 | 0.25 | 1.00 | 0.99 |
| 88 | families | scat | mid | fpc | 100 | 0.13 | 0.30 | 0.26 | 0.62 | 0.98 | 0.86 |
| 89 | families | betar | mid | iid | 100 | 0.08 | 0.11 | 0.20 | 0.28 | 0.99 | 0.98 |
| 90 | families | betar | mid | fpc | 100 | 0.15 | 0.34 | 0.28 | 0.72 | 0.97 | 0.80 |
| 91 | families | nb | mid | iid | 100 | 0.11 | 0.14 | 0.28 | 0.38 | 0.97 | 0.96 |
| 92 | families | nb | mid | fpc | 100 | 0.21 | 0.52 | 0.39 | 1.08 | 0.93 | 0.62 |
| 93 | families | nb | mid | iid | 100 | 0.12 | 0.17 | 0.25 | 0.40 | 0.98 | 0.96 |
| 94 | families | nb | mid | fpc | 100 | 0.22 | 0.92 | 0.37 | 1.81 | 0.93 | 0.46 |
| 95 | ar1_home | gaussian | mid | ou0 | 100 | 0.16 | 0.38 | 0.29 | 0.76 | 0.97 | 0.77 |
| 96 | oscillating | gaussian | mid | osc | 100 | 0.14 | 0.35 | 0.26 | 0.68 | 0.97 | 0.84 |
| 97 | oscillating | poisson | mid | osc | 100 | 0.16 | 0.37 | 0.28 | 0.75 | 0.96 | 0.81 |
| 98 | oscillating | binomial | mid | osc | 100 | 0.39 | 0.82 | 0.70 | 1.79 | 0.77 | 0.39 |
| 99 | warp_ar1 | gaussian | high | warp | 100 | 0.21 | 0.24 | 0.34 | 0.49 | 0.97 | 0.91 |
| 100 | heteroskedastic | gaussian | mid | fpc_raw | 40 | 0.23 | 0.73 | 0.41 | 1.39 | 0.91 | 0.50 |
| 101 | heteroskedastic | gaussian | mid | fpc_raw | 100 | 0.16 | 0.37 | 0.29 | 0.76 | 0.97 | 0.77 |
| 102 | heteroskedastic | gaussian | mid | fpc_het | 40 | 0.22 | 0.75 | 0.41 | 1.46 | 0.92 | 0.52 |
| 103 | heteroskedastic | gaussian | mid | fpc_het | 100 | 0.15 | 0.36 | 0.29 | 0.74 | 0.97 | 0.79 |
| 104 | heteroskedastic | gaussian | mid | fpc_raw_het | 40 | 0.22 | 0.74 | 0.41 | 1.39 | 0.92 | 0.52 |
| 105 | heteroskedastic | gaussian | mid | fpc_raw_het | 100 | 0.15 | 0.37 | 0.29 | 0.76 | 0.97 | 0.78 |
| 106 | lowrank_covariate | gaussian | mid | iid | 100 | 0.09 | 0.12 | 0.22 | 0.34 | 0.99 | 0.97 |
| 107 | lowrank_covariate | gaussian | mid | fpc | 100 | 0.15 | 0.51 | 0.29 | 1.06 | 0.96 | 0.63 |
| 108 | lowrank_covariate | poisson | mid | iid | 100 | 0.10 | 0.13 | 0.25 | 0.38 | 0.98 | 0.96 |
| 109 | lowrank_covariate | poisson | mid | fpc | 100 | 0.16 | 0.57 | 0.32 | 1.20 | 0.95 | 0.60 |
| 110 | lowrank_covariate | binomial | mid | iid | 100 | 0.27 | 0.29 | 0.70 | 0.84 | 0.81 | 0.78 |
| 111 | lowrank_covariate | binomial | mid | fpc | 100 | 0.44 | 1.50 | 0.78 | 3.13 | 0.68 | 0.22 |
| 112 | basis_size | gaussian | mid | iid | 100 | 0.10 | 0.14 | 0.28 | 0.44 | 0.98 | 0.96 |
| 113 | basis_size | gaussian | mid | iid | 100 | 0.11 | 0.16 | 0.37 | 0.60 | 0.97 | 0.91 |
| 114 | basis_size | gaussian | mid | fpc | 100 | 0.17 | 0.86 | 0.34 | 1.75 | 0.96 | 0.37 |
| 115 | basis_size | gaussian | mid | fpc | 100 | 0.18 | 1.80 | 0.39 | 3.78 | 0.95 | 0.12 |
| 116 | basis_size | poisson | mid | iid | 100 | 0.11 | 0.16 | 0.31 | 0.48 | 0.97 | 0.94 |
| 117 | basis_size | poisson | mid | iid | 100 | 0.12 | 0.18 | 0.40 | 0.66 | 0.95 | 0.88 |
| 118 | basis_size | poisson | mid | fpc | 100 | 0.18 | 0.96 | 0.37 | 1.94 | 0.95 | 0.35 |
| 119 | basis_size | poisson | mid | fpc | 100 | 0.19 | 1.88 | 0.42 | 3.97 | 0.93 | 0.12 |
| 120 | basis_size | binomial | mid | iid | 100 | 0.30 | 0.33 | 0.85 | 1.04 | 0.71 | 0.64 |
| 121 | basis_size | binomial | mid | iid | 100 | 0.32 | 0.35 | 1.04 | 1.30 | 0.56 | 0.47 |
| 122 | basis_size | binomial | mid | fpc | 100 | 0.48 | 2.64 | 0.87 | 5.69 | 0.59 | 0.08 |
| 123 | basis_size | binomial | mid | fpc | 100 | 0.49 | 5.83 | 0.96 | 12.44 | 0.51 | 0.04 |
16.8 S6 Files
| file |
|---|
| ablation-contrasts-summary.csv |
| ablation-coverage-wide.csv |
| ablation-se-sd.csv |
| comparators-cl2-arms.csv |
| comparators-coverage.csv |
| comparators-estimation-plasmode-beta.csv |
| comparators-estimation.csv |
| comparators-time.csv |
| computing-time.csv |
| cross-study-assessment.csv |
| cross-study-dataset-outcomes.csv |
| cross-study-synthetic-cells.csv |
| dti-fits.csv |
| dti-recipes.csv |
| lambda-ncv-plasmode-beta.csv |
| lambda-ncv-synthetic.csv |
| lambda-reml-plasmode-beta.csv |
| lambda-reml-synthetic-coverage.csv |
| lambda-reml-synthetic-width-score.csv |
| log-sp-spread-core.csv |
| metrics-plasmode-beta-per-dataset.csv |
| metrics-plasmode.csv |
| metrics-synthetic.csv |
| misspec-basis-contrasts.csv |
| misspec-bias.csv |
| misspec-bump-region.csv |
| misspec-coverage-wide.csv |
| misspec-mse.csv |
| misspec-truths.csv |
| ncv-intervals-combination-synthetic.csv |
| ncv-intervals-misspec.csv |
| ncv-intervals-plasmode.csv |
| ncv-intervals-scores.csv |
| ncv-intervals-synthetic.csv |
| ncv-jackknife-recheck.csv |
| plasmode-ar1.csv |
| plasmode-attached-detached.csv |
| plasmode-attribution.csv |
| plasmode-block-arm-coverage.csv |
| plasmode-block-arm.csv |
| plasmode-block-table.csv |
| plasmode-consistency.csv |
| plasmode-data-table.csv |
| plasmode-datasets.csv |
| plasmode-dependence-table.csv |
| plasmode-ecg-beats.csv |
| plasmode-edf-table.csv |
| plasmode-flips.csv |
| plasmode-G40-paired.csv |
| plasmode-hybrid-by-truth.csv |
| plasmode-hybrid-digest.csv |
| plasmode-lower-tails.csv |
| plasmode-model-table.csv |
| plasmode-mse-ratio-by-truth.csv |
| plasmode-pool-means.csv |
| plasmode-relative-error.csv |
| plasmode-remlcl2-by-truth.csv |
| plasmode-rule-both-criteria-point.csv |
| plasmode-rule-G40-point.csv |
| plasmode-rule.csv |
| plasmode-test-cohort.csv |
| plasmode-zstats-reml.csv |
| relative-error-cells.csv |
| relative-error-core-beta.csv |
| relative-error-summary.csv |
| S1-synthetic-coverage-per-cell.csv |
| S2-synthetic-coverage-alpha-gamma-f.csv |
| S3-synthetic-mse-per-cell.csv |
| S4-plasmode-coverage-per-cell.csv |
| S5-plasmode-mse-ratio-per-cell.csv |
| satterthwaite-contrasts-summary.csv |
| satterthwaite-df.csv |
| ssn-g-ladder-relative-error.csv |
| ssn-g-ladder.csv |
| ssn-g20-coverage.csv |
| ssn-g20-satterthwaite.csv |
| ssn-null-exclusion.csv |
| ssn-null-fpr.csv |
| ssn-null-other-estimands.csv |
| ssn-null-rms.csv |
| ssn-satt-contrasts.csv |
| ssn-satt-df.csv |
| ssn-satt-extensions-low.csv |
| ssn-satt-extensions.csv |
| ssn-satt-plasmode.csv |
| synthetic-ar1.csv |
| synthetic-basis-size.csv |
| synthetic-bias-variance.csv |
| synthetic-coverage-by-estimand.csv |
| synthetic-design-blocks.csv |
| synthetic-design-effects.csv |
| synthetic-extended-families.csv |
| synthetic-fits.csv |
| synthetic-G40-vs-100.csv |
| synthetic-grid-contrasts.csv |
| synthetic-heteroskedastic-coverage.csv |
| synthetic-hybrid-bias-aware.csv |
| synthetic-lower-tail-core.csv |
| synthetic-lowrank-mse.csv |
| synthetic-misregistration-coverage.csv |
| synthetic-mse-ratio-by-error.csv |
| synthetic-oscillating-coverage.csv |
| synthetic-pointwise-ncv.csv |
| synthetic-question1.csv |
| synthetic-recommendation-by-signal.csv |
| synthetic-recommendation.csv |
| synthetic-rule-variants.csv |
| synthetic-signal-truth-beta.csv |
| synthetic-term-edf.csv |
| synthetic-term-type-contrasts.csv |
| synthetic-zstats-reml.csv |
| zstats-plasmode.csv |
| zstats-synthetic.csv |