| Title: | Cumulative Calibration Assessment for Prediction Models |
|---|---|
| Description: | Tools for visualization of, and inference on, the calibration of prediction models on the cumulative domain. This provides a method for evaluating calibration of risk prediction models without having to group the data or use tuning parameters (e.g., loess bandwidth). This package implements the methodology described in Sadatsafavi and Patkau (2024) <doi:10.1002/sim.10138>. The core of the package is cumulcalib(), which takes in vectors of binary responses and predicted risks. The plot() and summary() methods are implemented for the results returned by cumulcalib(). |
| Authors: | Mohsen Sadatsafavi [aut, cre] |
| Maintainer: | Mohsen Sadatsafavi <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.0.1 |
| Built: | 2025-01-27 05:41:38 UTC |
| Source: | https://github.com/resplab/cumulcalib |
This is the core function for performing cumulative calibration assessment
cumulcalib(y, p, method = c("BB", "BM"), ordered = FALSE, n_sim = 0)cumulcalib(y, p, method = c("BB", "BM"), ordered = FALSE, n_sim = 0)
y |
vector of binary responses |
p |
vector of predicted probabilities. |
method |
string with either BB (Brownian bridge test, default method), BM (Brownian motion test), BM2p (two-part BM test - experimental), BB1p (one-part BB test wit only the 'bridge' component). Multiple methods can be specified. The first one will be the 'main' method (e.g., when submitting the resulting object to plot()). Default is c("BB","BM") |
ordered |
if TRUE, y and p are already ordered based on ascending values of p. This is to speed up simulations. |
n_sim |
if >0, indicates a simulation-based test is requested for inference. |
an objective of class cumulcalib that can be printed or plotted
pi <- rbeta(1000,1,2) Y <- rbinom(length(pi),1,pi) res <- cumulcalib(Y, pi, method="BB") summary(res) plot(res)pi <- rbeta(1000,1,2) Y <- rbinom(length(pi),1,pi) res <- cumulcalib(Y, pi, method="BB") summary(res) plot(res)
CDF of the Kolmogorov distribution
pKolmogorov(q, summands = ceiling(q * sqrt(72) + 3/2))pKolmogorov(q, summands = ceiling(q * sqrt(72) + 3/2))
q |
the quantity at which CDF will be evaluated. Currently accepts only a scalar |
summands |
maximum number of terms to be evaluated in the infinite series (default=ceiling(q*sqrt(72)+3/2)) |
a scalar value
Generates cumulative calibration plots
## S3 method for class 'cumulcalib' plot( x, method = NULL, draw_stat = TRUE, stat_col = c("blue", "red"), draw_sig = TRUE, sig_level = c(0.95, 0.95), x2axis = TRUE, y2axis = TRUE, ... ) ## S3 method for class 'cumulcalib' plot( x, method = NULL, draw_stat = TRUE, stat_col = c("blue", "red"), draw_sig = TRUE, sig_level = c(0.95, 0.95), x2axis = TRUE, y2axis = TRUE, ... )## S3 method for class 'cumulcalib' plot( x, method = NULL, draw_stat = TRUE, stat_col = c("blue", "red"), draw_sig = TRUE, sig_level = c(0.95, 0.95), x2axis = TRUE, y2axis = TRUE, ... ) ## S3 method for class 'cumulcalib' plot( x, method = NULL, draw_stat = TRUE, stat_col = c("blue", "red"), draw_sig = TRUE, sig_level = c(0.95, 0.95), x2axis = TRUE, y2axis = TRUE, ... )
x |
An object of class cumulcalib generated by cumulcalib() |
method |
Which method to use. Options are BB (Brownian bridge test), BM (Brownian motion test), BB1p (1-part Brownian bridge test), and BM2p (2-part Brownian bridge test). If unspecified, returns the default method used in the cumulcalib() call |
draw_stat |
Should the statistic (terminal value an/or maximum drift, depending on method) be drawn? Default is TRUE |
stat_col |
The color(s) to draw the stat. Default is c('blue','red'). For single-part tests (BM and BB1p) only the first element is used |
draw_sig |
Whether significance lines should be drawn. Default is T. Colors will be the same as stat_col |
sig_level |
If to draw significance lines, at what level? Default is c(0.95,0.95). For single-part tests (BM and BB1p) only the first element is used |
x2axis |
If true, draws a second x-axis (on top) showing predicted risks |
y2axis |
If true, draws a second y-axis (on right) showing scaled partial sums |
... |
Parameters to be passed to plot() |
None
None
CDF of the distribution of the maximum absolute deviation of Brownian motion in [0,1] interval
pMAD_BM(q, summands = 100)pMAD_BM(q, summands = 100)
q |
the quantity at which CDF will be evaluated. Currently accepts only a scalar |
summands |
maximum number of terms to be evaluated in the infinite series (default=100) |
a scalar value
CDF of the distribution of the maximum absolute deviation of Brownian motion in [0,1] interval, conditional on its terminal value
pMAD_BM_c( q, w1, method = 1, exp_tolerance = -30, summands = ceiling(q * sqrt(72) + 3/2) )pMAD_BM_c( q, w1, method = 1, exp_tolerance = -30, summands = ceiling(q * sqrt(72) + 3/2) )
q |
the quantity at which CDF will be evaluated. Currently accepts only a scalar |
w1 |
the terminal value |
method |
different infinite series to use (1,2,3) |
exp_tolerance |
numerical tolerance as the stopping rule when evaluating the infinite sum (default -30 on the exponential scale) |
summands |
number of terms to evaluate (default is ceiling(q * sqrt(72) + 3/2)) |
a scalar value
Quantile function of the Kolmogorov distribution
qKolmogorov(p)qKolmogorov(p)
p |
the quantity at which the quantile function will be evaluated. Currently accepts only a scalar |
a scalar value
Quantile function of the distribution of the maximum absolute deviation of Brownian motion in [0,1] interval
qMAD_BM(p)qMAD_BM(p)
p |
the quantity at which the quantile function will be evaluated. Currently accepts only a scalar |
a scalar value
Quantile function of the distribution of the maximum absolute deviation of Brownian motion in [0,1] interval, conditional on its terminal value
qMAD_BM_c(p, w1)qMAD_BM_c(p, w1)
p |
the quantity at which the quantile function will be evaluated. Currently accepts only a scalar |
w1 |
the terminal value |
a scalar value
Summarizes a cumulcalib object
Summarizes a cumulcalib object
## S3 method for class 'cumulcalib' summary(object, method = NULL, ...) ## S3 method for class 'cumulcalib' summary(object, method = NULL, ...)## S3 method for class 'cumulcalib' summary(object, method = NULL, ...) ## S3 method for class 'cumulcalib' summary(object, method = NULL, ...)
object |
An object of class cumulcalib generated by cumulcalib() |
method |
Which method to use. Options are BB (Brownian bridge test), BM (Brownian motion test), BB1p (1-part Brownian bridge test), and BM2p (2-part Brownian bridge test). If unspecified, returns the default method used in the cumulcalib() call |
... |
Other parameters passed to summary() |
None
None