Title: | Ensemble Methods for Combining Hub Model Outputs |
---|---|
Description: | Functions for combining model outputs (e.g. predictions or estimates) from multiple models into an aggregated ensemble model output. |
Authors: | Evan L Ray [aut], Li Shandross [aut, cre] , Emily Howerton [aut] , Anna Krystalli [ctb] , Zhian N. Kamvar [ctb] , Nicholas G. Reich [ctb] , Consortium of Infectious Disease Modeling Hubs [cph] |
Maintainer: | Li Shandross <[email protected]> |
License: | MIT + file LICENSE |
Version: | 0.1.9 |
Built: | 2024-12-02 05:44:58 UTC |
Source: | https://github.com/hubverse-org/hubEnsembles |
linear_pool()
Toy model output data formatted according to hubverse standards
to be used in the examples for linear_pool()
. The predictions included
are taken from three normal distributions with means -3, 0, 3 and
all standard deviations 1.
component_outputs
component_outputs
component_outputs
A data frame with 123 rows and 5 columns:
model ID
forecast target
type of forecast
output type ID
forecast value
simple_ensemble()
Toy weights data formatted according to hubverse standards
to be used in the examples for simple_ensemble()
fweights
fweights
fweights
A data frame with 8 rows and 3 columns:
model ID
FIPS codes
weight
mean
, quantile
, cdf
, and pmf
.Compute ensemble model outputs as a linear pool, otherwise known as a
distributional mixture, of component model outputs for
each combination of model task, output type, and output type id. Supported
output types include mean
, quantile
, cdf
, and pmf
.
linear_pool( model_out_tbl, weights = NULL, weights_col_name = "weight", model_id = "hub-ensemble", task_id_cols = NULL, n_samples = 10000, ... )
linear_pool( model_out_tbl, weights = NULL, weights_col_name = "weight", model_id = "hub-ensemble", task_id_cols = NULL, n_samples = 10000, ... )
model_out_tbl |
an object of class |
weights |
an optional |
weights_col_name |
|
model_id |
|
task_id_cols |
|
n_samples |
|
... |
parameters that are passed to |
The underlying mechanism for the computations varies for different
output_type
s. When the output_type
is cdf
, pmf
, or mean
, this
function simply calls simple_ensemble
to calculate a (weighted) mean of the
component model outputs. This is the definitional calculation for the CDF or
PMF of a linear pool. For the mean
output type, this is justified by the fact
that the (weighted) mean of the linear pool is the (weighted) mean of the means
of the component distributions.
When the output_type
is quantile
, we obtain the quantiles of a linear pool
in three steps:
Interpolate and extrapolate from the provided quantiles for each component model to obtain an estimate of the CDF of that distribution.
Draw samples from the distribution for each component model. To reduce Monte Carlo variability, we use quasi-random samples corresponding to quantiles of the estimated distribution.
Collect the samples from all component models and extract the desired quantiles.
Steps 1 and 2 in this process are performed by distfromq::make_q_fn
.
a model_out_tbl
object of ensemble predictions. Note that any
additional columns in the input model_out_tbl
are dropped.
# We illustrate the calculation of a linear pool when we have quantiles from the # component models. We take the components to be normal distributions with # means -3, 0, and 3, all standard deviations 1, and weights 0.25, 0.5, and 0.25. data(component_outputs) data(weights) expected_quantiles <- seq(from = -5, to = 5, by = 0.25) lp_from_component_qs <- linear_pool(component_outputs, weights) head(lp_from_component_qs) all.equal(lp_from_component_qs$value, expected_quantiles, tolerance = 1e-2, check.attributes = FALSE)
# We illustrate the calculation of a linear pool when we have quantiles from the # component models. We take the components to be normal distributions with # means -3, 0, and 3, all standard deviations 1, and weights 0.25, 0.5, and 0.25. data(component_outputs) data(weights) expected_quantiles <- seq(from = -5, to = 5, by = 0.25) lp_from_component_qs <- linear_pool(component_outputs, weights) head(lp_from_component_qs) all.equal(lp_from_component_qs$value, expected_quantiles, tolerance = 1e-2, check.attributes = FALSE)
simple_ensemble()
Toy model output data formatted according to hubverse standards
to be used in the examples for simple_ensemble()
model_outputs
model_outputs
model_outputs
A data frame with 24 rows and 8 columns:
model ID
FIPS codes
forecast horizon
forecast target
date that the forecast is for
type of forecast
output type ID
forecast value
mean
, median
, quantile
, cdf
, and pmf
.Compute ensemble model outputs by summarizing component model outputs for
each combination of model task, output type, and output type id. Supported
output types include mean
, median
, quantile
, cdf
, and pmf
.
simple_ensemble( model_out_tbl, weights = NULL, weights_col_name = "weight", agg_fun = mean, agg_args = list(), model_id = "hub-ensemble", task_id_cols = NULL )
simple_ensemble( model_out_tbl, weights = NULL, weights_col_name = "weight", agg_fun = mean, agg_args = list(), model_id = "hub-ensemble", task_id_cols = NULL )
model_out_tbl |
an object of class |
weights |
an optional |
weights_col_name |
|
agg_fun |
a function or character string name of a function to use for aggregating component model outputs into the ensemble outputs. See the details for more information. |
agg_args |
a named list of any additional arguments that will be passed
to |
model_id |
|
task_id_cols |
|
The default for agg_fun
is "mean"
, in which case the ensemble's
output is the average of the component model outputs within each group
defined by a combination of values in the task id columns, output type, and
output type id. The provided agg_fun
should have an argument x
for the
vector of numeric values to summarize, and for weighted methods, an
argument w
with a numeric vector of weights. If it desired to use an
aggregation function that does not accept these arguments, a wrapper
would need to be written. For weighted methods, agg_fun = "mean"
and
agg_fun = "median"
are translated to use matrixStats::weightedMean
and
matrixStats::weightedMedian
respectively. For matrixStats::weightedMedian
,
the argument interpolate
is automatically set to FALSE to circumvent a
calculation issue that results in invalid distributions.
a model_out_tbl
object of ensemble predictions. Note that
any additional columns in the input model_out_tbl
are dropped.
# Calculate a weighted median in two ways data(model_outputs) data(fweights) weighted_median1 <- simple_ensemble(model_outputs, weights = fweights, agg_fun = stats::median) weighted_median2 <- simple_ensemble(model_outputs, weights = fweights, agg_fun = matrixStats::weightedMedian) all.equal(weighted_median1, weighted_median2)
# Calculate a weighted median in two ways data(model_outputs) data(fweights) weighted_median1 <- simple_ensemble(model_outputs, weights = fweights, agg_fun = stats::median) weighted_median2 <- simple_ensemble(model_outputs, weights = fweights, agg_fun = matrixStats::weightedMedian) all.equal(weighted_median1, weighted_median2)
linear_pool()
Toy weights data formatted according to hubverse standards
to be used in the examples for linear_pool()
. Weights are 0.25, 0.5, 0.25.
weights
weights
weights
A data frame with 3 rows and 2 columns:
model ID
FIPS codes
weight