Accurate uncertainty quantification in Bayesian calibration accounting for correlated targets

SMDM 47th Annual Meeting

Conference abstract presenting a method for accurate uncertainty quantification in Bayesian calibration of health decision models with correlated calibration targets. SMDM 47th Annual Meeting, 2025.

Bayesian Calibration with Correlated Targets

SMDM 47th Annual Meeting

June 2025 DOI: 10.1177/0272989X251408817

Roa, J., Gracia, V., Goldhaber-Fiebert, J.D., & Alarid-Escudero, F.

Thomas Cole — The Voyage of Life: Manhood (1842) National Gallery of Art · Washington D.C. A man navigates a violent rapid alone — the guardian angel of his youth has vanished. Stormy skies press down, shadowy phantoms drift through the clouds, and the boat is swept toward darkness. Part of Cole’s four-part allegory of human life.

The Problem

Model calibration using a likelihood on multiple targets often assumes those targets are independent when constructing the overall goodness-of-fit measure. But calibration targets – such as survival, prevalence, and disease proportions over time – are inherently correlated. When this independence assumption is violated, calibration underestimates parameter uncertainty and, consequently, the uncertainty in model-predicted outcomes.

The Approach

We calibrated a Sick-Sicker cohort Markov model to three time-dependent targets – survival, disease prevalence, and the proportion Sick of the total Sick-or-Sicker cohort – using four different approaches to compute the overall likelihood. First, using all target observations assuming independence. Second, selecting a subset of observations based on the effective sample size (ESS) of each target treated as a time series. Third, constructing a correlated likelihood assuming multivariate normality with a covariance matrix estimated from an autocorrelated error of order 1. Fourth, estimating sufficient statistics for each target and using those as calibration targets instead. All approaches used the IMIS algorithm to draw 1,000 parameter sets from the joint posterior distribution.

What We Found

4

Likelihood approaches compared

3

Correlated calibration targets

1,000

Posterior parameter sets via IMIS

Assuming independence underestimated parameter and model-predicted uncertainty. The three methods that account for correlation produced wider, more realistic uncertainty bounds. The multivariate normal (MVN) approach produced the widest bounds – even wider than the target data itself. The ESS and sufficient statistics approaches produced similar, well-calibrated uncertainty bounds that fell within target uncertainty. However, ESS requires deciding which observations to include, whereas sufficient statistics avoids this arbitrary choice entirely.

Model-predicted 95% posterior credible bounds for survival (left), prevalence (center), and proportion Sick (right) under four likelihood approaches: independence assumption (top row), effective sample size, multivariate normal, and sufficient statistics (bottom rows). The independence assumption systematically underestimates uncertainty.

Why It Matters

Using sufficient statistics from the target data as calibration targets quantifies uncertainty more accurately – without incorrectly assuming independence, picking an arbitrary subset of observations, or assuming a specific correlation structure. This provides a principled, practical solution for any health policy model calibrated to time-series targets.

Citation

Roa, J., Gracia, V., Goldhaber-Fiebert, J.D., & Alarid-Escudero, F. (2025). Accurate uncertainty quantification in Bayesian calibration accounting for correlated targets. Medical Decision Making, 45(4). Society for Medical Decision Making (SMDM) 47th Annual Meeting, University of Michigan, Ann Arbor, USA. DOI: 10.1177/0272989X251408817

Read the abstract