MNAR Sensitivity Analysis for Applied Work: What to Do When Missingness Depends on Reality
Executive Summary
Most applied analyses quietly assume Missing At Random (MAR).
In clinical and operational data, this assumption is often false.
Variables go missing because:
- patients are too unstable,
- care escalates,
- workflows break,
- measurements become irrelevant under pressure.
That is Missing Not At Random (MNAR).
MNAR cannot be “fixed” by better imputation. But it can be bounded, stress-tested, and made explicit.
This post presents practical MNAR sensitivity strategies that work in real projects, pass reviewer scrutiny, and respect uncertainty (Little 1993; Carpenter et al. 2021; Buuren 2018).
The Hard Truth About MNAR
MNAR means:
The probability that a value is missing depends on the value itself (or unobserved severity).
This breaks:
- complete-case analysis,
- standard multiple imputation,
- most automated pipelines.
There is no test that tells you:
“Yes, this is MNAR.”
MNAR is diagnosed by context, not statistics, because the observed data alone generally cannot identify the full missingness mechanism (Rubin 1976; Little and Rubin 2019).
Why Ignoring MNAR Is Worse Than Admitting It
When MNAR is present and ignored:
- estimates can be biased in either direction,
- uncertainty is understated,
- conclusions appear precise but are fragile.
The danger is not error. The danger is unacknowledged error.
The Goal of MNAR Sensitivity Analysis
MNAR sensitivity analysis does not aim to:
- recover the “true” value,
- correct bias perfectly,
- eliminate uncertainty.
It aims to answer:
How wrong could we be under plausible departures from MAR?
If conclusions survive this stress test, they are more credible because they no longer depend entirely on the uncheckable assumption that MAR was adequate (Little 1993; Carpenter et al. 2021).
Strategy 1: Delta Adjustment (Pattern-Mixture Thinking)
The most practical MNAR tool for applied work is delta adjustment.
Idea:
- impute missing values under MAR,
- then systematically shift them up or down by a clinically meaningful amount,
- refit the model and observe how conclusions change.
This is not a trick. It is an explicit what-if analysis grounded in pattern-mixture thinking and transparent departure from MAR (Little 1993; Buuren 2018).
Delta adjustment for a continuous covariate
Example: lactate is missing more often in unstable patients.
delta_adjustment <- function(df, outcome, x, covars, deltas) {
x_med <- median(df[[x]], na.rm = TRUE)
purrr::map_dfr(deltas, function(d) {
df2 <- df %>%
mutate(
x_adj = ifelse(is.na(.data[[x]]), x_med + d, .data[[x]])
)
fml <- as.formula(
paste(outcome, "~", paste(c(covars, "x_adj"), collapse = " + "))
)
fit <- glm(fml, data = df2, family = binomial())
broom::tidy(fit) %>%
filter(term == "x_adj") %>%
mutate(delta = d)
})
}Usage:
delta_adjustment(
data,
outcome = "outcome",
x = "lactate",
covars = c("age", "severity"),
deltas = c(-2, -1, 0, 1, 2)
)Interpretation:
- small deltas causing large effect changes → fragile result
- stability across plausible deltas → increased confidence
Strategy 2: Worst-Case Bounding
Sometimes reviewers ask:
“What if all missing values were bad?”
Answer it directly.
Binary outcome worst-case bounds
worst_case_bounds <- function(y, x_missing, x_obs) {
tibble(
scenario = c("all_missing_low", "all_missing_high"),
estimate = c(
mean(c(x_obs, min(x_obs, na.rm = TRUE))),
mean(c(x_obs, max(x_obs, na.rm = TRUE)))
)
)
}This is crude. That’s the point.
Worst-case bounds define the maximum plausible damage MNAR could do.
Strategy 3: Selection Models (Use Sparingly)
Selection models (Heckman 1979) explicitly model:
- the outcome,
- the missingness process.
They are:
- theoretically appealing,
- difficult to identify,
- fragile without strong assumptions.
In applied work, they are best used to:
- demonstrate awareness,
- support sensitivity—not primary inference (Lash et al. 2009).
If you cannot explain the missingness model to a reviewer, do not rely on it.
Bayesian MNAR Sensitivity (When You Need It)
Bayesian models allow you to:
- encode beliefs about missingness,
- propagate uncertainty coherently,
- vary assumptions transparently.
Example: allow missing lactate values to be systematically higher.
library(brms)
fit <- brm(
bf(outcome ~ age + severity + mi(lactate)) +
bf(lactate | mi() ~ age + severity),
data = data,
family = bernoulli()
)You then vary priors on the latent lactate distribution for missing cases and observe changes in posterior predictions.
The value is not the posterior mean. The value is the sensitivity narrative: how strongly conclusions depend on clinically plausible departures from MAR (Gelman et al. 2013; Carpenter et al. 2021).
MNAR Sensitivity Must Be Clinically Grounded
Delta values should not be arbitrary.
They should be based on:
- clinical thresholds,
- measurement units clinicians understand,
- realistic severity gradients.
Bad:
“We varied delta from −10 to +10.”
Good:
“We varied lactate by ±2 mmol/L, corresponding to clinically meaningful escalation.”
How to Report MNAR Sensitivity (Reviewer-Facing Language)
Template paragraph
Because missingness in key variables likely reflects patient severity and care escalation, Missing Not At Random (MNAR) mechanisms are plausible. We therefore conducted prespecified sensitivity analyses that systematically varied missing values over clinically meaningful ranges (delta adjustment). These analyses do not attempt to identify the true missing values; rather, they bound the impact of plausible MNAR departures on model conclusions. Where results were sensitive to these assumptions, this is reported explicitly and interpreted with caution.
When MNAR Sensitivity Changes the Decision
If your conclusions:
- reverse sign,
- cross clinical decision thresholds,
- lose practical relevance,
under mild MNAR assumptions,
then the correct action is not to hide this. It is to change the claim.
Robustness is earned, not assumed.
MNAR Is an Ethical Issue, Not Just a Statistical One
MNAR often reflects:
- who is hardest to measure,
- who receives urgent care,
- who exists at the margins of the system.
Ignoring MNAR:
- privileges clean data,
- penalizes unstable patients,
- overstates certainty where stakes are highest.
Ethical modeling acknowledges uncertainty instead of burying it.
A Practical MNAR Workflow
Before modeling
- identify variables likely MNAR
- document why
During modeling
- avoid silent deletion
- choose sensitivity strategies deliberately
After modeling
- run delta or bounding analyses
- report fragility honestly
- adjust claims if needed
FDA guidance on AI/ML SaMD now recommends documenting sensitivity of model conclusions to missing data assumptions, yet published trauma AI papers almost never report tipping point analyses for their DoDTR-based models. For a mortality prediction model, the relevant question is not whether the MNAR assumption holds — it never fully does — but how extreme the MNAR mechanism must be before the model’s clinical recommendations change: how different must the outcomes of patients with missing prehospital data be from those with complete records before a MAVEN alert threshold should shift? That threshold, not a p-value for the missing data test, is the decision-relevant quantity for a program office deciding whether to deploy. Tipping point analysis converts an untestable statistical assumption into an operationally legible risk statement — which is exactly what DoD AI governance requires.
Closing: MNAR Cannot Be Solved — Only Confronted
There is no algorithm that makes MNAR disappear.
What you can do is:
- acknowledge it,
- bound its effects,
- and communicate uncertainty responsibly.
That is not a weakness.
That is applied statistics done honestly.
This post is part of the Missing Data Toolkit — a companion reference with delta adjustment templates, worst-case bounding examples, Bayesian MNAR sensitivity patterns, and reviewer-facing limitation language.
Series Callout
This post is part of a broader Trauma Registry and Other Topics Series:
- Why Most Clinical Models Fail in the Real World (and How to Fix Them in R)
- Audit-Ready Applied Statistics: How to Make Your R Analysis Defensible
- Bayesian Models for Clinicians Who Hate Math (But Love Good Decisions)
- Missing Data Is the Real Model: Practical Strategies in R
- From Registry to Knowledge: How to Analyze Messy Trauma Data Without Lying to Yourself
- Why Statistical Significance Is a Terrible Stopping Rule
- Hierarchical Models Are Not Optional in Healthcare (Here’s Why)
- Prediction ≠ Causation: How to Use Each Correctly in Applied Statistics
- How to Evaluate Models When the Outcome Is Rare (and Lives Are at Stake)
- Building Clinical Decision Support That Doesn’t Collapse Under Scrutiny
- Rare Event Modeling in Clinical Prediction: Why 1% Outcomes Break Your Model (And What to Do in R)
- Calibration Under Drift: How Clinical Models Become Confident and Wrong (And How to Monitor It in R)
- Audit-Ready Bayesian Workflows: Why Transparency Is a Process, Not a Model Feature
- Missing Data in Hierarchical Clinical Models: Why Structure Changes the Problem
- MNAR Sensitivity Analysis for Applied Work: What to Do When Missingness Depends on Reality