Audit-Ready Applied Statistics: How to Make Your R Analysis Defensible
Executive Summary
“Reproducible” is not the same as “defensible.”
A reproducible analysis can still fail review if it lacks: - a clear question tied to a decision, - traceability from raw data to results, - documented assumptions, - pre-specified rules for edge cases, - transparent QC and sensitivity checks, - and a stable output that can be re-run months later.
Audit-ready applied statistics is a workflow: intent → provenance → transformation → analysis → QC → reporting → archiving.
This post lays out a practical blueprint in R, drawing on the same reporting logic that underlies modern guidance such as SAMPL, TRIPOD, and related recommendations for transparent statistical reporting (Lang and Altman 2015; Moons et al. 2015; Ou et al. 2021).
1. What “Audit-Ready” Means (In Plain Language)
Audit-ready work is not about formality for its own sake. It is about making the inferential path visible: what data were used, what was decided, what was derived, and how the analytic choices map to the final claim (Lang and Altman 2015; Steyerberg 2019).
An audit-ready analysis is one where an independent reviewer can answer:
- What decision or claim is this analysis supporting?
- What data was used, exactly?
- How was the data transformed, exactly?
- Which assumptions were made and why?
- What checks were run to detect errors or fragility?
- Can we re-run it and get identical results?
- If results change, can we explain what changed?
If your analysis cannot answer these questions quickly and unambiguously, it may still be “good science”—but it is not audit-ready.
2. The Audit Triangle: Claim, Data, Code
Audit failure almost always happens in one of these:
- Claim drift: results don’t match the stated question/endpoint.
- Data ambiguity: unclear cohorts, unclear time windows, unclear versions.
- Code fragility: code runs “on your machine” but not elsewhere, or outputs are not stable.
Your goal is to reduce each to a set of explicit, testable objects.
3. Start With a Decision-First Analysis Contract
Before any modeling, create a single “analysis contract” section:
3.1 Problem Statement (Decision Context)
- Who is the audience?
- What decision will this support?
- What failure would be unacceptable?
3.2 Endpoints + Cohort Definitions (Operationalized)
Write cohort rules as deterministic code, not narrative.
# Example: cohort definition as code
define_cohort <- function(df) {
df |>
dplyr::filter(age >= 18) |>
dplyr::filter(!is.na(index_time)) |>
dplyr::filter(index_time >= as.POSIXct("2020-01-01"))
}3.3 Pre-specify Analytic Choices
- Primary model + alternatives
- Primary metric(s)
- Handling of missingness
- Subgroup checks
- Sensitivity analyses
Even if you revise later, you want the diff of what changed and why.
4. Provenance: Make the Data Version Explicit
4.1 Never Analyze “A Folder of Files”
Every analysis should record:
- File names
- Hashes (fingerprints)
- Row counts
- Date pulled/extracted
- Query parameters (if from a database)
hash_file <- function(path) {
# Lightweight hash via tools available in base R environment
# (For production, consider digest::digest(file = path, algo = "sha256"))
paste0(path, "::", file.info(path)$size, "::", file.info(path)$mtime)
}
record_provenance <- function(paths) {
tibble::tibble(
file = paths,
fingerprint = vapply(paths, hash_file, character(1)),
size = file.info(paths)$size,
modified_time = file.info(paths)$mtime
)
}4.2 Store Provenance in the Output
Write provenance into an appendix table in the final report, and save it as CSV alongside outputs.
5. Transformations: Make Your Pipeline Traceable
A defensible pipeline is:
- modular
- named
- documented
- tested
- stable
5.1 “Raw → Clean → Analytic” Stages
raw/= immutable inputderived/= cleaned, standardized intermediate productsanalysis/= analytic datasets (ADaM-like or “analysis-ready”) (Analysis Data Model (ADaM) Implementation Guide 2009)
make_adsl_like <- function(raw_df) {
raw_df |>
dplyr::mutate(
# Standardize types
subject_id = as.character(subject_id),
sex = factor(sex),
age = as.numeric(age)
) |>
dplyr::mutate(
# Derived variables with explicit rules
age_group = dplyr::case_when(
age < 18 ~ "<18",
age < 40 ~ "18-39",
age < 65 ~ "40-64",
TRUE ~ "65+"
)
)
}5.2 Avoid “Silent” Rules
If you:
- drop NAs,
- cap outliers,
- recode categories,
…log it and count it.
qc_counts <- function(df_before, df_after, label = "step") {
tibble::tibble(
step = label,
n_before = nrow(df_before),
n_after = nrow(df_after),
dropped = nrow(df_before) - nrow(df_after)
)
}6. QC as a First-Class Artifact
QC is not a vibe. It’s an output.
6.1 Build a QC Checklist Into the Report
Minimum QC artifacts:
- row counts by key stages
- missingness summary
- distribution checks (by site/time)
- key variable range checks
- duplicate checks on identifiers
- cohort flow diagram (even simple counts)
range_check <- function(df, var, lo, hi) {
x <- df[[var]]
tibble::tibble(
variable = var,
lo = lo,
hi = hi,
n_out_of_range = sum(!is.na(x) & (x < lo | x > hi)),
pct_out_of_range = mean(!is.na(x) & (x < lo | x > hi))
)
}6.2 QC Must Fail Loudly
If a critical check fails, stop.
assert_no_duplicates <- function(df, key) {
dups <- df |>
dplyr::count(dplyr::across(dplyr::all_of(key))) |>
dplyr::filter(n > 1)
if (nrow(dups) > 0) {
stop("Duplicate keys detected: ", paste(key, collapse = ", "))
}
invisible(TRUE)
}7. Sensitivity Analyses: Demonstrate Robustness, Not Certainty
Audit-ready work anticipates the reviewer’s question:
“If we changed a reasonable assumption, would the conclusion hold?”
Examples:
- missingness assumptions (MAR vs not)
- alternative outcome definitions
- alternative time windows
- alternative models (parsimonious vs flexible)
You don’t need 20 sensitivity analyses. You need the 3 that matter.
8. Reporting: Make Outputs Stable and Reviewable
8.1 Use Deterministic Seeds
set.seed(20260125)8.2 Lock Session Details
sessionInfo()8.3 Save Machine-Readable Outputs
Alongside the HTML/PDF:
tables/*.csvfigures/*.pngmodel_objects/*.rdsqc/*.csv
saveRDS(model_fit, "model_objects/fit_primary.rds")
readr::write_csv(qc_table, "qc/qc_summary.csv")9. Archiving: Results Must Survive You
An audit-ready analysis should be runnable by:
- future-you,
- a teammate,
- or a reviewer,
without Slack archaeology (Peng 2011; Sandve et al. 2013).
Minimum archive:
- report (HTML/PDF)
- code snapshot
- data fingerprints
- config file (parameters)
- model objects
- QC outputs
10. A Practical Folder Template (Copy/Paste)
project/
README.md
renv.lock # if used
config.yml # analysis parameters
data/
raw/
derived/
analysis/
R/
01_ingest.R
02_derive.R
03_analysis.R
04_qc.R
05_report_helpers.R
reports/
audit_ready_analysis.Rmd
output/
figures/
tables/
qc/
model_objects/
logs/
run_log.csvFDA and DoD audit requirements for clinical AI — including RAIMF and the FDA’s AI/ML SaMD action plan — demand full reproducibility of model development: every preprocessing step, train/test split, and hyperparameter choice must be logged and recoverable. DoDTR-based models deployed via MAVEN must meet this standard, yet most published trauma AI papers report insufficient methodological detail to reproduce their results, let alone audit them. When an adverse event occurs and a clinical AI tool’s recommendation is implicated, an unauditable model cannot be investigated — and an uninvestigable model cannot be trusted with subsequent patients. An unauditable model is an unaccountable model.
11. Closing: What “Defensible” Looks Like
A defensible analysis is one where:
- the question is clear,
- the cohort is reproducible,
- the pipeline is traceable,
- QC is explicit,
- assumptions are documented,
- sensitivity is demonstrated,
- and outputs can be re-run identically.
This is not bureaucracy. It is how applied statistics earns trust (Simera et al. 2010).
This post is part of the Trauma Registry Analytics Toolkit — a companion reference with audit-ready code templates, QC checklists, and reviewer-facing language.
Series Callout
This post is part of a broader Trauma Registry and Other Topics Series:
- Why Most Clinical Models Fail in the Real World (and How to Fix Them in R)
- Audit-Ready Applied Statistics: How to Make Your R Analysis Defensible
- Bayesian Models for Clinicians Who Hate Math (But Love Good Decisions)
- Missing Data Is the Real Model: Practical Strategies in R
- From Registry to Knowledge: How to Analyze Messy Trauma Data Without Lying to Yourself
- Why Statistical Significance Is a Terrible Stopping Rule
- Hierarchical Models Are Not Optional in Healthcare (Here’s Why)
- Prediction ≠ Causation: How to Use Each Correctly in Applied Statistics
- How to Evaluate Models When the Outcome Is Rare (and Lives Are at Stake)
- Building Clinical Decision Support That Doesn’t Collapse Under Scrutiny
- Rare Event Modeling in Clinical Prediction: Why 1% Outcomes Break Your Model (And What to Do in R)
- Calibration Under Drift: How Clinical Models Become Confident and Wrong (And How to Monitor It in R)
- Audit-Ready Bayesian Workflows: Why Transparency Is a Process, Not a Model Feature
- Missing Data in Hierarchical Clinical Models: Why Structure Changes the Problem
- MNAR Sensitivity Analysis for Applied Work: What to Do When Missingness Depends on Reality