STAI-X 2026 Reproducibility Checklist (https://statsupai.org/STAIX2026/)
\begin
Discover why over 25 million people worldwide trust Overleaf with their work.
\documentclass{article}
\usepackage[preprint]{staix_2026}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{hyperref}
\usepackage{url}
\usepackage{booktabs}
\usepackage{array}
\usepackage{tabularx}
\usepackage{longtable}
\usepackage{enumitem}
\usepackage{xcolor}
\setlength{\LTpre}{6pt}
\setlength{\LTpost}{6pt}
\setlist[itemize]{leftmargin=1.5em}
\newcommand{\fieldTODO}[1]{\textcolor{red}{\bf [FILL: #1]}}
\begin{document}
\section*{STAI-X 2026 Reproducibility Checklist}
\noindent
\textbf{Accepted STAI-X papers will undergo a reproducibility check.
Authors should provide code, data, and instructions that are as complete as possible
so that as many main text figure or table result units as possible can be reproduced.
For this checklist, authors only need to consider result units that appear as
figures or tables in the main text;
appendix and supplementary material results do not need to be included for now.}
\vspace{1ex}
%%% BEGIN AUTHOR INSTRUCTIONS %%%
For each checklist item below:
\begin{itemize}
\item For Item~1, replace \verb+\answerTODO{}+ with exactly one of \verb+\answerYes{}+, \verb+\answerPartial{}+, \verb+\answerNo{}+, or \verb+\answerNA{}+.
\item For Items~2 and 4, replace \verb+\answerTODO{}+ with exactly one of \verb+\answerYes{}+, \verb+\answerNo{}+, or \verb+\answerNA{}+.
\item Use \verb+\answerPartial{}+ for Item~1 when coverage is only partial, and explain the partial coverage in the justification.
\item Replace \verb+\justificationTODO{}+ with a short 1--3 sentence explanation.
\item If code is hosted on GitHub or another repository, provide the URL and the exact branch, tag, release, or commit used for this checklist.
\end{itemize}
After the numbered checklist, please complete the execution elements table.
%%% END AUTHOR INSTRUCTIONS %%%
\vspace{1ex}
\noindent
\textbf{Paper title:} \fieldTODO{paper title} \\
\textbf{Primary supported platform(s):} \fieldTODO{e.g., Ubuntu 22.04, Python 3.11} \\
\textbf{GitHub / code repository URL (optional):} \fieldTODO{repository URL, if applicable} \\
\textbf{Repository version / location (optional):} \fieldTODO{branch / tag / release / commit SHA; subdir if needed}
\begin{enumerate}
\item {\bf Main text figure or table reproducibility}
\item[] Question: Are the main text figures and tables reproducible from your submitted code on at least one supported platform?
\item[] Answer: \answerTODO{}
\item[] Justification: \justificationTODO{}
\item[] Guidelines:
\begin{itemize}
\item Main text outputs should include the paper's central figures and tables from the main text.
\item If only a subset is reproducible, answer \answerPartial{} and specify which outputs are omitted or blocked.
\item If no main text outputs are reproducible from the release materials, answer \answerNo{} and identify the blocking issue.
\end{itemize}
\item {\bf Dependency specification}
\item[] Question: Is a dependency specification included?
\item[] Answer: \answerTODO{}
\item[] Justification: \justificationTODO{}
\item[] Guidelines:
\begin{itemize}
\item Examples include \verb+requirements.txt+, \verb+environment.yml+, \verb+renv.lock+, \verb+Project.toml+, Dockerfiles, or explicit system package lists.
\end{itemize}
\item {\bf Data availability}
\item[] Question: What is the availability status of the data required for the main text results?
\item[] Answer: \textcolor{red}{\bf [TODO: Included / Public / Partly available / Restricted / Not available]}
\item[] Justification: \justificationTODO{}
\item[] Guidelines:
\begin{itemize}
\item Use \textbf{Included} if the required data are bundled directly with the release materials.
\item Use \textbf{Public} if the required data are available through public links or repositories with clear access instructions.
\item Use \textbf{Partly available} if only a subset of the required data are included or publicly available.
\item Use \textbf{Restricted} if the main results depend on data that require controlled, licensed, or case-by-case access.
\item Use \textbf{Not available} if the required data cannot be made available to STAI-X organizers or readers.
\item The justification should state which parts are available, which parts are restricted or missing, and whether a reduced or demo substitute is provided.
\end{itemize}
\item {\bf External services, credentials, or hosted infrastructure}
\label{item:external-services}
\item[] Question: Are any external services, credentials, hosted models, public APIs,
paid APIs, rate limited APIs, external database endpoints,
or similar hosted dependencies required?
\item[] Answer: \answerTODO{}
\item[] Justification: \justificationTODO{}
\item[] Guidelines:
\begin{itemize}
\item Answer \answerYes{} if reproduction requires any external hosted dependency
(for example, hosted models, public or paid APIs, rate limited APIs,
external database endpoints, private infrastructure, gated services,
non-public credentials, or mutable public endpoints); otherwise answer \answerNo{}.
\end{itemize}
\end{enumerate}
\paragraph{Execution Elements Table Requirements}
\begin{itemize}
\item For this checklist, include only main text figure or table result units. Do not include appendix
or supplementary material results for now.
\item Complete one row per main text figure or table result unit that requires running
experiments, processing data, or producing released outputs.
\item Each row must describe exactly one main text figure or one main text table result unit.
Do not combine multiple result units in one row, even if they are produced by the same
script, command, or workflow.
\item Do not include purely conceptual workflow diagrams, architecture diagrams,
or schematic figures unless they are generated from experiment data or release artifacts.
\item Use exact relative paths and commands whenever possible.
\item In the result description column, state any expected tolerance, randomness,
or approximate-match criteria when exact agreement with the manuscript value is not expected.
\item In the reproducibility column, enter exactly one of \answerYes{},
\answerPartial{}, or \answerNo{} for the corresponding result item.
\item If reproducibility is \answerYes{} or \answerPartial{}, fill in the expected output and runtime column.
If reproducibility is \answerNo{}, leave the expected output and runtime column blank; this result item
will not be included in the STAI-X reproducibility check.
\item If reproducibility is \answerPartial{} or \answerNo{}, explain the omitted or blocked parts
in the notes on computation and stability column.
\item In the notes on computation and stability column, state any large compute requirement
(for example, LLM fine tuning or large model training) or reproducibility stability issue
(for example, live LLM API calls, hosted judges, mutable external APIs, or nondeterministic services).
\end{itemize}
\begingroup
\scriptsize
\setlength{\tabcolsep}{1pt}
\begin{longtable}{@{}>{\raggedright\arraybackslash}p{0.10\linewidth}%
>{\raggedright\arraybackslash}p{0.16\linewidth}%
>{\raggedright\arraybackslash}p{0.10\linewidth}%
>{\raggedright\arraybackslash}p{0.18\linewidth}%
>{\raggedright\arraybackslash}p{0.10\linewidth}%
>{\raggedright\arraybackslash}p{0.08\linewidth}%
>{\raggedright\arraybackslash}p{0.12\linewidth}%
>{\raggedright\arraybackslash}p{0.12\linewidth}@{}}
\toprule
\textbf{Paper locator} &
\textbf{Result description and tolerance} &
\textbf{Entrypoint} &
\textbf{Canonical command} &
\textbf{Data availability} &
\textbf{Repro\-ducibility} &
\textbf{Expected output and runtime} &
\textbf{Notes on computation and stability} \\
\midrule
\endfirsthead
\toprule
\textbf{Paper locator} &
\textbf{Result description and tolerance} &
\textbf{Entrypoint} &
\textbf{Canonical command} &
\textbf{Data availability} &
\textbf{Repro\-ducibility} &
\textbf{Expected output and runtime} &
\textbf{Notes on computation and stability} \\
\midrule
\endhead
\fieldTODO{Figure / Table / section} &
\fieldTODO{what this result shows; randomness and tolerance statement} &
\fieldTODO{script or notebook path} &
\fieldTODO{exact command} &
\fieldTODO{Included / Public / Partly available / Restricted / Not available} &
\fieldTODO{Yes / Partial / No} &
\fieldTODO{output file or directory; expected runtime} &
\fieldTODO{hardware, large compute, API or stability caveats} \\
\addlinespace
\fieldTODO{Figure / Table / section} &
\fieldTODO{what this result shows; randomness and tolerance statement} &
\fieldTODO{script or notebook path} &
\fieldTODO{exact command} &
\fieldTODO{Included / Public / Partly available / Restricted / Not available} &
\fieldTODO{Yes / Partial / No} &
\fieldTODO{output file or directory; expected runtime} &
\fieldTODO{hardware, large compute, API or stability caveats} \\
\addlinespace
\fieldTODO{Figure / Table / section} &
\fieldTODO{what this result shows; randomness and tolerance statement} &
\fieldTODO{script or notebook path} &
\fieldTODO{exact command} &
\fieldTODO{Included / Public / Partly available / Restricted / Not available} &
\fieldTODO{Yes / Partial / No} &
\fieldTODO{output file or directory; expected runtime} &
\fieldTODO{hardware, large compute, API or stability caveats} \\
\addlinespace
\fieldTODO{Figure / Table / section} &
\fieldTODO{what this result shows; randomness and tolerance statement} &
\fieldTODO{script or notebook path} &
\fieldTODO{exact command} &
\fieldTODO{Included / Public / Partly available / Restricted / Not available} &
\fieldTODO{Yes / Partial / No} &
\fieldTODO{output file or directory; expected runtime} &
\fieldTODO{hardware, large compute, API or stability caveats} \\
\bottomrule
\end{longtable}
\endgroup
\paragraph{Example row}
Authors may delete this example from the completed checklist.
\begin{itemize}
\item \textbf{Paper locator:} Table 3
\item \textbf{Result description and tolerance:} Main predictive performance metrics on the full clinical dataset; manuscript values are means over seeds 1--5 and should match within 0.5 percentage points.
\item \textbf{Entrypoint:} \texttt{scripts/run\_main\_experiment.py}
\item \textbf{Canonical command:} \texttt{python scripts/run\_main\_experiment.py --config configs/full.yaml --seeds 1 2 3 4 5 --outdir outputs/table3}
\item \textbf{Data availability:} Restricted
\item \textbf{Reproducibility:} \answerPartial{}
\item \textbf{Expected output and runtime:} \texttt{outputs/table3/summary\_metrics.csv}; about 3.5 h.
\item \textbf{Notes on computation and stability:} 1 GPU with 24 GB VRAM, 32 GB RAM; exact manuscript values require licensed data access, so public release reproduces only the synthetic-data version.
\end{itemize}
\end{document}