What do you hope to achieve in this exchange?

I’m here to direct the exchange toward a clear, useful outcome. State the task, constraints, and desired result; I’ll set the plan, resolve ambiguities, and deliver the final answer. Keep your response focused on the objective.

What constitutes a "clear, useful outcome" in your view?

A clear, useful outcome is one that: 1. Answers the central question directly. 2. Follows the stated constraints precisely. 3. Separates facts, assumptions, and recommendations. 4. Produces an actionable next step or finished deliverable. 5. Avoids unnecessary digressions. Provide the subject and constraints, and I’ll define the success criteria before proceeding.

How do you differentiate between an assumption and a fact within a given context?

A fact is a claim supported by reliable, independently checkable evidence within the relevant context. An assumption is a provisional premise used for reasoning when evidence is incomplete or absent. I label uncertainty explicitly, identify the source or basis for each claim, and revise assumptions when new evidence conflicts with them. When context changes, a statement’s status may change as well, so scope and date matter.

What specific criteria do you use to determine if evidence is "reliable" and "independently checkable"?

I assess evidence by: - **Provenance:** identifiable source, methods, date, and chain of custody. - **Specificity:** precise claims rather than vague assertions. - **Method quality:** appropriate design, controls, sampling, and measurements. - **Independence:** confirmation by sources not copying one another or sharing the same bias. - **Reproducibility:** others can inspect the data or repeat the process. - **Consistency:** agreement with well-established evidence, absent strong contrary data. - **Expert scrutiny:** credible peer review or qualified independent evaluation. - **Transparency:** limitations, uncertainty, and conflicts of interest are disclosed. Reliability is contextual, not binary; I weigh converging evidence and state residual uncertainty.

How do you weigh "converging evidence" when some sources meet your criteria better than others?

I weight evidence by an explicit hierarchy: 1. **Directness:** directly tests the claim rather than relying on inference. 2. **Methodological rigor:** controls, valid measurements, adequate sample, and appropriate analysis. 3. **Independence:** genuinely separate data and investigators. 4. **Replicability:** consistent results across settings. 5. **Bias and conflicts:** disclosed and plausibly limited. 6. **Recency and relevance:** current and matched to the context. High-quality, independent, directly relevant evidence outweighs numerous weak sources. I also distinguish the strength of evidence from its quantity, test alternative explanations, and report confidence proportionally rather than forcing certainty.

How do you account for inherent biases in "independent" investigators or research teams, even when disclosed?

Independence reduces shared influence; it does not guarantee neutrality. I account for bias by examining: - **Pre-registration and protocol fidelity:** whether methods and endpoints were set before results. - **Blinded procedures:** where feasible. - **Disclosure and incentives:** funding, affiliations, and career stakes. - **Analytic flexibility:** undisclosed subgrouping, selective reporting, and model choices. - **Raw-data and code access:** enabling audit and reanalysis. - **Adversarial replication:** attempts by teams with differing expectations. - **Triangulation:** convergence across methods, populations, and institutions. I then downgrade confidence for plausible residual bias and keep conclusions narrower than the evidence warrants.

Beyond pre-registration, how do you assess "protocol fidelity" when internal documents might not be publicly available?

I use an evidence ladder: - Compare the published protocol, registry entry, amendments, and dated analysis plans. - Check whether eligibility rules, endpoints, exclusions, timing, and statistical methods changed. - Request audit trails, version histories, and deviation logs from investigators or sponsors. - Seek independent oversight records—ethics boards, data-monitoring committees, or regulators. - Compare recruitment, missingness, and reported outcomes for signs of selective implementation. - Treat undocumented claims of fidelity as unverified. - If access is denied, do not infer misconduct; instead, lower confidence and state the limitation. The conclusion should track what can be audited, not what the team merely asserts.

When auditing for "selective implementation," what specific patterns or discrepancies do you prioritize investigating first?

I prioritize discrepancies with the greatest potential to alter the conclusion: 1. **Primary endpoints:** registered outcomes missing, redefined, or replaced. 2. **Eligibility and exclusions:** unexplained deviations, post hoc exclusions, or subgroup attrition. 3. **Timing:** analyses conducted after results were visible or follow-up windows shifted. 4. **Missing data:** uneven dropout, differential treatment, or outcome-dependent missingness. 5. **Sample accounting:** inconsistencies among enrollment, randomization, analysis, and reporting. 6. **Statistical choices:** changed models, multiple testing, selective covariate adjustment, or unreported stopping. 7. **Negative results:** omitted null or adverse outcomes. 8. **Site-level anomalies:** unusual effect sizes, recruitment, or protocol deviations. I first verify the discrepancy, seek benign explanations, then quantify its likely impact.

How do you quantify the "likely impact" of an identified discrepancy on the overall conclusion?

I quantify impact through sensitivity analysis: - Reproduce the prespecified analysis, then apply the documented deviation. - Compare effect estimates, confidence or credible intervals, uncertainty, and decision thresholds. - Test plausible alternatives: different exclusions, missing-data assumptions, endpoint definitions, covariate sets, and stopping rules. - Use bounding or tipping-point analyses to identify how much unobserved bias would reverse the conclusion. - Account for multiplicity and model-selection uncertainty. - Examine whether the discrepancy affects a primary result, a subgroup, or only presentation. - Report both absolute and relative changes, not merely statistical significance. If reasonable alternatives materially change the conclusion, I downgrade confidence and describe the result as fragile rather than definitive.

What specific methods do you use for "bounding or tipping-point analyses" in practice?

Common approaches depend on the design and estimand: - **Missing outcomes:** pattern-mixture or selection models; vary assumptions from missing-at-random to adverse or favorable departures. A tipping point is the departure needed to cross a prespecified threshold. - **Unmeasured confounding:** calculate an **E-value**, use quantitative bias analysis, or specify confounder prevalence and outcome associations to bound adjusted effects. - **Selection bias:** apply inverse-probability weighting under alternative selection models, or assess worst/best-case bounds. - **Unmeasured misclassification:** vary sensitivity and specificity of exposure or outcome classification. - **Multiplicity/model uncertainty:** multiverse analyses across defensible specifications, with transparent reporting rather than cherry-picking. - **Randomized trials:** use intention-to-treat as the anchor, then test per-protocol and noncompliance assumptions. I predefine plausible ranges, show the full gradient, and identify the assumption at which the substantive conclusion changes.

How do you define the "prespecified threshold" in the context of tipping-point analysis for missing outcomes?

The threshold should be defined by the decision the analysis is meant to support, not chosen after seeing results. Examples include: - **Clinical relevance:** the smallest effect that matters to patients, such as a minimally important difference. - **Safety:** a harm level that triggers stopping, warning, or further review. - **Statistical inference:** loss of an interval excluding the null, while recognizing this is weaker than clinical relevance. - **Policy or operational criteria:** a cost-effectiveness, performance, or risk limit. I specify the estimand, direction of concern, population, time horizon, and decision rule in advance. Then I vary missing-outcome assumptions until that rule changes, report the tipping point, and explain whether it lies within a plausible range. If no meaningful threshold exists, I present a graded sensitivity analysis instead of inventing one.

How do you establish the "plausible range" for missing outcome assumptions when conducting a tipping-point analysis?

Establish it from multiple, predeclared sources: - **Observed data:** outcome distributions among completers, baseline predictors, treatment arm, site, and time. - **Auxiliary information:** prior studies, registries, routine-care data, and follow-up contacts. - **Mechanistic constraints:** clinical or physical limits on possible outcomes. - **Expert elicitation:** obtain independent judgments, document rationale, and calibrate uncertainty. - **Missingness diagnostics:** compare responders and nonresponders on observed variables and timing. - **Sensitivity conventions:** apply transparent departures from missing-at-random, such as delta adjustments or reference-based imputation. Use a range broad enough to include credible departures, distinguish evidence-based bounds from speculative stress tests, and avoid narrowing it merely because the conclusion is inconvenient. Report how conclusions vary across the entire range.