On this page
Who bears the cost of research verification?
My questions after reading STM's integrity report: false positives, researcher burden, resources, and what better verification would look like.
After reading STM's report on research integrity, I kept thinking about the researcher whose legitimate work gets flagged.
Checking more submissions can be useful. It also creates questions about delay, explanation, cost, and appeal. Those questions deserve attention alongside the tools used to detect misconduct.
What evidence are we looking at?
The report draws on expert interviews and publisher practices. That makes it useful for understanding what organizations are doing. It does not establish that every described intervention reduces misconduct or that the same approach works across publishers. Read STM's account of the report or the full report (PDF).
I would separate adoption from effectiveness. A larger team or a higher screening rate describes capacity. It does not, by itself, describe the accuracy or fairness of decisions.
The false-positive problem
A screening system can identify something unusual without knowing why it happened. A researcher then needs an understandable question and a chance to answer it.
The workflow I would want is straightforward: explain the concern, ask for the relevant original material, have a qualified person review it, and communicate the outcome. An automated score should not become a misconduct accusation without that process.
This is familiar from product design. A tool's error handling determines what happens to the person on the other side of an imperfect output.
Who can afford verification?
Source records, original images, and reproducible code take work to maintain. Specialist investigation teams and tools take resources too.
My concern is how those requirements affect smaller institutions and early-career researchers. A standard can be useful while its implementation is difficult to meet. Shared training, accessible guidance, and clearly scoped evidence requests could help avoid making resources a proxy for credibility.
This is a question I bring to the report, rather than an outcome the interviews establish.
What would count as improvement?
I would want to see measures that account for both integrity and researcher experience: confirmed problems caught, false-positive rates, time to resolve concerns, quality of explanations, and whether corrections remain visible.
Even those measures need interpretation. More detections could reflect better screening or a larger problem. Fewer corrections could reflect prevention or weaker follow-up. A single number cannot settle the question.
My own next step
I can start with the part under my control: making source records and experiment decisions inspectable. Working on dataset provenance has made that less abstract.
The report helped me see verification as ongoing work involving people, evidence, and decisions. The challenge I want to keep thinking about is how to make that work rigorous while giving honest researchers a fair and understandable process.