Provocation statement: Damian Betebenner

Provocation statement: Damian Betebenner Statement (about 100 words) **Claim:** AI is moving the frontier of feasible measurement research. Problems that previously required a team of specialists, or simply remained unexplored, can now be attacked by a single researcher working with AI. **Example:** Deriving theoretical properties of new dependence-based models for educational growth, translating them into software, and exploring thousands of simulated conditions would previously have taken months or years, or a team. With AI participating across the entire loop, it became tractable on the scale of weeks. **Unresolved problem:** If AI makes it possible to produce work whose complexity exceeds what its author, or a conventional peer reviewer, can independently reconstruct, what makes the result warranted, and who is accountable for deciding that the evidence is sufficient? --- Longer version and alternatives Core provocation **Claim:** AI is moving the frontier of feasible measurement research. Problems that previously required a team of specialists, or simply remained unexplored because the cost of derivation, implementation, and validation was too high, can now be attacked by a single researcher working with AI. **Example:** In my own work, a mathematical problem involving new dependence-based models for educational growth required deriving theoretical properties, translating those derivations into software, designing simulation studies, exploring thousands of conditions, diagnosing failures, and repeatedly revising the mathematics and implementation. Work of that scope would previously have taken months or years, or required a team. With AI participating across the entire loop—mathematics, code, simulation design, debugging, and critique—it became tractable on the scale of weeks. **Unresolved problem:** If AI makes it possible to produce work whose complexity exceeds what its author—or a conventional peer reviewer—can independently reconstruct, what makes the result warranted? As production becomes cheaper, does verification become the new scarce resource, and who is accountable for deciding that the evidence is sufficient?

As production becomes cheaper, does verification become the new scarce resource, and who is accountable for deciding that the evidence is sufficient? Short provocations > **AI has made production cheap enough that verification may become the scarce resource in measurement science.** > **The question is no longer whether I can do the analysis. It is whether I can warrant the analysis that AI has made possible.** > **AI is not merely helping me do my old job faster. It is changing what my job is.** Alternative provocation: the bottleneck is changing **Claim:** AI does not merely increase productivity; it changes what the scarce professional skill is. When derivation, coding, simulation, documentation, and critique become inexpensive, the bottleneck moves from producing technical work to knowing what problem to pose, what constraints matter, what evidence would falsify the result, and when not to trust what has been produced. **Example:** I increasingly spend less of my time physically writing code or working through mathematical derivations and more of it specifying mathematical objects, constructing adversarial tests, examining failures, comparing alternative formulations, and deciding whether an apparently successful result is actually meaningful. AI performs much of the execution; my work is increasingly the design and governance of the epistemic process around it. **Unresolved problem:** If that is the emerging division of labor, are we training psychometricians for the wrong job? What knowledge must a measurement professional possess personally in order to supervise work that AI can perform better and faster than they can execute themselves?

**Unresolved problem:** If that is the emerging division of labor, are we training psychometricians for the wrong job? What knowledge must a measurement professional possess personally in order to supervise work that AI can perform better and faster than they can execute themselves? Alternative provocation: disciplinary boundaries are collapsing **Claim:** AI is collapsing the practical boundaries between mathematical theorist, statistician, programmer, simulation researcher, technical writer, and software developer. That changes not only how measurement research is done, but which research questions can realistically be pursued. **Example:** A methodological idea can now move in hours from an informal conjecture, to mathematical derivation, to executable code, to simulated counterexamples, to visualization, and back to a revised conjecture. In my recent work, that loop has repeated dozens of times around problems that previously would have required handoffs among people with different specialties. The important change is not that each step is faster; it is that the entire intellectual loop can remain continuous. **Unresolved problem:** Our systems of expertise, authorship, peer review, and graduate education were built around a world in which those capabilities were distributed across people and institutions. What happens when they can be concentrated in a single human–AI working unit?