AI is changing what my job is (provocation slides)

AI is changing what my job is Provocation · Damian Betebenner, Center for Assessment · AIME-Con 2026 Betebenner · Educational Measurement as an AI-Native Profession 1 / 10 One sentence AI is not merely helping me do my old job faster. It is changing what my job is. Betebenner · Educational Measurement as an AI-Native Profession 2 / 10 The frontier moved O N E C L A I M Problems that once required a team of specialists—or remained unexplored because they were too expensive to pursue—can now be attacked by a single researcher working with AI. Betebenner · Educational Measurement as an AI-Native Profession 3 / 10 What changed in my own work I N M Y W O R K F L O W Before AI With AI Theory derivation is slow and locally constrained derivation and counterexample generation iterate continuously Code implementation is a separate bottleneck code evolves alongside the mathematics Simulation designs limited by time and implementation cost thousands of conditions can be explored and revised quickly Critique review often arrives late adversarial checking can occur throughout the workflow Documentation frequently downstream of the analysis documentation can evolve with the work My role produce most of the technical artifacts specify, interrogate, verify, decide, and take responsibility Betebenner · Educational Measurement as an AI-Native Profession 4 / 10 One concrete example O N E E X A M P L E A recent methodological problem in educational growth required: 1. deriving properties of a new dependence-based model; 2. translating the mathematics into working software; 3. designing simulation studies; 4. exploring thousands of conditions; 5. diagnosing failures and constructing counterexamples; and 6. repeatedly moving back from computational results to revised mathematics. Previously: months or years, or a team with complementary specialties. With AI: a tightly coupled research loop operating on the scale of weeks. Betebenner · Educational Measurement as an AI-Native Profession 5 / 10 What became scarce? Cheaper mathematical derivation code production refactoring simulation implementation documentation first-pass critique More valuable problem formulation model specification adversarial testing recognizing nonsense deciding what evidence is enough accepting responsibility Betebenner · Educational Measurement as an AI-Native Profession 6 / 10

What became scarce? Cheaper mathematical derivation code production refactoring simulation implementation documentation first-pass critique More valuable problem formulation model specification adversarial testing recognizing nonsense deciding what evidence is enough accepting responsibility Betebenner · Educational Measurement as an AI-Native Profession 6 / 10 The unresolved problem O N E U N R E S O L V E D P R O B L E M The question is no longer whether I can do the analysis. It is whether I can warrant the analysis that AI has made possible. If AI makes it possible to produce work whose complexity exceeds what its author—or a conventional peer reviewer—can independently reconstruct, what makes the result warranted? As production becomes cheaper, does verification become the new scarce resource—and who is accountable for deciding that the evidence is sufficient? Betebenner · Educational Measurement as an AI-Native Profession 7 / 10 Backup slides Not part of the five minutes. For the dialogue, if useful. Betebenner · Educational Measurement as an AI-Native Profession 8 / 10 A broader possibility AI is collapsing the practical boundaries between mathematical theorist, statistician, programmer, simulation researcher, technical writer, and software developer. The important change is not that every step is faster. It is that the entire intellectual loop can remain continuous inside a human–AI working unit. Betebenner · Educational Measurement as an AI-Native Profession 9 / 10 Provocation for the field If this is the emerging division of labor, are we training psychometricians for the work they will actually do? What must a measurement professional be able to do personally in order to supervise, verify, and take responsibility for work that AI can execute faster (and, for some bounded tasks, better) than they can execute unaided? Betebenner · Educational Measurement as an AI-Native Profession 10 / 10