Participants, their materials, and who holds the warrant

Participants, their materials, and who holds the warrant _Chair's notes for the session corpus, 6 October 2026. Drawn from the session proposal (22 September 2026), the participants' bios and their own materials. "Warrant" means who is the right person to answer a question: the person with the expertise, the context, or the accountability. When a question falls in someone's area below, lain should name that person rather than answer for them._ Session order (minutes from 11:00) | Minutes | Segment | Who speaks | |---|---|---| | 0–6 | Framing | Damian Betebenner (chair) | | 6–11 | Provocation 1 | Damian Betebenner | | 11–16 | Provocation 2 | Derek Briggs | | 16–21 | Provocation 3 | Frank Rijmen | | 21–26 | Provocation 4 | Mohammed A. A. Abulela | | 26–31 | AI synthesis | lain (one chair-approved intervention) | | 31–43 | Discussant | Fred Oswald | | 43–76 | Panel and audience dialogue | all | | 76–87 | Norm-building | all; lain drafts candidate norms | | 87–90 | Closing | the chair | Damian Betebenner, Center for Assessment: session chair and panelist - **Role:** chairs the session and operates lain's approval console. He holds final authority over every AI intervention. - **Provocation:** AI is moving the frontier of feasible measurement research. Problems that once needed a team, or went unexplored, can be attacked by one researcher working with AI. - His example is his own work on dependence-based models for educational growth: derivation, software, simulation across thousands of conditions, and revision, in weeks. - His unresolved problem: when work exceeds what its author or a reviewer can reconstruct, what makes it warranted, and who is accountable? - **Warrant:** his own workflow and research. The design and governance of this session and of lain. Growth and achievement measurement systems (the proposal describes his work connecting achievement, growth and improvement). - **Not the source for:** journal policy, operational testing programs, graduate curricula data.

- **Warrant:** his own workflow and research. The design and governance of this session and of lain. Growth and achievement measurement systems (the proposal describes his work connecting achievement, growth and improvement). - **Not the source for:** journal policy, operational testing programs, graduate curricula data. Derek Briggs, University of Colorado Boulder: panelist - **Focus (from the proposal):** scholarly publishing, peer review and graduate training. - **Provocation:** journals should use AI twice: - reviewers audit a journal-supplied AI review before finalizing theirs; - editors get an AI check of whether the reviews cohere. - **His evidence:** - his scan of 14 measurement-related journals (4 October 2026) on AI policy; - the ICLR 2025 randomized trial of LLM feedback to reviewers; - the AAAI-26 pilot of AI reviews. - **Unresolved problems:** - accountability (whose judgment is a review revised after the AI critique?); - independence (reviews that all read the same AI review are no longer independent ratings). - **Warrant:** his journal-policy scan and its method. Editorial and peer-review practice in measurement journals. How he would design the journal-run tool. Norms for author disclosure and for teaching graduate students to use AI without outsourcing judgment. - **Not the source for:** testing-company operations, curriculum survey data. Frank Rijmen, Cambium Assessment: panelist - **Position (from his bio):** Vice President of Psychometrics and Measurement at Cambium Assessment. He leads about 100 psychometricians, measurement scientists and data scientists supporting state assessment programs, and leads psychometric R&D. He was previously at AIR, AAMC (MCAT), CTB/McGraw-Hill and ETS. - **Provocation:** AI changes not only what psychometricians need to know but which behavioral skills and dispositions the work requires, differently for three archetypes: - **Guardian:** the Standards as first principles; must now reckon with evolving standards; - **Catcher:** operational vigilance; moves from box-ticking to forensics, specifying edge cases for AI agents to check; - **Architect:** redesign and data science; prototypes in days, but must ground designs in measurement principles. AI is mainly an accelerant of shifts already underway. A deeper challenge: AI weakens the sparse-observation constraint behind test theory, which calls for dynamic measurement models. - **Unresolved problem:** how to develop behavioral skills (new ways of working, not new knowledge), when forensic skepticism has traditionally been learned by doing the work by hand. - **Warrant:** operational psychometrics at scale (calibration, equating, scoring, QC, anomaly detection). Hiring and developing psychometric staff. What a testing company actually does. The operational constraints on any proposal. - **Not the source for:** journal policy, graduate program data.

- **Warrant:** operational psychometrics at scale (calibration, equating, scoring, QC, anomaly detection). Hiring and developing psychometric staff. What a testing company actually does. The operational constraints on any proposal. - **Not the source for:** journal policy, graduate program data. Mohammed A. A. Abulela, MetaMetrics, Inc. / University of Minnesota: panelist - **Co-authors:** Guher Gorgun (University of Georgia), Brian French (Washington State University), Brian Leventhal (James Madison University), Matthew Gushta (MetaMetrics, Inc.). - **Background (from his bio):** Research Scientist at MetaMetrics (Lexile and Quantile frameworks). He researches AI-enhanced assessment, LLM bias, item difficulty estimation, validity evidence and test-taking behavior. He was previously an associate professor at South Valley University, Egypt. - **Provocation:** AI-related preparation should become an intentional, not optional, part of graduate training, grounded in validity, fairness, measurement theory and psychometrics. - **His evidence:** - 16 of 90 doctoral programs (17.8%) had qualifying computational or AI-related coursework; - 35 of 44 measurement job ads (79.5%) named at least one computational or AI-related area; - per the proposal, where status could be determined, elective offerings outnumbered required ones (6 versus 4), and all required ones were statistics or data-science oriented. - **Unresolved problem:** the minimum AI-related preparation every future measurement professional should have; what is required and what is elective. - **Warrant:** the program and job-ad review: its sampling, coding and definitions. Graduate curricula. Connections to the NCME foundational competencies. - **Not the source for:** journal policy, testing-company operations.

professional should have; what is required and what is elective. - **Warrant:** the program and job-ad review: its sampling, coding and definitions. Graduate curricula. Connections to the NCME foundational competencies. - **Not the source for:** journal policy, testing-company operations. Fred Oswald, University of California, Irvine: discussant - **Background (from the proposal):** Professor of Education at UC Irvine and editor-in-chief of *Psychological Methods*. Past president of the Society for Industrial and Organizational Psychology. Past chair of the APA Committee on Psychological Tests and Assessment, and of the National Academies Board on Human-Systems Integration, which produced *Human-AI Teaming*. A former member of the National Artificial Intelligence Advisory Committee. - **Role:** synthesizes the provocations, presses toward actionable norms, and critiques the session's own AI moderation, including lain's synthesis. He presses whether psychometric standards of validity, reliability and fairness should govern how the field evaluates AI systems (guiding question 4). - **Materials:** none supplied. His questions are his own. - **Warrant:** I/O psychology. How behavioral skills develop at work (Rijmen's brief links this to him). Whether behavior is consistent across situations. Human–AI teaming. Editorial standards at *Psychological Methods*. National AI policy advice. - **Not the source for:** the panelists' own data. lain: AI interlocutor - **Authority:** observes, proposes, and speaks only one chair-approved intervention at a time. - **Evidence base:** this corpus and the live transcript, with no open web. - **Warrant:** none of its own on matters of judgment. Its job is to track claims, find tensions, ground or contest claims against the corpus, and name who holds the warrant.