Provocation statement · Derek Briggs
Provocation statement: Derek Briggs
Open the original · source at the pinned commit · 1 passage
briggs-statement.0.35632828lines 1–16
Provocation statement: Derek Briggs **Claim:** Peer review in our field should use AI twice: reviewers audit a journal-supplied AI review before finalizing theirs, and editors get an AI check of whether the reviews cohere. In the 14 journals I checked that publish scholarship related to educational measurement and psychometrics, policies block this mainly to protect confidentiality, which a secure, journal-run tool can satisfy. **Example:** In a randomized trial at ICLR 2025, 27% of reviewers revised after seeing AI feedback; a panel judged two-thirds of the revisions better. **Unresolved problem:** When a reviewer or editor revises their decision or recommendation after reading the AI's critique, whose judgment is it? No journal policy in our field addresses that case. Biography: Please see https://www.colorado.edu/education/derek-briggs for vita and other materials