Automated item evaluation: Predicting item acceptance and rejection using LLM-generated critiques
Leveraging DeBERTa and LLM critiques to streamline the quality control of standardized testing assessments.
Automated item evaluation (AIE) refers to the use of computational methods to assess item quality without requiring manual expert review or field testing of the items under evaluation. We aimed to build a near-comprehensive AIE model by predicting item acceptance and rejection fr...
ArXiv CS.AI
Source
Generated insight by OnAirToday AI
Automated item evaluation significantly reduces the resource-heavy burden of manual testing for large-scale assessment programs. While AI effectively identifies technical and psychometric flaws, its limitations in detecting complex bias and fairness issues highlight the necessity of keeping human experts in the loop.

Comments
Keep comments relevant and respectful. No links, advertising, self-promotion, or spam.