Setting standards with multiple-choice tests: A preliminary intended-user evaluation of SmartStandardSet

Show simple item record

dc.contributor.author Brown, Gavin
dc.contributor.author Denny, P
dc.contributor.author San Jose, D
dc.contributor.author Li, E
dc.date.accessioned 2021-09-20T21:43:21Z
dc.date.available 2021-09-20T21:43:21Z
dc.date.issued 2021-10-31
dc.identifier.citation Frontiers in Education 6:735088 31 Aug 2021
dc.identifier.issn 2504-284X
dc.identifier.uri https://hdl.handle.net/2292/56595
dc.description.abstract Software that easily helps higher education instructors to remove poor quality items and set appropriate grade boundaries is generally lacking. To address these challenges, the SmartStandardSet system provides a graphical-user interface for removing defective items, weighting student scores using a two-parameter model IRT score analysis, and a mechanism for standard-setting. Stakeholders (i.e., teachers and students) from multiple faculties viewed and evaluated the system in a series of interviews and focus groups. Generally, both groups of participants reported high levels of feasibility, accuracy, and utility in SmartStandardSet’s statistical scoring of items and score calculation for test-takers. Teachers indicated the data displays would help them improve future test items; students indicated the system would be fairer and would motivate greater effort on more difficult test items. However, both groups had concerns about implementing the system without institutional policy endorsement. Students specifically were concerned that academics may set grade boundaries on arbitrary and invalid grounds. Our results provide useful insights into the perceived benefits of using the tool for standard setting, and suggest concrete next steps for gaining wider acceptance that will be the focus of future work.
dc.publisher Frontiers Media S.A.
dc.relation.ispartofseries Frontiers in Education
dc.rights Items in ResearchSpace are protected by copyright, with all rights reserved, unless otherwise indicated. Previously published items are made available in accordance with the copyright policy of the publisher.
dc.rights.uri https://researchspace.auckland.ac.nz/docs/uoa-docs/rights.htm
dc.rights.uri https://creativecommons.org/licenses/by/4.0/
dc.title Setting standards with multiple-choice tests: A preliminary intended-user evaluation of SmartStandardSet
dc.type Journal Article
dc.identifier.doi 10.3389/feduc.2021.735088
dc.date.updated 2021-08-17T22:05:49Z
dc.rights.holder Copyright: The author en
pubs.publication-status Accepted
dc.rights.accessrights http://purl.org/eprint/accessRights/OpenAccess en
pubs.subtype Article
pubs.elements-id 863246


Files in this item

Find Full text

This item appears in the following Collection(s)

Show simple item record

Share

Search ResearchSpace


Browse

Statistics