dc.contributor.author |
Yogarajan, Vithya |
|
dc.contributor.author |
Dobbie, Gill |
|
dc.contributor.author |
Pistotti, Timothy |
|
dc.contributor.author |
Bensemann, Joshua |
|
dc.contributor.author |
Knowles, Kobe |
|
dc.coverage.spatial |
Macau |
|
dc.date.accessioned |
2023-09-17T21:27:21Z |
|
dc.date.available |
2023-09-17T21:27:21Z |
|
dc.date.issued |
2023-08-21 |
|
dc.identifier.citation |
(2023, August 19-25). [Conference item]. Ethics and Trust in Human-AI Collaboration: Socio-Technical Approaches @ The 32nd International Joint Conference on Artificial Intelligence, Macau, 19 Aug 2023 - 25 Aug 2023. Proceedings of the Workshop on Ethics and Trust in Human-AI Collaboration: Socio-Technical Approaches (ETHAICS 2023) co-located with 32nd International Joint Conference on Artificial Intelligence (IJCAI 2023). 15 pages. 21 Aug 2023 |
|
dc.identifier.uri |
https://hdl.handle.net/2292/65876 |
|
dc.description.abstract |
Recent advances in artificial intelligence, including the development of highly sophisticated large language models (LLM), have proven beneficial in many real-world applications. However, evidence of inherent bias encoded in these LLMs has raised concerns about equity. In response, there has been an increase in research dealing with bias, including studies focusing on quantifying bias and developing debiasing techniques. Benchmark bias datasets have also been developed for binary gender classification and ethical/racial considerations, focusing predominantly on American demographics. However, there is minimal research in understanding and quantifying bias related to under-represented societies. Motivated by the lack of annotated datasets for quantifying bias in under-represented societies, we endeavoured to create benchmark datasets for the New Zealand (NZ) population. We faced many challenges in this process, despite the availability of three annotators. This research outlines the manual annotation process, provides an overview of the challenges we encountered and lessons learnt, and presents recommendations for future research. |
|
dc.relation.ispartof |
THE 32nd INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE |
|
dc.relation.ispartofseries |
Ethics and Trust in Human-AI Collaboration: Socio-Technical Approaches |
|
dc.rights |
Items in ResearchSpace are protected by copyright, with all rights reserved, unless otherwise indicated. Previously published items are made available in accordance with the copyright policy of the publisher. |
|
dc.rights.uri |
https://researchspace.auckland.ac.nz/docs/uoa-docs/rights.htm |
|
dc.rights.uri |
https://creativecommons.org/licenses/by/4.0/ |
|
dc.title |
Challenges in Annotating Datasets to Quantify Bias in Under-represented Society |
|
dc.type |
Conference Item |
|
dc.date.updated |
2023-08-19T10:38:54Z |
|
dc.rights.holder |
Copyright: The authors |
en |
pubs.author-url |
https://ceur-ws.org/Vol-3547/paper1.pdf |
|
pubs.finish-date |
2023-08-25 |
|
pubs.start-date |
2023-08-19 |
|
dc.rights.accessrights |
http://purl.org/eprint/accessRights/OpenAccess |
en |
pubs.subtype |
Conference Paper |
|
pubs.elements-id |
977337 |
|
pubs.org-id |
Science |
|
pubs.org-id |
School of Computer Science |
|
pubs.record-created-at-source-date |
2023-08-19 |
|