Abstract
Cultural heritage institutions are employing crowdsourcing techniques to enrich their collection. However, assessing the quality of crowdsourced annotations is a challenge for these institutions and manually evaluating all annotations is not feasible. We employ Support Vector Machines and feature set selectors to understand which annotator and annotation properties are relevant to the annotation quality. In addition we propose a trust model to build an annotator reputation using subjective logic and assess the relevance of both annotator and annotation properties on the reputation. We applied our models to the Steve.museum dataset and found that a subset of annotation properties can identify useful annotations with a precision of 90%. However, our studied annotator properties were less predictive.
Original language | English |
---|---|
Title of host publication | Proceedings of the 10th International Workshop on Uncertainty Reasoning for the Semantic Web (URSW 2014) co-located with the 13th International Semantic Web Conference (ISWC 2014) |
Publisher | CEUR-WS.org |
Pages | 25-36 |
Number of pages | 12 |
Publication status | Published - 2014 |
Externally published | Yes |
Event | 10th International Workshop on Uncertainty Reasoning for the Semantic Web (URSW2014) - Riva del Garda, Italy Duration: 19 Oct 2014 → 19 Oct 2014 Conference number: 10 http://ceur-ws.org/Vol-1259/ http://c4i.gmu.edu/ursw/2014/ |
Publication series
Name | CEUR Workshop Proceedings |
---|---|
Volume | 1259 |
ISSN (Print) | 1613-0073 |
Conference
Conference | 10th International Workshop on Uncertainty Reasoning for the Semantic Web (URSW2014) |
---|---|
Abbreviated title | URSW2014 |
Country/Territory | Italy |
City | Riva del Garda |
Period | 19/10/14 → 19/10/14 |
Internet address |