Loading...
Please wait, while we are loading the content...
Similar Documents
Quality control for comparison microtasks
| Content Provider | ACM Digital Library |
|---|---|
| Author | Venetis, Petros Garcia-Molina, Hector |
| Abstract | We study quality control mechanisms for a crowdsourcing system where workers perform object comparison tasks. We study error masking techniques (e.g., voting) and detection of bad workers. For the latter, we consider using gold-standard questions, as well as disagreement with the plurality answer. We perform experiments on Mechanical Turk that yield insights as to the role of task difficulty in quality control, and the effectiveness of the schemes. |
| Starting Page | 15 |
| Ending Page | 21 |
| Page Count | 7 |
| File Format | |
| ISBN | 9781450315579 |
| DOI | 10.1145/2442657.2442660 |
| Language | English |
| Publisher | Association for Computing Machinery (ACM) |
| Publisher Date | 2012-08-12 |
| Publisher Place | New York |
| Access Restriction | Subscribed |
| Content Type | Text |
| Resource Type | Article |
