Loading...
Please wait, while we are loading the content...
Finding True Beliefs : Applying Rank-Dependent Expected Utility Theory to Proper Scoring Rules
| Content Provider | Semantic Scholar |
|---|---|
| Author | Thammaiah, Pramod |
| Copyright Year | 2011 |
| Abstract | Proper scoring rules are designed to elicit truthful probability beliefs from expected-value maximizing agents. However, there is evidence that in certain contexts agents are not expected-value maximizers. Thus, we apply rank-dependent expected utility theory, a more general model of decision-making that incorporates probability weighting and non-linear utility functions, to the analysis of the quadratic scoring rule. Current literature provides a way to find an agent’s true beliefs in the case of two outcomes. We have two main theoretical contributions. First, we prove the existence of a unique optimal report and characterize its structure. Second, we use this characterization to find an agent’s true beliefs for any number of outcomes. We demonstrate the feasibility of our methodology by conducting an experiment on Amazon Mechanical Turk, an online crowdsourcing marketplace. The empirical analysis leads to surprising results. There was no statistically significant difference in performance between the control and treatment groups. In aggregate, subjects were extremely close to bayesian beliefs and there was no evidence of bias. Additionally, there was a statistically significant decrease in performance of adjusted reports over unadjusted reports. We offer three potential explantions: subjects are expected-value maximizers for small gains, subjects did not understand the payment scheme, or the incentives were too small for them to be considered. Applying our methodology and findings to contexts with larger stakes and in-person experimentation are interesting avenues for further research. |
| File Format | PDF HTM / HTML |
| Alternate Webpage(s) | http://www.eecs.harvard.edu/econcs/pubs/Thammaiah_thesis.pdf |
| Alternate Webpage(s) | http://econcs.seas.harvard.edu/files/econcs/files/thammaiah_thesis.pdf |
| Language | English |
| Access Restriction | Open |
| Content Type | Text |
| Resource Type | Article |
