NDLI: AxGames:Towards Crowdsourcing Quality Target Determination in Approximate Computing (OPSR, Vol. 50, Issue. 2)

Content Provider	ACM Digital Library
Author	Mahajan, Divya Thwaites, Bradley Esmaeilzadeh, Hadi Amaro, Emmanuel Park, Jongse
Abstract	Approximate computing trades quality of application output for higher efficiency and performance. Approximation is useful only if its impact on application output quality is acceptable to the users. However, there is a lack of systematic solutions and studies that explore users' perspective on the effects of approximation. In this paper, we seek to provide one such solution for the developers to probe and discover the boundary of quality loss that most users will deem acceptable. We propose AxGames, a crowdsourced solution that enables developers to readily infer a statistical common ground from the general public through three entertaining games. The users engage in these games by betting on their opinion about the quality loss of the final output while the AxGames framework collects statistics about their perceptions. The framework then statistically analyzes the results to determine the acceptable levels of quality for a pair of (application, approximation technique). The three games are designed such that they effectively capture quality requirements with various tradeoffs and contexts. To evaluate AxGames, we examine seven diverse applications that produce user perceptible outputs and cover a wide range of domains, including image processing, optical character recognition, speech to text conversion, and audio processing. We recruit 700 participants/users through Amazon's Mechanical Turk to play the games that collect statistics about their perception on different levels of quality. Subsequently, the AxGames framework uses the Clopper-Pearson exact method, which computes a binomial proportion confidence interval, to analyze the collected statistics for each level of quality. Using this analysis, AxGames can statistically project the quality level that satisfies a given percentage of users. The developers can use these statistical projections to tune the level of approximation based on the user experience. We find that the level of acceptable quality loss significantly varies across applications. For instance, to satisfy 90% of users, the level of acceptable quality loss is 2% for one application (image processing) and 26% for another (audio processing). Moreover, the pattern with which the crowd responds to approximation takes significantly different shape and form depending on the class of applications. These results confirm the necessity of solutions that systematically explore the effect of approximation on the end user experience.
Starting Page	623
Ending Page	636
Page Count	14
File Format	PDF
ISSN	01635980
DOI	10.1145/2954680.2872376
Journal	ACM SIGOPS Operating Systems Review (OPSR)
Volume Number	50
Issue Number	2
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	1975-04-01
Publisher Place	New York
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Crowdsourcing Approximate computing Games-with-a-purpose Energy-efficient computing
Content Type	Text
Resource Type	Article
Subject	Computer Networks and Communications Hardware and Architecture Information Systems

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

AxGames:Towards Crowdsourcing Quality Target Determination in Approximate Computing (CARN, Vol. 44, Issue. 2)

AxGames:Towards Crowdsourcing Quality Target Determination in Approximate Computing (SIGP, Vol. 51, Issue. 4)

Proactive Control of Approximate Programs (OPSR, Vol. 50, Issue. 2)

Sidewinder:An Energy Efficient and Developer Friendly Heterogeneous Architecture for Continuous Mobile Sensing (OPSR, Vol. 50, Issue. 2)

AxGames: Towards Crowdsourcing Quality Target Determination in Approximate Computing

An Energy-interference-free Hardware-Software Debugger for Intermittent Energy-harvesting Systems (OPSR, Vol. 50, Issue. 2)

HCloud:Resource-Efficient Provisioning in Shared Cloud Systems (OPSR, Vol. 50, Issue. 2)

High-Density Image Storage Using Approximate Memory Cells (OPSR, Vol. 50, Issue. 2)

memif:Towards Programming Heterogeneous Memory Asynchronously (OPSR, Vol. 50, Issue. 2)

AxGames:Towards Crowdsourcing Quality Target Determination in Approximate Computing (OPSR, Vol. 50, Issue. 2)

Similar Documents

AxGames:Towards Crowdsourcing Quality Target Determination in Approximate Computing (CARN, Vol. 44, Issue. 2)

AxGames:Towards Crowdsourcing Quality Target Determination in Approximate Computing (SIGP, Vol. 51, Issue. 4)

Proactive Control of Approximate Programs (OPSR, Vol. 50, Issue. 2)

Sidewinder:An Energy Efficient and Developer Friendly Heterogeneous Architecture for Continuous Mobile Sensing (OPSR, Vol. 50, Issue. 2)

AxGames: Towards Crowdsourcing Quality Target Determination in Approximate Computing

An Energy-interference-free Hardware-Software Debugger for Intermittent Energy-harvesting Systems (OPSR, Vol. 50, Issue. 2)

HCloud:Resource-Efficient Provisioning in Shared Cloud Systems (OPSR, Vol. 50, Issue. 2)

High-Density Image Storage Using Approximate Memory Cells (OPSR, Vol. 50, Issue. 2)

memif:Towards Programming Heterogeneous Memory Asynchronously (OPSR, Vol. 50, Issue. 2)

AxGames:Towards Crowdsourcing Quality Target Determination in Approximate Computing (OPSR, Vol. 50, Issue. 2)