NDLI: Online evaluation of bid prediction models in a large-scale computational advertising platform: decision making and insights

Content Provider	Springer Nature Link
Author	Shariat, Shahriar Orten, Burkay Dasdan, Ali
Copyright Year	2016
Abstract	Online media provides opportunities for marketers through which they can deliver effective brand messages to a wide range of audiences at scale. Advertising technology platforms enable advertisers to reach their target audience by delivering ad impressions to online users in real time. In order to identify the best marketing message for a user and to purchase impressions at the right price, we rely heavily on bid prediction and optimization models. Even though the bid prediction models are well studied in the literature, the equally important subject of model evaluation is usually overlooked or not discussed in detail. Effective and reliable evaluation of an online bidding model is crucial for making faster model improvements as well as for utilizing the marketing budgets more efficiently. In this paper, we present an experimentation framework for bid prediction models where our focus is on the practical aspects of model evaluation. Specifically, we outline the unique challenges we encounter in our platform due to a variety of factors such as heterogeneous goal definitions, varying budget requirements across different campaigns, high seasonality and the auction-based environment for inventory purchasing. Then, we introduce return on investment as a unified model performance (i.e., success) metric and explain its merits over more traditional metrics such as click-through rate or conversion rate. Most importantly, we discuss commonly used evaluation and metric summarization approaches in detail and propose a more accurate method for online evaluation of new experimental models against the baseline. Our meta-analysis-based approach addresses various shortcomings of other methods and yields statistically robust conclusions that allow us to conclude experiments more quickly in a reliable manner. We demonstrate the effectiveness of our evaluation strategy on real campaign data through some experiments.
Starting Page	37
Ending Page	60
Page Count	24
File Format	PDF
ISSN	02191377
Journal	Knowledge and Information Systems
Volume Number	51
Issue Number	1
e-ISSN	02193116
Language	English
Publisher	Springer London
Publisher Date	2016-07-27
Publisher Place	London
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	A/B test Experimentation framework Meta-analysis Real-time bidding Computational advertising Information Systems and Communication Service IT in Business
Content Type	Text
Resource Type	Article
Subject	Artificial Intelligence Information Systems Human-Computer Interaction Hardware and Architecture Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Multiknowledge for decision making

Decision-making and opinion formation in simple networks

Leveraging path information to generate predictions for parallel business processes

Distributed Smart-Home Decision-Making in a Hierarchical Interactive Smart Grid Architecture

Improving contextual advertising matching by using Wikipedia thesaurus knowledge

Real time prediction of closing price and duration of B2B reverse auctions

Fast and scalable support vector clustering for large-scale data analysis

Large-scale supervised similarity learning in networks

Approximating behavioral equivalence for scaling solutions of I-DIDs

Online evaluation of bid prediction models in a large-scale computational advertising platform: decision making and insights

Similar Documents

Multiknowledge for decision making

Decision-making and opinion formation in simple networks

Leveraging path information to generate predictions for parallel business processes

Distributed Smart-Home Decision-Making in a Hierarchical Interactive Smart Grid Architecture

Improving contextual advertising matching by using Wikipedia thesaurus knowledge

Real time prediction of closing price and duration of B2B reverse auctions

Fast and scalable support vector clustering for large-scale data analysis

Large-scale supervised similarity learning in networks

Approximating behavioral equivalence for scaling solutions of I-DIDs

Online evaluation of bid prediction models in a large-scale computational advertising platform: decision making and insights