NDLI: Toward Evaluation that Leads to Best Practices: Reconciling Dialog Evaluation in Research and Industry

Please wait, while we are loading the content...

Toward Evaluation that Leads to Best Practices: Reconciling Dialog Evaluation in Research and Industry

Content Provider	Semantic Scholar
Author	Paek, Tim
Copyright Year	2007
Abstract	Dialog evaluation is approached in different ways by research and industry. While researchers have sought commensurable evaluation metrics that allow for comparison of disparate systems with varying tasks and domains, industry engineers have focused mostly on best practices and delivering a return-on-investment to customers. In this paper, we contend that the problem of finding commensurable metrics also applies to commercial evaluation, and critically survey four candidate metrics for commensurability. Finally, in light of the problems faced by the candidate metrics, we advocate a collaborative agenda for dialog evaluation based on using statistical meta-analysis for empirically establishing best practices from any evaluation metric.
Starting Page	40
Ending Page	47
Page Count	8
File Format	PDF HTM / HTML
DOI	10.3115/1556328.1556334
Alternate Webpage(s)	http://www.aclweb.org/anthology/W/W07/W07-0306.pdf
Alternate Webpage(s)	http://aclweb.org/anthology/W/W07/W07-0306.pdf
Alternate Webpage(s)	http://aclweb.org/anthology/W07-0306
Alternate Webpage(s)	http://anthology.aclweb.org/W/W07/W07-0306.pdf
Alternate Webpage(s)	http://wing.comp.nus.edu.sg/~antho/W/W07/W07-0306.pdf
Alternate Webpage(s)	http://www.aclweb.org/anthology/W07-0306
Alternate Webpage(s)	http://aclweb.org/anthology//W/W07/W07-0306.pdf
Alternate Webpage(s)	https://doi.org/10.3115/1556328.1556334
Journal	HLT-NAACL 2007
Language	English
Access Restriction	Open
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Toward identification and adoption of best practices in algorithmic recommender systems research

Robust dialog management with n-best hypotheses using dialog examples and agenda

Best practices to foster a national information technology industry

Deep in benchmarking: using industry standards to assess a training program

Toward Best Practices in Integrating ETDs and Associated Data: UNT's Approach

Toward Best Practices in Quantitative, Qualitative, and Mixed- Method Research: A Social Justice Perspective

Transitioning to a software product family approach - challenges and best practices

From theory to practice of ecotourism: major obstacles that stand in the way and best practices that lead to success

Synthesis: review of tourism best practices in sub-saharan africa 1 world bank tourism industry: research and analysis phase ii tourism product development interventions and best practices in sub-saharan africa: (2010).

Toward Evaluation that Leads to Best Practices: Reconciling Dialog Evaluation in Research and Industry

Similar Documents

Toward identification and adoption of best practices in algorithmic recommender systems research

Robust dialog management with n-best hypotheses using dialog examples and agenda

Best practices to foster a national information technology industry

Deep in benchmarking: using industry standards to assess a training program

Toward Best Practices in Integrating ETDs and Associated Data: UNT's Approach

Toward Best Practices in Quantitative, Qualitative, and Mixed- Method Research: A Social Justice Perspective

Transitioning to a software product family approach - challenges and best practices

From theory to practice of ecotourism: major obstacles that stand in the way and best practices that lead to success

Synthesis: review of tourism best practices in sub-saharan africa 1 world bank tourism industry: research and analysis phase ii tourism product development interventions and best practices in sub-saharan africa: (2010).

Toward Evaluation that Leads to Best Practices: Reconciling Dialog Evaluation in Research and Industry