NDLI: Sample Size Analysis for Confidence Interval Estimation of Performance Metrics in ATR Evaluation

Content Provider	IEEE Xplore Digital Library
Author	Jun He Hongzhong Zhao Qiang Fu
Copyright Year	2007
Description	Author affiliation: Nat. Univ. of Defense Technol., Changsha (Jun He; Hongzhong Zhao; Qiang Fu)
Abstract	In this paper, we address the problem of sample size requirement for confidence interval estimation of performance metrics in ATR evaluation. The sample size of test data in ATR evaluation application has been reviewed firstly. We choose the Bayesian method to analyze the problem and select "minimum length criterion" to obtain the shortest confidence interval (CI) with the same estimation accuracy. The binomial probability density function is regarded as the likelihood function to calculate the posterior distribution about the metrics. Then CI accuracy requirement in ATR evaluation is discussed. Because the posterior distribution of the metrics depends on the test result, criteria to eliminate the uncertainty from the test result must be considered. The worst outcome (WOC) criterion is chosen to calculate sample sizes for various CI accuracy requirements and the corresponding sample sizes are listed in Table I. Table I shows that the sample size is very large when the CI accuracy requirement is high. Two approaches (specifications and prior information) to reduce the sample size are proposed and discussed. The absolute number of the sample size reduction is great when using the specifications in ATR evaluation applications. Table II contains those minimum sample sizes with various specifications, and its comparison with table I is shown. Whereas the sample size reduction is not large when using prior information because precise prior information about the metrics is always absent. The approximate sample size reductions can be got from table III and IV when considering beta distribution as the prior distribution.
Starting Page	585
Ending Page	589
File Size	2611400
Page Count	5
File Format	PDF
ISBN	1424402832
ISSN	10975659
DOI	10.1109/RADAR.2007.374284
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2007-04-17
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Performance analysis Measurement Testing Bayesian methods Computational Intelligence Society Statistical analysis Statistical distributions Uncertainty Helium Laboratories
Content Type	Text
Resource Type	Article
Subject	Electrical and Electronic Engineering

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Analysis of Key Comparisons Incorporating Knowledge About Bias

Confidence intervals for reliability-growth models with small sample-sizes

A statistical model to the expression of uncertainty and confidence in measurement

Sample size determination using ROC analysis (2007)

The Prediction and Measurement of System Availability: A Bayesian Treatment

Improved batching for confidence interval construction in steady-state simulation

Recursive non-linear bayesian state estimation and model recognition with application to contouring tasks

The Relation between Sample Size and Confidence in Test-to-Failure Reliability Programs

Experimental Uncertainty Estimation and Statistics for Data Having Interval Uncertainty (2007)

Sample Size Analysis for Confidence Interval Estimation of Performance Metrics in ATR Evaluation

Similar Documents

Analysis of Key Comparisons Incorporating Knowledge About Bias

Confidence intervals for reliability-growth models with small sample-sizes

A statistical model to the expression of uncertainty and confidence in measurement

Sample size determination using ROC analysis (2007)

The Prediction and Measurement of System Availability: A Bayesian Treatment

Improved batching for confidence interval construction in steady-state simulation

Recursive non-linear bayesian state estimation and model recognition with application to contouring tasks

The Relation between Sample Size and Confidence in Test-to-Failure Reliability Programs

Experimental Uncertainty Estimation and Statistics for Data Having Interval Uncertainty (2007)

Sample Size Analysis for Confidence Interval Estimation of Performance Metrics in ATR Evaluation