Loading...
Please wait, while we are loading the content...
Similar Documents
Comparing χ 2 tests for separability Interval estimation for the difference between a pair of differences between two proportions , and related tests
| Content Provider | Semantic Scholar |
|---|---|
| Author | Wallis, Sean |
| Copyright Year | 2017 |
| Abstract | This paper describes a series of novel statistical meta-tests for comparing experimental runs for significant difference, for conditions where experiments are carried out using Binomial or multinomial contingency statistics (χ, z, log-likelihood tests, etc.). The new tests permit us to evaluate whether experiments have failed to replicate on new data; whether a particular data source or subcorpus obtains a significantly different result than another; or whether changing experimental parameters obtains a stronger effect. Recognising when an experiment obtains a significantly different result and when it does not is an issue frequently overlooked in research publication. Papers are frequently published citing ‘p values’ or test scores suggesting a ‘stronger effect’, substituting for sound statistical reasoning. This paper sets out a series of tests which together illustrate the correct approach to this question, namely, to compute confidence intervals for differences between effect sizes and distributions. The meta-tests presented are derived mathematically from the χ test and the Wilson score interval, and cover both ‘goodness of fit’ and pairwise ‘homogeneity’ tests. Meta-tests for comparing tests with one degree of freedom (‘2 × 1’ and ‘2 × 2’ tests) are generalised to those of arbitrary size (‘r × 1’ and ‘r × c’). Finally, we compare our approach with a competing approach offered by Zar (1999), which, while straightforward to calculate, turns out to be both less powerful and less robust. |
| File Format | PDF HTM / HTML |
| Alternate Webpage(s) | http://www.ucl.ac.uk/english-usage/staff/sean/resources/comparing-x2-tests.pdf |
| Language | English |
| Access Restriction | Open |
| Content Type | Text |
| Resource Type | Article |
