The Tyranny of 0.05

A moral test of statistical significance. When the stakes are higher than Chrysler dealers, do we have cause to ignore arbitrary definitions of statistical significance?

From: Note on the presidential election in Iran, June 2009 Walter R. Mebane, Jr.

Nonetheless, pending the availability of less highly aggregated counts, it is easy to test the currently available data. A natural test is to check the distribution of the vote counts’ second significant digits against the distribution expected by Benford’s Law (Mebane 2008).

Such a test for the full set of counts for each candidate shows no significant deviations from expectations. In the following table, a test statistic χ2 2BL greater than 21.03 would indicate a deviation significant at the .05 test level (taking multiple testing—four candidates—into account; the critical value for the .10-level test would be 19.0).

The single statistic value of χ2 2BL = 17.08 for Rezaei is significant at the .05 level if the fact that statistics for four candidates are being tested is ignored. So a statistically sharp approach to statistical testing—taking the multiple testing into account—fails to provide evidence against the hypothesis that the second digits are distributed according to Benford’s Law. Tests based on the means of the second digits also fail to suggest any deviation from the second-digit Benford’s Law distribution. But arguably, in view of the χ2 2BL result for Rezaei, it’s a bit of a close call. Given the large aggregates being analyzed, such a close result warrents further examination.

Table 1: 2BL Test Statistics (Pearson chi-squared)
candidate χ2
2BL
Ahmadinejad 6.19
Rezaei 17.08
Karoubi 6.62
Mousavi 14.04

Nate Silver? Are you out there? Sphere: Related Content

Bookmark this post with:

• 
• 
• 
• 
• 
• 
• 
• 
• 

Print this post