statcheck // web
  • Home
  • Documentation
  • About/FAQ
  • Contact

How to interpret output from statcheck // web

Details

statcheck uses regular expressions to find statistical results in APA format. When a statistical result deviates from APA format, statcheck will not find it. The APA formats that statcheck uses are: t(df) = value, p = value; F(df1,df2) = value, p = value; r(df) = value, p = value; [chi]2 (df, N = value) = value, p = value (N is optional, delta G is also included); Z = value, p = value. All regular expressions take into account that test statistics and p values may be exactly (=) or inexactly (< or >) reported. Different spacing has also been taken into account.

Note that the conversion from PDF (and sometimes also HTML) to plain text and extraction of statistics can result in errors. Some statistical values can be missed, especially if the notation is unconventional. It is recommended to manually check some of the results.

Also, note that a seemingly inconsistent p value can still be correct when we take into account that the test statistic might have been rounded after calculating the corresponding p value. For instance, a reported t value of 2.35 could correspond to an actual value of 2.345 to 2.354 with a range of p values that can slightly deviate from the recomputed p value. statcheck will not count cases like this as errors.

The web implementation of statcheck will return an error if a PDF that does not contain any statistical results in APA format is submitted.

Output

Value Description
Source Name of the file of which the statistic is extracted.
Statistic Character indicating the statistic that is extracted.
df1 First degree of freedom (if applicable).
df2 Second degree of freedom.
Test Comparison Reported comparison of the test statistic, when importing from pdf this will often not be converted properly.
Value Reported value of the statistic.
Reported Comparison Reported comparison, when importing from pdf this might not be converted properly.
Reported P Value The reported p-value, or NA if the reported value was NS.
Computed The recomputed p-value.
Statistical Reference Raw string of the statistical reference that is extracted.
Error? Logical.The computed p value is not congruent with the reported p value.
Decision Error Logical. The reported result is significant whereas the recomputed result is not, or vice versa.
One-sided testing? Logical. Is it likely that the reported p value resulted from a correction for one-sided testing?
1-tail in text Logical. Does the text contain the string "sided", "tailed", and/or "directional"?
APA Factor The proportion of all detected p-values that was part of a complete statistical result reported in APA format.


statcheck by Sacha Epskamp and Michèle B. Nuijten // web implementation by Sean C. Rife