{"id":1966,"date":"2016-10-27T19:35:47","date_gmt":"2016-10-27T18:35:47","guid":{"rendered":"http:\/\/www.jutze.com\/?p=1966"},"modified":"2016-10-27T19:35:47","modified_gmt":"2016-10-27T18:35:47","slug":"accuracy-readability-and-mplus","status":"publish","type":"post","link":"https:\/\/www.jutze.com\/?p=1966","title":{"rendered":"Accuracy, Readability, and Mplus"},"content":{"rendered":"<p>The idea to create an algorithm that automatically scans scientific articles for the results of common statistical tests and evaluates the accuracy of these results seems straight-forward. <a href=\"http:\/\/statcheck.io\/\" target=\"_blank\">Statcheck<\/a> performs this, well, stat check. Now a lot of available papers have been <a href=\"http:\/\/retractionwatch.com\/2016\/09\/02\/heres-why-more-than-50000-psychology-studies-are-about-to-have-pubpeer-entries\/\" target=\"_blank\">automatically evaluated<\/a> and the outcomes were posted on <a href=\"http:\/\/pubpeer.com\" target=\"_blank\">PubPeer<\/a>.<\/p>\n<p>So far, none of <a href=\"https:\/\/pubpeer.com\/search?q=johannes+schult\" target=\"_blank\" rel=\"noopener\">the (two) papers on pubpeer I co-authored<\/a> raised an error flag. That&#8217;s reassuring. I went and (stat)checked my other publications, and behold: There was indeed an inconsistency in one of them. In <a title=\"Schult, J., Fischer, F. T., &amp; Hell, B. (2016). Tests of scholastic aptitude cover reasoning facets sufficiently. European Journal of Psychological Assessment, 32(3), 215-219. doi:10.1027\/1015-5759\/a000247\" href=\"http:\/\/dx.doi.org\/10.1027\/1015-5759\/a000247\" target=\"_blank\">Schult et al. (2016)<\/a> I reported &#8220;chi-square(33) = 59.11, <em>p<\/em> = .004&#8243;. Statcheck expected <em>p<\/em> = .003. The cause of this discrepancy is the rounding of rounded results. The Mplus output showed a chi-square value of 59.109 and a <em>p<\/em> value of 0.0035. I rounded both values to make the results more readable, accepting that, for example, a value such as 0.00347something would be mistakenly rounded to 0.004 instead of 0.003. For the record: Whenever a test statistic&#8217;s <em>p<\/em> value is close to the chosen alpha level, I do use all available decimal places to evaluate the decision of statistical significance. Of course, I could just report all available digits all the time. Still, that smells of pseudo-accuracy, plus I like to think that I write for human readers, not for computer algorithms.<\/p>\n<p>What&#8217;s the take home message here? I won&#8217;t be surprised when this error\/discrepancy\/inconsistency (what&#8217;s in a name?) is discovered and posted by the big machine. I will keep writing my papers with care, double-checking the results etc. (something my senior authors always condoned and enforced). And did I mention that I put <a href=\"https:\/\/www.jutze.com\/research\/\">replication materials<\/a> online (unless privacy\/copyright laws or, sadly, busyness prevent me from doing so)?<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The idea to create an algorithm that automatically scans scientific articles for the results of common statistical tests and evaluates the accuracy of these results seems straight-forward. Statcheck performs this, well, stat check. Now a lot of available papers have been automatically evaluated and the outcomes were posted on PubPeer. So far, none of the [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[16],"tags":[775,772,776,201,771,773,774,770],"class_list":["post-1966","post","type-post","status-publish","format-standard","hentry","category-data-analysis","tag-accuracy","tag-error","tag-mplus","tag-psychology","tag-reproducibility","tag-research","tag-rounding","tag-statcheck"],"_links":{"self":[{"href":"https:\/\/www.jutze.com\/index.php?rest_route=\/wp\/v2\/posts\/1966","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.jutze.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.jutze.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.jutze.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.jutze.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1966"}],"version-history":[{"count":1,"href":"https:\/\/www.jutze.com\/index.php?rest_route=\/wp\/v2\/posts\/1966\/revisions"}],"predecessor-version":[{"id":1967,"href":"https:\/\/www.jutze.com\/index.php?rest_route=\/wp\/v2\/posts\/1966\/revisions\/1967"}],"wp:attachment":[{"href":"https:\/\/www.jutze.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1966"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.jutze.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1966"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.jutze.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1966"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}