いつも思うのだが、なぜ (a) は両側で (b) は片側なのか。恐らく、全く別のグラフであるかのように誤認させる目的があるのかも。 ※ (a) の横軸にもP値はあるが非表示なだけ。第2横軸にP値の目盛を表示すべき。Z=1.96 が P=0.05 なやつ。 https://t.co/f5dCPUXxx0 https://t.co/garwhJ54Ou
RT 論文 (@david_colquhoun) のグラフ (a) と (b) はビン幅が揃ってない。私の画像で示した緑色の枠が同一のビン幅 (P=0.10) である。下段グラフでビン幅を等間隔で表示すると、中段グラフで「外側ほどビン幅を広くする」に相当するので、「正規分布は一様分布」へ見せかけることができてしまう。 https://t.co/lZZUkfTDnS
More on the false discovery rate here: https://t.co/4ZsORsjrie https://t.co/NTk7LPdG6B 11/
@lakens I find it useful for teaching to show the same thing by simulations eg Figs 3 and 5 in https://t.co/aD5ObRszF4 That approach also has the advantage of making it clear that p values often give an over-optimistic view of "statistical significance".
RT @moritzheiden: "If you use p=0.05 as a criterion for claiming that you have discovered an effect you will make a fool of yourself at lea…
RT @moritzheiden: "If you use p=0.05 as a criterion for claiming that you have discovered an effect you will make a fool of yourself at lea…
"If you use p=0.05 as a criterion for claiming that you have discovered an effect you will make a fool of yourself at least 30% of the time." https://t.co/sSd2ZLQ5HO
このRT右側グラフが、完全な一様分布ではなくガタガタしてるが、試行回数が100京回だとしてもガタガタは残るはず。シミュレーションだと「試行回数が少なくてガタガタしてる」のか「試行回数は十分だが、そもそも確率分布がガタガタしてる」のか区別つかないので、やめるべき。https://t.co/f5dCPUXxx0
@TylerClibbon @AtomsksSanakan @__anoop__ @JonathanSarfat1 @ChrisJohnsonMD @TigerlillySusan @RobertKennedyJr It is certainly not meaningless. See this: https://t.co/eKEnykzTap
@ArtKorenevsky @vaillancourt_dr @jonathanstea See for instance https://t.co/z5sxTth8Hn
@TomBoylesID @MythoFobe @ChronicUTIAus @xxChronicutixx @KeatingRachelle @StephenTristram I agree – I have read it and a few others: https://t.co/eKEnykzTap https://t.co/J6K0on3T4i https://t.co/nJSJCyIcLH https://t.co/sYp700KkX0 https://t.co/KgH5rM7Yb8
@MariannaRosso1 Perhaps useful for your lecture: https://t.co/YY9a2jwfB9
であるが故に、P=0.55 と P=0.60 の間にも、その他の同じ幅の区間にも、同じ数の #P値 が存在する。これは(#帰無仮説 の) #P値 には全く #再現性 が無いことを意味してる:出現する可能性が全ての #P値 で同等である。 https://t.co/B6fHq3OBO3 https://t.co/3cgXRzfpLJ
生成された 100,000 個の #P値 の分布は図3b:その 5%(5,000 個)は確かに P=0.05 以下であるが、分布が平坦であることに注意(#統計学者 の専門用語では、#帰無仮説 が真であるなら #P値 の分布は #一様分布 である)。 https://t.co/B6fHq3OBO3 https://t.co/ZCcdq6LtdB
〜 観測値が #正規分布(#ガウス分布)になり、P=0.05 を「有意性」の閾値(α=0.05)にすると、この #仮説検定 (#NHST) の 5% が「有意」であり、それら全てが #偽陽性(#αエラー)であることが分かる。古典的な手法で知らなければならないのはこれだけである。〜 https://t.co/B6fHq3OBO3 https://t.co/44V8ps3QtP
5.更に幾つかの複雑な問題:〜 2群の真の平均値が同等なら、平均値の群間差の真値はゼロである。このような #t検定 を10万回(@david_colquhoun の #ノートPC で約3分30秒かかった)シミュレーションした群間差の分布を図3aに示す。予想どおり平均値の群間差はゼロに近い。〜 https://t.co/B6fHq3OBO3 https://t.co/aVonC3lXTw
#誤発見率 #FDR と #P値 の誤解を調査 2014 #統計学者 @david_colquhoun 元教授 @ucl 要約:P=0.05 で発見したなら少なくとも 30% は誤り。実験が #検出力 不足(n が少なすぎる)なら殆どが誤り。〜 #FDR ≤ 5% にしたいなら #3σ規定 に従うか P≤0.001 〜 @RSocPublishing https://t.co/vT4eWNNEaA
@DKshad0w @GidMK Does specificity (and thus the false positive rate), and sensitivity (and thus the false negative rate), not depend on prevalence, as explained multiple times in your own link? Again, right answer is 'Yes'. https://t.co/2MQ3CaRzzu https
@DKshad0w @c0nc0rdance @elonmusk @hot_rod_co It actually is. Again, you'll never honestly admit that, because you're a disingenuous narcissist. Same feature a lot of non-expert crypto bros like Musk have. The definitions of specificity and FPR do not ment
@derdikman @GnarlyTinmen In 2014 I published a paper in volume 1 of a new #OA journal. It didn't have an impact factor. It's had more than 300,000 pdfs (the paper was trivial so that's nothing to do with merit, just the size of the audience) https://t.co/
@blogBRHP "If you use p = 0.05 to suggest that you have made a discovery, you will be wrong at least 30% of the time. If, as is often the case, experiments are underpowered, you will be wrong most of the time." - Colquhoun 2014 https://t.co/IYg6dOO1iG A c
RT @tangming2005: 2/ An investigation of the false discovery rate and the misinterpretation of p-values https://t.co/b7S5QS8juo
2/ An investigation of the false discovery rate and the misinterpretation of p-values https://t.co/b7S5QS8juo
Spot on Here’s a link to the #drugbaron piece on false discovery rates too: https://t.co/ELORaPTLxO
I’d add an add’l Q to their list: “How concerned are we that the results are a false (+)?” Background on this topic in this very readable (& free) review by Coloquon: “An investigation of the false discovery rate and the misinterpretation of p-values”
@d_uth245 @oforbes22 The long-run performance of tests will depend on the prior probability, see here for details https://t.co/j6ZxzwcvD2
RT @david_colquhoun: Well, well. My first attempt to understand the problems with p values, published in a new journal with no impact facto…
@oforbes22 @marespadafor @aidybarnett If interested in reading a paper using a similar approach (i.e., assume x% of hypotheses are true a priori, alpha = x, and power = x, what is the false discovery rate) check out https://t.co/gll4iIYQSM
RT @david_colquhoun: Well, well. My first attempt to understand the problems with p values, published in a new journal with no impact facto…
An investigation of the false discovery rate and the misinterpretation of p-values
@jfiksel1 @oforbes22 Definitely not my intention, basically lifted from this paper https://t.co/EavlxTah5o
An investigation of the false discovery rate and the misinterpretation of p-values via @david_colquhoun https://t.co/zZOuU2drlz
RT @david_colquhoun: Well, well. My first attempt to understand the problems with p values, published in a new journal with no impact facto…
RT @david_colquhoun: Well, well. My first attempt to understand the problems with p values, published in a new journal with no impact facto…
Bibliometrics is the use of statistical methods to analyse books, articles and other publications, especially in regard with scientific contents
RT @david_colquhoun: Well, well. My first attempt to understand the problems with p values, published in a new journal with no impact facto…
RT @david_colquhoun: Well, well. My first attempt to understand the problems with p values, published in a new journal with no impact facto…
RT @david_colquhoun: Well, well. My first attempt to understand the problems with p values, published in a new journal with no impact facto…
Well, well. My first attempt to understand the problems with p values, published in a new journal with no impact factor, has just passed 300,000 pdf downloads. That's the ultimate condemnation of the uselessness of #bibliometrics. https://t.co/aD5ObRszF4
RT @ChristosArgyrop: @david_colquhoun @stephensenn This is not how your paper https://t.co/2L9Gm7NALT reads: The FDR simply builds on the…
@ErinWestgate One more that may be of interest https://t.co/gll4iIYj3e
RT @Kangerine: The more I read about p-values, the more I appreciate the line quoted below. Most recently: https://t.co/mwTHzWf8hl and: ht…
If your p-value is 0.05, your false discovery rate — at a minimum — is 0.289. Why? Because for those tests where the null hypothesis is false, 100% of positives are true — but where the null is true, 100% of positives are false. Both must be considered.
@cwhalleypub How's this for an anecdote? My first publication about p values was in Vol 1 of a new journal -it didn't even have an impact factor. It took off like wildfire (now downloaded almost 300k times). Paywalls are dead https://t.co/aD5ObRszF4
@MartinRoosli @KFosterUPenn @DavidWedege ...also nice I this area: "If you use p=0.05 to suggest that you have made a discovery, you will be wrong at least 30% of the time." https://t.co/J3i5S0bJ0M
@ryanbeed @BorisBarbour @GidMK To be clear, p<.05 does not mean 95% certainty of not due to chance. https://t.co/7I6q0bfFWy (misinterpretation 2) and: https://t.co/eKEnykzTap
RT @MichaelJSlade: A phenomenal, MD-accessible summary of why underpowered trials are a huge problem, even (and especially) when p < 0.05.…
RT @MichaelJSlade: A phenomenal, MD-accessible summary of why underpowered trials are a huge problem, even (and especially) when p < 0.05.…
RT @MichaelJSlade: A phenomenal, MD-accessible summary of why underpowered trials are a huge problem, even (and especially) when p < 0.05.…
A phenomenal, MD-accessible summary of why underpowered trials are a huge problem, even (and especially) when p < 0.05. https://t.co/QT6rtYjuVc
https://t.co/4TesaRpqwq p=0.05... Great read for those interested in how to use statistics in engineering/life sciences, and interested in reproducibility in science
RT @ueafam: 『P=0.05 を魔法のような閾値として用いると「効果が無いのに、効果がある」と主張して自分自身を馬鹿にしてしまう可能性が高いことは既に明白です。』
RT @ueafam: 『効果ゼロが真実である場合に観測値が「稀」であることを知っていても、効果がある場合でも「稀」であるかどうかを判断しなければ意味がありません。』
The more I read about p-values, the more I appreciate the line quoted below. Most recently: https://t.co/mwTHzWf8hl and: https://t.co/MP5Yf9bNkN (I found Q7 especially damning).
@brembs @BorisBarbour @mbeisen @AMartinezArias @peiferlabunc @LabLerit @eLife My first p value paper was rejected by eLife, as were comments about reproducibility. In Vol1 of Roy Soc Open Science it's had 287.000 downloads and over 400 citations (over 600
@Andy_Tattersall @LizzieGadd Exactly. An example. My first p value paper (2014) was in volume 1 of a then unknown OA journal . It's been downloaded 279,973 times. Not because it's good -just some simple-minded simulations. https://t.co/zbPxDmhz2l
@acerbialberto @HeldLeonhard https://t.co/JCY6LZ9QgS Figure 7. Even for mildly underpowered studies one would in this simple scenario expect an effect size shrinkage.
RT @david_colquhoun: @TomChivers Hmm *your* campaign? "Never, ever, use the word ‘significant’ in a paper" DC 2014 https://t.co/FCHr53C1Ne…
@TomChivers Hmm *your* campaign? "Never, ever, use the word ‘significant’ in a paper" DC 2014 https://t.co/FCHr53C1Ne and 800+ signatories here https://t.co/AxfANyQg81
@Camburi @frapink1 @bihan35 Best read you'll ever have on that topic An investigation of the false discovery rate and the misinterpretation of p-values, by David Colquhoun https://t.co/mtD8wsZhjg
"...your chance of making a fool of yourself by declaring a result to be real when it is not will be 45/125=36%. "It may go some way to explain why so many false positive tests corrupt the literature." https://t.co/IkkVWz9Gj9
@HansPetterKj @opherdonchin @learnfromerror @EJWagenmakers @f2harrell @richarddmorey @jwalkrunski @lakens @neuro_data @matloff @chrisfcarroll @StatModeling I think that you are over-complicating the question. Just do 100k t tests and count how many false p
Kick off the #PaperADayChallenge with a well-written #openaccess article on the misuse of p-values in statistics. https://t.co/amJv7OLlTJ "Never, ever, use the word ‘significant’ in a paper... Still less should you use ‘almost significant'..."
@noop_noob @wikkit Thanks! I found this article to be helpful to my understanding too! https://t.co/y0roZoMhPs
@felixjannitsch @_open_science_ @juli_tkotz And this one: An investigation of the false discovery rate and the misinterpretation of p-values (Colquhoun, 2014) https://t.co/0b3OyjySfd
@CJohnston1903 @mikegravenor @DogICUma @StephenMakin @Jopo_dr @ADAlthousePhD @andymoz78 Don’t know the thread... but this might help: https://t.co/LSkLjeqGEn
@RubinPsyc @paulpharoah @mvholmes @minouye271 @TheLancet The false discovery rate isn't a P value though. https://t.co/SIJZxy7cJm
@JusiMD @KariTikkinen @hsfi @IivoHetemaki @KantaHameenSHP @amchelsinki @helsinkiuni @NuoriLaakari @Laakariliitto @lasleh @raittioe @AleksiReito @myllarni Liitynpä jaaritteluun: Colquhounin (2014) artikkelissa on mielestäni kuvattu aika osuvasti "massascree
@tmorris_mrc @article well the p value paper in 2014 was also simulation but that was more hobby than serious work (the 270,628 pdf downloads just show what nonsense bibliometrics are) https://t.co/zbPxDmhz2l
@sjwdebates While I'd recommend these three links for best understanding what p values are and what statistical significance means as well as the best way to interpret it, (https://t.co/2VCualjmB6, https://t.co/OTq14b9YtN, https://t.co/kdkjCR5jfq)
The more detailed paper by @david_colquhoun An investigation of the false discovery: https://t.co/GJH65a7vmX 2/3
RT @ChristosArgyrop: @david_colquhoun @stephensenn This is not how your paper https://t.co/2L9Gm7NALT reads: The FDR simply builds on the…
@IoanaA_Cristea Reminds me of the discussions regarding screening for dementia in general populations, e.g. https://t.co/qeIDtanUCp section 3.
This was a beautiful read. "All of the approaches above suggest that if you use p=0.05 as a criterion for claiming that you have discovered an effect you will make a fool of yourself at least 30% of the time. This alone implies that many published claims
@liina_kaisa Näen kyllä pointin ja olen itsekin samaa mieltä, mutta ei ole ainakaan omaan silmään osunut tietoa testien sensitiivisyydestä ja spesifisyydestä? Huomioitava myös populaatiotason prevalenssi, esim "3. The screening problem": https://t.co/1gIm6
@fvguima @GigiKaupe @rittner_daniel @marcusvbp Obrigado! Acho que é esse mesmo! Aberto https://t.co/bYqhYpo3Mm Interessante também citar sobre "significância" nos artigos https://t.co/CNOwezVOAU E contradições e efeitos diferente dos achados https://t.c
RT @david_colquhoun: @Usman_skhan @alan_winfield If you want just one paper, try https://t.co/MyOwecVIpg For more background, the first (2…
RT @david_colquhoun: @Usman_skhan @alan_winfield If you want just one paper, try https://t.co/MyOwecVIpg For more background, the first (2…
RT @david_colquhoun: @Usman_skhan @alan_winfield If you want just one paper, try https://t.co/MyOwecVIpg For more background, the first (2…
@Usman_skhan @alan_winfield If you want just one paper, try https://t.co/MyOwecVIpg For more background, the first (2014) might be helpful: https://t.co/zbPxDmhz2l
@tmorris_mrc I like your emphasis on simulation: https://t.co/GvPawbePwG I used it (necessarily) in my erstwhile job https://t.co/6Hzxw8JBKW and more recently to try to work out the problems of p values: https://t.co/zbPxDmhz2l
@westwoodsam1 Don't frighten people off! It took a week or so to learn enough R to write https://t.co/zbPxDmhz2l
@BobSiegerink Colquhoun's 2013 p-value paper: https://t.co/1gIm6WBPJQ Wagenmakers et al. 2012 confirmatory research paper: https://t.co/rM8VHMZLFd
@neuronast Quizás esto te sea de ayuda a la hora de decidirlo (de Derrick Wade, editor de Clinical Rehabiltation): https://t.co/Q0jTPm8eFD