How reproducible p-values are? They are totally not!! I was gasped when I saw this figure, comparing p-values reported in the original studies (x-axis) and ones replicated (y-axis) for three psychology journals. https://t.co/WjMRuliESQ #AcademicTwitter
Aarts et al. describe the replication of 100 experiments reported in papers published in 2008 in three high-ranking psychology journals. (…), they find that about one-third to one-half of the original findings were also observed in the replication study. h
@eugenioproto I will send you a couple of papers of ours on this seems to say more about the field of psychology which seems in disarray - it turns out very little of what is published in the psychology journals is reproducible https://t.co/SDyVdLwvzQ
@leifsrask @jeff_jjef @michaelshermer @jordanbpeterson Given that psychology is currently in shambles, I think I'll wait until science rights itself: https://t.co/s0AEyMpQ4X
@scaldacsc @nntaleb daudz par so runa. Psihologija notiek milziga petijumu Replikacijas krize https://t.co/GihXabZ3XR
@wduyck @didbruyn @IstvanHajnal @gertdecooman @philipdutre De psychologie staat al jaren bekend voor hun betrouwbare experimenten... tot ze gerepliceerd worden, dan is er nogal wat dat niet klopt. https://t.co/DZGc1byQGn
@MaiteSaura Data from psychology, by a team that gets nuance, well worth reading: https://t.co/L5AJB9Objx Commentary: https://t.co/PiCQ6uTFL9 by @edyong209 and https://t.co/Zp24ozDsu3
☕️📑This Wednesday, March 11 from 3-4.p.m BYOTea! Following University guidelines we will be meeting via Zoom to discuss 'Estimating the reproducibility of psychological science' https://t.co/ExXlh6MqZO Zoom (Princeton Affiliates only): https://t.co/M
@joejohnstun Here's the famous study calling into question the reproducibility of psych research: https://t.co/zfrsSaDy1i The studies were not randomly selected (possible bias), relied on three scientific journals (which mostly included two fields of psy
@adam_zethraeus A broader problem here is that epistemically we're in a bad place. AFAIK, nobody wins prizes for replicating research, so a lot of "knowledge" is B.S. E.g: https://t.co/8Z0HfvySlw
@Gamal_Saadeddin 1. ساينس: https://t.co/qfjahqFPrC سجن ستانفورد: https://t.co/LVtzVrZome 3. روزنهان: https://t.co/Hj9bn8HTV2 المريض الأبيض وأسود: https://t.co/5LCdi3rlZG 4. الأوهام الإيجابية: https://t.co/7FlYLW7BjV المكتئبين أقدر على التخلص من الأوهام
Paper 2 of #156papers is "Estimating the reproducibility of psychological science" (Open Science Collaboration 2015, https://t.co/lV4tcIIlJl) Understanding reproducibility, replicability, etc, is important for understanding how we can improve #NLProc &
@lfaliaj @igorthiriez @Om_Guerrero Ioannidis : of course. Vraiment pas certain, en retour, qu’on soit si bien lotis... https://t.co/KK6ffCniKB
@realsci_DE Aber ist es nicht so, das gerade in der Psychologie Probleme in der Methodik und reproduzierbarkeit von Studien auftauchen? https://t.co/698OyWKNzn
RT @goando: 社会的プライミングに限らず、心理学の多くの研究が再現性の低さを指摘されています。(心理学の再現性危機) サイエンス誌による大規模な追試で、ハイランクとされる100の実験のうちオリジナルと同等の再現性が確認できた実験が僅か39%に留まるという事が確認され…
@IonSource @FasciottiMaira @francesarnold It varies with discipline. This paper https://t.co/eXRY4dVOyp estimated less than 50% of papers in the psychological sciences were reproducible. A Nature survey highlighted major concerns - see https://t.co/UstdGlv
RT @UKRNSwansea: Happy New Year from UKRN Swansea branch! 1st session for 2020 is on Tuesday Jan 7th at 12 in the Surf Room, Fulton House,…
RT @UKRNSwansea: Happy New Year from UKRN Swansea branch! 1st session for 2020 is on Tuesday Jan 7th at 12 in the Surf Room, Fulton House,…
Happy New Year from UKRN Swansea branch! 1st session for 2020 is on Tuesday Jan 7th at 12 in the Surf Room, Fulton House, Singleton Campus. Next theme - Reproducibility Now: Why many studies are not reproducible. Message me for PDF https://t.co/dxuBRnFgUe
Science誌の論文 https://t.co/tLGu44NYXc
@tema57 Я смотрю вы читаете, только, что вам удобно. Почитайте это https://t.co/beHKUToYxT с большой долей вероятности, эта теория попадет в те самые 70%, и уж точно эмпирическому опыту и критическому мышлению я доверяю больше.
RT @goando: 社会的プライミングに限らず、心理学の多くの研究が再現性の低さを指摘されています。(心理学の再現性危機) サイエンス誌による大規模な追試で、ハイランクとされる100の実験のうちオリジナルと同等の再現性が確認できた実験が僅か39%に留まるという事が確認され…
2. Whereas retractions are rare events (1.4 in 10,000 papers for biology and medicine, less for other fields according to LBUJ), failure to replicate in many fields is disturbingly high. https://t.co/z0v0rgwSGG
RT @souvik_neo: Earlier smaller trials with lactoferrin showed benefit. But follow-up large trial like the ELFIN showed no difference. Surp…
Earlier smaller trials with lactoferrin showed benefit. But follow-up large trial like the ELFIN showed no difference. Surprised! Probably not. Emphasizes the need for good quality replication of studies. Link to this highly cited paper https://t.co/04l7If
RT @ShaunBoustani: Less than half the studies conducted in psychology are reproducible, effects that are reproducible are only about half t…
RT @ShaunBoustani: Less than half the studies conducted in psychology are reproducible, effects that are reproducible are only about half t…
Less than half the studies conducted in psychology are reproducible, effects that are reproducible are only about half the magnitude originally described Source: https://t.co/8DHVVGvKoD
RT @KallmesKevin: Scientific communication is broken. #StartAnArgumentIn4Words #BigData #sciencetwitter @BrianNosek @OSFramework https://t.…
RT @SingReproTea: Tomorrow Thursday from 1 to 2pm we'll be talking about #Reproducibility of published research findings in our Singapore #…
RT @SingReproTea: Tomorrow Thursday from 1 to 2pm we'll be talking about #Reproducibility of published research findings in our Singapore #…
Tomorrow Thursday from 1 to 2pm we'll be talking about #Reproducibility of published research findings in our Singapore #ReproducibiliTea journal club with @gabrock94 from @SANLabNTU as our discussion leader and this target paper #OpenScience @NTUsgLibrary
RT @T_T3_M5: https://t.co/EDO7ouMUeg 100個の心理学の論文を無作為抽出、追試→36%の論文しか結果を再現できなかった https://t.co/7DrU17xM0K
https://t.co/EDO7ouMUeg 100個の心理学の論文を無作為抽出、追試→36%の論文しか結果を再現できなかった
RT @LinkopingTea: It’s a beautiful frosty morning in Linköping and we’re looking forward to another #ReproducibiliTea journal club. 11:15 i…
It’s a beautiful frosty morning in Linköping and we’re looking forward to another #ReproducibiliTea journal club. 11:15 in I:207 (I building, campus Valla). Today we’ll be discussing this paper: https://t.co/HLRgWTllOA https://t.co/RSlYVhzW9w
now being put into question because when scientists try to reproduce it, they get weaker to no results than the original study. In fact, a large study by a collaboration of many dozens of scientists (https://t.co/M3buY4GkFH ), they attempted to replicate 1
@asaplsnr @FredrikHuddinge Har för mig att denna blev omtalad också, eller så tar jag fel. Tror ska handla om psykologiska experiment som ej visat sig reproducerbara. (Har ej access här.) https://t.co/Q5fif1pV5i
RT @stdebove: Le type interviewé @BrianNosek fait partie de l’équipe de recherche qui a essayé de répliquer 100 études de psychologie pour…
Le type interviewé @BrianNosek fait partie de l’équipe de recherche qui a essayé de répliquer 100 études de psychologie pour s’apercevoir qu’un résultat significatif n’était obtenu que dans 36% des cas, contre 97% dans les 100 études originales. https://t.
À noter aussi qu’il existe des sous-disciplines en psychologie plus touchées par la crise de réplicabilité que d’autres. La psychologie sociale semble plus touchée que la psychologie cognitive par exemple : https://t.co/C6aPfeavE4
@alain_p1 Est-ce qu'on ne s'appliquerait pas la chose à soi même? https://t.co/iR7k2X43BA
@AdilSaribay @cjsotomatic Since published (4 months) Desoto got 2,092 downloads. The nonreplication has 205 news articles on it and nearly 100,000 views in the first month, 15,218 downloads... do you really not see it? https://t.co/sM9CrqNhZN https://t.c
@Keith_Laws ~50% of studies fail to replicate & the ones that do replicate with ES less than 1/2 as large. Many studies underpowered to find results reported. P-hacking & associated problems not unique to psychology, but we can't accept them as nor
心理学研究は再現性が低い、と報告した有名なサイエンス論文(グーグル3000回以上引用!)。俗説となってる心理学の結果(マシュマロ実験や20-80の法則とか)もどうなんだろ?いちいち調べるのめんどくさい。 Estimating the reproducibility of psychological science https://t.co/SkNi2ZjeRe
It is also somewhat ironic that it was @sciencemagazine that published this 2015 article on the issues of replicability in psychology and that only 40% of studies held up to further scrutiny... 2/2 https://t.co/qhQroCju4v
@MaruSilvestri No se si hay una confusión entre terapia y psicología, pero aclaro por las dudas que no es lo mismo. Por otro lado la psicología es una “ciencia” en crisis. Los mismos psicólogos lo sostienen https://t.co/6iJouY8ANO Saludos
@anty_lewiatan @LadyM30_ @blekknajt @PaniDubito Ja nie mówię o likwidacji. Ja mówię dużo ostrożniejszym podchodzeniu do statystyk w naukach społecznych od nauk przyrodniczyh i scisłych! Przecież jest wielkie halo, że większość testów psychologicznych nie j
"Innovation points out paths that are possible; replication points out paths that are likely; progress relies on both." The paper we're discussing in today's @GrazOpenScience #reproducibiliTea is full of inspirational quotes! https://t.co/ZfY9LAwhUx
RT @emljames: If you're York-based (or visiting!) and would like to join us, tomorrow we're kicking off with the Open Science Collaboration…
RT @emljames: If you're York-based (or visiting!) and would like to join us, tomorrow we're kicking off with the Open Science Collaboration…
If you're York-based (or visiting!) and would like to join us, tomorrow we're kicking off with the Open Science Collaboration (2015) paper https://t.co/V4EPubBkd1. But not always psychology-focused, all welcome, see here for this term's schedule: https://t
@DrGroquik Je ne le fais presque jamais mais je vais m'autociter https://t.co/Uyx3YyLKMW
@delamare41 @Erdnalexa @docdu16 @monosynaptik @lfaliaj @DrGroquik c'est plus que 1% mais c'est en effet assez peu reproductible https://t.co/6tgCwiPXo5
RT @KingCrocoduck: @ScientistMel All three articles are reacting to this one: https://t.co/Gg07fTRKwb
Nope. Pay wall
Posting pics in the replies. I don't have access to their full article to check their methods as they back pedal a bit in their conclusion. The title even says it's an estimation. It's certainly not definitive. Posting critique below with pics
Their first estimate suggests a difference of Pearson's r=0.2 (0.36 traditional vs 0.16 with prereg). This may sound small, but it's HUGE if you consider that it's equivalent to the mean ES found for replication attempts in the Reproducibility Project! htt
@mike_jd_daniels @DrSamMoxon @Geekazoid @Maclomaclee @justsaysinmice In fact, it seems that humans aren't a good model for humans. #RepeatabilityCrisis https://t.co/fKMfX7IodD
RT @goando: 社会的プライミングに限らず、心理学の多くの研究が再現性の低さを指摘されています。(心理学の再現性危機) サイエンス誌による大規模な追試で、ハイランクとされる100の実験のうちオリジナルと同等の再現性が確認できた実験が僅か39%に留まるという事が確認され…
@images_mc @nailbomb3 Only 1/3 - 1/2 of landmark studies are reproducible: https://t.co/oCt8cUFKty
RT @VHugoBustos: Great quote from this paper: "Innovation points out paths that are possible; replication points out paths that are likely;…
https://t.co/vNOAJuthIg この論文は心理学で売っている人があまり知られたくないモノだろうなぁ
We will be discussing this paper on 'Estimating the reproducibility of psychological science': https://t.co/orP4dP7BnM
Science study showing that a large percentage of psychological studies could not be reproduced. Author's quote "Replication can increase certainty when findings are reproduced and promote innovation when they are not" https://t.co/Yb5brdn33h
@MalikMashail Also, some evidence that surprising findings are less valid, e.g., they are less replicable in Nosek et al. (2015) https://t.co/7XtgqNGZ0j
@ucdavis @ralphhexter At @ucdavis Ioannidis: another approach to assess reproducibility is brute force attempts to redo classic papers / studies (e.g., https://t.co/cT7MzzSjF3)
RT @realsci_DE: 2015: Nosek et al. versuchen, klassische Forschungsergebnisse der Psychologie zu replizieren. Sie finden, dass sich davon l…
RT @realsci_DE: 2015: Nosek et al. versuchen, klassische Forschungsergebnisse der Psychologie zu replizieren. Sie finden, dass sich davon l…
2015: Nosek et al. versuchen, klassische Forschungsergebnisse der Psychologie zu replizieren. Sie finden, dass sich davon lediglich ca. 39% bestätigen lassen. Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. htt
@RVWetjo Ik zie het niet direct, weet jij het @lakens ? Artikel erover staat in Science https://t.co/NwwYgtDDkg
@dylanwiliam @FiveThirtyEight @cragcrest I found this an interesting paper too https://t.co/phzfabbTCt
@jon_m_bryan There is not a standard, but there is recognition that each statistical approach provides somewhat different insight. See these two papers for most in depth treatment: https://t.co/fvrc3HqUNS and https://t.co/5VSJ86avAC
@juanjobatista74 @perezreverte @juansotoivars Fuente (de pago pero más relevante) https://t.co/dBeRvDMoQA
RT @BrianNosek: Across 6 large-scale replication projects, replication rate is 90 of 190 (47%). ML1: https://t.co/845t16w5zT ML2: https://…
RT @BrianNosek: Across 6 large-scale replication projects, replication rate is 90 of 190 (47%). ML1: https://t.co/845t16w5zT ML2: https://…
Estimating the reproducibility of psychological science | Science #bishtok https://t.co/qY3o7l8niZ
RT @BrianNosek: Across 6 large-scale replication projects, replication rate is 90 of 190 (47%). ML1: https://t.co/845t16w5zT ML2: https://…
Why is this article (https://t.co/jGwroUKDEA) in @sciencemagazine not #openacess? I couldn't access this without logging in.
RT @BrianNosek: Across 6 large-scale replication projects, replication rate is 90 of 190 (47%). ML1: https://t.co/845t16w5zT ML2: https://…
RT @BrianNosek: Across 6 large-scale replication projects, replication rate is 90 of 190 (47%). ML1: https://t.co/845t16w5zT ML2: https://…
RT @BrianNosek: Replication generalizability, 3 questions: 1. What are the boundaries you generalize these replication project findings (…
Is this another supporting evidence for RCTs? Can we researchers really test for random effects?
RT @BrianNosek: Replication generalizability, 3 questions: 1. What are the boundaries you generalize these replication project findings (…
RT @BrianNosek: Across 6 large-scale replication projects, replication rate is 90 of 190 (47%). ML1: https://t.co/845t16w5zT ML2: https://…
RT @BrianNosek: Replication generalizability, 3 questions: 1. What are the boundaries you generalize these replication project findings (…
RT @BrianNosek: Replication generalizability, 3 questions: 1. What are the boundaries you generalize these replication project findings (…
RT @BrianNosek: Replication generalizability, 3 questions: 1. What are the boundaries you generalize these replication project findings (…
This is centerpiece of a long, excellent thread by @BrianNosek on replication difficulties in psychology. Bottom line, a running empirical replication rate is @ 50%. Over 90% of studies in the main social journal, JPSP, report statistically significant fi
RT @BrianNosek: Replication generalizability, 3 questions: 1. What are the boundaries you generalize these replication project findings (…
RT @BrianNosek: Replication generalizability, 3 questions: 1. What are the boundaries you generalize these replication project findings (…
Replication generalizability, 3 questions: 1. What are the boundaries you generalize these replication project findings (e.g., applies to psych, just these findings)? 2. What do you generalize (e.g., the % rate, that reproducibility is more challenging t
RT @rikunert: When I started off in Psychology research, I expected every published effect to be real & replicable. When I left 2 years ago…
RT @CTennie: If you flipped a coin to decide whether or not a reported big claim/classic finding in psychology is real, you'd give psycholo…
If you flipped a coin to decide whether or not a reported big claim/classic finding in psychology is real, you'd give psychology three percent slack.