Dilbilim Araştırmalarında Açık Bilim

Esra Ataman; Ozan Can Çağlar; Bilal Kırkıcı

doi:10.18492/dad.936072

TR EN

Dilbilim Araştırmalarında Açık Bilim

Öz

Daha açık, şeffaf, tekrarlanabilir ve yeniden üretilebilir çalışmalar yürütmeye dikkat çeken açık bilimle ilgili tartışmalar, son yıllarda alanyazında küresel olarak görünürlük kazanmaktadır. Bu tartışmalar, dilbilim alanında da kendine yer bulmuş, fakat ülkemizde bu alanda açık bilime dair kayda değer bir alanyazın henüz oluşmamıştır. Bu doğrultuda, bu makale tekrarlama krizi, bilimsel sahtekarlık ve yayın yanlılığı gibi açık bilime zemin hazırlayan gelişmelerin güncel bir özetini sunarak, şeffaflık, tekrarlanabilirlik ve yeniden üretilebilirlik gibi ilkelerin dil araştırmalarında nasıl vücut bulabileceğini irdelemiştir. Dahası, makalede gizli esneklik ve sakıncalı araştırma uygulamaları sorunlarına dil araştırmaları özelinde odaklanılmış ve bu sorunların ciddiyetine karşı bir farkındalık oluşturmak amaçlanmıştır. Ayrıca, bu sorunlara karşı önerilmiş dört temel çözüm (ön kayıt, yeniden üretilebilir iş akışı, ön baskı ve veri/materyal paylaşımı) artı ve eksileriyle tartışılarak dil araştırmacılarının dikkatine sunulmuştur. Araştırmacıların bu çözüm önerilerini kendi alan ve koşullarını düşünerek değerlendirmeleri ve olabildiğince uygulamaları, dil araştırmalarında şeffaflık, tekrarlanabilirlik ve yeniden üretilebilirliği artırmaya katkı sağlayacaktır.

Anahtar Kelimeler

Open Science in Linguistic Research

Abstract

Discussions about open science, which draws attention to conducting more open, transparent, replicable, and reproducible studies, have been making an appearance in the global literature in recent years. These discussions have also found their place in linguistics, though there is no well-established literature on open science in this field in Turkey yet. Accordingly, by presenting an up-to-date summary of developments that pave the way for open science, such as the replication crisis, scientific fraud, and publication bias, this article examines how related principles such as transparency, replicability, and reproducibility can manifest themselves in linguistic research. In the article, the problems of hidden flexibility and questionable research practices are discussed within the scope of linguistic research with the aim of raising awareness about the seriousness of these problems. Four basic solutions suggested in the literature to avoid these problems (pre-registration, reproducible workflow, preprint, and data/material sharing) are presented together with their pros and cons and are brought to the language researchers' attention. It is possible to increase transparency, replicability, and reproducibility in linguistic research when researchers evaluate these proposed solutions in the literature by considering their own fields and conditions and apply them to the extent possible.

Keywords

References

Baker, M., & Dolgin, E. (2017). Cancer reproducibility project releases first results. Nature News, 541(7637), 269. https://doi.org/10.1038/541269a
Berez-Kroeker, A. L., Gawne, L., Kung, S. S., Kelly, B. F., Heston, T., Holton, G., Pulsifer, P., Beaver, D. I., Chelliah, S., Dubinsky, S., Meier, R. P., Thieberger, N., Rice, K., & Woodbury, A. C. (2018). Reproducible research in linguistics: A position statement on data citation and attribution in our field. Linguistics, 56(1), 1-18. https://doi.org/10.1515/ling-2017-0032
Camerer, C. F., Dreber, A., Forsell, E., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Almenberg, J., Altmejd, A., Chan, T., Heikensten, E., Holzmeister, F., Imai, T., Isaksson, S., Nave, G., Pfeiffer, T., Razen, M., & Wu, H. (2016). Evaluating replicability of laboratory experiments in economics. Science, 351(6280), 1433-1436. https://doi.org/10.1126/science.aaf0918
Center for Open Science. (2020). Preregistration. https://www.cos.io/initiatives/prereg
Chambers, C. (2017). The Seven Deadly Sins of Psychology. Princeton University Press; JSTOR. https://doi.org/10.2307/j.ctvc779w5
Chládková, K., & Šimáčková, Š. (2021). Distributional Learning of Speech Sounds: An Exploratory Study Into the Effects of Prior Language Experience. Language Learning, 71(1), 131-161. https://doi.org/10.1111/lang.12432
Christensen, G., Freese, J., & Miguel, E. (2019). Transparent and Reproducible Social Science Research (1. bs). University of California Press; JSTOR. https://doi.org/10.2307/j.ctvpb3xkg
Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49(12), 997-1003. https://doi.org/10.1037/0003-066X.49.12.997

Coretta, S. (2020). Open Science in phonetics and phonology [Preprint]. Open Science Framework. https://doi.org/10.31219/osf.io/4dz5t
Crüwell, S. (2019). Seven easy steps to open science: An annotated reading list. Zeitschrift Für Psychologie, 227(4), 237-248. https://doi.org/10.1027/2151-2604/a000387
Devezer, B., Navarro, D. J., Vandekerckhove, J., & Ozge Buzbas, E. (2021). The case for formal methodology in scientific reform. Royal Society Open Science, 8(3), 200805. https://doi.org/10.1098/rsos.200805
Dirnagl, U. (2020). Preregistration of exploratory research: Learning from the golden age of discovery. PLOS Biology, 18(3), e3000690. https://doi.org/10.1371/journal.pbio.3000690
Ferguson, C. J., & Heene, M. (2012). A Vast Graveyard of Undead Theories: Publication Bias and Psychological Science’s Aversion to the Null. Perspectives on Psychological Science, 7(6), 555-561. https://doi.org/10.1177/1745691612459059
Franco, A., Malhotra, N., & Simonovits, G. (2014). Publication bias in the social sciences: Unlocking the file drawer. Science, 345(6203), 1502-1505. https://doi.org/10.1126/science.1255484
Haven, T., & Van Grootel, Dr. L. (2019). Preregistering qualitative research. Accountability in Research, 26(3), 229-244. https://doi.org/10.1080/08989621.2019.1580147
Havron, N., Bergmann, C., & Tsuji, S. (2020). Preregistration in infant research—A primer. Infancy, 25(5), 734-754. https://doi.org/10.1111/infa.12353
Hoekstra, R., & Vazire, S. (2020). Hoekstra & Vazire (2020), Intellectual humility is central to science [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/edh2s
Hui, B., & Huntley, E. (2020). Embracing Open Science in Applied Linguistics: Graduate Students’ Perspectives [Preprint]. Open Science Framework. https://doi.org/10.31219/osf.io/vmtza
Ioannidis, J. P. A. (2005). Why Most Published Research Findings Are False. PLoS Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124
John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling. Psychological Science, 23(5), 524-532. https://doi.org/10.1177/0956797611430953
Kathawalla, U. K., Silverstein, P., & Syed, M. (2021). Easing Into Open Science: A Guide for Graduate Students and Their Advisors. Collabra: Psychology, 7(18684). https://doi.org/10.1525/collabra.18684
Kerr, N. L. (1998). HARKing: Hypothesizing After the Results are Known: Personality and Social Psychology Review. https://doi.org/10.1207/s15327957pspr0203_4
Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., Cemalcilar, Z., Chandler, J., Cheong, W., Davis, W. E., Devos, T., Eisner, M., Frankowska, N., Furrow, D., Galliani, E. M., … Nosek, B. A. (2014). Investigating Variation in Replicability: A “Many Labs” Replication Project. Social Psychology, 45(3), 142-152. https://doi.org/10.1027/1864-9335/a000178
Lindsay, D. S., Simons, D. J., & Lilienfeld, S. O. (2016). Research Preregistration 101. APS Observer, 29(10). https://www.psychologicalscience.org/observer/research-preregistration-101
Marsden, E., Morgan-Short, K., Thompson, S., & Abugaber, D. (2018). Replication in Second Language Research: Narrative and Systematic Reviews and Recommendations for the Field: Replication in Second Language Research. Language Learning, 68(2), 321-391. https://doi.org/10.1111/lang.12286
Marsden, E., Thompson, S., & Plonsky, L. (2017). Open science in second language acquisition research: The IRIS repository of research materials and data. SHS Web of Conferences, 38, 00013. https://doi.org/10.1051/shsconf/20173800013
Marsden, E., Mackey, A., & Plonsky, L. (2016). The IRIS Repository: Advancing research practice and methodology. Içinde Advancing Methodology and Practice: The IRIS Repository of Instruments for Research into Second Languages (ss. 1-21). Routledge. https://www.routledge.com/Advancing-Methodology-and-Practice-The-IRIS-Repository-of-Instruments-for/Mackey-Marsden/p/book/9780415833646
Mertzen, D., Lago, S., & Vasishth, S. (2021). The benefits of preregistration for hypothesis-driven bilingualism research. Bilingualism: Language and Cognition, 1-6. https://doi.org/10.1017/S1366728921000031
Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., Glennerster, R., Green, D. P., Humphreys, M., Imbens, G., Laitin, D., Madon, T., Nelson, L., Nosek, B. A., Petersen, M., Sedlmayr, R., Simmons, J. P., Simonsohn, U., & Van der Laan, M. (2014). Promoting Transparency in Social Science Research. Science, 343(6166), 30-31. https://doi.org/10.1126/science.1245317
Moshontz, H., Binion, G. E., Walton, H., Brown, B. T., & Syed, M. (2020). A Guide to Posting and Managing Preprints [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/dp4x9
Mudrak, B. (2020, February). What are Preprints, and How Do They Benefit Authors?. AJE. https://www.aje.com/arc/benefits-of-preprints-for-researchers/
Munafò, M., & Neill, J. (2016). Null is beautiful: On the importance of publishing null results. Journal of Psychopharmacology, 30(7), 585-585. https://doi.org/10.1177/0269881116638813
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600-2606. https://doi.org/10.1073/pnas.1708274114
Olsson-Collentine, A., van Assen, M. A. L. M., & Hartgerink, C. H. J. (2019). The Prevalence of Marginally Significant Results in Psychology Over Time. Psychological Science, 30(4), 576-586. https://doi.org/10.1177/0956797619830326
Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716-aac4716. https://doi.org/10.1126/science.aac4716
Roettger, T. (2021). Preregistration in experimental linguistics: Applications, challenges, and limitations. Linguistics, 0(0), 000010151520190048. https://doi.org/10.1515/ling-2019-0048
Roettger, T. , & Baer-Henney, D. (2018). Toward a replication culture: Speech production research in the classroom. PsyArXiv. https://doi.org/10.31234/osf.io/q9t7c
Roettger, T., Winter, B., & Baayen, H. (2019). Emergent data analysis in phonetic sciences: Towards pluralism and reproducibility. Journal of Phonetics, 73, 1-7. https://doi.org/10.1016/j.wocn.2018.12.001
Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638-641. https://doi.org/10.1037/0033-2909.86.3.638
Schönbrodt, F. (2019). Training students for the Open Science future. Nature Human Behaviour, 3(10), 1031-1031. https://doi.org/10.1038/s41562-019-0726-z
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359-1366. https://doi.org/10.1177/0956797611417632
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2013). P-Curve: A Key to the File Drawer (SSRN Scholarly Paper ID 2256237). Social Science Research Network. https://papers.ssrn.com/abstract=2256237
Spellman, B. A., Gilbert, E. A., & Corker, K. S. (2018). Open Science. İçinde Stevens’ Handbook of Experimental Psychology and Cognitive Neuroscience (ss. 1-47). American Cancer Society. https://doi.org/10.1002/9781119170174.epcn519
Szollosi, A., Kellen, D., Navarro, D. J., Shiffrin, R., Rooij, I. van, Zandt, T. V., & Donkin, C. (2020). Is Preregistration Worthwhile? Trends in Cognitive Sciences, 24(2), 94-95. https://doi.org/10.1016/j.tics.2019.11.009
The Retraction Watch Leaderboard. (2015, Haziran 16). Retraction Watch. https://retractionwatch.com/the-retraction-watch-leaderboard/
United Nations Educational, Scientific and Cultural Organization (UNESCO). (2016). The Future of scientific advice to the United Nations: A summary report to the Secretary-General of the United Nations from the Scientific Advisory Board.
Vasishth, S., Mertzen, D., Jäger, L. A., & Gelman, A. (2018). The statistical significance filter leads to overoptimistic expectations of replicability. Journal of Memory and Language, 103, 151-175. https://doi.org/10.1016/j.jml.2018.07.004
Vazire, S. (2019). A toast to the error detectors. Nature, 577(7788), 9-9. https://doi.org/10.1038/d41586-019-03909-2
Wicherts, J. M., Bakker, M., & Molenaar, D. (2011). Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results. PLOS ONE, 6(11), e26828. https://doi.org/10.1371/journal.pone.0026828
Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016). Degrees of Freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking. Frontiers in Psychology, 7. https://doi.org/10.3389/fpsyg.2016.01832

Details

Primary Language

Turkish

Subjects

Language Studies

Journal Section

Research Article

Authors

Esra Ataman ^*
0000-0003-2359-6620
Türkiye

Ozan Can Çağlar
0000-0002-4824-3832
Türkiye

Bilal Kırkıcı
0000-0002-6423-7774
Türkiye

Publication Date

December 31, 2021

Submission Date

May 12, 2021

Acceptance Date

October 8, 2021

Published in Issue

Year 2021 Volume: 32 Number: 2

DOI

https://doi.org/10.18492/dad.936072

IZ

https://izlik.org/JA53JX46GR

Cite

RIS / Bibtex

APA

Ataman, E., Çağlar, O. C., & Kırkıcı, B. (2021). Dilbilim Araştırmalarında Açık Bilim. Dilbilim Araştırmaları Dergisi, 32(2), 149-175. https://doi.org/10.18492/dad.936072

Cited By

DERLEM DİLBİLİM İLE GENÇLİK DİLİNE BAKIŞ: TÜRKÇE İÇİN İHTİYAÇLAR VE GELİŞİM ALANLARI

Dil Dergisi

https://doi.org/10.33690/dilder.1548981

Looking at Online Language Comprehension with the Visual World Paradigm: A Systematic and Cross-Linguistic Review of Three Decades

Dilbilim Araştırmaları Dergisi

https://doi.org/10.18492/dad.1692960