8 Sep 2022

S1: Why replication? How is it done? Where to find replication material?

Webinar REPLICATE REPLICATE: HOW YOU CAN TAKE PART SESSION 1 discusses the basics of replication addressing: Why is the topic of replication relevant to promoting a renewed approach to Economics and other social sciences What is Philosophy of Economics, and how does it relate to replication Which purposes can it serve to replicate in the narrow sense (i.e. rerunning original studies' data and code to check results) How are results produced How to update data or how to use new data or methods to check for robustness and/or generalizability How to identify studies suitable for replication How to share results How do replications help to question published findings, detect errors, contribute to scientific progress and incentivize discussion but also how do they face resistance from authors of original studies and editors Why is replication useful and relevant to teachers and researchers, and how does the Webinar facilitate the integration of replication in your syllabus or research How can students overcome being intimidated by replication, why is it that everyone can do it, and in which way does it prepare you to produce more reliable and robust results This session includes a video-lecture available here and live Q&A. Use the discussion page to comment or ask questions in written form. SIGN UP! REPLICATION PLAN SUBMISSIONS were due on August 15, 2022. We already have a sufficient number of interesting contributions. Further ones may be accepted on a rolling basis. The webinar series may be continued after December 2022 depending on participant interest. Of course feel free to watch the videos and join the discussions also if you have not yet submitted an own replication plan. Project details and how to make the most of the Webinar here or at ReplicationWiki. Do not hesitate to address your questions to replication@uni-goettingen.de or to project organizers.

15 Sep 2022

S2: Two Sides of the Replication Coin

WEBINAR REPLICATE REPLICATE: HOW YOU CAN TAKE PART SESSION 2: Two Sides of the Replication Coin: Ex Post Reproduction and Ex Ante Documentation We will begin by drawing a distinction between two distinct, but complementary, activities: ex post reproduction and ex ante documentation.Ex post reproduction refers to reproduction by an independent investigator of empirical results reported by a researcher in a previous publication. This notion of ex post reproduction corresponds to one of the types of replication that this symposium is focused on. Ex ante documentation refers to materials--data, code, and various forms of supplementary information--that are assembled by the original researchers who conduct a study, and posted publicly with the paper or report in which they present their results. This documentation should contain everything necessary to enable an interested reader to reproduce the results of the study, easily and exactly. The "replication files" that many journals now require authors of quantitative articles to submit are examples of ex ante documentation. We will then give an overview of Project TIER, an initiative that promotes incorporating instruction in transparent and reproducible methods in the research training of undergraduate and graduate students in quantitative disciplines. The focus of Project TIER has been on ex ante documentation. We will highlight the complementarities between teaching ex ante documentation and ex post reproduction. The central message: Conducting an ex post reproduction of a previously published study is excellent preparation for students who will be constructing ex ante documentation for an empirical project of their own. Constructing ex ante documentation for an empirical project of their own is excellent preparation for students who will be conducting an ex post reproduction of a previously published study. SESSION 2: Video are available here and the part on the Open Science Framework (OSF) here, slides here. | Live Q&A Sep 15, 2022For discussion please use this session's page on the ReplicationWiki. SHORT BIOSRichard Ball is Professor of Economics, and Norm Medeiros is Associate Librarian, both at Haverford College. Ball and Medeiros have been Co-Directors of Project TIER -Teaching Integrity in Empirical Research since they founded the initiative in 2013.

29 Sep 2022

S3: Replication in Quantitative Macroeconomics

WEBINAR REPLICATE REPLICATE: HOW YOU CAN TAKE PART SESSION 3: Replication in Quantitative Macroeconomics Macroeconomics differs from other fields in Economics in both the sources of data used and in the style of computation involved. In particular, data most often comes from reputable existing datasets, such as various IMF datasets, or those run by large Government organizations. On the other hand, many Macroeconomics papers involve substantial computing, often of models that have been coded specifically for that paper, rather than using standard software. As a result, there is both a substantial overlap in issues around replication in Macroeconomics and other fields, and also some important differences. We will discuss the state of replication in Macroeconomics (embryonic) and many related questions: Why is replication important? What does a successful/failed replication mean? How to find out if a paper has been replicated? How to go about performing a replication? How to make your work more easily replicable? Has the field improved over the past decade? What is most important to improve over the coming years? We will finally discuss how to do replication yourself: choosing a paper, performing the replication, useful resources, common issues, what if you need to contact the original authors?, what to do with the replication once completed? A full working paper is available here. A recorded VIDEO PRESENTATION is available to view prior to the session: https://vimeo.com/752871416A live question and answer session was recorded: https://vimeo.com/755328940For further discussion please use this session's page on the ReplicationWiki. SHORT BIORobert Kirkby is a Macroeconomist at Victoria University of Wellington. He works on macroeconomic policy analysis as well as teaching macroeconomics. A substantial component of his research is the development of theoretical and computational tools to better perform such analysis.

13 Oct 2022

S4: Advice and Information on Computational Reproductions

WEBINAR REPLICATE REPLICATE: HOW YOU CAN TAKE PART SESSION 4: Advice and Information on Computational Reproductions: Students May Find Them Messier Than Expected Drawing on experience teaching and conducting replications this talk focuses on the most basic form of replication: the computational reproduction. This means simply trying to reproduce the numerical results of a previous study using the same data and statistical routines from that study. As it turns out, there are many potential pitfalls that come from both the original study and the replicator, that could lead to different results. Different computing environments or data versioning and formatting are also culprits. This talk reviews these pitfalls using examples from published literature and several attempts across disciplines to computationally reproduce findings published in journals. It also reports results from a crowdsourced replication, where eighty-five independent teams attempted a computational replication of results reported in an original study that is central to economics and many other disciplines on the links between policy preferences and immigration. Although when teams had the original data and code they were able to achieve a high rate of similar results (95.7%), a random half of teams did not get the original code and struggled (89.3%). What was more surprising was that exact numerical reproductions to the second decimal place were far less common (76.9% and 48.1%). This has wider implications for science obviously, but it should serve as a lesson for students using replication that things are not a clean cut as one might hope. (full paper on the crowdsourced study).Please watch the pre-recorded video for this session here. The live session will then focus on discussion.For discussion please use the session's page on the ReplicationWiki. SHORT BIONate Breznau, Postdoctoral Fellow at the University of Bremen. Researcher at the Comparative Research Center “The Global Dynamics of Social Policy”. Principal Investigator of the research project, “The Reciprocal Relationship of Public Opinion and Social Policy”. Principal Investigator of “The Crowdsourced Replication Initiative”. Open Science advocate. Open Science Fellow at Wikimedia with the project, “Giving the Results back to the Crowd”. User of preprints. Crowdsourcing and Mertonian-norm advocate. Author of the open science and crowdsourcing blog “Crowdid”.

27 Oct 2022

S6: Questionable research practices outside of the lab

WEBINAR REPLICATE REPLICATE: HOW YOU CAN TAKE PART SESSION 6: Questionable research practices outside of the lab Social scientists are increasingly interested in questionable research practices and their role in producing false and misleading results. Many scientists are prescribing open practices (e.g., open data, prospective registration) as a way to restore the credibility of academic research. But, what about the broader public significance of continued reliance on questionable research practices? What do lay people think about them? And, what happens when questionable research practices enter courtrooms and affect legal decisions? What can economists learn from this? This talk attempts to answer these questions. As we follow a flipped classroom approach, please watch the pre-recorded video of the presentation here: https://vimeo.com/753809900The live session will then focus on discussion. If you cannot make it but still want to comment or ask a question, please use the discussion of this session's page on the ReplicationWiki. SHORT BIOJason Chin. I am a Lecturer at the School of Law at the University of Sydney, the former (2020-21) President of the Association for Interdisciplinary Meta-research and Open Science (AIMOS), and the inaugural registered reports editor for Forensic Science International: Synergy. I have a PhD in Social Psychology from the University of British Columbia and a JD from the University of Toronto. Prior to returning to academia, I practiced litigation at a large international law firm and was called to the bar in both New York and Ontario. Whenever feasible, I try to offer pragmatic solutions and guidance to the challenges faced by practicing lawyers. My research has been featured in the New York Times and Sydney Morning Herald.

20 Oct 2022

S5: Student Replication on Fiscal Shocks in the Globalized World

SESSION 5: Student Replication Presentation by Franziska Strunz (PhD student at the Universität der Bundeswehr in Munich, DE) ARTICLE BEING REPLICATED:Auerbach and Gorodnichenko (2016) “Effect of Fiscal Shocks in a Globalized World”, published in the IMF Economic Review (available here: https://link.springer.com/article/10.1057/imfer.2015.15). Also: data series extended with data from 2014 to 2022. ABSTRACT:"Although theoretical models consistently predict that government spending shocks should lead to appreciation of the domestic currency, empirical studies have regularly found depreciation. Using daily data on U.S. defense spending (announced and actual payments), the paper documents that the dollar immediately and strongly appreciates after announcements about future government spending. In contrast, actual payments lead to no discernible effect on the exchange rate. It examines the responses of other variables at the daily frequency and explores how the response of the exchange rate to fiscal shocks varies over the business cycle as well as at the zero lower bound and in normal times." The replication progress will be presented by Franziska, we will discuss the results so far and how to continue. For discussion please use the session's page on the ReplicationWiki.

10 Nov 2022

S7: Reopening After Covid: A Replication and Extension

WEBINAR REPLICATE REPLICATE - HOW YOU CAN TAKE PART SESSION 7: Reopening After Covid: A Replication and Extension of Chetty et al. (2020) "The Economic Impacts of COVID-19." Presentation by Annais Gangolf, Devansh Goyal, and Samuel Ross (Haverford College) November 10, 2022 at 11am Bogotá/ 12am Pennsylvania, USA/ 5pm Lisbon/ 6pm Berlin/ 9:30pm New Dehli ABSTRACT"We replicate Section IV.A of Chetty et al. (2020), which investigates the response of consumer spending, business reopenings, and employment to state reopening policies. We employ the dataset they construct in their paper that uses micro-level data from private firms. They have not published replication materials, so we implemented their methods ourselves in Stata. We were unable to exactly reproduce Chetty et al.’s results, but our conclusions are broadly consistent with theirs. Going forward, we plan to extend their paper by applying their methodology to a broader set of reopening policies."A working paper is available here. Chetty, R., Friedman, J.N., Hendren, N., Stepner, M., and the Opportunity Insights Team (2020). "The Economic Impacts of COVID-19: Evidence from a New Public Database Built Using Private Sector Data." NBER Working Paper 27431. DOI: 10.3386/w27431 The presentation will be live — no pre-recorded video! For discussion please use this session's page on the ReplicationWiki. SHORT BIOSThe authors are undergraduates at Haverford CollegeAnnais Gangolf (Research Assistant & Impact Capital Sourcing Project Student); Devansh Goyal (Peer Tutor and Incoming Summer Analyst at Cornerstone Research); Samuel Ross (Economics Research Assistant, Professor Carola Binder)

17 Nov 2022

S8: Replication in political science

This session features a set of two presentations: Nov 17, 2022 at 8am Colorado (US)/ 10am Quito (EC), Bogotá (CO) New York (US)/ 3pm Lisbon (PT) and Dublin (IE)/ 4pm Berlin (DE)/ 8:30pm New Dehli (IN) PRESENTATION 1 byCarolina Curvale (FLACSO-Ecuador) & Gustavo Pérez-Arrobo (FLACSO-Ecuador & University of Colorado Boulder) AbstractWe explore the incidence of open science in academic research in the field of Political Science. Open science´s goal is to shed light on information about data, research procedures, and results of academic work, thus making information accessible to reviewers and the general public. This practice is not prevalent across the social sciences although there is increasing interest on the importance of reproducibility in building or rejecting theory and knowledge. However, important epistemological and methodological debates have evolved around the feasibility and desirability of adopting these practices as a standard in the discipline. We systematically collected and analyzed data on publishing requirements of the top journals in the fields at point, including pre-registration. We also provide a state of the art on the implementation of these best practices in Latin America. For more information please refer to the published article "Documentation Requirements in Political Science Journals: Moving Towards Open Access Practices", Revista de ciencia política (Santiago) 41(3), 2021, http://dx.doi.org/10.4067/S0718-090X2021005000108 SHORT BIOS Carolina Curvale is an Associate Professor of Political Science at FLACSO Ecuador. She holds a PhD in Political Science from New York University, where she also earned a master´s degree. She received her baccalaureate from San Andres University in Argentina. Her main research areas are political institutions and political economy, with a regional focus in Latin America. Her research has been published by Oxford University Press and the Economic Commission for Latin America and the Caribbean, among others. Gustavo Pérez-Arrobo is a research associate in the Political Studies Department at Facultad Latinoamericana de Ciencias Sociales, Ecuador, and a PhD student at the University of Colorado Boulder. He holds a master´s degree in comparative politics from Facultad Latinoamericana de Ciencias Sociales, Ecuador and a BSc in economics from Pontificia Universidad Católica del Ecuador. His main areas of research are political economy of development, research methodology in political science and Latin American politics. PRESENTATION 2In What Direction? Replication and Extension of Oskooii 2020 'Perceived Discrimination and Political Behavior'by Andrej Cvetić (Trinity College Dublin, IE) AbstractThis paper replicates and extends Oskooii 2020. The original paper uses the Ethnic Minority British Election Study (EMBES) dataset to study how social and political discrimination on individual level influence voting behaviour of minorities. My paper successfully replicates the original results and extends the paper by performing the same statistical procedure on the European Social Survey (ESS) wave 5 dataset for UK and Western European countries. In the extension I analyse the effect of group discrimination on voting. My results indicate group discrimination lowers the propensity to cast a ballot, which runs counter to prevalent findings in the literature. The replication paper, coding scripts in R and used datasets are available in this link. The original study that was replicated is "Perceived Discrimination and Political Behavior", Kassra A.R. Oskooii, British Journal of Political Science 50(3), 2020, pp. 867 - 892. https://doi.org/10.1017/S0007123418000133. SHORT BIO Andrej Cvetić is a Ph.D. student and Research Assistant at the Department of Political Science, Trinity College Dublin. He is interested in political psychology and immigration. Currently, his research is focused on political integration of immigrants in Western Europe. Please, use the discussion page at ReplicationWiki to comment or ask questions in written form.

1 Dec 2022

S9: Student Replication on Minimum Wage in Latin America

SESSION 9: STUDENT REPLICATION ON MINIMUM WAGE IN LATIN AMERICA Presentation by Oscar Jaramillo (Torcuato de Tella University, Buenos Aires)Dec 1, 2022 at 12pm Colorado (US)/ 2pm Quito (EC), Bogotá (CO) New York (US)/ 4pm Buenos Aires (AR)/ 7pm Lisbon (PT)/ 8pm Berlin (DE) ARTICLE BEING REPLICATED: Orlando J.Sotomayor (2021), “Can the minimum wage reduce poverty and inequality in the developing world? Evidence from Brazil”, World Development 138, https://doi.org/10.1016/j.worlddev.2020.105182 ABSTRACT: “Even though there is growing social support for higher minimum wages as anti-poverty policy tools, very little is known about their effectiveness in reducing poverty or inequality in the developing world. Latin America’s largest economy offers a fertile setting for shedding light on the issue, in being a large and data-rich country where frequent increases in the minimum wage can allow for direct estimation of influence on the distribution of income.” (Sotomayor 2021) Considering this to be an important question, this replication will use data from Ecuador incorporating some changes. To understand the developing world, we need more evidence from different countries. Since the 2000s, in Latin America, minimum wage increase was an economic tool frequently used and we should try to better understand the relationship between this policy and poverty in all Latin America countries. The replication progress will be presented by Oscar, we will discuss the results so far and how to continue. SHORT BIOOscar Jaramillo is an economist from Central University of Ecuador, currently studying for a Master's degree in economics at Torcuato de Tella University in Argentina. For discussion please use the session's page on the ReplicationWiki.

1 Mar 2023

S11: Replication from a publisher's perspective

WEBINAR REPLICATE REPLICATE: HOW YOU CAN TAKE PART Replication from a publisher's perspective March 1 (Wed), 20239 am Eau Claire (WI-US), 10am Bogotá, 3pm Lisbon and 4pm Paris(*Our apologies, due to unforseen events, this session was postponed several times. We now hope this is it! Thank you for your understanding*Feb 22 UPDATE: Due to severe weather conditions in Eau Claire, the session was postponed to March 1, Wednesday, at the same time.) Pre-recorded video now available HERE ! For discussion please use the session's page on the ReplicationWiki. SHORT BIOJack is founding editor of the International Journal of Pluralism and Economics Education. He has written three books, including the widely-read, A Handbook for Pluralist Economics Education (Routledge 2009). His most recent book, Introducing a New Economics (Pluto, 2017) reconceptualizes economics and how we should educate economists with an emphasis on sustainability and pluralism. He holds a Ph.D. in economics from the University of Notre Dame. He is a research fellow at Cusanus University in Bernkastel-Kues, Germany, and visiting lecturer at NMIMS in Hyderabad, India. He is currently finishing a novel, and writing a history of thought module for the Global Development and Environment Institute (Tufts University, USA).

15 Dec 2022

S10: Replication in Behavioural Economics

WEBINAR REPLICATE REPLICATE: HOW YOU CAN TAKE PART Replication in Behavioural Economics December 15, 2022 at 12am Bogotá and New York/ 11am Wisconsin, USA/ 12am Pennsylvania, USA/ 5pm Lisbon/ 6pm Stockholm and Berlin/ 10:30pm New Dehli ABSTRACTI discuss several recent large replication projects in behavioral economics (psychology and economics), where my coauthors and I have redone experiments published in high impact journals with new and larger samples to see whether the main result replicates. We discuss the statistical power of these replications, since true original results might be exaggerated, which might make replications underpowered if we have high power to detect 100% of the original effect size. I briefly discuss our studies with prediction markets and forecasting surveys where researchers attempt to predict these replication outcomes. Finally, I discuss ways to increase the reliability of scientific results, and ways to avoid publication bias (with Registered Reports). Please watch the pre-recorded video for this session here. The live session will then focus on discussion. For discussion please use the session's page on the ReplicationWiki. SHORT BIOI'm the Johan Björkman professor of economics at the Department of Economics, mainly doing meta-science and behavioral and experimental economics. I am an Editor at the Journal of Political Economy Microeconomics and an Associate Editor at the Journal of Political Economy. I am a Wallenberg Scholar, a member of the Royal Swedish Academy of Sciences (KVA), and a member of the Royal Swedish Academy of Engineering Sciences (IVA). I am also affiliated with the Department of Economics at the University of Innsbruck and the Credence Goods, Incentives and Behavior group.

16 Mar 2023

S12: Women's Participation in the Peace Process

WEBINAR REPLICATE REPLICATE: HOW YOU CAN TAKE PART SESSION 12: Women's Participation in the Peace Process - Replication of Krause, Krause and Bränfors March 16th (Thursday), New York & Bogotá 4pm; Lisbon 9pm; Paris 10pm; March 17th (Friday), 8am Sydney In this session Lenka presents a replication of the 2018 article titled "Women’s Participation in Peace Negotiations and the Durability of Peace" by Jana Krause, Werner Krause and Piia Bränfors. https://doi.org/10.1080/03050629.2018.1492386 Main reserach question: Does women's participation in the peace process lead to more peace? You can watch the PRE-RECORDED VIDEO for this session HERE. The live session will be a Q&A focusing on discussion! For a written discussion of the session, please use the session's page on the ReplicationWiki. SHORT BIOLenka Olejníková, PhD, is a researcher and Sessional Academic at the School of Social Sciences at UNSW Sydney. Her research focuses on the utility and limitations of empirical modelling and the study of gender, utilising insights from statistics, philosophy of science, and feminist theory.