Examinando por Autor "Rachinger, Heiko"

Mostrando 1 - 4 de 4
  • Miniatura
    Ítem
    Accounting for sample overlap in economics meta-analyses: the generalized-weights method in practice
    (John Wiley and Sons Inc, 2025-05-06) Bom, Pedro; Rachinger, Heiko
    Meta-analyses in economics frequently exhibit considerable overlap among primary samples. If not addressed, sample overlap leads to efficiency losses and inflated rates of false positives at the meta-analytical level. In previous work, we proposed a generalized-weights (GW) approach to handle sample overlap. This approach effectively approximates the correlation structure between primary estimates using information on sample sizes and overlap degrees in the primary studies. This paper demonstrates the application of the GW method to economics meta-analyses, addressing practical challenges that are likely to be encountered. We account for variations in data aggregation levels, estimation methods, and effect size metrics, among other issues. We derive explicit covariance formulas for different scenarios, evaluate the accuracy of the approximations, and employ Monte Carlo simulations to demonstrate how the method enhances efficiency and restores the false positive rate to its nominal level.
  • Miniatura
    Ítem
    Conventional wisdom, meta-analysis, and research revision in economics
    (John Wiley and Sons Inc, 2025-05-20) Gechert, Sebastian; Mey, Bianka; Opatrny, Matej; Havránek, Tomáš ; Stanley, T. D.; Bom, Pedro; Doucouliagos, Hristos; Heimberger, Philipp; Irsova, Zuzana; Rachinger, Heiko
    Over the past several decades, meta-analysis has emerged as a widely accepted tool to understand economics research. Meta-analyses often challenge the established conventional wisdom of their respective fields. We systematically review a wide range of influential meta-analyses in economics and compare them to “conventional wisdom.” After correcting for observable biases, the empirical economic effects are typically much closer to zero and sometimes switch signs. Typically, the relative reduction in effect sizes is 45%–60%.
  • No hay miniatura disponible
    Ítem
    A generalized-weights solution to sample overlap in meta-analysis
    (John Wiley and Sons Ltd, 2020-11) Bom, Pedro; Rachinger, Heiko
    Meta-studies are often conducted on empirical findings obtained from overlapping samples. Sample overlap is common in research fields that strongly rely on aggregated observational data (eg, economics and finance), where the same set of data may be used in several studies. More generally, sample overlap tends to occur whenever multiple estimates are sampled from the same study. We show analytically how failing to account for sample overlap causes high rates of false positives, especially for large meta-sample sizes. We propose a generalized-weights (GW) meta-estimator, which solves the sample overlap problem by explicitly modeling the variance-covariance matrix that describes the structure of dependence among estimates. We show how this matrix can be constructed from information that is usually available from basic sample descriptions in the primary studies (ie, sample sizes and number of overlapping observations). The GW meta-estimator amounts to weighting each empirical outcome according to its share of independent sampling information. We use Monte Carlo simulations to (a) demonstrate how the GW meta-estimator brings the rate of false positives to its nominal level, and (b) quantify the efficiency gains of the GW meta-estimator relative to standard meta-estimators. The GW meta-estimator is fairly straightforward to implement and can solve any case of sample overlap, within or between studies. Highlights: Meta-analyses are often conducted on empirical outcomes based on samples containing common observations. Sample overlap induces a correlation structure among empirical outcomes that harms the statistical properties of meta-analysis methods. We derive the analytic conditions under which sample overlap causes conventional meta-estimators to exhibit high rates of false positives. We propose a generalized-weights (GW) solution to sample overlap, which involves approximating the variance-covariance matrix that describes the correlation structure between outcomes; we show how to construct this matrix from information typically reported in the primary studies. We conduct Monte Carlo simulations to quantify the efficiency gains of the proposed GW estimator and show how it brings the rate of false positives near its nominal level. Although we focus on meta-analyses of regression coefficients, our approach can, in principle, be modified and extended to effect sizes more commonly used in other research fields, such as Cohen's d or odds ratios.
  • No hay miniatura disponible
    Ítem
    A kinked meta-regression model for publication bias correction
    (John Wiley and Sons Ltd, 2019-12) Bom, Pedro; Rachinger, Heiko
    Publication bias distorts the available empirical evidence and misinforms policymaking. Evidence of publication bias is mounting in virtually all fields of empirical research. This paper proposes the endogenous kink (EK) meta-regression model as a novel method of publication bias correction. The EK method fits a piecewise linear meta-regression of the primary estimates on their standard errors, with a kink at the cutoff value of the standard error below which publication selection is unlikely. We provide a simple method of endogenously determining this cutoff value as a function of a first-stage estimate of the true effect and an assumed threshold of statistical significance. Our Monte Carlo simulations show that EK is less biased and more efficient than other related regression-based methods of publication bias correction in a variety of research conditions.