Quantitative research literature is often biased because studies that fail to find a significant effect (or that demonstrate effects that are not in the desired or expected direction) are less likely to be published. This phenomenon, termed publication bias, can cause problems when researchers attempt to synthesize results through the set of techniques known as meta-analysis. Various methods exist that estimate and correct meta-analyses for publication bias. However, no single method exists that 1) can account for continuous moderators by including them within the model, 2) allow for substantial data heterogeneity, 3) produce an adjusted mean effect size, 4) include a formal test for publication bias, and 5) allow for correction when only a small number of effects is included in the analysis. This dissertation develops a method that tries to encompass those characteristics. The model uses the beta density as a weight function that estimates the selection process in order to produce adjusted parameter estimates. The model is implemented both by maximum-likelihood (ML) and by Bayesian estimation. The utility of the model is assessed by simulations and through use on real data sets. The ML simulations indicate that the likelihood-ratio test has good Type I error performance. However, the test is not very powerful for small data sets. Coverage rates indicate that the model’s 95% confidence intervals based on adjusted parameter estimates (those that correct for bias) are more likely to contain the true parameter values than are CIs around the unadjusted parameter estimates (those that do not account for bias). Bias and root mean squared errors of the estimates are better for the adjusted mean effect than the unadjusted mean effect whenever bias is present. The ML simulations also show that the model is good at distinguishing systematic study differences from publication bias. The utility of the Bayesian implementation of the model is demonstrated in two ways: 1) when ML estimation produces nonsensical parameter estimates for real data sets, Bayesian estimation does a good job of adjusting to appropriate estimates; and 2) when bias is present in small data sets, the adjusted Bayesian parameter estimates are generally closer to the true population values than the adjusted ML estimates.

In this dissertation, I explore challenges that simulation researchers face. First, I argue that compared to using inferential models, assessing simulation convergence is the superior method to determine the number of dataset replications to use when conducting a simulation. I devise a novel way of assessing simulation convergence with rounded cumulative means and apply it to four examples alongside a more conventional, analytical technique. Second, I highlight the importance of incorporating statistical decisions into the simulation process. I illustrate that with examples of decisions surrounding model selection, convergence of an individual model’s estimates, and modifications made during preliminary statistical analyses (e.g., due to outliers or a perceived assumption failure). Third, I compare the use of continuous manipulated variables that have been discretized into levels to those generated along the full continuum of possible values. Based on linear and non-linear simulation examples, discretized manipulated variables appear more effective than continuous variables, mainly due to the taxing process of establishing simulation convergence along a continuum as opposed to a point estimate.

The conditional use of the random-effects model as the primary choice of inference is common practice in everyday meta-analysis. This method of using the random-effects model for inference only if the Q statistic test for heterogeneity is significant after having used the fixed-effects approach ignores the most important criterion for choosing the correct inference model: the nature of the question one is investigating. Furthermore, using the wrong inference model could have unfortunate consequences. It is these potential consequences that we investigated via simulation in order to help decide what model is best to use. Power and error rates for heterogeneity testing and model fit testing, and bias and efficiency of parameter estimation were the criteria for comparing procedures. We found that while, overall, fixed-effects inference is more powerful than mixed-effects inference for testing possible components of models, it also produces much higher Type I error rates when heterogeneity is present. Furthermore, when heterogeneity is present, (so that mixed-effects inference is more appropriate), the Q statistic test for heterogeneity is not significant for small variance components and effect-sizes when a Type I error has been committed in the course of model selection using fixed-effects procedures. Thus, using the conditional approach will too often lead us to using the wrong inference model. Furthermore, maximum-likelihood mixed-effects inference is found to work much better. Thus, researchers should choose wisely and weigh their options carefully when choosing an appropriate inference model.

This dissertation begins by demonstrating that publication bias can depend on factors other than statistical significance, including study characteristics like social preferences and source of funding. After providing an empirical example of differing bias patterns, the dissertation presents a weight-function model that is capable of accommodating moderators of publication bias. Subsequent chapters describe a version of the model designed for sensitivity analyses, a Bayesian version, and an R package for implementing the model. Throughout the dissertation, the performance of each version of the model is assessed via simulation. Overall, the model outperforms competing models across simulation cells, and appears to be an effective tool in cases where study characteristics affect publication bias.

Life satisfaction is universally important and a pinnacle of human pursuits. Uncontrollable factors, such as personality, influence life satisfaction. Interventions, such as volunteering, also impact life satisfaction. Previous studies have provided evidence that personality traits affect life satisfaction, that volunteerism impacts life satisfaction, and that personality traits influence volunteerism. After synthesizing these prior results, I employed methods from structural equation modeling (SEM) to explore the partial mediation of volunteerism on the effects of the Big Five personality traits on life satisfaction. Contrary to the hypothesis, the SEM did not fit the observed data well. However, the analyses were limited by highly skewed distributions of item responses. Nonetheless, there was not sufficient statistical evidence to support the claim that frequency of volunteering partially mediates the effects of the Big Five personality traits on current life satisfaction.

This dissertation is comprised of a set of four simulations investigating the model-building approach to assessing Q3 values. This method allows researchers to take a confirmatory approach to detecting violations of local item independence in item response theory (IRT) models. The first simulation deals with the 3 parameters logistic model and local item dependence of pairs. The second simulation identifies the usefulness of the approach for assessing item clusters. The third and fourth extends the previous work to polytomous data for pairs and for polytomous item clusters, respectively. The aim is to determine under what circumstances the model-building approach correctly identifies local item dependence.