Skip to main content
eScholarship
Open Access Publications from the University of California

Scholarly Works (23 results)

  • Article
  • Peer Reviewed
UCLA Previously Published Works (2023)
Researchers with mediation hypotheses must consider which design to use: within-subject or between-subject? In this paper, I argue that three factors should influence design choice: validity, causality, and statistical power. Threats to validity include carry-over effects, participant awareness, measurement, and more. Causality is a core element of mediation, and the assumptions required for causal inference differ between the two designs. Between-subject designs require more restrictive no-confounder assumptions, but within-subject designs require the assumption of no carry-over effects. Statistical power should be higher in within-subject designs, but the degree and conditions of this advantage are unknown for mediation analysis. A Monte Carlo simulation compares designs under a broad range of sample sizes, effect sizes, and correlations among repeated measurements. The results show within-subject designs require about half the sample size of between-subject designs to detect indirect effects of the same size, but this difference can vary with population parameters. I provide an empirical example and R script for conducting power analysis for within-subject mediation analysis. Researchers interested in conducting mediation analysis should not select within-subject designs merely because of higher power, but they should also consider validity and causality in their decision, both of which can favor between-subject designs.
    Creative Commons 'BY' version 4.0 license
    • Article
    • Peer Reviewed
    UCLA Previously Published Works (2019)
    Moderation hypotheses appear in every area of psychological science, but the methods for testing and probing moderation in two-instance repeated measures designs are incomplete. This article begins with a short overview of testing and probing interactions in between-participant designs. Next I review the methods outlined in Judd, McClelland, and Smith (Psychological Methods 1; 366-378, 1996) and Judd, Kenny, and McClelland (Psychological Methods 6; 115-134, 2001) for estimating and conducting inference on an interaction between a repeated measures factor and a single between-participant moderator using linear regression. I extend these methods in two ways: First, the article shows how to probe interactions in a two-instance repeated measures design using both the pick-a-point approach and the Johnson-Neyman procedure. Second, I extend the models described by Judd et al. (1996) to multiple-moderator models, including additive and multiplicative moderation. Worked examples with a published dataset are included, to demonstrate the methods described throughout the article. Additionally, I demonstrate how to use Mplus and MEMORE (Mediation and Moderation for Repeated Measures; available at http://akmontoya.com ), an easy-to-use tool available for SPSS and SAS, to estimate and probe interactions when the focal predictor is a within-participant factor, reducing the computational burden for researchers. I describe some alternative methods of analysis, including structural equation models and multilevel models. The conclusion touches on some extensions of the methods described in the article and potentially fruitful areas of further research.
      Cover page: Moderation analysis in two-instance repeated measures designs: Probing methods and multiple moderator models
      • Article
      • Peer Reviewed
      UCLA Previously Published Works (2019)
      Previous research has found that interdependent self-construals are related to poorer intergroup outcomes. Here we examine interdependent self-construal specifically in relation to comfort in contexts in which people are a numeric minority (i.e., outgroup comfort), and also examine the moderating roles of racial nominalism and racial essentialism. Among a racially diverse sample ( N = 577), interdependent self-construals were related to more outgroup comfort. Two dimensions of racial nominalism—humanist and sociopolitical—were established with exploratory and confirmatory factor analyses. Humanist, sociopolitical, and essentialist beliefs about race were examined as moderators of the interdependent self-construal and outgroup comfort relationship. Among participants of color with higher sociopolitical beliefs, and unexpectedly among participants with higher essentialist beliefs, interdependent self-construal was more positively related to outgroup comfort. Findings are discussed in relation to theory on self- and group-level construals, and in relation to the role of multicultural education for fostering sociopolitical beliefs about race.
        Cover page: Construals of self and group: How racial nominalism can promote adaptive intergroup outcomes for interdependent selves
        • Article
        Library Prize for Undergraduate Research (2020)
          Cover page: Lack of Robustness of Lasso and Group Lasso with Categorical Predictors: Impact of Coding Strategy on Variable Selection and Prediction  Creative Commons 'BY-NC-ND' version 4.0 license
          • Article
          • Peer Reviewed
          UCLA Previously Published Works (2020)
          In this article, the authors describe how multiple indicators multiple cause (MIMIC) models for studying uniform and nonuniform differential item functioning (DIF) can be conceptualized as mediation and moderated mediation models. Conceptualizing DIF within the context of a moderated mediation model helps to understand DIF as the effect of some variable on measurements that is not accounted for by the latent variable of interest. In addition, useful concepts and ideas from the mediation and moderation literature can be applied to DIF analysis: (a) improving the understanding of uniform and nonuniform DIF as direct effects and interactions, (b) understanding the implication of indirect effects in DIF analysis, (c) clarifying the interpretation of the "uniform DIF parameter" in the presence of nonuniform DIF, and (d) probing interactions and using the concept of "conditional effects" to better understand the patterns of DIF across the range of the latent variable.
            Cover page: MIMIC Models for Uniform and Nonuniform DIF as Moderated Mediation Models.Creative Commons 'BY-NC' version 4.0 license
            • Article
            • Peer Reviewed
            UCLA Previously Published Works (2017)
            Researchers interested in testing mediation often use designs where participants are measured on a dependent variable Y and a mediator M in both of 2 different circumstances. The dominant approach to assessing mediation in such a design, proposed by Judd, Kenny, and McClelland (2001), relies on a series of hypothesis tests about components of the mediation model and is not based on an estimate of or formal inference about the indirect effect. In this article we recast Judd et al.'s approach in the path-analytic framework that is now commonly used in between-participant mediation analysis. By so doing, it is apparent how to estimate the indirect effect of a within-participant manipulation on some outcome through a mediator as the product of paths of influence. This path-analytic approach eliminates the need for discrete hypothesis tests about components of the model to support a claim of mediation, as Judd et al.'s method requires, because it relies only on an inference about the product of paths-the indirect effect. We generalize methods of inference for the indirect effect widely used in between-participant designs to this within-participant version of mediation analysis, including bootstrap confidence intervals and Monte Carlo confidence intervals. Using this path-analytic approach, we extend the method to models with multiple mediators operating in parallel and serially and discuss the comparison of indirect effects in these more complex models. We offer macros and code for SPSS, SAS, and Mplus that conduct these analyses. (PsycINFO Database Record
              Creative Commons 'BY' version 4.0 license
              • Thesis
              • Peer Reviewed
              UCLA Electronic Theses and Dissertations (2023)

              Moderated mediation models are used commonly in psychological research and other academic fields to model how and when effects occur. Researchers must choose which paths from the mediation model are moderated when specifying this type of model. This dissertation examines how model specification impacts statistical power and type I error rate for the index of moderated mediation. In a meta-analytic review, we found that six model specifications account for 85% of published moderated mediation analyses, so this dissertation focuses on those six models. When considering power and type I error rate, two attributes matter: the data analysis model, and the data generating process (DGP). In reference to the DGP, the data analysis model can either be correctly specified, over-specified, under-specified, or completely misspecified. A Monte Carlo simulation study was run to examine the impacts of model specification on power and type I error rate, and results were analyzed using multi-level logistic regression along with figures and tables. Over-specified models had lower statistical power to detect a significant index of moderated mediation compared to correctly specified models. Under-specified models had slightly higher power when moderation on the direct effect was omitted, but otherwise, under-specified models had much lower power than correctly specified models. Parameter bias was also unacceptably high for most under-specified models. Completely misspecified models generally still had acceptable type I error rates, with a notable exception of inflated type I error rates where moderation was omitted from the direct effect. Overall, while many published moderated mediation models may not have large enough sample sizes for adequate statistical power, over-specifying or under-specifying models can lead to lower statistical power as well, while complete model misspecification risks an inflated type I error rate.

                Cover page: Impacts of Model Specification on Statistical Power and Type I Error Rate in Moderated Mediation Analysis
                • Article
                • Peer Reviewed
                UCLA Previously Published Works (2021)
                Model fit indices are being increasingly recommended and used to select the number of factors in an exploratory factor analysis. Growing evidence suggests that the recommended cutoff values for common model fit indices are not appropriate for use in an exploratory factor analysis context. A particularly prominent problem in scale evaluation is the ubiquity of correlated residuals and imperfect model specification. Our research focuses on a scale evaluation context and the performance of four standard model fit indices: root mean square error of approximate (RMSEA), standardized root mean square residual (SRMR), comparative fit index (CFI), and Tucker-Lewis index (TLI), and two equivalence test-based model fit indices: RMSEAt and CFIt. We use Monte Carlo simulation to generate and analyze data based on a substantive example using the positive and negative affective schedule (N = 1,000). We systematically vary the number and magnitude of correlated residuals as well as nonspecific misspecification, to evaluate the impact on model fit indices in fitting a two-factor exploratory factor analysis. Our results show that all fit indices, except SRMR, are overly sensitive to correlated residuals and nonspecific error, resulting in solutions that are overfactored. SRMR performed well, consistently selecting the correct number of factors; however, previous research suggests it does not perform well with categorical data. In general, we do not recommend using model fit indices to select number of factors in a scale evaluation framework.
                  • Article
                  • Peer Reviewed
                  UCLA Previously Published Works (2023)
                  Several options exist for conducting inference on indirect effects in mediation analysis. Although methods that use bootstrapping are the preferred inferential approach for testing mediation, they are time-consuming when the test must be performed many times for a power analysis. Alternatives that are more computationally efficient are not as robust, meaning accuracy of the inferences from these methods is more affected by nonnormal and heteroskedastic data. Previous research has shown that different sample sizes are needed to achieve the same amount of statistical power for different inferential approaches with data that meet all the statistical assumptions of linear regression. By contrast, we explore how similar power estimates are at the same sample size, including when assumptions are violated. We compare the power estimates from six inferential methods for between-subjects mediation using a Monte Carlo simulation study. We varied the path coefficients, inferential methods for the indirect effect, and degree to which assumptions are met. We found that when the assumptions of linear regression are met, three inferential methods consistently perform similarly: the joint significance test, the Monte Carlo confidence interval, and the percentile bootstrap confidence interval. When the assumptions were violated, the nonbootstrapping methods tended to have vastly different power estimates compared with the bootstrapping methods. On the basis of these results, we recommend using the more computationally efficient joint significance test for power analysis only when no assumption violations are hypothesized a priori. We also recommend the joint significance test to pick an optimal starting sample size value for power analysis using the percentile bootstrap confidence interval when assumption violations are suspected.
                    Creative Commons 'BY-NC-ND' version 4.0 license
                    • Article
                    • Peer Reviewed
                    UCLA Previously Published Works (2022)
                    The bias-corrected bootstrap confidence interval (BCBCI) was once the method of choice for conducting inference on the indirect effect in mediation analysis due to its high power in small samples, but now it is criticized by methodologists for its inflated type I error rates. In its place, the percentile bootstrap confidence interval (PBCI), which does not adjust for bias, is currently the recommended inferential method for indirect effects. This study proposes two alternative bias-corrected bootstrap methods for creating confidence intervals around the indirect effect: one originally used by Stine (1989) with the correlation coefficient, and a novel method that implements a reduced version of the BCBCI's bias correction. Using a Monte Carlo simulation, these methods were compared to the BCBCI, PBCI, and Chen and Fritz (2021)'s 30% Winsorized BCBCI. The results showed that the methods perform on a continuum, where the BCBCI has the best balance (i.e., having closest to an equal proportion of CIs falling above and below the true effect), highest power, and highest type I error rate; the PBCI has the worst balance, lowest power, and lowest type I error rate; and the alternative bias-corrected methods fall between these two methods on all three performance criteria. An extension of the original simulation that compared the bias-corrected methods to the PBCI after controlling for type I error rate inflation suggests that the increased power of these methods might only be due to their higher type I error rates. Thus, if control over the type I error rate is desired, the PBCI is still the recommended method for use with the indirect effect. Future research should examine the performance of these methods in the presence of missing data, confounding variables, and other real-world complications to enhance the generalizability of these results.
                      Cover page: Correcting the Bias Correction for the Bootstrap Confidence Interval in Mediation AnalysisCreative Commons 'BY' version 4.0 license
                      Top