Comprehensive Evaluation of Reference Values of Parametric and Non-Parametric Effect Size Methods for Two Independent Groups

Authors

  • Ayşegül Yabacı Tak Bezmialem Vakıf University, Faculty of Medicine, Department of Biostatistics and Medical Informatics, Istanbul, Turkey https://orcid.org/0000-0002-5813-3397
  • Ilker Ercan Bursa Uludag University, Faculty of Medicine, Department of Biostatistics, Bursa, Turkey

DOI:

https://doi.org/10.6000/1929-6029.2022.11.11

Keywords:

Effect Size, Parametric Effect Size, Non-Parametric Effect Size, Two Independent Groups

Abstract

In the field of health and other sciences, effect size (ES) provides a scientific approach to the effectiveness of treatment or intervention. The p-value indicates whether the statistical difference depends on chance, while ES gives information about the effectiveness of the treatment or intervention, even if the difference is not significant. For this reason, ES has become a very popular measure in recent years. It depends on which ES will be used based on the distribution of data and the number of groups. In this study, parametric and non-parametric ES were evaluated for two independent groups.

When the literature was examined, there were no studies aimed at evaluating the reference values of the parametric and non-parametric ES methods used for two independent groups. In this study, the reference values of parametric and non-parametric ES methods for two independent groups were re-evaluated by a simulation study. As a result, the very small reference value of parametric ES methods was determined differently from the literature. It has been seen that the reference values of non-parametric ES methods are valid in cases where the skewness is low, and new reference values have been proposed at the varying skewness level.

References

Kelley K, Preacher KJ. On effect size. Psychological Methods 2012; 17: 137. https://doi.org/10.1037/a0028086

Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd edn. Hillsdale, NJ: Lawrence Erlbaum Associates 1988.

Valentine JC, Cooper H. A systematic and transparent approach for assessing the methodological quality of intervention effectiveness research: The Study Design and Implementation Assessment Device (Study DIAD). Psychological Methods 2008; 13: 130. https://doi.org/10.1037/1082-989X.13.2.130

Sawilowsky SS. New effect size rules of thumb. Journal of Modern Applied Statistical Methods 2009; 8: 26. https://doi.org/10.22237/jmasm/1257035100

Cohen J. The statistical power of abnormal-social psychological research: a review. The Journal of Abnormal and Social Psychology 1962; 65: 145. https://doi.org/10.1037/h0045186

Peng C-YJ, Chen L-T. Beyond Cohen's d: Alternative effect size measures for between-subject designs. The Journal of Experimental Education 2014; 82: 22-50. https://doi.org/10.1080/00220973.2012.745471

Borenstein M, et al. Converting among effect sizes. Introduction to Meta-Analysis 2009; pp. 45-49.

Fleishman AI. A method for simulating non-normal distributions. Psychometrika 1978; 43: 521-532. https://doi.org/10.1007/BF02293811

Luo H. Generation of Non-normal Data: A Study of Fleishman’s Power Method, ed, Department of Statistics, Uppsala University, 2011.

Caliński T, Harabasz J. A dendrite method for cluster analysis. Communications in Statistics-theory and Methods 1974; 3: 1-27. https://doi.org/10.1080/03610927408827101

Rousseeuw PJ. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis Journal of Computational and Applied Mathematics 1987; 20: 53-65. https://doi.org/10.1016/0377-0427(87)90125-7

Ben-Shachar MS, et al. Package ‘effectsize’, 2021.

Cebeci Z, et al. Package ‘pplust’. CRAN, 2019.

Fialkowski A. SimMultiCorrData: Simulation of Correlated Data with Multiple Variable Types. 2018, URL https://CRAN. R-project. org/package= SimMultiCorrData. R package version 0.2 1, pp. p251.

Kassambara A, Mundt F. Package ‘factoextra’. Extract and Visualize the Results of Multivariate Data Analyses 2017; 76.

Mangiafico S, Mangiafico MS. Package ‘rcompanion’. Cran Repos 2017; 20: 1-71.

Rogmann JJ, Rogmann MJJ. Package ‘orddom’, 2013.

Torchiano M, Torchiano MM. Package ‘effsize’. Package “Effsize”[(accessed on 9 March 2021)] 2020.

Wickham H, et al. Package ‘readxl’ 2019.

Yarberry W. DPLYR, in CRAN Recipes, Springer, 2021; pp. 1-58. https://doi.org/10.1007/978-1-4842-6876-6_1

MacQueen J. Some methods for classification and analysis of multivariate observations, in Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, Oakland, CA, USA, 1967; pp. 281-297.

Ellis P. Thresholds for interpreting effect sizes. Retrieved January 13, 2009; pp. 2014.

Sullivan GM, Feinn R. Using effect size—or why the P value is not enough; Journal of graduate medical education 2012; 4: 279-282. https://doi.org/10.4300/JGME-D-12-00156.1

---, Primary, secondary, and meta-analysis of research. Educational Researcher 1976; 5: 3-8. https://doi.org/10.3102/0013189X005010003

Hedges LV. Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics 1981; 6: 107-128. https://doi.org/10.3102/10769986006002107

Cliff N. Dominance statistics: Ordinal analyses to answer ordinal questions. Psychological Bulletin 1993; 114: 494. https://doi.org/10.1037/0033-2909.114.3.494

Glass GV. A ranking variable analogue of biserial correlation: Implications for short-cut item analysis. Journal of Educational Measurement 1965; 2: 91-95. https://doi.org/10.1111/j.1745-3984.1965.tb00396.x

Vargha A, Delaney HD. A critique and improvement of the CL common language effect size statistics of McGraw and Wong. Journal of Educational and Behavioral Statistics 2000; 25: 101-132. https://doi.org/10.3102/10769986025002101

Downloads

Published

2022-09-19

How to Cite

Tak, A. Y., & Ercan, I. (2022). Comprehensive Evaluation of Reference Values of Parametric and Non-Parametric Effect Size Methods for Two Independent Groups. International Journal of Statistics in Medical Research, 11, 88–96. https://doi.org/10.6000/1929-6029.2022.11.11

Issue

Section

General Articles