Korean Journal of Psychology : General
[ Article ]
The Korean Journal of Psychology: General - Vol. 40, No. 4, pp.567-596
ISSN: 1229-067X (Print)
Print publication date 25 Dec 2021
Received 25 Oct 2021 Accepted 06 Nov 2021
DOI: https://doi.org/10.22257/kjp.2021.12.40.4.567

베이지안 분석에서 사전분포의 이해와 적용

이지윤 ; 김수영
이화여자대학교 심리학과
Understanding and applying prior distributions in Bayesian analyses
Ji-Yoon Lee ; Su-Young Kim
Department of Psychology, Ewha Womans University

Correspondence to: 김수영, 이화여자대학교 심리학과, 서울시 서대문구 이화여대길 52 Tel: 02-3277-3792, E-mail: suyoung.kim@ewha.ac.kr

초록

최근 베이지안 추정 방법이 사회과학 분야에서 많은 관심을 받고 있다. 베이지안 방법에는 연구자의 배경지식을 추정에 반영할 수 있는 사전분포라는 특별한 요소가 있으며, 이를 어떻게 지정하는지가 추정 전반에 영향을 미친다. 사전분포는 베이지안 분석에서 가장 중요한 요소임에도 불구하고, 사전분포를 이해하고 적절히 지정하기 위해 참고할 수 있는 방법론적 연구는 부족한 상황이다. 본 연구는 사전분포의 중요성과 사전분포 지정에 대한 전반적인 내용을 다룬다. 먼저, 연구자가 사전분포를 직접 지정하지 않는, 즉 프로그램이 제공하는 디폴트 사전분포 방법을 알아본다. 자주 사용되는 프로그램들의 디폴트 사전분포를 알아봄과 더불어 디폴트 사전분포의 알려진 문제점도 확인한다. 다음으로는 연구자가 사전분포를 직접 지정하는 방법에 대해 다룬다. 직접 지정할 수 있는 사전분포에는 무정보 사전분포와 정보 사전분포가 있으며, 어떤 사전분포를 이용할지는 모수에 대한 사전 정보의 유무에 따라 결정된다. 무정보 사전분포의 필요성과 이를 지정할 때 참고할 수 있도록 제안된 방법을 다루고, 정보 사전분포를 지정할 때 참고할 수 있는 연구들을 제공하며, 여러 연구의 기준을 종합해 연구자의 정보성 선택에 참고할 수 있는 기준을 탐색한다. 이후 본문에서 논의한 방법들을 적용한 자료 예시를 통해 실질적 도움을 제공하고자 하였으며, 마지막으로 본 연구의 의의와 한계에 대해 논의한다.

Abstract

The Bayesian estimation method has recently received a lot of attention in the social sciences. The Bayesian method has a special factor of prior distribution that can reflect researchers’ background knowledge in the estimation process. The specification of the prior distribution affects the overall estimation. Despite prior distribution being the most important factor in Bayesian analysis, there is a lack of methodological research for understanding and appropriately specifying the prior distribution. Therefore, the present study tries to help researchers to apply the prior distribution to their estimation by addressing the importance of the prior distribution and the overall content of the prior specification. First, we explore the method that researchers do not directly specify the prior distribution. This method means selecting the default prior distribution automatically provided by the program, and if you want to use this option, you must know exactly what kind of the default prior distribution is actually provided. For this, we discuss the default priors of frequently used programs, as well as the known problem of the default priors. Second, we address the method that researchers do specify the prior distribution by themseleves. The prior distributions that can be directly specified include noninformative prior distributions and informative prior distributions. Which prior distribution to use is determined by the presence of prior information on parameters. This study deals with the necessity of noninformative prior distributions and the proposed method when specifying them, provides studies that can be referenced when specifying informative prior distributions, and explores criteria that can be referenced for the select of informativeness by synthesizing the criteria across many studies. We provide practical help through data examples applying the methods discussed in the text, and finally discuss the significance and limitations of the present study.

Keywords:

Bayesian method, prior distribution, default priors, non-informative priors, informative priors

키워드:

베이지안 방법, 사전분포, 디폴트 사전분포, 무정보 사전분포, 정보 사전분포

Acknowledgments

이 논문은 2019년 대한민국 교육부와 한국연구재단의 지원을 받아 수행된 연구임 (NRF-2019S1A5A2A03041362).

References

  • Asparouhov, T., & Muthén, B. (2010). Bayesian analysis of latent variable models using Mplus. Retrieved from http://www.statmodel.com/download/BayesAdvantages18.pdf
  • Baldwin, S. A., & Fellingham, G. W. (2013). Bayesian methods for the analysis of small sample multilevel data with a complex variance structure. Psychological Methods, 18(2), 151-164. https://doi.org/10.1037/a0030642 [https://doi.org/10.1037/a0030642]
  • Barnard, J., McCulloch, R., & Meng, X. L. (2000). Modeling covariance matrices in terms of standard deviations and correlations, with application to shrinkage. Statistica Sinica, 10(4), 1281-1311. Retrieved from http://www3.stat.sinica.edu.tw/statistica/oldpdf/a10n416.pdf
  • Baron, R. M., & Kenny, D. A. (1986). The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173-1182. https://doi.org/10.1037/0022-3514.51.6.1173 [https://doi.org/10.1037/0022-3514.51.6.1173]
  • Browne, W. J., & Draper, D. (2006). A comparison of Bayesian and likelihood-based methods for fitting multilevel models. Bayesian Analysis, 1(3), 473-514. https://doi.org/10.1214/06-BA117 [https://doi.org/10.1214/06-BA117]
  • Can, S., van de Schoot, R., & Hox, J. (2015). Collinear latent variables in multilevel confirmatory factor analysis: A comparison of maximum likelihood and Bayesian estimations. Educational and Psychological Measurement, 75(3), 406-427. https://doi.org/10.1177/0013164414547959 [https://doi.org/10.1177/0013164414547959]
  • Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., ... & Riddell, A. (2017). Stan: A probabilistic programming language. Grantee Submission, 76(1), 1-32. https://doi.org/10.18637/jss.v076.i01 [https://doi.org/10.18637/jss.v076.i01]
  • Chiorri, C., Day, T., & Malmberg, L. E. (2014). An approximate measurement invariance approach to within-couple relationship quality. Frontiers in Psychology, 5, 983. https://doi.org/10.3389/fpsyg.2014.00983 [https://doi.org/10.3389/fpsyg.2014.00983]
  • Choi, J. G. & Song, W. Y. (2018). The Effect of Socially-Prescribed Perfectionism of College Students to Depression: Testing the Mediation effect of Intolerance of Uncertainty and Unconditional Self-Acceptance. Journal of Convergence for Information Technology, 8(3), 183-191. https://doi.org/10.22156/CS4SMB.2018.8.3.183
  • Choi, Y. J. & Hong, H. Y. (2020). Mediating Effects of Mattering and Self-Acceptance in the Relationship between Socially Prescribed Perfectionism and Social Anxiety. Journal of the Korea Contents 20(1), 259-270. https://doi.org/10.5392/JKCA.2020.20.01.259
  • Cortopassi, A. C., Starks, T. J., Parsons, J. T., & Wells, B. E. (2017). Self-concealment, ego depletion, and drug dependence among young sexual minority men who use substances. Psychology of Sexual Orientation and Gender Diversity, 4(3), 272-281. https://doi.org/10.1037/sgd0000230 [https://doi.org/10.1037/sgd0000230]
  • Depaoli, S. (2012). Measurement and structural model class separation in mixture CFA: ML/EM versus MCMC. Structural Equation Modeling: A Multidisciplinary Journal, 19(2), 178-203. https://doi.org/10.1080/10705511.2012.659614 [https://doi.org/10.1080/10705511.2012.659614]
  • Depaoli, S. (2014). The impact of inaccurate “informative” priors for growth parameters in Bayesian growth mixture modeling. Structural Equation Modeling: A Multidisciplinary Journal, 21(2), 239-252. https://doi.org/10.1080/10705511.2014.882686 [https://doi.org/10.1080/10705511.2014.882686]
  • Depaoli, S., & Clifton, J. P. (2015). A Bayesian approach to multilevel structural equation modeling with continuous and dichotomous outcomes. Structural Equation Modeling: A Multidisciplinary Journal, 22(3), 327-351. https://doi.org/10.1080/10705511.2014.937849 [https://doi.org/10.1080/10705511.2014.937849]
  • Falkenström, F., Hatcher, R. L., & Holmqvist, R. (2015a). Confirmatory Factor Analysis of the Patient Version of the Working Alliance Inventory–Short Form Revised. Assessment, 22(5), 581–593. https://doi.org/10.1177/1073191114552472 [https://doi.org/10.1177/1073191114552472]
  • Falkenström, F., Hatcher, R. L., Skjulsvik, T., Larsson, M. H., & Holmqvist, R. (2015b). Development and validation of a 6-item working alliance questionnaire for repeated administrations during psychotherapy. Psychological Assessment, 27(1), 169–183. https://doi.org/10.1037/pas0000038 [https://doi.org/10.1037/pas0000038]
  • Fang, J., Wen, Z., & Hau, K. T. (2019). Mediation effects in 2-1-1 multilevel model: evaluation of alternative estimation methods. Structural Equation Modeling: A Multidisciplinary Journal, 26(4), 591-606. https://doi.org/10.1080/10705511.2018.1547967 [https://doi.org/10.1080/10705511.2018.1547967]
  • Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper). Bayesian Analysis, 1(3), 515-534. https://doi.org/10.1214/06-BA117A [https://doi.org/10.1214/06-BA117A]
  • Gelman, A., Carlin, J. B., & Stern, H. S. (2013). Bayesian Data Analysis (3rd ed.). CRC Press. https://doi.org/10.1201/9780429258411 [https://doi.org/10.1201/9780429258411]
  • Gelman, A., Jakulin, A., Pittau, M. G., & Su, Y. S. (2008). A weakly informative default prior distribution for logistic and other regression models. Annals of Applied Statistics, 2(4), 1360-1383. https://doi.org/10.1214/08-AOAS191 [https://doi.org/10.1214/08-AOAS191]
  • Hagger, M. S., & Hamilton, K. (2018). Motivational predictors of students’ participation in out-of-school learning activities and academic attainment in science: An application of the trans-contextual model using Bayesian path analysis. Learning and Individual Differences, 67, 232-244. https://doi.org/10.1016/j.lindif.2018.09.002 [https://doi.org/10.1016/j.lindif.2018.09.002]
  • Hobert, J. P., & Casella, G. (1996). The effect of improper priors on Gibbs sampling in hierarchical linear mixed models. Journal of the American Statistical Association, 91(436), 1461-1473. https://doi.org/10.1080/01621459.1996.10476714 [https://doi.org/10.1080/01621459.1996.10476714]
  • Holtmann, J., Koch, T., Lochner, K., & Eid, M. (2016). A comparison of ML, WLSMV, and Bayesian methods for multilevel structural equation models in small samples: A simulation study. Multivariate Behavioral Research, 51(5), 661-680. https://doi.org/10.1080/00273171.2016.1208074 [https://doi.org/10.1080/00273171.2016.1208074]
  • Hox, J. J., & Bechger, T. M. (1998). An introduction to structural equation modeling. Family Science Review, 11, 354-373. Retrieved from http://joophox.net/publist/semfamre.pdf
  • Hox, J. (2020). Important yet unheeded. In van de Schoot, R., & Miocević, M. (Eds.). Small sample size solutions: A guide for applied researchers and practitioners (pp. 255-265). Taylor & Francis. Retrieved from http://library.oapen.org/handle/20.500.12657/22385 [https://doi.org/10.4324/9780429273872-21]
  • Hoyle, R. H. (2000). Confirmatory Factor Analysis. In Tinsley, H. E., & Brown, S. D. (Eds.). Handbook of Applied Multivariate Statistics and Mathematical Modeling (pp. 465-497). Academic Press. https://doi.org/10.1016/B978-0-12-691360-6.X5000-9 [https://doi.org/10.1016/B978-0-12-691360-6.X5000-9]
  • Jeffreys, H. (1946). An invariant form for the prior probability in estimation problems. Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences, 186(1007), 453-461. https://doi.org/10.1098/rspa.1946.0056 [https://doi.org/10.1098/rspa.1946.0056]
  • Kaplan, D. & Depaoli, S. (2013). Bayesian statistical methods. In T. D. Little (Ed.), Oxford handbook of quantitative methods (pp. 407-437). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199934874.001.0001 [https://doi.org/10.1093/oxfordhb/9780199934874.001.0001]
  • Kelly, B. C., Rendina, H. J., Vuolo, M., Wells, B. E., & Parsons, J. T. (2015a). Influences of motivational contexts on prescription drug misuse and related drug problems. Journal of Substance Abuse Treatment, 48, 49-55. https://doi.org/10.1016/j.jsat.2014.07.005 [https://doi.org/10.1016/j.jsat.2014.07.005]
  • Kelly, B. C., Vuolo, M., Pawson, M., Wells, B. E., & Parsons, J. T. (2015b). Chasing the bean: Prescription drug smoking among socially active youth. The Journal of Adolescent Health, 56, 632-638. https://doi.org/10.1016/j.jadohealth.2015.02.008 [https://doi.org/10.1016/j.jadohealth.2015.02.008]
  • Kim, S. Y., Suh, Y., Kim, J. S., Albanese, M. A., & Langer, M. M. (2013). Single and multiple ability estimation in the SEM framework: A noninformative Bayesian estimation approach. Multivariate Behavioral Research, 48, 563-591. https://doi.org/10.1080/00273171.2013.802647 [https://doi.org/10.1080/00273171.2013.802647]
  • Kim, S. Y., Huh, D., Zhou, Z., & Mun, E. Y. (2020). A comparison of Bayesian to maximum likelihood estimation for latent growth models in the presence of a binary outcome. International Journal of Behavioral Development, 44(5), 447-457. https://doi.org/10.1177/0165025419894730 [https://doi.org/10.1177/0165025419894730]
  • Kruschke, J. K. (2011). Introduction to special section on Bayesian data analysis. Perspectives on Psychological Science, 6(3), 272-273. https://doi.org/10.1177/1745691611406926 [https://doi.org/10.1177/1745691611406926]
  • Lee, D. Y. (2020). The Effects of adolescent’s Perfectionistic Self-Presentation on Social anxiety: focusing on the mediation effects of unconditional self-acceptance [Unpublished master’s thesis]. The Catholic University of Korea.
  • Lee, J. E. & Kim, S. Y. (2018). The Mediating Effects of Unconditional Self-Acceptance and Moderating Effects of Gender on the Relationship between Evaluative Concerns Perfectionism and Social Anxiety among College Students. Journal of Human Understanding and Counseling, 39(2), 25-45. https://doi.org/10.30593/JHUC.39.2.2 [https://doi.org/10.30593/JHUC.39.2.2]
  • Lee, S. Y., & Song, X. Y. (2004). Evaluation of the Bayesian and maximum likelihood approaches in analyzing structural equation models with small sample sizes. Multivariate Behavioral Research, 39(4), 653-686. https://doi.org/10.1207/s15327906mbr3904_4 [https://doi.org/10.1207/s15327906mbr3904_4]
  • Lüdtke, O., Marsh, H. W., Robitzsch, A., & Trautwein, U. (2011). A 2×2 taxonomy of multilevel latent contextual models: Accuracy–bias trade-offs in full and partial error correction models. Psychological Methods, 16(4), 444-467. https://doi.org/10.1037/a0024376 [https://doi.org/10.1037/a0024376]
  • Lunn, D., Jackson, C., Best, N., Thomas, A., & Spiegelhalter, D. (2012). The BUGS book: A practical introduction to Bayesian analysis. CRC Press. https://doi.org/10.1080/09332480.2013.868759 [https://doi.org/10.1080/09332480.2013.868759]
  • Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. (2000). WinBUGS-a Bayesian modelling framework: concepts, structure, and extensibility. Statistics and Computing, 10(4), 325-337. https://doi.org/10.1023/A:1008929526011 [https://doi.org/10.1023/A:1008929526011]
  • McNeish, D. (2016). On using Bayesian methods to address small sample problems. Structural Equation Modeling: A Multidisciplinary Journal, 23(5), 750-773. https://doi.org/10.1080/10705511.2016.1186549 [https://doi.org/10.1080/10705511.2016.1186549]
  • McNeish, D. (2019) Two-Level Dynamic Structural Equation Models with Small Samples. Structural Equation Modeling: A Multidisciplinary Journal, 26(6), 948-966. https://doi.org/10.1080/10705511.2019.1578657 [https://doi.org/10.1080/10705511.2019.1578657]
  • McNeish, D., & Stapleton, L. M. (2016). Modeling clustered data with very few clusters. Multivariate Behavioral Research, 51(4), 495-518. https://doi.org/10.1080/00273171.2016.1167008 [https://doi.org/10.1080/00273171.2016.1167008]
  • Merkle, E. C., & Rosseel, Y. (2015). blavaan: Bayesian structural equation models via parameter expansion. Journal of Statistical Software, 85(4), 1-30. https://doi.org/10.18637/jss.v085.i04 [https://doi.org/10.18637/jss.v085.i04]
  • Miočević, M., Levy, R., & MacKinnon, D. P. (2021) Different Roles of Prior Distributions in the Single Mediator Model with Latent Variables, Multivariate Behavioral Research, 56(1), 20-40,https://doi.org/10.1080/00273171.2019.1709405 [https://doi.org/10.1080/00273171.2019.1709405]
  • Miočević, M., MacKinnon, D. P., & Levy, R. (2017). Power in Bayesian mediation analysis for small sample research. Structural equation modeling: a multidisciplinary journal, 24(5), 666-683. https://doi.org/10.1080/10705511.2017.1312407 [https://doi.org/10.1080/10705511.2017.1312407]
  • Mohajerin, B., Dolatshahi, B., Pour Shahbaz, A., & Farhoudian, A. (2013). Differences between expressive suppression and cognitive reappraisal in opioids and stimulant dependent patients. International Journal of High Risk Behaviors & Addiction, 2, 8–14. https://doi.org/10.5812/ ijhrba.8514 [https://doi.org/10.5812/ijhrba.8514]
  • Muthén, B., & Asparouhov, T. (2012). Bayesian structural equation modeling: a more flexible representation of substantive theory. Psychological MMethods, 17(3), 313-335. https://doi.org/10.1037/a0026802 [https://doi.org/10.1037/a0026802]
  • Muthen, L. K., & Muthen, B. O. (1998–2019). Mplus user’s guide. (8th ed.) [Computer software manual]. Muthen & Muthen. Retrieved fromhttps://www.statmodel.com/download/usersguide/MplusUserGuideVer_8.pdf
  • Plummer, M. (2003). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. In K. Hornik, F. Leisch, & A. Zeileis (Eds.), In Proceedings of the 3rd international workshop on distributed statistical computing (Vol. 124). Technische Universität Wien. Retrieved fromhttps://www.r-project.org/conferences/DSC-2003/Proceedings/
  • Polson, N. G., & Scott, J. G. (2012). On the half-Cauchy prior for a global scale parameter. Bayesian Analysis, 7(4), 887-902. https://doi.org/10.1214/12-BA730 [https://doi.org/10.1214/12-BA730]
  • Preacher, K. J., Rucker, D. D., & Hayes, A. F. (2007). Addressing moderated mediation hypotheses: Theory, methods, and prescriptions. Multivariate Behavioral Research, 42(1), 185-227. https://doi.org/10.1080/00273170701341316 [https://doi.org/10.1080/00273170701341316]
  • R Core Team. (2015). R: A language and environment for statistical computing [Computer software manual]. R Foundation for Statistical Computing. Retrieved fromhttps://www.R-project.org/
  • Seo, Y. I., Kim, J. M., & Jo, H. B. (2018). The effects of multidimensional perfectionism on the social anxiety and depression of college students: The mediating effect of unconditional self-acceptance. Korea Society for the Emotional & Behavioral Disorders, 34(1), 197-215. https://doi.org/10.33770/JEBD.34.1.16 [https://doi.org/10.33770/JEBD.34.1.16]
  • Smid, S. C., McNeish, D., Miočević, M., & van de Schoot, R. (2020a). Bayesian Versus Frequentist estimation for Structural Equation Models in Small Sample Contexts: A Systematic Review. Structural Equation Modeling: A Multidisciplinary Journal, 27(1), 131-161. https://doi.org/10.1080/10705511.2019.1577140 [https://doi.org/10.1080/10705511.2019.1577140]
  • Smid, S. C., Depaoli, S., & van De Schoot, R. (2020b). Predicting a distal outcome variable from a latent growth model: ML versus Bayesian estimation. Structural Equation Modeling: A Multidisciplinary Journal, 27(2), 169-191. https://doi.org/10.1080/10705511.2019.1604140 [https://doi.org/10.1080/10705511.2019.1604140]
  • Smid, S. C., & Winter, S. D. (2020). Dangers of the Defaults: A Tutorial on the Impact of Default Priors When Using Bayesian SEM With Small Samples. Frontiers in Psychology, 11, 3536. https://doi.org/10.3389/fpsyg.2020.611963 [https://doi.org/10.3389/fpsyg.2020.611963]
  • Spiegelhalter, D. J., Thomas, A., Best, N. G., & Lunn, D. (2003). WinBUGS user manual, Version 1.4. MRC Biostatistics Unit. Retrieved fromhttp://www.mrc-bsu.cam.ac.uk/bugs/winbugs/manual14.pdf
  • Steer, R. A., & Clark, D. A. (1997). Psychometric characteristics of the Beck Depression Inventory-II with college students. Measurement and Evaluation in Counseling and Development, 30(3), 128-136. https://doi.org/10.1080/07481756.1997.12068933 [https://doi.org/10.1080/07481756.1997.12068933]
  • van de Schoot, R., Broere, J. J., Perryck, K. H., Zondervan-Zwijnenburg, M., & Loey, N. E. (2015). Analyzing small data sets using Bayesian estimation: The case of posttraumatic stress symptoms following mechanical ventilation in burn survivors. European Journal of Psychotraumatology, 6(1), Article 25216. https://doi.org/10.3402/ejpt.v6.25216 [https://doi.org/10.3402/ejpt.v6.25216]
  • van de Schoot, R., Kaplan, D., Denissen, J., Asendorpf, J. B., Neyer, F. J., & Aken, M. A. (2014). A gentle introduction to Bayesian analysis: Applications to developmental research. Child development, 85(3), 842-860. https://doi.org/10.1111/cdev.12169 [https://doi.org/10.1111/cdev.12169]
  • van de Schoot, R., Winter, S. D., Ryan, O., Zondervan-Zwijnenburg, M., & Depaoli, S. (2017). A systematic review of Bayesian articles in psychology: The last 25 years. Psychological Methods, 22(2), 217-239. https://doi.org/10.1037/met0000100 [https://doi.org/10.1037/met0000100]
  • van Erp, S., Mulder, J., & Oberski, D. L. (2018). Prior sensitivity analysis in default Bayesian structural equation modeling. Psychological Methods, 23(2), 363-388. https://doi.org/10.1037/met0000162 [https://doi.org/10.1037/met0000162]
  • Xu, C., Wang, W., Liu, P., & Li, Z. (2015). Calibration of crash risk models on freeways with limited real-time traffic data using Bayesian meta-analysis and Bayesian inference approach. Accident Analysis & Prevention, 85, 207-218. https://doi.org/10.1016/j.aap.2015.09.016 [https://doi.org/10.1016/j.aap.2015.09.016]
  • Yu, R., & Abdel-Aty, M. (2013). Investigating different approaches to develop informative priors in hierarchical Bayesian safety performance functions. Accident Analysis & Prevention, 56, 51-58. https://doi.org/10.1016/j.aap.2013.03.023 [https://doi.org/10.1016/j.aap.2013.03.023]
  • Yuan, Y., & MacKinnon, D. P. (2009). Bayesian mediation analysis. Psychological Methods, 14(4), 301-322. https://doi.org/10.1037/a0016972 [https://doi.org/10.1037/a0016972]
  • Zondervan-Zwijnenburg, M., Peeters, M., Depaoli, S., & van de Schoot, R. (2017). Where do priors come from? Applying guidelines to construct informative priors in small sample research. Research in Human Development, 14(4), 305-320. https://doi.org/10.1080/15427609.2017.1370966 [https://doi.org/10.1080/15427609.2017.1370966]