Keywords
assessment
psychometric properties
bullying
drug use
How to Cite
Abstract
The present paper analyses the psychometric properties of the My-School (MS) 3D simulation program, which was designed to detect students at risk of drug abuse and bullying. In particular, two empirical studies have been carried out to evaluate, on the one hand, the content validity of the scenes, and on the other, the validity criterion (predictive) of the software. Specifically, in terms of the evaluation of content validity, the results showed the existence of a substantial agreement among the expert judges regarding the relevance, coherence, clarity and low bias of situations. In relation to the criterion validity, there were no differences between the answers given in the program and the interviews. The MS software thus has adequate validity properties for detecting students at risk of substance use and bullying.
Bates, S. C., & Cox, J. M. (2008). The impact of computer versus paper–pencil survey, and individual versus group administration, on self-reports of sensitive behaviours. Computers in Human Behavior, 24(3), 903-916. https://doi.org/10.1016/j.chb.2007.02.021
Buchanan, T., Johnson, J.A., & Goldberg, L.R. (2005). Implementing a five-factor personality inventory for use on the Internet. European Journal of Psychological Assessment, 21(2), 115-127. https://doi.org/10.1027/1015-5759.21.2.115
Carmona, J. A., Cangas, A. J., & Langer, A. I. (2012). Applications of 3D simulation in Mental Health: Utilities and new developments. En L. L’Abate (Ed.), Mental Illnesses. Evaluation, Treatments and Implications (pp. 37-56). Rijeka: Intech.
Carmona, J. A., Cangas, A. J., García, G. R., Langer, A. I., & Zárate, R. (2012). Early Detection of Drug Use and Bullying in Secondary School Children Using a 3-D Simulation Program CyberPsychology, Behavior, and Social Networking, 15(1), 43-49. https://doi.org/10.1089/cyber.2010.0589
Carmona, J. A., Espínola, M., Cangas, A. J., & Iribarne, L. (2010a). MII School: New 3D Technologies Applied in Education to Detect Drug Abuses and Bullying in Adolescents. En M. Lytras et al. (Eds.), Technology Enhanced Learning: Quality of Teaching and Educational Reform (pp. 65-72). Heidelberg: Springer.
Carmona, J. A., Espínola, M., Cangas, A. J., & Iribarne, L. (2010b). Detecting Drug Use in Adolescents Using a 3D Simulation Program. Psychology, Society, & Education, 2(2), 143-153. Recuperado de http://ojs.ual.es/ojs/index.php/psye
Carmona, J. A., Espínola, M., Cangas, A. J., & Iribarne, L. (2011). MII-School: A 3D Videogame for the Early Detection of Abuse of Substances, Bullying and Mental Disorders in Adolescents. European Journal of Education and Psychology, 4(1), 75-85. Recuperado de http://www.formacionasunivep.com/ejep/
Escobar-Pérez, J., & Cuervo-Martínez, A. (2008). Validez de contenido y juicio de expertos: Una aproximación a su utilización. Avances en Medición, 6(1), 27-36.
Freitas, S., & Neumann, T. (2009). The use of “exploratory learning” for suppporting immersive learning in virtual environments. Computers & Education, 52(2), 343-352. https://doi.org/10.1016/j.compedu.2008.09.010
Gutiérrez-Maldonado, J., Alsina-Jurnet, I., Carvallo-Becíu, C., Letosa-Porta, A., & Magallón-Neri, E. (2007). Aplicaciones clínicas de la realidad virtual en el ámbito escolar. Cuadernos de medicina psicosomática y psiquiatría de enlace, 82, 32-51. Recuperado de http://www.editorialmedica.com/editorialmedica_publicacion.php
Johnson, J. A. (2005). Ascertaining the validity of individual protocols from Webbased personality inventories. Journal of Research in Personality, 39(1), 103-129. https://doi.org/10.1016/j.jrp.2004.09.009
Kendall, M. G., & Babington, B. (1939). The Problem of m Rankings. The Annals of Mathematical Statistics, 10(3), 275-287. https://doi.org/10.1214/aoms/1177732186
Kim, B., Park, H., & Baek, Y. (2009). Not just fun, but serious strategies: using meta-cognitive strategies in game-based learning. Computers & Education, 52(4), 800-810. https://doi.org/10.1016/j.compedu.2008.12.004
Kyriakakis, C., Van Rooyen, A., & Sisemore, D. (2000). The virtual classroom: a virtual reality environment for the assessment and rehabilitation of attention deficits. CyberPsychology & Behavior, 3(3), 483-499. Recuperado de http://www.liebertpub.com/overview/cyberpsychology-behavior-and-social-networking/10/
Landis J. R., & Koch G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159-174. Recuperado de http://www.biometrics.tibs.org/
López Feal, R. (1986). Construcción de instrumentos de medida en ciencias conductuales y sociales. Barcelona: Alamex.
McCabe, S. E., Boyd, C. J., Young, A., Crawford, S., & Pope, D. (2005). Mode effects for collecting alcohol and tobacco data among 3rd and 4th grade students: A randomized pilot study of Web-form versus paper-form surveys. Addictive Behaviors, 30(4), 663-671. https://doi.org/10.1016/j.addbeh.2004.08.012
McGartland, D., Berg-Weger, M., Tebb, S. S., Suzanne, E., & Rauch, S. (2003). Objectifying content validity: Conducting a content validity study in social work research. Social Work Research, 27(2), 94-104. Recuperado de http://www.naswpress.org/publications/journals/swr.html
McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2), 153-157.
Millman, J., & Greene, J. (1989). The specification and development of test of achievement and ability. En R. L. Linn (Ed.), Educational Measurement (pp. 335-366). Londres: Macmillan.
Observatorio Español sobre Drogas. (2008). Encuesta Estatal Sobre Uso de Drogas en Enseñanzas Secundarias (ESTUDES). Madrid: Delegación del Gobierno para el Plan Nacional sobre Drogas.
Osterlind, S. J. (1989). Constructing test items. Boston: Kluwer Academic Publishers.
Reips, U. D. (2000). The web experiment method: Advantages, disadvantages, and solutions. En M.H. Birnbuam (Ed.), Psychological experiments on the internet (pp. 89-117). San Diego: Academic Press.
Serrano, A., & Iborra, I. (2005). Informe Violencia entre compañeros en la escuela. Valencia, España: Centro Reina Sofía para el Estudio de la Violencia. Recuperado de http://www.centroreinasofia.es
Siegel, S., & Castellan, N. J. (1995). Estadística no paramétrica aplicada a alas ciencias de la conducta. México D. F.: Trillas.
Zoll, C., Enz, S., Schaub, H., Aylett, R., & Paiva, A. (2006, abril). Fighting bullying with the help of autonomous agents in a virtual school environment. The 7th International Conference on Cognitive Modeling. Trieste, Italia. Recuperado de http://www.inesc-id.pt/ficheiros/publicacoes/3057.pdf
This journal is registered under a Creative Commons Attribution 4.0 International Public License. Thus, this work may be reproduced, distributed, and publicly shared in digital format, as long as the names of the authors and Pontificia Universidad Javeriana are acknowledged. Others are allowed to quote, adapt, transform, auto-archive, republish, and create based on this material, for any purpose (even commercial ones), provided the authorship is duly acknowledged, a link to the original work is provided, and it is specified if changes have been made. Pontificia Universidad Javeriana does not hold the rights of published works and the authors are solely responsible for the contents of their works; they keep the moral, intellectual, privacy, and publicity rights. Approving the intervention of the work (review, copy-editing, translation, layout) and the following outreach, are granted through an use license and not through an assignment of rights. This means the journal and Pontificia Universidad Javeriana cannot be held responsible for any ethical malpractice by the authors. As a consequence of the protection granted by the use license, the journal is not required to publish recantations or modify information already published, unless the errata stems from the editorial management process. Publishing contents in this journal does not generate royalties for contributors.