Confirmatory Factor Analysis of the Invalidating Childhood Environment Scale in Spanish (ICES)*
Análisis Factorial Confirmatorio de la Escala de Ambiente Invalidante Infantil en Español (ICES)
Universitas Psychologica, vol. 19, 2020
Pontificia Universidad Javeriana
Martín M. Puddington a email@example.com
Universidad Tres de Febrero, Argentina
Fundación Foro, Argentina
Fundación Foro, Argentina
Date received: 07 May 2017
Date accepted: 04 November 2019
Abstract: Background/Objective: The Invalidating Childhood Environment Scale (ICES) is an instrument designed to assess retrospectively parental invalidation experience, according to Marsha Linehan’s definition of invalidation. The main aim of this paper is to determine the factorial structure of the ICES. Three factor structures: Standard Model, Reduced Model, and 2 Factor Model are examined. Also, the association between invalidating experience and borderline symptoms is studied. Method: Data from a non-clinical, volunteer sample of 274 adults was collected. Participants completed 4 surveys: Demographic surveys, Invalidating Childhood Environment Scale (ICES), Borderline Symptoms List (BSL 23), and Parental Bonding Instrument (PBI). Statistical analyses included confirmatory factor analysis and multiple linear regression. Results: The results of the analysis presented indicate that the ICES is a reliable instrument, and that the 2 Factor Model is good fit and allows for a broader understanding of invalidation and it’s qualities. The results of the linear regression analysis indicated a mild but significant association between invalidation and BPD symptoms. Internal consistency was acceptable for all levels. Conclusion: The outcome of this study shows that the ICES is a useful instrument that can be used to make progress in uncovering the role of childhood invalidation in BPD.
Keywords:invalidating environment, ICES, Borderline Personality Disorder, Confirmatory Factor Analysis, emotional dysregulation.
Resumen: Objetivos: La escala de Ambiente Invalidante Infantil (ICES) es un instrumento diseñado para evaluar de forma retrospectiva la invalidación (entendida según la definición de Marsha Linehan) experimentada por el sujeto durante su infancia. El objetivo principal del presente artículo es el de determinar la estructura factorial de la ICES. Se evaluaron tres modelos mediante un análisis factorial confirmatorio (AFC). Adicionalmente, se realizó un análisis de regresión para evaluar la relación entre la invalidación y la sintomatología del Trastorno Límite de la Personalidad (TLP). Método: Se tomó una muestra de 274 adultos de población general no clínica. Cada participante completó cuatro cuestionarios: una encuesta demográfica, la escala ICES, la escala de sintomatología TLP (BSL23) y el Instrumento de Lazos Parentales (PBI). Los datos fueron examinados mediante un AFC y un estudio de regresión lineal múltiple. Resultados: Los resultados muestran que la escala ICES tiene buenas propiedades psicométricas. El análisis de regresión mostró que la asociación entre la experiencia de invalidación y la sintomatología TLP es pequeña pero significativa. Conclusión: La escala ICES es un instrumento útil, que puede ser usado para futuros estudios que busquen echar luz sobre el rol de la invalidación en el TLP.
Palabras clave: ambiente invalidante, ICES, Trastorno Límite de la Personalidad, Análisis Factorial Confirmatorio, desregulación emocional.
An invalidating environment is one in which the caregivers punish, diminish or ignore the child’s emotional experience (Chapman & Linehan, 2005). This concept is central in Linehan’s (1993) biosocial theory (BT), which was conceptualized in order to explain Borderline Personality Disorder’s (BPD) etiology. According to BT, BPD is frequently the result of a vulnerable child experiencing constant emotional invalidation and negative reinforcement of emotional arousal from his parents. The parenting style is not necessarily a pre-existing condition. Since there is a complex reciprocal transaction, the child itself can contribute to and shape the caregivers behavior (Crowell, Beauchaine, & Linehan, 2009). In order to test BT statements, emotional invalidation consequences must be understood. Through a transversal study it has been observed that, in adults, receiving an invalidating response after performing a behavioral task, increases heart rate and skin conductance (Shenk & Fuzzetti, 2011). Effects of invalidation in youth have been studied only retrospectively, through a scale developed by Mountford, Corstorphine, Tomlinson, and Waller (2007), to quantitatively evaluate, in adults, parental invalidating responses during childhood. The Invalidating Childhood Environment Scale (ICES) has 14 items which describe validating and invalidating behaviors of parents; subjects have to indicate the frequency of occurrence of the behaviors during their childhood (up to the age of 18). Each item has to be rated separately for the father and the mother. Additionally, it has four descriptions corresponding to the family types according to BT (Chaotic, Perfect, Typical and Validating), and subjects have to rate each description considering how similar is to their own family.
Several studies have linked high ICES scores with psychopathology. For example, women presenting an eating disorder showed higher perceived childhood invalidation than non-clinical subjects (Mountford et al., 2007), while bulimic patients perceived higher paternal invalidation than anorexic patients (Haslam, Mountford, Meyer, & Waller, 2008). Also, in a non-clinical sample ICES scores were positively associated with BPD symptomatology (Robertson, Kimbrel, & Nelson-Gray, 2013). In adolescents, participants who engaged in non-suicidal self-injurious behavior reported higher ICES scores than control participants (Tan, Rehfuss, Suárez, & Parks-Savage, 2012). Another study conducted with a clinical sample of adolescents, showed a correlation between mother’s invalidation and BPD symptomatology (Valentin et al., 2015). Though there is a growing body of evidence, it is not yet clear in which way parental invalidation leads to psychopathology. Avoidance of affect has been proposed as a mediating variable between invalidation on both BPD symptoms (Sturrock, Francis, & Carr, 2009; Sturrock, & Mellor, 2014) and eating disorder symptoms (Mountford et al., 2007). However we are far from having a complete empirically based theory on how invalidation works. In this sense, determining the factor structure of the ICES could be an important step.
The ICES factorial structure has been explored with two separate exploratory factorial analyses done in two different linguistic adaptations. Compagnone and Monaco (2015), found, in the French version of the ICES, a 2 factor structure that later showed good fit in a confirmatory factor analysis. The exploratory analysis of the Spanish adaptation also showed a 2 factor structure replicating the same structure that was proposed for the French version (Puddington, Duthu, & Gagliesi, 2017). A third structural model was proposed by Robertson et al. (2013), who eliminated 5 items and tested a 1 factor structure in a confirmatory factor analysis that showed good fit indices.
In this paper, a non-clinical sample was evaluated with the Spanish-ICES. Three confirmatory factor analyses were performed comparing the three models proposed, one factor with 14 items (standard model), one factor with 9 items (reduced model) and two factors with 14 items (2 factors model). Also comparisons with Parental Bonding Instrument responses are presented to test criterial validity, and a linear regression, using BPD symptoms inventory as a regression variable, to test Linehan’s model in a Latinomerican population.
Volunteers were 274 adults (49.27% men), relatives and friends of psychology students. The sample was collected by students who were instructed to distribute envelopes containing the instruments to volunteers interested in participating. Students received academic credit for collaborating in the sample collection. To avoid an overrepresentation of mental health workers in the sample, volunteers could not be professionals or students from any career related to psychology.
Every student received 5 or more envelopes to distribute to volunteers. Each envelope contained an informed consent, and 4 surveys. After having filled out the instruments, the envelopes were sealed and returned to the first author.
The following instruments were used:
Demographic survey: Only basic demographic information was collected. Age, gender, degree of education, civil state and number of children.
Invalidating Childhood Environment Scale ([ICES]; adapted to Spanish by Puddington et al., 2017 form the original Mountford et al., 2007). This is a scale composed of 14 items that describe attitudes of the parent towards the child in different situations. The subject has to indicate how alike the situations are to what he experienced in his childhood. Responses are provided separately for mother and father. The instrument also has 4 extra items that describe the three types of invalidating families proposed by Linehan (1993), and a validating family. In this case the subject has to indicate how similar the description is to his actual family while he was a child, before he turned 18. Two separated exploratory factorial analyses determined that this instrument has a two factors structure. Items 1; 3; 4; 6; 7; 9; 10; 11 and 13 correspond to the first factor: Negative Response (previously labelled as Personal Distress by Compagnone & Monaco, 2015)), while items 2; 5; 8; 12 and 14 correspond to the second factor Lack of Support (previously labelled as Empathetic Incapacity Compagnone and Monaco (2015)). Since items in the first factor describe the progenitors’ behavior rather than the emotional state of the child we propose the use of the term Negative Response, instead of Personal Distress (Puddington et al., 2017). Also, we consider that the term Empathetic Incapacity represents an unsubstantiated inference on the parents ability to be empathetic, thus we put forward the more descriptive terms of Maternal or Paternal Lack of Support.
Borderline Symptoms List ([BSL 23]; Spanish version, Soler et al., 2013 adapted from Bohus et al., 2009), is a 23 item self-report measure that indicates the occurrence of symptoms typically observed in the borderline personality disorder population, within the previous week. All items are scored on a scale from 0 (not at all) to 4 (very strong). Absolute scores range from 0 to 92, with higher scores indicating a greater prominence of BPD symptoms. In previous research (Bohus et al., 2009; Soler et al., 2013), the original and Spanish adaptation of the BSL 23 was found to have strong reliability and validity, and was clearly able to differentiate between BPD symptoms and Axis I symptoms.
Parental Bonding Instrument ([PBI]; Parker, Tupling, & Brown, 1979; the Gómez Maquet, Vallejo Zapata, Villada Zapata, & Zambrano Cruz, 2010 adaptation was used). Is a 25 items self-report questionnaire that, as well as the ICES, describes parental behavior during the subjects childhood. The questionnaire contains two scales: care/affection, represented by 12 items, and overprotection/control, represented by 13 items. All items are scored on a scale from 0 (very unlike) to 3 (very like), and answered separately for each parent. The absolute score for the Care/Affection scale ranges 0 to 36 with higher scores representing the presence of parental care and affection. The absolute score for the Overprotection/Control scale ranges from 0 to 39, with higher scores reflecting the presence of excessive parental control.
All the analyses were performed using SPSS. To compare the three factorial structures proposed, a Confirmatory Factor Analysis (CFA) was performed using AMOS plugin. Additionally, a multiple linear regression was done using age, gender, and all the measures provided by the ICES as predictors, and BSL 23 scores as target. Gender variable was codified as 1 for men and 0 for women.
Demographics: Participants’ ages varied between 18 and 75 (Mean = 32.63; SD = 14.15). 6.5% of the subjects, did not indicate their educational level, the rest were primary completers or less 4.01%, secondary incomplete 4.37%, completed secondary 16.05%, coursing university or terciarian carriers 43.7%, terciarian or universitarian carrier completed 25.18%). 6.5% of subjects did not indicate their educational level. Regarding marital status, 50.9% of the volunteers were married or in a stable relationship, 42.4% were single or not in a stable relationship and 6.7% did not answer. 35.7% had at least one child.
Confirmatory Factor Analysis: The three models were tested in a CFA performed separately for answers related to the mother and the father, thus 6 different models were tested. The default model showed poor fit in both Mother’s answers χ 2 (77) = 328.633; p < 0.001; CFI = 0.762; AIC = 384.633; SRMR = 0.0862; RMSEA = 0.109; 90% confidence interval (CI) [0.097, 0.122]) and Father’s χ 2 (77) = 407.773; p < 0.001; CFI = 0.704; AIC = 463.773; SRMR = 0.1009; RMSEA = 0.125; 90% confidence interval (CI) [0.114, 0.138]). Both alternative models showed a better adjustment, (Reduced Model Mother, χ 2. (27) = 64.445; p < 0.001; CFI = 0.929; AIC = 100.445; SRMR = 0.0526; RMSEA = 0.071; 90% confidence interval (CI) [0.049, 0.094]; Reduced Model Father, χ 2. (27) = 60.494; p < 0.001; CFI = 0.942; AIC = 96.494; SRMR = 0.0532; RMSEA = 0.067; 90% confidence interval (CI) [0.045, 0.09]; 2 Factors Model Mother χ 2. (76) = 141.012; p < 0.001; CFI = 0.939; AIC = 199.012; SRMR = 0.0525; RMSEA = 0.056; 90% confidence interval (CI) [0.041, 0.07]; 2 Factors Model Mother X²(76) = 123.528; p < 0.001; CFI = 0.957; AIC = 181.528; SRMR = 0.0541; RMSEA = 0.048; 90% confidence interval (CI) [0.032, 0.063]). Results are summarized in Table 1.
Descriptive statistics: Mean and standard deviation of the different measures, along with the alpha coefficient for each of the scales are presented in Table 2. Gender comparisons for each of the variables were performed with a one way ANOVA. Gender differences were detected only for FNR, F(1,272) = 6.927; p < 0.01 and for Family Type Typical, F (1,272) = 7,57; p < 0.01, in both cases males showing higher invalidation levels.
Correlations between the measures of each instrument are presented in Table 3. Significant correlations (p < 0.01) were observed between every measure except for Family Type Typical and Paternal Overprotection.
Linear Regression Analysis: All variables included in the analysis are listed in Table 4. Regression resulted significant (F(10, 273) = 9.345; p < 0.001). The resultant model reached to explain 26% of the variance.
The main purpose of the present research was to test the fitness of three different factorial models of the ICES, through a CFA. The default model, originally proposed by the authors, showed poor fit indicators, while two alternative models showed better results. Both, 2 Factor Model and Reduced Model showed good to excellent values of CFI, SRMR and RMSEA indictors. Since the AIC indicator penalizes the number of items, as was expected, it showed better adjustment for the 9 items version than for the 2 factors version. However, we consider that both versions of the instrument can be used: the 9 item version is recommended if items economy is needed, and the 2 factors version, if a broader description of the invalidation is desired.
In comparison to other known factorial studies, our results with a Spanish version of the ICES match the results of a CFA performed in a sample from the United States (Robertson et al., 2013), as well as the CFA performed with the French version (Compagnone & Monaco, 2015), and with the Portuguese version (Vieira et al., 2018) which suggest that ICES factorial structure is consistent across western cultures.
Internal consistency, which in some previous papers using the ICES was reported as low, reached acceptable levels for all the possible scores. Criterial validity was assessed by studying the correlation of the ICES and the PBI. Results were very similar to those observed by Robertson et al. (2013), with high negative correlations between invalidation and care, and moderate positive correlations between invalidation and overprotection. These results are to be anticipated, as it can be inferred that parents who are more caring toward their children are less apt to invalidate them. Similarly, parents who are more controlling of their children may engage in increased invalidation. Family type chaotic, perfect and typical also correlated negatively with care and positively with overprotection, while the opposite was observed for validating families.
BPD symptoms were poorly but significantly associated with the parental behavior with correlations oscillating between 0.154 and 0.355. ICES in general terms showed higher correlation with BSL than PBI, thus invalidation as described by ICES could be more accurate for the study of BT.
The ICES Negative Response sub-scale rates the presence of invalidating responses while Lack of Support the absence of validating responses. Similarly PBI instrument also has one subscale rating desirable parental behavior, Care, and a second subscale that assess undesirable parental behaviors, Control. As can be seen in Table 3, BSL 23 correlated more strongly to items describing undesirable parental behaviors (ICES’ Negative Response, and PBI’s Control) than to items describing desirable behaviors (ICES’ Lack of Support, and PBI’s Care). Additionally, the three invalidating family types also showed a higher association with BSL 23 than the validating type. A rather speculative interpretation could be that, invalidation is stronger as a risk factor than validation as a protective factor. Thus, avoiding invalidation could be a better way to prevent mental disorders than being validating. This hypothesis requires further research.
In a resent experimental study, participants reported more negative emotional reactions to invalidation coming from men than from women (Weber & Herr, 2019). It is not clear if such effect is valid for parental gender since available results are contradictories. In contrast to what was found by Robertson et al. (2013) the model resulting in the linear regression analysis excluded maternal invalidation and included paternal negative responses, as well as the three types of invalidating families. The resulting model also includes age as a predictive factor with a negative beta, which indicates that in this non clinical sample, probability of high BSL levels decrease with age. This result is consistent to the decrement of BPD symptomatology after 10 years, observed in a prospective study with a clinical sample (Álvarez-Tomás et al., 2016). Finally, gender was also selected by the model with a negative beta. As male was codified with 1 and female with 0, the model found that levels of BPD symptoms were reduced in males. Although this result has to be taken with caution since an ANOVA comparing BSL scores between men (ẋ = 9.45) and women (ẋ = 11.05) failed to find differences between genders (p > 0.18).
The model reached to explain only 26% of the variance in the BSL 23 scores. This amount of explained variance does not differ greatly from what was found by other studies using a similar approach (Keng & Wong, 2017; Robertson et al., 2013). In order to generate a better model, other important variables must be included. Hernández, Arntz, Gaviria, Labad, and Gutiérrez-Zotes (2012), using the PBI instead of the ICES to study parental invalidation concluded that traumatic childhood experiences rather than invalidating environment, explained BPD symptomatology better. However, according to BT a big proportion of the variance should presumably be explained by biological variables such as gene polymorphism (Martín-Blanco et al., 2016) or even epigenetics (Dammann et al., 2011; Streit, Colodro-Conde, Hall, & Witt, 2020). In any case, the effect of exposition to invalidation, and neglect during childhood is far from being understood. In fact, parental invalidation was associated with other psychological variables not predicted by Biosocial Therory. For example Gill, Warburton, and Beath (2018) showed that parental invalidation was related to chronic worry, and Arnett, Roach, Elzy, and Jelsone-Swain (2019) founded an association with empathy.
In order to make progress in uncovering the role of childhood invalidation in BPD and development, three types of studies seem to be needed. Studies that assess invalidation in BPD population, general population longitudinal studies, and big samples general population studies with putative gene polymorphism assessment. The results in this paper indicate that the ICES is a reliable instrument potentially useful for any of these proposed studies.
We show our appreciation and gratitude to the president of Fundación Foro for Mental Health, Javier Martín Camacho, for providing funding for this research.
Álvarez-Tomás, I., Soler, J., Bados, A., Martín-Blanco, A., Elices, M., Carmona, C., ... Pascual, J. C. (2016). Long-term course of borderline personality disorder: A prospective 10-year follow-up study. Journal of Personality Disorders, 35(5), 590-605. https://doi.org/10.1521/pedi_2016_30_269
Arnett, K., Roach, A., Elzy, M., & Jelsone-Swain, L. (2019). Childhood emotional invalidation and right hemispheric mu suppression during a pain empathy task: An EEG study. Social Neuroscience, 14(2), 236-250. https://doi.org/10.1080/17470919.2018.1441905
Bohus, M., Kleindienst, N., Limberger, M. F., Stieglitz, R. D., Domsalla, M., Chapman, A. L., Steil, R.,... Wolf, M. (2009). The short version of the Borderline Symptom List (BSL-23): Development and initial data on psychometric properties. Psychopathology, 42, 32-39. https://doi.org/10.1159/000173701
Chapman, A. L., & Linehan, M. M. (2005). Dialectical behavior therapy for borderline personality disorder. In M. C. Zanarini (Ed.), Borderline Personality Disorder. Boca Ratón, FL: Taylor & Francis Group.
Compagnone, P. D., & Monaco, G. L. (2015). Validation française du questionnaire d’Évaluation de L’environnement Invalidant durant l’Enfance : The Invalidating Childhood Environments Scale (ICES). Revue Européenne de Psychologie Appliquée/European Review of Applied Psychology, 65(1), 43-52. https://doi.org/10.1016/j.erap.2014.11.001
Crowell, S. E., Beauchaine, T. P., & Linehan, M. M. (2009). A biosocial developmental model of borderline personality: Elaborating and extending Linehan’s Theory. Psychological Bulletin, 135(3), 495-510. https://doi.org/10.1037/a0015616
Dammann, G., Teschler, S., Haag, T., Altmüller, F., Tuczek, F., & Dammann, R. H. (2011). Increased DNA methylation of neuropsychiatric genes occurs in borderline personality disorder. Epigenetics, 6(12), 1454-1462. https://doi.org/10.4161/epi.6.12.18363
Gill, D., Warburton, W., & Beath, K. (2018). The specificity of the biosocial model to borderline traits. Clinical Psychologist, 22(1), 27-36. https://doi.org/10.1111/cp.12104
Gómez Maquet, Y., Vallejo Zapata, V. J., Villada Zapata, J., & Zambrano Cruz, R. (2010). Propiedades psicométricas del Instrumento de Lazos Parentales (Parental Bonding Instrument) en la población de Medellín, Colombia. Pensando Psicología, 6(11), 65-73. Retrieved from https://revistas.ucc.edu.co/index.php/pe/article/view/367
Haslam, M., Mountford, V., Meyer, C., & Waller, G. (2008). Invalidating childhood environments in anorexia and bulimia nervosa. Eating Behaviors, 9(3), 313-318. https://doi.org/10.1016/j.eatbeh.2007.10.005
Hernández, A., Arntz, A., Gaviria, A. M., Labad, A., & Gutiérrez-Zotes, J. A. (2012). Relationships between childhood maltreatment, parenting style, and borderline personality disorder criteria. Journal of Personality Disorders, 26(5), 727-736. https://doi.org/10.1521/pedi.2012.26.5.727
Keng, S. L., & Wong, Y. Y. (2017). Association among self-compassion, childhood invalidation, and borderline personality disorder symptomatology in a Singaporean sample. Borderline Personality Disorder and Emotion Dysregulation, 4(1), 24. https://doi.org/10.1186/s40479-017-0075-3
Linehan, M. M. (1993). Cognitive-behavioral treatment of borderline personality disorder. New York: Guilford Press.
Martín-Blanco, A., Ferrer, M., Soler, J., Arranz, M. J., Vega, D., Calvo, N., ... Carmona, C. (2016). The role of hypothalamus–pituitary–adrenal genes and childhood trauma in borderline personality disorder. European Archives of Psychiatry and Clinical Neuroscience, 266(4), 307-316. https://doi.org/10.1007/s00406-015-0612-2
Mountford, V., Corstorphine, E., Tomlinson, T., & Waller, G. (2007). Development of a measure to assess invalidating childhood environments in the eating disorders. Eating Behaviors, 8(1), 48-58. https://doi.org/10.1016/j.eatbeh.2006.01.003
Parker, G., Tupling, H., & Brown, L. B. (1979). A parental bonding instrument. British Journal of Medical Psychology, 52(1), 1-10. https://doi.org/10.1111/j.2044-8341.1979.tb02487.x
Puddington, M., Duthu, F., & Gagliesi, P. (2017). Adaptación argentina de la Escala de Ambiente Invalidante Infantil (ICES-A). Revista Argentina de Clínica Psicológica, 26(3), 307-312. https://doi.org/10.24205/03276716.2017.1029
Robertson, C. D., Kimbrel, N. A., & Nelson-Gray, R. O. (2013). The Invalidating Childhood Environment Scale (ICES): Psychometric properties and relationship to borderline personality symptomatology. Journal of Personality Disorders, 27(3), 402-410. https://doi.org/10.1521/pedi_2012_26_062
Shenk, C. E., & Fruzzetti, A. E. (2011). The impact of validating and invalidating responses on emotional reactivity. Journal of Social and Clinical Psychology, 30(2), 163-183. https://doi.org/10.1521/jscp.2011.30.2.163
Soler, J., Vega, D., Feliu-Soler, A., Trujols, J., Soto, Á., Elices, M., ... Pascual, J. C. (2013). Validation of the Spanish version of the Borderline Symptom List, short form (BSL-23). BMC Psychiatry, 13(1), 139. https://doi.org/10.1186/1471-244X-13-139
Streit, F., Colodro-Conde, L., Hall, A. S., & Witt, S. H. (2020). Genomics of borderline personality disorder. In B. T. Baune (Ed.), Personalized psychiatry (pp. 227-237). London: Elsevier Inc.
Sturrock, B. A., Francis, A., & Carr, S. (2009). Avoidance of affect mediates the effect of invalidating childhood environments on borderline personality symptomatology in a non-clinical sample. Clinical Psychologist, 13(2), 41-51. https://doi.org/10.1080/13284200902936927
Sturrock, B., & Mellor, D. (2014). Perceived emotional invalidation and borderline personality disorder features: A test of theory. Personality and Mental Health, 8(2), 128-142. https://doi.org/10.1002/pmh.1249
Tan, A. C., Rehfuss, M. C., Suárez, E. C., & Parks-Savage, A. (2012). Nonsuicidal self-injury in an adolescent population in Singapore. Clinical Child Psychology and Psychiatry, 19(1), 58-76. https://doi.org/10.1177/1359104512467273
Valentin, M., Sasha, M., Theodora, M., Cecilia, M., Jean-Chrysostome, Z., & Johanne, R. (2015). Family functioning and parental invalidation of depressed adolescents with borderline personality disorder traits. Journal of Child & Adolescent Behavior, 3(4), 235. https://doi.org/10.4172/2375-4494.1000235
Vieira, A. I., Gonçalves, M., Machado, B. C., Rodrigues, T., Machado, P. P., Brandão, I., ... & Gonçalves, S. (2018). Psychometric properties of the Portuguese version of the Invalidating Childhood Environment Scale. Eating and Weight Disorders-Studies on Anorexia, Bulimia and Obesity, 1-9. https://doi.org/10.1007/s40519-018-0550-x
Weber, D. M., & Herr, N. R. (2019). The messenger matters: Invalidating remarks from men provoke a more negative emotional reaction than do remarks from women. Psychological Reports, 122(1), 180-200. https://doi.org/10.1177/0033294117748618
a Corresponding author. E-mail: firstname.lastname@example.org
How to cite: Puddington, M. M., Wright, E., & Gagliesi, P. (2019). Confirmatory factor analysis of the Invalidating Childhood Environment Scale in Spanish (ICES). Universitas Psychologica, 19https://doi.org/10.11144/Javeriana.upsy19.cfai