<?xml version="1.0" encoding="UTF-8"?><?xml-model type="application/xml-dtd" href="http://jats.nlm.nih.gov/publishing/1.1d3/JATS-journalpublishing1.dtd"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.1d3 20150301//EN" "http://jats.nlm.nih.gov/publishing/1.1d3/JATS-journalpublishing1.dtd">
<article xmlns:ali="http://www.niso.org/schemas/ali/1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" dtd-version="1.1d3" specific-use="Marcalyc 1.2" article-type="research-article" xml:lang="en">
<front>
<journal-meta>
<journal-id journal-id-type="redalyc">647</journal-id>
<journal-title-group>
<journal-title specific-use="original" xml:lang="es">Universitas Psychologica</journal-title>
<abbrev-journal-title abbrev-type="publisher" xml:lang="es">Univ. Psychol.</abbrev-journal-title>
</journal-title-group>
<issn pub-type="ppub">1657-9267</issn>
<issn pub-type="epub">2011-2777</issn>
<publisher>
<publisher-name>Pontificia Universidad Javeriana</publisher-name>
<publisher-loc>
<country>Colombia</country>
<email>revistascientificasjaveriana@gmail.com</email>
</publisher-loc>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="art-access-id" specific-use="redalyc">64762919003</article-id>
<article-id pub-id-type="doi">https://doi.org/10.11144/Javeriana.upsy19.cfai</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Artículos</subject>
</subj-group>
</article-categories>
<title-group>
<article-title xml:lang="en">Confirmatory Factor Analysis of the Invalidating Childhood Environment Scale in Spanish (ICES)<xref ref-type="fn" rid="fn1">*</xref>
</article-title>
<trans-title-group>
<trans-title xml:lang="es">Análisis Factorial Confirmatorio de la Escala de Ambiente Invalidante Infantil en Español (ICES)</trans-title>
</trans-title-group>
</title-group>
<contrib-group>
<contrib contrib-type="author" corresp="yes">
<contrib-id contrib-id-type="orcid">https://orcid.org/0000-0002-9908-2792</contrib-id>
<name name-style="western">
<surname>Puddington</surname>
<given-names>Martín M.</given-names>
</name>
<xref ref-type="corresp" rid="corresp1"><sup>a</sup></xref>
<xref ref-type="aff" rid="aff1"/>
<email>mmpuddington@gmail.com</email>
</contrib>
<contrib contrib-type="author" corresp="no">
<contrib-id contrib-id-type="orcid">https://orcid.org/0000-0002-6533-6088</contrib-id>
<name name-style="western">
<surname>Wright</surname>
<given-names>Erika</given-names>
</name>
<xref ref-type="aff" rid="aff2"/>
</contrib>
<contrib contrib-type="author" corresp="no">
<contrib-id contrib-id-type="orcid">https://orcid.org/0000-0001-5593-7616</contrib-id>
<name name-style="western">
<surname>Gagliesi</surname>
<given-names>Pablo</given-names>
</name>
<xref ref-type="aff" rid="aff3"/>
</contrib>
</contrib-group>
<aff id="aff1">
<institution content-type="original">Universidad Tres de Febrero, Argentina</institution>
<institution content-type="orgname">Universidad Tres de Febrero</institution>
<country country="AR">Argentina</country>
</aff>
<aff id="aff2">
<institution content-type="original">Fundación Foro, Argentina</institution>
<institution content-type="orgname">Fundación Foro</institution>
<country country="AR">Argentina</country>
</aff>
<aff id="aff3">
<institution content-type="original">Fundación Foro, Argentina</institution>
<institution content-type="orgname">Fundación Foro</institution>
<country country="AR">Argentina</country>
</aff>
<author-notes>
<corresp id="corresp1"><sup>a</sup> Correspondence author. E-mail: <email>mmpuddington@gmail.com</email>
</corresp>
</author-notes>
<pub-date pub-type="epub-ppub">
<year>2020</year>
</pub-date>
<volume>19</volume>
<history>
<date date-type="received" publication-format="dd mes yyyy">
<day>07</day>
<month>05</month>
<year>2017</year>
</date>
<date date-type="accepted" publication-format="dd mes yyyy">
<day>04</day>
<month>11</month>
<year>2019</year>
</date>
</history>
<permissions>
<ali:free_to_read/>
<license xlink:href="https://creativecommons.org/licenses/by/4.0/">
<ali:license_ref>https://creativecommons.org/licenses/by/4.0/</ali:license_ref>
<license-p>Esta obra está bajo una Licencia Creative Commons Atribución 4.0 Internacional.</license-p>
</license>
</permissions>
<abstract xml:lang="en">
<title>Abstract</title>
<p>Background/Objective: The Invalidating Childhood Environment Scale (ICES) is an instrument designed to assess retrospectively parental invalidation experience, according to Marsha Linehan’s definition of invalidation. The main aim of this paper is to determine the factorial structure of the ICES. Three factor structures: Standard Model, Reduced Model, and 2 Factor Model are examined. Also, the association between invalidating experience and borderline symptoms is studied. Method: Data from a non-clinical, volunteer sample of 274 adults was collected. Participants completed 4 surveys: Demographic surveys, Invalidating Childhood Environment Scale (ICES), Borderline Symptoms List (BSL 23), and Parental Bonding Instrument (PBI). Statistical analyses included confirmatory factor analysis and multiple linear regression. Results: The results of the analysis presented indicate that the ICES is a reliable instrument, and that the 2 Factor Model is good fit and allows for a broader understanding of invalidation and it’s qualities. The results of the linear regression analysis indicated a mild but significant association between invalidation and BPD symptoms. Internal consistency was acceptable for all levels. Conclusion: The outcome of this study shows that the ICES is a useful instrument that can be used to make progress in uncovering the role of childhood invalidation in BPD.</p>
</abstract>
<trans-abstract xml:lang="es">
<title>Resumen</title>
<p>Objetivos: La escala de Ambiente Invalidante Infantil (ICES) es un instrumento diseñado para evaluar de forma retrospectiva la invalidación (entendida según la definición de Marsha Linehan) experimentada por el sujeto durante su infancia. El objetivo principal del presente artículo es el de determinar la estructura factorial de la ICES. Se evaluaron tres modelos mediante un análisis factorial confirmatorio (AFC). Adicionalmente, se realizó un análisis de regresión para evaluar la relación entre la invalidación y la sintomatología del Trastorno Límite de la Personalidad (TLP). Método: Se tomó una muestra de 274 adultos de población general no clínica. Cada participante completó cuatro cuestionarios: una encuesta demográfica, la escala ICES, la escala de sintomatología TLP (BSL23) y el Instrumento de Lazos Parentales (PBI). Los datos fueron examinados mediante un AFC y un estudio de regresión lineal múltiple. Resultados: Los resultados muestran que la escala ICES tiene buenas propiedades psicométricas. El análisis de regresión mostró que la asociación entre la experiencia de invalidación y la sintomatología TLP es pequeña pero significativa. Conclusión: La escala ICES es un instrumento útil, que puede ser usado para futuros estudios que busquen echar luz sobre el rol de la invalidación en el TLP.</p>
</trans-abstract>
<kwd-group xml:lang="en">
<title>Keywords</title>
<kwd>invalidating environment</kwd>
<kwd>ICES</kwd>
<kwd>Borderline Personality Disorder</kwd>
<kwd>Confirmatory Factor Analysis</kwd>
<kwd>emotional dysregulation</kwd>
</kwd-group>
<kwd-group xml:lang="es">
<title>Palabras clave</title>
<kwd>ambiente invalidante</kwd>
<kwd>ICES</kwd>
<kwd>Trastorno Límite de la Personalidad</kwd>
<kwd>Análisis Factorial Confirmatorio</kwd>
<kwd>desregulación emocional</kwd>
</kwd-group>
<counts>
<fig-count count="0"/>
<table-count count="4"/>
<equation-count count="0"/>
<ref-count count="27"/>
</counts>
<custom-meta-group>
<custom-meta>
<meta-name>How to cite</meta-name>
<meta-value>Puddington, M. M., Wright, E., &amp; Gagliesi, P. (2019). Confirmatory factor analysis of the Invalidating Childhood Environment Scale in Spanish (ICES). <italic>Universitas Psychologica, 19</italic><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.11144/Javeriana.upsy19.cfai">https://doi.org/10.11144/Javeriana.upsy19.cfai</ext-link>
</meta-value>
</custom-meta>
</custom-meta-group>
</article-meta>
</front>
<body>
<sec>
<title/>
<p>An invalidating environment is one in which the caregivers punish, diminish or ignore the child’s emotional experience (<xref ref-type="bibr" rid="ref4">Chapman &amp; Linehan, 2005</xref>). This concept is central in <xref ref-type="bibr" rid="ref13">Linehan’s (1993)</xref> biosocial theory (BT), which was conceptualized in order to explain Borderline Personality Disorder’s (BPD) etiology. According to BT, BPD is frequently the result of a vulnerable child experiencing constant emotional invalidation and negative reinforcement of emotional arousal from his parents. The parenting style is not necessarily a pre-existing condition. Since there is a complex reciprocal transaction, the child itself can contribute to and shape the caregivers behavior (<xref ref-type="bibr" rid="ref6">Crowell, Beauchaine, &amp; Linehan, 2009</xref>). In order to test BT statements, emotional invalidation consequences must be understood. Through a transversal study it has been observed that, in adults, receiving an invalidating response after performing a behavioral task, increases heart rate and skin conductance (<xref ref-type="bibr" rid="ref19">Shenk &amp; Fuzzetti, 2011</xref>). Effects of invalidation in youth have been studied only retrospectively, through a scale developed by Mountford, Corstorphine, Tomlinson, and Waller (2007), to quantitatively evaluate, in adults, parental invalidating responses during childhood. The Invalidating Childhood Environment Scale (ICES) has 14 items which describe validating and invalidating behaviors of parents; subjects have to indicate the frequency of occurrence of the behaviors during their childhood (up to the age of 18). Each item has to be rated separately for the father and the mother. Additionally, it has four descriptions corresponding to the family types according to BT (Chaotic, Perfect, Typical and Validating), and subjects have to rate each description considering how similar is to their own family.</p>
<p>Several studies have linked high ICES scores with psychopathology. For example, women presenting an eating disorder showed higher perceived childhood invalidation than non-clinical subjects (<xref ref-type="bibr" rid="ref15">Mountford et al., 2007</xref>), while bulimic patients perceived higher paternal invalidation than anorexic patients (<xref ref-type="bibr" rid="ref10">Haslam, Mountford, Meyer, &amp; Waller, 2008</xref>). Also, in a non-clinical sample ICES scores were positively associated with BPD symptomatology (<xref ref-type="bibr" rid="ref18">Robertson, Kimbrel, &amp; Nelson-Gray, 2013</xref>). In adolescents, participants who engaged in non-suicidal self-injurious behavior reported higher ICES scores than control participants (<xref ref-type="bibr" rid="ref24">Tan, Rehfuss, Suárez, &amp; Parks-Savage, 2012</xref>). Another study conducted with a clinical sample of adolescents, showed a correlation between mother’s invalidation and BPD symptomatology (<xref ref-type="bibr" rid="ref25">Valentin et al., 2015</xref>). Though there is a growing body of evidence, it is not yet clear in which way parental invalidation leads to psychopathology. Avoidance of affect has been proposed as a mediating variable between invalidation on both BPD symptoms (<xref ref-type="bibr" rid="ref22">Sturrock, Francis, &amp; Carr, 2009</xref>; <xref ref-type="bibr" rid="ref23">Sturrock, &amp; Mellor,  2014</xref>) and eating disorder symptoms (<xref ref-type="bibr" rid="ref15">Mountford et al., 2007</xref>). However we are far from having a complete empirically based theory on how invalidation works. In this sense, determining the factor structure of the ICES could be an important step.</p>
<p>The ICES factorial structure has been explored with two separate exploratory factorial analyses done in two different linguistic adaptations.<xref ref-type="bibr" rid="ref5"> Compagnone and Monaco (2015)</xref>, found, in the French version of the ICES, a 2 factor structure that later showed good fit in a confirmatory factor analysis. The exploratory analysis of the Spanish adaptation also showed a 2 factor structure replicating the same structure that was proposed for the French version (<xref ref-type="bibr" rid="ref17">Puddington, Duthu, &amp; Gagliesi, 2017</xref>). A third structural model was proposed by Robertson et al. (2013), who eliminated 5 items and tested a 1 factor structure in a confirmatory factor analysis that showed good fit indices.</p>
<p>In this paper, a non-clinical sample was evaluated with the Spanish-ICES. Three confirmatory factor analyses were performed comparing the three models proposed, one factor with 14 items (standard model), one factor with 9 items (reduced model) and two factors with 14 items (2 factors model). Also comparisons with Parental Bonding Instrument responses are presented to test criterial validity, and a linear regression, using BPD symptoms inventory as a regression variable, to test Linehan’s model in a Latinomerican population.</p>
</sec>
<sec sec-type="methods">
<title>Method</title>
<sec>
<title>
<italic>Participants</italic>
</title>
<p>Volunteers were 274 adults (49.27% men), relatives and friends of psychology students. The sample was collected by students who were instructed to distribute envelopes containing the instruments to volunteers interested in participating. Students received academic credit for collaborating in the sample collection. To avoid an overrepresentation of mental health workers in the sample, volunteers could not be professionals or students from any career related to psychology.</p>
<p>Every student received 5 or more envelopes to distribute to volunteers. Each envelope contained an informed consent, and 4 surveys. After having filled out the instruments, the envelopes were sealed and returned to the first author.</p>
</sec>
<sec>
<title>
<italic>Instruments</italic>
</title>
<p>The following instruments were used:</p>
<p>
<italic>Demographic survey:</italic> Only basic demographic information was collected. Age, gender, degree of education, civil state and number of children.</p>
<p>
<italic>Invalidating Childhood Environment Scale</italic> ([ICES]; adapted to Spanish by Puddington et al., 2017 form the original <xref ref-type="bibr" rid="ref15">Mountford et al., 2007</xref>). This is a scale composed of 14 items that describe attitudes of the parent towards the child in different situations. The subject has to indicate how alike the situations are to what he experienced in his childhood. Responses are provided separately for mother and father. The instrument also has 4 extra items that describe the three types of invalidating families proposed by<xref ref-type="bibr" rid="ref13"> Linehan (1993)</xref>, and a validating family. In this case the subject has to indicate how similar the description is to his actual family while he was a child, before he turned 18. Two separated exploratory factorial analyses determined that this instrument has a two factors structure. Items 1; 3; 4; 6; 7; 9; 10; 11 and 13 correspond to the first factor: Negative Response (previously labelled as Personal Distress by <xref ref-type="bibr" rid="ref5">Compagnone  &amp; Monaco, 2015</xref>)), while items 2; 5; 8; 12 and 14 correspond to the second factor Lack of Support (previously labelled as Empathetic Incapacity <xref ref-type="bibr" rid="ref5">Compagnone  and Monaco (2015)</xref>). Since items in the first factor describe the progenitors’ behavior rather than the emotional state of the child we propose the use of the term Negative Response, instead of Personal Distress (<xref ref-type="bibr" rid="ref17">Puddington et al., 2017</xref>). Also, we consider that the term Empathetic Incapacity represents an unsubstantiated inference on the parents ability to be empathetic, thus we put forward the more descriptive terms of Maternal or Paternal Lack of Support.</p>
<p>
<italic>Borderline Symptoms List</italic> ([BSL 23]; Spanish version, <xref ref-type="bibr" rid="ref20">Soler et al., 2013</xref> adapted from <xref ref-type="bibr" rid="ref3">Bohus et al., 2009</xref>), is a 23 item self-report measure that indicates the occurrence of symptoms typically observed in the borderline personality disorder population, within the previous week. All items are scored on a scale from 0 (<italic>not at all</italic>) to 4 (<italic>very strong</italic>). Absolute scores range from 0 to 92, with higher scores indicating a greater prominence of BPD symptoms. In previous research (<xref ref-type="bibr" rid="ref3">Bohus et al., 2009</xref>; <xref ref-type="bibr" rid="ref20">Soler et al., 2013</xref>), the original and Spanish adaptation of the BSL 23 was found to have strong reliability and validity, and was clearly able to differentiate between BPD symptoms and Axis I symptoms.</p>
<p>
<italic>Parental Bonding Instrument</italic> ([PBI]; <xref ref-type="bibr" rid="ref16">Parker, Tupling, &amp; Brown, 1979</xref>; the <xref ref-type="bibr" rid="ref9">Gómez Maquet, Vallejo Zapata, Villada Zapata, &amp; Zambrano Cruz, 2010</xref> adaptation was used). Is a 25 items self-report questionnaire that, as well as the ICES, describes parental behavior during the subjects childhood. The questionnaire contains two scales: care/affection, represented by 12 items, and overprotection/control, represented by 13 items. All items are scored on a scale from 0 (<italic>very unlike</italic>) to 3 (<italic>very like</italic>), and answered separately for each parent. The absolute score for the Care/Affection scale ranges 0 to 36 with higher scores representing the presence of parental care and affection. The absolute score for the Overprotection/Control scale ranges from 0 to 39, with higher scores reflecting the presence of excessive parental control.</p>
</sec>
<sec>
<title>
<italic>Data analysis</italic>
</title>
<p>All the analyses were performed using SPSS. To compare the three factorial structures proposed, a Confirmatory Factor Analysis (CFA) was performed using AMOS plugin. Additionally, a multiple linear regression was done using age, gender, and all the measures provided by the ICES as predictors, and BSL 23 scores as target. Gender variable was codified as 1 for men and 0 for women.</p>
</sec>
</sec>
<sec sec-type="results">
<title>Results</title>
<p>Demographics: Participants’ ages varied between 18 and 75 (Mean = 32.63; <italic>SD </italic>= 14.15). 6.5% of the subjects, did not indicate their educational level, the rest were primary completers or less 4.01%, secondary incomplete 4.37%, completed secondary 16.05%,  coursing university or terciarian carriers 43.7%, terciarian or universitarian carrier completed 25.18%). 6.5% of subjects did not indicate their educational level. Regarding marital status, 50.9% of the volunteers were married or in a stable relationship, 42.4% were single or not in a stable relationship and 6.7% did not answer. 35.7% had at least one child.</p>
<p>Confirmatory Factor Analysis: The three models were tested in a CFA performed separately for answers related to the mother and the father, thus 6 different models were tested. The default model showed poor fit in both Mother’s answers <inline-formula>
<alternatives><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mglyph style="border:1px;" width="0.66em" height="1em"></mml:mglyph> <mml:mi>&#x3C7;</mml:mi> </mml:math>
<graphic xlink:href="64762919003_gi2.png"/>
</alternatives>
</inline-formula>
<sup>2</sup> (77) = 328.633; <italic>p</italic> &lt; 0.001; CFI = 0.762; AIC = 384.633; SRMR = 0.0862; RMSEA = 0.109; 90% confidence interval (CI) [0.097, 0.122]) and Father’s <inline-formula>
<alternatives><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mglyph style="border:1px;" width="0.66em" height="1em"></mml:mglyph> <mml:mi>&#x3C7;</mml:mi> </mml:math>
<graphic xlink:href="64762919003_gi3.png"/>
</alternatives>
</inline-formula>
<sup>2</sup> (77) = 407.773; <italic>p</italic> &lt; 0.001; CFI = 0.704; AIC = 463.773; SRMR = 0.1009; RMSEA = 0.125; 90% confidence interval (CI) [0.114, 0.138]). Both alternative models showed a better adjustment, (Reduced Model Mother, <inline-formula>
<alternatives><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mglyph style="border:1px;" width="0.66em" height="1em"></mml:mglyph> <mml:mi>&#x3C7;</mml:mi> </mml:math>
<graphic xlink:href="64762919003_gi4.png"/>
</alternatives>
</inline-formula>
<sup>2</sup>. (27) = 64.445; <italic>p</italic> &lt; 0.001; CFI = 0.929; AIC = 100.445; SRMR = 0.0526; RMSEA = 0.071; 90% confidence interval (CI) [0.049, 0.094]; Reduced Model Father, <inline-formula>
<alternatives><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mglyph style="border:1px;" width="0.66em" height="1em"></mml:mglyph> <mml:mi>&#x3C7;</mml:mi> </mml:math>
<graphic xlink:href="64762919003_gi5.png"/>
</alternatives>
</inline-formula>
<sup>2</sup>. (27) = 60.494; <italic>p</italic> &lt; 0.001; CFI = 0.942; AIC = 96.494; SRMR = 0.0532; RMSEA = 0.067; 90% confidence interval (CI) [0.045, 0.09]; 2 Factors Model Mother <inline-formula>
<alternatives><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mglyph style="border:1px;" width="0.66em" height="1em"></mml:mglyph> <mml:mi>&#x3C7;</mml:mi> </mml:math>
<graphic xlink:href="64762919003_gi6.png"/>
</alternatives>
</inline-formula>
<sup>2</sup>. (76) = 141.012; <italic>p</italic> &lt; 0.001; CFI = 0.939; AIC = 199.012; SRMR = 0.0525; RMSEA = 0.056; 90% confidence interval (CI) [0.041, 0.07]; 2 Factors Model Mother X²(76) = 123.528; <italic>p</italic> &lt; 0.001; CFI = 0.957; AIC = 181.528; SRMR = 0.0541; RMSEA = 0.048; 90% confidence interval (CI) [0.032, 0.063]). Results are summarized in <xref ref-type="table" rid="gt1">Table 1</xref>.</p>
<p>
<table-wrap id="gt1">
<label>Table 1</label>
<caption>
<title>
<italic>Model fit statistics for Confirmatory Factor Analysis</italic>
</title>
</caption>
<alt-text>Table 1 Model fit statistics for Confirmatory Factor Analysis</alt-text>
<graphic xlink:href="64762919003_gt2.png" position="anchor" orientation="portrait"/>
</table-wrap>
</p>
<p>
<italic>Descriptive statistics</italic>: Mean and standard deviation of the different measures, along with the alpha coefficient for each of the scales are presented in <xref ref-type="table" rid="gt2">Table 2</xref>. Gender comparisons for each of the variables were performed with a one way ANOVA. Gender differences were detected only for FNR, <italic>F</italic>(1,272) =  6.927; <italic>p </italic>&lt; 0.01 and for Family Type Typical,  <italic>F </italic>(1,272) =  7,57; <italic>p</italic> &lt; 0.01, in both cases males showing higher invalidation levels.</p>
<p>
<table-wrap id="gt2">
<label>Table 2</label>
<caption>
<title>
<italic>Means, Standard Deviations, and Internal Consistency of each variable</italic>
</title>
</caption>
<alt-text>Table 2 Means, Standard Deviations, and Internal Consistency of each variable</alt-text>
<graphic xlink:href="64762919003_gt3.png" position="anchor" orientation="portrait"/>
</table-wrap>
</p>
<p>
<italic>Correlations</italic> between the measures of each instrument are presented in <xref ref-type="table" rid="gt3">Table 3</xref>. Significant correlations (<italic>p</italic> &lt; 0.01) were observed between every measure except for Family Type Typical and Paternal Overprotection.</p>
<p>
<table-wrap id="gt3">
<label>Table 3</label>
<caption>
<title>
<italic>Correlations among variables of the three instruments used</italic>
</title>
</caption>
<alt-text>Table 3 Correlations among variables of the three instruments used</alt-text>
<graphic xlink:href="64762919003_gt4.png" position="anchor" orientation="portrait"/>
<table-wrap-foot>
<fn-group>
<fn id="fn4" fn-type="other">
<label>
<italic>Note</italic>
</label>
<p>*<italic>p</italic> &lt; 0.05; ** <italic>p</italic> &lt; 0.005</p>
</fn>
</fn-group>
</table-wrap-foot>
</table-wrap>
</p>
<p>Linear Regression Analysis: All variables included in the analysis are listed in <xref ref-type="table" rid="gt4">Table 4</xref>. Regression resulted significant (<italic>F</italic>(10, 273) = 9.345;<italic> p </italic>&lt; 0.001). The resultant model reached to explain 26% of the variance.</p>
<p>
<table-wrap id="gt4">
<label>Table 4</label>
<caption>
<title>
<italic>Multiple Lineal Regression Analysis</italic>
</title>
</caption>
<alt-text>Table 4 Multiple Lineal Regression Analysis</alt-text>
<graphic xlink:href="64762919003_gt5.png" position="anchor" orientation="portrait"/>
<table-wrap-foot>
<fn-group>
<fn id="fn5" fn-type="other">
<label>
<italic>Note</italic>
</label>
<p>
<bold>Bold</bold> indicates p &lt; 0.05</p>
</fn>
</fn-group>
</table-wrap-foot>
</table-wrap>
</p>
</sec>
<sec sec-type="discussion">
<title>Discussion</title>
<p>The main purpose of the present research was to test the fitness of three different factorial models of the ICES, through a CFA. The default model, originally proposed by the authors, showed poor fit indicators, while two alternative models showed better results. Both, 2 Factor Model and Reduced Model showed good to excellent values of CFI, SRMR and RMSEA indictors. Since the AIC indicator penalizes the number of items, as was expected, it showed better adjustment for the 9 items version than for the 2 factors version. However, we consider that both versions of the instrument can be used: the 9 item version is recommended if items economy is needed, and the 2 factors version, if a broader description of the invalidation is desired.</p>
<p>In comparison to other known factorial studies, our results with a Spanish version of the ICES match the results of a CFA performed in a sample from the United States (<xref ref-type="bibr" rid="ref18">Robertson et al., 2013</xref>), as well as the CFA performed with the French version (<xref ref-type="bibr" rid="ref5">Compagnone &amp; Monaco, 2015</xref>), and with the Portuguese version (<xref ref-type="bibr" rid="ref26">Vieira et al., 2018</xref>) which suggest that ICES factorial structure is consistent across western cultures.</p>
<p>Internal consistency, which in some previous papers using the ICES was reported as low, reached acceptable levels for all the possible scores. Criterial validity was assessed by studying the correlation of the ICES and the PBI. Results were very similar to those observed by <xref ref-type="bibr" rid="ref18">Robertson et al. (2013)</xref>, with high negative correlations between invalidation and care, and moderate positive correlations between invalidation and overprotection. These results are to be anticipated, as it can be inferred that parents who are more caring toward their children are less apt to invalidate them. Similarly, parents who are more controlling of their children may engage in increased invalidation. Family type chaotic, perfect and typical also correlated negatively with care and positively with overprotection, while the opposite was observed for validating families.</p>
<p>BPD symptoms were poorly but significantly associated with the parental behavior with correlations oscillating between 0.154 and 0.355. ICES in general terms showed higher correlation with BSL than PBI, thus invalidation as described by ICES could be more accurate for the study of BT.</p>
<p>The ICES Negative Response sub-scale rates the presence of invalidating responses while Lack of Support the absence of validating responses. Similarly PBI instrument also has one subscale rating desirable parental behavior, Care, and a second subscale that assess undesirable parental behaviors, Control. As can be seen in <xref ref-type="table" rid="gt3">Table 3</xref>, BSL 23 correlated more strongly to items describing undesirable parental behaviors (ICES’ Negative Response, and PBI’s Control) than to items describing desirable behaviors (ICES’ Lack of Support, and PBI’s Care). Additionally, the three invalidating family types also showed a higher association with BSL 23 than the validating type. A rather speculative interpretation could be that, invalidation is stronger as a risk factor than validation as a protective factor. Thus, avoiding invalidation could be a better way to prevent mental disorders than being validating. This hypothesis requires further research.</p>
<p>In a resent experimental study, participants reported more negative emotional reactions to invalidation coming from men than from women (<xref ref-type="bibr" rid="ref27">Weber &amp; Herr, 2019</xref>). It is not clear if such effect is valid for parental gender since available results are contradictories. In contrast to what was found by <xref ref-type="bibr" rid="ref18">Robertson et al. (2013)</xref> the model resulting in the linear regression analysis excluded maternal invalidation and included paternal negative responses, as well as the three types of invalidating families. The resulting model also includes age as a predictive factor with a negative beta, which indicates that in this non clinical sample, probability of high BSL levels decrease with age. This result is consistent to the decrement of BPD symptomatology after 10 years, observed in a prospective study with a clinical sample (<xref ref-type="bibr" rid="ref1">Álvarez-Tomás et al., 2016</xref>). Finally, gender was also selected by the model with a negative beta. As male was codified with 1 and female with 0, the model found that levels of BPD symptoms were reduced in males. Although this result has to be taken with caution since an ANOVA comparing BSL scores between men (ẋ = 9.45) and women (ẋ = 11.05) failed to find differences between genders (<italic>p</italic> &gt; 0.18).</p>
<p>The model reached to explain only 26% of the variance in the BSL 23 scores. This amount of explained variance does not differ greatly from what was found by other studies using a similar approach (<xref ref-type="bibr" rid="ref12">Keng &amp; Wong, 2017</xref>; <xref ref-type="bibr" rid="ref18">Robertson et al., 2013</xref>). In order to generate a better model, other important variables must be included. <xref ref-type="bibr" rid="ref11">Hernández, Arntz, Gaviria, Labad, and Gutiérrez-Zotes (2012)</xref>, using the PBI instead of the ICES to study parental invalidation concluded that traumatic childhood experiences rather than invalidating environment, explained BPD symptomatology better. However, according to BT a big proportion of the variance should presumably be explained by biological variables such as gene polymorphism (<xref ref-type="bibr" rid="ref14">Martín-Blanco et al., 2016</xref>) or even epigenetics (<xref ref-type="bibr" rid="ref7">Dammann et al., 2011</xref>; <xref ref-type="bibr" rid="ref21">Streit, Colodro-Conde, Hall, &amp; Witt, 2020</xref>). In any case, the effect of exposition to invalidation, and neglect during childhood is far from being understood. In fact, parental invalidation was associated with other psychological variables not predicted by Biosocial Therory. For example <xref ref-type="bibr" rid="ref8">Gill, Warburton, and Beath (2018)</xref> showed that parental invalidation was related to chronic worry, and <xref ref-type="bibr" rid="ref2">Arnett, Roach, Elzy, and Jelsone-Swain (2019)</xref> founded an association with empathy.</p>
<p>In order to make progress in uncovering the role of childhood invalidation in BPD and development, three types of studies seem to be needed. Studies that assess invalidation in BPD population, general population longitudinal studies, and big samples general population studies with putative gene polymorphism assessment. The results in this paper indicate that the ICES is a reliable instrument potentially useful for any of these proposed studies.</p>
</sec>
</body>
<back>
<ack>
	<title>Acknowledgment</title>
	<p>We show our appreciation and gratitude to the president of Fundación Foro for Mental Health, Javier Martín Camacho, for providing funding for this research.</p>
</ack>
<ref-list>
<title>References</title>
<ref id="ref1">
<mixed-citation>Álvarez-Tomás, I., Soler, J., Bados, A., Martín-Blanco, A., Elices, M., Carmona, C., ... Pascual, J. C. (2016). Long-term course of borderline personality disorder: A prospective 10-year follow-up study. <italic>Journal of Personality Disorders, 35</italic>(5), 590-605. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1521/pedi_2016_30_269">https://doi.org/10.1521/pedi_2016_30_269</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Álvarez-Tomás</surname>
<given-names>I.</given-names>
</name>
<name>
<surname>Soler</surname>
<given-names>J</given-names>
</name>
<name>
<surname>Bados</surname>
<given-names>A</given-names>
</name>
<name>
<surname>Martín-Blanco</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Elices</surname>
<given-names>M</given-names>
</name>
<name>
<surname>Carmona</surname>
<given-names>C</given-names>
</name>
<name>
<surname>Pascual</surname>
<given-names>J. C.</given-names>
</name>
</person-group>
<article-title>Long-term course of borderline personality disorder: A prospective 10-year follow-up study.</article-title>
<source>Journal of Personality Disorders</source>
<year>2016</year>
<volume>35</volume>
<issue>5</issue>
<fpage>590</fpage>
<lpage>605</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1521/pedi_2016_30_269">https://doi.org/10.1521/pedi_2016_30_269</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1521/pedi_2016_30_269</pub-id>
</element-citation>
</ref>
<ref id="ref2">
<mixed-citation>Arnett, K., Roach, A., Elzy, M., &amp; Jelsone-Swain, L. (2019). Childhood emotional invalidation and right hemispheric mu suppression during a pain empathy task: An EEG study. <italic>Social Neuroscience, 14</italic>(2), 236-250. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1080/17470919.2018.1441905">https://doi.org/10.1080/17470919.2018.1441905</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Arnett</surname>
<given-names>K</given-names>
</name>
<name>
<surname>Roach</surname>
<given-names>A</given-names>
</name>
<name>
<surname>Elzy</surname>
<given-names>M</given-names>
</name>
<name>
<surname>Jelsone-Swain</surname>
<given-names>L</given-names>
</name>
</person-group>
<article-title>Childhood emotional invalidation and right hemispheric mu suppression during a pain empathy task: An EEG study.</article-title>
<source>Social Neuroscience</source>
<year>2019</year>
<volume>14</volume>
<issue>2</issue>
<fpage>236</fpage>
<lpage>250</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1080/17470919.2018.1441905">https://doi.org/10.1080/17470919.2018.1441905</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1080/17470919.2018.1441905</pub-id>
</element-citation>
</ref>
<ref id="ref3">
<mixed-citation>Bohus, M., Kleindienst, N., Limberger, M. F., Stieglitz, R. D., Domsalla, M., Chapman, A. L., Steil, R.,... Wolf, M. (2009). The short version of the Borderline Symptom List (BSL-23): Development and initial data on psychometric properties. <italic>Psychopathology, 42,</italic> 32-39. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1159/000173701">https://doi.org/10.1159/000173701</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Bohus</surname>
<given-names>M</given-names>
</name>
<name>
<surname>Kleindienst</surname>
<given-names>N</given-names>
</name>
<name>
<surname>Limberger</surname>
<given-names>M. F.</given-names>
</name>
<name>
<surname>Stieglitz</surname>
<given-names>R. D</given-names>
</name>
<name>
<surname>Domsalla</surname>
<given-names>M</given-names>
</name>
<name>
<surname>Chapman</surname>
<given-names>A. L</given-names>
</name>
<name>
<surname>Steil</surname>
<given-names>R</given-names>
</name>
<name>
<surname>Wolf</surname>
<given-names>M.</given-names>
</name>
</person-group>
<article-title>The short version of the Borderline Symptom List (BSL-23): Development and initial data on psychometric properties.</article-title>
<source>Psychopathology</source>
<year>2009</year>
<volume>42</volume>
<fpage>32</fpage>
<lpage>39</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1159/000173701">https://doi.org/10.1159/000173701</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1159/000173701</pub-id>
</element-citation>
</ref>
<ref id="ref4">
<mixed-citation>Chapman, A. L., &amp; Linehan, M. M. (2005). Dialectical behavior therapy for borderline personality disorder. In M. C. Zanarini (Ed.), <italic>Borderline Personality Disorder</italic>. Boca Ratón, FL: Taylor &amp; Francis Group.</mixed-citation>
<element-citation publication-type="book">
<person-group person-group-type="author">
<name>
<surname>Chapman</surname>
<given-names>A. L</given-names>
</name>
<name>
<surname>Linehan</surname>
<given-names>M. M.</given-names>
</name>
</person-group>
<person-group person-group-type="editor">
<name>
<surname>Zanarini</surname>
<given-names>M. C.</given-names>
</name>
</person-group>
<article-title>Dialectical behavior therapy for borderline personality disorder.</article-title>
<source>Borderline Personality Disorder.</source>
<year>2005</year>
<publisher-loc>Boca Ratón, FL</publisher-loc>
<publisher-name>Taylor &amp; Francis Group.</publisher-name>
</element-citation>
</ref>
<ref id="ref5">
<mixed-citation>Compagnone, P. D., &amp; Monaco, G. L. (2015). Validation française du questionnaire d’Évaluation de L’environnement Invalidant durant l’Enfance : The Invalidating Childhood Environments Scale (ICES). <italic>Revue Européenne de Psychologie Appliquée/European Review of Applied Psychology, 65</italic>(1), 43-52. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1016/j.erap.2014.11.001">https://doi.org/10.1016/j.erap.2014.11.001</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Compagnone</surname>
<given-names>P. D.</given-names>
</name>
<name>
<surname>Monaco</surname>
<given-names>G. L.</given-names>
</name>
</person-group>
<article-title>Validation française du questionnaire d’Évaluation de L’environnement Invalidant durant l’Enfance : The Invalidating Childhood Environments Scale (ICES).</article-title>
<source>Revue Européenne de Psychologie Appliquée/European Review of Applied Psychology</source>
<year>2015</year>
<volume>65</volume>
<issue>1</issue>
<fpage>43</fpage>
<lpage>52</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1016/j.erap.2014.11.001">https://doi.org/10.1016/j.erap.2014.11.001</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1016/j.erap.2014.11.001</pub-id>
</element-citation>
</ref>
<ref id="ref6">
<mixed-citation>Crowell, S. E., Beauchaine, T. P., &amp; Linehan, M. M. (2009). A biosocial developmental model of borderline personality: Elaborating and extending Linehan’s Theory. <italic>Psychological Bulletin, 135</italic>(3), 495-510. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1037/a0015616">https://doi.org/10.1037/a0015616</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Crowell</surname>
<given-names>S. E</given-names>
</name>
<name>
<surname>Beauchaine</surname>
<given-names>T. P.</given-names>
</name>
<name>
<surname>Linehan</surname>
<given-names>M. M.</given-names>
</name>
</person-group>
<article-title>A biosocial developmental model of borderline personality: Elaborating and extending Linehan’s Theory.</article-title>
<source>Psychological Bulletin</source>
<year>2009</year>
<volume>135</volume>
<issue>3</issue>
<fpage>495</fpage>
<lpage>510</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1037/a0015616">https://doi.org/10.1037/a0015616</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1037/a0015616</pub-id>
</element-citation>
</ref>
<ref id="ref7">
<mixed-citation>Dammann, G., Teschler, S., Haag, T., Altmüller, F., Tuczek, F., &amp; Dammann, R. H. (2011). Increased DNA methylation of neuropsychiatric genes occurs in borderline personality disorder. <italic>Epigenetics, 6</italic>(12), 1454-1462. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.4161/epi.6.12.18363">https://doi.org/10.4161/epi.6.12.18363</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Dammann</surname>
<given-names>G.</given-names>
</name>
<name>
<surname>Teschler</surname>
<given-names>S.</given-names>
</name>
<name>
<surname>Haag</surname>
<given-names>T.</given-names>
</name>
<name>
<surname>Altmüller</surname>
<given-names>F.</given-names>
</name>
<name>
<surname>Tuczek</surname>
<given-names>F.</given-names>
</name>
<name>
<surname>Dammann</surname>
<given-names>R. H.</given-names>
</name>
</person-group>
<article-title>Increased DNA methylation of neuropsychiatric genes occurs in borderline personality disorder.</article-title>
<source>Epigenetics</source>
<year>2011</year>
<volume>6</volume>
<issue>12</issue>
<fpage>1454</fpage>
<lpage>1462</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.4161/epi.6.12.18363">https://doi.org/10.4161/epi.6.12.18363</ext-link>
</comment>
<pub-id pub-id-type="doi">10.4161/epi.6.12.18363</pub-id>
</element-citation>
</ref>
<ref id="ref8">
<mixed-citation>Gill, D., Warburton, W., &amp; Beath, K. (2018). The specificity of the biosocial model to borderline traits. <italic>Clinical Psychologist, 22</italic>(1), 27-36. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1111/cp.12104">https://doi.org/10.1111/cp.12104</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Gill</surname>
<given-names>D.</given-names>
</name>
<name>
<surname>Warburton</surname>
<given-names>W.</given-names>
</name>
<name>
<surname>Beath</surname>
<given-names>K.</given-names>
</name>
</person-group>
<article-title>The specificity of the biosocial model to borderline traits.</article-title>
<source>Clinical Psychologist</source>
<year>2018</year>
<volume>22</volume>
<issue>1</issue>
<fpage>27</fpage>
<lpage>36</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1111/cp.12104">https://doi.org/10.1111/cp.12104</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1111/cp.12104</pub-id>
</element-citation>
</ref>
<ref id="ref9">
<mixed-citation>Gómez Maquet, Y., Vallejo Zapata, V. J., Villada Zapata, J., &amp; Zambrano Cruz, R. (2010). Propiedades psicométricas del Instrumento de Lazos Parentales (Parental Bonding Instrument) en la población de Medellín, Colombia. <italic>Pensando Psicología, 6</italic>(11), 65-73. Retrieved from <ext-link ext-link-type="uri" xlink:href="https://revistas.ucc.edu.co/index.php/pe/article/view/367">https://revistas.ucc.edu.co/index.php/pe/article/view/367</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Gómez Maquet</surname>
<given-names>Y.</given-names>
</name>
<name>
<surname>Vallejo Zapata</surname>
<given-names>V. J.</given-names>
</name>
<name>
<surname>Villada Zapata</surname>
<given-names>J.</given-names>
</name>
<name>
<surname>Zambrano Cruz</surname>
<given-names>R.</given-names>
</name>
</person-group>
<article-title>Propiedades psicométricas del Instrumento de Lazos Parentales (Parental Bonding Instrument) en la población de Medellín, Colombia.</article-title>
<source>Pensando Psicología</source>
<year>2010</year>
<volume>6</volume>
<issue>11</issue>
<fpage>65</fpage>
<lpage>73</lpage>
<comment>Retrieved from <ext-link ext-link-type="uri" xlink:href="https://www.revistas.ucc.edu.co/index.php/pe/article/view/367">https://www.revistas.ucc.edu.co/index.php/pe/article/view/367</ext-link>
</comment>
</element-citation>
</ref>
<ref id="ref10">
<mixed-citation>Haslam, M., Mountford, V., Meyer, C., &amp; Waller, G. (2008). Invalidating childhood environments in anorexia and bulimia nervosa. <italic>Eating Behaviors, 9</italic>(3), 313-318. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1016/j.eatbeh.2007.10.005">https://doi.org/10.1016/j.eatbeh.2007.10.005</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Haslam</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Mountford</surname>
<given-names>V.</given-names>
</name>
<name>
<surname>Meyer</surname>
<given-names>C.</given-names>
</name>
<name>
<surname>Waller</surname>
<given-names>G.</given-names>
</name>
</person-group>
<article-title>Invalidating childhood environments in anorexia and bulimia nervosa.</article-title>
<source>Eating Behaviors</source>
<year>2008</year>
<volume>9</volume>
<issue>3</issue>
<fpage>313</fpage>
<lpage>318</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1016/j.eatbeh.2007.10.005">https://doi.org/10.1016/j.eatbeh.2007.10.005</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1016/j.eatbeh.2007.10.005</pub-id>
</element-citation>
</ref>
<ref id="ref11">
<mixed-citation>Hernández, A., Arntz, A., Gaviria, A. M., Labad, A., &amp; Gutiérrez-Zotes, J. A. (2012). Relationships between childhood maltreatment, parenting style, and borderline personality disorder criteria. <italic>Journal of Personality Disorders, 26</italic>(5), 727-736. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1521/pedi.2012.26.5.727">https://doi.org/10.1521/pedi.2012.26.5.727</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Hernández</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Arntz</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Gaviria</surname>
<given-names>A. M.</given-names>
</name>
<name>
<surname>Labad</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Gutiérrez-Zotes</surname>
<given-names>J. A.</given-names>
</name>
</person-group>
<article-title>Relationships between childhood maltreatment, parenting style, and borderline personality disorder criteria.</article-title>
<source>Journal of Personality Disorders</source>
<year>2012</year>
<volume>26</volume>
<issue>5</issue>
<fpage>727</fpage>
<lpage>736</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1521/pedi.2012.26.5.727">https://doi.org/10.1521/pedi.2012.26.5.727</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1521/pedi.2012.26.5.727</pub-id>
</element-citation>
</ref>
<ref id="ref12">
<mixed-citation>Keng, S. L., &amp; Wong, Y. Y. (2017). Association among self-compassion, childhood invalidation, and borderline personality disorder symptomatology in a Singaporean sample. <italic>Borderline Personality Disorder and Emotion Dysregulation, 4</italic>(1), 24. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1186/s40479-017-0075-3">https://doi.org/10.1186/s40479-017-0075-3</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Keng</surname>
<given-names>S. L.</given-names>
</name>
<name>
<surname>Wong</surname>
<given-names>Y. Y.</given-names>
</name>
</person-group>
<article-title>Association among self-compassion, childhood invalidation, and borderline personality disorder symptomatology in a Singaporean sample.</article-title>
<source>Borderline Personality Disorder and Emotion Dysregulation</source>
<year>2017</year>
<volume>4</volume>
<issue>1</issue>
<fpage>24</fpage>
<lpage>24</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1186/s40479-017-0075-3">https://doi.org/10.1186/s40479-017-0075-3</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1186/s40479-017-0075-3</pub-id>
</element-citation>
</ref>
<ref id="ref13">
<mixed-citation>Linehan, M. M. (1993). <italic>Cognitive-behavioral treatment of borderline personality disorder</italic>. New York: Guilford Press.</mixed-citation>
<element-citation publication-type="book">
<person-group person-group-type="author">
<name>
<surname>Linehan</surname>
<given-names>M. M.</given-names>
</name>
</person-group>
<source>Cognitive-behavioral treatment of borderline personality disorder.</source>
<year>1993</year>
<publisher-loc>New York</publisher-loc>
<publisher-name>Guilford Press.</publisher-name>
</element-citation>
</ref>
<ref id="ref14">
<mixed-citation>Martín-Blanco, A., Ferrer, M., Soler, J., Arranz, M. J., Vega, D., Calvo, N., ... Carmona, C. (2016). The role of hypothalamus–pituitary–adrenal genes and childhood trauma in borderline personality disorder. <italic>European Archives of Psychiatry and Clinical Neuroscience, 266</italic>(4), 307-316. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1007/s00406-015-0612-2">https://doi.org/10.1007/s00406-015-0612-2</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Martín-Blanco</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Ferrer</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Soler</surname>
<given-names>J.</given-names>
</name>
<name>
<surname>Arranz</surname>
<given-names>M. J.</given-names>
</name>
<name>
<surname>Vega</surname>
<given-names>D.</given-names>
</name>
<name>
<surname>Calvo</surname>
<given-names>N</given-names>
</name>
<name>
<surname>Carmona</surname>
<given-names>C.</given-names>
</name>
</person-group>
<article-title>The role of hypothalamus–pituitary–adrenal genes and childhood trauma in borderline personality disorder.</article-title>
<source>European Archives of Psychiatry and Clinical Neuroscience</source>
<year>2016</year>
<volume>266</volume>
<issue>4</issue>
<fpage>307</fpage>
<lpage>316</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1007/s00406-015-0612-2">https://doi.org/10.1007/s00406-015-0612-2</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1007/s00406-015-0612-2</pub-id>
</element-citation>
</ref>
<ref id="ref15">
<mixed-citation>Mountford, V., Corstorphine, E., Tomlinson, T., &amp; Waller, G. (2007). Development of a measure to assess invalidating childhood environments in the eating disorders. <italic>Eating Behaviors, 8</italic>(1), 48-58. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1016/j.eatbeh.2006.01.003">https://doi.org/10.1016/j.eatbeh.2006.01.003</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Mountford</surname>
<given-names>V.</given-names>
</name>
<name>
<surname>Corstorphine</surname>
<given-names>E.</given-names>
</name>
<name>
<surname>Tomlinson</surname>
<given-names>T.</given-names>
</name>
<name>
<surname>Waller</surname>
<given-names>G.</given-names>
</name>
</person-group>
<article-title>Development of a measure to assess invalidating childhood environments in the eating disorders.</article-title>
<source>Eating Behaviors</source>
<year>2007</year>
<volume>8</volume>
<issue>1</issue>
<fpage>48</fpage>
<lpage>58</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1016/j.eatbeh.2006.01.003">https://doi.org/10.1016/j.eatbeh.2006.01.003</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1016/j.eatbeh.2006.01.003</pub-id>
</element-citation>
</ref>
<ref id="ref16">
<mixed-citation>Parker, G., Tupling, H., &amp; Brown, L. B. (1979). A parental bonding instrument. <italic>British Journal of Medical Psychology, 52</italic>(1), 1-10. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1111/j.2044-8341.1979.tb02487.x">https://doi.org/10.1111/j.2044-8341.1979.tb02487.x</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Parker</surname>
<given-names>G.</given-names>
</name>
<name>
<surname>Tupling</surname>
<given-names>H.</given-names>
</name>
<name>
<surname>Brown</surname>
<given-names>L. B.</given-names>
</name>
</person-group>
<article-title>A parental bonding instrument.</article-title>
<source>British Journal of Medical Psychology</source>
<year>1979</year>
<volume>52</volume>
<issue>1</issue>
<fpage>1</fpage>
<lpage>10</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1111/j.2044-8341.1979.tb02487.x">https://doi.org/10.1111/j.2044-8341.1979.tb02487.x</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1111/j.2044-8341.1979.tb02487.x</pub-id>
</element-citation>
</ref>
<ref id="ref17">
<mixed-citation>Puddington, M., Duthu, F., &amp; Gagliesi, P. (2017). Adaptación argentina de la Escala de Ambiente Invalidante Infantil (ICES-A). <italic>Revista Argentina de Clínica Psicológica, 26</italic>(3), 307-312. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.24205/03276716.2017.1029">https://doi.org/10.24205/03276716.2017.1029</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Puddington</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Duthu</surname>
<given-names>F.</given-names>
</name>
<name>
<surname>Gagliesi</surname>
<given-names>P.</given-names>
</name>
</person-group>
<article-title>Adaptación argentina de la Escala de Ambiente Invalidante Infantil (ICES-A).</article-title>
<source>Revista Argentina de Clínica Psicológica</source>
<year>2017</year>
<volume>26</volume>
<issue>3</issue>
<fpage>307</fpage>
<lpage>312</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.24205/03276716.2017.1029">https://doi.org/10.24205/03276716.2017.1029</ext-link>
</comment>
<pub-id pub-id-type="doi">10.24205/03276716.2017.1029</pub-id>
</element-citation>
</ref>
<ref id="ref18">
<mixed-citation>Robertson, C. D., Kimbrel, N. A., &amp; Nelson-Gray, R. O. (2013). The Invalidating Childhood Environment Scale (ICES): Psychometric properties and relationship to borderline personality symptomatology. <italic>Journal of Personality Disorders, 27</italic>(3), 402-410. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1521/pedi_2012_26_062">https://doi.org/10.1521/pedi_2012_26_062</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Robertson</surname>
<given-names>C. D.</given-names>
</name>
<name>
<surname>Kimbrel</surname>
<given-names>N. A.</given-names>
</name>
<name>
<surname>Nelson-Gray</surname>
<given-names>R. O.</given-names>
</name>
</person-group>
<article-title>The Invalidating Childhood Environment Scale (ICES): Psychometric properties and relationship to borderline personality symptomatology.</article-title>
<source>Journal of Personality Disorders</source>
<year>2013</year>
<volume>27</volume>
<issue>3</issue>
<fpage>402</fpage>
<lpage>410</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1521/pedi_2012_26_062">https://doi.org/10.1521/pedi_2012_26_062</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1521/pedi_2012_26_062</pub-id>
</element-citation>
</ref>
<ref id="ref19">
<mixed-citation>Shenk, C. E., &amp; Fruzzetti, A. E. (2011). The impact of validating and invalidating responses on emotional reactivity. <italic>Journal of Social and Clinical Psychology, 30</italic>(2), 163-183. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1521/jscp.2011.30.2.163">https://doi.org/10.1521/jscp.2011.30.2.163</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Shenk</surname>
<given-names>C. E.</given-names>
</name>
<name>
<surname>Fruzzetti</surname>
<given-names>A. E.</given-names>
</name>
</person-group>
<article-title>The impact of validating and invalidating responses on emotional reactivity.</article-title>
<source>Journal of Social and Clinical Psychology</source>
<year>2011</year>
<volume>30</volume>
<issue>2</issue>
<fpage>163</fpage>
<lpage>183</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1521/jscp.2011.30.2.163">https://doi.org/10.1521/jscp.2011.30.2.163</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1521/jscp.2011.30.2.163</pub-id>
</element-citation>
</ref>
<ref id="ref20">
<mixed-citation>Soler, J., Vega, D., Feliu-Soler, A., Trujols, J., Soto, Á., Elices, M., ... Pascual, J. C. (2013). Validation of the Spanish version of the Borderline Symptom List, short form (BSL-23). <italic>BMC Psychiatry, 13</italic>(1), 139. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1186/1471-244X-13-139">https://doi.org/10.1186/1471-244X-13-139</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Soler</surname>
<given-names>J.</given-names>
</name>
<name>
<surname>Vega</surname>
<given-names>D.</given-names>
</name>
<name>
<surname>Feliu-Soler</surname>
<given-names>A</given-names>
</name>
<name>
<surname>Trujols</surname>
<given-names>J.</given-names>
</name>
<name>
<surname>Soto</surname>
<given-names>Á.</given-names>
</name>
<name>
<surname>Elices</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Pascual</surname>
<given-names>J. C.</given-names>
</name>
</person-group>
<article-title>Validation of the Spanish version of the Borderline Symptom List, short form (BSL-23).</article-title>
<source>BMC Psychiatry</source>
<year>2013</year>
<volume>13</volume>
<issue>1</issue>
<fpage>139</fpage>
<lpage>139</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1186/1471-244X-13-139">https://doi.org/10.1186/1471-244X-13-139</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1186/1471-244X-13-139</pub-id>
</element-citation>
</ref>
<ref id="ref21">
<mixed-citation>Streit, F., Colodro-Conde, L., Hall, A. S., &amp; Witt, S. H. (2020). Genomics of borderline personality disorder. In B. T. Baune (Ed.), <italic>Personalized psychiatry</italic> (pp. 227-237). London: Elsevier Inc.</mixed-citation>
<element-citation publication-type="book">
<person-group person-group-type="author">
<name>
<surname>Streit</surname>
<given-names>F.</given-names>
</name>
<name>
<surname>Colodro-Conde</surname>
<given-names>L.</given-names>
</name>
<name>
<surname>Hall</surname>
<given-names>A. S.</given-names>
</name>
<name>
<surname>Witt</surname>
<given-names>S. H.</given-names>
</name>
</person-group>
<person-group person-group-type="editor">
<name>
<surname>Baune</surname>
<given-names>B. T.</given-names>
</name>
</person-group>
<article-title>Genomics of borderline personality disorder.</article-title>
<source>Personalized psychiatry</source>
<year>2020</year>
<fpage>227</fpage>
<lpage>237</lpage>
<publisher-loc>London</publisher-loc>
<publisher-name>Elsevier Inc.</publisher-name>
</element-citation>
</ref>
<ref id="ref22">
<mixed-citation>Sturrock, B. A., Francis, A., &amp; Carr, S. (2009). Avoidance of affect mediates the effect of invalidating childhood environments on borderline personality symptomatology in a non-clinical sample. <italic>Clinical Psychologist, 13</italic>(2), 41-51. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1080/13284200902936927">https://doi.org/10.1080/13284200902936927</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Sturrock</surname>
<given-names>B. A.</given-names>
</name>
<name>
<surname>Francis</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Carr</surname>
<given-names>S.</given-names>
</name>
</person-group>
<article-title>Avoidance of affect mediates the effect of invalidating childhood environments on borderline personality symptomatology in a non-clinical sample.</article-title>
<source>Clinical Psychologist</source>
<year>2009</year>
<volume>13</volume>
<issue>2</issue>
<fpage>41</fpage>
<lpage>51</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1080/13284200902936927">https://doi.org/10.1080/13284200902936927</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1080/13284200902936927</pub-id>
</element-citation>
</ref>
<ref id="ref23">
<mixed-citation>Sturrock, B., &amp; Mellor, D. (2014). Perceived emotional invalidation and borderline personality disorder features: A test of theory. <italic>Personality and Mental Health, 8</italic>(2), 128-142. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1002/pmh.1249">https://doi.org/10.1002/pmh.1249</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Sturrock</surname>
<given-names>B.</given-names>
</name>
<name>
<surname>Mellor</surname>
<given-names>D.</given-names>
</name>
</person-group>
<article-title>Perceived emotional invalidation and borderline personality disorder features: A test of theory.</article-title>
<source>Personality and Mental Health</source>
<year>2014</year>
<volume>8</volume>
<issue>2</issue>
<fpage>128</fpage>
<lpage>142</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1002/pmh.1249">https://doi.org/10.1002/pmh.1249</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1002/pmh.1249</pub-id>
</element-citation>
</ref>
<ref id="ref24">
<mixed-citation>Tan, A. C., Rehfuss, M. C., Suárez, E. C., &amp; Parks-Savage, A. (2012). Nonsuicidal self-injury in an adolescent population in Singapore. <italic>Clinical Child Psychology and Psychiatry, 19</italic>(1), 58-76. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1177/1359104512467273">https://doi.org/10.1177/1359104512467273</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Tan</surname>
<given-names>A. C.</given-names>
</name>
<name>
<surname>Rehfuss</surname>
<given-names>M. C.</given-names>
</name>
<name>
<surname>Suárez</surname>
<given-names>E. C.</given-names>
</name>
<name>
<surname>Parks-Savage</surname>
<given-names>A.</given-names>
</name>
</person-group>
<article-title>Nonsuicidal self-injury in an adolescent population in Singapore.</article-title>
<source>Clinical Child Psychology and Psychiatry</source>
<year>2012</year>
<volume>19</volume>
<issue>1</issue>
<fpage>58</fpage>
<lpage>76</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1177/1359104512467273">https://doi.org/10.1177/1359104512467273</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1177/1359104512467273</pub-id>
</element-citation>
</ref>
<ref id="ref25">
<mixed-citation>Valentin, M., Sasha, M., Theodora, M., Cecilia, M., Jean-Chrysostome, Z., &amp; Johanne, R. (2015). Family functioning and parental invalidation of depressed adolescents with borderline personality disorder traits. <italic>Journal of Child &amp; Adolescent Behavior, 3</italic>(4), 235. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.4172/2375-4494.1000235">https://doi.org/10.4172/2375-4494.1000235</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Valentin</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Sasha</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Theodora</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Cecilia</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Jean-Chrysostome</surname>
<given-names>Z.</given-names>
</name>
<name>
<surname>Johanne</surname>
<given-names>R.</given-names>
</name>
</person-group>
<article-title>Family functioning and parental invalidation of depressed adolescents with borderline personality disorder traits.</article-title>
<source>Journal of Child &amp; Adolescent Behavior</source>
<year>2015</year>
<volume>3</volume>
<issue>4</issue>
<fpage>235</fpage>
<lpage>235</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.4172/2375-4494.1000235">https://doi.org/10.4172/2375-4494.1000235</ext-link>
</comment>
<pub-id pub-id-type="doi">10.4172/2375-4494.1000235</pub-id>
</element-citation>
</ref>
<ref id="ref26">
<mixed-citation>Vieira, A. I., Gonçalves, M., Machado, B. C., Rodrigues, T., Machado, P. P., Brandão, I., ... &amp; Gonçalves, S. (2018). Psychometric properties of the Portuguese version of the Invalidating Childhood Environment Scale. <italic>Eating and Weight Disorders-Studies on Anorexia, Bulimia and Obesity,</italic> 1-9. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1007/s40519-018-0550-x">https://doi.org/10.1007/s40519-018-0550-x</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Vieira</surname>
<given-names>A. I.</given-names>
</name>
<name>
<surname>Gonçalves</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Machado</surname>
<given-names>B. C.</given-names>
</name>
<name>
<surname>Rodrigues</surname>
<given-names>T.</given-names>
</name>
<name>
<surname>Machado</surname>
<given-names>P. P.</given-names>
</name>
<name>
<surname>Brandão</surname>
<given-names>I.</given-names>
</name>
<name>
<surname>Gonçalves</surname>
<given-names>S.</given-names>
</name>
</person-group>
<article-title>Psychometric properties of the Portuguese version of the Invalidating Childhood Environment Scale.</article-title>
<source>Eating and Weight Disorders-Studies on Anorexia, Bulimia and Obesity</source>
<year>2018</year>
<fpage>1</fpage>
<lpage>9</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1007/s40519-018-0550-x">https://doi.org/10.1007/s40519-018-0550-x</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1007/s40519-018-0550-x</pub-id>
</element-citation>
</ref>
<ref id="ref27">
<mixed-citation>Weber, D. M., &amp; Herr, N. R. (2019). The messenger matters: Invalidating remarks from men provoke a more negative emotional reaction than do remarks from women. <italic>Psychological Reports, 122</italic>(1), 180-200. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1177/0033294117748618">https://doi.org/10.1177/0033294117748618</ext-link>
</mixed-citation>
<element-citation publication-type="journal">
<person-group person-group-type="author">
<name>
<surname>Weber</surname>
<given-names>D. M.</given-names>
</name>
<name>
<surname>Herr</surname>
<given-names>N. R.</given-names>
</name>
</person-group>
<article-title>The messenger matters: Invalidating remarks from men provoke a more negative emotional reaction than do remarks from women.</article-title>
<source>Psychological Reports</source>
<year>2019</year>
<volume>122</volume>
<issue>1</issue>
<fpage>180</fpage>
<lpage>200</lpage>
<comment>
<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1177/0033294117748618">https://doi.org/10.1177/0033294117748618</ext-link>
</comment>
<pub-id pub-id-type="doi">10.1177/0033294117748618</pub-id>
</element-citation>
</ref>
</ref-list>
<fn-group>
<fn id="fn1" fn-type="other">
<label>*</label>
<p>Research article.</p>
</fn>
</fn-group>
</back>
</article>