Following the tradition of analysing psychological constructs based on language, this instrumental research aims to validate a dictionary of intergroup emotions, whose process was based on the design, conformation, and validation of word lists associated with 38 intergroup emotions, useful to identifying emotions in linguistic corpus analysed, using the Linguistic Inquiry and Word Count software (LIWC), whose main function is to identify linguistic and psychological categories in texts. Nine judges divided into three groups rated the words in two parameters (pertinence and relevance of each word with the emotion) and the scores were complemented with an incomplete random block methodology, to estimate the qualification of the remaining words. Acceptable consistency was found among the judges, as well as adjustment of their scores in the Rasch model, which allowed accepting the obtained results. From these ratings, 9 % of the root words associated with 38 emotions (18 negative, 17 positive, and 3 neutral), subsequently supplemented with their respective endings, were cleared.

Keywords:
Content analysis, linguistic analysis, intergroup emotions, content validity, political psychology

RLP_52_15_20033

Alpers, G. W., Winzelberg, A. J., Classen, C., Roberts, H., Dev, P., Koopman, C., & Taylor, C. B. (2005). Evaluation of computerized text analysis in an Internet breast cancer support group. Computers in Human Behavior, 21(2), 361-376. https://doi.org/ 10.1016/j.chb.2004.02.008

Andrei, A., Dingwall, A., Dillon, T., & Mathieu, J. (2014). Developing a Tagalog linguistic inquiry and word count (liwc) “Disaster” dictionary for understanding mixed language social media: A work-in-progress paper. EACL 2014, 91, 91-94.

American Psychological Association (APA) (2018). APA Dictionary of Psychology. Recuperado de https://dictionary.apa.org/

Ato, M., López, J. J., & Benavente, A. (2013). Un sistema de clasificación de los diseños de investigación en psicología. Anales de Psicologia, 29(3), 1038-1059. https://doi.org/10.6018/analesps.29.3.178511

Balage, P. P., Pardo, T. A., & Aluísio, S. M. (2013). An evaluation of the Brazilian Portuguese liwc dictionary for sentiment analysis. 9th Brazilian Symposium in Information and Human Language Technology, 7(6), 215-219. https://www.aclweb.org/anthology/W13-4829.pdf

Bantum, E. O., & Owen, J. E. (2009). Evaluating the validity of computerized content analysis programs for identification of emotional expression in cancer narratives. Psychological Assessment, 21(1), 79-88. https://doi.org/10.1037/a0014643

Barreto, I., & Medina-Arboleda, I. F. (en prensa). Political communication and ideology. En C. Zúniga, & W. López-López (Eds.), Political Psychology from Latin America (APA).

Bekman, R. M. (2001). Aplicação dos blocos incompletos balanceados na teoria de resposta ao item. Estudos em Avaliação Educacional, 24, 119-136. https://doi.org/10.18222/eae02420012202

Bjekić, J., Lazarević, L. B., Živanović, M., & Knežević, G. (2014). Psychometric evaluation of the Serbian dictionary for automatic text analysis –liwcser. Psihologija, 47(1), 5-32. https://doi.org/10.2298/PSI1401005B

Boyd, R. L., & Pennebaker, J. W. (2015). A way with words: Using language for psychological science in the modern era. En C. V. Dimofte, C. P. Haugtvedt & R. F. Yalch (Eds.), Consumer Psychology in a Social Media World [e-book] (pp. 222–236). Nueva York: Taylor & Francis Group. https://doi.org/10.4324/9781315714790

Campos, J. J. (1981). Human emotions: their new importance and their role in social referencing. Research & Clinical Center for Child Development, 1-7.

Carrobles, J. A., Becerra, A. A., Blanco Abarca, A., & Palomo, T. (1996). Gran diccionario de psicología. Madrid: Ediciones Del Prado.

Castiblanco-González, P. T., Medina-Arboleda, I. F., & Barreto, I. (2020). Estilo e conteúdo linguístico de mensangens publicadas no Twitter® durante um processo de transição à paz.  En D. Hung Hur, & J. M. Sabucedo (Eds.), Psicologia dos extremismos políticos (pp. 200-239). Petrópolis: Editora Vozes Ltda.

Chung, C. K., & Pennebaker, J. W. (2001). Desarrollo y propiedades psicométricas del liwc. LIWC Inc. Recuperado de http://www.liwc.net/liwcespanol/liwcdescription.php

Consuegra, N. (2010). Diccionario de psicología. Segunda edición. Bogotá: Ecoe Ediciones.

Devos, T., Silver, L. A., Mackie, D. A., & Smith, E. R. (2003). Experiencing intergroup emotions. En D. Mackie & E. Smith (Eds.), From prejudice to intergroup emotions: Differentiated reactions to social groups (pp. 111-134). Sussex: Psychology Press.  https://doi.org/10.4324/9781315783000

Fernández-Cabana, M., Rúas-Araújo, J., & Alves-Pérez, M. T. (2014). Psicología, lenguaje y comunicación: análisis con la herramienta liwc de los discursos y Tweets de los candidatos a las elecciones gallegas de 2012. Anuario de Psicología/The UB Journal of Psychology, 44(2), 169-184.

García-Prieto, P., Mackie, D. M., Tran, V., & Smith, E. R. (2007). Intergroup emotions in workgroups: Some emotional antecedents and consequences of belonging. Research on Managing Groups and Teams, 10, 145-184. https://doi.org/10.1016/S1534-0856(07)10007-4

Giner-Sorolla, R., Mackie, D. M., & Smith, E. R. (2007). Special issue on intergroup emotions: Introduction. Group Processes and Intergroup Relations, 10(1), 5-8. https://doi.org/10.1177/1368430206071661

Guevara Rojas, L. A., & Espinosa, A. (2014). Estereotipos y emociones intergrupales en Facebook durante las elecciones generales peruanas del 2011. Revista Electrónica de Psicología Política, 13(33), 41-64.

Gulías, E. J., López, P. C., & Boubeta, M. A. (2020). Economía, política social y Twitter: análisis de las emociones negativas en cuatro elecciones presidenciales latinoamericanas a través del liwc. Revista Ibérica de Sistemas e Tecnologias de Informação, E26, 270-282.

Hareli, S., & Parkinson, B. (2008). What’s social about social emotions? Journal for the Theory of Social Behaviour, 38(2), 131-156. https://doi.org/10.1111/j.1468-5914.2008.00363.x

Hayeri, N. (2014). Does gender affect translation? Analysis of English talks translated to Arabic (Tesis doctoral). The University of Texas, Austin, Texas, EE. UU.

Henríquez Henríquez, D. T., Urzúa Morales, A., & López López, W. (2020). Fusión de identidad: una revisión sistemática. Acta Colombiana de Psicología, 23(2), 383-437. https://doi.org/10.14718/ACP.2020.23.2.15

Hill, C. E. (2014). Emotion words checklist. En Helping Skills: Facilitating Exploration, Insight, and Action (4^a ed., pp. 1-8). Washington: American Psychological Association.

Huang, C.-L., Chung, C. K., Hui, N., Lin, Y.-C., Seih, Y.-T., Lam, B. C. P.,…, Pennebaker, J. W. (2012). The development of the Chinese linguistic inquiry and word count dictionary. Chinese Journal of Psychology, 54(2), 185–201.

Instituto Colombiano para la Evaluación de la Educación (icfes) (2017). El armado de las pruebas Saber y la comparabilidad en el tiempo. Boletín Saber en Breve, 25, 1-3. Recuperado de https://bit.ly/34caMft

Kahn, J., Tobin, R., Massey, A., & Anderson, J. (2007). Measuring emotional expression with the linguistic inquiry and word count. American Journal of Psychology, 120(2), 263-286. Recuperado de https://www.jstor.org/stable/pdf/20445398.pdf

Kjell, O. N., Kjell, K., García, D., & Sikström, S. (2018). Semantic measures: Using natural language processing to measure, differentiate, and describe psychological constructs. Psychological Methods, 24(1), 92-115. https://doi.org/10.1037/met0000191

Laboratorio Latinoamericano de Evaluación de la Calidad de la Educación (llece) (2010). Compendio de los manuales del serce. Santiago: Oficina Regional de Educación para América Latina y el Caribe (orealc/unesco). Recuperado de http://unesdoc.unesco.org/images/0019/001919/191940s.pdf

Lee, C., Sim, J. M., & Yoon, A. (2008). The review about the development of Korean linguistic inquiry and word count. Korean Journal of Cognitive Science, 16(4), 93-121.

Lewis, M., Haviland-Jones, J. M., & Feldman, L. (2008). Handbook of Emotions. Contemporary Sociology (3^a ed., Vol. 24). New York: The Guilford Press.

Mackie, D. M., Devos, T., & Smith, E. R. (2000). Intergroup emotions: Explaining offensive action tendencies in an intergroup context. Journal of Personality and Social Psychology, 79(4), 602-616. https://doi.org/10.1037/0022-3514.79.4.602

Mackie, D. M., Maitner, A. T., & Smith, E. R. (2015). Intergroup Emotion Theory. En T. D. Nelson (Ed.), Handbook of Prejudice, Stereotyping, and Discrimination (2^a ed., pp. 149-174). New York: Psychology Press. https://doi.org/10.4324/9780203361993

Mackie, D. M., & Smith E. R. (1998). Intergroup relations: Insights from a theoretically integrative approach. Psychological Review, 105(3), 499-529. https://doi.org/10.1037/0033-295x.105.3.499

Mackie, D. M., & Smith, E. R. (2001). Intergroup emotions and the social self: Prejudice preconceptualized as differentiated reactions to out-groups. En J. P. Forgas, & K. D. Williams (Eds.), The Social Self Cognitive, Interpersonal and Intergroup Perspectives (pp. 309-326). Nueva York: Francis & Taylor Group. https://doi.org/10.4324/9781315800516

Mackie, D. M., & Smith, E. R. (2017). Group-based emotion in group processes and intergroup relations. Group Processes and Intergroup Relations, 20(5), 658-668. https://doi.org/10.1177/1368430217702725

Maitner, A. T., Smith, E. R., & MacKie, D. M. (2016). Intergroup emotions theory: Prejudice and differentiated emotional reactions toward outgroups. En Ch. Sibley & F. K. Barlow  (Eds.),  The Cambridge Handbook of the Psychology of Prejudice (pp. 111-130). Cambridge: Cambridge University Press. https://doi.org/10.1017/9781316161579.006

Mauss, I. B., & Robinson, M. D. (2009). Measures of emotion: A review. Cognition and Emotion, 23(2), 209-237. https://doi.org/10.1080/02699930802204677

McDonnell, M. (2014). Comparison of text analysis programs for identification of emotional expression (Tesis doctoral). Loma Linda University, Loma Linda, California, EE.UU. Recuperado de https://scholarsrepository.llu.edu/etd/308/

Miller, D. A., Smith, E. R., & Mackie, D. M. (2004). Effects of intergroup contact and political predispositions on prejudice: Role of intergroup emotions. Group Processes and Intergroup Relations, 7(3), 221-237. https://doi.org/10.1177/1368430204046109

Mir, J. R., & Revert, X. (2014). Modelos teóricos en el estudio de la emoción. En F. Palmero, F. Martínez, & J. Huertas (Coords.),  Motivación y emoción (pp. 95-138). Madrid: McGraw-Hill.

Montero, I., & León, O. G. (2002). Clasificación y descripción de las metodologías de investigación en psicología. Revista Internacional de Psicología Clínica y de la Salud, 2, 503-508.

Moreno-Murcia, L., & Medina-Arboleda, I. F. (2020). Efectos de la reexperimentación emocional mediante lengua de señas colombiana sobre la sintomatología depresiva en personas sordas. Suma Psicológica, 27(2), 88-97. https://doi.org/10.14349/sumapsi.2020.v27.n2.3

Palmero, F., Guerrero, C., Gómez, C., Carpi, A., & Gorayeb, R. (2011). Manual de teorías emocionales y motivacionales. Castelló de la Plana (Valencia): Publicacions de la Universitat Jaume I.

Pardo, C. A. (2007). Estimación del desempeño de estudiantes de arquitectura en el diseño de un proyecto arquitectónico a partir de la evaluación por jueces utilizando el modelo de facetas. En Segunda Reunión Regional Norte, Centroamérica y Caribe de Evaluación Educativa. Bogotá.

Pennebaker, J. W. (2011). The secret life of pronouns. New Scientist, 211(2828), 42-45. https://doi.org/10.1016/S0262-4079(11)62167-2

Pennebaker, J. W. (2018). liwc, how it works. Recuperado de https://liwc.wpengine.com/how-it-works/

Pennebaker, J. W., Boyd, R. L., Jordan, K., & Blackburn, K. (2015). The Development and Psychometric Properties of liwc 2015. Recuperado de https://repositories.lib.utexas.edu/handle/2152/31333

Pennebaker, J. W., Chung, C. K., Ireland, M., Gonzales, A., & Booth, R. J. (2007). The development and psychometric properties of liwc 2007. Journal of Infectious Diseases and Therapy, 2(6), 1-22.

Pennebaker, J. W., & Francis, M. E. (1996). Cognitive, emotional and language processes in disclosure. Cognition & Emotion, 10(6), 601-626.

Pennebaker, J. W., & King, L. A. (1999). Linguistic styles: Language use as an individual difference. Journal of Personality and Social Psychology, 77(6), 1296–1312. https://doi.org/10.1037/0022-3514.77.6.1296

Pennebaker, J. W., Mayne, T., & Francis, M. E. (1997). Linguistic predictors of adaptive bereavement. Journal of Personality and Social Psychology, 72, 863-871.

Pennebaker, J. W., & Seagal, J. D. (1999). Forming a story: The health benefits of narrative. Journal of Clinical Psychology, 55(10), 1243-1254. https://doi.org/10.1016/j.psfr.2011.07.002

Pulido, P., Rojas, J., Restrepo, H., Zambrano-Hernandez, S., & Barreto, I. F. (2020). Estilos lingüísticos y emociones intergrupales en el plebiscito. Un análisis en la red social digital Twitter®. En D. Rico & I. F. Medina-Arboleda (Eds.), Construcción de paz en el postacuerdo: avances, tensiones y desafíos, (pp. 17-37). Barranquilla: Universidad del Norte – Universidad Católica de Colombia.

Ramírez Esparza, N., Pennebaker, J. W., García, F. A., & Suriá, R. (2007). La psicología del uso de las palabras: un programa de computadora que analiza textos en español. Revista Mexicana de Psicología, 24(1), 85-99.

Rico, D., Alzate, M., & Sabucedo, J. (2020) Participación en comportamientos extremos: fusión de la identidad en la resistencia pacífica. En D. Rico & I. F. Medina-Arboleda (Eds.),  Construcción de paz en el posacuerdo: avances, tensiones y desafíos (pp. 201-223). Barranquilla: Universidad del Norte – Universidad Católica de Colombia.

Roselló, G. (1980). Diccionario de psicología. Barcelona: Elicien. Recuperado de https://library.biblioboard.com/content/cf5492 4f-6023-4c14-ae28-22b5d74afba5

Rúas-Araújo, J., Alves-Pérez, M. T., & Fernández-Cabana, M. (2016). Comunicación, lenguaje y política. Análisis de los discursos institucionales del presidente de Ecuador, Rafael Correa (2007-2015), con la herramienta liwc. Razón y Palabra, 20(95), 591-607.

Scherer, K. R. (2005). What are emotions? And how can they be measured? Social Science Information, 44(4), 695-729. https://doi.org/10.1177/0539018405058216

Smith, E. R., & Mackie, D. M. (2008). Intergroup emotions. Handbook of Emotions, 2, 428-439. https://doi.org/10.1037/14342-010

Smith, E. R., & Mackie, D. M. (2016). Group-level emotions. Current Opinion in Psychology, 11, 15-19. https://doi.org/10.1016/j.copsyc.2016.04.005

Smith, E. R., Seger, C. R., & Mackie, D. M. (2007). Can emotions be truly group level? Evidence regarding four conceptual criteria. Journal of Personality and Social Psychology, 93(3), 431-446. https://doi.org/10.1037/0022-3514.93.3.431

Stone, P. J., Dunphy, D. C., Smith, M. S., & Ogilvie, D. M. (1967). The General Inquirer, a computer approach to content analysis. Cambridge: The MIT Press.

Tackman, A. M., Sbarra, D. A., Carey, A. L., Donnellan, M. B., Horn, A. B., Holtzman, N. S., …,Mehl, M. R. (2019). Depression, negative emotionality, and self-referential language: A multi-lab, multi-measure, and multi-language-task research synthesis. Journal of Personality and Social Psychology, 116(5), 817-834. https://doi.org/10.1037/pspp0000187

Tausczik, Y. R., & Pennebaker, J. W. (2010). The psychological meaning of words: liwc and computerized text analysis methods. Journal of Language and Social Psychology, 29(1), 24-54. https://doi.org/10.1177/0261927X09351676

Universidad de Granada (2015). Diseños en bloques incompletos aleatorizados. En Diseño de experimentos (pp. 1-11). Granada (España): Universidad de Granada. Recuperado de http://wpd.ugr.es/~bioestad/wp-content/uploads/Bloques-Incompletos.pdf

Van Kleef, G. A., & Fischer, A. H. (2016). Emotional collectives: How groups shape emotions and emotions shape groups. Cognition and Emotion, 30(1), 3-19. https://doi.org/10.1080/02699931.2015.1081349

Wolf, M., Horn, A. B., Mehl, M. R., Haug, S., Pennebaker, J. W., & Kordy, H. (2008). Computergestützte quantitative Textanalyse: Äquivalenz und Robustheit der deutschen Version des Linguistic Inquiry and Word Count. Diagnostica, 54(2), 85-98. https://doi.org/10.1026/0012-1924.54.2.85

Yentzen, E. (2008). Diccionario de emociones, actitudes y conductas. Santiago (Chile): Universidad Bolivariana.

Yzerbyt, V., Kuppens, T., & Mathieu, B. (2016). When talking makes you feel like a group: The emergence of group-based emotions. Cognition and Emotion, 30(1), 33-50. https://doi.org/10.1080/02699931.2015.1045454

Zijlstra, H., van Meerveld, T., van Middendorp, H., Pennebaker, J. W., & Geenen, R. (2004). De Nederlandse versie van de Linguistic Inquiry and Word Count (liwc), eenge computeriseerd tekst analyse programma [Dutch version of the Linguistic Inquiry and Word Count (liwc), a computerized text analysis program]. Gedrag & Gezondheid, 32, 273-283.