政治情感的动态探索性图分析

Introduction

High-quality images and video footage of politicians addressing the electorate have become prevalent throughout all forms of media, especially on online platforms. Broadcasted political performances can capture the attention of veoters and provide audiences with a great deal of information – not only about political agendas and issue stances, but also about the character and capabilities of political figures themselves (Benoit et al., 2003).

Part of a Special IssueNetwork PsychometricsThis article belongs to a curated special issue. Discover more research on this topic from the full collection.Explore the special issue →

At the same time, political communication has become increasingly emotional in recent decades, with emotional appeals now being a common element of almost every political speech. Populist leaders seem to be especially adept at crafting their public images by utilizing emotions (Engesser et al., 2017; Salmela & von Scheve, 2018). They are more likely overall to express emotions in their public performances, with a particular focus on negative ones (Bucy et al., 2020; Major & Tomašević, 2023; Widmann, 2021). This emotional framing of political communication contributes both to the moralization of social issues (Clifford, 2019) and the overall persuasiveness of appeals (Wirz, 2018). The use of negative emotions to emphasize the nefarious actions of the political establishment is also linked to the populist style of communication (Demasi et al., 2024). On the demand side, exposure to the negative emotional content of political campaigns is more likely to drive affective polarization among respondents scoring high in populist attitudes (Nai & Maier, 2024).

While numerous studies have explored the tendency of populist politicians to be more emotional or more likely to express and elicit certain types of emotions (Breeze, 2019; HidalgoTenorio & Benítez-Castro, 2022; Richman & Moorman, 2000; Widmann, 2021), there is a significant gap in the literature on the affective dynamics of emotions expressed by politicians in their political speeches. At the same time, the field of affective dynamics in psychology is rapidly developing, thanks to new data collection methods such as the Experience Sampling Method. This field has shifted its focus from viewing emotions as stable traits that switch on and off and has provided a framework in which emotions continually change, unfold, fluctuate, synchronize, merge, and influence each other (Bringmann et al., 2016; Kuppens & Verduyn, 2017; Kuppens et al., 2010).

However, these studies have primarily focused on the experience of emotions rather than their outward expression. When studying the emotional content of political communication, especially in its public-facing forms such as the affective dimensions of political speeches, our interest lies in the emotions that are visibly expressed by political leaders, rather than their internal states. As a result, the study of affective dynamics in political communication poses a challenging and underdeveloped problem.

How do politicians express emotions during a political speech? As public performances, speeches are constructed, scripted, tailored to a specific audience, and aimed at influencing public opinion and political behavior (Stewart et al., 2009; Sullivan & Masters, 1988). It is reasonable to assume that the emotional expression of a speaker during a political speech follows a specific pattern.

Starting from the idea that political speeches are heavily scripted, we divide the emotional content of speeches into segments characterized by positive or negative sentiment. Sentiment expresses the speaker’s opinion in relation to the topic of their speech (Dorle & Pise, 2018). Politicians strategically use positive and negative language and emotional expressions to affect voter behavior (Crabtree et al., 2020). Given this strategic use of sentiment, it is important to consider how it is expressed non-verbally, such as through facial expressions. The facial expression of emotion depends on the content of the speech, and strategic components of the speech can further be linked to the expression of positive sentiment, negative sentiment, or neutrality.

For example, if a politician is referring to a war crime being committed, we can expect to see a negative sentiment in their facial expression for a longer period of time. It takes time for the speaker to move away towards another topic and transition to a positive sentiment. In both cases, during an emotional episode, we can expect to see expressions of different negative or positive emotions with varying intensity. Because there is a tendency for the speaker to remain within the same sentiment for a longer period of time, we can expect to see a pattern of covariation of emotions over time.

Unlike in everyday life, where emotions are responses to situations, in a scripted setting like a political speech, emotional expressions are not triggered by external events. Instead, they follow the speaker’s internal representation of a specific part of the speech. This conceptualization of emotion dynamics aligns with the generative computational model presented by Ryan et al. (2023). However, it excludes the feedback loop where an emotional reaction influences the next situation due to the linear nature of the script. In this context, once the speaker focuses their attention on a particular part of the speech, their emotional reaction will follow. Facial expressions of these emotions provide a non-verbal signal that can give us information about segments of the speech characterized by positive or negative sentiment.

Given the arguments regarding the low signal value of facial expressions, supported by evidence that humans do not routinely express prototypical facial expressions (Barrett, 2011), we should not expect to retrieve rich information about a specific emotional state at any specific moment of the political speech. Instead, we can expect to see patterns in the expression of emotions over time, which can be informative about the structure of the emotional content of the speech.

The starting point of this study is the idea that labels used for categorizing facial expressions of emotions represent different states of affective trajectories at specific time points (Cunningham et al., 2013; Kirkland & Cunningham, 2012). Measuring the presence of each label through time can provide us with information about these affective trajectories in terms of neutrality, positive sentiment, and negative sentiment. In other words, although facial expressions are low signals for categories, they may be useful for recovering the latent affect dimension.

These expectations are based on consistent evidence (Thornton & Tamir, 2017) that people have highly accurate mental models of emotional transitions in others. Through a complex process of social learning, individuals tend to identify and predict regular patterns of emotional expressions in others (Zhao et al., 2022). These regularities are experienced in one’s own emotional life and observed in others. This knowledge is leveraged in social situations to form accurate predictions about the future emotional states of others. While these results are based on the interpersonal context, we can expect that the same principles apply to the public context of political speeches, as audiences observe the emotional transitions of the speaker in relation to the context and content of the speech.

Given the potential consequences of affective dynamics for political behavior in modern democracies, it is surprising that we know very little about the coevolution and covariation of different emotions, as well as the formation of emotional sequences and patterns. When a politician expresses anger during a speech, what comes next? Do they become less angry and more sad, or does a burst of anger followed by a sharp increase in positive emotion, such as happiness? Due to the lack of theoretical knowledge, the present study is exploratory in nature, aiming to fill this gap and provide preliminary descriptions of the patterns and structure of covariation of emotions expressed by political leaders during their public performances. More precisely, the aims of this study are to: (1) investigate the dimensionality of the structural organization of covariation of facial expressions of emotions in political speeches and (2) examine the differences in the structure of these relationships between leaders with different levels of populist rhetoric. The second aim helps us understand how the structure of the relationships between emotions changes in relation to the level of populism of the speaker, given that populist leaders are known to be more strategic in their use of emotions (Bucy et al., 2020).

In order to achieve these aims, we present a new multivariate time series dataset that describes the intensity of six emotions and neutral expressions. This dataset was obtained by applying transformer machine learning models with zero-shot learning capabilities (Tomaševic et al., 2024) for emotion recognition from facial expressions of political leaders during their public appearances. The dataset includes both populist and non-populist leaders from around the globe. Analyzing non-verbal communication provides a global perspective, unaffected by the specifics of language that may influence the dynamics and intensity of emotional expressions. The focus of this study is on the relationships between basic emotions in a simple data structure that describes the intensity of their facial expressions over time. We examine the structure of these relationships using a network psychometric method called dynamic Exploratory Graph Analysis (Golino et al., 2021a, 2022).

This study extends the application of dynamic EGA to new types of data and presents the first account of network models of emotion dynamics in political speeches. Our approach is based on the recommendations given by Barrett et al. (Barrett et al., 2019) for studies of facial expressions of emotions. It includes a big data approach, automated detection in natural conditions, integration of classical psychological methods and machine learning approaches, and sampling of visual material from different contexts and cultures. Furthermore, we combine a computational approach to facial expression detection with computational modeling of emotion dynamics, representing a ”natural confluence” (Hall et al., 2024) of approaches that is already being used in psychological research in a similar manner.

The field of emotion dynamics has experienced rapid growth due to the increasing availability of emotion time series data in psychology (Kuppens & Verduyn, 2017; Kuppens et al., 2010). However, this field has remained largely separate from the study of emotional content in political communication, particularly in the context of nonverbal communication by political leaders. This paper aims to bridge this gap by presenting results that contribute to the integration of network psychometrics (Borsboom et al., 2021), the psychometric network theory of emotions (Lange, 2023; Lange et al., 2020), and the computational study of political communication (Bucy, 2023; Joo et al., 2019; Major & Tomašević, 2023).

Data & Method

Emotion Time-Series

The list of source videos used in this study was obtained from a previous study where a deep learning-based computer vision algorithm was applied to a sample of 220 YouTube videos depicting political leaders from 15 different countries (Major & Tomašević, 2023). This study implemented a convolutional neural network for facial expression recognition, which was trained on the FER-2013 dataset (Goodfellow et al., 2013). However, one limitation of this approach is its reliance on a fixed set of emotion labels present in the training set (anger, disgust, fear, surprise, happiness, sadness, and neutral). A notable drawback of this approach is the imbalance between positively and negatively valenced emotions, which limits the modeling of emotion scores, especially their underlying dynamics.

In our work, we utilize the transforEmotion R package (Tomaševic et al., 2024), which leverages OpenAI’s CLIP model (Radford et al., 2021) to detect emotional expressions from images. Following previous research (Tomaševic et al., 2024), we define a customized set of labels consisting of three positive emotions (excitement, happiness, pride), three negative emotions (anger, fear, sadness), and a neutral expression.

This model is applied to every video from the source dataset in the following way: 300 frames are uniformly sampled from each video, and CLIP inference is applied to every extracted image. The output of the inference is numerical scores for each emotion label, representing the predicted proportion of an expressed emotion (FER scores). The FER scores range from 0 to 1, with 0 indicating the absence of a particular emotion and 1 indicating a ”clean” expression of a single emotion. These scores provide relative information about the intensity of the expressed emotion and can also be interpreted as the degree of uncertainty about the presence of a particular emotion in a complex facial expression.

For each video \(i\), we have a multivariate time series \(X^i_{j,t}\), where \(j \in \{1, \ldots, 7\}\) represents the score of one of the 7 emotional states, and \(t\) is the number of the processed frame. Each time series contains values between 0 and 1, which sum up to 1 across all time series: \(\sum_j X^i_{j,t} = 1\). For every video, we also have an ordinal measure of populist rhetoric from the Global Party Survey (GPS) dataset (Norris, 2020), denoted as \(1 \leq Y^i_{pop} \leq 4\).

We used the time series of FER scores for the six emotions as input data for Dynamic Exploratory Graph Analysis (Dynamic EGA) (Golino et al., 2021a).

Table 1

Descriptive statistics for FER and populism scores for all processed videos (n = 132, N = 33233 frames in total)

Emotion	Min.	Avg.	SD	Max.
Excitement	0.002	0.043	0.05	0.924
Happiness	0.000	0.021	0.025	0.528
Pride	0.003	0.056	0.057	0.784
Anger	0.003	0.169	0.111	0.748
Fear	0.003	0.173	0.088	0.934
Sadness	0.003	0.180	0.125	0.869
Neutral	0.004	0.357	0.185	0.912
Populism	1.000	2.721	1.092	4.000

Dynamic Exploratory Graph Analysis

Dynamic EGA (Golino et al., 2021a) combines techniques from the analysis of dynamical systems and the network psychometric method of Exploratory Graph Analysis (EGA) (Golino et al., 2020; Golino & Epskamp, 2017). Network methods for data analysis in psychology represent different combinations of multivariate statistics and network science methods used to investigate the structure of relationships in multivariate data (Borsboom et al., 2021; Christensen et al., 2023). In the case of EGA, it combines Gaussian Graphical Model (GGM) estimation and community detection to assess the dimensionality of the data. EGA has been proven to be an interesting and robust alternative to factor analysis methods for cross-sectional data (Golino et al., 2020), with innovative approaches to assessing the fit of network models using tools from information theory (Golino et al., 2021b).

The dynamic EGA method extends EGA to the analysis of time series and provides a platform for analyzing clusters of co-evolving variables over time. Unlike similar methods based on GGM estimation (Dablander et al., 2020; Epskamp et al., 2018; Haslbeck et al., 2021), EGA leverages methods from dynamical systems analysis, specifically the generalized local linear approximation (GLLA) (Boker et al., 2010), to estimate the first-order derivatives of the multivariate data. This provides insights into the correlations between the rate of change (or velocity) of the variables. This is important for studying affective dynamics because it helps identify which emotions co-occur with the same rate of change. Additionally, this method enables the identification of clusters of co-evolving emotions, allowing for the description of the dynamical affective system in terms of a network.

For each video \(i\) in our sample, the dynamic EGA starts by transforming each FER score \(X_{j}^i, j \in {1..7}\) into a time-delay embedding matrix \(\mathbf{Y}^{(n)}_i\). This matrix is of size \(M \times n\), where \(M = N – (n-1)\tau\), \(N\) is the length of the time series, \(n\) is the number of embedding dimensions, and \(\tau\) is the reconstruction delay. In this matrix, each row represents a phase-state vector:

\(\begin{equation}Y = [Y_1, Y_2, \ldots, Y_M]’\end{equation}\tag{1}\)

where Y_k represents the emotion state in the kth frame, given by the FER score of video i and emotion j:

\(\begin{equation}Y_k = [X^i_{j,k}, X^i_{j,k+\tau}, \ldots, X^i_{j,k + (n-1)\tau}]\end{equation}\tag{2}\)

This state can be seen as a short sequence describing what happens after frame k. It enables us to determine whether the value observed at frame k will remain stable over a short period of time or if it is part of a changing state, such as a sharp rise in anger. For all models, we set τ = 1 and implement a grid search for the number of dimensions n. We select the value of n that provides the best fit of the resulting model to the data, as measured by the Total Entropy Fit Index (Golino et al., 2021b). Across different models, the range of n is between 5 and 20. Given that the time distance between frames in the dataset is, on average, 3 seconds (for details, see Major & Tomašević, 2023), this means that each emotion state provides information about the microdynamics of the expression of a particular emotion within a range of 15 to 60 seconds.

The next step of the analysis is to apply GLLA, which estimates how the time-delayed variable changes over time. The zeroth derivative obtained by the GLLA gives us the final transformation of the observed FER score after applying time-delay embeddings. The first derivative obtained by the GLLA gives us the rate of change of a particular emotion over time. Negative firstorder derivatives indicate a reduction in the intensity of the emotion at that specific time point in the video. The output of GLLA for the zeroth and first-order derivatives is combined for each emotion variable, resulting in two M ×7 matrices D_i,0 and D_j,0 for each video.

These matrices are used as inputs for estimating the following Gaussian graphical models using default estimation methods provided by the EGAnet package (Golino & Christensen, 2023):

• Model 1: Single population-level model, FER scores
• Model 2: Single population-level model, first derivatives
• Model 3: Four subgroup-level models, FER scores

Population-level models are estimated from large matrices D₀ and D₁ created by stacking the corresponding matrices estimated from individual videos. Group models are estimated from a similar type of matrix, filtered based on the level of populism using the Type_Populism variable from the GPS dataset. Videos featuring leaders for which the GPS Populism variable is missing are excluded from the analysis.

We used the Ergodicity Information Index (EII) and its bootstrap test (Golino et al., 2022) to determine whether the structure of the relationships between emotions over time in our sample is best represented by multiple individual structures (multiple individual or subgroup structures) or a single between-individual structure (unique single network). This index quantifies the relative algorithmic complexity of the population structure compared to multiple individual structures. The result of the EII bootstrap test will indicate whether the populationlevel network exhibits ergodicity, meaning that little information is lost by representing the affective dynamics of each individual with a single network structure.

Results

After processing the data as described in the previous section, we excluded videos that had more than 10% missing rows. This means that more than 10% of the selected frames either didn’t contain the face of the speaker or the face was not successfully detected. The Dynamic EGA models were estimated on a sample of 132 videos, with an average length of 251.76 frames, totaling 33,233 frames. In other words, we have 132 multivariate time series of 7 emotions, each with an average length of 251 frames.

We estimated a population-level Dynamic EGA model of FER scores for the entire sample (Model 1). This model captures the structure of cooccurrence of emotions over time. The bootstrap Ergodicity Index test showed that the empirical Ergodicity Information Index (EII) was significantly different (p = 0.02, see section A in the SI) from what would be expected from random variation in the population structure. This indicates that significant information is lost when aggregating the results into a single population network. Therefore, we can interpret this network as only representing the average covariation of emotions through time based on our sample. The network is represented on the left panel of Figure 1 and shows a two-factor structure of co-varying FER scores over time.

On the left side of the graph shown in this panel, we have three negative emotions: fear, anger, and sadness. On the right side, we have three positive emotions: pride, happiness, and excitement. This Dynamic EGA model shows distinctive communities for each class of emotions. Within the negative community, anger and fear are strongly positively correlated, while there is a negative correlation between the expression of fear and sadness. On the other hand, all three positive emotions are positively correlated. Looking at the strength of the correlations, we see that the primary axes of this dynamic system are edges denoting the covariation of three pairs of emotions: anger and fear, fear and sadness, and pride and happiness. When we examine the network of first-order derivatives (Model 2), we observe a similar structure with two communities of positive and negative emotions. However, due to its strong negative correlations with pride and happiness, anger is grouped together with the positive emotions community. The network of first-order derivatives is depicted in the right panel of Figure 1.

Figure 1

Network representing the affective dynamics of political speeches. Nodes represent emotions and edges represent partial correlations between FER scores (left panel, Model 1) and the first derivative of FER scores (right panel, Model 2) over time. The thickness of the edges represents the strength of the correlations.

Table 2 presents the network scores for each emotion in both models. Network scores represent the standardized sum of correlations that nodes have within their respective communities. In Model 1, we observe that pride has the strongest score within the positive emotion community and a strong cross-community score, indicating that its expression is highly influenced by the expression of other emotions. Fear has the largest score within the negative emotion community, while sadness has the largest cross-community score. It is worth noting that anger is almost completely isolated from the positive emotions community.

Table 2

Network scores for FER scores(Model 1) and their first derivatives (Model 2). Only scores greater than 0.1 are shown.

2*Emotions	Model 1		Model)
	1	2	1	2
Pride	0.317	-0.201	0.314
Happiness	0.314	0.123	0.413
Excitement	0.266	0.173	0.318	0.227
Fear	0.199	0.317		0.209
Anger		0.234	-0.269	0.172
Sadness	-0.233	-0.197	0.246	0.209

In Model 2, happiness and excitement have the highest network scores within their communities. Changes in these emotions occur at a similar rate of change, and they are strongly negatively correlated with negative emotions. This suggests that a rapid increase in happiness can be followed by a rapid increase in excitement and a rapid decrease in anger. The rate of change of happiness and excitement is most strongly associated with changes in other emotions in this case.

However, this network structure does not represent the emotion dynamics of all videos in our sample. To examine the subtle differences in structure among videos with leaders of varying strength of populist rhetoric, we can use additional dynamic EGA models for subsamples. Figure 2 shows two separate networks representing the Dynamic EGA models of FER scores.

Figure 2

Network representing affective dynamics of political speeches according to different levels of populist rhetoric (1: strong pluralists, 4: strong populists). Nodes represent basic emotions and edges represent partial correlations of FER scores. Thickness of the edge represents the strength of the correlations.

There are several differences in the network structures. First of all, for strong pluralists, the network shows a unidimensional character. It has been shown that pluralist leaders are more neutral in their expressions (Major & Tomašević, 2023), and here we see that the correlations between FER scores do not exhibit a high level of structural organization between positive and negative emotions. Notably, there is no strong correlation between fear and anger (present in other networks), which can be explained by the tendency of pluralists not to rely on strong expressions of negative emotions.

As we move towards stronger levels of populist rhetoric, key correlations become stronger and the networks exhibit a structure with two communities. Correlations between positive emotions become stronger, as well as fear-anger and fear-sadness correlations. Figure 3 shows the standardized network scores for all emotions across populism groups, where we find a two-dimensional network structure. We can see that with the increase in the level of populism, the score of anger decreases, while the score of happiness increases. This tells us that with the increase of populist rhetoric, anger becomes a more autonomous emotion and less contingent on the expression of other emotions, while the reverse is true for happiness as it becomes more constrained by the expression of other positive emotions in both populist groups.

Figure 3

Network scores in two-dimensional structures across different levels of populism (Model 3).

Discussion

The global rise of populism in recent years has highlighted the importance of studying the affective aspects of political communication (Engesser et al., 2017; Salmela & von Scheve, 2018). While numerous studies have explored the tendency of populist politicians to be more emotional or more likely to express certain types of emotions (Hidalgo-Tenorio & Benítez-Castro, 2022; Widmann, 2021), there has been a significant gap in the literature on the affective dynamics of emotions expressed by politicians in their political speeches. Our study addresses this gap by examining the structure and dynamics of emotional expressions in political speeches using a novel combination of machine learning-based emotion recognition and Dynamic Exploratory Graph Analysis.

Our main finding is that the temporal covariation of emotions in political videos exhibits a twodimensional structure, indicating different patterns of covariation between positive and negative emotions. Specifically, positive emotions such as pride, happiness, and excitement are positively correlated with each other. On the other hand, negative emotions show a positive correlation between fear and anger, while both fear and anger are negatively correlated with sadness. This suggests that the expression of emotions in political speeches can be represented as a dynamical system structured around two latent dimensions, where the expression of one emotion is contingent on the expression of the emotion from the same cluster.

The first aim of the study was to investigate the structural organization (dimensionality) of the covariation of facial expressions of emotions in political speeches. Both the dependencies and covariations of FER scores and the correlations of first-derivatives (rate of change) of FER scores are well described by a two-dimensional network structure. Regarding the second aim, we investigated the differences in the structure of these relationships between leaders with different levels of populist rhetoric. We found that the network structure in subsamples of videos grouped by different levels of populist rhetoric is two-dimensional, except for strong pluralists where it is unidimensional. However, we observed variations in the strength of connectivity between emotions in the networks of different groups. Specifically, we found that anger becomes more autonomous with the increase of populist rhetoric, while happiness becomes more constrained by the expression of other positive emotions. These differences highlight the strategic use of emotions in political communication and the distinct affective landscapes created by populist and pluralist leaders.

Our study started from the idea that facial expressions of emotions can be taken as a noisy signal of a specific state of an affective trajectory taken by the speaker during a public appearance. If a politician makes an angry face at a particular point during a speech, it is a signal that indicates that we are in the segment of the speech characterized by negative sentiment, and the expressions of the speaker are helping the audience navigate the affective landscape painted by the speech. Strong patterns of correlation over time, both in the expression of emotions and their rate of change, suggest that emotion expressions are contingent on one another, and the constraints placed on the emotions are stronger within the same cluster of emotions. This points towards the idea that the expression of emotions in political speeches is structured around two latent dimensions whose intensities determine the affective trajectories of the speech.

Future work and extensions of this study should be performed on a larger sample with a more diverse selection of political speeches and videos, such as debates and interviews. Additionally, integrating the detection of facial expressions of emotions with sentiment analysis of political speeches holds great potential for future computational studies in political communication. While previous research has focused on publicly available text data retrieved from political speeches (Maerz & Schneider, 2020; Maia Polo et al., 2023), the emergence of machine learning tools like Whisper AI (Radford et al., 2022) allows for the retrieval of speech transcripts from video footage. By combining these two data sources, we can gain unprecedented insight into the emotional content of public performances by political leaders worldwide. The methods presented in this study can be expanded to incorporate different data sources, especially considering that Dynamic EGA has already been applied to textual data (Golino et al., 2021a).

Our study also has significant limitations. First, the dataset used in this study is relatively small and is not representative of the global political landscape. Second, the approach based on OpenAI’s CLIP model (Tomaševic et al., 2024) has not been validated by human coders, and it is unclear how successfully it performs zero-shot classification in the detection of emotions in political speeches. Therefore, the results of this study should be interpreted with caution, and it is recommended to replicate the study on a larger dataset with a more accurate emotion recognition algorithm.

In conclusion, our study provides novel insights into the dynamics of emotions in political speeches through the application of advanced computational and network analysis techniques. By revealing the interconnected nature of emotional expressions and the differences between populist and pluralist leaders, we open new avenues for understanding the strategic use of emotions in political communication. As research in this area continues to evolve, we anticipate further advances in our ability to decode the complex emotional landscapes of political discourse, contributing to a deeper understanding of the role of emotions in shaping political realities.

Data Availability

Emotion time-series dataset in CSV format is available at: https://osf.io/3gduc/. The OSF repository also contains R scripts used for the main and supplementary analyses, as well as all outputs of the analysis.

Conflicts of Interest

The authors report no competing interests.

Author Contributions

A.T. designed the study, wrote the analysis code, analyzed the data and drafted the paper. S.M. designed the study, drafted the introduction section and edited the entire paper. Both authors provided critical revisions.

References

Barrett, L. F. (2011). Was Darwin Wrong About Emotional Expressions? Current Directions in Psychological Science, 20(6), 400–406. Retrieved from https://doi.org/10.1177/0963721411429125

Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements. Psychological Science in the Public Interest, 20(1), 1–68. https://doi.org/10.1177/1529100619832930

Benoit, W. L., Hansen, G. J., & Verser, R. M. (2003). A meta-analysis of the effects of viewing U.S. presidential debates.    Communication Monographs, 70(4), 335–350.  https://doi.org/10.1080/0363775032000179133

Boker, S. M., Deboeck, P. R., Edler, C., & Keel, P. K. (2010). Generalized local linear approximation of derivatives from time series. Statistical methods for modeling human dynamics: An interdisciplinary dialogue, 161–178. Retrieved 2023-03-17, from https://pure.mpg.de/pubman/faces/ViewItemOverviewPage.jsp?itemId=item (Publisher: Routledge)

Borsboom, D., Deserno, M. K., Rhemtulla, M., Epskamp, S., Fried, E. I., McNally, R. J., … Waldorp, L. J. (2021). Network analysis of multivariate data in psychological science. Nature Reviews Methods Primers, 1(1), 1–18. https://doi.org/10.1038/s43586-021-00055-w

Breeze, R. (2019). Emotion in politics: Affective-discursive practices in UKIP and Labour. Discourse & Society, 30(1), 24–43. https://doi.org/10.1177/0957926518801074

Bringmann, L. F., Pe, M. L., Vissers, N., Ceulemans, E., Borsboom, D., Vanpaemel, W., … Kuppens, P. (2016). Assessing Temporal Emotion Dynamics Using Networks. Assessment, 23(4), 425–435. https://doi.org/10.1177/1073191116645909

Bucy, E. P. (2023). Politics through machine eyes: What computer vision allows us to see. Journal of Visual Political Communication, 10(1), 59–68. https://doi.org/10.1386/jvpc_00023_1

Bucy, E. P., Foley, J. M., Lukito, J., Doroshenko, L., Shah, D. V., Pevehouse, J. C., & Wells, C. (2020). Performing populism: Trump’s transgressive debate style and the dynamics of Twitter response. New Media & Society, 22(4), 634–658. https://doi.org/10.1177/1461444819893984

Christensen, A. P., Garrido, L. E., Guerra-Peña, K., & Golino, H. (2023). Comparing community detection algorithms in psychometric networks: A Monte Carlo simulation. Behavior Research Methods. https://doi.org/10.3758/s13428-023-02106-4

Clifford, S. (2019). How Emotional Frames Moralize and Polarize Political Attitudes: Moralizing Effects of Persuasive Frames.    Political Psychology, 40(1), 75–91. https://doi.org/10.1111/pops.12507

Crabtree, C., Golder, M., Gschwend, T., & Indridason, I. H. (2020). It is not only what you say, it is also how you say it: The strategic use of campaign sentiment. The journal of politics, 82(3), 1044–1060. https://doi.org/10.1086/707613

Cunningham, W. A., Dunfield, K. A., & Stillman, P. E. (2013). Emotional States from Affective Dynamics. Emotion Review, 5(4), 344–355. https://doi.org/10.1177/1754073913489749

Dablander, F., Ryan, O., & Haslbeck, J. M. B. (2020). Choosing between AR(1) and VAR(1) models in typical psychological applications. PLOS ONE, 15(10), e0240730. https://doi.org/10.1371/journal.pone.0240730

Demasi, C., McCoy, J., & Littvay, L. (2024). Influencing people’s populist attitudes with rhetoric and emotions: An online experiment in the united states. The American behavioral scientist. https://doi.org/10.1177/00027642241240359

Dorle, S., & Pise, N. (2018). Political sentiment analysis through social media. In 2018 second international conference on computing methodologies and communication (iccmc) (pp. 869–873). IEEE. https://doi.org/10.1109/iccmc.2018.8487879

Engesser, S., Ernst, N., Esser, F., & Büchel, F. (2017). Populism and social media: How politicians spread a fragmented ideology. Information, Communication & Society, 20(8), 1109–1126. https://doi.org/10.1080/1369118X.2016.1207697

Epskamp, S., Waldorp, L. J.,  Mõttus, R., & Borsboom, D. (2018). The Gaussian Graphical Model in Cross-Sectional and Time-Series Data. Multivariate Behavioral Research, 53(4), 453–480. https://doi.org/10.1080/00273171.2018.1454823

Golino, H., & Christensen, A. (2023). Eganet: Exploratory graph analysis – a framework for estimating the number of dimensions in multivariate data using network psychometrics [Computer software manual]. (R package version 2.0.0)

Golino, H., Christensen, A. P., Moulder, R., Kim, S., & Boker, S. M. (2021). Modeling Latent Topics in Social Media using Dynamic Exploratory Graph Analysis: The Case of the Right-wing and Left-wing Trolls in the 2016 US Elections. Psychometrika. https://doi.org/10.1007/s11336-021-09820-y

Golino, H., Christensen, A. P., & Nesselroade, J. R. (2022). Towards a psychology of individuals: the ergodicity information index and a bottom-up approach for finding generalizations (preprint). PsyArXiv. https://doi.org/10.31234/osf.io/th6rm

Golino, H., Moulder, R., Shi, D., Christensen, A. P., Garrido, L. E., Nieto, M. D., … Boker, S. M. (2021). Entropy Fit Indices: New Fit Measures for Assessing the Structure and Dimensionality of Multiple Latent Variables. Multivariate Behavioral Research, 56(6), 874–902. https://doi.org/10.1080/00273171.2020.1779642  

Golino, H., Shi, D., Christensen, A. P., Garrido, L. E., Nieto, M. D., Sadana, R., … MartinezMolina, A. (2020). Investigating the performance of exploratory graph analysis and traditional techniques to identify the number of latent factors: A simulation and tutorial. Psychological Methods, 25(3), 292–320. https://doi.org/10.1037/met0000255

Golino, H. F., & Epskamp, S. (2017). Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research. PLOS ONE, 12(6), e0174035. https://doi.org/10.1371/journal.pone.0174035

Goodfellow, I. J., Bulatov, Y., Ibarz, J., Arnoud, S., & Shet, V. (2013). Challenges in representation learning: A report on three machine learning contests. arXiv preprint arXiv:1307.0414.

Hall, N. T., Hallquist, M. N., Martin, E. A., Lian, W., Jonas, K. G., & Kotov, R. (2024). Automating the analysis of facial emotion expression dynamics: A computational framework and application in psychotic disorders. Proceedings of the National Academy of Sciences, 121(14), e2313665121. https://doi.org/10.1073/pnas.2313665121

Haslbeck, J. M. B., Bringmann, L. F., & Waldorp, L. J. (2021). A Tutorial on Estimating Time-Varying Vector Autoregressive Models. Multivariate Behavioral Research, 56(1), 120–149. https://doi.org/10.1080/00273171.2020.1743630

Hidalgo-Tenorio, E., & Benítez-Castro, M.-A. (2022). Trump’s populist discourse and affective´ politics, or on how to move ‘the People’ through emotion. Globalisation, Societies and Education, 20(2), 86–109. https://doi.org/10.1080/14767724.2020.1861540

Joo, J., Bucy, E. P. T., & Seidel, C. (2019). Automated Coding of Televised Leader Displays: Detecting Nonverbal Political Behavior With Computer Vision and Deep Learning. International Journal of Communication. https://ijoc.org/ index.php/ijoc/article/view/10725

Kirkland, T., & Cunningham, W. A. (2012). Mapping emotions through time: How affective trajectories inform the language of emotion. Emotion, 12(2), 268–282. https://doi.org/10.1037/a0024218

Kuppens, P., Oravecz, Z., & Tuerlinckx, F. (2010). Feelings change: Accounting for individual differences in the temporal dynamics of affect. Journal of Personality and Social Psychology, 99(6), 1042–1060. https://doi.org/10.1037/a0020962

Kuppens, P., & Verduyn, P. (2017). Emotion dynamics. Current Opinion in Psychology, 17, 22–26. Retrieved 2023-09-18, from https://www.sciencedirect.com/science/ article/pii/S2352250X16302019

Lange, J. (2023). Embedding Research on Emotion Duration in a Network Model. Affective Science. https://doi.org/10.1016/j.copsyc.2017.06.004

Lange, J., Dalege, J., Borsboom, D., van Kleef, G. A., & Fischer, A. H. (2020). Toward an Integrative Psychometric Model of Emotions. Perspectives on Psychological Science, 15(2), 444–468. https://doi.org/10.1177/1745691619895057

Maerz, S. F., & Schneider, C. Q. (2020). Comparing public communication in democracies and autocracies: Automated text analyses of speeches by heads of government. Quality & Quantity, 54(2), 517–545. https://doi.org/10.1007/s11135-019-00885-7

Maia Polo, F., Saxena, S., & Alexopoulos, C. (2023). Stop the War, for Godsake!: A Text Analysis of Speeches of Volodymyr Oleksandrovych Zelenskyy, the Ukrainian President. International Journal of Public Administration, 0(0), 1–19. https://doi.org/10.1080/01900692.2023.2199174

Major, S., & Tomasevic, A. (2023). The Face of Populism: Examining Differences in Facial Emotional Expressions of Political Leaders Using Machine Learning. ArXiv(arXiv:2304.09914).

Morzy, M., Kajdanowicz, T., & Kazienko, P. (2017). On Measuring the Complexity of Networks: Kolmogorov Complexity versus Entropy. Complexity, 2017, e3250301. https://doi.org/10.1155/2017/3250301

Nai, A., & Maier, J. (2024). Polarized Populists: Dark Campaigns, Affective Polarization, and the Moderating Role of Populist Attitudes. The American behavioral scientist, 00027642241242056. https://doi.org/10.1177/00027642241242056  

Norris, P. (2020). Measuring populism worldwide. Party Politics, 26(6), 697–717. https://doi.org/10.1177/1354068820927686

Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., … Sutskever, I. (2021). Learning transferable visual models from natural language supervision. arXiv. http://arxiv.org/abs/2103.00020

Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., & Sutskever, I. (2022). Robust Speech Recognition via Large-Scale Weak Supervision (No. arXiv:2212.04356). arXiv. https://doi.org/10.48550/arXiv.2212.04356

Richman, J. S., & Moorman, J. R. (2000). Physiological time-series analysis using approximate entropy and sample entropy. American Journal of Physiology-Heart and Circulatory Physiology, 278(6), H2039-H2049. https://doi.org/10.1152/ajpheart.2000.278.6.H2039

Ryan, O., Dablander, F., & Haslbeck, J. (2023). Towards a generative model for emotion dynamics. PsyArXiv. https://psyarxiv.com/x52ns

Salmela, M., & von Scheve, C. (2018). Emotional Dynamics of Right- and Left-wing Political Populism. Humanity & Society, 42(4), 434–454. https://doi.org/10.1177/0160597618802521

Stewart, P. A., Waller, B. M., & Schubert, J. N. (2009). Presidential speechmaking style: Emotional response to micro-expressions of facial affect. Motivation and emotion, 33(2), 125–135. https://doi.org/10.1007/s11031-009-9129-1

Sullivan, D. G., & Masters, R. D. (1988). “happy warriors”: Leaders’ facial displays, viewers’ emotions, and political support. American journal of political science, 32(2), 345. https://doi.org/10.2307/2111127

Thornton, M. A., & Tamir, D. I. (2017). Mental models accurately predict emotion transitions. Proceedings of the National Academy of Sciences, 114(23), 5982–5987. https://doi.org/10.1073/pnas.1616056114

Tomaˇsevic, A., Golino, H., & Christensen, A. P. (2024). Decoding emotion dynamics in videos using dynamic exploratory graph analysis and zero-shot image classification: A simulation and tutorial using the transforEmotion R Package. PsyArXiv. https://doi.org/10.31234/osf.io/hf3g7

Widmann, T. (2021). How Emotional Are Populists Really? Factors Explaining Emotional Appeals in the Communication of Political Parties. Political Psychology, 42(1), 163–181. https://doi.org/10.1111/pops.12693

Wirz, D. (2018). Persuasion Through Emotion? An Experimental Test of the Emotion-Eliciting Nature of Populist Communication. International Journal of Communication, 12, 1114– 1138. https://doi.org/10.5167/UZH-149959

Zhao, Z., Thornton, M. A., & Tamir, D. I. (2022). Accurate emotion prediction in dyads and groups and its potential social benefits. Emotion, 22(5), 1030–1043. https://doi.org/10.1037/emo0000890

Supplementary Information: Ergodicity information index

Ergodicity Information Index (EII) (Golino et al., 2022) was inspired by algorithmic information theory and the concept of Kolmogorov complexity (Morzy et al., 2017). The EII calculates the amount of information lost when representing a set of measures (emotion time-series) as a single interindividual structure instead of multiple individual structures (within-video structures). Larger values of EII indicate that more information is encoded in intraindividual networks, and a significant amount of information is lost when representing the entire population with a single network structure.

The Bootstrap EII test begins by obtaining a bootstrap distribution of EII values assuming that all participants in the data have the population structure. It then compares this null distribution to the empirical EII value at the population level. Significant differences indicate that the empirical data cannot be generated from an ergodic process, and the population structure is insufficient to describe all the dynamics of the videos.

In the case of our population networks, the empirical EII is 1.9 (dashed blue line in Figure 4) and significantly differs from the bootstrap distribution of EII values (p = 0.0198). This suggests that the population network is not a good representation of the affective dynamics of all the videos, and there is a significant amount of information lost by representing the entire population with a single network structure.

Figure 4

Histogram of bootstrapped Ergodicity Information Index for all videos.