The (In)effectiveness of Populist Rhetoric: A Conjoint Experiment of Campaign Messaging

The (In)effectiveness of Populist Rhetoric: A Conjoint Experiment of Campaign Messaging

Resources

• Figures & Tables

Authors: Yaoyao Dai and Alexander Kustov

Journal: Political Science Research and Methods (2023)

DOI: 10.1017/psrm.2023.55

---

Abstract

Is populism electorally effective and, if so, why? Scholars agree that populism is a set of people-centric, anti-pluralist, and anti-elitist ideas that can be combined with various ideological positions. It is difficult, yet important, to disentangle populism from its hosting ideology in evaluating populism’s effectiveness and its potential conditional effects on the hosting ideology. We conduct a novel US conjoint experiment asking respondents to evaluate pairs of realistic campaign messages with varying populism-related messages and hosting policy positions given by hypothetical primary candidates. Although party-congruent policy positions are expectedly much more popular, we find that none of the populist features have an independent or combined effect on candidate choice.

Over the past two decades, populist parties and candidates have been gaining increasing electoral support across the world. Populism has little programmatic content and is almost always combined with other ideologies in reality [@Mudde2018]. As a result, it is easy to conflate the effects of populism with those hosting ideologies or related policy stances when they are employed together [@hunger2022s]. Despite the observed success of populist parties and candidates, recent survey experiments find that populist appeals are much less effective than programmatic policy priorities in attracting votes [@neuner2020populist; @silva2022populism]. These results further highlight the importance of disentangling these two distinct concepts when evaluating the effectiveness of populism. So, does combining populism with certain policy positions make populist claims-making or related policies more appealing to voters? How effective is populism compared to non-populist alternatives, such as those related to pluralism? Since populism is multi-dimensional, to what extent is the effect of populism based on each of its components or on their particular combination?

To determine which combinations of populist ideas and various ideological stances are most effective for candidate choice, we conduct a novel conjoint survey experiment. In our study, we ask our respondents from a nationally diverse US online survey to evaluate four pairs of realistic campaign messages by hypothetical primary candidates for their party in the next US House of Representatives elections. The campaign messages have been randomized to include the necessary components of populism, common economic and immigration-related policy stances, and relevant background candidate characteristics. In line with the populism literature, our general expectation has been a positive interaction effect between various components of populist messaging so that they are especially effective when combined together. We have also hypothesized that populist campaign messaging might be especially effective among those voters who hold populist attitudes.

Overall, although party-congruent policy positions are expectedly much more popular among the respondents, we find that none of the necessary components of populism have had an independent or a combined effect on candidate choice, including for those who hold populist attitudes themselves. In addition, we do not find any significant interaction effects of populism and various policy positions. We conclude by discussing the implications of these findings for understanding the role of populism in politics and its apparent (in)effectiveness on vote choice.

Populism and Its Effectiveness

Although populism is still a contested concept, many scholars across disciplines have increasingly adopted a minimum “ideational” definition of populism over the past decade. According to this definition, populism is primarily a set of ideas depicting society as divided into two homogeneous and antagonistic groups-the “good” people and the “corrupt” elites-and emphasizing that politics should reflect the general will of the people [@Mudde2004; @Mudde2018; @hawkins2018ideational]. Unlike classical ideologies such as socialism or nationalism, populism lacks programmatic content and can be combined with a wide range of “hosting” ideologies from across the left-right spectrum [@Mudde2018].

As populist parties from the left and right have gained significant voter support worldwide in the past decades, there have been significant research efforts to explain the reasons behind the electoral success of populist parties and candidates. Some scholars have investigated various individual attitudes, such as dissatisfaction with representative democracy, and structural factors, such as financial and immigration crises, that might fuel the demand for populism [@Bakker2016a; @Akkerman2013; @kustov2023testing]. However, since populism is always attached to other ideologies in practice, it has been difficult to disentangle its independent effects in observational studies.

The two recent studies that have managed to separate “thin populism” from the policy positions of populist parties in conjoint experiments find that policy positions, in and of themselves, have much larger effects on respondents’ vote choice than populism [@neuner2020populist; @silva2022populism]. However, we still lack empirical evidence on the interaction between the components of populism and their hosting ideologies. Building on this new strand of experimental research, we seek to further disentangle the effect of populism from the hosting ideology, as well as explicitly test the possible interaction effect between the two.

Although parties or voters can only be considered “populist” if they express or hold all of the core components of populism [@wuttke2020whole; @hawkins2018ideational], when one considers real-world political campaigns, the presence of all dimensions is much less common than the various combinations of two dimensions [@engesser2017populism]. Similarly, only the most politically sophisticated voters hold consistent populist or anti-populist ideology [@spruyt2021ideologically]. In other words, it is possible that it might be beneficial for political parties and candidates to only utilize or emphasize certain combinations of populism’s main components.

Several recent experimental studies have investigated the effects of these components of populism on voter support and mobilization. Some scholars find that the morally charged anti-elitist rhetoric is the most appealing aspect of populist candidates, especially among those who are low in agreeableness [@silva2022populism; @Bakker2021; @bos2020effects]. At the same time, other scholars find that people-centrism is the most appealing populist aspect [@neuner2020populist]. However, it is still unclear whether each of these components is more effective in the presence of other components and certain policy positions. This paper seeks to fill this gap by explicitly testing the interactional effects between the components of populism.

Data and Methods

In April 2022 we administered a nationally diverse US online survey experiment on Amazon MTurk (N = 1004) with quality controls recommended in the literature [@Kennedy2020]. Similar to analogous online surveys, our respondents were slightly more likely to be young, male, white, and educated than the general population. 36% of our respondents identified as Republicans and 58% as Democrats (including leaners). 56% of our respondents also could be categorized as displaying populist attitudes. For full descriptive statistics comparing our sample to a probability-based benchmark, see Table A3.

While the US political system offers much less opportunity for organized populist parties than countries with proportional representation, it still provides ample opportunities for populist candidacies [@Lee2019]. Indeed, populist campaigns have been common among political candidates across all parties in the US [@Bonikowski2015; @hawkins2018measuring; @Dai2022]. Therefore, the US case arguably provides a good opportunity to create realistic candidate profiles with a combination of various components of populism and policy positions in a single experiment.

The survey, which was fielded as a part of the broader omnibus study, first presented our respondents with a conjoint task vignette and four pairs of candidate choices, and, following a few unrelated questions, asked about their demographic characteristics and populist attitudes. Our conjoint experiment asked our respondents to choose their preferred candidate from four realistic US House primary candidate profile pairs for their party with randomized attributes. We focus on hypothetical primary contests since any choice of candidates in the general elections would come down to the candidates’ partisan identity, which we would like to avoid since it is not our primary variable of interest [@kirkland2018candidate].

Compared to previous conjoint experiments on the topic that introduce populism in the form of candidates’ listed policy priorities [@Neuner2022; @silva2022populism], our study embeds the components of populism in the form of candidates’ campaign message features. Populist ideas embedded in campaign messages arguably make the overall conjoint procedure more realistic, and better resemble the information voters receive during elections. Our approach also enables us to better disentangle the potential electoral effects of populism and its components with various policy content from this content itself. We operationalize populism along the dimensions of people-centrism, anti-elitism, and anti-pluralism. In addition to populist messages, we specify the non-populist attributes using the direct ideological opposites of populism, such as elitism and pluralism (for a detailed discussion on the operationalization of populism, see Appendix B).

We also include policy positions (immigration and economy) and background candidate characteristics. All candidate attributes were randomly selected from those discrete predefined levels. The main binary outcome was a forced choice from a given pair of candidate profiles [@Hainmueller2014causal]. The list of all conjoint attributes and their possible levels is presented in Appendix B Table B1. Each theoretically distinct dimension of populist and non-populist elements had four distinct manifestations paraphrased from real-world political campaigns. This helped us ensure the relative realism of our experiment despite the fictitious nature of our candidates [@Brutger2020abstraction].

To test our main empirical expectation regarding the effectiveness of populist rhetoric, we follow the conventional empirical approach and estimate average marginal component effects (AMCEs) of various attributes on candidate choice using simple linear regression with robust standard errors clustered by respondent. The AMCE represents the average difference in the probability of being chosen when comparing two different attribute values (e.g., a candidate who worked as a lawyer compared to a candidate who worked as a firefighter) where the average is taken over all other possible attribute combinations. To ensure our results are not driven by our choice of reference categories and to make valid subgroup comparisons, we also provide the estimates of marginal means in addition to AMCEs [@Leeper2020measuring].

To simplify the presentation of our results, our main specification recodes pro-immigration and liberal economic stances as party-congruent for those who identify as Democrats and anti-immigration and conservative economic stances as party-congruent for those who identify as Republicans (with 5% of pure Independent respondents dropped from the analysis). For our subgroup analyses, in addition to partisan identification, we measure the respondents’ populist attitudes using a previously validated set of questions from [@Akkerman2013] and [@wuttke2020whole] adjusted for the US context. In line with [@wuttke2020whole], we define “populists” as only those respondents who at least weakly endorse all core components of populism. For the full list of items, see Appendix C. For power analysis, see Appendix D.

Analysis and results

Our main results are summarized in Figures 1-3 below. Table A1 provides the detailed coefficients for the models in Figures 1-3 (model 1). Figures A1-3 provide an original coding of policy positions and an alternative coding of our two complex populist-related conjoint treatments as the four simpler, mutually exclusive binary attributes with no change in the underlying results. For other robustness checks, see Appendix E.

Figure: fig:Figure1

Overall, as can be seen from Figures fig:Figure1 and A1, populist rhetoric does not have a statistically significant effect on candidate choice. At the same time, the effects of policy stances are both statistically and substantively significant. A candidate with a party-congruent stance on immigration and the economy has a respectively 7-8 percentage point higher probability of being selected. Importantly, this holds true regardless of the use of populist rhetorical elements in their campaign message (see Tables A1 and tab:interactionYD). Unlike populist rhetoric, these sizable effects of either economic or immigration issue positioning are also robust to Holm-Bonferroni correction for multiple comparisons across all 128 coefficients and models tested in this paper $(p < 0.01)$. The magnitude of these effects is also comparable to the estimates for congruent issue positioning uncovered in the previous candidate choices experiments [@Graham2020a; @silva2022populism].

Of course, the lack of statistical significance or the failure to reject the null hypothesis does not necessarily imply that populist rhetoric has no effect. To test whether there is a practical null effect, we follow the recent methodological literature and consider the “two one-sided test” (TOST) procedure [@lakens2018equivalence]. In particular, we test for equivalence against the AMCE interval of $0.04$, which is the minimum effect size that can be considered of substantive political significance given our experimental design. Since the observed confidence intervals for all possible populist features compared to non-populist features are fully contained in this interval, we consider this as evidence for the null practical effect of populist rhetoric.

We then test whether the effects differ among certain subgroups of voters such as among those who hold populist attitudes (Figures fig:FigureA3, A2, and A6). Interestingly, we see that such people are somewhat less sensitive to economic policy. Nonetheless, populists are not more responsive to populist rhetoric than non-populist voters. This holds true across various operationalizations of populist attitudes (see Appendix A).

Figure: fig:FigureA3

To further test whether there is an interaction effect between populism and various policy positions, we interact recoded populism variable with the recoded party congruent positions and the original immigration and economic policy positions (see Table tab:interactionYD). Populist messages are the ones that contain all of the anti-pluralist, people-centric and moralized anti-elitist attributes. We also report results from models with interaction terms between each of the binary components of populism and policy positions in Table A2. We find no significant interaction effects between populist rhetoric and policy positions (regardless of their operationalization). There is also no interaction between any of populism’s components and policy positions. All in all, the populist rhetoric of any form does not seem to impact the electoral effectiveness or attractiveness of immigration and economic policies.

Table: Effects of policy and populist rhetoric on vote choice

\begin{table}[!htbp] \centering \caption{Effects of policy and populist rhetoric on vote choice} \label{tab:interactionYD} \resizebox{0.65\textwidth}{!}{ \begin{tabular}{@{\extracolsep{-5pt}}lcc} \\[-1.8ex]\hline \hline \\[-1.8ex] \\[-1.8ex] & \multicolumn{2}{c}{Probability of selection} \\ \\[-1.8ex] & (1) & (2)\\ \hline \\[-1.8ex] Job background (non-elite) & 0.010 & 0.008 \\ & (0.011) & (0.011) \\ Political experience (none) & $-$0.017 & $-$0.014 \\ & (0.011) & (0.012) \\ Current polling (trailing) & $-$0.001 & $-$0.002 \\ & (0.011) & (0.012) \\ Immigration policy (anti) & $-$0.036$^{*}$ &  \\ & (0.013) &  \\ Economic policy (right) & $-$0.038$^{*}$ &  \\ & (0.013) &  \\ Immigration (congruent) &  & 0.073$^{**}$ \\ &  & (0.013) \\ Economic policy (congruent) &  & 0.072$^{**}$ \\ &  & (0.013) \\ Populist rhetoric & 0.016 & 0.020 \\ & (0.031) & (0.031) \\ Immigration policy (anti) : Populist rhetoric & $-$0.006 &  \\ & (0.035) &  \\ Economic policy (right) : Populist rhetoric & $-$0.006 &  \\ & (0.036) &  \\ Immigration (congruent) : Populist rhetoric &  & 0.029 \\ &  & (0.036) \\ Economy (congruent) : Populist rhetoric &  & $-$0.040 \\ &  & (0.036) \\ \hline \\[-1.8ex] Observations & 8,032 & 7,616 \\ Adjusted R$^{2}$ & 0.002 & 0.010 \\ \hline \hline \\[-1.8ex] \multicolumn{3}{r}{} \\ \end{tabular} } \begin{flushleft} \footnotesize{The table shows the AMCEs of the randomly assigned profile and speech attributes on candidates' probability of being selected. Robust standard errors clustered by respondent are given in parentheses, $^{*}$p$<$0.05; $^{**}$p$<$0.01; $^{***}$p$<$0.001.} \end{flushleft} \end{table}

We also break down our results by partisanship (Figures fig:FigureA2 and A3). Overall, there is some indication that Democrats are relatively more sensitive to policy positions vis-a'-vis populist rhetoric. Specifically, unlike those who lean Democrat, the choices of Republican respondents appear to be almost equally driven by candidates’ economic positions and anti-establishment rhetoric. However, none of these subgroup differences are statistically significant when one accounts for multiple comparisons.

Figure: fig:FigureA2

Discussion

These findings have important implications for our understanding of the role of populist rhetoric in politics. Fascinated by the rise of populism over the last several decades, many commentators have attributed its success to a particular rhetorical style that may be appealing to a substantial part of the electorate. Our experimental results suggest that a substantial part of the appeal that populists have may actually lie in their substantive policy positions (which are hard to disentangle in available observational data). In that sense, while some attributed the electoral success of Donald Trump in 2016 to his populist style, it is possible that his appeal was more related to his (rhetorical) moderation on economic issues and the emphasis on immigration issues.

Similar to previous conjoint experiments on populism [@Neuner2022; @silva2022populism], we also do not find that populist rhetoric is particularly appealing to those who hold populist attitudes. Importantly, our study was conducted during the Biden administration (while these previous studies were conducted during the Trump administration), which indicates that the null effects hold regardless of whether the incumbents themselves are populist [@jungkunz2021populist]. In line with previous research on democratic attitudes [@Graham2020a], our paper also suggests that congruent substantive policy positions are the most important factor for vote choice.

Of course, our research is not without limitations. First, our sample is arguably not large enough to reliably detect three-way and small two-way interaction effects even when they exist. Second, our results may be potentially driven by the conjoint design choices, including the paired task and/or particular speech examples. While it is important to see further replications of our study, the current results do suggest that populist rhetoric is unlikely to be effective in itself, especially compared to candidate issue positioning. In this respect, future research would benefit from considering larger and/or more targeted samples, as well as other possible conjoint and vignette experimental designs.

Source Appendix: Appendix.tex

Appendix A: Tables and Figures

Figure: Effects of using various features of populist rhetoric and other attributes (alternative coding)

Caption: Effects of using various features of populist rhetoric and other attributes (alternative coding)

Figure: Effects of using various features of populist rhetoric by populist attitudes (alternative coding)

Caption: Effects of using various features of populist rhetoric by populist attitudes (alternative coding)

Figure: Effects of using various features of populist rhetoric by partisanship (alternative coding)

Caption: Effects of using various features of populist rhetoric by partisanship (alternative coding)

Figure: Pair one

Caption: Pair one

Figure: Effects of using various features of populist rhetoric and other attributes (attentive respondents only)

Caption: Effects of using various features of populist rhetoric and other attributes (attentive respondents only)

Figure: Effects of using various features of populist rhetoric by strong populist attitudes

Caption: Effects of using various features of populist rhetoric by strong populist attitudes

Figure: Example of a random conjoint task

Caption: Example of a random conjoint task

Table: Effects of various candidate and speech characteristics on vote choice

\begin{table}[!htbp] \centering \caption{Effects of various candidate and speech characteristics on vote choice} \label{tab:interactions} \resizebox{0.7\textwidth}{!}{ \begin{tabular}{@{\extracolsep{-4pt}}lcccc} \\[-1.8ex]\hline \hline \\[-1.8ex] \\[-1.8ex] & \multicolumn{4}{c}{Probability of selection} \\ \\[-1.8ex] & (1) & (2) & (3) & (4)\\ \hline \\[-1.8ex] Job background (non-elite) & 0.008 & 0.009 & 0.009 & 0.008 \\ & (0.011) & (0.011) & (0.011) & (0.011) \\ Political experience (none) & $-$0.014 & $-$0.014 & $-$0.014 & $-$0.013 \\ & (0.012) & (0.012) & (0.012) & (0.012) \\ Current polling (trailing) & $-$0.003 & $-$0.003 & $-$0.003 & $-$0.002 \\ & (0.012) & (0.012) & (0.012) & (0.012) \\ Immigration policy (congruent) & 0.077$^{**}$ & 0.077$^{**}$ & 0.077$^{**}$ & 0.077$^{**}$ \\ & (0.012) & (0.012) & (0.012) & (0.012) \\ Economic policy (congruent) & 0.067$^{**}$ & 0.067$^{**}$ & 0.067$^{**}$ & 0.067$^{**}$ \\ & (0.012) & (0.012) & (0.012) & (0.012) \\ Pluralist / People-centric & 0.013 & 0.012 & 0.0004 & $-$0.0001 \\ & (0.014) & (0.025) & (0.030) & (0.029) \\ Anti-pluralist / People-centric & 0.008 & 0.004 & 0.013 & 0.017 \\ & (0.013) & (0.024) & (0.028) & (0.029) \\ Non-moralist / Anti-establishment & 0.010 & $-$0.009 & 0.013 & $-$0.015 \\ & (0.014) & (0.024) & (0.030) & (0.029) \\ Moralist / Anti-establishment & 0.017 & 0.032 & 0.059 & 0.005 \\ & (0.014) & (0.024) & (0.031) & (0.028) \\ Partisanship (Democrat) &  &  & 0.025 &  \\ &  &  & (0.023) &  \\ Populist Attitudes (Populist) &  &  &  & $-$0.020 \\ &  &  &  & (0.023) \\ Pluralist / People-centric : Non-moralist / Anti-establishment &  & 0.037 & 0.037 & 0.037 \\ &  & (0.034) & (0.034) & (0.034) \\ Anti-pluralist / People-centric : Non-moralist / Anti-establishment &  & 0.020 & 0.020 & 0.020 \\ &  & (0.034) & (0.034) & (0.034) \\ Pluralist / People-centric : Moralist / Anti-establishment &  & $-$0.036 & $-$0.037 & $-$0.037 \\ &  & (0.035) & (0.035) & (0.035) \\ Anti-pluralist / People-centric : Moralist / Anti-establishment &  & $-$0.008 & $-$0.008 & $-$0.008 \\ &  & (0.035) & (0.035) & (0.035) \\ Pluralist / People-centric : Democrat &  &  & 0.020 &  \\ &  &  & (0.029) &  \\ Anti-pluralist / People-centric : Democrat &  &  & $-$0.016 &  \\ &  &  & (0.027) &  \\ Non-moralist / Anti-establishment : Democrat &  &  & $-$0.036 &  \\ &  &  & (0.028) &  \\ Moralist / Anti-establishment : Democrat &  &  & $-$0.044 &  \\ &  &  & (0.030) &  \\ Pluralist / People-centric : Populist &  &  &  & 0.022 \\ &  &  &  & (0.028) \\ Anti-pluralist / People-centric : Populist &  &  &  & $-$0.023 \\ &  &  &  & (0.027) \\ Non-moralist / Anti-establishment : Populist &  &  &  & 0.011 \\ &  &  &  & (0.028) \\ Moralist / Anti-establishment : Populist &  &  &  & 0.047 \\ &  &  &  & (0.029) \\ \hline \\[-1.8ex] Observations & 7,616 & 7,616 & 7,616 & 7,616 \\ Adjusted R$^{2}$ & 0.010 & 0.010 & 0.010 & 0.010 \\ \hline \hline \\[-1.8ex] \multicolumn{5}{r}{} \\ \end{tabular} } \begin{flushleft} \footnotesize{The table shows the AMCEs of the randomly assigned profile and speech attributes on candidates' probability of being selected. Robust standard errors clustered by respondent are given in parentheses, $^{*}$p$<$0.05; $^{**}$p$<$0.01; $^{***}$p$<$0.001.} \end{flushleft} \end{table}

Table: Effects of policy and populist rhetoric (components) on vote choice

\begin{table}[!htbp] \centering \caption{Effects of policy and populist rhetoric (components) on vote choice} \label{tab:interaction2} \resizebox{0.5\textwidth}{!}{ \begin{tabular}{@{\extracolsep{-5pt}}lcc} \\[-1.8ex]\hline \hline \\[-1.8ex] \\[-1.8ex] & \multicolumn{2}{c}{Probability of selection} \\ \\[-1.8ex] & (1) & (2)\\ \hline \\[-1.8ex] Job background (non-elite) & 0.010 & 0.009 \\ & (0.011) & (0.012) \\ Political experience (none) & $-$0.017 & $-$0.014 \\ & (0.011) & (0.012) \\ Current polling (trailing) & $-$0.002 & $-$0.002 \\ & (0.011) & (0.012) \\ Immigration policy (anti) & $-$0.049 &  \\ & (0.025) &  \\ Economic policy (right) & $-$0.051$^{+}$ &  \\ & (0.026) &  \\ Immigration (congruent) &  & 0.087$^{**}$ \\ &  & (0.025) \\ Economic policy (congruent) &  & 0.080$^{*}$ \\ &  & (0.026) \\ Pluralist / People-centric & 0.024 & 0.026 \\ & (0.032) & (0.031) \\ Anti-pluralist / People-centric & $-$0.008 & 0.018 \\ & (0.031) & (0.030) \\ Non-moralist / Anti-establishment & $-$0.037 & $-$0.031 \\ & (0.032) & (0.030) \\ Moralist / Anti-establishment & 0.032 & 0.058 \\ & (0.031) & (0.031) \\ Pluralist / People-centric : Non-moralist / Anti-establishment & 0.040 & 0.038 \\ & (0.034) & (0.034) \\ Anti-pluralist / People-centric : Non-moralist / Anti-establishment & 0.015 & 0.021 \\ & (0.033) & (0.034) \\ Pluralist / People-centric : Moralist / Anti-establishment & $-$0.043 & $-$0.037 \\ & (0.034) & (0.035) \\ Anti-pluralist / People-centric : Moralist / Anti-establishment & $-$0.023 & $-$0.008 \\ & (0.034) & (0.035) \\ Immigration policy (anti) : Pluralist / People-centric & 0.013 &  \\ & (0.027) &  \\ Immigration policy (anti) : Anti-pluralist / People-centric & 0.043 &  \\ & (0.027) &  \\ Immigration policy (anti) : Non-moralist / Anti-establishment & 0.006 &  \\ & (0.027) &  \\ Immigration policy (anti) : Moralist / Anti-establishment & $-$0.025 &  \\ & (0.028) &  \\ Economic policy (right) : Pluralist / People-centric & $-$0.041 &  \\ & (0.028) &  \\ Economic policy (right) : Anti-pluralist / People-centric & $-$0.010 &  \\ & (0.027) &  \\ Economic policy (right) : Non-moralist / Anti-establishment & 0.051 &  \\ & (0.028) &  \\ Economic policy (right) : Moralist / Anti-establishment & 0.038 &  \\ & (0.028) &  \\ Immigration (congruent) : Pluralist / People-centric &  & $-$0.005 \\ &  & (0.028) \\ Immigration (congruent) : Anti-pluralist / People-centric &  & $-$0.017 \\ &  & (0.027) \\ Immigration (congruent) : Non-moralist / Anti-establishment &  & $-$0.003 \\ &  & (0.028) \\ Immigration (congruent) : Moralist / Anti-establishment &  & $-$0.005 \\ &  & (0.028) \\ Economy (congruent) : Pluralist / People-centric &  & $-$0.024 \\ &  & (0.029) \\ Economy (congruent) : Anti-pluralist / People-centric &  & $-$0.013 \\ &  & (0.027) \\ Economy (congruent) : Non-moralist / Anti-establishment &  & 0.047 \\ &  & (0.028) \\ Economy (congruent) : Moralist / Anti-establishment &  & $-$0.047 \\ &  & (0.029) \\ \hline \\[-1.8ex] Observations & 8,032 & 7,616 \\ Adjusted R$^{2}$ & 0.003 & 0.011 \\ \hline \hline \\[-1.8ex] \multicolumn{3}{r}{} \\ \end{tabular} } \begin{flushleft} \footnotesize{The table shows the AMCEs of the randomly assigned profile and speech attributes on candidates' probability of being selected. Robust standard errors clustered by respondent are given in parentheses, $^{*}$p$<$0.05; $^{**}$p$<$0.01; $^{***}$p$<$0.001.} \end{flushleft} \end{table}

Table: Descriptive Statistics (MTurk, n = 1004/8032 vs ANES 2020 Benchmark)

\begin{table}[t!] \centering \caption{Descriptive Statistics (MTurk, n = 1004/8032 vs ANES 2020 Benchmark)} \label{tab:descriptive} \begin{tabular}{@{\extracolsep{5pt}}lccccccc} \\[-1.8ex]\hline \hline \\[-1.8ex] Statistic  & \multicolumn{1}{c}{ANES Mean} & \multicolumn{1}{c}{Mean} & \multicolumn{1}{c}{St. Dev.} & \multicolumn{1}{c}{Min} & \multicolumn{1}{c}{Pctl(25)} & \multicolumn{1}{c}{Pctl(75)} & \multicolumn{1}{c}{Max} \\ \hline \\[-1.8ex] Age & 49.92 & 38.35 & 11.38 & 18 & 30 & 44 & 78 \\ Female  & 0.52 & 0.38 & 0.49 & 0 & 0 & 1 & 1 \\ White & 0.69 & 0.79 & 0.41 & 0 & 1 & 1 & 1 \\ College+ & 0.37 & 0.78 & 0.42 & 0 & 1 & 1 & 1 \\ Republican & 0.42 & 0.36 & 0.48 & 0 & 0 & 1 & 1 \\ Democrat & 0.46 & 0.58 & 0.49 & 0 & 0 & 1 & 1 \\ Populist & --- & 0.56 & 0.50 & 0 & 0 & 1 & 1 \\ \hline \\[-1.8ex] \end{tabular} \end{table}

Appendix B: Operationalization of Populism and its Components

Following the theoretical literature, our experiment operationalizes populism along the dimensions of people-centrism, anti-elitism, and anti-pluralism. Unlike previous studies that only included populist attributes [@Neuner2022; @silva2022populism], we also include elitist and pluralist attributes, populism’s direct opponents, as the non-populist attributes [@mudde2017populism]. People-centrism specifies that people should be the only legitimate source of political power. Elitism, on the other hand, considers the elites should make political decisions. People-centrism is a shared feature between populism and liberal democracy. However, unlike liberal democracy’s pluralist view of the people, populist understanding of the people is homogenous and virtually pure (anti-pluralist) [@mudde2017populism]. In other words, there are two types of people-centrism: pluralist and anti-pluralist. Therefore, we include three categories of attributes in the people-centrism dimension: pluralist people-centrism, anti-pluralist people-centrism, and elitism.

Anti-elitism, the other core component of populism, is not simply about criticizing the status quo or the establishment [@muller2017populism]. Elites, as the opposite of the pure people, in populism are morally corrupted and intentionally subverting the people’s interests. The distinction between the people and the elite in populism is purely moral [@mudde2017populism]. In other words, a populist anti-elitist message should be morally charged [@Bonikowski2015; @bonikowski2023populism]. However, one can criticize the status quo or the establishment without vilifying the elite. We, therefore, include three categories of attributes in the anti-elitism dimension: moralized anti-establishment, non-moralized anti-establishment, and pro-establishment.

Following the multi-dimensional theoretical structure of populism [@wuttke2020whole], a campaign message is only considered populist when it contains all of the anti-pluralist, anti-elitist, and anti-pluralist attributes.

In Table tab:Conjoint_attributes below, we include the list of all conjoint attributes and their possible levels.

Table: List of conjoint speech attributes

\begin{table}[!htbp] \centering \caption{List of conjoint speech attributes} \label{tab:Conjoint_attributes} \vspace*{-25pt} \includegraphics[width=\linewidth]{TableA1.pdf} \vspace*{-25pt} \end{table}

Appendix C: Power Analysis

According to our calculations, our conjoint design with the resulting effective sample size of 8032 provides 90% statistical power to detect a small effect (AMCE $>$ 0.06) and near 80% statistical power to detect a smaller effect of substantive significance > 0.04 of the target 3-level substantive attributes at $alpha = 0.05$ with a near zero probability that estimated coefficients have incorrect signs (Stefanelli and Lukac 2020). For the possible interaction of two populism treatments or their possible heterogeneity by populist attitudes, given the proportion of populism treatment (1/3) and the distribution of populist attitudes $( 1/2)$, our design can only reliably detect interaction effects greater than 0.07 with 80% power (Schuessle and Freitag 2020).\

References:

Stefanelli, Alberto, and Martin Lukac. 2020. “Subjects, trials, and levels: Statistical power in conjoint experiments.” (https://mblukac.shinyapps.io/conjoints-power-shiny/).

Schuessler, Julian, and Markus Freitag. 2020 “Power analysis for conjoint experiments.” \(https://markusfreitag.shinyapps.io/cjpowr/).

Appendix D: Survey instrument

*Populist attitudes** [binary variable, defined as 1 if and only if respondents agree with all three subcomponents of populism)]

“How much do you agree or disagree with the following statements?” (Strongly disagree; Somewhat disagree; Neither agree nor disagree; Somewhat agree; Strongly agree)

Anti-pluralism:*

• “Ordinary people share the same values and interests”
• “Ordinary people are of good and honest character”

People-centrism:*

• “The people, not the elites, should make our most important policy decisions”
• “The politicians need to follow the will of the people”

Moralized anti-elitism:*

• “The government is pretty much run by a few big interests looking out for themselves”
• “Quite a few of the people running the government are crooked”

*Partisanship** [binary variable: Republican or Democrat, including leaners]

“Which of the following categories best describes your political affiliation?” (Democrat; Republican; Independent; Other)

[if Independent]”Would you say that you are…” (Independent, lean Republican; Independent, do not lean towards either party; Independent, lean Democrat”)

Appendix E: Robustness checks

We conduct a number of robustness checks with no change in the underlying substantive conclusions. First, decompose our two complex populist-related conjoint attributes into the four simpler, mutually exclusive binary variables as an alternative conceptualization (see Figures A1-3). Second, we replicate our subgroup analysis by populist attitudes with the stronger, more exclusive definition of these attitudes in which the respondents have to strongly agree with all six populist items (see Figure fig:FigureA6). Third, we test for the possible carryover and profile order effects by restricting our sample to each of the candidate pairs (see Figure fig:FigureA4 and interacting the profile order with the attribute effects (not shown). Fourth, we restrict our sample to the 70% of respondents who passed a relatively demanding attention check (see Figure fig:FigureA5). [note: The question fielded at the end of the omnibus survey was worded as follows: “In the survey, you read about:” (Information Theory, Critical Race Theory, Theory of Knowledge, Theoretical Rationality). The second option was the correct one.]

---