Blog post by Whitney Afonso, Zach Mohr, and Scott Powell

Questions for Practitioners:

• When (as in time of year) should you engage citizens in budget simulations?
• Can I expect trust in our government to increase if we engage citizens in budget simulations?
• How do starting budget conditions affect revenue and expenditure preferences, and what role do deficits, surpluses, and balanced conditions play in shaping these preferences?
• In what ways can local government practitioners enhance the design of budget simulations based on the findings, and what considerations should they take into account when implementing such simulations to engage citizens in the budget process?

Introduction

In the rapidly evolving landscape of budget engagement, online budget simulations have gained prominence. This blog addresses the increasing use of these simulations, particularly in local governments, and emphasizes the need for a thoughtful evaluation of design choices. While previous studies focused on federal budgets, this blog delves into municipal budget simulations, posing the question: does beginning the simulation in balance, deficit, or surplus affect respondents’ engagement and budget preferences?

The blog post summarizes an article published in the journal Public Budgeting & Finance.  We also discuss findings that were not presented in the article that suggests that starting the simulations in deficit really does not affect trust. The blog is useful to anyone that is looking to think more deeply about how to set up an online budget simulation.  They may also be useful to those that want to do experiments in budget and finance settings, which have some important practical challenges.

The findings from the research are highly relevant to local government budget practitioners who wish to engage citizen to participate in the budget process. As budgeting preferences are shown to be a function of design, budgeteers should think critically about the design of their simulations.  Specifically, we think that starting from a small budget deficit position, such as the level of inflation, may be the best way to practically get more engagement with the budget simulation, but this may influence the budget outcomes from the simulation.  At any rate, a choice on how to start the simulation must be made.

Summary of the Article

While traditional in-person engagement has been the norm, the shift to online methods, accelerated by the COVID-19 pandemic, brings benefits such as broad accessibility to the simulation and speed to complete them (although this may also be a bit of a problem).  The article first traces the evolution of online budget simulations from national to local levels, emphasizing their growth in popularity and recognition as a best practice in public engagement, with examples of awards received by local governments. The newer “dynamic” simulations, allowing simultaneous adjustments to revenue and expenditures, are highlighted for their widespread adoption and relevance for both local governments and scholars interested in testing budget engagement theories.

Behavioral Model and Hypotheses

In the article, we discuss the impact of information presentation in budget simulations on participant engagement and budgetary preferences. Referring to previous studies, we establish that the way information is presented, especially concerning cost and performance, influences budget preferences. The model proposed suggests that starting a simulation with different budgetary conditions affects participant engagement and preferences. Unlike prior assumptions of intrinsic motivation, this research explores how the simulation structure influences engagement levels, potentially influencing other perceptions like trust in the government, which we discuss more below.

Research Design and Analysis

Participants engaged in the simulation as part of an online survey experiment. The study aimed to understand how starting conditions (balanced, a 5% deficit, or a 5% surplus) influenced participant engagement, preferences, and perceptions.

Due to local leaders’ concerns about public perception, the simulation had to be conducted in a controlled lab environment rather than in the field. This field-in-lab experiment involved randomly assigning participants to different budgetary conditions, mirroring those of a nearby local government. The participants were younger and more diverse than the general population, being students from the university, thus the specific policy preferences indicated in the simulation are not generalizable to the population.

The analysis focused on three main aspects: level of engagement, budgetary preferences, and final budgetary balance. Contrary to expectations, the time spent on the simulation did not significantly differ between deficit, surplus, and balanced conditions. However, the deficit treatment significantly reduced the number of participants completing the simulation, indicating potential difficulties. The deficit treatment increased the number of changes made to both expenditures and revenues, while the surplus treatment primarily influenced changes in revenues.

Starting conditions significantly affected both revenues and expenditures. Participants tended to maintain taxes when starting in balance, reduce them in surplus, and either maintain or increase them in deficit. Expenditure changes were idiosyncratic, with deficits leading to more significant cuts.

The results partially supported hypotheses related to the size of changes and the final budgetary balance. Participants starting in deficit made larger changes to expenditures, but the final balance was only significantly influenced by the surplus condition, aligning with anchoring and status quo bias. The study sheds light on how starting conditions in budget simulations impact participant behavior, providing valuable insights for both academic research and practical applications in local government budget engagement.

Discussion

The experiment demonstrates that different starting conditions significantly impact engagement and budgetary preferences. Two out of three engagement outcomes were influenced by starting conditions, indicating their substantial role in shaping participant behavior. The deficit condition particularly amplified cuts to services like the police, suggesting that unpopular services might be more vulnerable during financial downturns. Despite the expected increase in time spent on the simulation due to varied starting conditions, time was not significantly affected, potentially influenced by the participants’ choice to avoid difficult tasks.

The deficit condition reduced the likelihood of participants completing the simulation, highlighting cognitive limitations and avoidance of challenging situations. This finding has practical implications for practitioners, emphasizing the trade-off between obtaining more information through deficit starting conditions and the decrease in completions. Future research could explore predictors of completion, such as math literacy, to enhance engagement. Additionally, practitioners may need to consider when to run simulations, as participants’ responses could vary depending on the budget preparation cycle.

Observations of “chunking” behavior in deficit conditions, where participants made larger changes, suggest a dynamic relationship between engagement and budgetary preferences. The study encourages further behavioral theory development in the context of budget simulations.

The research contributes to understanding individual psychological processes like anchoring, loss aversion, and decisional inertia in budget engagement, offering insights for practitioners on designing effective simulations. Practical implications include adjusting starting conditions based on desired outcomes and considering the impact of performance information on budget engagement. The study advocates for more field experiments and “field-in-lab” experiments to refine the choice architecture of politically sensitive topics like budget simulations, enhancing the usability and effectiveness of engagement efforts. Overall, the findings underscore the importance of behaviorally informed changes in shaping public engagement and preferences in budget-related decision-making.

The Simulation’s Effect on Trust

One of the things that was particularly important to our local government partner that we were helping to test the effect of starting position was to make sure that this did not have an effect on citizen’s trust.  The city had carefully maintained its well earned reputation for being a good steward of taxpayers money and it did not want to see that reputation tarnished because citizens’ were asked to do a budget simulation that started in deficit.  This was a major reason that we ran the simulation on students where we could properly debrief them that the scenario was not real and we took out all mentions to this specific city in the simulation.

We also saw that it was an opportunity to test the effect of the simulation and the simulation starting position on the outcome of trust in local government.  So, we actually designed an experiment within the experiment that would look at the effect of seeing no simulation relative to seeing a simulation.  We also are able to see if the effect of the treatment of balance, deficit, and surplus had an effect on trust.  So, the experiment participants were first given a series of demographic questions and a question about trust in different levels in government, they were then randomly assigned into one of the four treatments shown in figure 1 and were asked to complete the simulation, and then the participants were asked about their level of trust in government again.  So, this can be treated as either a between subject or pre-post design.

Figure 1: The four treatment conditions

It really doesn’t matter, though, how we analyze the data because there was no statistically significant effect of the simulation on trust.  There was likewise no effect between the different starting treatments on the levels of trust between the treatments or the change in the trust.  Put simply, starting a budget in deficit is not going to influence the level of trust – at least as we measured it.  One of the things that we are doing with future surveys is looking at whether the simulations had effects on other perceptions about the government.

Conclusion

The results of the published study and the results that we are presenting here suggest that there are only minimal negative effects of starting a budget in deficit.  Most importantly it may modestly reduce the number of completions.  However, starting the budget simulation in a small deficit – such as the current rate of inflation – may lead the respondents that complete the budget to adjust more types of revenue and expenditure and provide budgeteers with more realistic information about how citizens would balance the budget.  Starting the budget simulation in deficit is also found not to reduce trust.  Future research is ongoing about the responses to different levels of deficit (2.5%, 5%, 7.5% or 10%) and effects on other perceptual outcomes.  We would invite the budget community to propose other types of experiments that they would like to see researchers address and the research community should continue to work with local governments to test their most promising behavioral changes to these simulations.

Citation to journal article:

Mohr, Zach, and Whitney Afonso. “Budget starting position matters: A “field‐in‐lab” experiment testing simulation engagement and budgetary preferences.” Public Budgeting & Finance (2023).