Examining the Impact of Six Pro-Vaccination Messages on MMR Vaccine Hesitancy Among Mothers in Ukraine A

The Ukrainian government began delivering mandatory MMR vaccination letters to parents across the country in 2019. In this two-phase online RCT, we aimed to test the effectiveness of this national mandatory vaccination letter against five behavioural science-informed letters, in terms of their effects on the vaccination attitudes, intentions and behaviours of Ukrainian mothers ( N = 738). One letter was focused on the simplicity and accessibility of vaccination procedures; one contained a testimonial from a family doctor; two letters contained pro-vaccination social norm statements (one signed by a family doctor and one by a school director); and one contained a loss-framed message underlining the risks of non-vaccination. The results showed no difference between the conditions in terms of change in vaccination attitudes and intentions but there was an effect on behaviour (measured through clicking a link to schedule a vaccination). The letters signed by a family doctor, outlining how

Even before the Covid-19 pandemic struck, vaccine hesitancy was amongst the top global public health threats (WHO, 2019) and a particularly acute problem in Ukraine.Hesitancy towards, and uptake of, the combined measles, mumps and rubella (MMR) vaccination has been an issue of particular concern because measles remains a significant threat to children's health in countries with low vaccination rates (Patel et al., 2019).Ukraine is one such country with only 41% of children under 6 vaccinated (Ministry of Health of Ukraine, 2020), and where the latest measles epidemic occurred in the 2017-2019 period (Rodyna, 2019).The Wellcome Global Survey (2019) attributes the high levels of hesitancy and low levels of vaccine uptake to Ukrainian people holding one of the lowest levels of trust in vaccination globally.Since the Covid-19 pandemic began in 2019, the issue of vaccine hesitancy, in general, has taken on an even greater importance because Covid-19 vaccinations will play a vital role in controlling the virus and the success of the vaccinations will hinge on people's willingness to vaccinate against the new virus.This means that the development of strategies for reducing vaccine hesitancy in Ukraine are of critical importance for the prevention of further measles outbreaks and to support the Covid-19 vaccine roll-out.
Public health bodies have implemented a variety of strategies to increase vaccination uptake in Ukraine, including the mandating of vaccination.One component of Ukrainian mandatory vaccination policy has been the delivery of a letter to all parents that informs them of the requirement to vaccinate their child in order to attend school (Law of Ukraine, 2020). 1 However, there is evidence that mandating vaccination for a sceptical population may be ineffective (Opel et al., 2017) and it is open to debate whether stressing the mandatory nature of vaccination in the text of letters to parents is the most effective way to increase vaccine uptake in Ukraine (and beyond).Another common approach is to stress the risks of nonvaccination, for which the evidence on behaviour change is also sparse (O' Keefe & Nan, 2012;Parsons et al., 2018).
Existing research has shown that applying behavioural science-informed techniques to assist in the design of official communications, such as vaccination letters or text messages, has the potential to influence vaccine uptake, when compared to no message (Regan et al., 2017;Yokum et al., 2018), as well as to messages designed without the use of behavioural science techniques (Milkman et al., 2011).Thus, there may be some particular approaches to the framing of such messages that could potentially increase their effectiveness relative to other letter-types, by influencing attitudes towards vaccination, intentions to vaccinate and, most importantly from a public health perspective, vaccination behaviour.This study aimed to compare the efficacy of the mandatory vaccination message with five other communications with alternative message framings that could be effective in a Ukrainian context, including those commonly used, based on existing theory and evidence (Parsons et al., 2018).

The Increasing Vaccination Model
In this paper we base our contribution on the increasing vaccination model (IVM; Brewer, 2021).The model is founded on the theory of planned behavior (TPB) and explains behaviour in terms of (1) attitudes, (2) subjective norms and (3) perceived behavioural control (Ajzen I., 1984).In the case of vaccine hesitancy, a person's attitudes towards vaccines, their understanding of what is normative and appropriate within their social network / the society they live in, and their ease of access to vaccines will all play a significant role in determining their decision to get vaccinated or not.The IVM (Brewer, 2021) extends TPB theory and explains vaccination outcomes in terms of what people think and feel, social processes (including social norms and networks), and opportunities for behaviour change.The focus on thinking and feeling includes risk appraisal (loss-and gain-framed messaging, see below), vaccine confidence, and motivation.Although these show to be important drivers of vaccination uptake, meta-analytic evidence shows that attempts to change risk appraisals are not always effective (Parsons et al., 2018), nor is a focus on increasing vaccine effectiveness or motivation (Brewer et al., 2017).Brewer et al. (2021) suggest that interventions that address the subjective norms, building on foundations of trust or other social processes and opportunity for vaccination are more likely to increase vaccination uptake.In the context of an invitation letter, and based on the common use of risk appraisal strategies in the Ukrainian context, we set out to test the effect of various messages focused on social processes, and an opportunitydirected message, against the mandatory message, and one of risk appraisal.

Risk Appraisal: Loss-Framed and Gain-Framed Messages
Much of the existing research in the area of vaccine communication framings has focused on the relative effectiveness of two of the most commonly employed message framings: lossframed and gain-framed strategies (O'Keefe & Jensen, 2008;Gallagher & Updegraff, 2012).Both loss-framed and gain-framed communications can be seen as examples of messages that focus mostly on influencing attitudes and/or what people think and feel.Loss-framed communications work by highlighting the potential costs of not following a course of action.One commonly employed loss-framed strategy for reducing vaccine hesitancy (common in the Ukrainian context) is pointing out the serious health risks that result from not vaccinating.There is evidence that messages of this type are effective in some situations (e.g., Abhyankar et al., 2008;Brewer et al., 2016).
The currently employed Ukrainian mandatory vaccination message could be seen as an example of a loss-framed message due to the loss of access to school for the child of the parent contacted.Gain-framed messages, in contrast, outline the benefits of following a given course of action, such as avoiding infection for oneself and one's family.Early theoretical work postulated that gain-framed messages would be more effective for encouraging uptake of preventative medicine such as vaccines (e.g., Rothman & Salovey, 1997) but a 2012 metaanalysis (O'Keefe & Nan, 2012) found no difference between the two message frames for encouraging vaccination.More recent evidence from a meta-analysis of 16 studies suggests that neither gain-or loss-framing provide strong enough intervention to change vaccination context (Parsons et al., 2018).As loss-framed messages are commonly used in Ukraine, we opted to test the loss-framed message against mandatory and other messages.One potential problem with loss-framed strategies is that there is a risk that people feel threatened and/or coerced, and that this could reduce perceptions of response efficacy, or even increase fear and anger (Witte & Allen, 2000).Nyhan et al.'s (2014) research demonstrated the potential dangers of loss-framed messages for uptake in sceptical populations by showing how not only were loss-framed messages ineffective in shifting beliefs and vaccination intent, compared to a control and to other messages, they also had the unanticipated consequence of increasing misperceptions about MMR.In the Ukrainian context, this problem could be particularly acute because trust in vaccination as well as national governance and institutions is very low, in comparison to similar European countries (Izha et al., 2020).This means that loss-framed strategies may not only be ineffective but could even be counter-productive.

Social Processes and Opportunity: Additional Messages Suitable for a Ukrainian Context
In this study, we aimed to test the effectiveness of additional IVM-informed yet currently under-researched messages and compare them to the mandatory vaccination message currently used in Ukraine.We wanted to select messages that might be effective in a Ukrainian context and that focused on social norms and social networks, as well as opportunity.In addition to the two common messages: mandatory control and the loss-framed message, we aimed to select different formulations of messages which would leverage social processes or norms, and attention to opportunity as potential mechanisms for changing vaccination behaviour.We opted to test messages signed by two messengers: school directors and family doctors.This was done for practical reasons: as the Ukrainian vaccination letters are distributed through schools and because the mandatory vaccination policy affects school attendance, a school director could make for a more feasible messenger than for a local doctor.Finally, we included a letter which could be expected to directly influence vaccination behaviour by underlining the lack of barriers to getting one's child vaccinated.
Messages signed by trusted authorities (and explicitly referencing the relevant expertise of the person's authority) could be effective because some groups in society are more trusted than others.Doctors are one particularly promising group here because they are consistently rated as some of the most trusted and respected members of society worldwide (GFK Verein, 2018) and particularly in a Ukrainian context as trust in other forms of institutional authority is low (Wellcome Global Monitor, 2018).Furthermore, recent studies show that appropriate interactions with doctors are effective in emphasising the benefits of vaccines and encouraging parental vaccination decisions (Kao et al., 2015;Wiley et al., 2013).
Messages which communicate that vaccination is a social norm may increase the sense of social pressure or expectation to get vaccinated.There are two ways in which a social norm can be used.A so-called injunctive norm expresses how people should behave, while a socalled descriptive norm describes how the majority behaves.There is various evidence from observational survey-type analysis on vaccination (Betsch et al., 2015;Gerend et al., 2013) and from experiments in other fields (e.g., Gerber et al., 2008) that people tend to change their behaviour in accordance with descriptive social norms, more so than the injunctive norms with respect to vaccination.Drawing attention to social norms has also had various reported adverse consequences in other policy domains: it could outline that not everyone is doing it (Kahan, 2014;Rimal & Real 2005).One such example is from Palm et al. (2021) who showed in an experimental study that people who received vaccination messages framed with a negative perspective towards vaccination were more likely to suggest that they themselves would not get vaccinated, compared to a control.In various other ways, socials norms have shown to be related to vaccination uptake.For example, Allen et al. (2009) found that social norms (reporting of a friend's behaviour), was the strongest predictor of the intent to be vaccinated against human papillomavirus (HPV).Similarly, Brunson (2013) identified descriptive social norms as key in parental decisions about their children's vaccinations.Vaccine coverage in one's social circle also predicted vaccination behaviour for influenza in De Bruin et al. (2019) and Parker et al. (2013).Therefore, we chose to use a descriptive norms-based message.
As IVM suggests, there is a growing body of behavioural science literature which underlines the importance of attending to opportunity.Recent studies in the U.S. have provided evidence in support of messages emphasising the default nature of vaccination (e.g., suggesting a vaccination appointment is already scheduled; Chapman et al., 2010;Chapman et al., 2016;Milkman et al., 2021), as well as the use of implementation intention prompts (or nudges) (Milkman et al, 2011).Further evidence has demonstrated that non-vaccination is predicted by practical barriers to vaccination such as the distance to site, digital literacy or making an appointment (Gerend et al., 2013;Wiley et al., 2013).In our final message, we attempt to employ a frame that underlines the lack of barriers to getting a vaccination.
These selected messages may be particularly effective for reducing vaccine hesitancy in the specific Ukrainian context.Not only could emphasising how vaccination is a social norm and / or messages from a trusted messenger such as a family doctor be especially effective due to the low levels of trust in other institutions, but a barrier-reduction message could also be effective in Ukraine due to public perceptions of vaccine shortages (DiBonaventura & Chapman, 2005), common reporting of unavailability of vaccines in emergency departments (i.e., Pappano et al., 2004) and logistical obstacles to vaccine distribution (UNICEF Ukraine, 2019; Witteman et al., 2015).
A related area of interest is the varying effects of different messages on attitudes or beliefs towards vaccination, intentions to vaccinate and vaccination behaviour.Ultimately, from a public health perspective, it is the effect on vaccination behaviour that is important, but research on how attitudes and intentions feed into vaccination behaviour is under-developed.Many large-scale studies have restricted their focus to behaviour only (albeit for sound empirical or methodological reasons) (e.g., Milkman et al., 2021;Yokum et al, 2018) but this does mean that studies examining the causal pathways from nudge to needle are relatively rare.Some research has shown that both vaccine attitudes and beliefs are affected in a similar way by vaccine messaging (e.g., Nylan et al, 2016) but where research has examined the links between attitudes, intentions and behaviour, the evidence has been mixed.For example, although attitudes do appear to predict vaccination intentions (Askelson et al., 2010), changes in intentions on their own do not always lead to corresponding behavioural outcomes (Sheeran & Webb, 2016;Webb & Sheeran, 2006).
In summary, the aim of this RCT was to compare the effectiveness of different message designs, in vaccination letters sent to mothers in Ukraine, in terms of their effects on vaccination attitudes, intentions and behaviour.We also aimed to explore the extent to which there were different effects for attitudes, intentions and behavioural measures.Six different messages were compared, including a control letter that was a copy of the original mandatory vaccination letter currently distributed in Ukraine, and five other letters that differed in terms of the behaviourally informed content in the letter.We hypothesise that each of the five intervention group messages will significantly improve pro-vaccination attitudes (H1), intentions (H2) and behaviour (H3) compared to the control.

Design
An online RCT was conducted to compare the effectiveness of six different MMR vaccination letters to mothers in terms of their effects on vaccination attitudes, intentions and behaviour.
The RCT used a mixed design with pre-and post-intervention data collection phases and six between-subjects framing conditions to test the effectiveness of five intervention letters vs a control on change in vaccine-related attitudes, change in vaccine-scheduling intention, and appointment scheduling behaviour of mothers.The research was approved by LSE's Research Ethics Policy and Procedures Committee and the National University of Kyiv-Mohyla Academy's Ethical Review Committee.

Procedure
Participants provided consent to take part in a two-part study about child vaccination, in which they would be asked their opinions and be shown a vaccination-related letter.In phase 1, participants answered a series of questions about vaccination attitudes and intentions, present and prior vaccination behaviour, and provided sociodemographic information.In phase 2, which followed two weeks after phase 1, all participants received one of six virtual letters in their email.After viewing the letter, participants were presented with the option to book a vaccination, followed by a request to answer the same attitude and intention questions from phase 1. Participants were informed they could refuse to answer questions or withdraw from the study at any time.At the end of the study participants were debriefed.The debrief explained that various messages were used, that all the information presented was true, but that there can be various unintended consequences when different messages are being studied.It was also explained that vaccination behaviour was measured based on their choice of clicking to schedule a vaccination or not.

Participants
Participants were invited to participate through Ukrainian commercial polling platform AskInMind (50,000 panellists across Ukraine, in all regions, female, aged 18 to 55 years, with at least one child aged 18 years or younger).Participants were selected from the AskInMind panel using stratified random sampling to the 6 conditions by age and regional representation, returning 2,017 panellists, who were invited to provide pre-intervention information (phase 1).This sample size was based on a power-calculation that indicated a sample size of 620 would detect a small to medium effect size (f = 0.18) in change in attitude change scores with an α = .05and power (1−β) of 0.95, as well as an estimated 50% dropout between the two phases of the project.Phase 1 data collection was stopped at a total of 1,277 responses (Mage = 34.35,SD = 6.58,Range = 18 -55).Phase 2 was completed by 58% of the phase 1 sample (N = 738).Table 1 shows the demographic breakdown of the phase 1 and phase 2 samples, by condition.There were no significant differences between the two samples in terms of region, settlement type, age of mother or child, highest educational level of mother, financial situation, or marital or vaccination status.The demographic equivalence of the samples indicates there was no demographic-based bias in the phase 2 sample caused by people from certain demographic groups dropping out at higher rates between phase 1 and phase 2. Additionally, the six conditions were well-matched in terms of demographic characteristics at phase 2. The loss-framed and mandatory vaccination conditions showed slight over-representation of single and divorced mothers compared to the other conditions (χ 2 (20) = 32.31,p = .04)but there were no other significant demographic differences between the conditions, again indicating minimal evidence of bias being introduced due to the high dropout rate.

Materials
Each of the six letters (see Figure 1) contained the same opening and concluding statement but differed in terms of the behavioural science-informed content in the middle section, and in some cases the signatory.The letters were comprised of a mandatory vaccination letter, that at the time of the study had been mandated by Ukrainian law but not yet sent out to parents, a loss-framed letter signed by a school director focused on the health consequences of catching measles, a barrier-reduction letter signed by a school director outlining the ease with which MMR vaccinations can be given in Ukraine, a doctor testimonial letter signed by a family doctor in which the doctor stressed it was their personal and professional recommendation that children should be vaccinated, and two social norm letters that stressed the normative nature of vaccination in Ukraine, one signed by a doctor and one by a school director.The letters varied in length from 53 to 66 words.

Measures
To measure vaccination attitudes, the pre-intervention (phase 1) and post-intervention (phase 2) questionnaires contained the same seven commonly employed survey items on attitudes towards vaccination (vaccine confidence project: Larson et al., 2015).This included a general "What is your attitude towards vaccination?"question (5-point Likert scale, response range 1 = definitely disapprove to 5 = definitely approve) and six 5-point Likert scale items measuring agreement with statements regarding the safety, importance and effectiveness of vaccines in general, and the MMR vaccination specifically.Pre-and post-intervention composite attitude measures were constructed from the seven items (Pre-intervention α = .96;Post-intervention α = .96).
To measure intentions to vaccinate, both pre-and post-intervention questionnaires contained a single item on intentions to vaccinate one's children.The question varied depending on if the children of the participants were already vaccinated.For the mothers whose children were not vaccinated (n = 332) the questions was phrased "How likely are you to have your child do all the necessary measles, rubella and mumps vaccinations?"and for the participants with children who did not require further MMR vaccination (n = 390) the question was phrased "If you have another child, how likely is it that you will get all the necessary measles, rubella and mumps vaccinations for him/her?"Participants answered one or the other question and responses were recorded on a 5-point Likert scale (1 = strongly disagree to 5 = strongly agree).Generalised According to the order of the Education Department and the order from our school, children that are not vaccinated in accordance with the vaccination calendar won't be allowed to attend schools.All children must be vaccinated with two doses of MMR vaccine at 1 and 6 years of age in order to be protected.
It is important to vaccinate before the beginning of the school year.

Dear parent,
The measles outbreak is continuing in Ukraine.Measles is one of the most contagious and dangerous diseases affecting children in Ukraine.
Vaccines are safe, quick, free of charge and simple.It is difficult is to treat complications from measles.All children must be vaccinated with two doses of MMR vaccine at 1 and 6 years of age in order to be protected.
It is important to vaccinate before the beginning of the school year.

Dear parent,
The measles outbreak is continuing in Ukraine.Measles is one of the most contagious and dangerous diseases affecting children in Ukraine.
As your doctor, I recommend a vaccination.I vaccinate against measles not only for myself but also for my children and family.All children must be vaccinated with two doses of MMR vaccine at 1 and 6 years of age in order to be protected.
It is important to vaccinate before the beginning of the school year.

Loss-framed -school representative (Intervention 5) Dear parent,
The measles outbreak is continuing in Ukraine.Measles is one of the most contagious and dangerous diseases affecting children in Ukraine.
The majority of Ukrainian children receive vaccination against measles, according to the recommended schedule.You should protect your children, as other parents do.All children must be vaccinated with two doses of MMR vaccine at 1 and 6 years of age in order to be protected.
It is important to vaccinate before the beginning of the school year.

Dear parent,
The measles outbreak is continuing in Ukraine.Measles is one of the most contagious and dangerous diseases affecting children in Ukraine.
The majority of Ukrainian children receive vaccination against measles, according to the recommended schedule.You should protect your children, as other parents do.All children must be vaccinated with two doses of MMR vaccine at 1 and 6 years of age in order to be protected.
It is important to vaccinate before the beginning of the school year.

Dear parent,
The measles outbreak is continuing in Ukraine.Measles is one of the most contagious and dangerous diseases affecting children in Ukraine.
Children that get measles won't be able to attend school for a long period and the infection can cause further complications or even death.All children must be vaccinated with two doses of MMR vaccine at 1 and 6 years of age in order to be protected.
It is important to vaccinate before the beginning of the school year.

Figure 1. English Translations of the Six Letter Conditions
linear modelling analysis with a normal probability distribution indicated there was no interaction between a child's current vaccination status and message framing on change in intentions to vaccinate (interaction omnibus Wald χ 2 (5) = 6.72, p = .243)so the two groups were pooled for hypothesis testing.
Separately, in order to measure behaviour, three links were provided below the intervention letter (phase 2).Participants with unvaccinated children could select between: 1) "Click this link to schedule your child's vaccination or visit your doctor" 2) "Click this link if you do not plan to vaccinate your child" or 3) "Click this link if the information is not relevant or if you are undecided".Although parents were not aware of this, these options only referred them to information on appointment scheduling, as opposed to actual scheduling due to restrictions in the panel forum.Participants whose children were already vaccinated were asked to select which of the three options they would hypothetically click if their child was not vaccinated.
For both groups of parents, those with vaccinated children and those with unvaccinated children, options 2 and 3 were combined to create a binary outcome variable indicating whether participants clicked to schedule a vaccination or not.A binary logistic analysis estimated using Generalised Linear Modelling analysis indicated there was no interaction for parents of vaccinated and unvaccinated children and message framing on likelihood to vaccinate (interaction omnibus Wald χ 2 (5) = 0.63, p = .987)(see also Supplemental Figure 1) so to maximise power, the two groups were pooled for hypothesis testing (however, for interested parties, see the unpooled analyses below for separate vaccinated and unvaccinated analyses).

Data Analysis
Descriptive statistics were calculated for the seven attitude items, the attitude composite measure and the intentions item.Wilcoxon signed rank tests were conducted to compare preintervention and post-intervention scores on for the whole sample and broken down by intervention.In order to test for an effect of message type on change in attitudes (H1), a 6*2 Mixed ANOVA was conducted, with phase as the within-subjects factor and message type as the between-subjects factor.To address whether change in intentions was linked to message type (H2), a 6*2 Mixed ANOVA was conducted with phase as the within-subjects factor and message type as between-subjects factor.To investigate whether behaviour was linked to message type (H3), simple binary logistic regression was employed to test if there was a significant difference between message groups, in terms of the odds that participants would click to schedule a vaccination, with the control message used as the reference group.Followup ANCOVAs with post-intervention scores as dependent variables and pre-intervention scores as covariates (H1, H2) and logistic (H3) multiple regressions were employed to confirm the results held after controlling for all demographic variables shown in Table 1.We opted for both analyses to allow comparison of adjusted and unadjusted scores with relevant counterparts in the literature.Analysis was conducted in SPSS 25 and an alpha value of 0.05 was used in all analyses.
For the behaviour outcomes, the main and follow-up analyses were repeated separately for mothers whose children were not vaccinated and for those who were already vaccinated.Based on the estimated small to medium effect size we specified in our power analysis, these analyses are not sufficiently powered to reliably detect a significant effect.

Pre-and Post-Intervention Attitude and Intentions
Table 2 displays the difference between vaccination attitudes and intentions between pre-and post-intervention.We observe a significant increase in general attitudes, vaccine safety attitudes and the composite attitude score across the sample.In addition, we see a significant increase in general attitudes for the doctor testimonial and social norm -family doctor conditions, and increased perceived general and MMR-specific effectiveness in the barrierreduction condition.No significant differences between pre-and post-intervention scores were found in the mandatory vaccination condition.We found no significant differences between pre-and post-intervention intentions to vaccinate.

H3: Effects of Message Type on Vaccination Behaviour
Figure 2 shows the proportions of participants from each group who clicked to schedule a vaccination.The social norm -family doctor message had the highest percentage of participants who clicked to schedule a vaccination (75.9%) and the loss-framed condition had the lowest percentage (52.5%).Simple binary logistic regression analysis showed a significant effect of message on clicking to vaccinate (N = 738, Nagelkerke R 2 = .03,omnibus χ 2 (5) = 15.97,p = .007).The social norm -family doctor message had significantly higher odds of clicking to vaccinate compared to the control mandatory vaccination letter currently employed in Ukraine (57.7% clicked to vaccinate; OR = 2.310, p = .004,95%

Follow-Up Analyses
Follow-up analyses testing each of the three hypotheses but adjusting for region of Ukraine, size of settlement, age of mother, age of child, educational level, marital status, financial situation, and current vaccination status of child showed similar findings to the unadjusted analyses.The results of the follow-up analyses are shown in Supplemental Tables 1, 2 and 3.

Analysis of Unpooled Data
In the unvaccinated group, there was no significant effect of message on vaccination behaviour (N = 341, Nagelkerke R 2 = .03,omnibus χ 2 (5) = 7.19, p = .207),as expected.However, the social norm -family doctor condition had significantly higher odds of clicking to schedule a vaccine than the loss-framed message (p = .014).Strikingly in view of the power calculation, in the vaccinated group, there was a significant effect of message on vaccination behaviour (N = 378, omnibus χ 2 (5) = 11.06,p = .050).The social norm -family doctor condition had significantly higher odds of clicking to schedule a vaccine than the mandatory vaccination condition (p = .011)and the loss-framed condition (p = .008).

Discussion
Using an RCT, we set out to test for the effect of five behaviourally informed MMR vaccination invitation letters, compared to the Ukrainian national template letter, on pro-vaccination attitudes, intentions and behaviours amongst Ukrainian mothers.We measured their effectiveness using three outcome measures: Change in vaccine-based attitudes, change in vaccine-based intentions and vaccine behaviour, measured through clicking through to schedule a vaccination appointment.Our findings provide evidence to support an overall increase in positive attitudes between pre-and post-intervention but no support for an effect of message type on attitudes (H1).This indicates that letters such as those employed in the study can positively impact attitudes even if no one behavioural frame shifted attitudes more than another.Given that there was an increase in positive attitudes from pre-to post-intervention but no difference between the conditions, it could be that the change in attitudes from pre-to post-intervention were due to an exposure effect.In terms of vaccination intentions, we observed no changes between pre-and post-intervention and found no evidence for an effect of message type (H2).Previous research (e.g., Abhyankar et al., 2008) has shown that different message framings can affect vaccine intentions so our finding could be seen as surprising.H3 (message type affects vaccine behaviour) was partially supported, with the different letters showing significantly different behavioural outcomes.This result indicates that the framing used in MMR vaccination letters matters for vaccine uptake and adds to the body of literature showing the importance of behaviourally-informed message framing specifically (e.g., Milkman et al., 2011;Milkman et al., 2021;Yokum et al, 2018) The results show that 75.9% of the participants who received the social norm -family doctor message clicked the button to schedule a vaccination.This was significantly more than the proportion who received the mandatory vaccination (57.7%), barrier-reduction (58.7%), lossframed (52.5%), and doctor testimonial (61.1%) letters.Further, 64.7% of participants in the social norm -school director condition also clicked to vaccinate, the second highest percentage clicking to schedule a vaccination, and a significantly higher proportion than the loss-framed condition.Given the two most effective letters employed the social norm framing, the logical conclusion is that letters that emphasise vaccination to be a social norm are the most promising candidate for increasing vaccination rates, in the Ukrainian context at least.To the best of our knowledge this is the first finding from an RCT showing social norm messages to be more effective than loss-framed messages, or any other types of messages.It is worth noting that both social norm messages were signed by an authority figure, so it is possible that perhaps a social norm letter is only effective when signed by an authority figure.Theoretically, this finding could be explained through social identity theory whereby compliance with the behaviour of others provides people with a feeling of belonging and minimises social rejection (Cialdini & Goldstein, 2004).
In contrast, only 57.7% of participants who received the Ukrainian national template letter -mandatory vaccination -clicked to book a vaccination.This adds support to the idea that stressing the mandatory nature of vaccinations may not be the most effective way of encouraging vaccination.One reason for this could be that enforcing vaccination, particularly in a situation where the government's legitimacy is in doubt, may be perceived as curbing people's freedoms, leading to resistance or opposition (Brehm & Brehm, 2013;Hendrix et al., 2016).It's worth noting that the Mandatory Vaccination letter could be seen as a form of lossframed letter in that it stresses the consequence of non-vaccination (non-attendance at school for the child) and that the only message that had a (non-significantly) lower uptake rate was the researcher-designed loss-framed letter, with only 52% clicking to schedule a vaccination.Taken together, these findings are interesting because they provide evidence that loss-framed messages are likely not effective in a Ukrainian context.It may be that increasing the social acceptability of vaccinations before such policies come into effect, at least in (countries similar to) Ukraine, could be important (Hendrix et al., 2016).
One area where our study provides somewhat more opaque evidence is in the case of the importance of a letter coming from a trusted authority.The Doctor Testimonial letter performed moderately well in terms of the behavioural outcome, although this was not significantly different from any other condition.Additionally, although the most effective message in our study was signed by a doctor, our results do not provide conclusive evidence for a messenger effect (as there was no significant difference between the social norm letters signed by a doctor and signed by a school director in attitude change, intention change or behaviour).Existing research indicates that doctor recommendations can be effective for encouraging vaccination (Reiter et al., 2013;Wiley et al., 2013) but our findings do not provide any additional evidence to support this contention.

Limitations
We note that our study has limitations.For one, our sampling was conducted using an online panel.This means that the representativeness of the sampling frame from which we sampled could be called into question, because the participants in the panel are volunteers and not sampled randomly from the Ukrainian population.Our use of only mothers rather than only fathers could be seen as a limitation but in Ukraine mothers make most of the decisions related to child health, including vaccination, and typically attend health facilities with children (UNFOA, 2020).The effect of the interventions on behaviour can only be assessed in relative terms because there is no comparison to a control group who don't receive any letter.In terms of the design of the different messages, we didn't include a gain-framed message so are unable to compare the effectiveness of such a message framing in the Ukrainian context.We also did not use mutually exclusive messages.In particular, some letters were signed by a school director and some by a family doctor so it is hard to pick apart the effect of each.It is important to acknowledge both that there is more than one way to design a letter so that it is consistent with a certain framing and that combinations of different message types may be more effective than the use of a single framing strategy.All the messages were signed by an authority figure of some sort so it is impossible to say if any of them would be effective without the reference to authority.We also have no way of knowing how letters with different wordings or emphasis may have performed but it is interesting to note that the letter that combined the vaccination as a social norm wording with a signature from a family doctor was the letter with the most vaccine positive behavioural outcomes.
Furthermore, it is also unclear exactly how valid the behavioural outcome is as an analogue for real-world behaviour.For one, we used a hypothetical frame for parents whose children were already vaccinated and combined the responses of the unvaccinated and vaccinated parents in our main analysis, although it is worth noting that the responses for vaccinated and unvaccinated parents on the behaviour outcome were very similar, both in terms of the proportion of mothers in the different conditions who clicked to vaccinate and in the results of the potentially underpowered analyses that examined the two groups separately.Further research should aim to work with a larger sample of parents who have children that are not vaccinated.
Additionally, the full pathway to vaccination was not tracked -only clicking to schedule a vaccination (leaving it open to interpretation how many of those vaccinations would occur)and the letter was attached to an email rather than on paper.Participants were also given the option to schedule an appointment immediately after the letters were read, potentially boosting pro-vaccination behaviour more than would be the case in a real-world scenario.This means that actual rates of vaccination based on different letters cannot be extrapolated from the rates reported here, but we would still suggest that the differences between letters are likely to generalise to some extent to real-world vaccination behaviour, even if one cannot extrapolate to actual vaccination rates.Further research would be needed to follow the effects of these behaviourally informed messages through to actual scheduling and attendance rates of their child's vaccination appointment.

Conclusion
This research shows that the content and framing of vaccination letters can have an effect on vaccination scheduling behaviour.It also serves as an example of how interventions may affect behaviour independently of attitude and intention (Jarrett et al., 2015).Vaccination letters that emphasised vaccination as the social norm were effective in shifting vaccine-related behaviours in a positive direction.In contrast to this, the mandatory vaccination messages currently used by Ukrainian public authorities, as well as the loss-framed message (effective in other contexts), did not appear effective.The findings illustrate the importance of careful testing when framing government health communications and provide an important example of how behaviourally informed messaging can help in the roll-out of current and future vaccines.

Notes
1. Based on the Law of Ukraine on Protection from Infectious Diseases and ruling by the Supreme Court of Ukraine, children need to have vaccination in order to attend nursery or school.However, the decision on enforcing this policy is taken by each education institution in Ukraine independently.Starting from 2020, some education facilities started to send parents letters informing them about the policy as an attempt to increase vaccination uptake.The policy is thus the basis for many communication messages.
is continuing in Ukraine.Measles is one of the most contagious and dangerous diseases affecting children in Ukraine.

Figure 2 .
Figure 2. Percentage of Mothers Who Clicked to Schedule a Vaccination by Message Type Note.Error bars represent bootstrapped 95% confidence intervals.N = 738.

Table 1 .
Demographic Characteristics of the Sample at Pre-and Post-Intervention, by Message Group (% Within Demographic Group)

Table 1 .
Demographic Characteristics of the Sample at Pre-and Post-Intervention, by Message Group (% Within Demographic Group)[continued]

Table 2 .
Pairwise Comparisons for Pre-and Post-Intervention Attitude and Intention Scores (Means), by Experimental Condition

Vaccination Messages and Vaccine Hesitancy Atif et al. European Journal of Health Communication 2024
, Vol. 5(1) 46-66 CC BY 4.0 58 Table3shows full pairwise comparisons across the conditions.Note.Values greater than 1 indicate the row message type had higher odds of clicking to vaccinate, compared to the column condition.