By Vince Hopkins, BC BIG
Losing a job can affect the rest of your life.
The early weeks of COVID-19 saw mass layoffs in BC. Our province lost nearly 400,000 jobs. Unemployment doubled.
During this difficult time, the Ministry of Social Development and Poverty Reduction (SDPR) has played a major role.
Through its WorkBC program, the Ministry offers valuable job-seeking services. These include employment counselling and training support — even something as simple as looking over a resume. Services are free and WorkBC has a great track record of success. But for a variety of reasons, some clients delay applying.
To ensure timely uptake, the Ministry partnered with the BC Behavioural Insights Group.
Ultimately, our goal was to help as many British Columbians as possible return to work whenever they were ready.
We followed the RIDE model for behaviour shift.
It starts with user research.
Before the pandemic, BC BIG travelled to the Okanagan, Metro Vancouver, and Southern Vancouver Island. We conducted 90+ interviews with WorkBC staff and clients. We offered virtual consultations and spoke with people across the province.
Talking to users of the service helped us understand the intention-action gap — i.e. why clients who want to connect with WorkBC sometimes don’t. For example, we heard that people may not understand what WorkBC is, or what’s expected of them after losing their job. We also heard that people sometimes underestimate the popularity of WorkBC.
We leveraged these insights to design two new emails, working collaboratively with frontline staff. We also drew on best practices in behavioural science, like using personalization and a clear Call-to-Action.
For each email, we varied the subject line and slightly modified the content. Based on what we heard during user research, one email used a social norms message to indicate the popularity of the service. The other contained a simple checklist indicating that individuals were on the final step towards accessing support.
Both emails highlighted the benefits of WorkBC services and directed recipients to an online, pre-populated form. Once a client clicked “submit”, our form automatically contacted their nearest WorkBC office.
We asked a simple question: Which email works best — checklist or social norms?
To find the answer, we used the NotifyBC platform. Notify is an amazing, powerful new way to send emails — lots of emails. It’s built in-house by our friends at Government Digital Experience in the Ministry of Citizens’ Services. It’s open-source and based on code from the UK and Canada.
Notify is ideal for “A/B testing.” This is a way to compare two versions of something — like a web page or email — and measuring the difference in performance. Basically, you give one version to one group and another version to another group. Then you can see how each version performs, and which one is best.
For our A/B test, we worked with a large sample of people who had applied for EI during the pandemic. We randomly assigned them to two groups. One group received the social norms email; the other group received the checklist email. Finally, we compared engagement, measured by the percentage of email recipients who opened the email, clicked on the link, and submitted the online form.
What did we find?
The checklist email performed the best (see below). Compared to the social norms email, it generated a 13% increase in email openings (+4 percentage points), 43% increase in click-throughs (+2 percentage points), and 59% increase in online submissions (+2 percentage points). Among this group, the conversion rate — the number of people who submitted the form after clicking the link — was high, at 64%. The checklist email also reduced the delay in submissions by 1.5 days. After two weeks, we sent a reminder email to people who had not yet submitted an expression of interest. This further increased engagement. We were also able to identify the optimal wait period before sending a reminder email — reducing delay by another 7 days.
Finally, we examined results from a Gender-based Analysis Plus (GBA+) lens. For example, in BC, women made up more than 60% of the job losses in the hardest-hit sectors. When we compared across treatment groups, we were encouraged to see engagement was strongest among women, especially those with postsecondary education. In the spirit of continuous improvement, our results show the value of testing, and underscore the need to find the right communication medium for the audience.
Conclusion
This was an exciting project, and we’re proud to have collaborated with our partners at SDPR. Our results demonstrate the importance of building capacity rather than a one-time solution. Now, our client can make testing a core part of how they deliver services.
For more information on COVID-19, visit the BC Government Support and Information Page (gov.bc.ca/covid).
By Vince Hopkins
Vince is a Senior Behavioural Scientist with the BC Behavioural Insights Group.