Supporting Dementia Care Toolkit
Pilot phase

Evaluation


Aim

Evaluate the effectiveness of the Digital Befriending Kit in reducing social isolation and digital exclusion for people living with dementia and their carers

Participants

We invited 11 people to take part in the pilot although two did not attend any of the sessions. Appendix A  contains a profile of each participant and describes their attitudes and behaviours during the sessions

Findings

Significant carer support is required

Almost all participants had significant help from a carer to operate the kit and join the calls. Our hypothesis is that this was because carers felt it was easier to do it for the participant than to teach them and because they were keen for the participant to focus on engaging in the online group rather than worrying about the technology. At the end of the pilot, most participants were not able to join the call using the kit without some help, although it seemed that they were limited by a lack of confidence as much as their cognitive ability.

Elements of success with the kit

Tablet stand: All participants used the stand and some commented on how it proved a useful place to store all components of the kit (e.g. Zoom Tickets and instructions) and reduce the risk of losing them in between sessions. One participant commented that it would be useful to have a more distinctive slot to put the iPad in so that you can align it more easily with the QR code on the ticket. The Zoom Tickets work without the stand (people can still point the tablet at the ticket to read the QR code). The bespoke stand is specifically designed to align an iPad to the Zoom Ticket so it would need to be adapted or adjusted for other devices (e.g. Samsung Tablet).

Zoom Tickets: Participants enjoyed having a tangible printed invitation to the group sessions and the tickets were successfully used by carers and participants throughout the pilot. Some carers would have been comfortable with email invitations but were generally very happy to use printed invitations. On one occasion, we had to send an email link to a carer who was unable to join the call with the QR code for an unknown reason. With another iteration of the design, there could be improvements made to the stand and Zoom Ticket combination to make aligning the QR code even more reliable.

Tap & Chat app: Participants found the distinctive pink colour of the app icon easy to identify, remember and click on as part of the process of joining a call.

Home screen: Some participants commented that the instructions on the home screen helped them to join the call (although it seemed like some participants did not focus on the screen to read the instructions). To make tapping on the app more intuitive, we recommend experimenting with a plain home screen that has no distractions.

Printed instructions: Everyone has different learning styles so it is beneficial to have a range of materials and prompts to support them with using the kit. A set of printed instructions with pictures of each stage of joining a call was given to each participant – some found this useful and others less so.

 

Expect some dropout from participants

Of the 11 participants in the pilot, two did not attend any online community group sessions despite many attempts to support and engage with them (calling, offering to visit etc).

One participant struggled with scheduling and keeping the router, iPad and phone plugged in, resulting in it running out of battery and they therefore could not respond to phone calls from the team offering help and reminders.

Another participant seemed generally less motivated to attend the group sessions (this was not due to difficulty in using the kit as they successfully managed a 1:1 call).

Even the most engaged participants often had clashing appointments so, of six participants, we believe it is reasonable to expect four to attend each session.

Considering the varied needs of the demographics and competing demands on daily routines, we are satisfied that we achieved a high level of engagement throughout the pilot.

Structured activities work well

For each week of the pilot, a different structured activity was trialled, including poetry reading, listening to live music, playing bingo and drawing. Participants had differing preferences towards the activities, which made it hard to please everyone. 

Despite some initial scepticism, the bingo was universally enjoyed and was felt to be the most successful activity. As a format, it was particularly conducive to encouraging conversation among participants and felt like a shared experience. The live music session was enjoyed by most as they found it fun and uplifting although some found it overstimulating (too loud and jolly).

Week 1: Poetry reading

This was the first session so the atmosphere in the session was likely affected by participants getting to know each other. 

We brought two poems to the session and provided printed versions in advance, partly because we thought paper versions would be more accessible and also because participants seem to respond enthusiastically to tangible, physical items. 

We chose one poem that participants were unlikely to be familiar with (it was used by a poetry group we attended for inspiration) and ‘The Owl and the Pussy-cat’. We were slightly worried that the second poem (a children’s poem) might be considered patronising but, in practice, participants enjoyed it the most as it drew on their long-term memory and was easier to relate to. 

Participants enjoyed reading and hearing the poems more than talking about them.

 

Week 2: Listening to live music 

A musician sang and played popular songs on the piano. The music was the most polarising of all the sessions it was loved by some and not enjoyed at all by others. 

Several participants enjoyed joining in with the singing although at times the session felt like ‘one way’ entertainment and more discussion with participants could have been encouraged. A few participants commented that this activity felt like sensory overload and that the experience on Zoom was somewhat disorientating. Personal music preference also plays an important role here and it may be hard to find a genre or style that appeals to everyone.

In general, and intentionally, participants did not mute their microphones although this did affect the sound quality from the musician’s feed and there were occasional interruptions when participants sang along (with various levels of talent). The overall effect was positive and contributed to it feeling like a live and shared experience.
 

Week 3: Bingo and Show & Tell

All participants enjoyed the bingo activity even those who said they thought they would not enjoy it beforehand. The task was simple and practical “without requiring a lot of mental strain”. Participants found it fun and exciting to be looking to win and commented it was “mentally uplifting”. It also reminded some of physically going to bingo in the past. As one participant said, “It was both enjoyable and stimulating and a sense of achievement when you find the numbers.”

At the start of the bingo session, we also introduced a Show & Tell session where we asked participants to speak about something they love (e.g. a book, activity, object or memory). This was in response to feedback from previous weeks where people had wanted more of an opportunity to speak and engage with one another. This worked well as it combined a personal element and an opportunity for connection balanced against a task-based activity that had a clear purpose, set-up and outcome.

 

Week 4: Drawing 

During the drawing session, we showed a variety of pictures one at a time via screen sharing. We were interested to see if participants would focus on the screen to view the picture and, in practice, no one had difficulty with that. One participant with vision impairment was supported by the picture being described to them

We asked participants to bring paper and a pen or pencil to the session and gave them differing amounts of time (between one and five minutes) to draw each picture. Everyone was happy to participate with drawing but it was considered less engaging as an activity and stimulated less discussion.

Art-based sessions may work better when participants are given a brief to produce art before the session and then bring it to share and discuss on the day.

 

Group discussion helped to build connections

The group discussions appeared to be the section of the online sessions that resulted in the most connection building as participants started to talk amongst themselves. This was most successfully facilitated by asking an open question to each individual and follow-up questions based on their responses. Some participants started to ask each other these follow-up questions and reflect together on similar memories or places visited. 

The value of participants living in the same part of London became apparent in these conversations as they discussed activities and facilities in the local area. Some participants also expressed a desire to meet in person. This was an unanticipated consequence of the geographical grouping, which was initially planned for simpler communications via community group leads and in the hope that the groups might carry on in some form beyond the pilot (i.e. being run by local organisations).

Some participants felt the group discussion was the most important part of the sessions as they considered the purpose of the pilot was to feel part of a community and get to know other people. These individuals commented that it felt odd to jump into a virtual activity with people they did not really feel they knew yet. Other participants said they enjoyed the group discussion less because they felt some people dominated the conversation.

 

Variety of personalities dictates tone of the group

The atmosphere in group sessions was dependent on the personalities in the group (which could change from week to week even within the same participant). Some groups had chatty participants who kept the conversation going with little facilitation and others required more encouragement and closer management of participants who were less engaged.

Speaking over each other

A constant challenge throughout the sessions was participants speaking over each other and understanding when to speak, be quiet and respond to questions. One way to mitigate this is for the facilitator to direct questions or comments to one individual at a time by using their name. It is also important to be patient and wait for participants to finish speaking and to encourage those who are more reluctant to take a turn.

Consistency, predictability and preparedness is key

Within this pilot, we wanted to try a variety of activities to see what worked well. Inevitably, some participants preferred some activities over others and would return to sessions wanting what they had previously liked. In particular, the music activity had a different style to the other sessions and divided the group in terms of enjoyment. 

Furthermore, due to the experimental nature of the group, it was hard to give it a useful name and set expectations accordingly. It was often referred to as a ‘reading group’ (based on the first session around poetry), which set the wrong expectations for participants coming to subsequent sessions. 

One learning from this was the benefit of briefing participants in advance so that they come to the session prepared for the activity. In particular, participants enjoyed receiving the brief in the post with a dedicated Zoom Ticket for the session. We also extended this learning to the group session itself, making sure we gave time at the beginning of each session to introduce ourselves, any new participants, what we would do and how long for.

Meeting in person:

Many participants commented on the importance of our team delivering the iPads in person. This was due the fact it provided an opportunity for a personal connection to be established before meeting online. They believed this greatly enhanced the experience.

One participant even suggested it would have been nice for the whole group to meet in person before moving the sessions online as this would have provided:

  • Opportunity for learning the iPad together – feeling as though they are “in it together” and that they are part of a shared experience of trying the new technology with others who are in the same boat
  • Opportunity for casual 1:1 conversations to get to know each other better before meeting online
  • Better establishment of group dynamics so online conversations are easier to facilitate and they can feel more comfortable and confident speaking up in group sessions

Facilitator roles:

As one participant commented: “Who runs the project is really important”. There are two aspects of the facilitator roles to note:

  1. Connection:  Many participants commented on the importance of feeling connected and listened to by the facilitators. 
  2. Support roles:  Throughout the process, we found there was a lot of offline support needed to help people join calls whilst the facilitators were running the sessions. We refined this process throughout the pilot and found it worked best when we had someone offline who supported participants remotely if and when needed. This meant that participants on the call were not distracted by facilitators on the call who were contacting and speaking to other participants.

Use of other apps:

During the pilot, the iPads were wiped of potentially distracting software/apps but participants reported wanting to use the tablets between sessions for their own interests, such as playing Sudoku or reading the news. We were able to remotely add an additional app to one of the participant’s iPads to enable them to make better use of it.

One particular finding is that participants could benefit from using the iPad for simple games in order to become more fluent in using tablet computers, especially learning the ‘tap’ gestures and the use of buttons. This would, in turn, better prepare participants to be able to join calls independently.

We noticed in the community group sessions that participants who were not familiar with tablets did not know how hard or long to tap the screen or the difference in gesture compared to pressing a physical button (e.g. the home button). Long presses could produce unexpected results, such as invoking ‘Siri’, which would lead to confusion. It is easy to underestimate the importance of these ‘small’ factors as barriers to access.

Handset attachment and wireless router:

We gave a handset attachment to four of the participants to see if this was a useful addition to the kit in terms of helping them to associate it with making a call and also helping with hearing. None of the participants used the telephone attachment and said that it was not necessary. 

One of the reasons for this was that most participants joined the call with a friend or relative by their side so using the phone would have prevented the relative from also hearing the call. In addition, the audio environment of the participants’ homes was far more suitable to online meetings compared to the community group settings we previously tested at.

One participant was given a Wi-Fi router and SIM card because they did not have internet access at home. This participant did not take part in the groups and, on a support visit to their home, the router was unplugged. 

Agenda and invitations in the post:

Before one of the group sessions, we sent a new Zoom Ticket to participants in the post with a note detailing the agenda of the session and some printed bingo cards. This physical element appeared to positively engage many of the participants.  

Digital inclusion, social satisfaction and loneliness score:

Alongside qualitative questioning, we attempted to use a quantitative measure of digital inclusion and loneliness by asking a set of scored questions at the start and end of the pilot (see section 5 of this toolkit).

Method:

To evaluate quantitative changes, we asked participants the following questions at the start and end of the pilot:

  • I feel satisfied with my opportunities to socialise with people
  • I am satisfied with the amount of time I spend doing leisure or social activities 
  • How often do you feel lonely? 
  • [Questions to create a score on the government’s Digital Inclusion Scale]

The chart below demonstrates the change in social satisfaction for participants from the beginning of the pilot to the end of the pilot:

  • A positive score indicates they felt more satisfied
  • 0 (no bar shown) indicates there was no change in satisfaction
  • A negative score indicates they felt less satisfied

Chart showing change in social satisfaction from evaluation conducted at start of pilot and end of pilot

All charts showing reported changes in loneliness, social and leisure satisfaction as well as Digital Inclusion can be viewed in Appendix B.

Discussion:

On reflection, we did not feel this method was reflective of the experience for the following reasons:

  • Asking both the carer and person living with dementia means there are two different perspectives, which is hard to draw conclusions from
  • Short-term memory limitations of the participants makes reflecting on their satisfaction of recent activities unreliable
  • Many people found the phrasing of the questions challenging to answer (e.g. How satisfied are you with your opportunity to socialise?) meaning their answers had to be interpreted by our team, which could create variability in participants’ answers between the start and end of the pilot
  • The time frame of only four weeks meant that loneliness and digital inclusion is unlikely to have changed significantly irrespective of our intervention

Although the qualitative results included in the appendices do not show a general improvement or deterioration in loneliness, neither do they reflect the positive comments the team received from participants and carers during and after sessions, which indicate a high level of satisfaction with the activities. Quotes include:

  • “My mother is very excited about today’s meeting 🙏🏼 [after previous meeting]” (WhatsApp message)
  • “Thank you for everything, my father really enjoyed it, and he said it was very nice, .. i hope that you guys, next week, will keep on doing it..i think you guys should keep on, these are really nice classes you are having” (voicemail message)
  • “Thanks again for today [the participant] really enjoyed it. I had a good time as well singing along with the lyrics on my iPhone” (WhatsApp message)
  • “Thank you for your support in setting ‘D’ on the road to using IT again.” (email message)
  • “We are secretly hoping they bring you back to run [sessions] again – it was so much fun and social for ‘N’🤞🏼” (text message)