Engaging patients and triggering self-care: using Intelligent Augmented Reality apps

USE OF AUGMENTED REALITY FOR SCREENING FOR ATRIAL FIBRILLATION

AR is a technology where a phone camera is used to add to (or augment) real world capture with virtual 3D objects.

Know My Heart and Know My Beat apps have been designed specifically to use this technology. The aim is to engage members of the public and individual patients in visual ways to assist them to check their pulse manually at regular intervals and become more aware of the potential consequences of AF, so that those with it are more likely to adhere to treatment (www.virtualhealthshed.com/know-my-heart).

Know My Beat (Figure 1) uses AR to create a virtual hand that appears on any hard surface (for example, a table top), showing in the viewport of the phone’s camera. It is also ‘trained’ to detect a specially-printed drinks mat. This has become part of an at-scale campaign across Staffordshire to transmit information to the general public via a pub chain, where hopefully, the apps will be of benefit to the customers drinking and eating there. The concept provides health information at a location and time where users are more relaxed and receptive to learning about checking their own pulse, or getting friends and family members to do so.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Know My Beat — Virtual Pulse.

HOW IT WORKS: KNOW MY BEAT

The AR feature is designed to display a virtual 3D hand. As the feature is a 3D model, the app is designed to detect if the user has pressed two fingers on the correct location of the virtual wrist. When the user does this correctly, the app responds by using the device’s vibrate feature (if available) to simulate a normal pulse and also one that is irregular. It displays a 3D graphic of the heart to show the relation to normal and irregular pulse rhythms. If the user does not find the virtual pulse, a ‘tour’ is provided whereby virtual clinicians and an informed patient appear on the hard surface or drinks mat and demonstrate how to do this. A selection of questions and answers are provided in the app using the same technology to allow the clinical team and patient to answer those questions as ‘avatars’ in the app.

HOW IT WORKS: KNOW MY HEART

Know My Heart (Figure 2) uses AR in a different way, showing two versions of the heart from which the user can select one — a beating realistic 3D model of the heart or a cartoon version. The content of the app was co-designed with patients, and the cartoon version added in response to some patients’ fears that the realistic-looking heart model could be alarming.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

Know My Heart — Tour.

The app offers the user a similar option to direct the heart image to any hard surface that can be seen through the phone camera. Once the heart is placed on the surface, the ‘tour’ function uses AR to feature the portion being discussed by the clinician avatars. As the clinicians explain and point to parts of the heart, AR is used to illuminate these sections in 3D. This allows the user to move their device’s camera to explore the heart from any angle. As they do so, the tour illuminates the correct section of the heart, regardless of the angle the user is looking at. The user can also change the delivery of the beat to an irregular rhythm, which shows the blood flow and the consequences of how a clot can form in the left atrial appendage to be expelled via the aorta to the rest of the body.

AR apps are an ideal way to engage with the general public at scale and trigger interest and behaviour change, thus improving health and wellbeing.

In a minority of cases, using the apps could exacerbate health fears and in turn generate unnecessary clinical consultations, but on the whole, these innovative apps should support patients to take more responsibility for their own health and reduce demand on the NHS.