MIT Reality Virtually Hackathon 2020
Overview
Lewis Gardner and I had met at the ‘19 MIT hackathon and decided to compile a team to address a pressing healthcare issue.
Alzheimer's Disease (AD) is the leading cause of cognitive decline in the western world, affecting a third of Americans over age 85. The disease slowly degrades memory and cognitive skills, hindering the ability to carry out daily activities, inevitably leading to full-time care and death.
Inspired by recent publications in Alzheimer's disease that emitting light at 40 Hz frequency can drastically reduce beta-amyloid plaques in various areas of the brain in rodent models of Alzheimer's Disease. With similar effects when light is absorbed through the optic nerve, it seemed like a VR or AR headset would a fantastic piece of technology to explore.
What we built quickly expanded from a use case of 40 Hz to other methods that spatial computing could assist the lives, and the lives of caregivers, for people suffering from AD.
Blinkenlights is a AR assistant that accompanies individuals with AD throughout their disease progression to have a better quality of life by promoting independence, assisting with memory recall and encouraging social interaction.
Features
Temporal & Spatial Reminder System provides users with temporal or location-tagged reminders to assist with day-to-day activities.
Engagement platform for team of caregivers and remote family members to engage in care for the individual by sending reminders, adding events to calendar and send voice memos, and monitor user's task progress.
Facial Recognition & Memory Recall (Reminiscence) uses facial recognition to help users identify people and pulls up shared videos and images for user to review and trigger memory recall of relationship.
When we implemented Facial Recognition it was the first time ever to run it locally on a Magic Leap AR device. With the help of the official Magic Leap Dev Support team, we published the build and the code to the Magic Leap Store.
Research
We conducted secondary research and interviewed 3 family members of Alzheimer patients, along with two experts (neurologist & PhD candidate in Gerontology and Dementia studies) to better understand the problem space and identify user needs. Here are a few of our key findings:
1. Memory loss interferes with daily tasks such as medication compliance, turning off burner, and self care
"She would...forget how many pills to take and so we had to monitor the pills" -_S.O. on her mother_
"Hard time with personal care/grooming" - _M.T. on his father_
2. Social interaction slows down disease progression and helps the patients feel happier
"Loneliness is a risk factor for depression which can cause a quicker decline. They would not take as good care of themselves if they're also experiencing depression. There is total loss of motivation to do so." - _AS, Corporate Director of Dementia Programs at Tutera Senior Living and Health Care
"More visits from family, people who knew him and loved him [would help him feel happier and independent]" - _M.T. on his father_
3. Photos and sounds are helpful in reminiscence therapy to help patients trigger memories
"We really rely on their family photos. A lot of times, people will look at photos and go through them to make sure they still remember in early stages. Photos are good at sparking memory." - _A.S.
Ethical Considerations
Having serverless facial recognition in our society has obvious social implications. One can imagine the impact on privacy if police or another entity had the power to reference our faces to past discrepancies or legal problems. With that said, technological innovation is inevitable, the only way to combat illicit uses of technology is to educate everyone on technological literacy and democratize the tools so gate keepers never hold all the power.
If you are interested in implementing facial recognition in a Magic Leap device, check out our open source github.