What's new ?
CoachMyLife - Teaser
And... ACTION!
(sound on)
To be continued...
February 2023
Project presentation at the SmartCities International Conference 10th Edition, 2022 in Bucharest, Romania
The theme of the event was Accelerating innovation.
Participation to the conference is open to academics, policy makers, early-stage researchers and students in social sciences and other related fields. Experts in comparative public administration and elected officials of local governments are strongly encouraged to engage and submit their contributions to one of the Conference tracks. This anniversary edition aims to focus more on longitudinal studies of digital reforms, locally and globally.
December 2022
Poster presentation at the 10th Edition of the Congress of the University Carol Davila in Bucharest, Romania
The event hosted more than 10000 participants, MDs, academia and medical students from all medical specialties, along with people from related domains, including people activating in space science and medical robotics. The congress included a session dedicated to Rehabilitation Medicine, presided by Mr Professor Mihai Berteanu.
The abstracts will be published in a supplement of the journal Maedica.
November 2022
Project presentation at the 45th National Congress of Physical and Rehabilitation Medicine in Bucharest, Romania
The event had a massive national participation, with more than 700 participants, mostly medical doctors specialised in Physical and Rehabilitation Medicine. During the three days of scientific event, participants had the opportunity to meet important national decision makers in healthcare in Romania, international and national personalities from Rehabilitation Medicine domains, take note of the new pharmacological and non-pharmacological methods of intervention and evidences regarding their efficiency and also had the opportunity to see and test latest technologies brought by our technology providers.
October 2022
Installation at user's home for the pilot
Here is the installation at an end-user's kitchen.
August 2022
What does the installation look like?
Intensive testing happened at UMFCD, TERZ and UNIGE. Here is what the whole installation looks like:
UMFCD
The team presented all the devices of the solution to a primary end-user: the CML mug recognised by the system for drinking detection, the pill dispenser for medication management, the bracelet for eating detection and the Hometab (tablet).
UMFCD, Canary Tech and Caretronic have met in Romania for the solution CoachMyLife. Participants have made the final testing in a living lab at UMFCD and decided how to proceed with the solution on the international market.
TERZ
TERZ is using a regular android tablet (instead of the ERT Hometab). On the right picture appears the system running on a computer for activity detection based on predefined regions of interest (red and green squares). The right picture shows the camera and computer used to collect and analyse the data.
UNIGE
Before running tests with users,
UNIGE installed the CML solution
in their offices. They didn't use a
tablet but a computer for the
interface.
June 2022
.jpg)
Complex flow integration
A new complex flow has been developed by our technical partners. It defines the behaviour of the full system and shows how the user interface must react according to the situation in the user's kitchen. The details of the flow can't be shared as it represents the core of the CML system.
April 2022
Kitchen configuration... a better definition of the regions of interest
To allow the system to detect what is happening in the kitchen and provide adequate guidance to the user, our partners worked on the definition of the regions of interest.
Hands and feet positions are important to make the detection easier for the system. However, depending on the kitchen organisation and its size, it can be difficult for the system to identify where the user is standing and what he is achieving.
It is now possible to draw the exact areas for the hands and feet on the tablet, with the finger. By proceeding this way, there won't be overlapping of the regions corresponding to the fridge, stove, sink, microwave or any other appliance present in the kitchen.
November 2021
.png)
.png)
New looks of the system!
The intent recognition has been tested by the technical partners of the Consortium and is now reliable, allowing the system to detect defined activities happening in the kitchen. The main concern is now to facilitate the installation of the system at user's homes and to guarantee that all data collected is processed locally. To achieve that goal, the system must be powerful enough. The technical partners found the perfect device with the following characteristics :
1- Computer Vision recognition & Artificial Intelligence algorithms running locally on this "computation box" ;
2- Full privacy respect with video stream processed locally ;
3- 1 click connection with WPS button of the participants' router for an
easy auto configuration
To cover the additional device, UNIGE 3D printed a box to make it more attractive.
Here is how the prototype looks like!
October 2021
Medication management: what alternative PPSA and UNIGE developed to help CML users with their daily medical treatment?
The medication management is one of the most critical daily activities with which elderly people tend to struggle. Pill boxes or connected pill dispensers are usually used. To address the issue, a solution has been elaborated by PPSA and UNIGE, and is currently being tested by PPSA chronic customers.
To achieve this, the PPSA technical team adapted the medication robot process, which was at first dedicated to nursing home’s residents. Individual medication packages are prepared by a robot, based on the doctor’s prescription. A personalised roll is produced with the medication for a month. All important information appears on each individual package (name of the patient, date, time, drugs present in the pouch, dosages, QR code).
To help the user with the medication management, a reminder appears on the app so he notices it is time to take his medical treatment. The medication intake can be validated by scanning the package or confirmed by user speech. The next development steps will be to generate a report, accessible to the general practitioner. This feature can also predict medication adherence.
June 2021
Intent recognition and computer vision: how does it work?
The intent recognition is a key element of the CML solution and works through computer vision. Some activities happening in the kitchen were identified (use a microwave, the stove, the coffee machine, etc.) and decomposed into steps so the system can provide the adequate guidance to the user at the right moment.
Step 1: Kitchen configuration and appliances localization
Step 2: Recognise the activity achieved by the user
Once the activity has been detected by the system, guidance can be provided directly on the user interface to help him achieve all the steps required to complete the activity...
April 2021
New publication!
Check out the paper published by our partner from the Department of Intelligent Systems at the Jozeph Stefan Institute - Slovenia.
March 2021
CML promotional video in progress...
The promotional video of our solution has been recorded! We are currently working on it and looking forward to sharing it with you!
January 2021
How does the CML application look like and which are the main functionalities ?
The requirements for the application have been discussed during co-creation workshops, organised by the end-user organisations.
Some very interesting feedback was given and
taken into account when improving the design
of the application such as:
- front size
- buttons colours
- icons design
- user interactio
December 2020
The home page will give an overview of the activities completed throughout the day.
When activities have been detected by the system, the icons will change colours and turn to green.
However, if the CML system hasn't detected any drinking activity, the corresponding icon will turn to red and a notification will be sent in order to remind the user to take his/her medication, drink or eat.
The general menu will allow the user to travel around and access the functionalities offered by the CoachMyLife application.
From the menu, the user have the possibility go back to the Homepage, create new reminders, have a look at the four main activities and access the corresponding instructions when needed.
On the "Reminder" page, the user will be able to create a new reminder and select the category in which it belongs.
For example, the user can create a medication reminder, with a specific title, a description of the reminder, set a time for the alarm to ring and a notification to pop up if the action wasn't detected by the system.
The user can as well consult all the reminders saved on the tablet.
The activity page allows the user to access some information regarding the four main activities identified during the co-creation workshop:
- medication
- hygiene (teeth brushing)
- eating
- drinking
The use of the devices connected to the application are explained and instructions are available on videos as well as text.
The UMFCD Team participated to the 43rd National Congress of Rehabilitation Medicine. Click here to access the abstract.
October 2020
Which camera was chosen for computer vision ?
Good news: our technical partners have found the perfect camera to detect drinking activities !
After testing different models of cameras, here is the one which was chosen:
SV-B01W-1080P-HX camera model.
It supports the protocols needed, provides a good prediction for computer vision. It has been successfully integrated to the solution and connected to the Hometab. In order to keep our accuracy when deploying our solution to the end-user’s home, we must stick to this model.
September 2020
Second Co-Creation activities with end-users
The end-user organisations involved in the project in Switzerland and Romania conducted workshops and heuristic evaluations in August 2020 to evaluate the status of the application. The goal was to get feedback on the current interface design of the CoachMyLife application in terms of usability and accessibility before the start of the first field tests in the real user environment. In addition to seniors aged 65+ as primary users, medical experts, caregivers and further usability experts were involved in the evaluation.
An evaluation by experts provides feedback on how a product or concept idea can be optimised. Therefore, we have used the method of heuristic evaluation for gaining feedback of medical experts and caregivers. In order to involve seniors in the co-designing process, workshops in the end-user organisations were originally planned. In Switzerland they were conducted remotely by using the virtual meeting tool “GoToMeeting” since we wanted to prioritise telematic contact over face-to-face interactions in order to protect the seniors due to the Covid19 pandemic.
The results of the expert evaluation and the senior's feedback were very positive and helpful. The structure of the app appears to be understandable for both experts and seniors. The application was found to be very helpful for future users as long as individuality and adaptability to the user's needs are taken into account. Data protection is also an issue that concerns seniors nowadays. Therefore, steps are being taken in the project to create a trustworthy system that will meet with great acceptance by future users.
Thanks to the great participation of end-users and healthcare professionals, important insights for the further development of the CML application could be gained. This brings the project team closer to a solution that creates real added value and wants to be used by the target group.
August 2020
The UMFCD Team participated to the 22nd edition of the European Congress of Physical and Rehabilitation Medicine, in Belgrade (Serbia).
September 2020
3rd "in-person" Meeting
Because of the pandemic, the in-person meeting was canceled and the partners met virtually. However, each partner has been able to make a presentation and a demonstration of what had been achieved during the lockdown.
The home-office didn't impact the progress of the CML solution and the technical partners have been very creative:
- JSI tested the Computer Vision at home
- PPSA and UNIGE elaborated a video for the ERT smart pill dispenser utilisation
- TERZ organised a virtual workshop with end-users
- CTI worked on the CML application
- UMFCD finalised the study protocol
Given the COVID19 pandemic and its uncertain evolution, the Consortium decided to postpone the first pilot phase to mid-October 2020.
The next in-person meeting will hopefully be maintained in Slovenia, beginning of December 2020...
July 2020
2. During the setup, the user can manually select some areas and objects of interest in the room.
The algorithm for detecting the activity of drinking is now working !
JSI is in charge of developing computer vision for hydration activities. The whole process is organised this way:
6. After being alerted, the system analyzes for some time what the user is doing, and checks if they are drinking or not.
If they don’t drink for too long of a time, the tablet will receive a notification...better to get some water!
June 2020
1. A camera will be placed next to the ceiling of the kitchen, in order to observe the whole room.
3. The camera observes the scene, and nothing happens as long as the user is just wandering around.
4. However, if the user steps inside one of the selected areas, or touches ones of the selected objects, the program is alerted that it’s time to get to work!
5. One of the situations that will alert the system is the displacement of the CoachMyLife mug.
How will our solution help users take their medication ?
An automatic pill dispenser is a perfect solution for patients suffering from cognitive impairment or mild dementia, mentally or visually impaired patients, or patients who need to take multiple different medications and need notifications when certain medications have to be taken.
The automatic pill dispenser can be filled with a supply of pills that lasts between one week and one month, depending on the number of daily doses. A loud acoustic signal announces the treatment needs to be taken. Only the specific pills are available to the patient, as previously set in the dosing area.
If the medication is not taken at the set time, it is safely stored in a locked tablet dispenser.
May 2020
How will our solution detect eating ?
Understanding the dietary habits of people plays a crucial role in interventions promoting a healthy lifestyle. People with mild cognitive impairment and dementia require monitoring of their eating behaviour because they often forget whether they have already eaten. As a result, they eat lunch or dinner multiple times a day or not at all, which might cause additional health problems. Proper treatment of these issues requires an objective measurement of the time at which the meal takes place, the duration of the meal and what the person eats.
For this purpose, we are developing an eating-detection method using a wristband. Wristband devices and smartwatches are increasingly popular, mainly because people are accustomed to wearing watches, which makes the wrist placement one of the least intrusive placements to wear a device. Additionally, the cost of these devices can be relatively small, which makes them easily accessible to everyone.
Our method works with motion data from an accelerometer and gyroscope sensors. Using these two sensors, we are able to monitor the movement of the hand, from which we can extract information whether eating takes place. The method makes a decision using an advanced AI algorithm. Additionally, our eating-detection method is energy-efficient, which means that even though the wristband is a device with limited battery life, it will be able to run the advanced method without significantly affecting the battery. At the moment, our method detects periods and duration of eating. However, in the near future, we also plan to focus on detecting bites.
Ref: https://dl.acm.org/doi/abs/10.5555/3400306.3400354
April 2020
What are the Hometab functionalities ?
The HomeTab is a desktop based touch-screen IP-based carephone specially adapted both from hardware and software sides for it's end users (mostly elderly people, not having much technology skills).
Those are all the possible functionalities, but some of them will be used in CML pilot. The CML application will be managed through the HomeTab developed and provided by Caretronic.
Functionalities for users:
• Emergency call that can be routed to formal or informal caregivers
• Service calls (request for cleaning, food ordering)
• Smart-home management (switching on-off lights, heating, curtains …)
• Reminders and appointments (birthdays, doctor appointments, medications …)
• Video calls (with family, friends …)
• Playing games
• Possible integration of other apps / platforms
Functionalities for caregivers:
• List of pending / requested tasks to perform at the user (elderly)
• Documentation of all services done at / to the user (health-care related and other, such as cleaning, food delivery)
• History of tasks accomplished
March 2020
Which devices will be used to detect the identified activities and send the right reminder at the right time ?
Our partner Caretronic provides professional ICT solutions for nursing homes, hospitals, and other care facilities, as well as for elderly individuals who feel the need for independent living, more efficient communication and want to simplify their everyday activities with this kind of systems.
For the pilot phases, three devices will be provided to the participants:
- A tablet (Hometab) with the CML application
- A wristband
- A smart pill dispenser
March 2020
2nd in-person Meeting (Sinaia - Romania)
The UMFCD team welcomed the Consortium and allowed the partners to discover a beautiful region, after long days of brainstorming and decision making.
Activities which will be recognised through computer vision have been identified, the inclusion criteria for the participants to our first pilot phase were defined and a prototype of the application has been shown by CTI.
Among the activities which have been enunciated by the participants of the TERZ first Co-creation workshop, four have been chosen by the CML partners:
1. Medication
2. Hydration
3. Eating
4. Teeth brushing
The Computer vision will recognise the activity and allow the CML solution to provide the adequate reminder to the user.
Next stop: Slovenia in summer 2020...
February 2020
First Co-Creation Workshop with end-users
In November 2019, initial workshops were carried out with seniors at terzStiftung in Switzerland to identify user needs and user requirements with regard to the CML solution. Following a user-centered design approach, end-users in the CML project are involved in the entire development process and thus participate as co-creators.
From November 28th to December 5th a total of 4 workshops took place in the premises of terzStiftung, each with 4-7 participants.
After a short introduction to the idea of CoachMyLife, the participants had the task to write down important activities of their daily life on cards and place them on a timeline on the pinboard. This was to determine which activities are important in daily life, when they are usually performed and how flexible the CML system should be. In a second step the cards were assigned to rooms using room sketches to find out where these activities are primarily relevant.
Furthermore, the possible challenges/problems related to the performance of the identified daily activities were discussed. Finally, the activities were prioritized according to their importance, which served as a starting point for the development of the system.
With regard to the implementation of CML, the participants agreed that it should be a modular system that grows with the user’s needs. Individuality and flexibility are mentioned as well as reliability and privacy. CML is considered a chance of living longer in one's own home.
However, the workshop also showed that it will be a challenge for developers to meet all user requirements, since activities of daily life are based on individual needs and these cannot be generalized. Each person organizes his or her daily life differently and has different assistance needs, which makes the system very complex. This challenge encourages the CML team all the more to develop a system that creates added value for everyone.
November-December 2019
The UMFCD Team participated to the 42nd National Congress of Rehabilitation Medicine, in Cluj-Napoca (Romania), where they presented their research : „Errorless Learning For Cognitively Challenged Older Adults”.
October 2019
KickOff Meeting
The CML project started in Geneva - Switzerland, where both general and technical coordinators (PPSA & UNIGE) are from.
All partners, coming from Slovenia, Romania and Switzerland, finally met in person. During this journey, they have been working on the CoachMyLife strategy and learned to know each other more.
At the end of their stay, they all went back home, with some swiss chocolate and many promising perspectives.
And so it begins...
July 2019