Mar 13, 2023
KI.ASSIST (“AI.ASSIST”) research project successfully evaluates assistance systems supported by Artificial Intelligence (AI) to increase the occupational participation of people with various disabilities. [By Lisa Gruber]
The purpose of the research project KI.ASSIST, which was launched in 2019, was to increase or even make possible the occupational participation of people with disabilities by using AI-supported assistance systems. The task was to identify which technologies are suitable for specific limitations and which framework conditions they require to promote the vocational independence of the rehabilitees.
The goal of the funded research project KI.ASSIST was to increase the participation of people with a wide range of disabilities in vocational rehabilitation. Several vocational promotion centres (BFW), vocational training centres (BBW) and workshops for people with disabilities, as well as their umbrella organizations, participated in the research project KI.ASSIST (2019-2022), which was funded by the German Federal Ministry of Labour and Social Affairs (BMAS). The research partner was the German Research Centre for Artificial Intelligence (DFKI). In so-called learning and experimentation spaces, one of the questions investigated was whether AI-supported assistance technologies can support the acquisition of skills as well as the learning of processes in a wide range of professions. The project also evaluated the extent to which these technologies can be used outside vocational rehabilitation facilities to promote the participation of people with disabilities in working life.
TeamViewer contributed its Augmented Reality (AR) system Frontline to projects that focused on trainees in IT and technical professions, as well as on nursing professions inside and outside the facility.
In industry, clear step-by-step instructions have been improving business processes in such areas as logistics and maintenance over many years. In particular, the AR-solution TeamViewer Frontline xMake, which focuses on manual manufacturing processes, was used in the learning and experimentation rooms in Munich and Pirna (Germany).
xMake operates on data glasses, wearables (i.e. mini-computers that can be worn on the body) and mobile devices. In the KI.ASSIST projects, the choice fell on smart glasses models from the TeamViewer partners Vuzix and RealWear. These devices, already established in industrial contexts, are worn on the head and have a small screen located directly in the user’s field of vision. In addition, the devices are equipped with a camera, microphone and speaker.
xMake assists wearers of data glasses with pre-programmed step-by-step instructions (workflows). Depending on the task and information to be conveyed, videos, images or texts can be projected onto the screen of the data glasses. A spoken instruction is played back via the speaker of the data glasses. Users can control the flow of the instructions individually by voice command. This means they can pause the tutorial to perform the action that was just described. In addition, they can repeat individual steps, comment on them, or even cancel them. If work results are to be validated, the workflow can be set up in such a way that individual work steps must be confirmed before the instructions can be continued. Speech recognition and control of the data glasses are based on an AI model pre-trained by TeamViewer for speech and image recognition.
In all learning and experimentation rooms, the use of data glasses was found to be beneficial for many target groups, especially for routine activities with the same procedure over and over again. Only in the case of people with severe physical or cognitive disabilities did the use of data glasses prove rather difficult because of the technical handling. Speech recognition did not always work properly in case of speech problems.
Since some AI-supported assistance systems have to be prepared for each task with a precise fit, they cannot necessarily be used spontaneously. Skills for the various AI assistance systems, such as data glasses, biofeedback tracking, navigation system, voice control, need to be built through training and consultation.
At BFW Munich (Vocational Promotion Centre Munich), TeamViewer Frontline was used by over 40 trainees to build up expertise and to learn and practice work processes. The use of AR technology has proven particularly promising for people with learning disabilities or those with mental and physical impairments.
First, trainees working in IT and electronics tested the step-by-step instructions via data glasses. Positive side effect for these trainees: They are working their way into next-gen technologies. The rehabilitation patients confirmed that the speech recognition worked well. All found the work and learning support helpful. Only the unfamiliar handling of the new high-tech devices initially led to habituation effects such as dizziness for some users.
Testing the technology, especially in low-threshold work such as the installation of network cables, should provide initial empirical values for further application scenarios. Podiatry and the commissioning and control of electronic circuits are envisaged. In other areas of application, for example in the kitchen, the data glasses were perceived as rather annoying and were not considered providing helpful support.
The AWO (Workers Welfare Institution) Workshops Pirna for people with disabilities also tested TeamViewer Frontline, to teach 33 of their 550 employees work processes in the care sector in a structured way. Some rehabilitees with cognitive disabilities and learning disabilities also work in jobs in the primary labour market, such as in residential homes and in assembly. Different step-by-step instructions (workflows) were created for their tasks and tested with two different pairs of data glasses. It was important to bear in mind that data protection guidelines limited the use of data glasses with cameras at certain public locations outside workshops.
Within the framework of the project, the experts agree that the supportive function of the technology in the workplace has already reached a high level of maturity and is suitable for a broad target group. Frontline can be used flexibly for a wide range of applications and has demonstrated direct added value in productive use. For many activities, this means that there is no need for accompanying personnel, which frees up capacity and reduces the cost of guidance and support. Overall, the technology was received very positively during the project, so that individual institutions have decided to continue it independently – for example BFW Munich – and it remains to be seen how many will follow and trust the advantages of AR in vocational rehabilitation.