Robot assistants are able to help users in their daily lives, and teleoperation and telepresence are just some of the capabilities that make this possible. Teleoperation and telepresence help to provide an opportunity for human-robot collaboration to perform cooperative tasks in environments that may be dynamic or unstructured. An example of this is in project SANDRo which PAL Robotics is a project partner in. SANDRo stands for (Semi-Autonomous Night and Day Robot) and through PAL Robotics’ TIAGo robot provides assistive services to people with difficulties in Activities of Daily Living (ADL). In project pilots, TIAGo which forms the base of the SANDRo solution will support users. The project is conducting trials in several care centers and test facilities across Europe.
SANDRo can be tele-operated by a remote operator when needed (e.g. on-call or when an alert is triggered). This operator, located in a support center, can communicate with users through voice and video, which enables safe and detailed remote visual inspection. Once the action has been defined, the operator uses a specialized cockpit interface with integrated haptic feedback, so the operator feels the forces that are applied and can accurately operate the robot and its arm. Making it possible to conduct special tasks such as picking up things from the ground (it would take more time to call a care professional to solve this situation) or more complex tasks such as putting a mug or plate in a drawer, and performing close up wound inspection via the wrist camera.
The remote cockpit technology in the SANDRo project for telepresence tasks is based on 14 years of heritage work by project partner HIT, in remote handling maintenance. Proper haptics is an important factor in achieving transparent remote control. This technology is realised on a novel, dexterous and cost-efficient system, based on PAL Robotics’ TIAGo platform.
Remote control guarantees the safe and successful execution of more complex Activities of Daily Living (ADL) tasks. A human operator involved is the key to making a service robot more robust and efficient in the execution of practical tasks, using human judgement, sensor, and motor skills interactively. Especially in care environments, where situations are unpredictable and variable, a semi-autonomous approach helps to provide a reliable and robust service system.
SANDRo project use cases including as a personal assistant, activity counsellor and patrolling robot
Personal Assistant: Here the robot executes user-requested Activities of Daily Living (ADL) via teleoperation. The tasks are requested by either clients or care workers. Examples of such tasks are: picking objects from the floor, fetching and bringing household items, setting the table for lunch, and transferring medical equipment. During the task, the operator is able to talk to the user and explain what the robot is doing or have an informal chat.
Activity Counsellor: Here the users call the robot either via verbal commands (if the robot is nearby and able to listen to them) or via an interface. The robot approaches the user and instantiates an interaction session to identify the user’s request. The robot uses Natural Language Understanding (NLU) methods to infer the request of the client, match it to one of the predefined requests and address it. The Activity Counsellor role also includes skype-like communication with remote consultants or care professionals, relatives, and family members.
Patrolling Robot: This role includes patrolling, and approaching clients to offer them assistance. The robot stands close to the client and starts an interaction session. The session may include autonomous interaction or communication with the operator via an interface.
As well as the trials carried out in the Netherlands the project will carry out more trials in Paris and Barcelona healthcare centres.
Project trials in Paris healthcare centre including facilitating entertainment and remote communications
The project trial in a healthcare centre in Paris includes some of the following activities.
Tasks via telepresence
A nurse sends a robot to a patient’s room or to a communal area to start a remote screen care session. The session includes the following:
- Communication with a therapist
- Communication with a relative or family member
- Entertainment: playing music, reading text, showing a video with exercises, or playing a simple cognitive game in the form of questions
- Monitoring session: a nurse is monitoring the wellbeing of patients and speaks via the robot to give instructions
Other potential telepresence tasks may include:
- Act as a single point of contact (receptionist/informant role)
- Webcalls with relatives or therapists
- Webcalls with the operator for direct assistance
- Emergency calls or notification of a care worker
- Entertainment and activation (play video/music, read news)
- Remote screen care (incl. would inspection)
- Remote client monitoring (sound & video)
Tasks via teleoperation
The robot helps clients with Assisted Daily Living (ADL) tasks, such as bringing a glass of water, a medicine cup, or a towel. The patient can also call the robot to his/her room. A nurse can give an object to the robot to bring it to another place and hand it over to a patient.
Via teleoperation, SANDRo also helps with setting the table (placing objects in dedicated places), sorting groceries from a bag, serving/delivering goods on a tray or cart, throwing away garbage, fetching and bringing objects, or picking up objects from the floor.
Project trials in Barcelona healthcare centre including fall detection and fetching objects
At project pilots in Barcelona, the robot also takes part in checking client status, in addition to fall detection, as well as sending notifications to the nurse app. In addition, in this pilot the robot also enables telepresence tasks such as communication with a therapist, relative or family member, entertainment and monitoring sessions via nurses to provide instructions, as well as teleoperation tasks such as support with fetching and bringing objects.
Project trials future tasks including robot self-learning execution of tasks
In these trials, planned future robot tasks include:
- Automation of basic and repetitive manipulation tasks
- Human/face detection and autonomous addressing of requests
- Screen rehabilitation
- Self-learning execution of tasks by demonstration
PAL Robotics’ TIAGo robot used in project SANDro
The SANDRo solution is based on PAL Robotics’ TIAGo robot, with various integrations provided by project partners. TIAGo is an omnidirectional mobile base with a lifting torso (height 110-145cm) and a 7 DOF arm with a payload of a minimum 3kg. The design of the arm and torso and the implementation of a whole-body-controller software allow for flexible motions as well as reaching the floor to pick up fallen objects. Additional equipment can be easily mounted on the platform, such as grippers that will allow the robot to manipulate and tiny objects that request significant dexterity such as medicine cups, pills and credit cards.
The autonomous interaction module of the SANDRo system, entails a software module that is equipped with the following capabilities:
- Speech Recognition via the robot’s microphone to capture what the user says
- Understanding the user’s request and selection of an appropriate response
- Robot speech via the robot’s speaker
Project SANDRo: overall goals
The overall aims of project SANDRo include the following:
- Generate insight into the added value of robot assistants in healthcare, and the future acceptance of this from the perspective of care users and care workers/professionals.
- Evaluation of the communication (client control & social interaction), remote manipulation of ADL tasks with clients, informal carers and care workers, and autonomous navigation.
- Evaluate the experiences of healthcare workers and clients with the system by means of questionnaires and interviews. This gives insight into the perspective of the care recipients and care employees/professionals with regard to the potential added value of the robot.
- Learn from the results for the future development of the robot.
SANDRo is a project by DIH Hero with project partners PAL Robotics, and project leader HIT (Heemskerk Innovative Technology) which provides advice and support to innovative high-tech projects in the field of robotics and mechatronics. DIH Hero is an independent platform that connects Digital Innovation Hubs across Europe to create a sustainable network for all those who are active in the healthcare robotics sector.
At PAL Robotics we partner in a number of collaborative projects in healthcare and ambient assisted living, as well as projects covering areas such as agri-food, smart cities, factories of the future, Artificial Intelligence, and Deep Learning. We are always on the look for new collaborations, to find out more about EU-funded projects or request information, visit PAL Robotics’ collaborative projects webpage and don’t hesitate to get in touch.
Provided by PAL Robotics