A fundamental source of information that we exploit in social interactions comes from the way we move. Our movements are characterized by certain regularities that make the motion of living beings – or biological motion – unique, both in terms of how it is performed and how it is perceived. Humans are particularly good at perceiving biological movements and at sensing their properties, and even newborns exhibit the ability to recognize biological motion in the scene. With the aim of deepening the understanding of the complex mechanisms at the basis of human motion, we started to implement a form of visual processing that allowed iCub to detect biological motion. The developed model is based on low-level visual motion features like average speed and curvature computed from the optic flow of the scene. These geometrical features are inspired by the two-third power law, a motor control law proper of biological motion. The biological motion detection algorithm proved to be successful in face of the heterogeneity of different actions to be judged.
The mechanism supporting this ability scales well also to more complex tasks, as coordination in simple rhythmic exercise or simple action understanding – suggesting that it could represent a reasonable building block for more complex social skills.

Here you can find our dataset about actions understanding across views:

https://sites.google.com/view/themocaproject/welcome

Ref.

• Vignolo, A., Rea, F., Noceti, N., Sciutti, A., Odone, F., & Sandini, G. (2016, November). Biological movement detector enhances the attentive skills of humanoid robot iCub. In 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids) (pp. 338-344). IEEE.
• Vignolo, A., Noceti, N., Rea, F., Sciutti, A., Odone, F., & Sandini, G. (2017). Detecting biological motion for human–robot interaction: A link between perception and action. Frontiers in Robotics and AI, 4, 14.

In typical human-human interaction, a considerable amount of information is exchanged through non-verbal signals. The outstanding ability to detect implicit cues in everyday gestures makes the interaction between humans smooth and seamless. Our aim is to provide robots with the same capability: enabling them to detect implicit signals in human actions but also to enrich their movements with the same cues to make them more natural and readable by the human partners. We focused for example on the automatic inferring of object properties like weight and fragility observing human kinematics during the manipulation.

Ref.

• Sciutti, A., Patane, L., Nori, F., & Sandini, G. (2014). Understanding object weight from human and humanoid lifting actions. IEEE Transactions on Autonomous Mental Development, 6(2), 80-92.
• Lastrico, L., Carfì, A., Vignolo, A., Sciutti, A., Mastrogiovanni, F., & Rea, F. (2021). Careful with That! Observation of Human Movements to Estimate Objects Properties. In Human-Friendly Robotics 2020: 13th International Workshop (pp. 127-141). Springer International Publishing.

Social interactions often entail complex and dynamic situations that follow non-explicit, unwritten rules. In this particular projects, we intend to study first if robots are capable of evoking Comfortability in humans during a live interaction. We defined Comfortability as disapproving of (or approving of) the situation that arises as a result of an interaction which influences one’s own desire of maintaining or withdrawing from it. Second, which are the non-verbal cues associated with this internal state (i.e., facial expressions, body movements, and some physiological features). The final aim is to endow the robot with the ability of automatically recognize such state and take decisions accordingly.

Ref.

• Redondo, M. E. L., Vignolo, A., Niewiadomski, R., Rea, F., & Sciutti, A. (2020, November). Can Robots Elicit Different Comfortability Levels?. In International Conference on Social Robotics (pp. 664-675). Springer, Cham.
• Redondo, M. E. L., Vignolo, A., Niewiadomski, R., Rea, F., & Sciutti, A. (2020, November). Can Robots Elicit Different Comfortability Levels?. In International Conference on Social Robotics (pp. 664-675). Springer, Cham.
   

Eyes are very effective source to exchange information during interaction and get intuitions about the partner’s needs and intensions. The ability of detecting mutual gaze with the partner has been shown to be enough in simple turn taking task, allowing iCub to guess correctly when to intervene during the interaction. An even more precise estimate of gaze direction can be effective to recognize which, among different possible objects in front of the person, is the one desired. 
Gaze is also an important cue to reveal what we want to hide. Indeed, if the robot can access subtler proprieties of gaze behaviour like pupillometry, it can even understand whether the interacting partner is lying or not. In a recent study, we demonstrated that the cognitive load associated to lying can be evaluated by monitoring pupil dilation alone and so let iCub correctly guess when humans were lying in a game-like scenario, by relying on a simple heuristic approach.
 

Ref.

• Palinko, O., Rea, F., Sandini, G., & Sciutti, A. (2016, October). Robot reading human gaze: Why eye tracking is better than head tracking for human-robot collaboration. In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 5048-5054). IEEE.
• Dario Pasquali, Jonas Gonzalez-Billandon, Francesco Rea, Giulio Sandini, and Alessandra Sciutti. 2021. Magic iCub: A Humanoid Robot Autonomously Catching your Lies in a Card Game. In Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (HRI '21). Association for Computing Machinery, New York, NY, USA, 293–302. DOI:https://doi.org/10.1145/3434073.3444682
• Gonzalez-Billandon, J., Aroyo, A. M., Tonelli, A., Pasquali, D., Sciutti, A., Gori, M., ... & Rea, F. (2019). Can a robot catch you lying? a machine learning system to detect lies during interactions. Frontiers in Robotics and AI, 6, 64.

Current state-of-the-art models for automatic facial expression recognition (FER) are based on deep neural networks that are effective but rather expensive to train. This characteristic hinders such models of been used as a general affect recognition.
To address this problem, we formalized the FaceChannel, a light-weight neural network that has much fewer parameters than common deep neural networks.  To evaluate our model, we performed a series of experiments on different benchmark datasets and demonstrate how the FaceChannel achieves a comparable, if not better, performance to the current state-of-the-art in FER.
Moreover, we applied the model to analyze affective behavior in different HRI scenarios, like children-robot interaction or motor learning with robotic assistance.
To analyze the affective response of people interacting with the humanoid robot iCub, we leveraged also an open-source library like OpenFace which relies on the analysis of facial Action Units (AUs), as we did in the development of the affective cognitive architecture.

Ref.

• Barros, P., Churamani, N., & Sciutti, A. (2020). The FaceChannel: A Fast and Furious Deep Neural Network for Facial Expression Recognition. SN Computer Science, 1(6), 1-10.
• Tanevska, A., Rea, F., Sandini, G., & Sciutti, A. (2018, March). Designing an affective cognitive architecture for human-humanoid interaction. In Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction (pp. 253-254).

Interacting with others requires the ability to evaluate their attitudes based on how actions are performed. Even a simple everyday act as handing over an object acquires a different meaning if it is performed gently or harshly. Body motion is important to convey why and how that particular action is carried out, revealing the intention and affective state of the counterpart.
Concerning human-robot interaction, it is important to be aware of the impact that robot motion features might have in the partner’s interpretation of their actions. These different ways of communicating have been called vitality forms and play a crucial role in social relations. It is important in the scope of human-robot interaction to communicate intuitively the attitude of the other and enable a fast and immediate adaptation. The same action, in fact, acquires a different meaning, also in terms of urgency or importance, when associated with different vitality forms. Robots might need to express different attitudes in particular contexts such as elderly care or law enforcement. Our goal is to investigate the expression of an emotional status through movement and to endow the iCub humanoid robot with the capacity to communicate intuitively positive and negative attitudes in its own actions.

Ref.

• Vannucci, F., Sciutti, A., Lehman, H., Sandini, G., Nagai, Y., & Rea, F. (2019). Cultural differences in speed adaptation in human-robot interaction tasks. Paladyn, Journal of Behavioral Robotics, 10(1), 256-266.

• Vannucci, F., Di Cesare, G., Rea, F., Sandini, G., & Sciutti, A. (2018, November). A robot with style: can robotic attitudes influence human actions?. In 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids) (pp. 1-6). IEEE.

The modulation of the motor behaviour of the robot might have an impact not only on the immediate response of the human partner but also on its willingness to maintain the interaction over time. In particular, the way an action is performed can express the amount of effort a person is putting in performing the task. For instance, in a teaching context, making slower and larger movements and segmenting more the actions can be a strategy to help scaffolding the learning. In turn, perceiving a larger effort from the partner could lead the collaborator to persist longer and to invest greater effort in the joint action. In other words, certain properties of the way an agent moves can lead to the establishment of a sense of commitment. 
In our research, after having proved that a robot can trigger a sense of commitment in a game with a human partner, we investigated the impact of different degrees of commitment expressed by the robot kinematics on participants’ motor behavior and their performance while performing a joint task.
The results showed that humans are particularly sensitive to changes in the degrees of effort the robot seems to put in the task and respond by changing the amount of effort they put in the task. Therefore, the way an action is performed can have important effects on the establishment of a rapport between the agents.

Ref.

• Vignolo, A., Sciutti, A., & Michael, J. (2020, November). Using Robot Adaptivity to Support Learning in Child-Robot Interaction. In International Conference on Social Robotics (pp. 428-439). Springer, Cham.
• Székely, M., Powell, H., Vannucci, F., Rea, F., Sciutti, A., & Michael, J. (2019). The perception of a robot partner’s effort elicits a sense of commitment to human-robot interaction. Interaction Studies, 20(2), 2
• Vignolo, A., Powell, H., McEllin, L., Rea, F., Sciutti, A., & Michael, J. (2019, October). An adaptive robot teacher boosts a human partner’s learning performance in joint action. In 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) (pp. 1-7). IEEE.34-255.