Researcher(s)
- Khadija Mohammadi, Computer Science, University of Delaware
Faculty Mentor(s)
- Maryam Vaziri-Pashkam, Psychological and Brain Science, University of Delaware
Abstract
How We Grasp the World – Literally
Our hands play an important role in our daily lives—picking up a glass, opening a door, or texting a friend all require interacting with objects using our hands. Not only do we execute these movements ourselves, but we also watch others perform them and can predict their goals from their movements.
The primary goal of this research is to investigate if hand grasp movements convey information about the intentions, and how human and AI algorithms can predict these intentions from the movements. To achieve this, we conducted a series of experiments involving naturalistic grasping behaviors. Sixteen motion sensors were placed on participants’ hands to record their movements in three-dimensional space (x, y, z) as they grasped 58 distinct 3D-printed objects. We are currently analyzing the collected data to parse out the beginning and end of grasp movements to be able to further allow for investigations of AI and human compatibility.
In the future we plan to use these data to assess human-AI alignment in observing grasp movement. In behavioral experiments, participants will view videos of these grasp movements—displaying only the hand, without the object—and we will ask them to identify the object being grasped. This allows us to examine which aspects of the grasp are most informative in conveying the goal of the action. We will also assess AI performance in similar tasks by inputting these same videos into a machine learning model to assess whether AI can reliably infer the object being grasped based solely on the hand’s movements, further elucidating the informational content of human grasp kinematics.
This research could have implications for physical rehab after injury and for designing better AI algorithms for robots that need to mimic human gestures and can interact with humans in daily life.