Reinforcement Learning for Haptic Guidance

​

Haptic interactions are perceived differently by different people, and guidance from haptic devices can result in different motions. Haptic device designers must take this uncertainty into account and use data driven models of users' responses to haptic cues. This is especially true for ungrounded haptic devices, which cannot provide a net force or torque on a user to prescribe a particular motion. Using my ungrounded CMG device, I recorded user responses to haptic cues in different directions throughout the workspace. I used Gaussian Processes to generate predictive models, including uncertainty, for how the user would move in response to different cues. Then, I used Gaussian Process Dynamic Programming to optimize the haptic guidance provided to the user based on the model. Reinforcement Learning techniques like these are rarely applied in haptic interactions, but could lead to improved interactions, haptic cues that adapt to users' behavior, or user-specific haptic cues.