Virtual reality has a long way to go before it feels like reality. The visual part is getting more immersive with the new VR headsets, but things like the sense of touch are missing. Heather Culbertson hopes to change that.
At the University of Southern California Viterbi School of Engineering, Culbertson and her team of graduate students are working on a “haptic” armband that gives you a sense of touch. It’s not using an actuator, which gives you a sense of buzzing like in typical video game controllers. Rather, Culbertson has engineered a prototype arm so that it feels like the sensation of a finger moving along your arm. I tried it out on a recent trip to Culbertson’s lab. The current prototype is a sleeve that wraps around your forearm and has a row of wired small speakers on it. It mimics the sensation of a finger moving along your arm.
“That’s what we focus on in this lab, a way to create hardware and signals that make haptic interactions feel natural, mimicking what you feel in real life, said Culbertson, who is an assistant professor of computer science, aerospace, and mechanical engineering. “With this device, it’s a set of voice coil actuators. These are capable of playing vibration. They’re essentially speakers.”
As Culbertson described it, I felt something like a soft touch moving up along my arm. She said it was a “pleasant sensation on the arm, a soothing, stroking sensation.”
No, we’re not going to start imagining what this could mean for virtual sex. It’s not going in that direction. Rather, the lab is looking to improve social VR, like when you tap someone on the shoulder to reassure the person. She calls it social haptics. The sense of touch can be surprisingly effective at things like decreasing depression or reducing stress.
Culbertson has been working on the tech since January, when she joined USC. Before that, she worked on it with colleagues at Stanford University.
Here’s an edited transcript of our interview.
Above: Dean Takahashi tries out the haptic armband at USC.
VentureBeat: Tell us what you have here.
Heather Culbertson: We’re in the department of computer science and we deal with everything that has to do with the sense of touch. Right now I have two Ph.D students and three master’s students, and one undergrad for the summer. I just started here in January, so we’re still ramping up a little bit. We’re exclusively working on haptics and human-computer interaction in this lab.
We have a couple of devices we’re building around here. We’re going to demo this one here. Specifically, it’s dealing with social touch. We’re trying to create a pleasant sensation on the arm, a soothing, stroking sensation.
If you think of haptic devices up until now, they’ve been very—kind of annoying, almost? Buzzing, not feeling very natural. That’s what we focus on in this lab, a way to create hardware and signals that make haptic interactions feel natural, mimicking what you feel in real life. With this device, it’s a set of voice coil actuators. These are capable of playing vibration. They’re essentially speakers. We can play vibration through them, but we actually use them to display motion. They move up and down by a couple of millimeters. We use this to create that pleasant stroking sensation on the arm.
With this project, we’re coming up with a way to use haptics to convey social touch cues from one person to another. If you think of Skype or other interactions online, that’s limited to video and audio now. But when we’re interacting face to face, you begin with a handshake. If you’re talking to your family you might have a pat on the back, a sympathetic touch on the arm. We want to convey those very natural, very common social touch interactions. This device is a way to create those stroking sensations, because those are very common in creating and displaying sympathy or sadness or love.
These are going to move up and down, one at a time. We call this a sequential normal indentation. We’re creating the illusion of motion. Because creating a device that has a long lateral motion is very difficult. Instead we create that illusion of motion by indenting on your arm one at a time, and then we control how we actuate them, how long a delay we have between them. You can see we have a wave that passes down. When you wear that on your arm it’ll feel like something stroking your arm.
We could definitely use this in nursing homes or the homes of elderly parents who want to create a calming sensation, or just a social presence as part of communication, that social touch that they’re not receiving. Research has shown that not receiving these social touch cues, not receiving touch at all, can lead to a sense of isolation amongst individuals. We’re planning on working with elderly individuals, people with anxiety disorders, and others.
Above: The haptic armband prototype has six little speakers that simulate touch.
VentureBeat: Is this going back and forth, or just one direction?
Culbertson: We can change the direction. For this demo, it just goes in one direction – it only goes one way at a time – but we can switch that direction and it’ll go back the other way. By changing the actuation parameters, we can make it go faster. We originally created this device for that stroking sensation, but we can also, using the same kinds of motions—we have pulse width, which is how long it takes to indent, and then the delay is the percentage of time between when one starts indenting and then the next. We can make it go faster and provide something that isn’t as pleasant, but we can provide, say, directional cues. With a really high delay we can make it very discrete, which some people feel as something crawling up your arm.
We originally created this device for social interactions, but we can change around the parameters and use it in VR scenarios to create different sensations. You could wear this in a game and feel a bug crawling up your arm. People have said this one’s pretty creepy.
VentureBeat: The number of buttons, the number of actuators, is that one thing that will determine how much you can control the feeling?
Culbertson: Sure. If we had 200 actuators and they were smaller, we could definitely create an entire sleeve of this to provide multidirectional cues. We’re building a 2D system, so instead of having a single row, it has two rows. We could provide different sensations in addition to stroke. We could do a grasp or additional gestures.
We just bought off-the-shelf components for the actuators, because we’re computer scientists, but if we worked with mechanical engineers and materials scientists to design and miniaturize the actuators, we could pack these a lot tighter and make them a lot stronger. We could build an armband, a vest. I’m sure you can tell that the normal force is a lot more natural than what you get from vibration. It’s a lot more pleasant.
VentureBeat: Why haven’t some of the different haptics companies thought of this earlier? The haptics devices I’ve tried so far have mostly been these buzzing things. Nobody’s really ventured into this territory yet?
Above: Heather Culbertson and her grad students in the haptics lab.
Culbertson: Not with normal, no. Most commercial devices have typically used the buzzing. One startup I know, Tactical Haptics, is using skin stretch, which came out of the university lab as well.
VentureBeat: That’s the Utah one, right?
Culbertson: Right, Utah, Will Provancher. A lot has to do with availability of actuators and miniaturizing the actuators. These are a bit big right now, but it’s not that heavy. It’s looking at what you can do with the actuators. These are actually speakers, as I said. They’re not designed for haptics at all. They’re just exciter speakers, speakers without cones.
VentureBeat: I remember one company more recently was using speakers to blow air at you, to make you feel something without having something attached to you. It’s limited, but it’s something different.
Culbertson: There’s a lot more going on in haptics research than what you typically see in commercially available haptics. What’s really novel about this device is the use of a haptic illusion, how we can create the sensation of long lateral motion without a very mechanically complex device. We’re going to continue exploring that to see how we can change the signals, how we can play around with novel control of the devices, to create the illusion that something is moving along your arm, something is happening when it actually isn’t.
VentureBeat: Do you eventually want to get to those Ready Player One suits?
Culbertson: Yeah, that’s definitely the dream, the goal of all haptics. [laughs] It comes down to how much is needed to be really immersive, how much of the body needs to be covered. Do you need the full body? Are the hands enough, or the arms? Do you need arms and a vest? Now that we’re getting into more wearable devices, rather than grounded, table-based haptic devices, these are questions we can start to answer.
Above: A haptic suit enables Wade to feel touch in the Ready Player One trailer.
VentureBeat: How long a process do you think you have ahead of you, to get something like a product defined? Do you still see a lot of years of research?
Culbertson: We’re pretty close in terms of ideas for how we could integrate this with currently available games or movies, entertainment. In order to get this into a commercially viable system, I think we’d have to partner with more mechanical engineers or materials scientists to miniaturize the actuators, rather than going with what we have available off the shelf.
VentureBeat: Something like gloves, are they the first step? Would that be useful by itself?
Culbertson: Gloves are really challenging because your hands are so sensitive. Fooling your hands is a challenging problem. You have miniaturization problem with getting the actuators in a glove, and then if you want to display something to your hand that delivers as many signals as your hand is capable of sensing—you have a lot more receptors in your hand than you do in your arm. If you’re working in a VR scenario, you aren’t getting a lot of signals in your arm and other parts of your body, so that’s where we’re starting, more in armbands or vests.
I can show you our other two devices. They’re not quite demo-able, but we have the hardware. None of this is proprietary, by the way, nothing that we’re showing. This is a project I had an undergrad working on this summer. He was a physics undergrad from MIT, on an exchange program. It’s a thermal haptic display. We were trying to display cool and hot, very different temperatures, in addition to pressure. We have a bunch of different pumps. This was in an armband device. He would have these different actuators or pouches in here, and we would use these pumps to pump water from a hot reservoir tank and a cool reservoir tank. We’d be able to directly change the temperature of what you were feeling on the arm with the water we were sending.
What are typically used in thermal displays for haptics are these Peltier pumps. They’re these little ceramic squares controlled with voltage. You send the voltage and one side gets hot while the other gets cold. But they’re pretty slow to actuate. By using liquid, we can instantly change the temperature from hot to cold, depending on which reservoir we send from. We can also mix it to get arbitrarily warm temperatures. In addition, using the liquid gives us pressure as well. We have a couple of immersion pumps in here, and these are solenoid valves that control which pouch the water is going to, whether it’s hot or cold.
Above: This contraption lets you feel sensations of hot or cold.
VentureBeat: What would you use this for?
Culbertson: Part of it is, if you want a more immersive scenario in VR—say you’re walking through a forest, and then all of a sudden you come out of the forest and it’s hotter in the sunshine. You get warm all of a sudden. Or you could reach out toward a virtual stove and it feels instantly hot.
We’re also thinking about therapy devices. There’s a thing called deep pressure therapy, where they have weighted vests or swaddling clothes for people with anxiety. We’re working on a grant around whether we can use these types of devices to help those individuals. We want to look at whether temperature can affect that as well, whether receiving warm pressure cues is better than cold pressure cues. You could more closely mimic the feeling of a hug with warm rather than cool. We want to look at therapeutic systems as well as VR systems.
This last one we have over here, we’re working with air. We’re trying to create a more lightweight version of these wearable systems using pneumatic pouches. We have a compressed air system here, and we can pump air directly into the pouches. That we can provide pressure cues to the arm. This one’s more in the beginning stages of the project, but I wanted to give you an idea of the range of things we’re working on in the lab.
VentureBeat: It gives you a lot more appreciation for what the human body can sense.
Culbertson: The sense of touch is really, really complex. We have a lot of things going on, a lot of sensations you need to mimic when you want to build something that feels immersive and natural.
