Chris Harrison of Carnegie Mellon University and Microsoft researchers have been collaborating on using the human body as an input surface. They call this approach “OmniTouch”. Chris Harrison’s earlier project called ‘Skinput’ had similar goals and had a more interesting name but OmniTouch has some interesting advances over Skinput. Read on.
If this looks familiar to you, you may be thinking of Pranav Mistry’s SixthSense project at MIT Medialab a while back. OmniTouch has some interesting advances over SixthSense, though.
Mistry’s SixthSense also projected an image on ones body but used fiducials (colored marks) on the tips of the users fingers. A wearable camera tracked the fiducials and a computer deduced if a finger was touching a projected input point.
Harrison’s Skinput used a picoprojector to display an image on a body surface like a hand or a forearm. Bioacoustic techniques using a specially-designed armband detected taps on the skin and, with a bit of signal processing, could determine where on the skin the tap occurred. Skinput needed an armband which could be covered by clothing so it was a bit less obtrusive than the SixthSense fingertip markers.
OmniTouch enables a wide variety of surfaces to be input devices, not just a body surface. It uses a wearable projector and camera like SixthSense, but doesn’t require SixthSense’s markers on ones fingertips. OmniTouch uses a depth sensing camera, similar to Kinnect, but capable of shorter focus distances which increases flexibility. So, for example, one could use a wall or a pad of paper as an interactive surface. Depth sensing allows touch as well as hovering gestures.
The concept is very interesting. Wearing a projector and a depth sensing camera is clunky, but the concept is interesting…. interactive pixels everywhere in front of you. It’s just a proof of concept… now we need smart companies to make it small, stylish, and usable. Check out the video below from Chris Harrison’s website.
Here are a couple of links if you want more detail: http://chrisharrison.net/projects/skinput/SkinputHarrison.pdf and http://chrisharrison.net/projects/omnitouch/omnitouch.pdf
Yesterday, researchers at the University of Illinois-Urbana announced skin-like patches (here) that have electronic circuits on them and can be put directly on the surface of the skin. Here is an excerpt (emphasis added by me):
“The circuit bends, wrinkles, and stretches with the mechanical properties of skin. The researchers demonstrated their concept through a diverse array of electronic components mounted on a thin, rubbery substrate, including sensors, LEDs, transistors, radio frequency capacitors, wireless antennas, and conductive coils and solar cells for power.”
The announcement goes on to say:
“The technology can connect you to the physical world and the cyberworld in a very natural way that feels very comfortable.”
They emphasize medical applications, but I think this has big implications on the broader digital light/pixels everywhere future. For example:
- we’ll be able to put pixels all over our bodies. They’ll be almost part of us, not just projected onto us (see http://www.pixelizedlight.com/2011/07/pixels-everywhere-i-mean-all-over/) . Why? To reflect our mood, perhaps, or maybe as art or most likely just for fun.
- if this technology truly wrinkles, stretches and bends as well as the researchers claim, why restrict it to skin patches? Maybe this is a breakthrough that enables a big leap forward in highly flexible materials for
- fashion … think what Moritz Waldemeyer or Vega Wang could do with this (see: http://www.pixelizedlight.com/2011/07/wearable-pixels/)
- interior design… imagine LED enhanced curtains, furniture fabrics, and wall coverings …potentially taking what Kvadrat and Philips have done to a whole new level (see http://www.pixelizedlight.com/2011/07/big-beautiful-blurry-pixelized-walls/)
- architecture … the mind boggles at what architects could potentially do with flexible building material that illuminates and senses the environment.
Fun things to think about on a Friday afternoon! Maybe the pieces are falling into place for ‘pixels everywhere’. The world is going to be a canvas and you and I will be part of that canvas.
Oh no….are we approaching the digital light singularity … the pixelarity?
A while ago, I wrote about Sifteo’s game tiles. Pretty cool, but seemingly single-purpose.
WIMM Labs is developing a tiny tile-like device, but they’re treating it like a platform. In other words, it can be programmed for any number of uses and integrated into other products. The specs seem pretty impressive: 32GB storage, touchscreen, Wi-Fi, bluetooth, accelerometer, Android OS. Apparently it also includes an expansion connector, too. Small — 32mm x 36mm x 12.5mm. I guess they’re battery operated but I couldn’t find power consumption or battery life specs on their web page.
No mention of the availability or price on their website as of this writing. But if it’s low enough in cost and with the claimed connectivity, I can see these little pixelized devices going pretty much everywhere. In singletons. In groups. In the hands of creative developers, some really interesting uses are going to occur. Sifteo, now WIMM. I think we’re going to see a lot more of these kinds of devices.
Hmmm…if they could just shrink the bezel a bit more….
Yesterday, I posed the question “Why shouldn’t all light be possible to modulate?” and promised to explore where that might take us. Wayfinding is one example.
Wayfinding using digital displays is becoming pretty commonplace. But that just makes signs smarter — we get the information we need and then we move on. Wayfinding using the GPS in our smartphones isn’t much different. It’s handier because we have the phone with us and we can refer to it whenever we want. But just like the digital sign example, we get the information we need and we move on.
What if the lights around us could actually lead us where we want to go?
After all, there are a lot of light sources. Consider this jogger running at night. If the street lights could be modulated, then maybe the lights could brighten the path where she is running and reduce brightness everywhere else. Maybe the lights could also help her find her way. Street lights today can’t do this, but new generations of lights perhaps could if we thought about light in new ways.
This would make our streets safer. It would reduce light pollution.
It will require big advances in lowering the cost of modulating, steering, and controlling light. This sort of wayfinding wouldn’t need very high resolution, so that would help. Even so, the new generation of street lights would cost more than today but I think the benefits could be worth it.
It will also require advances in the energy efficiency of modulated light… but some interesting work in novel MEMS devices, ultra-miniature motors and (perhaps) holographic projection might make this possible.
(thanks to Kristina Foster for the animation)