The Future is Here: Augmented Reality Storybooks

ar_storybookDisney has always been on the forefront of technological innovation whenever and wherever their animation is concerned. Augmented reality has been a part of their operations for quite some time, usually in the form of displays put on at Epcot Center or their Haunted Mansion. But now, they are bringing their efforts in AR to the kind of standard storybook that you would read to your children before bedtime.

Thanks to innovations provided by Nintendo DS, the PSP, tablets and smartphones, books have become alive and interactive in ways that were simply not possible ten or twenty years ago. However, one cannot deny that ebooks simply do not have the same kind of old world charm and magic that paperbacks do. Call it nostalgic appeal or tradition, but reading to a child from a bounded tome just seems somehow more meaningful to most people.

disneyhideout-640x353And that’s where Disney’s HideOut project comes into play, a mobile projector is used to create an augmented reality storybook. How it works is simple enough, and in a way, involves merging the best of electronic and paper media. Within the book, certain parts will be printed using special infrared-absorbing ink, so that sentences and images can be tracked.

The mobile projector, in turn, uses a built-in camera to sense the ink, then projects digital images onto the page’s surface that are animated to interact with the markers. In this way, it knows to show certain images when parts of the book call for them to be displayed, and can turn normal pictures into 3D animated segments.

disney_argameAnd storybooks aren’t the only application being investigated by Disney. In addition, they have been experimenting with game concepts, where a user would moves a mobile projector around a board, causing a character to avoid enemies. In another scenario, a characters projected onto a surface interacts with tangible objects placed around them. This would not be entertaining to a child, but could be educational as well.

The applications also extend to the world of work, as the demo below shows. in this case, HideOut projects a file system onto the top of a desk, allowing the user to choose folders by aiming the projector, not unlike how a person selects channels or options using a Wii remote by aiming it at a sensor bar. And the technology could even be used on smartphones and mobile devices, allowing people the ability to interact with their phone, Facetime, or Skype on larger surfaces.

disneyhideoutAnd of course, Disney is not the only company developing this kind of AR interactive technology, nor are they the first. Products like ColAR, an app that brings your coloring book images to life, and Eye of Judgment, an early PS3 game that accessed CCG cards and animated the characters on-screen, are already on the market. And while there does not appear to be a release date for Disney’s HideOut device just yet, its likely to be making the rounds within a few years tops.

For anyone familiar with the world of Augmented Reality and computing, this is likely to call to mind what Pranav Mistry demonstrated with his Sixth Sense technology, something which is being adopted by numerous developers for mobile computing. Since he first unveiled his concept back in 2009, the technology has been improving and the potential for commercial applications has been keeping pace.

In just a few years time, every storybook is likely to come equipped with its own projector. And I wouldn’t be surprised if it quickly becomes the norm to see people out on the streets interacting with images and worlds that only they can see. And those of us who are old enough will think back to a time when only crazy people did this!

In the meantime, check out this demo of the Disney’s HideOut device in action:


Source: extremetech.com

Omnitouch Projection Touch Screens

OmnitouchOne of the most interesting areas in which computing has been improving lately has been in the realm of interfacing. Concepts like the XBOX Kinect gaming console have not occurred in a vacuum; in fact, they are part of a larger research trend that wants to make projection and scanning the way of the future. After all, why rely on a monitor or a console when you can project images onto any surface and use them like a touchscreen?

That’s the idea behind the OmniTouch technology. Using the same technology from the Kinect gaming system, a research team made of developers from Microsoft and Carnegie-Mellon University revealed back in October that they’ve come up with a system that can turn virtually any surface into a touchscreen. By combining a miniature camera and a portable computer, the user can write documents, check email and carry out their daily computational tasks simply by finding a surface.

Chris Harrison, a postgraduate researcher at Carnegie Mellon’s Human-Computer Interaction Institute in Pittsburgh and a former intern at Microsoft Research and co-inventor of the device, describes the process as follows:

“OmniTouch works by bringing together a miniature projector and an infrared depth camera, similar to the kind used in Microsoft’s Kinect game console, to create a shoulder-worn system designed to interface with mobile devices such as smart phones. Instead of relying on screens, buttons, or keys, the system monitors the user’s environment for any available surfaces and projects an interactive display onto one or more of them.

“OmniTouch does this automatically, using the depth information provided by the camera to build a 3-D model of the environment. The camera acquires depth information about the scene by emitting a patterned beam of infrared light and using the reflections to calculate where surfaces are in the room. This eliminates the need for external calibration markers. The system rebuilds the model dynamically as the user or the surface moves—for example, the position of a hand or the angle or orientation of a book—so the size, shape, and position of these projections match those of the improvised display surfaces.”

Hrvoje Benko, another researcher who developed the device, admits that the current shoulder-mounted model is impractical. However, all the components involved in its creation are off-the-shelf electronics which are being miniaturized all the time. “But it’s not where you mount it that counts,” he says. “The core motivation was to push this idea of turning any available surface into an interactive surface… So I don’t think we’re so far from it being made into a pendant or attached to glasses.”

All of this calls to mind the SixthSense concept invented by Pranav Mistry many years ago. Presenting at TedTalks, he demonstrated how a wearable camera, finger mounted devices, and a portable computer, a person could use just about any surface to interface with their computer and even be able to transfer documents and programs to stationary computers.

While he may have beaten the IBM research team to the punch, this represents a step forward for portable computing and touch-technology since it shows that the results can be duplicated and made commercially viable.

Click on the video below to watch the OmniTouch device in action, and check out Pranav Mistry’s presentation to TedTalks at the bottom. Both enlightening and worthwhile video segments, trust me on that!