RAAK

Microsoft’s new controller for the Xbox 360, Kinect, sold 2.5 million units in the month of December 2010 and they’re projecting to sell 5 million units by end of 2011. This means, when you’re reading this, chances are very good that you either already own one, or you’re going to own one quite soon.

So, apart from playing supported Xbox 360 games with it, what else can you do with it?

A lot, actually.

How does it work?

The Kinect utilizes three types of sensors.

• RGB Camera
  Just a plain old full color webcam, so you and your environment can be captured the way humans see it.
• Infrared 3D Camera
  This does not show you the world the way humans see it, but it does show you how far every point in space is from the camera, in other words, it provides a 3D representation of you and your environment.
• Microphone Array
  A set of microphones, spatially arranged on the kinect in such a way that it provides both sound source localization and ambient noise reduction. In other words, it can be used to pinpoint sounds players make (spoken commands, hand claps, etc), link it to specific players, and isolate different players’ sounds from each other.

The following video demonstrates exactly the information that can be gained from the two camera sensors by combining the depth information with the 3D information:

What are people doing with it?

If you’ve got even just the tiniest droplet of hacker blood running in your veins, the above must have your brain racing through all the possibilities of this game changing piece of hardware.

Here are some of the coolest hacks I’ve come across:

Ultraseven Suit

This cool kid went and built himself a fully interactive Ultra Seven suit, complete with lasers bouncing off objects. Do watch it right up to the end, to see the amazing exit animation!

Virtual X-Ray

At the time of writing, this one was dominant in a kinect hack Twitter Search. This hack reproduces the scene as seen by the RGB camera, with a round window in the middle of the screen that, if your body is behind it, shows parts of a 3D model of a skeleton, positioned where your own should be.

Flying Quadrocopter

An autonomous flying robot, what else?

Cloaking Device

This guy turns himself invisible by superimposing an image of the background over his body. The result is an amazingly science-fiction-movie-like rendition of the ultimate cloaking technique.

What does the future hold?

The ROS project is working on a very exciting branch of development integrating Kinect with ROS.

To put it plainly, they are controlling robots by having the robot mimic the user’s movements, captured by a Kinect. Pretty wild!

There’s also this demonstration of how the Kinect can be used to capture movement for animation, previously done with very expensive setups of special dotted lycra suits and up to a few hundred cameras:

Given that the Kinect hardware will probably evolve quite a lot over time and given that it will soon have consumer drivers for just about every platform thinkable, this might just become one of the most important, and most game-changing controllers ever devised.

Ready to burn that mouse and keyboard, anyone?