IdentiToy and Digital Avatars research

One of the areas that we have been looking at in our research is digital puppetry. Many of our workshops involve people creating objects that reflect their identities in some way. People's toy hack creations and cardboard robot outfits often match their personalities in interesting ways. We are looking at ways which people can further inhabit these characters, and would like to use them as the starting point for a series of digital explorations of identity under the name IdentiToy. We have been looking at available tools and techniques to digitise people's creations, and also ways that they can control and interact with these avatars. This post summarises some of the possibilities we have discovered through our research.

Kinect Puppeteering

The first thing that sprung to mind when thinking of ways to digitally puppeteer things was Microsoft's Kinect sensor, a camera that can track your outline, and map the environment around you. This piece of hardware is really extremely cheap for what it does (most other depth cameras are in the thousands, or at least hundreds, whereas you can pick up a Kinect sensor for under £50 on ebay nowadays) The Kinect is able to track not only how far away different parts of the image are, but also does some rather clever processing to work out where all the different parts of your body are.

A "depth image" from the Kinect
In this image, instead of clolour, each pixel represents how far away from
the camera that part of the image is. The further away it is, the darker it is.

Here we see the kinect identifying the person's outline, and the
position of their joints in 3D space. Clever stuff!

Unfortunately, Microsoft did not have the foresight to make the Kinect compatible with windows from the outset, so  getting it to talk to your computer is not exactly simple. After it's release, Microsoft saw all the cool things that hackers were making with it, and jumped on the bandwagon. They have since released an official sensor and SDK (software development kit) for use with windows, although it is nearly £200 (more than twice the price of the original.) As we already had a Kinect for Xbox, we decided to try and get that working instead, and wait for the official windows release of the Kinect 2.0 rather than spend money on something that will be obsolete in a few months.

Installing all the right drivers and getting them to play nicely was quite an ordeal, and involved a lot of frustrating trial and error, uninstalling everything, and resintalling slightly different versions in a slightly different order a few times over!

There are libraries available to use the kinect sensor data in programming languages like OpenFrameworks and Processing, but after a little experimentation we quickly realised that this would be a pretty difficult and time consuming approach to get anything particularly impressive done in a reasonable amount of time. After doing a lot of research into potential software, I came up with the following list of things to investigate further :

- OSCeleton
- Kinectar
- NI mate
- Zigfu

IdentiToy