Arduino & Servos

Another thing that was looked at was the possibility of controlling servo motors from within the TouchDesigner graphical programming environment. I had previously looked at this some time ago and come up with a slightly hacky and complicated way to do this. During our R&D I came up with a greatly simplified and more reliable protocol for talking to the Arduino, and added a graphical control interface for controlling a servo motor from your computer :  

I then hooked it up to an existing system I had made for live looping of MIDI parameters :

Robotic CommuniToy

Scratch

MIT's Scratch programming software is a fantastic tool, and has become hugely popular over the last two years. It is now taught widely as part of key stage 1 and 2 computing in schools.

in August 2013, we collaborated with Neil Ford from  Code Club to run a pilot week of activities combining coding with Exploring Senses toy hacking, robot relays, and electronics building activities called "Code for Kids" (1.0)  blogs - (day 1, day 2, day 3 day 4, day 5)

During our R&D period, we have run two more events building on this :

In February half term, we ran a 3 day series of workshops with young people at Brighton Youth center called, yep, you guessed it.... "Code For Kids 2.0" Thanks to Sussex Community Foundation these workshops were free for all to attend. Here is a blog post documenting the project

Also, throughout June and July, we provided a series of further creative coding workshops at Brighton Youth Center's Junior Club, documented in a blog post here

My favourite game that anybody produced was this rather surreal one :

CommuniToy Animations

After Effects Puppet Tool

Creating and animating digital characters can be a pretty fiddly and involved process, and there are a ton of different ways to do it. For quickly and easily animating 2D characters, by a very long way, the best tool I have discovered is After Effects' puppet tool. It allows you to take a single flat image of a character, and manipulate it to animate in incredibly natural looking ways with a minimum amount of fiddliness.




In November 2013 Exploring Senses artist Louis d'Aboville created an interactive installation for Brighton Council's Shine on London Road project. The puppet tool was used extensively to create over 70 animations from photographs of toy hacks.

Project video :

Toy Hacking from Jeb Hardwick on Vimeo.

Needless to say after this I was reasonably familiar with this tool! Exploring Senses have used this for new activities a few times during this R&D period.

The first was in April, when we did a workshop at Lighthouse as part of the SprungDigi project, (ES blog post here) The project involved working with learning disabled creatives to create toy hacks, learn how to animate them, and put them in a bespoke version of the software used for the Shine On London Road project.

The second time we used it was in the final workshop for our ICT Art connect commission. #LINK# It was used to quickly animate on the spot toys that were being made in the workshop, and display them throughout the workshop as part of Brighton Urban Artfest, an event for Brighton Fringe festival.

It was also used to create animations for the Sensory Jumblies project #LINK#

IdentiToy

Puppeteering with Zigfu

I started off my research looking at potential technologies for puppeteering and interacting with digital versions of toy hacks in realtime, with a view to developing these into interactive exhibits.

The most promising looking solution for puppeteering, and the one we investigated first is something called Zigfu. Zigfu's ZDK development kit allows you to get the joint positions from a Microsoft Kinect sensor into other software, including a game engine called Unity3D. These can then be linked to the joints of a rigged 3D model, so that you get one to one puppeteering of the model's limbs. 

After much faffing around with Kinect drivers, I was able to get the example project running. I also did some research on Unity3D, and basic 3D modelling software so we could begin creating custom content to modify the project with.

I used Google sketchup to create some robot parts, and exported them as .obj files (a standard 3D model filetype) These could then be imported to Unity, positioned and linked to the existing model. I also customised the model's texture to have a CommuniToy logo, and attached some virtual lights to the model's hands.

This is really rather fun, and very entertaining to watch, although as you can see in the video, the tracking is not particularly smooth or accurate (especially when you turn to the side or turn around) The Kinect also requires you to do a "calibration pose" (stand like a cactus) before it will pick you up, and this can sometimes take a rather long time, with no feedback as to what you are doing wrong. Once you are tracked it also often loses track of your limbs. For these reasons, the technology was not really reliable enough at the time to be suitable for an installation, and we did not look into this much further.

In July 2014, Microsoft released the new version of their Kinect sensor for windows. It is massively improved in almost every way possible, and is extremely promising for developing puppeteering based interactions. Unfortunately, as with all new technologies initial support is extremely patchy, and it takes a while for it to be integrated into existing software, or for new products to be built around it. At the time of writing (August 2014) there is no support for the new Kinect in Zigfu, NImate, Faceshift, or any of the accessible solutions for realtime puppeteering, although this is definitely something we will be keeping a close eye on.

IdentiToy

Leap Motion Hooks Up to Oculus for Controller-Free Virtual Reality: 28th August 2014

Great news indeed!!: )

http://recode.net/2014/08/28/leap-motion-hooks-up-to-oculus-for-controller-free-virtual-reality/

This development is very exciting and possibly a step towards the dream multiplayer Toy Hack Disco/Rave experience, and game play within Unity environments.

We will be receiving the V2 Oculus rift soon.

Just ordered 3 x leap motion Oculus Rift mounts (2 for ES to use with our Rift DK1, and soon to arrive DK2 kit + one spare for use with future collaborators such as Blockbuilders, as the mount and cable is only €14 each: )

Arts Council

IdentiToy

CommuniToy Animations

Creating a 3D "aura" in Maya

Aurasma is an app for creating augmented reality experiences. I have been finding out how to make custom 3D content to put in the app, and have made my first ever video tutorial on using Maya to create, texture, and animate a model, and get it into Aurasma successfully.

IdentiToy

3D research

What with us getting a 3D printer and an Oculus rift and all, I figured now is as good a time as any to do a bit of research into the mysterious (to me anyway!) world of 3D modelling.

For the last few weeks, I have been looking into options for 3D scanning and 3D modelling software, investigating the options and finding out what we need to know to start creating models for printing, importing into Unity 3D experiences for Oculus Rift, and also for use with the Aurasma augmented reality app on iOS. This post is a quick summary of some of the things we have done and found out.



Quickly Creating 3D models

As Dave mentioned we have been looking at 123D's impressive suite of apps for creating content on the ipad. The 123D creature app in particular is a fantastic way to quickly and easily create great looking content which can easily be exported out as an .obj file and imported into Unity3D (or any other 3D software for that matter)

This abstract creature was made by one of the youngest learners at the self managed learning college within 2 minutes of seeing the app for the first time. It is definitely quick and easy.

Another good accessible entry point for creating 3D printable models is Doodle3D :

 
Doodle3D, makes 3D printing very easy from Rick Companje on Vimeo.

It is a wifi box that lets you create and send 3D models dirently from your ipad to your 3D printer. We saw Ivan Pope using it at a busy event a couple of months ago and in a situation where you don't have time or the ability to sit down at a computer, and want to give somebody a taste of the power of 3D printing, it looks great.


3D scanning existing objects :


I will go into more depth in another post about this in a seperate post, but I have also been experimenting with Autodesk's 123D catch software, which can automatically create 3D models from a series of photos.

I have been using it to try and capture some toy hacks, with mixed results. You can see all the models that I have created at :

http://www.123dapp.com/Project/Toy-Hacks/2513998


It is very easy to create models, and the colour and detail in the textures on some of them look really great. but the 3D models that it gives you are :

 - very complicated  - (millions of polygons)
 - noisy, bumpy and innacurate - (even more pronounced on small objects like toys)
 - often full of holes - (not good for 3D printing)
 - nearly always come with multiple texture maps  -  (no good for importing them into game engines and apps like Aurasma)

In short it is a lot of work to clean them up and make them anywhere near usable for a 3D printer or a game engine. I spent quite a lot of time feeling rather confused trying to make sense of all the funky 3D lingo (apparently I had to decimate my model, retopologize it and make some new UV maps...)

After many (many, many...!) failed attempts, I finally got a decent version of this smurf done.





The textures that 123D catch spits out are a huge mess though. Getting them to even appear on your model in the right way is a very long winded process, and projecting those textures onto a simplified version of the model is something that I still haven't had a lot of success with.



In the end I had to settle for keeping my sanity, giving up on Zbrush for now, and only having single colour cross dressing VR smurfs in Unity3D... (sometimes life is about compromise) Here they are in the Oculus Rift "Tuscany" demo.

I did also manage to get the Oculus rift working in TouchDesigner, and import some models, although haven't had time to investigate much further than that.

 Anyways, we have the Structure Sensor device on it's way to us extremely soon, and it looks like that will make getting nice clean, well textured models a hell of a lot easier and simpler.

In the next few posts I will go into detail on my research into using the Aurasma app for making Augmented Reality content, and creating animated 3D models to use in it.

IdentiToy