Over the past year I have been working for Seconde Nature designing a public multi-touch platform via my Atelier Hypermédia in Aix-en-Provence. It’s a fairly ambitious project and involves many partners and most importantly, a whole bunch of students and researchers from five different schools/departments exploring interactivity from the perspectives of art, design, architecture or some combination therein. While I’ve been tooling away at the project in some form or another over the past 12 months, the production team was officially formed at the beginning of 2010 and still has about 6 months to go before completing the project with an exhibition planned for early 2011 in Aix-en-Provence and Marseille. In other words, we’re only at the half-way point and anything you see here should be considered highly work-in-progress, and purely experimental/speculative in nature.
That said, we have amassed enough material from the exploratory/workshop phase of the project to create the following document which can be considered a collection of ideas that we found interesting enough to record and group into this 20-minute demo reel. There were actually far more ideas explored than those you will see in this reel but they were unfortunately either lacking decent documentation or were simply too preliminary/unpolished. That said, many if not all of these ideas will need to be completely reworked during the production phase of the project. Only a few will be retained and most of those will be redesigned in collaboration with our content partners.
All of the projects were built within one of the four workshops, with each workshop lasting either 1 or 2 weeks. In total, this reel represents the accumulation of about 6 weeks of direct prototyping.
Today we began our week-long « Mobility » workshop at the Aix-en-Provence Art School. We kicked off the session this afternoon with a conference from Thierry Marcou, director of the Villes 2.0 project at Fing who gave a general overview of the major issues facing the city as it evolves in the current networked era, as well as techno-social experiments, services and creations illustrating these tendencies.
Then, starting tomorrow, we’ll be working in two groups exploring the question of Mobility from an artistic point of view, using either the iPhone/iPodTouch platform via OpenFrameworks, a specifially-designed GPS platformed designed by the Atelier 3D, or some combination of both.
While Memo Akten couldn’t be here this week (cf. Decode), he nevertheless was able to come two weeks ago to the Atelier Hypermédia and help me (actually, the other way around) write a MapKit addon for the ofxiPhone project. This is a pretty cool little addition which allows you to run the standard Apple+Google MapKit library from within OpenFrameworks, and (magically) without breaking either. This addon is already sitting in the current svn/git of ofxiphone if you’re geek enough, otherwise it should be available within the next few weeks when the OpenFrameworks download is updated.
I’m posting this a bit late, but for some reason I totally missed this video. It’s a summary of some of the sound objects built by François Parra’s students in our joint sound-hypermedia mini-course for second year students. This year we split the two-week class into two: in the first week half of the class learned Processing while the other half learned PureData and audio acoustic instrumentation; the following week they reversed roles. While the hypermedia students were making Processing Monsters in preparation for Eniarof, François’ students were making these interesting instruments.
This is a short movie coming from a teaching project in Aix-en-Provence art school. Students build piezo amplified objects to compose a soundscape. They only used piezo mics, motors, a mixing board, speakers. You will see at the end the begining of a program, made to tie a string sound with generative pictures.
To be clear, I had little to nothing to do with the construction of the objects, I’m only in the video looking like the dork I am because I was the only professor present to conduct any sort of critique, which basically consisted of, « uhhh, how does it work? » followed by some random attempt at analyzing physical algorithmic phenomena. But I wanted to post the video here because it gives a fairly typical picture of the experimental method we use in our exploration of machines. Yeah, yeah, I know, plastic babies on a seesaw have that oh-so-art-school stench of ennui, but there are still some valid ideas in most of the instruments. Also, there’s some brief attempt at audiovisualisation, but on a very very basic level. I think we were far more succesful a few years back at approaching this connection, around the time of 8=8 for example, but again these are students who had only been programming for two or three days and these things go in cycles anyway.
On Saturday, I’ll be speaking at « code_source », an exhibit organized by Etienne Mineur (cf. Incandescence) as part of the Féstival international de l’affiche et du graphisme de Chaumont. The exhibit will attack the question « what is interaction design? » through a historical survey of software, hardware, theoretical productions and games from this still-emerging field. There will also be a section devoted to the creations of art and design students from France, Colombia, USA, Canada, Japan, Switzerland, China, etc. This is where I fit into the picture, as several works from our Atelier Hypermédia in Aix-en-Provence will be documented in the exhibit (more details to come).
Several designers, yours truly included, were also asked to discuss the subject of interaction design for the French graphic design magazine Étapes in conjunction with « Code_source ». We were all asked the same set of questions related to the definition itself of Interaction Design as well as the historical landmarks that influenced us. The selected designers were: Geoffrey Dorne, Jean-Jacques Birgé, Jean-Louis Fréchin, Gabriel Jorby, Éric Viennot, Projet Mü, and abstractmachine and have just been published in the May 2009 edition of Étapes (#168). Here are a few photos of my interview.
You can read the article yourself for my full comments, but here are video links to the four historical events I chose to highlight: Ivan Sutherland, Sketchpad (1963); Myron Krueger, Videoplace (1972-); Seymour Papert, Logo (1967-); Steve Russel et al., Spacewar! (1962). These are fairly standard responses, I know, but I was dealing with an audience (graphic designers) that often confuse (contemporary) software with the conceptual frameworks that made them possible. From this perspective, some history can do little harm.
Of all the interviews, I thought that Jean-Louis Fréchin’s list was the coolest:
The dissociation of gold and the dollar, thereby « announcing the birth of a society of immateriality and exchange »
The personal computer (« augmenting people’s capacities and means of expression »)
1969, the year of the moon landing, for its’ introduction of « mass transportation (747), hyperspeed (Concorde) and networks (Arpanet) », i.e. the concerns and constructs of our present world
So, as I mentioned above, several works produced in part at the Atelier Hypermédia over the past 10 years will be documented at « Code_source ». Here is a list of the works (sorry, in French, but at least I’ve included some images/videos) :
Le WebWaste est une poubelle sur Internet. Par l’activation d’un robot-éboueur, ses utilisateurs se font vider automatiquement le contenu de leurs poubelles (sur ordinateur) dans cette décharge collective (Internet). Les document s’empilent sur le serveur de la décharge. En visitant le site web www.webwaste.net, n’importe qui peut ensuite naviguer à travers les images, textes, sons et vidéos dont d’autres auraient souhaitaient se débarrasser. On peut y chercher de l’inspiration, ou y flâner par curiosité — avec la possibilité bien sûr de récupérer des fichiers pour une consommation chez soi. Les règles sont ouverts, on peut y faire ce que l’on veut : jouer, manifester, dire la vérité ou proliférer des mensonges — c’est aux utilisateurs de choisir.
Dans l’Atelier Hypermédia, nous trouvons qu’il est parfois pratique de suivre le principe de « Just Fuck Around® » quand il s’agit d’apprendre une nouvelle technologie. Dans ce visualiseur/jeu, il n’y a pas eu d’autre but que d’apprendre les fonctionnalités 3D de base du logiciel multimédia Director. Ce modéliseur 3D a été le résultat d’une heure d’expérimentation avec quelques heures d’ajustements par la suite. Il n’y a pas réellement d’autre but que celle de l’exploration d’une nouvelle forme.
Le flux télévisuel est pris en tant que potentiel, un potentiel d’images, de mouvements et de signes. Le processus y extrait, de façon automatisée et en temps réel, toutes les parties d’images qui ont changé lors de la diffusion de l’émission télévisuelle. Il les capture et les redistribue ensuite de façon spatio-temporelle sur une boucle de vingt-quatre images. En l’exposant au spectateur image par image, il agit ainsi sur la diffusion elle-même, en gardant en mémoire et en présentant potentiellement plusieurs fois chaque fragment d’image. L’esprit hypnotisé s’habitue aux fragments, voire les attend et tente de les recomposer. Mais cet effet recherché de rémanence n’empêche pas un effacement progressif de la configuration. Car chaque fragment additionné à la boucle est présent et répété jusqu’à ce qu’un autre le recouvre et ainsi de suite jusqu’à sa disparition et son oubli.
Lors de son passage à l’Atelier Hypermédia, Nao expérimentait principalement avec les formes interactives où image, son et jeu se mélangeaient en une seule entité audiovisuelle. Pour ses performances extra-muros, Nao a décidé de créer un instrument nomade qui nécessiterait uniquement une machine connecté à Internet pour être joué.
Tetris adventure est une variation autour du célèbre jeu de réflexion, mais se jouant désormais en ligne de commande. Il faut entrer manuellement les commandes au clavier dans la console pour faire se déplacer les pièces. Ce changement au sein même du gameplay instaure un rapport au temps différent. La réactivité du jeu n’est plus immédiate et impose au joueur de faire preuve de davantage de stratégie et de réflexion.
Il s’agit également d’un hommage aux premiers jeux d’aventure textuels (Adventure ), où le joueur devait entrer en toutes lettres les instructions.
Quelques faits intéressants à propos de Tetris adventure:
-Tetris adventure est déconseillé aux dylsexique, dixlesquique, dyslexiques.
-Tetris adventure est conseillé par le Medef pour former des dactylos plus performantes.
-L’homme qui a 8 doigts à chaque main est également le champion du monde de Tetris adventure.
Liste des commandes:
[gauche] déplacer la pièce à gauche
[droite] déplacer la pièce à droite
[tombe] faire tomber la pièce
[tourne] faire tourner la pièce
Tetris adventure a été conçu et présenté à l’occasion d’Eniarof 0.4, une fête foraine revisitée, qui laisse (entre autres) une large place aux détournements de jeux vidéos.
8=8 est un groupe de 4 programmeurs / 4 compositeurs / 4 VJs / 4 musiciens / 4 artistes. Tout les quatre apportent leur propre sensibilité esthétique et techniques dans un instrument numérique collectif: L’Hypertable. Par le déplacement de leurs mains sur la surface de l’Hypertable, images et sons sont générés, créant une opportunité unique d’improvisation musicale. 8=8 utilise l’Hypertable pour y jouer des programmes / instruments originaux, dans des contextes de concert / performance / demo.
En 2003, Douglas Edric Stanley crée « L’hypertable » – dispositif d’image/surface interactive. En 2005 il invite trois de son atelier à Aix-en-Provence à se joindre à lui : TM (a.k.a. Thomas Michalak), univers grouillant et mécanique ; Naoyuki Tanaka (artiste japonais résidant à Marseille) ; JankenPopp avec son monde teinté de fraises tagada et d’énergie punk ; et enfin l’univers d’ abstractmachine, plutôt sobre et conceptuel. L’hypertable a été co-produit par le CIREN, avec le soutien d’Arcadi, le DICREAM, et le SCAM.
The mashup-machine est un instrument electronique qui prend la forme d’une boite en bois surmontée de quatre boutons lumineux. Chaque bouton de la taille d’une main permet la manipulation et le mixage intuitif de plusieurs sources multimédia aléatoires. La machine est une interface polyvalente, dynamique et expérimentale qui a été utilisée de 2005 à 2007 lors de performances AV et de concerts lives, elle a également été présentée en tant qu’installation sonore interactive lors de festivals et d’expositions collectives (Eniarof 0.2.2, Festival Emergences).
Deux GameBoy reliées par un câble sont posées sur une réplique miniature d’une table de pingpong. Les visiteurs sont invités à jouer a une version classique du jeu vidéo « Pong » (jeu de tennis). À la différence du jeu original la balle n’est pas visible par les deux joueurs en même temps mais passe d’un écran à l’autre en créant la surprise.
Partant du concept de la machine schizophrénique chez Deleuze et Guattari, le « Gameboy nucleus » est une technologie pour laquelle les facultés de commandes auraient été mises à l’extérieur de la machine construisant ainsi une machine en perpétuelle reconstitution. Autrement dit, ce « nucleus » suit un processus de « physicalisation » dans laquelle l’algorithme, le programme, ou le code, s’exprime à travers des objets tirés entre fonctionnement physique et fonctionnement logiciel.
« Objets Orientées-Objet » (workshop), Douglas Edric Stanley avec l’assistance d’Anne-Laure Schneider et de Pierre Rossel, 2005
Workshop avec les étudiants de l’Haute École d’Art et de Design Genève avec la participation de l’Atelier Hypermédia ESA Aix-en-Provence
Ce workshop de 5 jours exploraient différentes possibilités de l’utilisation du dispositif « Hypertable » créé à l’origine par Douglas Edric Stanley pour son logiciel de cinéma algorithmique Concrescence. Trois étudiants et un enseignant ont terminé des projets pour lors de ce workshop : Pierre-Erick Lefvbre, Jana Korcjomkina, Pierre Rossel, Baptiste Coulon (dans l’ordre d’apparition des projets).
« Déplacements » est constitué de 24 ventilateurs formant un rectangle. Chaque ventilateur est « pixel », sa vitesse de rotation et l’intensité de la lumière de ses LED varient en fonction du niveau de gris correspondant au pixel de référence. Cet écran de ventilateurs est piloté par un ordinateur sur lequel tourne un programme simulant un automate cellulaire intitulé « Le jeu de la vie » (créé par John Horton Conway en 1970). C’est un modèle mathématique où chaque ventilateur est une cellule. «Déplacement » en tant que détournement de cet objet, composant de l’ordinateur, devenant image. Il ne s’agit pas d’un « déplacement » physique mais d’un mouvement, d’un flux.
Stefan Piat travaille sur la reconstitution de monuments et d’espaces en utilisant un système de mosaïques photographiques et vidéographiques qu’il relie dans des cartes dynamiques et interactives. Dans Forsinicola, Piat construit le Fort Saint Nicolas à Marseille. S’y mèle non seulement des images de formats divers, mais surtout des images avec des temporalités multiples, allant du cliché instantané à la chronophotographie (Marey/Muybridge) en passant par des ralentissements et des images-panoramiques. Le tout s’agrège dans une forme qui peut se décomposer et se recomposer selon les mouvements du joueur et du programme.
Que nous habitions là où Cézanne peignait, que nous y vivions ou que nous traversions quotidiennement les paysages que Cézanne a peints, nous sommes tous un peu des habitants de ses peintures. Le studio de création Digital Deluxe a conçu et réalisé Le Voyage immobile, une invitation à partager cet héritage commun. Mais de quoi s’agit-il ? D’un module itinérant que l’on va retrouver dans l’espace public : le centre ville, les villages, les lycées, etc. qui va offrir au passant une découverte sensorielle de l’univers du peintre.
C’est le Centre européen de création et de développement culturel (CECDC) qui a lancé cette ambitieuse réalisation en collaboration avec l’association Terre active. Elle permet d’évoquer, sans chercher à la reproduire ni à l’expliquer, l’expérience sensorielle que représentait l’acte de peindre chez Cézanne. Voici donc une promenade interactive, différente, et sans bouger d’un pouce, sur les pas de Cézanne. Une balade qui, du paysage à l’oeuvre, permet d’évoquer le regard de Cézanne sur la nature, de comprendre le passage du motif au tableau.
Production CECDC / Conception et réalisation Digital Deluxe / Itinérance et tournée Terre Active
Le système présenté est une installation venteuse, constituée de 15 ventilateurs disposés en cercle. Chaque ventilateur souffle en fonction d’informations fournies en temps réel par Internet et utilise le flux de stations météo du monde entier pour générer des dessins à la surface d’une feuille.
Chessynthesis est un dispositif sonore qui transforme deux joueurs d’échecs en des musiciens. Chessynthesis analyse en temps-réel les mouvements de chaque pièce sur le plateau, même celle qui se trouve dans la main du joueur. Avec ses informations, il interprète les indices tactiques/stratégiques du jeu, les tensions, les centres de gravité, et les transcode en de la musique avec de la synthèse granulaire.
« Episureo », Stefan Schwabe & Sebastian Neitsch, 2008
développé à l’Ecole supérieure d’art Aix-en-Provence et au Kunstuniversität Linz
The paths we choose to follow each day, to go to work, to move around in our own homes or even the ways we move during our free-time are mostly pre-defined by architecture, human crowds or subconsciously learned behaviours. During all this, the personal space we keep around ourselves depends on the place we are at and the people surrounding us. We all need privacy but also depend on social relationships and so-for on close contacts to other humans. Our aim is to show this never-ending, pulsing conflict and the trails the participants leave in space and time.
To realize this within an interactive installation, we had to find a place where the participants would stay for quite a while and where they would have to move consciously but also relaxed. We wanted the people to naturally communicate to each other so that groups would appear and we needed them to move in a quite large radius. A swimming pool is almost perfect for this. It is like a stage you step on and just by its architectural form, it already pre-defines paths the swimmers usually follow and with which we as authors can play. On top of that the swimming and the water might relax the visitors so that they have the time and mind to concentrate on the installation. This was never been done before, so not only the visualisation of movement and the dynamics of human crowds where our main topics now, but also the difficult task to convert a public swimming pool into a huge interface.
The picture on the ceiling responds via motion-tracking to the swimmers in the water. Depending on their movements, their speeds and their distances to each other, the visualisation changes and with the help of an underwater loudspeaker variant sounds are generated. Abstract graphics slowly appear and get more and more complex after time, showing trails, movements and the group dynamics in the pool. To picture
the whole movement and not only the one of each single swimmer, sometimes a kind of current appears that influences the whole visualisation.
développé au studio Lentigo, ENSBA Marseille avec l’assistance de l’Atelier Hypermédia ESA Aix-en-Provence
La ville est complexe, en perpétuelle mutation. Les flux d’images, d’information et de déplacement superposent en strates. Dans Node City, on se déplace à l’intérieur de ses strates via un système de navigation corporelle : autour de notre corps une projection au sol nous immerge à l’intérieur d’une carte que l’on peut explorer tout simplement en se déplaçant. Via un système de surveillance, on repère les promeneurs et leur propose un carrefour mobile de vidéos à explorer; en se baladant on découvre les différentes strates audiovisuelles de la ville.
shades of white, mêlant danse, images et son, a été pour le festival les affluents au pavillon noir d’aix-en-provence, en collaboration avec bruno péré. l’idée principale est une évolution de la lumière blanche au travers de ses trois composantes colorées : rouge, vert et bleu. les projections sont la seule source de lumière pour les danseurs sur scène. elles sont diffusées au sol, sur le mur du fond, voire les deux.
la majeure partie de ces histoires colorées est programmé, puis exportée en vidéo et montée sur logiciel vidéo. voilà un petit résumé sur ce que l’on a essayé de symboliser par ces temps colorés :
le rouge est le commencement, comme une naissance hésitante et fragile.
le vert correspond à la création d’un espace hésitant, une sorte de paysage sans dimension.
le bleu est le résidu, après l’explosion blanche des trois couleurs réunies.
tantôt au mur, tantôt au sol, la lumière est image. l’image est matière. le mouvement du corps évolue avec un espace coloré qui se raconte peu à peu. rouge. vert. bleu.
The whole process was a lot of fun, and an excellent pedagogic exercise. We only played around for about an hour, which was probably enough. But I have a feeling we will be doing more « exercises » like this in the future. We do a lot of theory in class, and try to mix that up with play, technical information, project critique, and open discussions. Since what the Atelier Hypermédia basically does is treat code as a « plastic » material, useable in any artistic context (i.e. considering the code itself a possible form of artistic exploration), games like this are really what we are all about. But I was struck by the level of mastery students displayed of the basic rules of Java syntax, and the collective part of the session was a real eye-opener on how good the students have gotten at collective coding. Talking about code is a Good Thing™®, coding with ten or more hands in and Even Better Thing™®. Just afterwards we had a great session with a young artist presenting her project, the exercise being: how would you develop her installation ? There too, the students showed a great capacity at collectively designing the project, even when discussing the complex details of the code. The trick, apparently, is playing such arcane and thick subjects fast and loose, and thereby ignoring the preciousness of the form itself of the code, and doing it collectively, and as a form of play. It’s an intellectual bait and switch which in the end allows for a rigorous form of play.
Some space was reserved yet again this year for the Atelier Hypermédia at the Festival Gamerz. Gamerz is a festival that explores the relationship between art & video games. This edition was co-curated by Isabelle Arvers who appears to be helping this promising little festival power up to the next level.
I’m a little late with this one, given that I was trapped in the snow in my little village for several days. I actually missed the opening and had to assist the last days of the Atelier Hypermédia’s contibution via lots of ftp and sms. Debugging and adjusting variables over cellular is not really my idea of fun, although it’s certainly not the first time. So I’ve only now been to the exhibit to actually see/document the works (and fix a couple bugs).
Our first contribution is from Florent Deloison, a 5th year student at the school and who is presenting his installation « Recession Kid »: an 8-bit role-playing romp through the fun of mass-layoffs, offshore banking, slum landords, and free-wheeling stock market pimping. While the game harkens back to old-skool RPG’s, it adds some minor sophistications to the genre by indexing the gameplay to real-time fluctuations of various on-line data, ranging from live stock market values to weather conditions on wall street.
The second installation is called « Twiist » and is a game from Jane Antoniotti which, while intended as a conceptual work, is actually quite fun to play. We built this one from the ground up in about three days when our previously planned installation went AWOL. Adding snow into the factor, I consider this quite a feat. Jane’s idea was to build a game using the Wii Balance Board, transforming it into an un-balanced board. Basically, during gameplay, the board periodically de-axes its center of gravity, and the player has to twist themselves into various improbable contorsions in order to re-produce, on screen, the ideal image of beauty. For Gamerz 04, Jane decided for a decidely Baywatch edition for the beauty model.
While making « Twiist », we wondered why Nintendo had never thought of this idea, for example by tying the Balance Board into their underwhelming Photo Channel. In the Atelier we were playing Flow on a Playstation 3 last week and were wondering, in a similar vein, why there aren’t any decent uses of the the Wiimote as simple and obvious as WiiPlay. Anyway, in Jane’s Twiist, the game was programmed to dynamically adjust to whatever photo is fed into it, and could easily be integrated with some simple OpenCV-style face detection + blob detection from within one’s own photo library. So any photo library could feasibly be fed into the game. There are in fact many potential variations to Jane’s game and would be pretty easy to program. Too bad we don’t have easy access to WiiWare (along the likes of the iPhone App Store, or even Microsoft’s XNA Creator’s Club), otherwise we would have been happy to port it to the real Wii. Again, the industry’s overprotective economic model is confounded with a contemporary version of shooting oneself in the foot. Nintendo has called one of their series « ArtStyle », with one of their games looking suprisingly similar to this old program created in 2004 in the Atelier by Yannick Aïvayan. Given that on rare occasions what we design is not so dissimilar, it makes no sense to me why we can’t distribute on that platform. But there you go, again: business models. But I’m rambling…
Of note: both games were built with Processing and Twiist uses Osculator to pull the data off the Balance Board and send it to Processing via Open Sound Control. For those looking to use the Balance Board, be careful of this little gem: early Balance Boards (April-May 2008) send their data out strangely and Osculator can register two of the four pads. This makes no sense of course, since the board works fine on the Wii itself, but there you go. Be forewarned. Camille Troillard, the creator of Osculator, has just sent me a beta test of the latest incarnation which fixes this problem. He is still testing, but my version works and on a balance board that previously worked incorrectly.
Also of note, both games use heavy doses of Polymorphism for more variation and evolution in the gameplay. Basically, if you are going to be doing any moderate to serious game design, you should know classes, objects, inheritance, and polymorphism. I have an old blog post on this subject.
Two other notable works at Gamerz, and from Atelier Hypermédia alumni to boot:
Antonin Fourneau’s R+R (at the Galerie Sextius), a totally killer head-banging apparatus where you don a heavy-metal-approved wig, and violently bob your head up and down, provoking ear-splitting guitar riffs that (with the proper frenzy) pump it up to eleven.
Manuel Braun and Virginie Le Gall’s « Fais-moi mal », a masochistic punching ball with an integrated speaker+accelerometer+arduino: as you punch the ball, the ball cries out for more. Probably one of the more gadgety installations by Manuel, but fun nevertheless and well-designed.
There are many other interesting installations, for example France Cadet’s Sweet Pads or Paul Destieu’s installation at the art school gallery which I can only describe as an autistic tennis match: the machine throwing and catching its own tennis ball, all day long. Er, uh, maybe that’s tautological tennis? Tennis recursion loop?
And finally, there appears to be a promising evening on January 28th with a contemporary take on the endlessly recycled ideas begun by Radúz Çinçera’s « Kinoautomat ». This time, the interface is your mobile phone: the film « E1000 » asks you to turn your cellphone on before the movie begins, and to use it in order to interact with the movie.
I love Processing Monsters, I think it’s a great idea. I saw it on Code & Form last week, and immediately gave it as an assignment to the 2nd-year students who, for the most part, have never programmed before and had only 3 days to learn the basics. Using Processing Monsters as an objective was great, as it kept us focused on some very basic functions (ellipse, bezier, shape, translate, etc) but which can quickly get out of control without some methodology. Also, looking forward to ENIAROF in March, monsters seems an appropriate theme.
I made the mistake of introducing class/objects on the final day, in a pretty funny class on fur, hair and tufts which I’ll have to reproduce in some form or other. I should have started directly with objects, as we did in the Algorithmic Design project we initiated last month in Orléans. In my experience, it’s easier to learn class/objects from day 1, rather than day 3, or week 5. Once you’ve become lazy programming spaghetti code, it’s too hard to break it off into objects. No matter how ugly it is, once comfort has settled in, it’s simply too easy to get stuck in linear thinking. That must have something to do with the brain’s natural tendencies. However, if you start from day 1, you stay organized, people tend to understand the code better, and probably can make cooler monsters. Alas! We did things ass-backwards, and the students’ code mastery suffered as a result. But a few of the monsters are fun nevertheless :
Stéphane Cousot and I are announcing today the public availability of our OpenCV Library for Processing. Although the library has been ready (in various states of undress) for a few months now, we have been using the intervening time to learn more in-depth how OpenCV works, debug, simplify method calls, test the library in real-world situations, add various features, plan out features for future releases, and — most importantly — write coherent documentation for those Processing users discovering OpenCV for the first time. It might seem like a light start, given the limited number of functions we’ve made available from the impressive Intel library, but we wanted to make sure each component worked as promised. Also, we wanted to make working with it as painless as possible for Processing users, and follow the Processing logic of getting complex things done with a limited number of simple methods. And finally, we wanted to make sure it was stable enough in a real-world installation context.
For the features, you have internal (via OpenCV) and external (via Processing) capture, basic image treatment (threshold, comparison, extraction, etc), contour tracking, face & body tracking, and a few other little goodies thrown in here and there. So, as it stands, you can (for example), recognize someone’s face, grab the outline of that face, and go into the image data of that person’s face to extract the face data. Or, you could use infrared filters with lights pointed at or placed on your body (see below), a multi-touch surface, or some other artificial lighting condition to grab light blobs for finger or body-part tracking and use that data somehow in Processing. There are obviously many possibilities.
Some of the things you cannot yet do, and which we plan to add to the library: motion history images and optical flow (pixel tracking), kalman predictions, color tracking, histograms, and obviously the list could go on and on. A lot of these functions I already have working in OpenFrameworks for an installation (soon to be announced) which will be exhibited later this summer. So consider the current release a starting point, with what we believe is a fairly clean start, but we could be wrong on that. The code is open, so go in and dig around — perhaps you can give us some good advice or add to the code yourself.
Special note: this library will also work for pure Java work, and yes, there is Java documentation.
So, why did it take so long? Well… when I say that we’ve been busy testing it in laboratory and real-world instances, I mean it. I’ve gotten some mail on this recently, so I should make things a little clearer: if you ever wondered why I don’t post as much as I (or apparently some of you) would like, it’s because I’m busy elsewhere working on so many @#&*$% projects. I do not just work on my own projects and I am definitely not a full-time blogger : I teach, run an atelier, collaborate with other artists, do research, write, write code, consult, curate, and somewhere in there, I’m a dad for two lovely and brilliant young (or youngish) women. Since I don’t have a secretary, nor a double, that means some creative Douglas-time-sharing. So when I’m quiet here, it most certainly means that I’m busy doing one of these other things. And over the past few months, that has worked out to about 50% of my creative work involving OpenCV in Processing and OpenFrameworks.
And on Stéphane’s side, he’s been just as busy working over the past six months on a gazillion projects for various artists, art students, and researchers; and only a part of that work involved this OpenCV library.
So, what have we been doing with it? The library has already been used in numerous projects at the Atelier Hypermédia, in external workshops at schools such as the Institut d’Arts Visuels in Orléans, as a research tool at the DRII laboratory (Dispositifs relationnels : Installations Interactives) at the École Nationale Supérieure d’Arts Décoratifs in Paris, and in two public works, one an installation for Gamerz 0.2 and the other as a component of a haptic dance performance-dispositif by Wolf Ka and his studio. Finally, we used the library to prototype an urban-design project by Lei Zhao for the Studio Lentigo, Marseille although this project was eventually finished in OpenFrameworks due to the high video performance demands of the installation. So all in all, about a dozen different projects over the past few months.
Here are a few images/videos with links for more information on the author(s)/works:
Lei Zhao, Node City (follow link for more videos).
Fabien Artal, Diplôme DNSEP (avec les félicitations du jury), L’école supérieure d’Aix-en-Provence. There is a video, but you’ll have to jump to 23:15 for Fabien’s installation.
Students of the Institut d’Arts Visuels, Workshop Légerté + Nuit des musées, Orléans (follow this link for — very poor quality — video).
I’ll leave off with these images from an installation Stefan Schwabe created with his collaborator Sebastian Neitsch in a public pool in Halle. As swimmers wade about, their movements are tracked by a camera and modify an image built out of 4 overlapping projectors, projecting onto the dome of the rotunda. It should be mentioned that, like Lei Zhao’s Node City, this piece used Processing only during the prototyping phase (the final work was created in vvvv). Nevertheless, Stefan & Sebastian’s project was an important one in our year-long experimentation with various forms of video surveillance in art and design installations. (See Stefan’s website for video of this installation).
Update: I used the wrong terminology. Oops. We decided to call this version v01, precisely to suggest that there is still much progress to be made. Previously I called it v1.0, which is a very different idea!
Vincent Cogne found some time (between evaluations and the whatnots of end-of-semester madness) to put up the source code to the solution he found for creating stereoscopic Processing sketches [link]. We tried a few different solutions before this one (well, to be honest, I just made teacherly suggestions with a lot of OpenGL theory and Vincent did all the work). This was all while Ben Chang was here teaching us his solution using Ygdrasil. I think Ben has done great work on making a viable solution, but I just cringed when I saw all the hackery-clickery that had to be done just to set the environment up. I figured there had to be a better way.
The advantage of Ygdrasil over Processing/OpenGL is that it’s higher level, so you can just say « put this model here, and that model there » and the thing takes care of it for you (sort of like Shockwave3d in that sense, but on a more serious foundation). But on the other end, from an artistic perspective, I actually prefer to work closer to the bone — screwing around with the OpenGL directly because you get more visually interesting results (= easier to make pretty mistakes). But that’s just me. Anyway, with Vincent’s solution we were able to play around with the basics of stereoscopic imagery very quickly, and in the little hour I found to fiddle around with it, I saw the power of working this way rather than the higher-level stuff. You can iterate the designs so much faster this way (thank you Processing). I wish I had more time to play with it, but ultimately that’s the great thing about running an atelier : you can follow several different directions simultaneously.
As a side note, we also noticed very quickly that the stereoscopic images really pop out of the stucture better if you work off of a black backround. In other words, if you lose the « image » reference, and even lose the frame, and work off of the structure as if it were an architectural element within the room and not just a big TV screen. This is the mistake most « Virtual Reality » environments make (god I hate that term). They make a huge effort to get stereoscopic images, and then project it onto a conventional surface.
Oh, if you don’t know how to make a stereoscopic image, it’s actually very similar to how to make a Hypertable. But in this case, you need two projectors, with polarized filters, and a rear-projection screen that respects the polarization. (Huh? That’s not much of an explanation) Oh, just ask Vincent, he’ll explain it to you. He’s quite brilliant at everything 3d-programmable, so go bug him. I just got off the plane for @&#§$*@#¡ and I need some sleep.
And while we’re on the subject of 3d engines (see that? it’s called the bait-and-switch), John Klima just finished a two week workshop at the school showing our students how to use the Torque Game Engine. I didn’t have the time to hang out with them as much as I would have liked, but I got enough of a glimse to know that it’s an even better soluton than XNA which was really bothering me because of the licence, the Microsoft-centric aspect, etc. Whereas Torque is Mac/PC/Linux/Xbox. Pretty impressive. So like XNA, you can make Xbox360 games with it (and even distribute over Xbox Live Arcade, although that’s far more complicated because you have to go through the Bill, even if Bill does give you a good cut).
Jack Stenner took some time out at ISEA2006 to walk me through Torque as well and I had left with a good impression. We were exhibiting next to one another and often up late plugging things in, so I got to see how the system worked for him. As with everything, there are some wonky aspects, but so what, it’s open-source and cheap. And I think that’s really cool that there is a viable open-source platform for gaming. In fact, I would even go so far as to suggest that we will not have a true massive gaming culture (on the same scale and deep influence as cinema, for example) until the platform becomes truly open and open-source. No one has a patent on 24-images-per-second, so too should it be for game platforms.
Enough techno-art-geek soap boxing for tonight. Zzzzz….
We’re very excited about this in the atelier. If you don’t know already:openframeworks is Zachary Lieberman’s project for an easy toolchain for artists wanting to work with C++. It’s notProcessing, i.e. it won’t teach you how to code. But if you know Processing well, and/or you understand how to work with classes, this project should make your process of discovery a whole lot easier.
It took my collegues some time to realize this with Arduino : it isn’t the actual difficulty of the code itself that keeps many people away from these technologies, it’s (gasp!) the actual difficultly of plugging all the crap together correctly and turning it on. Difficulty and obscurantism are two very different things, and people tend to forget this. Nobody complains about a word processor not being able to write a really difficult thesis for them (to give just one example ;-), but they do complain if that word processor actually gets in the way of writing it. In the same vein, writing code can be difficult, everyone knows that — but why the hell should I have to write code while balancing on one foot and whistling Dixie?
Just a few months ago, some students expressed the desire to move up (or down, in fact) to C++ development for more speed and a deeper reach into the machine. One student is actually in the process of making the shift right now. In fact, Processing became a reality for us just at the moment when I was starting to build a toolchain very similar to what Lieberman et Cie have come up with. And I too was worried about how to make the whole process simple enough for students with programming experience but no formal training in computer science. Unfortunately, as only a cursory glance at this blog would show, I’m still trying to tie up a lot of loose ends, and this project never came to fruition. I only had time to make a few short examples of developing in C with OpenGL on Mac/PC/Linux, and even those remained unfinished. So this is a project I definitely want to get involved with, as many hands are always better than two when it comes to building platforms.
Probably more interesting than the actual libraries themselves (their open nature, etc.), is the prospect of having an artistic community working around a single set of API’s. Even if this community is small, it’s still significant. This is really the key. Starting around the time of Mac OS X Beta, I began experimenting with Cocoa/Objective-C, for example, until I realized that there weren’t m/any artists out there working with it, making it an irresponsible choice for teaching young artists. You don’t want to teach students some obscure technology that doesn’t have a footprint with other artists/students. Of course this might have been okay for computer science students, but it wasn’t a good choice for ours, especially given then fact that at the time less than half of the students used Macs (although that number is now around 75% if I count just those working with me). So I refuse to teach some technology that won’t work (for free) on everyone’s machines. That is one of the reasons I’ve totally ignored Quarz Composer and vvvv, even if they are both pretty cool.
We will absolutely jump all over this as soon as it becomes a reality. If not me directly, definitely some of my students.
