abstractmachine

Stéphane Cousot and I are announcing today the public availability of our OpenCV Library for Processing. Although the library has been ready (in various states of undress) for a few months now, we have been using the intervening time to learn more in-depth how OpenCV works, debug, simplify method calls, test the library in real-world situations, add various features, plan out features for future releases, and — most importantly — write coherent documentation for those Processing users discovering OpenCV for the first time. It might seem like a light start, given the limited number of functions we’ve made available from the impressive Intel library, but we wanted to make sure each component worked as promised. Also, we wanted to make working with it as painless as possible for Processing users, and follow the Processing logic of getting complex things done with a limited number of simple methods. And finally, we wanted to make sure it was stable enough in a real-world installation context.

Download link: here

For the features, you have internal (via OpenCV) and external (via Processing) capture, basic image treatment (threshold, comparison, extraction, etc), contour tracking, face & body tracking, and a few other little goodies thrown in here and there. So, as it stands, you can (for example), recognize someone’s face, grab the outline of that face, and go into the image data of that person’s face to extract the face data. Or, you could use infrared filters with lights pointed at or placed on your body (see below), a multi-touch surface, or some other artificial lighting condition to grab light blobs for finger or body-part tracking and use that data somehow in Processing. There are obviously many possibilities.

Some of the things you cannot yet do, and which we plan to add to the library: motion history images and optical flow (pixel tracking), kalman predictions, color tracking, histograms, and obviously the list could go on and on. A lot of these functions I already have working in OpenFrameworks for an installation (soon to be announced) which will be exhibited later this summer. So consider the current release a starting point, with what we believe is a fairly clean start, but we could be wrong on that. The code is open, so go in and dig around — perhaps you can give us some good advice or add to the code yourself.

Special note: this library will also work for pure Java work, and yes, there is Java documentation.

So, why did it take so long? Well… when I say that we’ve been busy testing it in laboratory and real-world instances, I mean it. I’ve gotten some mail on this recently, so I should make things a little clearer: if you ever wondered why I don’t post as much as I (or apparently some of you) would like, it’s because I’m busy elsewhere working on so many @#&*$% projects. I do not just work on my own projects and I am definitely not a full-time blogger : I teach, run an atelier, collaborate with other artists, do research, write, write code, consult, curate, and somewhere in there, I’m a dad for two lovely and brilliant young (or youngish) women. Since I don’t have a secretary, nor a double, that means some creative Douglas-time-sharing. So when I’m quiet here, it most certainly means that I’m busy doing one of these other things. And over the past few months, that has worked out to about 50% of my creative work involving OpenCV in Processing and OpenFrameworks.

And on Stéphane’s side, he’s been just as busy working over the past six months on a gazillion projects for various artists, art students, and researchers; and only a part of that work involved this OpenCV library.

So, what have we been doing with it? The library has already been used in numerous projects at the Atelier Hypermédia, in external workshops at schools such as the Institut d’Arts Visuels in Orléans, as a research tool at the DRII laboratory (Dispositifs relationnels : Installations Interactives) at the École Nationale Supérieure d’Arts Décoratifs in Paris, and in two public works, one an installation for Gamerz 0.2 and the other as a component of a haptic dance performance-dispositif by Wolf Ka and his studio. Finally, we used the library to prototype an urban-design project by Lei Zhao for the Studio Lentigo, Marseille although this project was eventually finished in OpenFrameworks due to the high video performance demands of the installation. So all in all, about a dozen different projects over the past few months.

Here are a few images/videos with links for more information on the author(s)/works:

• Wolf Ka + cie, Moving By Numbers (see link for video).

• Atelier Hypermédia, C’est toi la patate.

• Lei Zhao, Node City (follow link for more videos).

• Fabien Artal, Diplôme DNSEP (avec les félicitations du jury), L’école supérieure d’Aix-en-Provence. There is a video, but you’ll have to jump to 23:15 for Fabien’s installation.

• Students of the Institut d’Arts Visuels, Workshop Légerté + Nuit des musées, Orléans (follow this link for — very poor quality — video).

• I’ll leave off with these images from an installation Stefan Schwabe created with his collaborator Sebastian Neitsch in a public pool in Halle. As swimmers wade about, their movements are tracked by a camera and modify an image built out of 4 overlapping projectors, projecting onto the dome of the rotunda. It should be mentioned that, like Lei Zhao’s Node City, this piece used Processing only during the prototyping phase (the final work was created in vvvv). Nevertheless, Stefan & Sebastian’s project was an important one in our year-long experimentation with various forms of video surveillance in art and design installations. (See Stefan’s website for video of this installation).

Update: I used the wrong terminology. Oops. We decided to call this version v01, precisely to suggest that there is still much progress to be made. Previously I called it v1.0, which is a very different idea!

Vincent Cogne found some time (between evaluations and the whatnots of end-of-semester madness) to put up the source code to the solution he found for creating stereoscopic Processing sketches [link]. We tried a few different solutions before this one (well, to be honest, I just made teacherly suggestions with a lot of OpenGL theory and Vincent did all the work). This was all while Ben Chang was here teaching us his solution using Ygdrasil. I think Ben has done great work on making a viable solution, but I just cringed when I saw all the hackery-clickery that had to be done just to set the environment up. I figured there had to be a better way.

The advantage of Ygdrasil over Processing/OpenGL is that it’s higher level, so you can just say « put this model here, and that model there » and the thing takes care of it for you (sort of like Shockwave3d in that sense, but on a more serious foundation). But on the other end, from an artistic perspective, I actually prefer to work closer to the bone — screwing around with the OpenGL directly because you get more visually interesting results (= easier to make pretty mistakes). But that’s just me. Anyway, with Vincent’s solution we were able to play around with the basics of stereoscopic imagery very quickly, and in the little hour I found to fiddle around with it, I saw the power of working this way rather than the higher-level stuff. You can iterate the designs so much faster this way (thank you Processing). I wish I had more time to play with it, but ultimately that’s the great thing about running an atelier : you can follow several different directions simultaneously.

As a side note, we also noticed very quickly that the stereoscopic images really pop out of the stucture better if you work off of a black backround. In other words, if you lose the « image » reference, and even lose the frame, and work off of the structure as if it were an architectural element within the room and not just a big TV screen. This is the mistake most « Virtual Reality » environments make (god I hate that term). They make a huge effort to get stereoscopic images, and then project it onto a conventional surface.

Oh, if you don’t know how to make a stereoscopic image, it’s actually very similar to how to make a Hypertable. But in this case, you need two projectors, with polarized filters, and a rear-projection screen that respects the polarization. (Huh? That’s not much of an explanation) Oh, just ask Vincent, he’ll explain it to you. He’s quite brilliant at everything 3d-programmable, so go bug him. I just got off the plane for @&#§$*@#¡ and I need some sleep.

And while we’re on the subject of 3d engines (see that? it’s called the bait-and-switch), John Klima just finished a two week workshop at the school showing our students how to use the Torque Game Engine. I didn’t have the time to hang out with them as much as I would have liked, but I got enough of a glimse to know that it’s an even better soluton than XNA which was really bothering me because of the licence, the Microsoft-centric aspect, etc. Whereas Torque is Mac/PC/Linux/Xbox. Pretty impressive. So like XNA, you can make Xbox360 games with it (and even distribute over Xbox Live Arcade, although that’s far more complicated because you have to go through the Bill, even if Bill does give you a good cut).

Jack Stenner took some time out at ISEA2006 to walk me through Torque as well and I had left with a good impression. We were exhibiting next to one another and often up late plugging things in, so I got to see how the system worked for him. As with everything, there are some wonky aspects, but so what, it’s open-source and cheap. And I think that’s really cool that there is a viable open-source platform for gaming. In fact, I would even go so far as to suggest that we will not have a true massive gaming culture (on the same scale and deep influence as cinema, for example) until the platform becomes truly open and open-source. No one has a patent on 24-images-per-second, so too should it be for game platforms.

Enough techno-art-geek soap boxing for tonight. Zzzzz….

If you’re coming to this from somewhere else than Processingblogs, and you’re interested in code, then you might want to check out Jesús Gollonet’s recent post about openframeworks.

We’re very excited about this in the atelier. If you don’t know already:openframeworks is Zachary Lieberman’s project for an easy toolchain for artists wanting to work with C++. It’s not Processing, i.e. it won’t teach you how to code. But if you know Processing well, and/or you understand how to work with classes, this project should make your process of discovery a whole lot easier.

It took my collegues some time to realize this with Arduino : it isn’t the actual difficulty of the code itself that keeps many people away from these technologies, it’s (gasp!) the actual difficultly of plugging all the crap together correctly and turning it on. Difficulty and obscurantism are two very different things, and people tend to forget this. Nobody complains about a word processor not being able to write a really difficult thesis for them (to give just one example ;-), but they do complain if that word processor actually gets in the way of writing it. In the same vein, writing code can be difficult, everyone knows that — but why the hell should I have to write code while balancing on one foot and whistling Dixie?

Just a few months ago, some students expressed the desire to move up (or down, in fact) to C++ development for more speed and a deeper reach into the machine. One student is actually in the process of making the shift right now. In fact, Processing became a reality for us just at the moment when I was starting to build a toolchain very similar to what Lieberman et Cie have come up with. And I too was worried about how to make the whole process simple enough for students with programming experience but no formal training in computer science. Unfortunately, as only a cursory glance at this blog would show, I’m still trying to tie up a lot of loose ends, and this project never came to fruition. I only had time to make a few short examples of developing in C with OpenGL on Mac/PC/Linux, and even those remained unfinished. So this is a project I definitely want to get involved with, as many hands are always better than two when it comes to building platforms.

Probably more interesting than the actual libraries themselves (their open nature, etc.), is the prospect of having an artistic community working around a single set of API’s. Even if this community is small, it’s still significant. This is really the key. Starting around the time of Mac OS X Beta, I began experimenting with Cocoa/Objective-C, for example, until I realized that there weren’t m/any artists out there working with it, making it an irresponsible choice for teaching young artists. You don’t want to teach students some obscure technology that doesn’t have a footprint with other artists/students. Of course this might have been okay for computer science students, but it wasn’t a good choice for ours, especially given then fact that at the time less than half of the students used Macs (although that number is now around 75% if I count just those working with me). So I refuse to teach some technology that won’t work (for free) on everyone’s machines. That is one of the reasons I’ve totally ignored Quarz Composer and vvvv, even if they are both pretty cool.

We will absolutely jump all over this as soon as it becomes a reality. If not me directly, definitely some of my students.

We’re still waiting for some stragglers, but here are some of the videos we have online from the ENIAROF group on YouTube and the ENIAROF Pool on Flickr. For more information on ENIAROF, read this article by Marie Lechner (Les nouvelles lois de l’attraction, or translated into English: The New Laws of Attraction), or visit the ENIAROF Blog. Most of the electronic work you see here is the result of a two-week workshop using Processing, Arduino, and Wiring. The rest are propositions by various artists and contributors to ENIAROF, for example Dekalko Studio, M2F, or a theatre troupe/military camp from Belgium that went by the title « Cookies Koolkies ».

Here’s the official description: « ENIAROF is a unique concept for a new form of carnaval, or « fête foraine », mixing the logic of flash mobs, gore, digital arts, classic video games, mashups, karaoke, BBQ, cotton candy, and anything weirdly kaiju-esque on a foggy night that just wants to have fun. ENIAROF (0.1) descended onto Aix-en-Provence in March 2005, followed by Paris (0.1.1), Marseille (0.1.2) and Aix-en-Provence (0.2) in late 2006. »

And don’t forget: GamerZ (bad name, fun exhibit) is still going on until this Friday afternoon (19h00). Much of ENIAROF ended up spilling into that exhibit and vice-versa.

Videos from the arcade:

{1} « MadNES »; Manuel Braun, Antonin Fourneau, Stéfan Piat; built with NES + modified controller; {2+3} « Immortal Combat »; Jean-Baptiste Alfonsi, Thomas Cheneseau, Wael Koudaih; built with Processing + camera + beamer + boxing gloves; {4} « Cui Cui Cabaret »; Cui Cui; built with carboard box + electricity + amplification; {5} « Pitch-Pong »; Émilie Brout; built with Processing + ESS + microphone; {6} « Simulateur de reportage TF1 »; Florian Deloison; built with Processing + camera; {7} « Sade »; Pascal Chirol; built with Processing + Arduino + camera; {8} « Open Your Eyes »; Marjorie Brunet, Tomek Jarolim; built with Processing + microphone; {9} « La dialectique des cailloux »; Maxime Marion; built with Processing + ReacTIvision + Live + camera + beamer; {10} « Tout le monde s’appelle Marcel Marceau »; Fabien Artal; built with Processing + ReacTIvision + camera + beamer; {11} « TekkenDrum »; Raphaël Isdant; built with Processing + Arduino + drumkit; {12} « L’ours mal leché »; Liza Gabry, Caroline Delieutraz; built with Processing + Arduino + fluffy bear.

Images from the Flickr pool:

More videos to come from those two locations, but I have just a few weeks left to finish my thesis, so enough ENIAROFing for me.

More just-in-time-for-ENIAROF portraits to give you an idea about who is behind this project, what sort of things we’re interested in, as well as a glimpse of some of the work produced at the Atelier Hypermedia. This time we’re featuring Antonin Fourneau: inventor and organizer of ENIAROF, researcher at the Atelier de Recherches Interactives (ENSAD), and hacker/artist/coder working in the field of video games.

Here is one of my favorite works of Antonin, which he developped in collaboration with Erational for the FAN project at Villette Numérique 2004:

It’s a mini foldout Ping Pong table with two Gameboy Advance consoles connected to each other via a local network (cf. GameboyPong). The ball is then sent back and forth across its « net » via the digital network connecting the two consoles. This fed into Téléférique’s concept of FAN, which created an actual/virtual mix somewhere between « PC Tuning » (cf. JackyPC), Origami, and networking.

Some aspects of this work grew out of two workshops at the school: PLAY+MOBILE and a workshop with Chris Csikszentmihalyi at the Laboratoire L.O.E.I.L.. He then evolved all of these experiments into two projects which he presented for his diploma in 2005: ENIAROF, an innovative alternative to digital arts festivals (i.e. Villette Numérique); and The Gameboy Nucleus:

Here is a YouTube video, or you can click here for Quicktime format: GBA Nucleus #1, GBA Nucleus #2:

The Gameboy Nucleus explores many of the issues we’re interested in at the Atelier, and is basically a physical externalization of the interior structure of the Gameboy. Each of the components you see here (Music Box, Woodpecker Toy, Button, …) are externalized forms of internal registers in the Gameboy. By connecting them up you activate or change the state of these internal registers, and thereby affect the internal algorithm. Although I have been speaking in my writings and workshops for many years about the spatialization of algorithms and the externalization of code, I never actually thought of it in such concrete terms. It really is fascinating to see how he designed it. I also find the drawings just as interesting as the actual working forms: it’s interesting to see how he created physical forms of counters (« i++ »), loops (« for(;;) »), randoms (« rand()% »), and so on. It’s quite lovely to hold code in your hands like that.

Here is a photo of one another work from about the same period, one of his contributions to ENIAROF, the Pince Vocale (scream to get an origami robot out of the onscreen « factory » game and into your hands):

Since September 2005, Antonin has been at ARI in Paris, where he has been expanding this work into a whole series of game console manipulations, some of which he presented at the Tokyo Dorkbot in late September:

One of these works, the Noisy Nucleus was also featured as one of the installations at this year’s Festival Emergences (note: as mentioned previously here, ENIAROF was given « carte blanche » at the festival this year, as a sort of bar-room entertainment between musical acts; a role we were — for once in our lives — happy to fill, given the nature of the project). The central idea to the Noisy Nucleus is that the output of one video game or console (joystick, video output, sound, etc) can be used as input into another video game. He basically hacks into the protocol of various game peripherals (SNES, etc), and creates a transcoder/recorder, allowing signals of one machine to pass into another. For example, while playing streetfighter you can also be mixing a breakcore soundtrack

Click the above images for this Quicktime video clip taken from Antonin’s presentation at ARI.

Finally, for this year’s Arborescence, he and Jankenpopp built a series of ENIAROF-style video game consoles. Two of Antonin’s proposals are worth mentioning. The first is a street-fighter with only one joystick: you basically fight against yourself. In the second, the « jump » button has been connected to the video output of Donkey Kong: whenever you jump the screen goes black; and since you have to jump a lot in Donkey Kong, this makes the game very hard to play:

Here’s a YouTube clip of the modified Street Fighter:

And then finally, comes this year’s ENIAROF, which Antonin is frantically preparing as we speak:

I will mention this later on the blog as we get closer, but if you happen to be at the Chicago Art Institute at the end of the week, Antonin and I will be giving a webcast presentation on the last wednesday of this month (11:00 am). This is part of our cross-atlantic teaching collaboration. We’ll see if we can record that somehow and put it onto some podcast (haven’t gotten around to making one yet).

To continue the series on cool student work from the Atelier Hypermédia, and just in time for ENIAROF, comes this small portrait of Jankenpopp (pronounced Jan-ken with a hard J), i.e. Pierre-Erick Lefebvre. Janken has been a longtime collaborator on projects such as 8=8, my assistant for the Objets orientés objet workshop in Genève as well as for the Concrescence algorithmic cinema platform where he contributed significantly to the project. Until recently he was a student in Aix-en-Provence, and is now starting his post-graduate studies in the Atelier de Recherches Interactives at the prestigious ENSAD in Paris. Finally, he and Antonin Fourneau will be bringing the rest of ARI to my Atelier this Tuesday to create the next great noisy, goofy, dopey, thumping, stumbling, bumping, @#!&ing, beeping and booping machines for the ENIAROF Video Arcade.

The first thing you need to know about Jankenpopp is that he’s one of of those kick-ass mashup DJ breakcore dweebs:

He has several online albums you can download at: jankenpopp mp3s, my favorite track being Nova Swap.

The second thing you need to know is that he’s a lazy-ass programmer who makes totally cool mini-software online, i.e. however he feels like it, just piling on the code until he get’s what he wants. Usually it looks something like this (click on images to play, requires Shockwave) :

And then, the final thing you need to know is that all that occasionally gets rolled up into a ball and turned into more complex (and well-programmed) audiovisual musical ensembles where the images/programs play/generate the music:

Both of the above videos are collaborative efforts, but there are certain sections that have his signature all over them.

Oh, and I forgot, although it was also mentioned here earlier: he created the Jankenpopp-666 soundfont, for my Cubed installation at ZeroOne San Jose:

Ok, so I was feeling a little inspired today, and sort of strung up a pretty diverse plate of references, and along with those from the students and my collegues it was a pretty fun mix. For those that were present, here are at least the few I was able to remember (we went very fast, despite our obsession with slowness ;-). This is a mix of everyone’s links, I can’t remember who brought what to the conversation: Slow Real-Time Systems, Kansas Flatter Than a Pancake, Computer Emotivity, The Room of Desires, Sowana, Eve Future, L’homme machine, John Von Neumann vs. Alan Turing (cf. Alan Turing with breasts), The Turing Test, Eric Cartman, Richard Dawkins, and the Wii, The Church of the Flying Spagetti Monster, The God Delusion, Stephen Jay Gould, Stephan Wolfram, Richard Dawkins & Memes, Probabalistic Robotics, Rube Goldberg Machines, Diet Coke & Mentos, Nucleation, Der Lauf der Dinge, Honda’s Fischli & Weiss rip-off, Widget Workshop vs . Max/MSP, I (heart) Huckabees, Heidegger - The Question Concerning Technology, Prédiction, variation, imprévu, Plotsème, …

There was a lot more, but that’s mostly what we talked about during the morning session.

• website: EasyDesign
• artist: Pascal Chirol
• link: http://www.easydesign.fr/

Continuing the series on interesting student work…

Pascal Chirol has just arrived this year, and has been working alongside our merry ragtag gang at the Atelier Hypermedia where he is currently maturing his Flash skills into a deeper understanding of object-oriented programming while learning Processing & Java. Later, he will be connecting up physical interfaces to his programs with my collegues at LOEIL (should be interresting).

He is also fine-tuning a hillarious and yet quite elegant, and usable (!) website he created as a student at the École régionale des beaux-arts de Valence where he studied with some of the rare (but growing) French art professors hip to generative design, interactivity, programming, etc. (cf. Luc Dall’armellina). His project is called EasyDesign and it generates beautiful vector-based imagery for you, so you don’t have to.

The part that I find hillarious is that you can also ask it to build you a poster in the style of John Maeda or the French design superstars M/M Paris. For an untrained eye, I’m sure it’s pretty damn close! There is also « Easy Swisse » (LOL), and all sorts of other jokes for anyone who has gone to design school (luckily I avoided all that).

Along with Maeda’s « Design Machine », Pascal’s work clearly references things like Autoshop, Vectorama, Josua Davis (who already is a bit of his own « easydesign », isn’t he?), and anything Jürg Lehni (Hektor, Scriptographer, Lineto, Lego Font Creator). I’m sure there are others that should be mentioned, but I immediately think of those four.

• Installation : Déplacements (français) / Déplacements (english)
• Artist : Manuel Braun
• Exhibition : bbb, Centre régional d’initiatives pour l’art contemporain, Toulouse, France
• Dates : October 2006
• Developped at : Aix-en-Provence School of Art
• Video: Déplacements

This is a work that Manuel Braun developped for his Diplôme Nationale Supérieure d’Arts Plastiques in June, and which has just been exhibited in Toulouse at the Centre régional d’initiatives pour l’art contemporain. It is a 5 x 5 pixel array made out of computer fans. Each fan represents one pixel which together make a very singular display. On that display runs an artificial life program based on Coway’s famous Game of Life. It‘s a beautiful work, quite mesmerizing and yet very simple. When Brigitte Bosch from the bbb gave me « carte blanche » to select the work of a young multimedia artist for an exhibit she was preparing, I chose this work — principally because I wanted to defend a certain tendancy we currently have been exploring in the Atelier Hypermedia : i.e. the move away from purely screen based work by introducing visual algorithms increasingly into the phyiscal space. But I was also particularly happy with this work having watched Manuel’s research over the years on the infinite variations one can inflict on the idea of the « pixel ». I felt with this work that he had evolved from the research stage into a coherent phase plastique. And finally there is the fairly obvious (and humorous) reversal of the role of matérial/mimetic component, a sort of digital form of the old support / surface debate.

I don’t usually talk about other people’s work here, using this blog mostly as an easy form of communication. But I probably should talk more about my students’ and former-students’ works, as their work is so influential to my own, especially given the very particular structure of the Atelier Hypermedia. I’m also mentioning it here because this work was the first final-year diploma installation to use Processing and more importantly the PicoIP Processing library Stéphane Cousot and I developped last year for Jean-Pierre Mandon’s PicoIP project. When I look at the work we were doing with Macrodobe’s Director and the work we’re now doing with Processing, I think the change was definitely worth it.

• presentation: instruments + plateformes interactives
• speaker: Douglas Edric Stanley
• conference: Symposium Audio/Espaces/Réseaux
• organizers: Locus Sonus
• pdf: destanley.pdf

This is a recording of my presentation during the Symposium Audio/Espaces/Réseaux organized by Locus Sonus. In the accompagnying pdf file (destanley.pdf) you will find links to all of the films and interactive animations described during the talk. This talk is in French (why the hell am I writing this in English? I have no idea)

 

 instruments + plateformes interactives [01:06:34m]: Play Now | Play in Popup | Download