abstractmachine

Stéphane Cousot and I are announcing today the public availability of our OpenCV Library for Processing. Although the library has been ready (in various states of undress) for a few months now, we have been using the intervening time to learn more in-depth how OpenCV works, debug, simplify method calls, test the library in real-world situations, add various features, plan out features for future releases, and — most importantly — write coherent documentation for those Processing users discovering OpenCV for the first time. It might seem like a light start, given the limited number of functions we’ve made available from the impressive Intel library, but we wanted to make sure each component worked as promised. Also, we wanted to make working with it as painless as possible for Processing users, and follow the Processing logic of getting complex things done with a limited number of simple methods. And finally, we wanted to make sure it was stable enough in a real-world installation context.

Download link: here

For the features, you have internal (via OpenCV) and external (via Processing) capture, basic image treatment (threshold, comparison, extraction, etc), contour tracking, face & body tracking, and a few other little goodies thrown in here and there. So, as it stands, you can (for example), recognize someone’s face, grab the outline of that face, and go into the image data of that person’s face to extract the face data. Or, you could use infrared filters with lights pointed at or placed on your body (see below), a multi-touch surface, or some other artificial lighting condition to grab light blobs for finger or body-part tracking and use that data somehow in Processing. There are obviously many possibilities.

Some of the things you cannot yet do, and which we plan to add to the library: motion history images and optical flow (pixel tracking), kalman predictions, color tracking, histograms, and obviously the list could go on and on. A lot of these functions I already have working in OpenFrameworks for an installation (soon to be announced) which will be exhibited later this summer. So consider the current release a starting point, with what we believe is a fairly clean start, but we could be wrong on that. The code is open, so go in and dig around — perhaps you can give us some good advice or add to the code yourself.

Special note: this library will also work for pure Java work, and yes, there is Java documentation.

So, why did it take so long? Well… when I say that we’ve been busy testing it in laboratory and real-world instances, I mean it. I’ve gotten some mail on this recently, so I should make things a little clearer: if you ever wondered why I don’t post as much as I (or apparently some of you) would like, it’s because I’m busy elsewhere working on so many @#&*$% projects. I do not just work on my own projects and I am definitely not a full-time blogger : I teach, run an atelier, collaborate with other artists, do research, write, write code, consult, curate, and somewhere in there, I’m a dad for two lovely and brilliant young (or youngish) women. Since I don’t have a secretary, nor a double, that means some creative Douglas-time-sharing. So when I’m quiet here, it most certainly means that I’m busy doing one of these other things. And over the past few months, that has worked out to about 50% of my creative work involving OpenCV in Processing and OpenFrameworks.

And on Stéphane’s side, he’s been just as busy working over the past six months on a gazillion projects for various artists, art students, and researchers; and only a part of that work involved this OpenCV library.

So, what have we been doing with it? The library has already been used in numerous projects at the Atelier Hypermédia, in external workshops at schools such as the Institut d’Arts Visuels in Orléans, as a research tool at the DRII laboratory (Dispositifs relationnels : Installations Interactives) at the École Nationale Supérieure d’Arts Décoratifs in Paris, and in two public works, one an installation for Gamerz 0.2 and the other as a component of a haptic dance performance-dispositif by Wolf Ka and his studio. Finally, we used the library to prototype an urban-design project by Lei Zhao for the Studio Lentigo, Marseille although this project was eventually finished in OpenFrameworks due to the high video performance demands of the installation. So all in all, about a dozen different projects over the past few months.

Here are a few images/videos with links for more information on the author(s)/works:

• Wolf Ka + cie, Moving By Numbers (see link for video).

• Atelier Hypermédia, C’est toi la patate.

• Lei Zhao, Node City (follow link for more videos).

• Fabien Artal, Diplôme DNSEP (avec les félicitations du jury), L’école supérieure d’Aix-en-Provence. There is a video, but you’ll have to jump to 23:15 for Fabien’s installation.

• Students of the Institut d’Arts Visuels, Workshop Légerté + Nuit des musées, Orléans (follow this link for — very poor quality — video).

• I’ll leave off with these images from an installation Stefan Schwabe created with his collaborator Sebastian Neitsch in a public pool in Halle. As swimmers wade about, their movements are tracked by a camera and modify an image built out of 4 overlapping projectors, projecting onto the dome of the rotunda. It should be mentioned that, like Lei Zhao’s Node City, this piece used Processing only during the prototyping phase (the final work was created in vvvv). Nevertheless, Stefan & Sebastian’s project was an important one in our year-long experimentation with various forms of video surveillance in art and design installations. (See Stefan’s website for video of this installation).

Update: I used the wrong terminology. Oops. We decided to call this version v01, precisely to suggest that there is still much progress to be made. Previously I called it v1.0, which is a very different idea!

• Installation: C’est toi la patate !?
• Authors: Fabien Artal, J.C. Loewe, Raphaël de Staël, Amaury Tatibouët, Roman Viguier
• Direction: Douglas Edric Stanley & Stéphane Cousot
• Workshop: PlayVision
• Exhibition: Gamerz 0.2
• Production: Atelier Hypermédia, Aix-en-Provence School of Art, Transatlab

Here are some screenshots from « C’est toi la patate !? », the design-it-and-build-it-as-fast-as-you-can « installation ludique » we created in just under 4 days during the PlayVision workshop. The main goal of this installation was to use our still-in-construction Open Computer Vision library for Processing in a real-world context and see if it could hold up. Conclusion? So far, so good.

The idea is very simple. You stand in front of a screen which acts as a surveillance-system/mirror. If you hold a part of your body still, the computer will take a picture of it and detach it from the rest of your body. Once you have detached the body part, you can play with it by moving your body around. The installation stores many body parts, leaving a sort of weak memory trail which (re)connects the successive players to one other over time.

We called it an « installation ludique » in response to an idea put forth by Jean-Baptiste Labrune during the opening Transatlab brainstorming session. I had been talking about Huizinga’s and Caillois’ theories around games and play, and Jean-Baptiste suggested we also keep in mind Winnicott’s theories around free play. The idea of a game open enough for the rules to more or less emerge during gameplay appealed to us greatly. This is one of the reasons there is no real specific goal to our « game », outside of the fairly obvious constraints of the interactive algorithm itself and the behavior required to achieve whatever it is you want to do. Jean-Baptiste also insisted that this model of gameplay has a power relation: it could be suggested that those who generate the rules, are exceptions to them.

Unsurprisingly, most people have been playing with the mirror from a purely narcissistic desire: fascinated by the pure game of self-recognition. Mirrors, apparently, still fascinate us. However, very quickly (in fact, almost immediately), people figured out that you could tell stories with it, and even salacious ones at that. By detaching your fingers in a suggestive manner and pushing them into suggestive configurations of body parts, one can in fact construct a compelling little marionette play, albeit a somewhat simplistic one. Funny that those two configurations — reflection and storytelling — should both be so immediately compatible in an interactive context, so easy to tap into. There is a great word for this which I used back in my pre-doctoral thesis on interactivity, namely « relation ». Relation, in French, can mean both a relation (in the English sense of relationship) and the act storytelling, i.e. the process of relating something to someone else. Interactivity, I wrote back then, was often about relation: what story is constructed in the liminary space generated between the player and the machine?

Ok, so we’ve still got a lot of work to do from here on (like making a distributable library package, for example), but this evening Stéphane and I at least got a pretty decent little edge detection example working with OpenCV humming along quite nicely inside of Processing. So we’re ready for the workshop that starts tomorrow. Of course, there are other libraries that already do this inside of Processing, but this will open a lot of future doors, so we’re pretty happy. Here’s a quick snapshot before I go to bed:

• Workshop: playVision
• Organizers: Douglas Edric Stanley & Stéphane Cousot
• Location: Atelier Hypermedia, Aix-en-Provence School of Art
• Dates: January 8 - 18, 2008
• Partners: Transatlab, School of the Art Institute of Chicago, M2FCréations, Hacking Lab, Atelier 3D, Locus Sonus

I’m a little giddy this evening, because Stéphane Cousot and I finally got a first very-alpha version working of the OpenCV library for Processing. We’re building this library for a workshop that begins next week and will eventually finish with a game to be installed at the Gamerz 0.2 exhibit on January 15th in Aix-en-Provence. After that, we’ll join up with the other groups working within the context of the Transatlab workshops throughout all of January, and see if we can’t put our OpenCV library to good use in their projects.

As for the library itself, talking back and forth from Processing to OpenCV turned out to be, as expected, quite a pain. Basically, we have to write bridge-code to all the OpenCV functions that we want to use, and implement inside of the JINI-side of the library all of the code required for manipulating and storing the image analysis data. Yikes. In fact just getting that Java-C++ bridge right without any memory leaks or other hazards was 90% of what took us the couple of days we’ve put into it so far. I’m also a bit obsessed with keeping the code as fast as possible (i.e. with as little communicated data as possible between the two environments), since native processor speed is one of the reasons you’d want to work with a library like this with Processing.

The original plan was to base our work on the fine code created by those wonderful people over at OpenFrameworks, and just add some JINI wrapper for Processing. Yeah right. Turns out you can’t just shoehorn these things like that, so we had to go back to the Intel source code and work from there. All was not lost though: we’ve been able to use OpenFrameworks as a cheat-sheet, and more importantly to try to match their methods to ours. We couldn’t make it 100% compatible, so you won’t be able to just copy your Processing code over. But we’re at least trying to keep the two projects as close as possible, so that people can take the essential routines from Processing and port them over to OpenFrameworks without too much hassle. So far we’ve found it pretty easy to go in this direction (it’s a fabulous way to learn C++, by the way), so why not return the favor and get inspired by OpenFrameworks to generate some code for Processing for a change?

For those that might be interested in this library, know that we’re working on it, but also that we don’t have any specific time-frame for releasing it. As Stéphane and I did for the UDP library, we’ll just fine tune it until it works within the context of the atelier, and then Stéphane will probably clean it up and make it all spiffy as he usually does.

That said, if you want any specific functions from the OpenCV library to be put at the top of the list (i.e. while we still have time to work on this project), let me know — perhaps in the comments here? As I mentioned above, we’re beginning with a simple demo-to-demo match of the OpenFrameworks example, and then we’ll maybe roll in some of the face detection behavior currently offered in the Face Detection library by Jaegon Lee. That’s a big maybe, because that library already works. But in the end, since it’s merely just a wrapper to a specific Mac OS X port of OpenCV, it would probably be good to just go back to the original OpenCV framework for Mac+Win+Linux compatibility. Anyway, edge/blob detection & face tracking are the first two on our list, along with all the fun little extra gadgets that come with making those two aspects work. Any others?

As you may or may not have noticed, I was teaching last month at the Chicago Art Institute. I met some pretty amazing people while there, but the person who really stood out for me was the very charming Iraqi artist Wafaa Bilal (cf. http://www.crudeoils.us/) who was preparing an interesting online performance entitled Domestic Tension: all throughout the month he has been living at the Flatfile Galleries where — via a locally housed server — you can chat with him, watch a live video feed of his life in the gallery, and shoot at him with a collective remote controlled gun.

Now, if you somehow think this is either lame (your call) or disturbing (whatever), you should at least know that while Wafaa is enacting this mostly symbolic performance in (relative) comfort as an art student in the United States, the rest of his family is very much enduring the real deal back home — and with very real casualties. So while it is symbolic for us in one way, it is symbolic in an entirely different way for him.

It is also interesting to frame this performance within the larger context of the displacement of the American discourse on casualties and friendly fire in Iraq: there is no longer the whimpy media smoke-and-mirrors proxy-debate that skirted shamelessly around the issue of cadavers within the video frame. The original debate (constructed pro and con by the pentagon) on how to honor lost american solders, has now transformed into a count not only of full cadavers and dead football players, but of all the missing body parts that will never be coming back. And on the backs of those incomplete bodies, we are finally feeling the weight of the enormous the narrative of the Iraqi body count, and particularly the breadth of this body count, i.e. it’s no longer about numbers, it’s about demographics.

Although I tend to yawn at on-line performances, especially heavy-handed ones, somehow I fell for this one. I suppose it’s the ambiguity of the whole thing (and the good nature of Wafaa) that warms me to it. Of course this work references some far more hardcore pieces of Chris Burden such as when he locked himself in a locker for several days, shot at planes, or had a collegue shoot him in the arm. Wafaa is also making some very broad strokes towards other famous works in the construction of an American mythology, for example Beuys’ I like America and America Likes Me.

But when it comes down to it, I have actually only seen the Burden performances via short crappy videos which grow their semiotic gravitas precisely out of of the crapiness of the video qua deficient document. Much of the 60’s and 70’s performance art has been tainted by this documentation process. I’m thinking precisely of the spic-and-span Los Angeles exhibit last year at the Pompidou Center where the contrast was particularly annoying. Coming back to Wafaa’s current plight under the gun makes me wonder to what degree the crappy webcam refreshing every n seconds helps to construct the mythology of the performance.

I had promised to Wafaa that I would post something about this performance, but I couldn’t get a decent enough connection until now to check it out myself. Watch the following video from Day 8 to get a sense of their troubles keeping the gun online:

• Conference: Douglas Edric Stanley
• Date: Tuesday April 17, 2007
• Location: School of the Art Institute of Chicago
• Room: 417
• Address: 112 S. Michigan
• Time: 12:00 - 13:30

I’ll be giving two public talks next week. On Tuesday, I’ll be speaking at The School of the Art Institute of Chicago where I’ll be focusing more on the Atelier Hypermédia and our work with coding as plastic medium. I’ll also try to find some time to talk about Plot and our postulate on slow real-time systems.

• Conference: Douglas Edric Stanley
• Date: Friday April 20, 2007
• Location: Gosia Koscielak Studio and Gallery
• Address: 1646 N. Bosworth Ave; Chicago, Il 60622
• Telephone: 847.858.1540
• Time: 19:00 - 20:00

And on Friday, I’ll be speaking at Gosia Koscielak Studio and Gallery where I’ll probably do more of the abstractmachine dog and poney show.

• Visiting professor: Douglas Edric Stanley
• Location: School of the Art Institute of Chicago
• Dates: 16 April - 20 April 2007
• Conference: To be announced

On Friday I’ll be flying to Chicago where I’ll be teaching at the School of the Art Institute of Chicago as part of our Transatlab between Chicago, Nice and Aix-en-Provence. I’ll be rebuilding my hypertable there, and if I have time I’ll show the students how to do FTIR tracking, as well as object tracking with ReacTIVision (using Processing, PureData, etc).

/* //////////////////////////////////////////////////////////////////////////////////////////////// */

After Chicago, I’ll fly to Montréal for two days where I have been invited by Jean-François Renaud to visit the University of Montréal à Québec and more specifically the École des médias. I’ll be in Montréal from the afternoon of the April 24th to noon April 26th. If you want to meet, show me work, or know of anything interesting going on, let me know.

This is an itsy-bitsy post. Just to let people know that Antonin Fourneau and I will be speaking tomorrow (damn! there we go again, make that today) through visioconference and various high-tech remote technologies (i.e. Ben Chang’s hand) to the students of the School of the Art Institute of Chicago as part of the Transatlab. We will be speaking at 17h00 Aix-en-Provence time, 10 AM Chicago time. Here’s the official announcement:

By day, the Atelier Hypermedia explores the relationship of algorithms to plasticity, and situates itself somewhere at the intersection of generative cinema, modular imagery, interactivity, robotics, simulation, physical computing, networks and play. On off-days it doubles as theoretical partner in the investigation of post-cybernetics and slow real-time systems as the research group PLOT. And by night it wanders further into unknown territories via the monster that is « ENIAROF » — a singular form of attraction for the emerging digital_kaiju culture. At exatly 48 hours before the unveiling of its latest orchestrated public accident, Douglas Edric Stanley and Antonin Fourneau will present « The-Thing-That-Was-Called-ENIAROF », along with other notable productions of the Atelier Hypermédia, though a video conference with the School of the Art Institute of Chicago. November 29th, 2006. 10:00 CST / 17:00 CET.

Oh, why « Transatlab » ? Well, of course, officially we were thinking of something like this:

But in reality, it’s probably something more like this:

…and maybe even this:

Still in preparation for ENIAROF 0.2, and while we’re waiting for things to start, my collegue Ben Chang over at the Chicago Art Institute sent me to Tale of Tales’ Realtime Art Manifesto [link]. He suggested it in relation to our DOGMeNIAROF, and indeed there are many similarities. But they’ve got even better slogans: « Do not render! », « Be a dictator », « Interactivity wants to be free », and other semantically ambiguous slogans. And then there is the last one, which ties in very well with the whole ENIAROF philosophy: « Develop a punk economy ».

In fact, on this last idea is absolutely something that inspired ENIAROF. We’ve been talking quite some time in the Atelier about punk rock, and how it developped not only its own community but its own economy. Of course it got co-opted, but I remember fondly as a youth things such as the Gilman Street Project (which still exists I’m told), along with MAXIMUMROCKNROLL which was always payable in cash. The shows and magazines were cheap, but you still paid for them. They were also easily sneak-in-able and photocopyable, which brings us also to digital distribution and filesharing. These two things are not mutually exclusive. Often, my students confuse anti-globalization with issues of monetary exchange (of any kind), whereas it’s far more instructive to look at the economies of scale rather than getting bogged down on whether or not to pay people for stuff-you-want™ anyway. They looked at me a little screwy last week when I suggested that it’s a good idea to sell your music and software for a few bucks. We were discussing these issues in relation to the recent Microsoft XNA Game Studio announcement for independent developers. They just snickered, « sure iTunes is cool, but it’s always cheaper on Emule ».

« Sell your games! »

All that said, I disagree when it comes to Tale of Tales’ « Game art is slave art » : « Make art-games, not game-art. Game art is just modern art - ironical, cynical, afraid of beauty, afraid of meaning. » I don’t know who they’re referencing here, but I can think of a few, and if so, I don’t agree. While I’m sympathetic to their attitude there, I think there is still a lot of fun to milked out of irony and cynicsm. It’s a little too easy to embrace the populist spirit 100% - down with sophistication and all that hogwash. That’s why I like the ENIAROF spirit - we can be highbrow and lobrow and whatever. What I personally find so interesting about the emerging Kaiju-culture is how it’s both cynical/ironic and fun. Like money and sharing, these two terms are not mutually exclusive.

Update: I forgot to mention that Tale of Tales exhibited their Endless Forest at the ZeroOne festival this summer. It looked very pretty but I didn’t really have time to play with it, unforunately.

• Conference: Virtual Artist Lecture Series
• Speakers: Peter Sinclair, Robb Drinkwater
• Locations: L’école supérieur d’art d’Aix-en-Provence, Chicago Art Institute
• Date: 31 March 2006
• Reference: FACE (French American Cultural Exchange), SAIC, Locus Sonus
• Photos: Marion Bonnefoy & Douglas Edric Stanley

Just a quick note to mention that, no, I have not dissapeared, it’s just that I’m working on two BIG PROJECTS®™ that you will hear more about at some other point. Yes, both have a lot to do with code. The real big project is of course a doctoral thesis, and that means as little of me gets around as possible. I’ll also post more soon about the two Processing libraries Stéphane Cousot and I are finishing up.

In the meantime, a quick reference the the exciting-event-of-the-day, i.e. a networked cross-conference between the Chicago School of the Art Institute and the Aix-en-Provence Art School. We already met the Chicago croud in Aix-en-Provence, France Cadet just got back from a trip over there, some of our students are currently visiting, and now we’re getting to work on the networked aspect from our respective huts. I’m also supposed to jump over the big lake, but I’m too busy right now, so they’ll have to settle for my pixeled mug.

So for today’s inaugural conference, my collegue Peter Sinclair discussed his work with Locus Sonus and quickly demonstrated the current streaming/sampling Pure Data patch they’re currently working on for Locus Sonus. On the Chicago end, Robb Drinkwater talked about Supercollider (love that website!), and showed us a Processing meets supercollider demo he prepared for Dorkbot

Since the Aix antenna set up shop in the Atelier Hypermedia, and since the other free translator was already talking, it was my turn to translate what was being said (oh, simultaneous? joy!).

I’m the next speaker on the list (gotta finish a research presentation for April 5th, so I was outta this round), probably on April 27th. But I’ll have to confirm that later when I have something solid. Peter Sinclair suggested that we multicast my conference with Quicktime Broadcaster, meaning that whoever wants to, will be able to listen in. I’ll be giving a brief overview of the history of the Atelier Hypermedia, how we got where we’re at, why we don’t know where we’re going, and why we try to make it look fun. If you don’t understand French, it might be one of the few occasions where you can see me speak. More on that later.

Ultimately, these crossed conferences will be followed by networked collaborative work between Aix-en-Provence and Chicago. For that, Ben Chang and I will probably be building a Hypertable in Chicago soon and see what they can do with it.