Over the past year I have been working for Seconde Nature designing a public multi-touch platform via my Atelier Hypermédia in Aix-en-Provence. It’s a fairly ambitious project and involves many partners and most importantly, a whole bunch of students and researchers from five different schools/departments exploring interactivity from the perspectives of art, design, architecture or some combination therein. While I’ve been tooling away at the project in some form or another over the past 12 months, the production team was officially formed at the beginning of 2010 and still has about 6 months to go before completing the project with an exhibition planned for early 2011 in Aix-en-Provence and Marseille. In other words, we’re only at the half-way point and anything you see here should be considered highly work-in-progress, and purely experimental/speculative in nature.
That said, we have amassed enough material from the exploratory/workshop phase of the project to create the following document which can be considered a collection of ideas that we found interesting enough to record and group into this 20-minute demo reel. There were actually far more ideas explored than those you will see in this reel but they were unfortunately either lacking decent documentation or were simply too preliminary/unpolished. That said, many if not all of these ideas will need to be completely reworked during the production phase of the project. Only a few will be retained and most of those will be redesigned in collaboration with our content partners.
All of the projects were built within one of the four workshops, with each workshop lasting either 1 or 2 weeks. In total, this reel represents the accumulation of about 6 weeks of direct prototyping.
Quick post to link up to the Gamerz 05 video that just went live and where you’ll find a few seconds of my bitPong installation. I also shot my own video of the piece, in which I take advantage of this little known device called a tripod, but since I still haven’t had the time to edit it I shouldn’t complain. When I finally find the time to link up the gazillion things I’m working on right now, you’ll hopefully understand why the silent treatment. Until then, here’s the video, with some of that all-around-sloppy-soup that we have come to know as Jankenpopp:
I have finally found a venue to show a decent working version of bitPong, a piece I created some time ago when I was still working on the close relationship with phsyical implementations of data and their aesthetic consequences.
The idea is simple: a two-player game, based on the uber-referenced Pong, here played with 8-bit controllers. When I say « 8-bit controller », I mean literally 8-bit, i.e. 8 buttons, each representing 1-bit of data. Collected together, this byte represents a 256 value variable which is used to control a visual paddle representation within the game. To aid players in the conversion of 1-bit discrete switches into their collective base-two 8-bit value, each button has been labelled: 2^n, i.e. two to the power of zero, two to the power of one, two to the power of two, and so on. This is otherwise known to mere mortals as the values 1, 2, 4, 8, 16, 32, 64, and 128. To move your paddle, you must add each of these values together in order to position it somewhere between position 0 and position 255.
For those who know little about how the computer works internally, this is how the computer moves from the well-known binary 0 | 1 value, to complex values such as the letters you are reading right now: by associating a different value to each bit (the 1 values of « 0 1 0 1 0 1 » get converted to « 0+2+0+4+0+8+0+32 », otherwise known as the value 42) the computer can use a physically limited scheme (0 or 1, on or off, yes or no, true or false) in order to represent a far greater sum of possibilities (here a number from 0 to 255). bitPong plays off of this configuration and brings its dynamics to the surface of the playing field. In order to take control of your paddle, you will have to quickly master binary encoding.
In this Victor Vasarely inspired version of bitPong, hexagons populate the playing field and create an added diversion. Therefore, bitPong has now turned into something like a two-player bitBreakout. I was actually inspired by the following sign which is posted on the wall just next to my installation, indicating the escape routes out of the museum.
I’ll be travelling tomorrow to Poitiers for what looks like a very rich roster of speakers discussing… oh yes… the subject of interactivity. Cough.
Oh, and apparently Ségolène Royal will be giving an opening pep-talk (oui, oui, that Ségolène Royal), which probably has something to do with the fact that she is currently the president of Poitou-Charentes where the conference is being held. You might also have noticed that she is currently making a bid to for the leadership of the French Socialist Party, so I don’t know how much to bet on her appearance.
I haven’t completely finished my talk yet, but from what I have so far, it looks like I’ll be sticking with this resumé that I sent a few weeks ago to the organizers:
L’image du monstre
Il y a trois ans, lors d’un précédent colloque à l’ÉESI sur le cinéma et l’interactivité, j’ai argumenté pour une approche “hydraulique” de l’image en mouvement : une approche dynamique autour d’une image fluctuante qui prendrait en compte notamment la fluidification que les machines algorithmiques apportaient à l’image. C’était une hypothèse intéressante, mais qui n’osait pas aller jusqu’au bout. L’épine du problème était une insistance à maintenir notre relation nostalgique avec la trace photographique à l’intérieur de l’image, face à l’horizontalité des nouvelles formes de stockage comme les bases de données qui ont tendance à brouiller les figures qui s’y trouvent.
Depuis, mon optique s’est totalement transformée. L’objet n’est plus pour moi un simple jeu de re-juxtaposition permanente, il est devenu un jeu de mutation, avec des images-croissance qui poussent à partir de n’importe quelle extrémité de la « Chose ». Il se peut qu’il y ait encore des traces anciennes dans cette image, mais ces traces jouent un tout autre rôle, et nourrissent la bête tout autrement. Je vois désormais dans cette image nouvelle une forme de « monstruosité » qui pousse à l’intérieur des images, et descend jusque dans les entrailles du GPU lui-même, ne remontant à la surface de l’écran pixelisé que le temps d’un court affichage.
Accepter le monstre dans l’image, transforme notre approche de celle-ci, et transforme aussi ce qu’on entend par figure, mimesis, et enfin narration. Cela change également les champs d’exploration qui permettent de saisir plus fermement les phénomènes que je considère comme les plus pertinents pour ces transformations, à commencer par les jeux vidéo.
Here is the symposium’s valiant attempt at an English translation, which makes absolutely no sense to me, and I wrote the damn thing. The words are right, it’s just that the meaning got lost in there somewhere. Apparently, my French is hard to translate, or perhaps just plain hard to understand:
Three years ago, during a previous conference on cine-film and interactivity at the ÉESI, I put forward the outline for a “hydraulic” approach to image in motion: a dynamic approach hinged on the fluctuating image ,which, notably, could factorise the fluidising import that algorithmic engines have brought to the image. It was an interesting hypothesis, which was just not bold enough to go all the way. The bane of the problem being insistence on maintaining our nostalgic affinity with the photographic trace within the image, at the hands of the horizontality of the new storing configurations, like those involving data bases, which tend to scramble the figures present.
Since then my assessment has been turned around. I no longer view the object as just a game of constant re-juxtaposition; it has become play on mutation, with image-growth sprouting from just about any appendage of the “Thing”. It is just possible that old traces still linger in that image, now however, they play a completely different role and feed the beast with different fodder. In this novel image, from now on, I can see a form of “monstrousness” germinating within the image, and getting right down to the entrails of the GPU itself, coming up to the pixelized surface of the screen for only a brief moment of display.
By accepting the monster in the image our approach to it becomes transformed, thus transforming that which we understand as figure, mimesis and finally narration. It also changes fields of inquiry which sanction and capture phenomena more securely and which I consider as being the most relevant for these transformations, starting with video games.
Stéphane Cousot and I are announcing today the public availability of our OpenCV Library for Processing. Although the library has been ready (in various states of undress) for a few months now, we have been using the intervening time to learn more in-depth how OpenCV works, debug, simplify method calls, test the library in real-world situations, add various features, plan out features for future releases, and — most importantly — write coherent documentation for those Processing users discovering OpenCV for the first time. It might seem like a light start, given the limited number of functions we’ve made available from the impressive Intel library, but we wanted to make sure each component worked as promised. Also, we wanted to make working with it as painless as possible for Processing users, and follow the Processing logic of getting complex things done with a limited number of simple methods. And finally, we wanted to make sure it was stable enough in a real-world installation context.
For the features, you have internal (via OpenCV) and external (via Processing) capture, basic image treatment (threshold, comparison, extraction, etc), contour tracking, face & body tracking, and a few other little goodies thrown in here and there. So, as it stands, you can (for example), recognize someone’s face, grab the outline of that face, and go into the image data of that person’s face to extract the face data. Or, you could use infrared filters with lights pointed at or placed on your body (see below), a multi-touch surface, or some other artificial lighting condition to grab light blobs for finger or body-part tracking and use that data somehow in Processing. There are obviously many possibilities.
Some of the things you cannot yet do, and which we plan to add to the library: motion history images and optical flow (pixel tracking), kalman predictions, color tracking, histograms, and obviously the list could go on and on. A lot of these functions I already have working in OpenFrameworks for an installation (soon to be announced) which will be exhibited later this summer. So consider the current release a starting point, with what we believe is a fairly clean start, but we could be wrong on that. The code is open, so go in and dig around — perhaps you can give us some good advice or add to the code yourself.
Special note: this library will also work for pure Java work, and yes, there is Java documentation.
So, why did it take so long? Well… when I say that we’ve been busy testing it in laboratory and real-world instances, I mean it. I’ve gotten some mail on this recently, so I should make things a little clearer: if you ever wondered why I don’t post as much as I (or apparently some of you) would like, it’s because I’m busy elsewhere working on so many @#&*$% projects. I do not just work on my own projects and I am definitely not a full-time blogger : I teach, run an atelier, collaborate with other artists, do research, write, write code, consult, curate, and somewhere in there, I’m a dad for two lovely and brilliant young (or youngish) women. Since I don’t have a secretary, nor a double, that means some creative Douglas-time-sharing. So when I’m quiet here, it most certainly means that I’m busy doing one of these other things. And over the past few months, that has worked out to about 50% of my creative work involving OpenCV in Processing and OpenFrameworks.
And on Stéphane’s side, he’s been just as busy working over the past six months on a gazillion projects for various artists, art students, and researchers; and only a part of that work involved this OpenCV library.
So, what have we been doing with it? The library has already been used in numerous projects at the Atelier Hypermédia, in external workshops at schools such as the Institut d’Arts Visuels in Orléans, as a research tool at the DRII laboratory (Dispositifs relationnels : Installations Interactives) at the École Nationale Supérieure d’Arts Décoratifs in Paris, and in two public works, one an installation for Gamerz 0.2 and the other as a component of a haptic dance performance-dispositif by Wolf Ka and his studio. Finally, we used the library to prototype an urban-design project by Lei Zhao for the Studio Lentigo, Marseille although this project was eventually finished in OpenFrameworks due to the high video performance demands of the installation. So all in all, about a dozen different projects over the past few months.
Here are a few images/videos with links for more information on the author(s)/works:
Lei Zhao, Node City (follow link for more videos).
Fabien Artal, Diplôme DNSEP (avec les félicitations du jury), L’école supérieure d’Aix-en-Provence. There is a video, but you’ll have to jump to 23:15 for Fabien’s installation.
Students of the Institut d’Arts Visuels, Workshop Légerté + Nuit des musées, Orléans (follow this link for — very poor quality — video).
I’ll leave off with these images from an installation Stefan Schwabe created with his collaborator Sebastian Neitsch in a public pool in Halle. As swimmers wade about, their movements are tracked by a camera and modify an image built out of 4 overlapping projectors, projecting onto the dome of the rotunda. It should be mentioned that, like Lei Zhao’s Node City, this piece used Processing only during the prototyping phase (the final work was created in vvvv). Nevertheless, Stefan & Sebastian’s project was an important one in our year-long experimentation with various forms of video surveillance in art and design installations. (See Stefan’s website for video of this installation).
Update: I used the wrong terminology. Oops. We decided to call this version v01, precisely to suggest that there is still much progress to be made. Previously I called it v1.0, which is a very different idea!
The abstractmachine is setup in San Jose, there is a breakcore Rubik’s Cube® kicking out the jams, a programmable video mosaic recorder is open for public abuse, and the Hypertable is unleashing a non-linear interactive documentary containing a telepathic virus. We’re setup in the main exhibition hall for the festival, South Hall.
Oh, I should also mention something after having watched a few visitors this afternoon: um, hello, people out there, yes, you can actually pick up the Rubik’s Cube® and play with it. And that interactive table, yes, you can put your hands on it. Most of you reading this are probably in the know, and would find it laughable that people would fear interacting with an installation during a festival dedicated to digital art. But hey, that’s apparently America. It’s my first show in my home country (yes, that’s right) and I guess the public is just like that. I always figured that the dopey Americans who looked confused were just like that because they were tourists lost in Europe. But who knows, maybe there are just a lot of tourists showing up today.
So with that out of the way, here’s the official statement.
Through various experiments, installations, and online software, the abstractmachine project asks the question of how we as artists and users can create, manipulate, and ultimately enact digital algorithms. If the specificity of the computer comes not only from it’s digital aspect, but even more so from it’s algorithmic aspect, how does this hyperprogrammable nature transform the media we manipulate — i.e. the images and sounds we design using these machines? Amongst the many machines available within the abstractmachine project, two creation platforms will be presented to illustrate our response to these questions: one dedicated to the creation and manipulation of algorithmic cinema, the other designed around algorithmic musical composition.
« 3 », a.k.a. « ^3 », a.k.a. « cubed » is a musical sequencer integrated into a Rubik’s Cube®. By manipulating the colors on the cube, users generate different sound algorithms within the sequencer. Using specially-designed soundfonts from Jankenpopp (cf. http://jankenpopp.com), math geeks can finally become the speedcubing breakcore supernerds they always feared were lurking underneath. With ^3 we are working against the idea that a musician has to create music with audio software where building the musical algorithm and manipulating the digital algorithm are two different processes. Often, making digital music looks a lot like someone working on their spreadsheet. In ^3, all of the notes of the musical process are visible and intrinsically intertwined. Using a universally known interface, a series of simple gestures cascade into a complex multitude of musical possibilities.
Concrescence is a platform for creating and manipulating moving images outside of the traditional linear time-code. Images grow in spatialized mosaics, allowing for infinite recomposition while avoiding purely random associations. This specialized software is then projected onto the Abstractmachine Hypertable: a multipurpose interactive table which allows multiple users to interact with the non-linear narratives by simply placing their hands on the surface. For the San Jose festival, two uses of the Concrescence platform will be presented: a fully developed algorithmic narrative entitled “The Signal”, accompanied by a simplified version of the Concrescence authoring software where the public can record their own audiovisual clips and create collective non-linear patchworks.
Concrescence was developed in France with assistance from the following institutions: ARCADI, DICREAM, SCAM, and was produced by the CIREN. All sounds for The Signal were designed by Julien Hô Kim, with a narration by Keith Evans. The Jankenpopp666 soundfont can be downloaded at http://jankenpopp.free.fr/666/
Ok, so 8=8 has returned back from Nantes. Ooops! That was a mistake! Scopitone is one of those amateurist multimedia festivals I have been complaining about recently. Actually, the festival is broken into two, with the whimpy multimedia selection during the day coupled with a brilliant and adventerous musical lineup at night. So it’s basically one of the best French festivals for electronic music with one of the crappiest non-selections of multimedia art. I don’t know what went wrong, but after five years you either have it or you don’t. If this year is any indication, they definitely have it all wrong and need a serious reboot. In typical fashion, yet again all the multimedia artists got squashed in the planning by by the disproportionate focus on (big name) musical acts. And of course, in spite of the fact that 8=8 is a musical performance guess where we were stuck performing? Sigh
Despite extremely poor conditions, we actually rocked the house. Don’t just take my word for it. Yeasterday, Marie Lechner wrote on 8=8’s performance in Libération and seemed quite happy with us. You can read the original article here: Nantes bombardé d’électro. Here is an excerpt of the part about us:
“Mécaniques infernales. On aurait aimé voir dans le [Scopitone Soir] 8 = 8, un dispositif où sons et images sont générés simultanément par simple déplacement des mains au-dessus d’une table basse. Pour découvrir l’instrument audiovisuel imaginé par Douglas Edric Stanley, il fallait se rendre aux Ateliers et Chantiers de Nantes qui accueillent le Scopitone «jour» et son lot d’ateliers et d’installations interactives. Assis autour de cette hypertable, avec trois autres performeurs (TM, Nao, JankenPopp), ils activent des univers punkoïdes, entre Donkey-Kong distordu, match de foot abstrait, arc-en-ciel déviant et mécaniques infernales.”
As Marie mentions in her article, we would have been better served as simply one of the opening acts for the nighttime concerts (more on that later). 8=8 was designed for a concert setting, although off the main stage. It was not designed as an installation, we already know how to do that, thank you very much. Despite our protests, the festival director begged us to present 8=8 as a daytime installation + performance, claiming that there wasn’t really anywhere for 8=8 to perform in the evening setting. But once we actually got on-site we realized how much we had been screwed, as there were several places/times we would have fit in fine. A truly shameful lie, especially considering he was with me in Marseille when I — sucessfully, albeit with much difficulty — fought against the tendencies a previous music festival had of treating multimedia performances as nothing more than an entertaining sideshow.
To further add insult to injury, the sound system sucked — really sucked — as did the acoustics, whereas at the night setting the sound was f§@#&?! brilliant.
Want the real story? In reality — according to a local — there wasn’t enough peanut gallery material to amuse the public and justify public funding, so 8=8 got sacrificed as a pleasant curiosity for your afternoon stroll. It’s true, 8=8 is fun for the public, but if we wanted to design it as an installation we would have designed it very differently. Ho hum there we are yet again distracting the public with amusing gadgets you can fiddle with. We really have to stop this dangerous cycle, it’s making us look like idiots…
If I could make a public plea to Scopitone: do what you do best — music — and just drop the multimedia part all together. That, or be more honest with your artists. Slapping together a couple minor installations with little to no means, and even worse no curatorial vision, just gives digital arts a bad name. The scary part is that they are currently renovating a former warehouse to make room for a permanent cultural center dedicated to multimedia art. Yikes! They better get a serious artistic director, and quick. It would be even better if they found someone who has travelled beyond the infamous invisible wall that protected France from Chernobyl.
That said, the public was very enthusiastic, and 8=8 actually started feeling more and more like a real group. I think we could actually do something with this ragtag band. I had fun spending time with my fellow performers, but I know it’s a pleasant illusion, and that I’ll always be the old professor who will eventually have to be carted back into his study. In the end, we knew that we would have a good response, hence our willingness to bore the weight of shitty conditions: ultimately we do enjoy meeting cool people and playing our programs for/with them. And for that, at least, the trip was a success — cheers to all the enthusiastic people we met, by the way…
Here are some photos of that “Punkoid universe, between distorted Donkey Kong, abstract football, deviant rainbows and infernal machines,” Marie described so well. All the photos were taken by Thomas or else by some intoxicated stranger ressembling one of the members of 8=8.
As previously mentioned, the performance schedule was ”to be announced”. It turns out we will be performing at 11h30 and at 17h for the two days of the festival.
