The Obsessive Drafter is a large autonomous robot arm that interpretively draws portraits of the people that approach him. Initiated and led by Guillaume Credoz who created the mechanical structure, and completed by collaborations with Nareg Karaoghlanian for mechatronics including mobility program, and myself for the sketch generation process.
Saint-Étienne Design Biennial 2017
March 9 until April 9 2017, Building H - Cité du design: TOD was exhibited as part of "Si automatique?" . During the month-long performance, it was to fill a 6 by 3 meters wall with sketched portraits of visitors…
StEt Panorama Photo: Panorama of 'Cité du design'
Sketch generation process
Using C# (visual studio 2013) and EMGU, an OpenCV wrapper.
The software will automatically capture the faces of visitors, through a webcam attached at TOD's wrist, then convert the image into a short sketch for feeding to the machine.

1) Computer Vision
I didn’t train my own sets, but used the default OpenCV eyes and faces haar cascade

YTD44EPcD5yngLoad video

OpenCV Face Detection Visualized - Adam Harvey

definitions that comes with the installation: A basic usage of those definitions alone returned a lot of false-positives .

Some filters were created:
  • Min & max dimensions
  • Mobility range check
  • Growth and shrinkage limits
  • Capture time
  • Face tracking
Here’s a capture testing session, against a 46 min documentary :
YTdJwODqj47AYLoad video
A face being tracked has a score that is updated each frame.

This score is then used while pooling for a portrait to sketch, including a final additional time filter: As the performance last a month, we initially intended for some of the captured visitors to still be around when their portraits are being sketched on the wall.


A sketch

Event captures
2) Sketch generation

The sketch routine was suggested to be a continuous circling line with variable diameter depending on the brightness of the underlying area.

The steering behavior of the pen tip was to be defined.

A quick prototype was created in AS3:
A wandering particle that is attracted by the color proximity of neighboring areas, in turn this particle influence the underlying area by progressively fading it out of existence.

Some parameters returned a cool (inefficient) contour tracer.

Gallery of results from initial prototype, using automated parameters space exploration:
  • Orbital wanderer
    New creatures
  • Orbital wanderer
    Parameters exploration
  • Orbital wanderer
    Parameters exploration

Orbital wanderer

Orbital wanderer
Another approach
A couple of different trials later, I decided to stop playing and define specific rules: Normalize and filter the capture image, create a dithered version, order the dither points to generate a path, detailing the picture along it.

Image filtering:
  1. Noise removal
  2. Masking against a pre-generated ‘face focus’ map.
  3. Image histogram equalization, using CLAHE .
  4. Minimizing large black patches
Edges are also extracted, using Hough transform, and were intended to add additional detailing around the center of the face: eyes, nose, mouth. But were not ultimately used.

Dither map:
  1. Used Floyd–Steinberg dithering.
  2. Probability of dither points based on distance from a central circle (fading out).

Filters

Ordering:
The line would have as many breaks as needed, otherwise it’s a traveling salesman problem!
Trial 1, minimum spanning tree:
Delaunay triangulation used to compute the euclidean minimum spanning tree of the dither points:
EMST
Trial 2, (simpler is better) weighted sort:

Weighted by distance (between vertices), hue, brightness and proximity to center. This will tend to build up the face in quasi-sequence by facial feature.

Each path node is encircled once, and the path can break when distances are above a threshold.

Below is one the final results, an unavoidable sketching the lovely Lena Soderberg:

YTowXyy5guzAsLoad video
Wall test

Wall filling & scenario:
A completed sketch is a normalized turtle graphic. It’s transformed to fit the next available cell on wall according to a running scenario, which data are written to disk.

The entire process runs on a windows laptop, the drawing commands get sent to the Arduino via serial communication.
Wall
Machine execution
The wall at the end of the month:
End of month wall
Photo by Guillaume Credoz.

Related

Another project featuring TOD's avatar: http://chadiik.com/?article=PublicCanvas