Empathic Painting: Interactive stylization through observed emotional state

1. Empathic Painting: Interactive stylization through observed emotional state Maria Shugrina , Margrit Betke , John Collomosse Computer Science Department, Boston University, Boston MA, USA. Computer Science Department, Boston University, Boston MA, USA. Department of Computer Science, University of Bath, Bath, England. NPAR 2006

3. Russell's 2D pleasure-arousal space

4. Facial action units (AUs)

5. Stroke placement • We begin by creating a colour band-pass pyramid segmentation of the source image using the EDISON algorithm. • For each layer, we first render the “interior” strokes of all regions, then the “boundary” strokes of all regions. • Brush strokes are formed using Catmull-Rom piecewise cubic splines, the control points of which are computed from the binary image of each region.

6. Interior Strokes • Scan-lines are then traversed, and strokes are started and terminated as region boundaries are encountered; the vertical interval between scan-lines during processing is proportional to stroke thickness. • Stroke control points are distributed uniformly over the stroke's length, and jittered via small translations to disguise the regularity of the stroke placement process. • Stroke colour is computed using the mean colour of the original image. • Stroke thickness is set on a per region basis, in proportion to area.

7. Boundary Strokes control points (magenta) • run-length compressed Freeman (chain) code • working set • the angle between adjacent control points rises above 50o, stroke is terminated chain code (cyan)

8. Rendering Parameters Rendering parameters were: p1 = 0:3, p2 = (-0.3, 0.0); p3 = 0.31, p4 = 0.0, p5 = 0.5 (left image), p1 = 0.0, p2 = (0.0), p3 = 0.0, p4 = 0.0, p5 = 0.0 (right image).

9. Region Turbulence(p1) The effect of the turbulence (p1) parameter on interior stroke regions

10. Tonal Variation (p2) • Happiness (pleasantness)---brightness (luninance), saturation • Arousal---saturation or hue • Calmness---blue [Wright] • Depression, cold---blue [Mahnke]

11. Tonal Variation (p2) G(x): greying U(x): un-greying D(x): darkening L(x): lightening T1(x): red colors are reddened and green(calm) is reduced T2(x): increase the blue

12. Stroke Denotation Style(p3-p5) A single brush stroke rendered with a range of undulation (p3) and jaggedness (p4) parameters. From left to right: (a) p3 = 0.0, p4 = 0.0, (b) p3 = 0.0, p4 = 1.0, (c) p3 = 0.7, p4 = 0.0, (d) p3 = 0.7, p4 = 0.8. We introduced p5 to dampen the effects of undulation (p3) on interior strokes.

13. Result- anger Russell's pleasure-arousal space p{1…5}= [0.64, (-0.86, 0.73), 0.62, 0.85, 0.58] The painterly output mirrors this through a general red-shift and luminance reduction of colours used in the rendering. Strokes become moderately undulated, and noticeably jagged and chaotic.

14. Result- sadness/despair p{1…5}= [0.33, (-0.96, -0.22), 0.31, 0.52, 0.24] The presence of user state in the third quad rant generates a colour shift towards the calmer blues and greens, with low arousal also manifesting itself through both low saturation and stroke undulation, as well as a much calmer, less chaotic background.

15. Result- jubilation p{1…5}= [0.87, (0.67, 0.73), 0.85, 0.47, 0.91] We see highly agitated and chaotic stroke placement, again reflecting high arousal scores.

16. Control data A neutral expression corresponding to approximately (0,0) in the Russell space (top-right) and the resulting neutral painting (bottom).