From Light to Enlightenment

1. From Light toEnlightenment Edition Honours Class ‘Big Images’ TU/e 2014 Kees van Overveld -1-

2. From Light toEnlightenment Introduction: what is looking? Exercise1. • Describe in at most one sentence (<20 words) the essence of what you will see next. ‘I see …’ Kees van Overveld -2-

3. Kees van Overveld -3-

4. Kees van Overveld -4-

5. From Light to Enlightenment Introduction: what is ‘looking’? Anwers: • I see the light of the beamer being reflected from the screen • I see a distribution of light shades in the middle, brownish near the borders • I see mainly smooth color distributions, granular in the middle and patches near the borders • I see few light, rounded, symmetric 2D shapes in the middle and a rounded triangle in the lower left • I see a roughly spherical shape in the middle and few flat, laying 3D shapes underneath • I see a cup of cappuccino and a newspaper • I see the cup being almost full and the newspaper not (yet) opened • I see the careless beginning of a promising holiday in Italy Kees van Overveld -5-

6. From Light to Enlightenment Introduction: what is ‘looking’?? Answers: • I see the light of the beamer being reflected from the screen • I see a distribution of shades of grey • I see mainly smooth distributions of grey, granular in places • I see few dark lines, few swirls, blotches and scratches • I see a – presumably – flat surface with some black shapes in it • I see some traces of elementary particles in a bubble chamber • I see a reaction between sub-atomic particles with various charges and masses, with lacking momentum • I see the first ever empirical evidence of a neutrino Kees van Overveld -6-

7. From Light to Enlightenment Introduction: what is an ‘image’? Exercise 2. • For the images of exercise 1, explain where they reside. • Hint: there are at least 10 different correct answers. Kees van Overveld -7-

8. From Light to Enlightenment Introduction: what is an ‘image’? Answers: • In the museum and in the bubble chamber at the instance of the nuclear reaction, respectively • In former downtown lunchroom Peacock (‘Heuvelgalerie’), where I took the photograph, and the Wikipedia archive, respectively • At the hard disk of my computer • In the beamer • In the space between the beamer and the screen, or between the screen and your eyes • In your eye • In your retina • In your brain • In your mind • In the sound waves in this room while we are talking about them Kees van Overveld -8-

9. From Light to Enlightenment Introduction: what is an ‘image’? We are certain that an image may reside in our head (‘I dream therefore I see’: immediate access to our internal virtual subjective omnimax theatre) All other answers apply only under certain circumstances So: the only thing that holds with certainty for each image, is that it has a mental, and therefore subjective representation. Kees van Overveld -9-

10. Mutually parallel or perpendicular lines or curves? From Light to Enlightenment Introduction: what is an ‘image’? Twoproblems: • We don’tknowhowsomething looks in reality -10-

11. From Light to Enlightenment Introduction: what is an ‘image’? Two problems: 2. We don’t know what is to be seen in someone else’s private theatre Kees van Overveld -11-

12. How could we ever intersubjectively know something about images??? From Light to Enlightenment Introduction: what is an ‘image’? Kees van Overveld -12-

13. From Light to Enlightenment Introduction: what is an ‘image’? Answer: Thanks to the miracle of equivalence and the tendency of clustering, innate in our brains. Kees van Overveld -13-

14. These are more similar … .. .than these From Light to Enlightenment Introduction: what is an ‘image’? Answer: Thanks to the miracle of equivalence and the tendency of clustering, innate in our brains. ‘being similar’ ’color’ Kees van Overveld -14-

15. These are more similar … … than these From Light to Enlightenment Introduction: what is an ‘image’? Answer: Thanks to the miracle of equivalence and the tendency of clustering, innate in our brains. ‘being similar’ ’shape’ Kees van Overveld -15-

16. … than these These are more similar … From Light to Enlightenment Introduction: what is an ‘image’? Answer: Thanks to the miracle of equivalence and the tendency of clustering, innate in our brains. ‘being similar’ size’ Kees van Overveld -16-

17. From Light to Enlightenment Introduction: what is an ‘image’? Preliminar conclusion: A visible property (‘color’, ‘shape’, ‘size’, …) is the same thing as ‘a way of clustering’ or an equivalence relation Kees van Overveld -17-

18. From Light to Enlightenment Introduction: what is an ‘image’? Q: what is an equivalence relation? A: a statement about relating two elements in a set, e.g. : ‘equally heavy’, ‘having the same father’, ‘is connected to’, … … where this relation is Reflective M(a,a) Symmetric M(a,b)  M(b,a) Transitive M(a,b) & M(b,c)  M(a,c) Kees van Overveld -18-

19. The class of all sets containing three elements, simply called ‘THREE’ The class of all sets containing four elements, simply called ‘FOUR’ From Light to Enlightenment Introduction: what is an ‘image’? Example (‘same amount’): ~ ~ ~ … ~ ~ ~ … Kees van Overveld -19-

20. From Light to Enlightenment Introduction: what is an ‘image’? So: Equivalence relations bring forward classes of elements, where all elements in a class are mutually equivalent, so called Equivalence classes. Equivalence classes are disjoint and covering. An equivalence class is a convenient way to define something abstract, such as THREE or FOUR, being equivalence classes of ‘equally many’. Kees van Overveld -20-

21. From Light to Enlightenment Introduction: what is an ‘image’? Apply to images: • ‘Looks similar to’ is (almost) an equivalence relation. • ‘Looks similar w.r.t. color’ has equivalence classes RED, GREEN, … etc • ‘Looks similar w.r.t. shape’ has equivalence classes ROUND, SQUARE, … etc • ‘Looks similar w.r.t. size’ has equivalence classes LARGE, SMALL, … etc Kees van Overveld -21-

22. … does not look similar to VARIANTS: the variety within one equivalence class INVARIANTS: that what distinguishes one equivalence class from another ..similar to.. Similar to Similar to Similar to The terminology to argue about properties of images From Light to Enlightenment This is an Inevitable shortcoming of our brain (and every measuring instrument), but at the same time an evolutionary advantage, provided that cluster boundaries have evolutionary meaningful interpretations Introduction: what is an ‘image’? Apply to images: • ‘Is similar to’ is (almost) an equivalence relation …. … but not quite: transitivity only holds in approximation. … this is also true for textures, shapes, 3D surfaces, objects and relalions and meaning Kees van Overveld -22-

23. From Light to Enlightenment Introduction: what is an ‘image’? Advantage of this ‘trick’ (=describing visible features in terms of equivalence relations): We don’t need to bother about the essential meaning of ‘red’ (just as we don’t need to bother about the essential meaning of ‘three’). In stead, we can concentrate on • interpretation of ‘is similar to’: kinds of similarities • identifying variants • identifying invariants Kees van Overveld -23-

24. From Light to Enlightenment Introduction: what is an ‘image’? There are many interpretations of ‘is similar to’ or ‘makes me think of’ (jigSaw!) Common to all: Every describable feature of an image is a <here-this> pair, Where ‘here’ denotes a location and ‘this’ is some equivalence class. Kees van Overveld -24-

25. Here is cylindrical Here is bigger than Here is saxofone Here is tasteless cliché (or: here is recommendation to buy porcellain statuette) Here is blue-white stripes Here is blue Here is ellipse From Light to Enlightenment Inleiding: wat is een beeld? Kees van Overveld -25-

26. Consider a process of visual communication, and study coding and decoding From Light to Enlightenment Image features in layers Proposal: let us group ‘this’-s in groups Groups have an ordering Properties in group n ‘follow’ (or ‘build on’) properties in group n-1 … in what sense ‘follow’? Kees van Overveld -26-

27. From Light to Enlightenment 1.Come and drink coffee with me 7.Meaning of the message virtual communication III 2. Sequence of characters typed onto keyboard 6. Letters on a screen virtual communication II 3. Bits en bytes 5. Software virtual communication I 4A. Electrical currents in wire 4B. Electronic detectionn physical communication Kees van Overveld -27-

28. From Light to Enlightenment representations: Difficult to define, but: • One representation can be converted into another oen • Can be replaced by other representations where lower- and higher layers stay the same (variants!) • Occur in a sequence of representation conversions, together fulfilling some purpose (where invariants, necessary for that purpose, stay the same) • In images: any representation can be written out as a series of here-this pairs In the example: Variants: color hue, reflectivity, thickness of the border, … Invariants: color saturation, shape, meaning,… Kees van Overveld -28-

29. From Light to Enlightenment representations in the context of communication: • 1st sequence of representations: sender • 2nd sequence of representations: receiver • Sender: initiates process with initial communication-impuls or intention • Receiver: concludes the process with understanding of (and perhaps response to) the message • Sender and receiver are connected with a physical link • Virtual communications occur between any two intermediate representations Kees van Overveld -29-

30. Representation conversion Representation conversion Representation conversion Representation conversion Representation conversion Representation conversion From Light to Enlightenment 1. 7. virtual communication 2. R 6. R’ decodes R virtual communication 3. R’ encodes R 5. R virtual communication 4A. 4B. physical communication Kees van Overveld -30-

31. Cause and/or intentiong Meaning and/or effect The net effect of visual communication Emitting light rays Reception of light raysn Light rays From Light to Enlightenment Which layers form a layered communication model for visual communication? • Lower most: light rays (physical communication) • Top most layer: intention and effect Kees van Overveld -31-

32. From Light to Enlightenment A 2-layer model is too naive: • We need additional layers to talk about • Representations in terms of … • Colors, textures, shapes, surfaces, objects, relations and meaning • Representation conversions such as … • Sending, reflecting and receiving light • Sampling and discretisation • Rendering and finding boundaries • Interpreting 2D as projected 3D • Understanding, recognizing and classifying objects and relations among them • Therefore we propose 8 layers: Kees van Overveld -32-

33. From Light to Enlightenment meaning relations objects surfaces shapes texture Color distributions Light rays Kees van Overveld -33-

34. From Light to Enlightenment Summary of essential concepts: • Clustering: natural tendency of the brain • Properties and values: features that cause clusters to occur • Here-this pairs: an image as a collection of here-this pairs • Equivalence: way to deal with the intersubjectivity-problem • Equivalence classes: collection of indistinguishable values for a given property • Variants and invariants: what is lost, resp. preserved in representation conversion • Coding and decoding: takes place in sender and receiver, respectively • Physical and virtual communication • Layers with representations and representation conversions Kees van Overveld -34-