C o lo ur an algorithmic approach

1. PhD Research Proposal Colouran algorithmic approach Thomas Bangert thomas.bangert@eecs.qmul.ac.uk

2. Human Visual Sensor Array

3. physical sensor response How the physical sensors respond to light… actually a measure of pigment’s ability to absorb photons

4. virtual sensor response implied sensor response based on perceptual studies

5. …derived from colour matching studies … using 3 primaries (700nm, 546nm, 436nm)

6. What is a colour matching study? Visual field divided into 2 regions region 1 illuminated by monochromatic light region 2 illuminated by primaries Subject is asked to adjust primaries until the colour of the 2 regions appears identical?  to match one region with the other R=175G=200B=25 R=200G=200B=50

7. What is being proposed? 4 primaries rather than 3: RGB + yellow Subject is asked to adjust primaries until the colour of the 2 regions appears identical?  to match one region with the other R=0G=25B=25y=175 R=0G=0B=50y=200

8. … and using modern LCD technology monochrome LCD with modified backlighting • one region lit by single spectrum source • the second region lit by 4 primaries

9. Why?(the simple answer) … to resolve the problem of negative primaries  ie. areas where colour matching with RGB fails Amount of red needed to add to monochromatic stimuli to get a match

10. … but really, because the human brain is wired with 4 sensors in mind – organized into 2 opponent channels

11. How would the brain like to see its visual sensor input? Colour information is packed into 2 ‘opponent channels’ (2 signed numbers).Driven by 4 sensors ideally, but otherwise what is available is used.

12. Why is this interesting? how a bird sees colour“… is difficult – impossible in fact – for humans to know” July 2006. “What birds see”. Scientific American.

13. Background to colour

14. sensor array of natural visual systems arrangement is random note:very few blue sensors, none in the centre

15. Sensors we build X Y

16. The naïve approach: Just measure R G B Opposite of what natural visual system do http://www.cvl.iis.u-tokyo.ac.jp/~zhao/database.html

17. Human Perceputal Responseto luminance

18. Luminance Sensor Idealized - note linear response in relation to wavelength

19. What does a light stimulus look like? - The sensor response is simple integration (summation across spectral range)

20. How do we code stimuli? - we assume intensity is equal throughout spectrum When spectral composition is approximately equal sensor response = luminous intensity

21. Spatial Opponency A Peculiarity of natural visual systems:Luminance is always measured by taking the difference between two sensor values.Produces: contrast value

22. Moving from Luminance to Colour • Primitive visual systems were luminance only • Night-vision remains luminance only • Evolutionary Path • Monochromacy • Dichromacy (most mammals – eg. the dog) • Tetrachromacy (birds, apes, some monkeys) • Vital for evolution: backward compatibility

23. Electro-Magnetic Spectrum Visible Spectrum Visual system must represent light stimuli within this zone.

24. Low resolution – equal distribution is okHigh resolution – not! - spectral distribution is more complexsimple luminous intensity fails to describe stimuli correctly

25. Given a light stimulus within the visible range: What information do we need to describe the stimulus fully? 1. Luminous Intensity - 2. Wavelength If we had a reference luminance we could calculate wavelength (by halves).

26. modify one sensor pair – shifting spectral sensitivity  reference sensor: Roughly speaking wavelength is: λ + (R – G ) One sensor can be used as a reference to measure intensity and the second to measure spectral position

27. the ideal light stimulus Monochromatic Light Allows wavelength to be measured relative to a reference.

28. Problem:natural stimuli are often not ideal • Light stimulus might not activate reference sensor fully. • Light stimulus might not be fully monochromatic.

29. Solution: Then reference sensor can be normalized A 3rd sensor is used to measure equiluminance. This means a 3rd piece of information: 3. Equiluminance Which is subtracted.

30. Coding colour With the assumption that a stimuli is monochromatic Any light stimulus (within the spectral range) can be represented exactly by 3 values: • luminous intensity • wavelength • equiluminance Wavelength is coded by taking a difference (or opponent) value of 2 sensors – simplest solution.

31. a 4 sensor opponent design 2 opponent pairs • only 1 of each pair can be active • min sensor is equiluminance

32. Pigment Absorption Data of human cone sensors Red > Green

33. human colour representation is circular! Which is not a new idea, but not currently in fashion. 480nm 620nm 540nm

34. Dual Opponency with Circularity an ideal model using 2 sensor pairs The Primaries: yellow - blue red - green

35. Defining Coloura working hypothesis ‘Colour’ means a stimulus is represented by 3 values • luminous intensity • distance between 2 primaries • equiluminance The primaries are fixed locations on the spectrum. The distance between primaries is measured.

36. Deliverables for 3 year Research Proposal

37. a 4-colour image standard • luminous intensity value • dual opponent value • equiluminance value

38. method to produce 4 primary colours from 3 sensors

39. technique to create images in 4-colour format 3 sensor raw output of conventional technology may be used algorithm is specific to device used need to be able to translate sensor values to wavelength Canon 400D

40. examine possibility of translating historical image archive to 4-colour format • photography • film

41. 4-colour display prototype adapt existing technology if there is no direct hardware access – use pixels as sub-pixels as long as each pixel is addressible

42. a diagnostic test to determine colour primaries in humans • opponency means colours can be ‘tweaked’ by opposing complement • human colour perception varies in the individual • individuals with variation outside normal bounds are called ‘colour blind’ • ‘colour blindness’ can be ‘cured’ by ‘tweaking’ the primaries

43. a colour matching study to confirm approach Does the 4 primary approach solve the ‘negative’ primary problem? primariesR = 650nmG = 530nmB = 460nm primariesR = 700nmG = 546.1nmB = 435.8nm

44. …using conventional lcd display technology for colour matching Light Source http://www.ccs.neu.edu/home/bchafy/monitor/crtlcd.html

45. … with modified backlighting – 2 light sources light from 4 primaries mono-chromatic light

46. (1) monochromatic light source User selectable

47. (2) 4 primaries (red, green, blue, yellow) high quality white light is already often produced by 4 primaries each primary individually adjustable

48. Apparatus • monochrome LCD display • spectrophotometer • monochromator • full spectrum light source • 4-primary light source

49. Further work: • colour arithmetic • Transparency • implied objects

50. Why is understanding colour correctly important? Colours are computed, not measured!Very important that colour information is in correct form!Starts with sensor information!