MOMA Display Screens

1. MOMA Display Screens K u r t R a l s k e

8. Technologies used in the project C - for graphics processing codeJava - for XML parsing and show schedulerXML - for storing show templates and show definitionsMax - for image display and talking to QuickTimeOSC - network protocol

12. Motion Graphics is extremely data-intensive One second of video = (720 (width) x 480 (height)) pixels x 30 frames per second =10,368,000 pixels per second…this is a lot of data for a hard disk to read, consistently

13. Motion Graphics is extremely CPU intensive 10,368,000 pixels per secondCPU uses four elements per pixel (alpha, red, green, blue) , so10,368,000 pixels x 4 =41,472,000 CPU operations per secper image layer…plus, many operations may need to be done to each element

14. Motion Graphics is extremely CPU intensive 10,368,000 pixels per secondCPU uses four elements per pixel (alpha, red, green, blue) , so10,368,000 pixels x 4 =41,472,000 CPU operations per secper image layer…plus, many operations may need to be done to each element

15. MoMA spec requires high resolution video ___Standard video format -- 720 x 480(720 x 480) pixels x 30 fps x 4 =41,472,000 CPU operations per sec per image layer___MoMA video format -- 1280 x 768(1280 x 768) pixels x 30 fps x 4 =117,964,800 CPU operations per sec per image layer(1280 x 768) / (720 x 480) = 2.84

16. Problem / Solution Problem: How to get huge amounts of video data off hard drive, quickly?

17. Problem / Solution Problem: How to get huge amounts of video data off hard drive, quickly?Solution: Don’t do it. Instead, render video in real-time.Technical hurdle: Compressed / uncompressed video?How to create specified video actions in real-time, using many layers of images?

18. Problem / Solution Problem: How to render video in real-time, at high resolution?

19. Problem / Solution Problem: How to render video in real-time, at high resolution?Solution: Work from still images. Create visual motion by moving, transforming, and layering the images.Technical hurdle: How to access hundreds of still images?

20. Problem / Solution Problem: How to access hundreds of large still images from the hard drive, with no latency?

21. Problem / Solution Problem: How to access hundreds of large still images from the hard drive, with no latency?Solution: Don’t do it. Store images in RAM.Technical hurdle: How many images will fit in RAM?How much RAM is available?

22. Problem / Solution Problem: Each machine needs a different set of images. How to catalog and retrieve hundreds of still images from 9 different machines?

23. Problem / Solution Problem: Each machine needs a different set of images. How to catalog and retrieve hundreds of still images from 9 different machines?Solution: Don’t do it. Make sure all the machines store all the images that any machine might need.Technical hurdle: How many images will fit in RAM?How much RAM is available?

24. Problem / Solution Problem: How to deal with network latency, from the master controller machine to the 9 player machines?

25. Problem / Solution Problem: How to deal with network latency, from the master controller machine to the 9 player machines?Solution: Structure commands so only very few messages need to be sent.Technical hurdle: How few messages can be sent, with still keeping time-sync between multiple machines?

26. Problem / Solution Problem: How to keep performance high, so each machine can create layers of video, without slowing down or dropping frames?

27. Problem / Solution Problem: How to keep performance high, so each machine can create layers of video, without slowing down or dropping frames?Solution: 1970’s-style optimizations: integers instead of floating point, fixed-point math, no divisions, bit shifting, keeping math outside of inner loop.Technical hurdle: How to keep code human-readable!

28. The development process 1) A dream2) Researching the limits of the technology3) Working with (or working around) the limits